loose alternator belt pulley

I have a 2.4L 1998 manual tacoma, and 2 of the three belts mysteriously shredded recently. I installed new ones, and noticed that the alternator belt pulley on the crankshaft is just hanging there loosely. It can move over and rub against the adjacent belt on the crankshaft. These two belts were the ones that shredded. Shouldn't there be something keeping this pulley from sliding into the next belt? I started the engine and the pulley in question moved over and began making contact with the middle belt. I noticed that there is some sort of belt guide on the side of the pulley near the alternator, but the guide only would stop the belt from sliding toward the engine on my truck. Is it broken? Is the pulley worn, and missing something that stablizes it? help please...