I always wondered about this.....

No. Prisons don't have custom fit pants and belts aren't allowed cause of obvious garrotting issues. Thus you wore what they give you. You obviously can't wear pants that are too small, but you can still wear pants that are too big.

But because they aren't allowed belts, they have to get used to having that swagger so they don't lose their baggy pants.

Now they wear them like that to show that they've done time and know how to walk with prison pants.