Towed at maximum capacity yesterday... Question.

First off, I'm a little jealous with the GSF. I went IS350, and should have just did the family cruiser, for kick ass families. Way more practical. Next one is the IS F or RC F. Love me some Lexus. Bonus for color....

For your truck, these are the stats:

640 lb tongue weight
6400 trailer capacity
5600 gross vehicle weight rating
4425 ~curb weight
11360 gross combined weight rating
1175 payload

Now you may have a yellow sticker on the inside of your drivers door sill, stating less (most likely).

Known weights:
2200 car hauler
4034 Lexus
6234 gross trailer rating. Now some of that weight does go to the trucks capacity via tongue weight. Your good there. You definitely did cut it close.

Tongue weight is tough to guess without a scale, and others have stated how craptastic the u-haul trailers are. These are built like brick shithouses for sure. My 14K lb rated equipment hauler is 2900 lbs, for reference.
usually tongue weight can be figured out by estimating GTWR, and usually this is 9-15% if that weight. Let's call it 11%. So you were at least 686 lbs on tongue weight. 46 lbs over.

Curb weight is 4425, and maximum what your truck can weight is 5600 lbs. So 1175 (hey, there's that payload number). You weigh 180, gear for trip 50lbs, tongue weight was ~686lbs. This comes to 916 lbs. You still had 259 lbs of capacity.

6234+4425+916=11575 lbs. You were over your gross combined vehicle tow rating by 215 lbs.

The trailer bounce and rough feeling is normal. Trailer tires are supposed to be aired up to Max Cold pressure. My flatbeds can get pretty rough. The 16' isn't bad, but the 28' GN lowboy is rather crappy unloaded.

Now this is just an estimate on your whole setup. You have any mods or added weight (off-roading gear, bigger tires, lifts, etc), this can directly effect this calculation.

Hope this helps a little.