We have a '99 Rockwood Freedom pop up that is just about that same size and weight. Like you, it's just me, my wife, and our daughter - and it's the perfect size for us. I've given a TON if thought to getting a larger hard-side camper, but I really don't want to be worried about the truck's capability. As you point out, even with a 4.0L double cab, I can definitely tell its back there. A few times, I'll have to get down to 3rd gear on long hills. I can't IMAGINE towing a 4K pound camper around here (upstate SC, northern GA, and western NC).
Instead, I think we'll just stick with the pop up and not give any thought to where we can or can't take it. Here's a pic from this weekend in Hiawassee, GA.