Any help

I have an 05 tacoma with a 3" toytec. So here is my problem my steering wheel is jerking,pulling to the right. Say i have my hand on steering wheel at twelve oclock i get bad jerks that pulls it to about two oclock this is continous until it stops one after another might last for 30 min then it goes away. It feels like someone else is trying to turn the wheel and im fighting it. This is usually off road at lower speeds. I can put in 4wd and it straitens right up and quits jerking but will be back as soon as i take it out. Any ideas what could be doing this???? Can the belt cause this?? Thats when it seems to start when i get it wet. Ty