Replacing door latch 99 tacoma

I have a 1999 Toyota Tacoma xtracab that needs new latches. The ones that attach to the side of doors and latch to the frame. I thought it was the outside handle but it turns out to be the rod it attaches to. Seems to have worn a bigger 'channel' in the rod where it attaches to the latch, which rides inbetween rubber bushings, and cannot be replaced. I got this far, but cannot get the latch out of the door. Is there a trick? My Haynes manual is no real help here. Any help would be greatly appreciated.