95 4Cyl A/C issue

hey all.
I have an odd A/C issue.

When I turn on the A/C after first starting the engine, idle drops down until the engine dies. After letting the engine warm up, it will idle up when turning on the A/C.

But here is the kicker.
Even after warming up. Once I start driving and say come up to a stop, I start down shifting. As soon I put in the clutch and foot off the accelerator, engine will die. If I turn off the AC before taking foot off the gas, engine will return to normal idle.
Also, say I am in 3rd coming up to said stop light with the A/C on. I let my foot off the gas and let the engine slow me down for about 10 seconds and I then put in the clutch, the engine will return to normal.

Following the troubleshooting in the FSM, my next step is to "try replacing the A/C Amplifier". But I'd really like to try to narrow it down more somehow, rather than "trying" new parts, when those parts are $30-40.
I plan on connecting a test light to the VSV input wire while driving to see if the amp is opening the VSV delayed/

 

Anyone else have a simular situation?