Seal cap between CV and spindle.

I just replaced one of my hubs and am having an issue with the seal cap between the CV and the spindle. (Back of spindle.)

I installed a new seal into the back of the spindle and placed the cap that goes between the seal and CV. The one that slides onto the plastic housing on the CV end. As you can see in on of the pictures the cap has popped off the top of the seal. I tried to reinstall it by pressing it into place but it won't go back in. I've also included a picture of the untouched side for a reference. Am I going to end up having to pull the entire thing apart again?