Stereo/Clock intermittent Outage

Sometimes my stereo will turn off mid-drive or not turn on at all. It can stay off for hours but sometimes it will just reboot at once. Occasionally the clock below the radio will shut off as well, but not always. This seems to especially happen when I drive over hard bumps. I used to think this issue was related to my add-a-fuse on the accessory slot because sometimes I could get the stereo to come back on by reaching down to the fuse box and jiggling it a bit.

The issue was infrequent for years, but lately, it's been happening a lot more often. Weeks ago I got fed up and took the add-a-fuse fuse out, but my issue persisted. So I took a closer look and realized if I messed with this (pictured) circled plug I could replicate the issue. It seems like this plug is loose, despite being pushed all the way in.

I also recognized an accompanying click sound coming from the fuse box area whenever the radio unexpectedly turned off or on. When I juggle the plug in question I can hear It clicking off/on over and over again until it finally stays on.

Any idea what going on? Any idea how to tighten up an old loose plug?