When I'm riding my bike I have a pocket pistol in a pocket holster plus a spare bag. Barely notice its there. Easy to grab.
If you want to carry a gun while exercising, there are several lightweight revolver and semi-autos to choose from, along with shirts made for concealed carry while running/biking, or a good fanny pack bag, like Blackhawk's. A fanny pack can be worn across the shoulder, bandolier style, too. A Maxpedition Versipack may also fit the bill.
If dogs were my concern while riding a bike, and I had to choose one item for defense, I'd still take spray over a gun, knife, or stick. Ideally, take more than one, but recognize that each has their strengths and weaknesses. A Kahr PM9, a can of spray, and a small folder don't weigh much, or take up much space.