We Americans have long history of prejudice against carrying stuff on our person while riding. Not all horse cultures did/do.
In many European armies the long arm was carried on the person of the rider. This was the practice in the U.S. and Confederate armies as well. The U.S. did not adopt “carbine boots” until the 1880s. There were some saddle mounted experiments during this period, but the thinking was that if the soldier lost his horse at least he had his weapon, giving him a chance at survival and completing his mission.
The British learned in India and it’s environs that carrying the water bottle on the person was a Very Good Thing. Much of the area the British occupied was desert or very dry and a man without water was a dead man.
So now you’ve got the soldier carrying, on his person, a rifle, a water bottle, a bandolier of ammunition, and a pistol. By the early 20th century add a couple of grenades. And they were quite effective as riders. 
A properly fitted “camelback” is not going to seriously impair rider balance nor will it subject the rider to any significant risk of harm (unless their riding at speed through heavy brush where the risk of a snag is significant).
In preparation for our upcoming military competitions I’ve been wearing a web pistol belt with a canteen, magazine pouch, first aid pouch, and M1911 pistol with a lanyard. I’m not riding in brush country. I don’t find think I’m taking any serious risk. And this is a whole lot more than a “fanny pack.” I don’t they they pose any real risk, either.
Of course risk is not a fixed target, but rather constantly moving. What works in environment A might be a very bad idea in environment B. So the rider has to use their head for something beside a mounting point for their helmet!!! :lol:
G.