As a rider you have two problems and those are a living thing that might be getting stung at the same time you are and distance above the ground If you come off you could hit hard on the device and that might render it useless at the very moment you need it. So any container will have to be robust enough to take that punishment.
Carrying on the saddle would be a solution except that if your horse is getting stung at the same time your are you’ll likely get separated from it, and separated from your medication, again at a time when you can least afford it.
I found these. http://www.kozygo.ca/category_s/60.htm They don’t look real robust so I don’t know if they would work or not.
Here is a better idea. https://www.amazon.com/Carrier-Epipen-Insulin-Pen-Concealed/dp/B00BR041N4
Here are a number of leather pouch alternatives. Some of these folks look like home manufacturers. Contact one of them and see if they could make the pouch into a “shoulder holster” design. That would put it safely on your body. Make the pouch a bit oversize so you could wrap the pen in a thin layer of bubble wrap or some other cushioning material while inside the pouch.
https://www.etsy.com/listing/620304685/leather-case-wmetal-belt-clip-rich-brown?ga_order=most_relevant&ga_search_type=all&ga_view_type=gallery&ga_search_query=epipen&ref=sr_gallery-1-6
https://www.etsy.com/listing/556220090/natural-leather-epipen-pouch-epipen?ga_order=most_relevant&ga_search_type=all&ga_view_type=gallery&ga_search_query=epipen&ref=sr_gallery-1-25&more_colors=1
https://www.etsy.com/listing/567998553/natural-leather-epipen-pouch-epi-pen?ga_order=most_relevant&ga_search_type=all&ga_view_type=gallery&ga_search_query=epipen&ref=sr_gallery-1-24
https://www.etsy.com/listing/236509797/handmade-leather-epinephrine-case-epipen?ga_order=most_relevant&ga_search_type=all&ga_view_type=gallery&ga_search_query=epipen&ref=sr_gallery-1-26&more_colors=1
If you talk to the maker ask them if the best way to orient the device is vertical, horizontal, or angled. My first impression would be angled but I’d have to think about it some before I made a final choice. The maker might be able to rig it such that you can adjust the way you carry to match your conformation and comfort.
I do some amateur leather work and this would be a fun and challenging project!
With a bit of imagination and talking to makers about your needs as a rider I’m sure you can come up with a viable solution. Good luck as you search.
G.