I mean, I love the traditional tweed or dark wool jacket when people are riding in the cold and damp (like out in the open in the UK, or at a show with cold weather). But most people’s schedules dictate showing during very hot weather, at least at some point in the year. And every time when it’s 100F in the shade, someone always post proudly that gleeful FB meme about how their trainer won’t permit them to remove their jacket, even if the requirement is waived. Obviously not as big a risk as riding helmetless, but wearing even a technical jacket when it’s extremely hot and doing an intense physical activity is a health risk. (And I know the hunters had their own kerfuffle, before my time, with the abandonment of unsafe velvet hunt caps without chinstraps).
I like top hats as costumes. Off-horse costumes. So if you’re dressing up as Freddy Eynsford-Hill, go for it. Just not on horseback at a competitive event.