Data collection and analytics in eventing

[aycaramba]

After another death, I’ve been wondering what I can do to help this sport that I love. My career is in data analytics (certainly not an expert, but good enough to get paid for it). To me, data always seems like a good way to try to solve a problem, and it seems like this could be one facet of making our sport safer. I’m inspired by Carol Kozlowski and her work to get rid of the weight requirement. My questions to y’all are:

1. is the USEA or USEF currently collecting any data at all on deaths/falls (or even better, all rides)? My guess is no but any data is a start.
2. does the USEA/USEF currently have anyone working on data analysis or a project like this?
3. who would I need to contact at the USEA to talk about getting a committee together to tackle this problem?
4. would anyone else be interested in joining me in this effort? I’m imagining the very least we could do would be to put together a proposal of what data we would like to see collected, along with some ideas of analysis for once we get the data.

”‹I’d love and ideas and input. I’ll probably never ride for the team and I’m not wealthy enough to donate tons of money, but the least I can do is offer up some time and brainpower.

[Bearx2]

I have no skill set to offer but just wanted to say I think you have a great idea. It would be interesting to see if the rider involved in catastrophic falls/ accidents have a history of issues or if it was a one time thing. I feel that by looking at results in an unemotional way might go a long way in helping to identify areas where things might need to be changed .

[Scribbler]

Years ago I read that airline safety engineers took near misses very seriously and analyzed and prevented them. They realized that every near miss indicated higher risk.

By contrast ordinary people tended to interpret a near miss as a sign of reduced risk. With driving people tend to think that a near miss is a sign of their superior skill in avoiding a crash or else a sign they are lucky.

After I read that I decided to interpret driving incidents more like airplane safety people and if I have any kind of near miss or even something a little alarming I dial it right back and slow down. Even if it’s “obviously” the other person’s fault. I’ve always found it interesting when people repeatedly have accidents that “aren’t their fault” even by the standards of police and insurance but nonetheless continue to rack up accidents when other people never have any.

Related to jumping, maybe more needs to be analyzed than just fatal accidents. Maybe all accidents and near accidents need analysis. When I’m watching juniors do the two foot nines I can usually predict a horse that’s going to dirty stop or runout several jumps before he does, and sometimes at Spruce Meadows on TV as well. Obviously these don’t usually result in much injury compared to cross country. There might be more data out there than just about fatal jumps.
”‹”‹”‹”‹”‹”‹

[aycaramba]

Thank you @Bearx2 ! It seems like it would be worth a try!
@Scribbler , that’s a really interesting perspective about near-misses! I agree, if we are going to do this right, we would ideally need data on ALL jumps so that we could find what the differences are between successful jumps, near misses, and accidents. Your comment made me think of one way we could do it - what if each jump judge rated each jump on a scale of 1-3 or 1-5 (fall to excellent) or something like that. Jump judges already have to mark on a paper or radio in that a jump was successful - giving them an easy rating system (with examples shown in the jump judge meeting beforehand) is one way to help easily standardize subjective data into something we could use for analysis. It certainly isn’t perfect, but it could be a start.

[Janet]

WRT falls-
TDs need to submit (to USEF) Fall Reports (for every fall on the cross country course related to a fence, whether or not there is an injury). Additional paperwork is submitted if there is an injury.
I do not know if /how this data is analyzed, or whether it is available for external analysis.

There is also paperwork submitted for “Dangerous Riding”.

I would suggest you contact the safety committee at USEA.

[enjoytheride]

I’m interested in finding out the parameters for yellow cards. What are officials looking for? What’s the difference between an action that causes a TD to warn you about getting a yellow card (which might be enough for some people to change) or actually giving you one (which might not be enough for some people to change). Do we need clearer yellow card guidelines? Would more help?

I’ve been working at events and heard TD’s say “I saw this rider at the last event, I’m going to watch them close today”

[enjoytheride]

@aycaramba

Thank you @Bearx2 ! It seems like it would be worth a try!
@Scribbler , that’s a really interesting perspective about near-misses! I agree, if we are going to do this right, we would ideally need data on ALL jumps so that we could find what the differences are between successful jumps, near misses, and accidents. Your comment made me think of one way we could do it - what if each jump judge rated each jump on a scale of 1-3 or 1-5 (fall to excellent) or something like that. Jump judges already have to mark on a paper or radio in that a jump was successful - giving them an easy rating system (with examples shown in the jump judge meeting beforehand) is one way to help easily standardize subjective data into something we could use for analysis. It certainly isn’t perfect, but it could be a start.

The biggest issue I see with this is a lot of the JJs I see have very little to no horse experience. Parents or grand parents. Usually your experienced JJs go near problem fences like ditches or water. I think it would scare off JJs, make them harder to find, and I think that riders would be prone to argue about a JJs analysis of their jump. Riders already argue about if it was a stop or not, or where or how many times they hit their horse. So if we went that route our thinking about fence judging would have to change.

It can be really hard being control and filtering out all the people who don’t know what roaring is and people describing dangerous riding, and then getting the TD there before something happens. Maybe something needs to change there.

[aycaramba]

@Janet , thank you, I will reach out to them! Hopefully they can make that data available.
@enjoytheride , that’s a good point about yellow cards, maybe a clear rubric or criteria could help (I’m not sure if they currently have that, but that’s a good idea to look into it). I think it would help to even record warnings that don’t result in yellow cards. And you do have a good point about the jump judges - it’s easy to forget that it’s not always horse people. Maybe we could create greater incentives for riders to jump judge, or create more flexibility for them to jump judge for just one level’s worth of divisions (I know time commitment is the biggest reason I don’t jump judge more often - if I am riding, it’s hard to volunteer even an entire half-day). Or maybe we could collect data in a different way, but jump judges seem like an underutilized resource if we want jump-level data since they are already there at each fence. This kind of brainstorming is exactly the kind of start we need.

[RSEventer]

Eventing Nation posted an op-ed piece today (https://eventingnation.com/a-plea-for-transparency/?fbclid=IwAR2vvIPauH7q2mXW7uGHV_ivMRCLzcJp6Rl3Zk1Ghr7c33mHuyB3mhdHiao) by a rider and when I was reading the comments after, there was a great article about what Australia did after back to back deaths on course of young riders. ( http://amp.abc.net.au/article/11575750) Here is a quick summary of a few points of the article:
The inquiry, chaired by Deputy State Coroner Derek Lee, considered whether safety procedures at NSW equestrian events were adequate to minimize unnecessary risk of serious injury or death of riders.
It also focused on concerns that equestrian sports heavily relied upon volunteers, who often lacked the necessary skills and training to ensure safety.
Further recommendations could also see:
Ӣ all fence judges view a critical incident training video prior to an event
Ӣ a national safety manager appointed on a full-time basis
Ӣ course designs be adequately tested for safety purposes
Ӣ a robust process developed for the review of serious incidents requiring a medical response
Ӣ a reporting system be implemented to allow competitors to confidentially communicate safety-related concerns

[CSU92]

Have you looked through the Charles Barnett audit of the FEI data?

[Janet]

For rules/guidelines on yellow warning cards see GR1037, GR1301.6, EV112. There is also a watch list, see EV102.5.h.

[BoringEventer]

@enjoytheride

I’m interested in finding out the parameters for yellow cards. What are officials looking for? What’s the difference between an action that causes a TD to warn you about getting a yellow card (which might be enough for some people to change) or actually giving you one (which might not be enough for some people to change). Do we need clearer yellow card guidelines? Would more help?

I’ve been working at events and heard TD’s say “I saw this rider at the last event, I’m going to watch them close today”

Do TDs know if another TD has already warned a rider for coming close to getting a yellow card? I mean, is there any sort of documentation of these talks/warnings? If Jane Doe is warned by TD1 at XYZ Horse Trials for her speed and looking like Evel Knievel coming through the down drop into the water, does TD2 at ABC Horse Trials get any notification that he should watch for that sort of riding and issue the yellow card if he sees it, instead of another warning? Or do we just hope the TDs talk to each other and happen to notice that these issues keep recurring?

And what else do we keep track of? I mean, do we keep track of who riders are listing as their coaches/trainers? Do we keep track of how often a specific coach/trainer’s students are receiving warnings or even yellow cards?

We can’t identify and correct patterns unless we have a very good method of tracking them to begin with. Maybe something is already in place, I hope it is, but I’m not sure it’s being used as well as it should be if it does exist.

[JER]

@BoringEventer

Do TDs know if another TD has already warned a rider for coming close to getting a yellow card? I mean, is there any sort of documentation of these talks/warnings?

After the carnage of 2008, there was supposed to be a Watch List. No idea what happened to that or if it was anything more than PR talk.

[add_leg]

This isn’t specific around falls/deaths, but I do analytics professionally and started compiling blog posts where people have used different analytical ideas in equine performance. There’s a small collection here (http://marescara.com/equestrian-anal…-data-analysis) and a few of us have some data sets floating around I’m sure we’d be happy to share. Doing more is a pet project, but I just don’t have the spare time and mental capacity right now to devote to looking into it. I’d be happy to collaborate with you or others though! (Also, horsey data people are my favorite people)

I have dreams of luring one of my trauma/emergency medicine epidemiologist friends into looking at this professionally, but again - haven’t pursued it. I do wonder if something like the University of Kentucky epi/biostats program would have anyone interested in looking at it, but I don’t know that I’m the right person to approach someone about it. Any epi people hanging out here?

[Divine_Comedy]

I collect data for the sport of eventing at the A/4/5* level (focused on the US but with significant worldwide data available at the 4/5* levels) and do some basic analytics with it. Unfortunately I don’t have the time or bandwidth to do it for all levels, and I have found it is nearly impossible to gather the type of data you are asking for without a huge overhaul of the way USEA/USEF gather data from shows.

For instance, the FEI doesn’t even record the dressage percentiles or which judge they are assigned to. That is something easily available at every show and yet the sport discards that data. Although that is not directly related to safety, it is an example of how casually the sport treats data.

I would love to see the sport try to assist someone like you with a project like this. I think a serious overhaul in the culture of what data events are responsible to provide to the organizations would be needed.

[Jealoushe]

I would 100% support this effort, more data has been something we have all been wanting for at least a decade on the board.

[Janet]

@BoringEventer

Do TDs know if another TD has already warned a rider for coming close to getting a yellow card? I mean, is there any sort of documentation of these talks/warnings? If Jane Doe is warned by TD1 at XYZ Horse Trials for her speed and looking like Evel Knievel coming through the down drop into the water, does TD2 at ABC Horse Trials get any notification that he should watch for that sort of riding and issue the yellow card if he sees it, instead of another warning? Or do we just hope the TDs talk to each other and happen to notice that these issues keep recurring?

There IS a watchlist. If rider is on the Watchlist, and enters other HTs, the TDs at the subsequent HTs are informed that the rider is on the Watchlist. I have never been informed that way, so I do not know how much detail is included.

But if a TD just “speaks to” a rider. with out doing the Watchlist paperwork, there is only “word of mouth”.

[anon69112984]

@Scribbler

Years ago I read that airline safety engineers took near misses very seriously and analyzed and prevented them. They realized that every near miss indicated higher risk.

By contrast ordinary people tended to interpret a near miss as a sign of reduced risk. With driving people tend to think that a near miss is a sign of their superior skill in avoiding a crash or else a sign they are lucky.

After I read that I decided to interpret driving incidents more like airplane safety people and if I have any kind of near miss or even something a little alarming I dial it right back and slow down. Even if it’s “obviously” the other person’s fault. I’ve always found it interesting when people repeatedly have accidents that “aren’t their fault” even by the standards of police and insurance but nonetheless continue to rack up accidents when other people never have any.

Related to jumping, maybe more needs to be analyzed than just fatal accidents. Maybe all accidents and near accidents need analysis. When I’m watching juniors do the two foot nines I can usually predict a horse that’s going to dirty stop or runout several jumps before he does, and sometimes at Spruce Meadows on TV as well. Obviously these don’t usually result in much injury compared to cross country. There might be more data out there than just about fatal jumps.
”‹”‹”‹”‹”‹”‹

Funny you should mention the airlines. I have worked high profile jobs in the airline industry and said over and over and over again to so called key actors in the world of eventing safety we could learn so much from the airline industry.

I even had a long conversation with Charles Barnett about it when he was collecting data for his audit and we were totally on the same page.

[add_leg]

@anon69112984

Funny you should mention the airlines. I have worked high profile jobs in the airline industry and said over and over and over again to so called key actors in the world of eventing safety we could learn so much from the airline industry.

I even had a long conversation with Charles Barnett about it when he was collecting data for his audit and we were totally on the same page.

My brother is a pilot and I work in healthcare and I see so much we could use - it’s not like we’re inventing the wheel. I don’t understand why we aren’t implementing these things that are proven best practices.
There should absolutely be the equivalent of an NTSB investigation/Morbidity and Mortality conference for every serious eventing injury or death, in my opinion.

[IFG]

If you look back on threads from 2008, several of us offered to help. I am a Ph.D. Epidemiologist. I worked to develop a case report form for falls with several others, including RAyers and JER. Nothing ever came of it. Part of the problem is the issue of jurisdiction over events that is divided between the USEA and the USEF and the FEI. If the PWB are more open now, great, but the experience left several of us with a bad taste in our mouths.