I’ve heard that for judge training there are lots of videos with “this horse should score X”.
Does anyone know if those are available to the public?
Janet Foy has a couple dressage symposium videos up on YouTube. Here is a playlist to get you started:
9 Likes
USDF puts out “On the Levels” videos whenever the tests change. They are more geared, IMO, to instructing riders on how to ride the tests, but I found them very helpful when going through the L program to help educate my eye. They include judge’s commentary throughout the tests.
5 Likes
If you have the chance to audit the L program, do it! It will include these types of videos along with great explanations of how the scoring works.
1 Like
If you look on ebay you’ll find a dvd called “Through the Judges eye” which has FEI judge Steven Clarke going through British Novice and Elementary tests which are comparable to US first and second level tests. Cost is just over $5 for a used DVD.
1 Like
You can search on Youtube - there are a lot of personal show videos where people post their scores along with the movement. L program and On the Levels are great recommendations.
Hmm I was really kind of hoping for a “here is the source of truth” videos, not just seeing what judges gave other people. Or even some documents like the FEI has for scoring specific mistakes and the required point deductions for different types of issues.
I’ve noticed some surprisingly large changes in score trends in my area recently (even among the same judges!) and I’m really trying to figure out if the scores were more “correct” before or after and change my mental scoring accordingly.
Maybe I’ll have to look into the L program. From what I recall, it’s pretty tough to get into, right?
2 Likes
@Mander - You can audit it, meaning you attend the training sessions but don’t practice judge or test. There are no requirements to audit although spaces are pretty limited.
4 Likes
Take the beginning sessions - or you can audit. It is totally worth it. They will go through videos in the sessions and it is helpful.
1 Like
If you’re looking for “here is the source of truth” this isn’t a video but a book and it was on the L program reading list when I went through.
It’s 40 Swiss francs ($46) I seem to remember paying closer to $80 for it. Ah, shipping to the US is another $28! Anyway, it’s a great resource.
For example, it describes the characteristics of the halt in the context of the training scale for scores of 10 all the way down to 0. It does the same for gaits, simple and flying changes, lateral movements, figures and collectives.
I also found this which is a PDF for free!
4 Likes
Oh heavens haha fill out a pdf to order a book - and who says that FEI is stuck in their ways 
Thank you though, I’m not sure I ever would have found that.
I’ve scoured the judging manual before and I did find it very helpful for understanding the “cost” of various mistakes. It sounds like the book will help me better understand the scoring of movements that didn’t have mistakes, but had more systemic issues (a shoulder in that wobbled a bit, for ex.)
1 Like
There are videos…but I would argue that there is NO “source of truth.”
For background, I spent 32 years in the petrochemical industry where I had to have discussions with Federal agencies on “compliance” with regulations. There are the regulations (rules) and how the various plants performed against the EPA/OSHA/RCRA/BPV…etc regs. So I felt qualified to ask question on “are you following the rules?”
I attended (audited) 2 of the L-judges seminars and was 2nd level demo rider. In one of the video sessions, I asked a question on how a judge would interpret a video rider’s performance against the rules. (I don’t think auditors are allowed to ask questions any more). The question had to do with the horse being “in front of the vertical” and how would a judge evaluate such a performance. This was an honest question to see how divergence from the rules would be evaluated.
The regs say
> DR-101 The head should remain in a steady position, as a rule slightly in front of the vertical, with a supple poll as the highest point of the neck…
> DR-102: The neck should be raised with the poll as the highest point and the head slightly in front of the vertical.
> DR:103: The nose must be clearly in front of the vertical.…
> DR: The rider allows the horse to carry the head a little more in front of the vertical than at the collected and the working trot, and to lower the head and neck slightly.
> DR:105: The rider allows. the horse to carry the head a little more in front of the vertical than in the collected and working canter…
So…you get the drift. The rider in the video had the horse obviously behind the vertical. The judge said, “The horse was in front of the vertical.”…I did not argue, but felt that the judge was blind. I saw the horse obviously BTV. She could have easily rewound the tape to show me wrong, but obviously did not want to go there.
So…forewarned is forearmed. There is NO TRUTH to dressage judging. There are statistical tests that can be done to validate testing protocols…think SAT testing etc. The old COTH Nerd Herd suggested that the USEF/USDF submit their L-program to this sort of statistical analysis and validation of the program. Of course there was no answer for any of the powers that be.
6 Likes
Auditors are allowed to ask questions during the L program; the paying candidates get to do so first so that the instructors time is not monopolized by auditors.
All judging is pretty much one person’s opinion at that particular point in time.
2 Likes
Ya know, would you want your radiologist reading your X-rays, or your welding inspector reading radiographs of a nuclear reactor to have “just one person’s opinion?”
My bet is no. Training programs undergo validation to see how well the training achieves proficiency in evaluating whatever subjective criteria is being trained.
There is no difference in training dressage judges to training anyone else in industry, medicine, sociology, psychology, that evaluates subjective criteria with serious health consequences.
Dressage is not rocket science. You would think the powers that be might be interested to apply known statistical techniques to the L-program if only to quantify where the training is working as intended and where there is room for improvement.
1 Like
But the training is not one person’s opinion. Judges work with quite a few different members of the Faculty during the various judge programs. Regular training sessions and small group apprentice programs and sitting in with a judge… so they get many different perspectives on developing their eye and become quite consistent in scoring the movements.
We must be doing something rights since many other federations keep asking for our judge program…
As the saying goes…“In the land of the blind, the one-eye man is king.” Eg., if other National Federations have no training programs, then something is better than nothing.
1 Like
Why are you blaming USEF/USDF when they are following FEI judging? It makes no sense for riders in America to be judged one way then have to switch completely at the international level.
Who is blaming USEF/USDF? …about what?
OP asked a question about “truth” in dressage judging. My premise there is no truth as there has never been quantification of how well judges judge to the standard. And back ~20 years ago a group of COTHers offered USDF/USEF to help with his and were ghosted.
The point is the USEF/USDF have the L-judge training program. That program has never been submitted to statistical rigor as to test whether the training is getting judges to be aligned in their judging…eg., to quantify whether judges are the all judging to the same standard. This is called “inter-rater reliability.” The 2 statistical tools used in industry to quantify inter-rater reliability is the Kappa Statistic (Cohen and Fleiss) and Kendall’s Coefficient of Concordance.
Kappa is a metric that is used to evaluate inter-rater reliability (IRR). IRR measures the extent to which different raters (eg., judges) assign the same score to the same variable (eg., dressage movement). You can use the kappa statistic to quantify the agreement of for Each Appraiser vs Standard and All Appraisers vs Standard. This requires you have a standard for each sample (dressage movement).
Which begs the question, is there a standard? And are the judges sticking to it? See above my example on the rules stating a horse must be “in front of the vertical” but that rule does not seem to be strictly applied.
Kendall’s Coefficient of Concordance is used to measure the level of agreement between multiple appraisers’ assessments of the same samples (riders).
For more info, here are links
Minitab on Kappa
Minitab on Kendall’s Coefficient of Concordance
Minitab on Inter-Rater Reliability techniques (Attribute Agreement Analysis)
1 Like
“I attended (audited) 2 of the L-judges seminars and was 2nd level demo rider.”
Do you mind sharing in what year that took place?
May I ask why you ask?
