DeepAI AI Chat
Log In Sign Up

Judging the Judges: Evaluating the Performance of International Gymnastics Judges

by   Hugues Mercier, et al.
University of Neuchâtel

Judging a gymnastics routine is a noisy process, and the performance of judges varies widely. In collaboration with the Fédération Internationale de Gymnastique (FIG) and Longines, we are designing and implementing an improved statistical engine to analyze the performance of gymnastics judges during and after major competitions like the Olympic Games and the World Championships. The engine, called the Judge Evaluation Program (JEP), has three objectives: (1) provide constructive feedback to judges, executive committees and national federations; (2) assign the best judges to the most important competitions; (3) detect bias and outright cheating. Using data from international gymnastics competitions held during the 2013--2016 Olympic cycle, we first develop a marking score evaluating the accuracy of the marks given by gymnastics judges. Judging a gymnastics routine is a random process, and we can model this process very accurately using heteroscedastic random variables. The marking score scales the difference between the mark of a judge and the theoretical performance of a gymnast as a function of the standard deviation of the judging error estimated from data for each apparatus. This dependence between judging variability and performance quality has never been properly studied. We then study ranking scores assessing to what extent judges rate gymnasts in the correct order, and explain why we ultimately chose not to implement them. We also study outlier detection to pinpoint gymnasts who were poorly evaluated by judges. Finally, we discuss interesting observations and discoveries that led to recommendations and rule changes at the FIG.


page 1

page 2

page 3

page 4


National Bias of International Gymnastics Judges during the 2013-2016 Olympic Cycle

National bias in sports judging is a well-known issue and has been obser...

Proceedings Fourth International Workshop on Rewriting Techniques for Program Transformations and Evaluation

This volume contains the formal proceedings of the 4th International Wor...

Judging the Judges: A General Framework for Evaluating the Performance of International Sports Judges

The monitoring of judges and referees in sports has become an important ...

Budget-Constrained Reinforcement of Ranked Objects

Commercial entries, such as hotels, are ranked according to score by a s...

A study of variability induced by events dependency in microelectronic production

-Complex manufacturing systems are subject to high levels of variability...

Stat-weight: Improving the Estimator of Interleaved Methods Outcomes with Statistical Hypothesis Testing

Interleaving is an online evaluation approach for information retrieval ...