“I don’t think that the components are judged fairly … [in order] to stay in that stupid corridor of average marks.” — Patrick Ibens, judge
“The [program component] marks do not reflect the skating but rather the starting order and the reputation of the skaters.” — Sonia Bianchetti, former referee and chair of the ISU Technical Committee
And we all turn a blind eye?!
The discontent with the current judging system among skaters, coaches, officials, and fans alike has been spreading like widlfire in recent years. Yet even the most unhappy of us seem resiged to the new status quo. “Too late to change it,” everyone says. “Let’s try to tweak the rules here and there.”
Tweak? How can you tweak something that is fundamentally flawed? How can you fix a system that straightjackets skaters into paint-by-numbers programs and doesn’t reward creativity and artistry? And how can you ignore dire warnings from high-level officials in the ISU that the five-year-old judging system — conceived by a speed skater — is not fit to judge figure skating?
Judges seldom if ever speak out about their jobs and their peers. So when international judge Patrick Ibens blew the lid open on major flaws in the judging system, his words sent shockwaves throughout the sport.
The flashiest quotes from Ibens’s interview concerned dishonesty among judges. (Ibens estimated that only 10 percent of judges are “completely honest.”) But the fact that judges have their own agendas is hardly news to anyone who’s ever rolled up their eyes at a skating result. (And anyone who has not, please raise your hand.) We can’t change human nature — nor can any judging system devised by man.
But the most serious charges leveled by Ibens (who judged last month at the Olympics) have to do with the way the system itself forces judges into poor and dishonest scoring by punishing them if they don’t mark program component scores (PCS’s) within an average “corridor.” The results are catastrophic, and that’s not inherent in the sport but specific to the International Judging System (IJS).
The corridor or shame
“What I hate the most about this system,” Ibens says, “is that it is made to save the “not-so-good” judges, while the really good judges who are marking the way it’s meant to be (every component separately) risk the chance of being out of the corridor of average marks, and risk getting some assessments. A judge who basically does not know anything can give all the wrong marks or completely guess and their marks fall into an average! But someone who wants to have wide margins between components might be singled out for doing so. For example, when scoring the first three groups at the World Championships, you give between 5.50 and 7.00 and you are in the safe corridor. When the last groups come on the ice, give between 7.00 and 8.50 and you’re safe again!”
This is a serious allegation — not the kind of thing you can sweep under the rug. Imagine the judges award a skater with a good reputation an extra point on each of the five component scores to make sure they fall within the magic corridor. That adds up to a whopping 10 extra points in the final mark (combined for the short and long program). With many competitions decided by as little as one point (e.g., men’s event at the Olympics) this built-in bias is certain to skew results consistently in favor of skaters with a solid reputation and against those who perform surprisingly well on a given night.
In plain English, we have consistently wrong results!
Skaters like Mirai Nagasu or Johnny Weir at the Olympics, for instance, could not break into the medals even with outstanding skates because their PCS’s have most likely been set based on expectations. The risk for the judges was too great to adjust each PCS. Conversely, a Patrick Chan or Stephane Lambiel can do no wrong. Sure, they’re wonderful skaters. But at the Olympics they had all the appeal of a patient walking away from a root canal. Yet the judges didn’t dare mark them down in PCS’s.
In a friendly response to the Ibens interview, former referee and ISU official Sonia Bianchetti reinforced the general sentiments about this infamous corridor. “The PC marks do not reflect the skating but rather the starting order and the reputation of the skaters,” she said.
Bianchetti also expanded on another big issue Ibens raised: the requirement that judges mark “absolutely,” without comparing skaters, something that defies both human nature and the nature of our sport.
The absolute mark debacle
“The only way to be consistent through the whole event is to be thinking all the time whether the marks given now make sense compared to the marks given before,” said Bianchetti, who has served as an ISU official at the highest levels of the sport for more than four decades.
“Under the IJS the judges are now asked to evaluate performances on an absolute point scale without comparison to any other performance. While this may be conceivable when evaluating individual elements of a program, for the program components it is not. These are entirely different ways of thinking,” Bianchetti said.
In other words, art is more than the sum of its parts and doesn’t lend itself to being quantified in points. As human beings we intuitively judge by comparing. We can look at a piece of paper and guess very accurately where the middle point is simply by comparing the two halves. But if we had to guess how many inches across the paper is, we’d not do nearly as well. Why are judges expected to do just that?
These issues are only two of many serious flaws afflicting the new judging system. Skaters and coaches have complained about being too constrained by the system. Fans don’t enjoy performances any longer. Skating is hurting financially as audiences are turning off their TVs. And much has been written about the controversial new judging system in the mainstream media (as well as in this blog, such as here and here). And yet most people are so afraid that the IJS has become entrenched by now that taking action seems unthinkable.
But the stakes are too high. Both the quality and integrity of figure skating have been seriously compromised under IJS. Why is it that a few people were able to throw into the trash bin of history the familiar 6.0 judging system, which has served our sport for more than a century, but only five short years after this experimental new system was launched, everyone thinks change is already impossible? It is only if you believe it to be so.
***
Join the Facebook group “Bring Back the 6.0 System in Figure Skating.”
GREAT analysis! good work M! i hope this heats up to the point that some change is indeed elicited. however, i think this is going to be the constant state of our sport...and is just part and parcel of walking that tightrope between sport and art/performance!
ReplyDeleteI am aware of the PCS problem, but I don't think it is any more of a problem than "Presentation Score" under the 6.0 system. Both are arbitrary, and both can't avoid the matter of subjectivity. I find it disingenuous that Ms. Bianchetti talks as if she's not aware of it. How exactly is it better or different to give out 5.7, 5.8, etc. different from 7.25, 7.50, etc.?
ReplyDeleteMs. Bianchetti comes off as if she's mourning the loss of judges' dramatic 'power' over ranking skaters. Sorry, but I think many will think that judges need not be at the front of any skating event. The 2010 Olympics, many agree, were one of the better judged events, and Ms. Bianchetti herself has expressed how well the top skaters performed throughout difficult programs. How does it then support the view that the system is killing the sport? You can't have it both ways.
I think the system needs to be tweaked, but going back to the 6.0 system or such is NOT an answer.
Both Ms. Bianchetti and Mr. Ibens explained very well how the "corridor" imposed by the system makes many judges mark unfairly. When you accuse someone of being "disingenuous," (Egg, above) I suggest you read the whole article first and try to understand what they're explaining. The corridor issue is unique to THIS system.
ReplyDeleteMoreover, just because skating is always arbitrary, under any system devised by man, doesn't mean one isn't better than the other. That's like saying that because we'll always have criminals we shouldn't try to pass better laws. If something doesn't work, we change it.
The other crucial difference between the presentation mark under 6.0 and the PCS's now is that under 6.0 it was all out in the open. Every single mark from every judge (with nationality) was posted on the boards and seen by every viewer on TV. Sure some judges cheated anyway. But if someone had a great performance, it would be very hard to give them a 5.1 for presentation with everyone seeing the marks. Conversely, a disastrous performance couldn't be rewarded with a 5.9 very easily.
But with the PCS this happens all the time, because no one sees the scores (except insiders who look them up after the event is over). People in the arena see a meaningless global score, like 145.35. Clear as mud, huh? And no one knows which judge gave which mark and which marks counted. It's all veiled in secrecy. Don't you think that makes it a lot more inviting for judges to cheat?