Judging music

When I was in middle/high school, I really enjoyed reading a lot of Piers Anthony's books. His first several books of a series were generally excellent, and I'll admit to reading some series well past that point (particularly Xanth, of which I read 10-12).

The point is, I read his Apprentice Adept series back then, and enjoyed it quite a bit. In it, there is this Game that is the center of the Proton (sci-fi-ish) world. A Game starts with a grid where progressively finer selections are made on a grid between two players. Eventually (whether cooperatively or competitively), they end up with a one-on-one challenge, which gets played out.

In it, the main character, Stile (a way over a fence between pastures, nothing to do with how he dresses) ends up in a musical performance competition with a professional musician. He ends up playing to the crowd, while the pro is a technical master. They play twice (the first ended in a draw, I believe, with the crowd choosing Stile, and the judges choosing their colleague), and the second time, the musician takes cues from Stile and plays much better.

The judges gave the victory to Stile, judging that if he was able to raise the level of playing of the other so much, he must be the better player. It always seemed a bit of a stretch, but now we have evidence that that never would have happened.

In her study, Tsay obtained complete audio and video recordings of 10 different classical music competitions, each of which featured three different finalists. At the end of each competition, a panel of expert judges had chosen a winner based on these performances.
...
When given the full performance, the novices performed about as well [at predicting winners, based on the clips they saw or heard] as the experts—and that turned out to be not well at all. With only three clips, they'd be expected to choose the winner a third of the time due to random chance alone. With the full performance, they only managed to guess correctly 35 percent of the time. Those who were given audio-only clips did even worse, getting it right 29 percent of the time.

But the surprise came from those given only visual clips. They got it right 46 percent of the time.

So, judging by this, it's very unlikely that the judges, even as well attuned as they are to the technical side of things, would have ever chosen the musician.

A bit of a surprise, and a good indication that any "sport" decided by judges is not terribly meritocratic. Not to belittle figure skaters, gymnasts, or others involved in similar sports, but it gives strong indication that the eyes lie (even when there is no prior bias, which is a definite issue as well, in international sports).

And as a side note, the first three books of that series ranged from excellent to pretty good. I wouldn't bother with the later ones.