How Accurate Were Olympics Medal Projections? It Depends on How You Look At It

With the Tokyo Olympics over, national committees will be deploying armies of data specialists to analyze performances, looking at what went right and what went wrong for their athletes. It’s an important job, and it allows nations to decide how money and resources should be spent in the future. The data that gets analyzed today helps shape performances for future Olympics, perhaps for athletes who have yet to be born.

Sports and data science enjoy an uneasy relationship, however. Yes, it’s fashionable to use data, but some are reluctant to go all-in on data science, particularly when it conflicts with the evidence in front of our eyes. Moneyball, the book and accompanying film, helped revolutionize the idea of analytics trumping traditional methods, but while Moneyball is in the list of best sports movies ever created, we now live in a more sophisticated time in terms of sports data and modeling – a post-Moneyball era.

Projecting Olympic medals is a massive task

And, perhaps the holy grail in using data to make sports predictions is the projected Olympic medals table. This is not a determination on how many rushing yards an NFL player will have or whether a team is going to win the Stanley Cup – this is a projection on 10,000 athletes in over 300 events aiming for over 1000 medals. It’s insanely difficult to model, or, at least, it should be. But it doesn’t stop analysts from trying.

Opinions might differ as to the foremost medals predictors, but Gracenote has got a lot of coverage in recent months. The company is a relative newcomer in sports data, and it is probably better recognised for its music recognition capabilities. It is owned by Nielsen, the tv-ratings company. Gracenote terms itself as “the New Player in Sports Data”, and its projections have been used in countless articles and press releases over the last few months.

But just how good is Gracenote and, indeed, other companies who try their hand at predicting Olympic medal tallies? Well, below we look at some of the predictions for the 2016 Olympics in Rio (gold medals only):

Nation	Final Gold Medal Tally	Gracenote Projections	Sports Illustrated (Brian Cassaneuve)	NBC Sports (Luciano Barra)
USA	46	38	45	35
Great Britain	27	18	16	16
China	26	31	45	32
Russia	19	20	14	23
Germany	17	16	14	16

 

As you can see from the above, Gracenote undershot the United States and Great Britain, slightly overshot China, and was almost perfect on Russia and Germany. The others had a similarly mixed bag of results. But, as we mentioned, it’s difficult to cover something that has such scope. And, you should also note that data scientists had to deal with the fall-out from the Russian doping scandal in the lead up to Rio – that caused havoc with many models.

While some might look at these medals tables and fail to be impressed by data scientists with their fancy models and algorithms, it can also be argued that people are looking at it the wrong way. Gracenote and other companies aren’t in some competition to see who is perfect in some expensive guessing game – they are publishing projections of expected medals. They are saying, “In a controlled environment, the data points to this outcome”. Sports, as we know, is far from being a controlled environment, scientifically speaking. So many factors – injury, human error, nerves – can play havoc with modelling what should occur.

Humans don’t do well with predictions  

As humans, we aren’t – by and large – comfortable with data modelling. Take, for instance, the furore over election polling. The pollsters got it wrong when backing Hillary Clinton in 2016, and they also undershot Donald Trump’s performance in 2020. But, as serious pollsters point out, it’s not supposed to be an exact science. If Hillary Clinton had an 83-85% chance of winning in 2016, and Trump 15-17% (as some pollsters said), the reader interprets that as certain victory for Clinton when, in fact, the implied probability for Trump is 1 in 6 – like correctly calling a dice roll. It’s unlikely, but not that unlikely.

Elections and sports events are, of course, different in terms of analysis, but some of our misperceptions are the same. We mistakenly believe the projections are claiming what will happen when they are saying what is most likely to happen. So, why do it all? There is value in that data, even if the results don’t match what the predictions claimed. Indeed, it can be almost as valuable to contrast how an athlete or team or nation was expected to perform with how they did perform.

You can see the latest Gracenote medals table for the Tokyo Olympics here.