Competitive Balance in Rugby League

One month ago saw the best Rugby League clubs from the European Super League and Australasia’s National Rugby League face each other in the World Club Series. St Helens took on the Sydney Roosters, Wigan faced the Brisbane Broncos and Leeds battled with the North Queensland Cowboys for the title of World Club champions.

The results left no doubt about which of the two is the dominant, stronger league – all three English clubs were completely outplayed by their Australian rivals, failing to score more than 12 points while conceding over 30 in all games.

That the NRL is the best rugby league competition is the world is nothing new. However, the ease with which the Australian clubs have been able to beat the best teams in the Super League has naturally lead to some soul searching. As relatively recent memories of Leeds, Saints and Bradford winning that competition contrast with the heavy losses of recent years, most English fans are asking themselves how they can possibly narrow the gap between the leagues.

Amid near-consensus that more investment in the game’s youth level is necessary, one commentator has suggested that, while top Super League clubs have decreased in quality over the last 4 to 5 years, the league as a whole has become more competitive, and needs to grow together.

I wonder: has it really? And how can we verify that?

I turned to a famous metric developed by Roger Noll and Gerald Scully, aptly named the Noll-Scully, to investigate whether Super League’s competitiveness has really advanced in recent years. This metric “compares the actual performance of a league to the performance that would have occurred if the league had the maximum degree of competitive balance (in the sense that all teams were equal in playing strength)”

To calculate this metric, you need first to calculate the Idealized Standard Deviation, defined as the average number of wins (in the case of sports like baseball and basketball, with no draws) or the average number of points (for sports like soccer, rugby league and hockey), divided by the square root of the number of games each team plays.

The Noll-Scully is then calculated by dividing the actual standard deviation of the number of wins or points in a certain competition by the Idealized Standard Deviation. A return of 1.0 would mean the competition is balanced, that is, the actual balance matches the ideal. Above 1.0, the greater the number is, the less balanced the competition is; and in the rare case the result is a value below 1.0, it means teams are closer in wins or points than we would have expected.

I have applied this method for the past 10 seasons of the Super League. The results are summarized in the chart below:


For the 2015 season, I calculated the Noll-Scully indicator using regular season results only, that is, after 23 games played by each team, before they are divided between Super 8s and Qualifiers. I did also calculate, however, the Noll-Scully for the top 8 teams after the Super 8s (30 games played by each club) and the result was encouraging: 1.03, extremely close to the ideal result of 1.0. Despite the flaws in that format, it seems that there is a considerably stronger balance between the teams in the Super 8s, and in this respect, at least, the Super League appears to have delivered on its promise of a more exciting competition where every game counts. Or perhaps, as our commentator suggested, this might have simply happened through a decrease in quality of the top sides; or even a combination of both.

Nonetheless, the downward trajectory of the Noll-Scully index in the last 5 years appears to vindicate our commentator’s observation that the Super League’s competitiveness has increased in that period – with the exception of 2014, when a small increase in the Noll-Scully number was observed, from 2011 to 2015 it has steadily gone down by a total of 27%. And indeed, by this metric the Super League XX (2015) was the most balanced competition of the past 10 year sample.

Moreover, if we are thinking about the gap between the NRL and Super League, it is worthy investigating how both leagues compare in that regard – could one of the NRL’s strengths be its competitive balance? I followed the same process as before to calculate the numbers below for the NRL, which I plotted along with the Super League indexes:


Make no mistake: in terms of sponsor’s revenue, investment in youth systems, overall quality and, as we can see above, competitive balance, the NRL is quite ahead of the Super League. Catching up the best rugby league in the world will not be easy – the Super League does indeed need to grow together.