Aggregate ranking of top universities
Like them or not, university rankings are here to stay. Trumpeted in both the higher-ed and broader media, they are the only regularly updated system for comparing institutions globally. For the sector, rankings are among the pivotal inputs of our time – powerful indicators of prestige and consequently key to attracting students and recruiting/retaining academic talent.
Essentially, global rankings cumulate readily quantifiable metrics such as citations and publications, more challenging qualitative indices such as reputation, and the trickier surrogates of teaching quality, variably smoothed for institutional size. Despite differing strengths and emphasis, pan-university rankings have a high degree of consistency both individually and at the top end.
Understandably universities cherry pick from among the myriad scoring systems available - now numbering well over ten. But three stand out as dominant influencers globally – the Times Higher Education (THE), the Academic Ranking of World Universities (ARWU), and the Quacquarelli Symonds (QS). We have combined these three rankings to produce the Aggregate Ranking of Top Universities (ARTU).
Meta-ranking offers an easily digestible if not simplistic ordering. Yet the only publicly available attempts at this to date are bedevilled on two counts. Firstly, neither are contemporaneous, being last updated in 2014 and 2017 respectively. But more importantly, they are flawed mathematically - both use the arithmetic average of THE, ARWU and QS in ranking the world’s top 200 and top 500 universities.[[readmore]]
Averaging data to generate an overall aggregate ranking would be accurate if each ranking system included the same universities and the average score followed a normal (parametric) distribution. In reality the rankings include different numbers of universities and average values do not follow a normal distribution. The consequence is that when the rankings are averaged, all but three universities in the top 200 have a worse average than their aggregate rank. For example at the top end, the leading university, Stanford has an average of 2.67 not 1, whilst lower down, the university with an aggregate rank of 200 has an average of 236.
So the data are best treated with a non-parametric approach, through ordinal ranking. Simply put, this means ordering universities by their aggregate score (=THE+ARWU+QS). This provides a more realistic reflection of a university’s overall position relative to its peers. Using the most recent releases of the three rankings, those with an “average” around 50, rank around 40th on aggregate, those averaging around 100 rank around 85th, 150th up to 130th etc. We first came across this because our own university UNSW Sydney, averaged 69th, yet only 54 universities have a better average or aggregate ranking.
We now make this publicly available as the Aggregate Ranking of Top Universities. The ARTU website orders the top 200 universities listed in all three rankings, and gives data for each of the last eight years.
Professor Ian Jacobs
President and Vice-Chancellor
Professor Nicholas Fisk
Deputy Vice-Chancellor, Research