The recent Times Higher Reputation Ranking also raises a number of more fundamental questions about the production of reputation. If we compare the reputation ranking with the overall THE World Universities ranking, it is striking that the reputation ranking is much more skewed. The top 6 universities eat almost the whole reputation pie. University number 50 (Osaka) has only 6 % of the "amount of reputation" that number 1 (Harvard) has, whereas number 50 in the overall THE ranking (Vanderbilt University) still has 69 % of the rating of number 1 (again Harvard). The reputation is based on a survey (of which the validity is unclear), but how do the respondents determine the reputation of universities of which they direct knowledge (for example because they do not work there)?
A recent issue of the New Yorker has an interesting analysis by Malcolm Gladwell about ranking American colleges (The order of things. What colleges rankings really tell us, The New Yorker, February 14 & 21, 2011, pp. 68-75). His topic is another ranking, perhaps even more famous than the THE Ranking: the Best Colleges Guide published by U.S. News & World Report. This is also based on a survey where university teachers are asked to rank the American colleges. When a university president is asked to assess the performance of a college, "he relies on the only source of detailed information at his disposal that assesses the relative merits of dozens of institutions he knows nothing about: U.S. News." According to Michael Bastedo, an educational sociologist at the University of Michigan, "rankings drive reputation". Gladwell concludes therefore that the U.S. News ratings are "a self-fulfilling prophecy".
The extremely skewed distribution of reputation is in itself an indication that this might also be true for the THE ranking. Performance ratings are ususally skewed because of network and scaling effects. A big research institute can mobilize more resources to produce top quality research, will therefore attract more external funding, and so on: this sustains a positive feedback loop. But if the resulting rankings are also strongly influencing the data that feed into the next ranking, the skewedness of the ranking becomes even stronger.
This would mean that the THE Reputation Ranking does not only show that, in the perception of the respondents, a few American universities plus Oxford dominate the world, it also indicates that these respondents use the THE ranking, and comparable rankings, to fill in the forms that subsequently determine the next ranking.
Thus, this type of ranking creates its own reality and truthfulness.