Mathematics – Statistics Theory
Scientific paper
2010-11-10
Annals of Statistics 2010, Vol. 38, No. 5, 2652-2677
Mathematics
Statistics Theory
Published in at http://dx.doi.org/10.1214/10-AOS794 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of
Scientific paper
10.1214/10-AOS794
For better or for worse, rankings of institutions, such as universities, schools and hospitals, play an important role today in conveying information about relative performance. They inform policy decisions and budgets, and are often reported in the media. While overall rankings can vary markedly over relatively short time periods, it is not unusual to find that the ranks of a small number of "highly performing" institutions remain fixed, even when the data on which the rankings are based are extensively revised, and even when a large number of new institutions are added to the competition. In the present paper, we endeavor to model this phenomenon. In particular, we interpret as a random variable the value of the attribute on which the ranking should ideally be based. More precisely, if $p$ items are to be ranked then the true, but unobserved, attributes are taken to be values of $p$ independent and identically distributed variates. However, each attribute value is observed only with noise, and via a sample of size roughly equal to $n$, say. These noisy approximations to the true attributes are the quantities that are actually ranked. We show that, if the distribution of the true attributes is light-tailed (e.g., normal or exponential) then the number of institutions whose ranking is correct, even after recalculation using new data and even after many new institutions are added, is essentially fixed. Formally, $p$ is taken to be of order $n^C$ for any fixed $C>0$, and the number of institutions whose ranking is reliable depends very little on $p$.
Hall Peter
Miller Hugh
No associations
LandOfFree
Modeling the variability of rankings does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.
If you have personal experience with Modeling the variability of rankings, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Modeling the variability of rankings will most certainly appreciate the feedback.
Profile ID: LFWR-SCP-O-613900