Ramblings on Rankings

Two weeks ago, U.S. News & World Report released its 2013 edition of America’s Best Colleges. For the second year in a row, Harvard and Princeton Universities tied to top the list of national universities, as Yale remained entrenched at third. In contrast, there was significant movement lower on the list, as the five-way tie for the fifth spot from last year’s ranking dissolved. The University of Chicago moved up to join Columbia University at fourth, inching ahead of Stanford University and the Massachusetts Institute of Technology, which tied at sixth. The California Institute of Technology, which in the 2000 edition managed to usurp the number one spot, dropped to 10th this year. As for the nation’s best liberal arts colleges, it was business as usual with Williams, Amherst and Swarthmore Colleges dominating the category. Amherst was ranked at second, while Williams skulked its way to the top slot. A cursory perusal of the rankings reveals some obvious deficiencies (the most egregious being the inflated position of Williams College). Nonetheless, U.S. News remains the most widely read and respected college ranking in publication, and it is therefore worth discussing how seriously it should — and should not — be taken.

U.S. News bases its ranking methodology on a variety of quantitative and subjective criteria. First, it categorizes institutions based on the type of higher education they offer. For example, liberal arts colleges, such as Amherst, which are devoted exclusively to undergraduate education, are listed in a separate ranking than national universities, which also offer master’s and Ph.D. programs. (A report published by Forbes in 2010, which ranked both liberal arts colleges and national universities but with a different methodology, placed Amherst at third, ahead of both Harvard and Yale.) Next, U.S. News collects, weighs and compiles 16 indicators of academics excellence. Some of the indicators, such as standardized test scores, are objective, whereas others, such as a peer assessment score, are subjective. Most data is self-reported by colleges, and while the publication strives to ensure accuracy, institutions — even elite ones such as Claremont McKenna College and Emory University — have in the past reported false data. Finally, U.S. News develops a ranking based on each institution’s composite weighted score.

Although the complexity of the U.S. News methodology lends it a veneer of objectivity, the report is far from scientific. A substantial percentage of the composite score is derived from subjective factors; academic reputation amounts to over 20 percent of the score. Furthermore, U.S. News assigns a weight to each indicator “that reflects [its] judgment about how much a measure matters,” which is but a tactful way to say: arbitrarily. For example, faculty resources are weighed as 20 percent of the total score and financial resources as 10 percent. Exactly why faculty resources are more important, moreover precisely twice as important, as financial resources is never explicated. Furthermore, U.S. News has readjusted its formula in the past in order to produce more savory results. The success and legitimacy of the U.S. News methodology does not — as commonly believed — proceed from producing controversial or sensationalist results, but rather from being congruent with pre-existing conceptions. To contrast, the Forbes ranking of America’s Top Colleges, which places the University of Notre Dame twenty-two places ahead of Dartmouth College, hardly seems credible. Therefore, after Caltech anomalously rose to the number one spot in the 2000 edition, displacing Harvard, Yale and Princeton Universities, U.S. News applied a “logarithmic adjuster” that disproportionately affected categories in which Caltech held a substantial advantage over Harvard, Yale and Princeton; consequently, in the 2001 edition, the three aforementioned institutions returned to their perennial positions at the top three.

If the U.S. News rankings are so flawed, what, then, should determine what constitutes the best college? The classic answer, as every high school counselor will advise, is: it depends. Each individual is unique; every college offers a distinct academic and social environment; thus, the optimal college experience depends on finding the best fit. To make an all-encompassing statement that a single college is the best for every individual, as U.S. News implicitly does, is fallacious. Nonetheless, the U.S. News methodology continues to represent a false degree of precision. While ranking Northwestern University above Northeastern University is probably a good call, to say decisively that Columbia University is better than Stanford University because its composite score is greater by exactly one point (on a 100-point scale) is spurious. Ultimately, the measured differences in the top universities and colleges are so marginal that a decision must come down to personal preference and fit.

Despite its shortcomings, the U.S. News & World Report College Rankings remain lucrative and influential. In 2007, three days after releasing the college ranking, the U.S. News Web site had received 10 million page views compared to an average 500,000 views in a normal month. In addition, the printed college-ranking issue generally sells 50 percent more than a routine issue. Although the U.S. News rankings can be useful resources when searching for colleges — as they provide a comprehensive compilation of relevant data — they tend to be misused and abused by both students and colleges. Anecdotes of students of committing to a certain institution simply because of a higher rank do little to recommend the rankings. Responding to the sway of college rankings over public perceptions, some colleges have been inclined to adopt some objectionable practices, such as intensively drawing upon Early Decision (which adversely affects low-income applicants) and allegedly rejecting overqualified applicants to boost yield rates. Taken with a grain of salt, however, the U.S. News College Rankings can be innocuous and amusing. After all, it is always nice to see Amherst on the front page.
While Amherst ought to congratulate itself for being recognized yet again as one of the best liberal arts colleges in the nation, let’s not give U.S. News too much credit for stating the obvious.