US News may have used incorrect data for one recent ranking

New vendor, new methodology attract questions about accuracy

Three educators from New York took to the Washington Post last week to argue that U.S. News & World Report used the incorrect data in its calculations for 2015's "best high schools." The magazine counters that it simply changed its methodology.

Principal Carol Burris and assistant principal John Murphy from South Side High School and Principal Sean Feeney from The Wheatley School argue that a change in vendors from the American Institute for Research to RTI International coincided with a switch to incorrect data that makes it impossible to fairly compare New York high schools to one another.

"Schools that have traditionally been on the list—and should have continued to be on it—fell off," they wrote.

Specifically, their two schools, which earned "Gold Medal" status every year since 2009, did not earn any rankings this year, despite that "our performance on the measures had improved, not dropped."

The three argue that data used by U.S. News, which included math and English proficiency rates, math distinction scores, and English Language Arts proficiency rates, were all incorrect.

"The data technicians used the wrong state data to evaluate New York schools for proficiency, because they did not understand or ignored how high-school proficiency in English Language Arts and math is determined in New York State," they wrote.

RTI says it used English, Algebra 1, Geometry, and Algebra 2 Regents exams for grade-based calculations. However, Geometry and Algebra 2 are not required across the state, but are optional, write the educators. Additionally, the firm did not account for students who took the Algebra test in eighth grade.

"We place far too much trust in big data," write the educators. "Lists like those of U.S. News and World Report incentivize behavior on the part of the institutions that they rank. They also have real effects on those universities, schools, and hospitals that they label," they say.

In response, U.S. News officials sent an email saying, "The methodology changes are not an error" and that the data were applied in the way the methodology and technical appendix explain.

"These rankings reflect that," officials added (Strauss, "Answer Sheet," Washington Post, 5/21).

The takeaway: A change in methodology—or a mistake in data usage—by U.S. News & World Report led three high school educators to take to the Washington Post and argue high schools in their state cannot be compared fairly in that manner.


Next in Today's Briefing

Are college admissions actually unfair? Expert explains the balancing act

Next Briefing

  • Manage Your Events
  • Saved webpages and searches
  • Manage your subscriptions
  • Update personal information
  • Invite a colleague