PayScale recently released its 10th annual report on the colleges that provide the best return on investment, but some in higher ed question the influential report's methodology.
To compile the report, PayScale researchers collected survey responses from U.S. college graduates, but restricted respondents to those who met several criteria, such as:
- Do not hold advanced degrees beyond the bachelor's degree;
- Work full-time;
- Are not self-employed or contract employees; and
- Work in the United States.
To predict salary outcomes for 2016 graduates, researchers used a calculation that accounts for:
- Years since each survey respondent graduated college;
- Inflation, based on the consumer price index; and
- School-specific models.
Finally, PayScale also created separate ROI calculations for public institutions for in-state and out-of-state students.
According to PayScale, the schools with the best 20-year ROI are:
- United States Merchant Marine Academy
- Harvey Mudd College
- Massachusetts Institute of Technology
- SUNY Maritime College (for in-state students)
- Colorado School of Mines (for in-state students)
- California Institute of Technology
- SUNY Maritime College (for out-of-state students)
- Webb Institute
- California State University, Maritime (for in-state students)
- Georgia Institute of Technology (for in-state students)
- Colorado School of Mines (for out-of-state students)
- Stevens Institute of Technology
- California State University, Maritime (for out-of-state students)
- Massachusetts Maritime Academy (for in-state students)
- Princeton University
- Stanford University
- Worcester Polytechnic Institute
- Georgia Institute of Technology (for out-of-state students)
- Kettering University
- Rose-Hulman Institute of Technology
But some higher education leaders are questioning PayScale's methodology, Doug Lederman writes for Inside Higher Ed.
Lederman argues that PayScale has fueled the public obsession with ROI and inspired a plethora of other graduate salary rankings. Given the ranking's influence, Lederman rounds up concerns about "fundamental flaws" within its methodology that have been raised by higher education officials and analysts.
Some, like Patricia McGuire, president of Trinity College Washington, argue more broadly that salary should not be our yardstick for measuring colleges.
Others point to drawbacks in PayScale's data and question whether the data is truly representative. Tod Massa, policy research and data warehousing director at the State Council of Higher Education for Virginia, points out that PayScale relies on voluntary, self-reported data, and some colleges have very small sample sets. In explaining its methodology, PayScale notes that its analysts take some steps to counteract limitations for schools with few respondents.
However, Lederman notes that other rankings of graduate salaries run into similar limitations and that PayScale has made consistent efforts to improve the ranking each year.
The most accurate information comes from comparing PayScale's ranking with the College Scorecard and state-level databases, according to Robert Kelchen, assistant professor of higher education at Seton Hall University. "If those data sources match up, I'd feel pretty comfortable," he says (PayScale methodology, accessed 4/19; PayScale ranking, accessed 4/19; Lederman, Inside Higher Ed, 4/18).
How are ROI concerns impacting students' enrollment decisions?
Next in Today's Briefing
5 simple ways to help students with test anxiety