What to Make of All These College Rankings?
Rankings provide one perspective on colleges and universities, but prospective students and their families should put them in context.By: Kathleen Harring Monday, September 18, 2023 01:41 PM
Over the past few weeks, a variety of sources have released a deluge of college rankings. First there was Princeton Review, then Niche, then Washington Monthly, The Wall Street Journal, The New York Times and, finally, today, U.S. News. And those are just the rankings the Muhlenberg news site covered.
Here, President Kathleen Harring, whose background in data analysis includes research in teaching and learning theory, program evaluation and more, answers common questions to help make sense of the noise. Harring formerly oversaw institutional research and planning at the College and has had a front-row seat to how these ranking systems have proliferated, and how their methodologies have changed, over her nearly 40 years at Muhlenberg.
How have college rankings systems changed what they measure?
Both new rankings and old rankings are moving to a methodology where outcomes are more important than inputs, particularly outcomes that focus on an economically diverse student body. It’s not just about recruiting and enrolling these students, but graduating them in a reasonable amount of time, at a reasonable cost, into successful post-collegiate lives and careers. In fact, U.S. News introduced a social mobility measure in 2019, looking at the graduation rate of students eligible for federal Pell grants, which are income-based, compared to the graduation rate for all students. In other methodology changes, U.S. News began elevating and increasing the weight of outcomes and particularly outcomes for a diverse student body. You see that with Washington Monthly. You see that in The Wall Street Journal rankings. They all have much more focus on educating for the common good and for a strong democracy, making sure that all students thrive, all students have access to opportunities and all students graduate.
What do you suppose is behind this shift in methodologies?
The most successful colleges, such as Muhlenberg, attract and enroll student bodies that look like America. It's a demographic fact that our country is more diverse racially, ethnically, culturally and economically than ever before. This is where institutions need to do things differently in order to increase access to the current population — to educate future leaders of a diverse society — and rankings need to follow along. We are also seeing questions about the value of higher education in general. The data are very compelling that a degree makes a difference in terms of overall income during your lifetime, but it also increases generational wealth. Earnings potential is one outcome these rankings seek to quantify.
If ranking systems are favoring outcomes more, what are they looking at less?
This year, in U.S. News, students’ high school qualifications (standardized test scores and class rank) contributed less to overall rankings. They’re not a reliable indicator of future success. And, for that matter, economically diverse students are not always able to afford SAT coaches or SAT classes. Having whole segments of students training for tests like these further dilutes their usefulness. Also, particularly after the pandemic, a large number of schools have joined Muhlenberg in being test-optional. Muhlenberg was one of the leaders in shifting to test-optional in the mid-’90s. We made that decision because we practice holistic admissions — the SAT was a data point, but it was not the be-all, end-all. And, we knew that the true talent of a diverse student body often was not reflected by standardized tests. In years past, U.S. News has also decreased the weighting of the peer rating [given by higher ed leaders from other institutions] and, this year, they eliminated alumni giving as a factor.
Some rankings factor in student perceptions — how should we interpret those?
While I think student perceptions are really valuable, it’s important to look at the context and the methodology. For example, look at the response rates. Qualifying for The Wall Street Journal rankings required at least 50 responses. Fifty responses from a school that has 2,000 students? We don’t know how representative those students are, how diverse those students are in terms of all types of features — racially, economically. Is there a diverse set of majors? A diverse set of students across the four class years?
How should prospective students and their families think about these rankings?
Rankings provide one perspective on colleges and universities, but prospective students and their families should put them in context. Looking at rankings to make decisions about whether or not an institution is a good fit is not at all the most reliable way for a prospective student and their family to gather data. Instead, when possible, they should be visiting campus, looking at the facilities and talking to students, faculty members and staff.
Coming out of the pandemic, virtual interviews have increased access for prospective students — we almost exclusively are doing virtual interviews. Prospective students and their families should be asking about opportunities for independent research, internships, studying abroad and other types of experiential learning. The research is pretty clear that students learn more from the kind of hands-on, active and engaged learning that’s core to how Muhlenberg prepares students for leadership in a dynamic global economy.
The rankings provide one piece of data, and now there are lots of different rankings, but actually learning more about the education students receive on college campuses, in the classroom, outside the classroom, in the residence hall, talking to recent alums — those are all things that provide a rich understanding of the actual experience that students have at a specific institution.
