Rankings and Ratings… It’s the world we live in...
Someone asked me the other day what I thought of the program rankings recently produced by “Academic Analytics'” and published in the November 2007 issue of The Chronicle of Higher Education. This for-profit company, owned in part by the State University of New York at Stony Brook, recently released its third annual Faculty Scholarly Productivity Index — a ranking of graduate programs at research universities based on what they report to be “an objective measurement of per-capita scholarly accomplishment.”
Frankly, I'm always a bit skeptical of these reports. While several areas of our program are ranked in the “Top 10” in recent studies, should we be ranked “2nd” nationally by “Academic Analytics”? I know the rating process did not include variables I think are important including information on our efforts to diversify our faculty and student body or our efforts to internationalize our program. However, as I took a very close look at the components of this ranking it started to make sense to me. Of course, if we had been ranked lower it would have made less sense!
At the same time I realize that this is just one data point. In February, the NRC will release its more authoritative report and that assessment is likely to have more impact with university administrations. I also know that other surveys, rankings, ratings and reports will continue to appear and we will be regularly asked to provide data on numbers of journal articles, grant dollars, awards, etc. for the metrics that each of our colleges and universities require for various purposes. At the end of the day, what does this really mean? I'm not sure.
What I do know is that these metric analyses are unlikely to cease and that readers will peruse them, sift through them and use them to make choices and recommendations. Maybe no single survey or study really captures what we do or “accurately” ranks programs but you can get an overall sense of things when you look at enough of these over time. Still everyone’s experience is different and programs of many kinds have value that are not captured in these studies. So much of our experience in a graduate program depends on that “special mentor”, that moment when a piece of literature brings clarity to our thinking or that time when we finally “get it” in a conversation with our peers.
Of course, in the end, it's better to be on the list than not :).
Frankly, I'm always a bit skeptical of these reports. While several areas of our program are ranked in the “Top 10” in recent studies, should we be ranked “2nd” nationally by “Academic Analytics”? I know the rating process did not include variables I think are important including information on our efforts to diversify our faculty and student body or our efforts to internationalize our program. However, as I took a very close look at the components of this ranking it started to make sense to me. Of course, if we had been ranked lower it would have made less sense!
At the same time I realize that this is just one data point. In February, the NRC will release its more authoritative report and that assessment is likely to have more impact with university administrations. I also know that other surveys, rankings, ratings and reports will continue to appear and we will be regularly asked to provide data on numbers of journal articles, grant dollars, awards, etc. for the metrics that each of our colleges and universities require for various purposes. At the end of the day, what does this really mean? I'm not sure.
What I do know is that these metric analyses are unlikely to cease and that readers will peruse them, sift through them and use them to make choices and recommendations. Maybe no single survey or study really captures what we do or “accurately” ranks programs but you can get an overall sense of things when you look at enough of these over time. Still everyone’s experience is different and programs of many kinds have value that are not captured in these studies. So much of our experience in a graduate program depends on that “special mentor”, that moment when a piece of literature brings clarity to our thinking or that time when we finally “get it” in a conversation with our peers.
Of course, in the end, it's better to be on the list than not :).
posted by hsypher at 2:51 PM
0 comments
