THERE has been a lot of publicity lately about University of Malaya (UM) getting into the top 100 of the Quacquarelli Symonds (QS) World University Rankings. It has now reached 87th place, ahead of Moscow State University,
St. Andrew’s University and the University of Science and Technology of China. Other Malaysian universities have also done well. Universiti Kebangsaan Malaysia (UKM) is in the top 200. Universiti Putra Malaysia (UPM), Universiti Sains Malaysia (USM) and Universiti Teknologi Malaysia are all in the world top 300.
NOT THE FIRST TIME
It would be well to recall that this is not the first time that UM has been in the top 100. Back in 2004 the first edition of what was then the THES-QS World University Rankings put UM in 89th place and USM made it into the top 200.
But a year later UM fell 80 places and USM dropped out of the top 200. What happened was that in 2004 QS made a simple mistake. They counted Chinese and Indian students and faculty as international, boosting the universities’ scores in the international student and faculty indicators. In 2005, QS realised their mistake and corrected it. The resulting “clarification of data” meant that UM and USM lost a few points and fell dozens of places.
GENUINE ACHIEVEMENTS BY UM
It should be noted that there are now 45 international university rankings listed in an Inventory published by the International Ranking Expert Group, including global, regional, subject and business school rankings. The QS world ranking is only one of them and it may not be the most reliable or the most appropriate for Malaysia. But it is the ranking where UM does best.
These are the ranks of UM in various rankings:
• The 2017-18 Times Higher Education (THE) World University Rankings, which include “teaching” indicators, 351-400
• The US News Best Global Universities, research-based, 301st
• The Shanghai Academic Ranking of World Universities, research-based, 401-500
• University Ranking by Academic Performance (published by Middle East Technical University), research-based, 234th
• Round University Ranking (RUR, published in Russia), which includes “teaching” indicators, 213
• Leiden Ranking (published by Leiden University, the Netherlands), publications indicator 110th, high quality research 264th
• Nature Index, highest quality research, not included in 500 ranked universities.
UM and, to a lesser extent, some other Malaysian universities have made significant progress in research output, the provision of resources and internationalisation. They are not doing so well when it comes to high quality research, as is shown by their failure to break into Nature Index, or quality of graduates or teaching. It also seems that the Malaysian university system is highly differentiated with four or five universities producing substantial research but many institutions doing no research, apart from a few isolated individuals, of any significance.
UM could claim that it has been very successful by noting its scores in the Leiden Ranking, the Round University Ranking and the Best Global Universities. Focussing on the QS rankings as a measure of achievement could be self-defeating.
QS RANKINGS BIASED AND UNBALANCED
Universities and stakeholders should be aware that the QS rankings are not the best measure of academic excellence. Take a look at the indicators that make up the QS world rankings. There are six: academic survey, employer survey, citations per faculty, faculty student ratio, international students, and international faculty.
The weighting of these indicators is very unbalanced. Forty per cent goes to academic reputation, which is about research, and only 20 per cent for citations. The employer survey gets another 10 per cent.
Some other rankings use reputation surveys, but they have smaller weightings. THE has 33 per cent for surveys of research and postgraduate teaching. The Russian based RUR rankings have 22 per cent, and the US News Best Global Universities 25 per cent.
The QS survey asks respondents about the best universities for research in chosen fields and regions. The citations per faculty indicator is supposed to be a measure of research excellence. One would expect that there would be some congruence between the two but that is not always the case. British universities do much better in the academic reputation survey than in the citations. Cambridge is second in the world for academic reputation but 71st for citations, Edinburgh is 24th and 181st, King’s College London 47th and 159th.
How do UK universities manage to do so well for academic reputation? Part of the answer may be that they are living off their intellectual capital, memories of fading scientific glory, but it could also be because the UK and other English-speaking countries are overrepresented in the survey. QS, to their credit, have listed the national affiliations of the participants in their 2018 survey. There are almost as many respondents from the UK as there are from the US, as many from Australia as there are from Russia, more from Canada than from Germany, more from New Zealand than from Switzerland.
Malaysia is also overrepresented. Out of the 80,000 plus respondents to the survey, 4.6 per cent are from Malaysia. That is more than any other country except the United States and the United Kingdom and more than three times the percentage five years ago. It is also more than the combined number for China and India. This is a lot more than Malaysia’s population, number of researchers, research output or research impact.
MALAYSIA DOES BETTER FOR RESEARCH REPUTATION THAN FOR RESEARCH
It is hardly surprising that the performance of UM and other Malaysian universities in the academic reputation indicator, with its 40 per cent weighting, is well ahead of its score for citations. The reputation survey has had a big influence on their ranking success and this is in large part the result of a very large number of respondents coming from Malaysia.
• UM is 99th for academic reputation and 399th for citations.
• UKM is 160th for reputation and below 601st for citations.
• USM is 168th for reputation and 556th for citations.
• UPM is 188th for reputation and 555th for citations.
Nobody is doing anything wrong. QS has its procedures for the survey and they include allowing universities to nominate up to 400 potential respondents and to alert potential supporters to the company’s sign up facility.
These procedures can be justified as a way of maximising participation in the survey and making it fairer and more inclusive by finding respondents from outside the world’s historical elite universities. But they also mean that QS might be suffering from survey inflation with many responses coming from universities with professional consultants and staff dedicated to rankings.
UM’s success and that of other Malaysian universities appears dependent on its research reputation score which has run ahead of its objective research achievements.
CONSIDER OTHER RANKINGS
UM is merely promoting its best interests by focusing on the QS rankings and it has every right to do so. The Malaysian government should, however, serve the interests of potential students and other stakeholders by keeping them informed about the position of Malaysian universities in a broad range of rankings, especially the Shanghai and Leiden rankings, which are relatively consistent and stable and useful for evaluating research.
The international ranking of university teaching is only just starting but if an assessment of general university quality is needed then it would be better to use the RUR rankings, which contain five out of six indicators used in the QS rankings plus another 15, none of which has a weighting of more than 10 per cent.
UM has good reason to be proud of its steady and genuine progress reflected in the Shanghai, Leiden and RUR. These should be given as much publicity as the QS world and regional rankings or more. The QS rankings should be treated with caution since a high score might be erased by changes in the collection and weighting of reputation survey responses.
The writer is News Editor for the International Ranking Expert Group, Brussels, and writer of the blog, University Ranking Watch
Get the latest World Cup 2018 scores, highlights and updates from our dedicated news page, click here