Changing the Ecology of Academic Research: Real Impact not Impact Factor
Professor Simon S.M. Ho
The purpose of academic research is to seek truth and new knowledge that enhance social development. Such research is one of the integral responsibilities of a faculty member working in an academic institution, and it is one of the key dimensions of his/her job performance. Faculty members who are not active in research probably use outdated teaching material that may not meet the needs of our fast-changing society.
It is understandable that some faculty members also pursue research to bolster their reputations and achieve promotion and tenure. Collectively, faculty members’ research performance (in terms of funding received, publications and awards, etc.) affects their academic institution’s resources, performance, reputation and rankings. However, these should be by-products and not the main purposes of research. Unfortunately the misuse of resources in the so-called ‘research activities’ in many publicly-funded research-intensive institutions has attracted the public attention in recent years.
Focusing on Quality, not Quantity
When academic institutions increasingly promote the use of quantitative performance indicators in research, many faculty members’ work serves the main purpose of chasing indicators – a dynamic that has numerous negative effects. Under this trend, researchers are often inclined to select agendas or topics that are favoured by the indicator system. This utilitarian approach subsequently discourages more meaningful or valuable research efforts/topics.
Overall, funding bodies and institutions should emphasise the quality of research based on social needs, personal interests/expertise and curiosity. Quality research should contribute to the advancement of new knowledge, have relevance (i.e. problem-based) and create an impact for the betterment of society.
Research for the purpose of publishing more articles in the so-called ‘high-impact’ or SCI/SCII journals would be a waste of time and energy if the publications do not help solve the world’s pressing problems. Thus, emphasising the importance higher quality and impacts over simply counting the total number of projects or publications will help to reduce the ‘publish-more-and-more’ syndrome.
The Myths of Impact Factor
For many years, a journal’s ‘impact factor’ – a problematic measure of how frequently the journal’s articles are cited in other journals – has mistakenly dominated faculty members’ publication choices and institutions’ assessment methods.
The importance of any individual article cannot be assessed based on its citation performance. Even in the highest impact factor journals, some articles are seldom citied by other researchers, and the number of citations is always affected by the popularity of the specific topics.
Some international research funding/assessment bodies have stopped using journal -based metrics such as impact factors as a surrogate measure of the quality or contribution of specific research articles.
Innovativeness and Riskiness
Research is also highly valued as a process, besides outcomes. Nobel Laureate Professor Myron Scholes expressed the following in a speech in Hong Kong in late 2013.
Persistence means that even if we fail, we learn from our failure along the way to our goals. We applaud success, but we also applaud failures that lead to success. Great academics are persistent and willing to fail in an attempt to succeed. Research and development are never easy. The great researchers ‘search’ for new ideas that break the tyranny of the ‘data mining’, and build new models and gain new insights from the information set. This is the critical difference between successful and unsuccessful hunters.
Mechanical assessment systems often discourage scholars from pursuing riskier, possibly innovative projects because it may be years before the first research articles are published.
Research projects feature intellectual, innovative, enterprising processes. In the same way that just taking part in the Olympic Games is lauded because we treasure the participants’ sportsmanship and spirit as much as the results, so should dedicated research participation be revered. Quick results that do little more than echo others’ views have little value, regardless of their final outlets of publication.
Enhancing Teaching Quality
Present-day faculty members are inclined to pursue more research and less teaching to satisfy research assessment requirements and compete for grants. Research-intensive universities’ over-emphasis on research and ranking has been criticised for its role in diminishing the importance of teaching and students’ personal development. These institutions typically look at candidates’ research records rather than at their teaching performance and student development when making recruitment, promotion and tenure decisions.
Universities are not pure research facilities, and institution leaders should encourage and support research activities that reinforce outstanding teaching and learning.
Holistic Assessment of Publications
All of these factors have implications for the assessment of research efforts. Assessing the achievements of researchers should amount to much more than simply counting the number of publications and the journals’ impact factors.
Evaluating a researcher’s contributions requires that some of his/her selected publications be read and analysed – a task that must not be passed automatically to journal editors or replaced with a blind reliance on journal classifications, citation indices, impact factors, etc. Many academic institutions and funding bodies now ask candidates to identify their five best articles. This makes it easier for reviewers to evaluate the selected publications in more detail, and reviewers should read each representative publication carefully to make a comprehensive and fair assessment.
In short, quality research is characterised by at least four factors: innovativeness, relevance, impact and the ability to enhance teaching. Ultimately, top-rated institutions will only look at such factors with little adherence to quantitative research requirements.
(Source: Real Purpose of Academic Research, China Daily (Hong Kong Edition), April 30, 2014.)