Time to Sign Agreement on Reforming Research Assessment

Numerous organisations including the European Commission have signed the Agreement on Reforming Research Assessment. The Agreement was inaugurated in 2022 by the Coalition for Advancing Research Assessment (CoARA). It was designed by more than 350 organisations from 40 countries. A half year later, the CoARA counts more than 500 organisations such as research funders, universities, institutes and infrastructures, associations of researchers, authorities, etc. The Agreement includes principles, commitments and timeframe for reforms of research assessment coupled with the CoARA's principles. Research assessment refers to researchers, research projects, research units and research organisations, and does not consider research performance at the level of individual economies. It builds on the San Francisco Declaration on Research Assessment (DORA) dating back to 2012. The DORA introduced several important recommendations for improving the evaluation of the quality of research outputs.

Why Agreement on Reforming Research Assessment?

The Agreement is a result of the need to reform research assessment practices. The overall goal is reducing dependence on journal-based metrics. The key measure of a researcher's contribution should be the intrinsic value of research instead of journal impact measures and citations. By dethroning 'publish or perish' culture, society at large will benefit from the higher quality and impact of research. The Agreement's vision is that

the assessment of research, researchers and research organisations recognises the diverse outputs, practices and activities that maximise the quality and impact of research.

The Agreement encompasses three types of assessments. First, research performing organisations and research units by assessment authorities, research funding and performing organisations. Second, the assessment of research projects and third, assessment of individual researchers and research teams. The purpose is to ensure adequate allocation of funding, increase public investment accountability and inform decisions on research priorities, making prize and award decisions, and recruitment and promotion of researchers.

Ten Commitments of the Agreement

The Agreement's core part is the diversity of research activities and practices, with a diversity of outputs, and reward early sharing and open collaboration. These may include, but are not limited to: peer review, training, mentoring and supervision of Ph.D candidates, leadership roles, entrepreneurship, knowledge valorisation, and industry-academia cooperation. Research outputs, beyond scientific publications, could be data, software, models, methods, theories, algorithms, protocols, exhibitions, policy contributions, etc. Also, it sets the qualitative difference between scientific fields and rewards open science and relevance of research for the whole society. It is based on the following ten commitments:

1. Recognise the diversity of contributions to, and careers in, research in accordance with the needs and nature of the research.
2. Base research assessment primarily on qualitative evaluation for which peer review is central, supported by responsible use of quantitative indicators.
3. Abandon inappropriate uses in research assessment of journal- and publication-based metrics, in particular inappropriate uses of Journal Impact Factor (JIF) and h-index.
4. Avoid the use of rankings of research organisations in research assessment.
5. Commit resources to reforming research assessment as is needed to achieve the organisational changes committed to.
6. Review and develop research assessment criteria, tools and processes.
7. Raise awareness of research assessment reform and provide transparent communication, guidance, and training on assessment criteria and processes as well as their use.
8. Exchange practices and experiences to enable mutual learning within and beyond the Coalition.
9. Communicate progress made on adherence to the Principles and implementation of the Commitments.
10. Evaluate practices, criteria and tools based on solid evidence and the state-of-the-art in research on research, and make data openly available for evidence gathering and research.

Should Western Balkans join?

By signing the Agreement, research and funding organisations agree to recognise and reward researchers' various contributions beyond citations and securing money for projects. Western Balkans' research output including productivity has increased over the last years but remains modest compared to the European regions. The region's participation through European funding instruments The EC has signed both the Agreement and DORA in its capacity of a research funding organisation. In other words, Western Balkans' policy and research organisations could follow suit. At the moment of writing this article, six research organisations from the Western Balkans have signed the Agreement. The front-runner is North Macedonia with three signatories: Ss. Cyril and Methodius University in Skopje, University "St. Kliment Ohridski" - Bitola and Institute for Research and European Studies (IRES). It is followed by Institute for Sustainable Development Environment and Tourism from Albania, Center for Development Evaluation and Social Science Research from Bosnia and Herzegovina, and Physical Activity and Sports Tech for Healthy Lifestyles society from Montenegro. These pioneering examples from the Western Balkans paved the way for others to join the Agreement. Perhaps the time to act is ripe.