Newsletter
How can open science practices be valued and rewarded in the evaluation of research and researchers? How do we use open science practices in research assessment? Which initiatives are available, and where can you find examples of good practice? These were the questions discussed by Brian Cahill, Lisanna Paladin, Sam Hall, and Gareth O’Neill in a session of the MCAA Annual Conference 2024.
Evolution of current research assessment
Since the topic of the Annual Conference was 10 years of MCAA, Brian Cahill started the session with a short overview of the policy changes in the research assessment within the last years. The endeavours of a joint effort between the MCAA and EuroDoc resulted in the Declaration on Sustainable Researcher Careers headed by Gábor Kismihók in May 2019 (1).
The following project, NewHoRRIzon (2), which started in 2019, delved into the questions of how to provide sustainable careers for researchers, how to deploy career management services at organisations, how to put focus on transferable skills training and recognition in PhD, postdoc, and research assessment, and how to provide networking opportunities within and outside of academia.
The NewHoRRIzon project resulted in a policy brief describing the uptake of responsible research and innovation practices in MSCA grants within Horizon 2020. The MCAA helped to establish the new evaluation criteria for the MSCA call in order to enlarge and modernise the notion of excellence, and to provide training for MSCA grantees within the multiple dimensions of research. From then on, MSCA grants have been supporting knowledge exchange and communities of practice by diverse and inclusive forms of excellence.
The Open and Universal Science project - OPUS
This joint effort set the grounds for the OPUS project (3) to begin in 2022, where the MCAA also plays an important part. OPUS develops coordination and support measures to reform the assessment of research and researchers towards a system that incentivises and rewards researchers to take up open science practices, with a focus on indicators as well as interventions to support open science.
In OPUS, open science is understood as practices providing open access to research outputs, the early and open sharing of research, participation in open peer review, measures to ensure the reproducibility of results, and involving all stakeholders in the co-creation process.
Details about OPUS were given by panellist Gareth O’Neill: “Research assessment should focus on the activities and outputs of researchers. The main principles for the assessment lay in the redefinition and new sets of indicators, priorities, or the coverage of the full spectrum of activities, which applies across countries, disciplines, and organisations.” There are several key initiatives at the EU level and also worldwide, the big current leader of change is the Coalition for Advancing Research Assessment (CoARA) (4).
Open Research Europe and Reforming Research Assessment
Sam Hall, a representative of the European Commission, shared his insights into open research projects and their influence on research assessment (5). He highlighted the main open science aspects as recommended by CoARA: promoting qualitative judgement with peer reviews supported by quantitative indicators, following the highest standards of ethics and integrity or reproducibility, focusing on the diversity of research outputs, transparent processes and methods, valuing teamwork, and supporting diverse profiles and career paths of researchers.
“How can we all specifically support research assessment? By transparency in all processes, reproducibility, and novelty, which altogether brings back recognitions and citable reports. Also, anyone can maximise research outputs through a variety of article types,” suggested Sam (6). He specifically mentioned the CRediT taxonomy, a useful tool that can be applied to capture every author's contribution to a research publication (7).
Good practice at EMBL
In the last part of the session, Lisanna Paladin shared a good practice used at the European Molecular Biology Laboratory (EMBL). This working place is international, dynamic, and with a high turnover of employees, but at the same time it offers a very collaborative working environment.
Lisanna is responsible for the EMBL BioIT project, a cumulative initiative that can serve as a model for any topic or area in research. Bio-IT is based on making sure that training opportunities are flexible, offered on demand, constantly evolving, and dependent on the recipients’ needs. The technological infrastructure and platforms that people use for working together or coding are maintained all the time by the Bio-IT team. Information dissemination is comprehensive, guided, and targeted to community development. All of this is based on volunteering contributions. The Bio-IT initiative is also seen as an opportunity to give knowledge and skills back to society. It is all based on do-ocracy, a system where responsibilities are attached to people who do the work.
All of this is also an open science principle. And the key message? Open Science is allowing, supporting, and valuing people to share. Importantly, open science is creating alternative career opportunities in science.
Why open science?
Towards the end of the session, Brian Cahill asked why open science was important for research assessment in Europe.
Lisanna Paladin concluded by answering that, simply put, open science is a fair way of assessing science: “The typical way of assessing science is very unfair since it is focused only on successful outputs, such as publications in Nature. We should seek alternative ways that are more intuitive because the impact of research on society is way wider.“
1. Gàbor K, et al. 2019. Declaration on Sustainable Researcher Careers. Zenodo. https://zenodo.org/records/3082245
2. Cohen JB, et al. 2019. Towards Responsible Research Career Assessment. Zenodo. https://zenodo.org/records/3560479
3. Open and Universal Science project, https://opusproject.eu/
4. Coalition for Advancing Research Assessment, https://coara.eu/
5. European Commission. Open Research Europe, https://open-research-europe.ec.europa. eu/
6. European Commission. Toward a reform of the research assessment. https://op.europa.eu/en/publication-detail/-/publication/36ebb96c-50c5-11ec-91ac01aa75ed71a1/language-en
7. CRediT. Contributor Roles Taxonomy. https://credit.niso.org/
Eliška Koňaříková
PR and PhD coordinator
Institute of Molecular Genetics
Czech Academy of Sciences
@EliskaKonarik