Newsletter
How can open science practices be valued and rewarded in the evaluation of research and researchers? How do we use open science practices in research assessment? Which initiatives are available and where to find examples of good practice? Those were the questions answered by Brian Cahill, Lisanna Paladin, Sam Hall and Gareth O’Neill during the MCAA Annual Conference 2024.
The evolution of the current research assessment
Since the topic of the Annual Conference was 10 years of MCAA, Brian Cahill started with a short overview of the policy changes in research assessment within the last decade. The endeavors of a joint effort of MCAA and EuroDoc resulted in May 2019 in the Declaration on Sustainable Researcher Careers headed by Gábor Kismihók (Kismihók et al., 2019).
How can we provide sustainable careers for researchers, how to deploy career management services at organizations, how to put focus on transferable skills training and recognition in PhD, postdoc, and research assessment, or how to provide networking opportunities within and outside of academia. Those were the questions encountered in the following project - NewHoRRIzon in December 2019 (Cohen et al., 2019).
A result of the NewHoRRIzon project was a policy brief describing the uptake of responsible research and innovation practices in MSCA grants within Horizon 2020. The MCAA helped to establish new evaluation criteria for the MSCA call in order to enlarge and modernize the notion of excellence and evaluators and training for MSCA grantees within the multiple dimensions of research. Also, MSCA grants from then on have been supporting knowledge exchange and communities of practice through diverse and inclusive forms of excellence.
The Open and Universal Science project - OPUS
All this joint effort resulted in the start of the OPUS project in September 2022, where the MCAA also played an important part. OPUS develops coordination and support measures to reform the assessment of research and researchers – towards a system that incentivises and rewards researchers to adopt open science practices with a focus on indicators/metrics as well as interventions to support open science.
Details about OPUS were described by the next panelist, Gareth O’Neill: “Research assessment should focus on activities and outputs of researchers. Main principles for the assessment lay in redefinition and new sets of indicators, priorities or the coverage of a full spectrum of activities, which applies across countries, disciplines and organizations.” There are several key initiatives at the EU level and also worldwide, the big current leader of change being the Coalition for Advancing Research Assessment (CoARA).
In OPUS, we understand the term ‘Open Science’ referring to practices providing open access to research outputs, early and open sharing of research, participation in open peer review, measures to ensure reproducibility of results, and involving all stakeholders in cocreation.
Open research Europe and reforming research assessment
Sam Hall, Associate Publisher for Open Research Europe, shared his insights into open research projects and their influence on research assessment. He highlighted the main open science aspects as recommended by CoARA: Promoting qualitative judgment with peer reviews supported by quantitative indicators, following the highest standards of ethics and integrity or reproducibility, focusing on diversity of research outputs, transparent processes and methods, valuing teamwork, and supporting diverse profiles and career paths.
“How can we all specifically support research assessment? By transparency in all processes, reproducibility, and novelty, which altogether brings back recognition and citable reports. Also, anyone can maximize research outputs through a variety of article types,” suggested Sam (European Commission, DirectorateGeneral for Research and Innovation, 2021). One useful tool that can be applied is Contributor Roles Taxonomy (CRediT) to capture every author’s contribution.
Good practice at EMBL
In the last part of the session, Lisanna Paladin shared a good practice used at the European Molecular Biology Laboratory (EMBL). This working place is very international and dynamic, with a high turnover of employees, but at the same time, it offers a very collaborative environment.
Lisanna is responsible for the EMBL Bio-IT project, which is a cumulative initiative and can serve as a model for any research topic or area. Bio-IT is based on the several core principles: flexible and on-demand training, evolving alongside community needs; infrastructure maintenance through collaborative platforms, coding platforms, and any kind of support for collaboration and interaction; comprehensive yet guided information dissemination;targeted to community development, based on volunteer contributions. This initiative is also seen as an opportunity to give back knowledge and skills to society. Bio-IT is all based on do-ocracy, a system where responsibilities are attached to people who do the work.
All of this is also open science – sharing the experience with concurrent check-ups, both top-down and bottom-up. Key message: Open Science is allowing, supporting, and valuing people to share. Importantly, open science is creating alternative career opportunities in science.
Why open science?
“Why is open science important for research assessment in Europe?” asked Brian to the audience.
“It is fair. The typical way of assessing science is very unfair since it is focused only on successful outputs, such as publications in Nature. And that is not always the case. Therefore, it should not be our only ambition. The impact on society is way wider. We should seek alternative ways that are more intuitive. Unfortunately, the system got focused on this one side of the full story,“ with these words Lisanna concluded the session.
Eliška Koňaříková
Managing Editor, MCAA Editorial Board
PR and PhD coordinator
Institute of Molecular Genetics
of the Czech Academy of Sciences
@EliskaKonarik
References
Kismihók, G., Cardells, F., Güner, P. B., Kersten, F., Björnmalm, M., Stroobants, K., Mol, S. T., Huber, F., Seipelt, J., Kretzschmar, W. W., Bajanca, F., Shawrav, M. M., Dahle, S., Carbajal, G. V., Harrison, S., Trusilewicz, L. N., Hnatkova, E., Cophignon, A., Keszler, Á., … Parada, F. (2019). Declaration on Sustainable Researcher Careers. Zenodo. https://doi.org/10.5281/ zenodo.3194228
Cohen, J. B., Bajanca, F., Lam, M. E., Stroobants, K., Novitzky, P., Björnmalm, M., Kismihók, G., & Loeber, A. (2019). Towards Responsible Research Career Assessment. Zenodo. https://doi. org/10.5281/zenodo.3560479
European Commission, Directorate-General for Research and Innovation, (2021). Towards a reform of the research assessment system : scoping report, Publications Office. https://data. europa.eu/doi/10.2777/707440