World Science Forum 2024: Breaking Barriers in Research Assessment: Embracing Open Infrastructures

The World Science Forum 2024 (WSF24) brought together people from all over the world to discuss how research and science can better serve society. One of the inspiring talks came from OpenAIRE CEO, Natalia Manola, whose presentation focused on the current issues around research assessment and how open infrastructures could change the game on that.

N. Manola pointed out a big paradox: while we’re pushing for open science—where research is shared and accessible to everyone—the systems we use to assess research are closed and outdated. These systems are expensive, focus too much on simple metrics like journal impact factors, and often don’t reflect the true value of modern research. For example, data sets, software, and collaborations between fields are essential today, but they’re often ignored in traditional evaluations.

Why Open Infrastructures Matter

Manola argued that open infrastructures are key to addressing these problems; unlike the proprietary systems that many institutions rely on, open infrastructures are inclusive, transparent, and designed to last. They can capture the full range of research outputs, from data and software to collaborations and methods, making assessments fairer and more meaningful.

But creating and using open infrastructures isn’t easy. They require funding, expertise, and a willingness to shift away from familiar systems. Some researchers and institutions are hesitant to make the change. This is the main focus of the work that the OpenAIRE team is working on: to make this transition smoother by creating tools, gathering examples of success, and building guidelines that help institutions embrace open systems.

A Bigger Discussion on Reform

Manola’s talk was part of a larger panel organized by CoARA (Coalition for Advancing Research Assessment). Experts from different countries and fields shared their thoughts on improving research assessment. 

Andrea Balla from Hungary discussed how his country is handling reform. In Hungary, people have traditionally trusted numerical metrics, like citation counts, but there’s now a push to balance these with qualitative methods, like peer reviews, that tell a fuller story.

Tung Tung Chan from Erasmus University spoke about the importance of peer reviewers, whose work often goes unnoticed. She suggested ways to reward them and help smaller research communities adopt better review practices.

Early career researchers were also a focus of the discussion. Pil Maria Saugmann and Katalin Solymosi highlighted the pressures young researchers face to publish as much as possible, sometimes at the cost of quality. They called for systems that support young researchers’ integrity and innovation, not just their publication records.

A Better Way Forward

Manola concluded her presentation by stating the clear challenge ahead: it’s time to move beyond quick, easy metrics and start assessing research based on values that matter. Open infrastructures, like the OpenAIRE Graph, show what is possible; these systems are transparent and collaborative, offering real alternatives to closed systems, like Scopus and Web of Science. Shifting to these systems will take time and effort, but the benefits are huge. Open infrastructures can make research even more inclusive, encourage collaboration across countries and disciplines, and ensure assessments reflect the real impact of science on society. This work is part of the CoARA Working Group, Towards Open Infrastructures for Responsible Research Assessment, that Natalia Manola co-chairs.

A Shared Responsibility

This year’s World Science Forum made it clear that improving research assessment isn’t just one group’s job. It will take collaborative efforts and teamwork from universities, funders, governments, and researchers. Natalia Manola’s insights, along with the input provided by her fellow panelists, offer hope for a future where research isn’t judged by numbers alone, but by its true value and impact. It’s a future that prioritizes openness, fairness, and the greater good - one worth striving for.