News

Published

May 3, 2021

Author

Elli Papadopoulou

Argos service and the Science Europe DMP evaluation rubric

May 3, 2021

The Science Europe DMP evaluation rubric and the Practical Guide to the International Alignment of Research Data Management

Science Europe released an extended version of the Practical Guide to the International Alignment of Research Data Management that offers targeted guidance for organisations, scientific communities, as well as individual researchers, to organise research data and preserve it appropriately. It includes a brand-new rubric to facilitate the evaluation of a data management plan (DMP) and also presents core requirements for DMPs, criteria for the selection of trustworthy repositories, and guidance for researchers to comply with organisational requirements. OpenAIRE was one of the organisations that participated in the consultation phase of the new version of the Science Europe DMP evaluation rubric with contributions from its RDM Task Force and Argos.

Argos - an interlinked Data Management Planning service

OpenAIRE has developed Argos, a tool for Data Management Planning which is based on the OpenDMP open source software. Argos is a ready-to-use RDA compliant service for creating and sharing DMPs. It is hooked into the OpenAIRE ecosystem and interconnects with its underlying services and the Research Graph to offer maximum capabilities for contextualisation, integration, validation and monitoring of DMPs.

More specifically, Argos ‘closes’ the DMP lifecycle by being fully integrated with Zenodo, offering the option to publish DMPs as outputs in an open and FAIR manner. This happens, among other things, by assigning DOIs and licenses and by supporting DMPs as living documents through versioning control. Furthermore, Argos is easy to use as key DMP entities are automatically linked to the OpenAIRE Research Graph, i.e., grant-ids, repositories, research outputs, services, people and semantics ensuring verification and integrity of research (check also useful resources at the end).

From the experience and unique viewpoint of Argos we see the following challenges and provide suggestions to Science Europe along with an invitation to explore further collaboration.

Areas of potential exploitation and collaboration

The next phase in the Argos development is to make the connection with the OpenAIRE Funder Monitoring Dashboard. This is important because it enables for indicators regarding DMP outputs to be designed and analysed based on funders criteria. That practically means that funders will be able to customise indicators for measuring DMPs characteristics and usage (e.g. number of DMPs per discipline, number of DMPs reusing data, number of DMPs with datasets deposited in x repositories, etc) on their OpenAIRE dashboards. The actual measurements / statistics rely on input data from the Research Graph.

Challenge 1. The global RDM community is moving towards machine-actionable solutions and tools contributing to open science and FAIR ecosystems. It is therefore important to commit to an evaluation mechanism that supports and integrates the next generation of machine-actionable DMPs, such as Argos outputs.

Suggestion: New criteria could be defined in consultation with the RDA DMP Common Standard properties and tested against existing DMPs (i.e. plain text documents) and maDMPs.

How Argos can help: Argos could provide the space for those tests (sandbox) to be performed by exploiting its data model and the Research Graph.

Challenge 2. Argos participation in the RDA Hackathon in 2020 stressed that there is a complexity and sensitivity on the approach with how re-used datasets should be handled in a DMP, mainly due to the fact that:

a. they already carry relationships with other research outputs,

b. have, possibly, already followed open and FAIR practices, and

c. are, possibly, already documented in a DMP.

Rising questions concern how re-used data are captured in DMPs (fields needed to validate copyrights and data ownership, etc). Existing datasets indexed in a repository or registry can be claimed by DMP tools and return back metadata about title, authors, formats, licenses, etc.

Suggestion: Separate re-used data from new data collected or generated that are described in order to be reusable.

How Argos can help: Argos could provide the means to incorporate aforementioned challenges that would support informed decisions. The latest version of Argos already incorporates specific fields for reused data so that the information about how they are used and how they can further be exploited is evident. Similarly, there are fields that draw information from OpenAIRE so the described datasets can be identified, located and accessed.

Challenge 3. Finally, some questions contained in DMPs diverge from researchers' core understanding and skills. As in Open Science practices, implementation of DMPs should be a collaborative effort supported by the collective skills Open Science ecosystems require stakeholders to thrive in. To accommodate this communication, DMPs could be structured in a way that it is clear for researchers to which parts of the DMP fall solely under their responsibility (such as data formats), and which parts are tied to external factors (such as repository preservation policies).

Suggestion: The requirement of creating DMPs by funders have standardised and strengthened qualitative characteristics in data management processes, such as data organisation. Additionally, the use of DMP tools has helped to familiarize users with data management concepts and to mainstream RDM best practices. Today, some DMP tools are found to embed data literacy methods as well as support communication between researchers, who have to complete a DMP, and data stewards/ librarians/ repositories/ national officers, who are more knowledgeable of internal procedures and policies. This shared responsibility could be facilitated by service integrations in DMP tools and by incorporating flexible, collaborative solutions.

How Argos can help: Argos offers both options, yet could be further exploited for the implementation of innovative, perhaps automated, solutions.

Remarks and Orientation

Argos enhances OpenAIRE’s continuous efforts towards Open Science Commons and a Linked Open Data environment that brings data closer to where they are stored and managed, aiming at better reproducibility and integrity of research.

Argos ma-DMP outputs and their exploitation in the OpenAIRE ecosystem

Argos integrates OpenAIRE underlying services, such as Zenodo, and progressively interconnects with others such as the Funder Monitoring Dashboard. A pilot is already in line with CHIST-ERA where Argos is called to assist with automating and simplifying internal evaluation workflows that the funders’ consortium has set based on their Open Science / RDM policy.

DMP evaluation is still today a less clear and standardised process whether it is happening at the proposal stage, during or at the end of the research project. For years now, OpenAIRE has been committed to supporting funders’ Open Science endeavours starting with the European Commission’s Open Access requirements back in 2007. Today OpenAIRE operates as a legal entity and continues to collaborate with a wide range of stakeholders and providers to develop and reinforce added value services in its portfolio and the EOSC. Argos welcomes Science Europe and other interested parties who share the same enthusiasm and eagerness regarding DMPs to work together on further exploring the area of DMP evaluations and assessments through Argos dynamic data model and integrations.

DaMaLOS paper on Data Management Plans and Linked Open Data: exploiting machine actionable data management plans through Open Science Graphs https://repository.publisso.de/resource/frl:6423283

Written by: Elli Papadopoulou

Contributors: Elly Dijk, Ellen Leenarts and Iryna Kuchma

Editors: Gwen Frank, Paula Moura, Emilie Hermans and Natalia Manola