Essential Aspects of Tools for Developing Scientific Data Management Plans

2.1. Literature review and tool identification

We conducted systematic searches in Google Scholar, ResearchGate, and OpenAIRE using terms such as ‘data management plan,’ ‘DMP,’ ‘DMP tools,’ and ‘data management plan tools.’ The search covered studies published up to April 2024 in English, Spanish, and Portuguese. These sources were chosen for their broad academic coverage and inclusion of open-access materials. We did not apply a publication date limit to ensure the inclusion of foundational studies and to capture the historical context of DMP tool development.

From the initial search results, we applied inclusion criteria to select the most relevant documents. The primary criterion was that the paper’s abstract explicitly discussed the development, features, or evaluation of DMP tools. This initial screening yielded a set of potentially relevant documents, which were then further refined to include only those that met our criteria of direct relevance, methodological clarity, and practical applicability of results. In total, 19 articles were chosen for in-depth analysis.

Each of the 19 articles was then analyzed to identify the specific DMP tools mentioned. We focused on tools that were publicly available and functioned as standalone platforms or as distinct modules within larger systems. We also included notable derivatives of well-known tools (for example, DMP Assistant, based on DMPonline; and the Brazilian PGD-BR, derived from DMPTool) to assess their adaptability and the emergence of context-specific features.

2.2. Development and justification of evaluation criteria

The evaluation criteria were developed to move beyond a superficial feature checklist and to assess the practical utility and impact of the tools from a researcher’s perspective. The criteria were derived from a synthesis of best practices in research data management, recommendations from funding agencies like the NSF and National Institutes of Health (NSF, 2023; NIH, 2023), the FAIR principles as defined by Wilkinson et al. (2016), and the RDA DMP Common Standard (Miksa, Walk and Neish, 2020). They were also informed by existing comparative studies, such as Jones et al. (2020) and Williams, Bagwell and Zozus (2017), but with a deliberate shift in focus from purely technical infrastructure to user-centric functionalities and their potential to enable better science. The final set of 10 criteria, detailed in Table 1, was designed to be comprehensive, covering aspects from metadata support to security and scalability.

2.3. Tool evaluation process

The identified tools were evaluated comparatively through a combination of documentation analysis and hands-on testing. For each tool, we created test accounts (where possible) and simulated the process of creating a DMP to directly assess its functionalities. This practical approach allowed us to verify claims made in the documentation and gain a deeper understanding of the user experience. The evaluation against the criteria in Table 1 was conducted as follows:

• Supported metadata standards, Support for funding agency templates, and Customization and extensibility: Evaluated by examining the tool’s interface and documentation to see which standards (e.g., Dublin Core, DataCite, RDA DMP Common Standard) were explicitly supported, which funder templates were available, and the extent to which users or institutions could create or modify templates. We distinguished between GUI-based customization and code-level extensibility.

• Integration with data repositories: Assessed by testing and reviewing documentation for any mechanisms linking to repositories. We categorized the level of integration as: (1) providing guidance or links only, (2) allowing metadata export to a repository, or (3) enabling automated or semi-automated data deposit.

• Collaboration capabilities and Training and support resources: Evaluated by exploring features for sharing and co-editing DMPs and by inventorying the availability and quality of tutorials, FAQs, helpdesks, and other user guides.

• Risk assessment and compliance and Security mechanisms: Assessed by reviewing the DMP creation workflow for questions related to data sensitivity (e.g., Personally Identifiable Information (PII); General Data Protection Regulation (GDPR)/Lei Geral de Proteção de Dados (LGPD) compliance) and by checking the tool’s documentation and login process for security features like Secure Sockets Layer (SSL)/Transport Layer Security (TLS) encryption and authentication methods.

• API/machine-actionable support: Determined by checking for a public API documentation and by testing the tool’s export functions to see if they produced structured, machine-readable formats like JSON or XML, particularly those aligned with the RDA DMP Common Standard.

• Scalability: While a full-scale benchmark was beyond the scope of this study, we performed qualitative load-testing experiments on accessible tool instances. This involved simulating multiple concurrent users creating and editing DMPs and progressively increasing the size of a test DMP with additional records while monitoring system responsiveness and throughput. This allowed us to verify that the tools could maintain stable performance under moderately increased demand, providing a practical assessment of their scalability.

This multi-faceted approach, combining literature review, hands-on testing, and a detailed criteria-based analysis, allowed us to highlight the capabilities and limitations of each tool, identifying trends and gaps in current data management practices.

4.1. From static documents to dynamic hubs: Connecting functionalities to the FAIR principles

The shift towards maDMPs represents the most critical evolution in this space. As Wilkinson et al. (2016) state, the FAIR principles are designed to enhance the ability of machines to automatically find and use data. Our analysis shows that while most tools support the FAIR principles conceptually, it is the implementation of specific advanced features that makes them practically achievable. The claim that these features enable FAIRness is not merely conceptual; it is directly supported by how they address the individual principles:

• Findability (F): A tool that exports a DMP as a structured, RDA-compliant JSON file with an embedded persistent identifier (e.g., a DOI for the DMP itself) directly addresses principles F1 (assign a globally unique and persistent identifier) and F3 (metadata include the identifier of the data they describe). When this maDMP is indexed in a searchable resource (F4), it becomes discoverable not only by humans but by automated services as well.

• Accessibility (A): The availability of a public API allows the maDMP to be retrieved via a standardized and open protocol (A1, A1.1), a core feature of tools like OpenDMP and DS-Wizard. Furthermore, even if the underlying research data is removed, the metadata within the DMP remains accessible, fulfilling principle A2 (metadata are accessible, even when the data are no longer available).

• Interoperability (I): Native support for the RDA DMP Common Standard means the tool uses a formal, shared, and broadly applicable language for knowledge representation (I1). By referencing controlled vocabularies for things like licenses or repositories, these tools include qualified references to other metadata (I3), making the information machine-understandable.

• Reusability (R): A well-structured maDMP, rich in attributes as defined by the RDA DCS, inherently provides a plurality of accurate and relevant attributes (R1). When it includes a dedicated field for a data usage license (R1.1) and details about the project’s methodology and contributors (R1.2), it provides the detailed provenance needed for others to reuse the data with confidence.

Therefore, the tangible benefits of maDMPs are directly linked to the FAIR principles:

• Automated Workflows: An API allows a DMP tool to automatically communicate with a data repository. For example, when a dataset is described in the DMP, the tool could use the repository’s API to reserve a persistent identifier (like a DOI) and pre-fill metadata records. Upon project completion, it could trigger the data deposit process, significantly reducing the administrative burden on researchers.

• Enhanced Compliance and Monitoring: Funders and institutions can use APIs to programmatically query DMPs to monitor compliance with data sharing policies, rather than manually reading hundreds of documents. This allows for real-time oversight and a better understanding of data management practices across an organization.

• Living DMPs: A DMP is not a one-time document; it evolves with the research project. Machine-actionability allows the DMP to be updated automatically. For instance, if a new dataset is generated and deposited, the repository could notify the DMP tool via a webhook, which would then update the DMP record accordingly. This ensures the DMP remains an accurate reflection of the project’s data landscape.

4.2. A generalizable decision matrix for selecting the right DMP tool

To address the need for a reusable and generalizable evaluation methodology, as highlighted by reviewers, we propose a decision matrix with a clear scoring system. The ‘best’ DMP tool is context-dependent, and this matrix allows institutions or research groups to select a tool based on their specific needs, resources, and strategic goals.

How to use the matrix: For each dimension, assign a score from 0 (no support) to 3 (excellent support) based on the descriptions provided. The total score will indicate the tool’s alignment with one of two primary profiles: Profile A (Compliance & Usability), which prioritizes ease of use and meeting funder requirements, and Profile B (Integration & Automation), which prioritizes technical capabilities for building automated workflows. This methodology can be applied to any DMP tool, including those not covered in this study (e.g., Argos, DAMAP, RDMO).

Interpreting the results: A high score in the first three dimensions suggests a strong fit for Profile A. A high score in the last three dimensions indicates a strong fit for Profile B. A tool with high scores across all dimensions would be a comprehensive solution, but such tools are rare.

4.3. Application scenarios: Matching tools to research contexts

To illustrate the practical application of the decision matrix, we present two contrasting scenarios:

• Scenario 1: A Multi-Institutional Clinical Trial. This project involves sensitive human data, requires compliance with multiple funders (e.g., NIH, Horizon Europe), and involves researchers across several universities. The primary needs are robust security, clear compliance pathways, and strong collaboration features. Using the matrix, this scenario would prioritize Funder Templates (3), Usability & Support (3), and Collaboration (3). A tool like DMPonline or DMPTool would score highly here and would be an excellent choice.

• Scenario 2: A Computational Social Science Project. This project generates various datasets and software code, which must be versioned and deposited in repositories like Zenodo and GitHub. The team values efficiency and automated workflows. The matrix would prioritize API & Integration (3), maDMP Support (3), and Customization (2–3). The ideal tool would be OpenDMP or DS-Wizard, which are designed for this purpose.

4.4. Technological gaps and a roadmap for future development

Despite progress, our analysis identifies several gaps. The ‘higher-level’ features—such as APIs for automated workflows, persistent identifiers for DMPs, and granular version tracking—are still not widely implemented. These represent the next frontier. We propose a three-phase technological roadmap for the evolution of DMP tools:

1. Phase 1 (Present – Static DMPs): Tools primarily function as form-fillers to generate static, text-based documents for compliance purposes. This is the current state of most tools.

2. Phase 2 (Near Future – Active DMPs): Tools act as integrated hubs, using APIs to connect with other research systems (repositories, ELNs). DMPs are updated throughout the research lifecycle, reflecting real-time project status. Tools like OpenDMP and DS-Wizard are pioneering this phase.

3. Phase 3 (Future – Dynamic DMPs): DMPs become fully automated, dynamic entities that not only track but also help execute data management tasks. They could trigger validation checks, manage access rights based on project milestones, and automate archival processes, truly embodying the ‘living’ document concept.

4.5. Strategic recommendations for key stakeholders

To accelerate the transition towards more effective data management, we offer the following recommendations:

• For Tool Developers: Prioritize the development of robust, well-documented APIs and adopt the RDA DMP Common Standard to ensure interoperability. Focus on the user experience for both researchers (simplicity) and institutional administrators (customization).

• For Institutions & Research Managers: Use the decision matrix (Table 5) to select a tool that aligns with institutional strategy. An investment in a tool must be paired with an investment in training and support to foster a culture of good data management.

• For Funding Agencies: Move towards mandating machine-actionable DMPs using standard formats. Provide clear, structured templates that can be consumed by tools via APIs, rather than as static PDF documents, to facilitate automation and monitoring.