Table 1
Evaluation criteria for DMP tools.
| CRITERIA | DESCRIPTION |
|---|---|
| Supported metadata standards | Does the tool allow researchers to report and use metadata standards for their data (e.g., the RDA DMP Common Standard schema or project-specific standards)? Supporting standard metadata is fundamental for ensuring data interoperability and reuse (FAIR principle I). |
| Integration with data repositories | Does the tool provide direct integration with data repositories (e.g., automated deposit or retrieval of repository metadata)? This simplifies complying with open-data mandates by easing the deposit and management of research data in repositories. |
| Support for funding agency templates | Does the tool offer templates or guidance aligned with the requirements of specific funding agencies or institutions? Compliance with funder guidelines is essential for project approval and funding; most tools meet this through customizable templates or profiles. |
| Collaboration capabilities | Does the tool allow collaborative development and management of the DMP? Collaborative features promote cooperation among research team members and stakeholders, enabling an integrated approach to data management. |
| Training and support resources | Does the tool provide tutorials, documentation, technical support, or other resources to help users effectively employ the tool? Ample support and training materials facilitate the adoption of good data-management practices by users. |
| Risk assessment and compliance | Does the tool include support for identifying sensitive data and meeting data protection regulations (e.g., GDPR in Europe or LGPD in Brazil)? It may ask researchers to declare sensitivity and applicable laws, ensuring measures like encryption or anonymization are considered. |
| Customization and extensibility | Does the tool allow customization of DMP templates and extensibility of content to meet specific project needs? Customization enables researchers and organizations to integrate their particular guidelines and requirements into DMPs. |
| API/machine-actionable support | Does the tool offer an API or output machine-readable formats (e.g., JSON, XML) to support automated workflows? API access and machine-actionable DMPs enable integration with other systems (repositories, project management tools) and facilitate automated data sharing processes. |
| Security mechanisms | Does the tool implement data security measures such as encryption, access controls, and secure authentication? Robust security protects sensitive information in transit and at rest, ensuring confidentiality and integrity of research data. |
| Scalability | Can the tool scale to accommodate growing data volumes and numbers of users without compromising performance? Scalability was evaluated via load testing with increasing user concurrency and dataset sizes to verify stable performance under high demand. |
Table 2
Documents retrieved from each source.
| SOURCE | ENGLISH | SPANISH | PORTUGUESE | TOTAL |
|---|---|---|---|---|
| Google Scholar | 8 | 3 | 4 | 15 |
| ResearchGate | 9 | 4 | 4 | 17 |
| OpenAIRE | 4 | 2 | 1 | 7 |
| Total | 21 | 9 | 9 | 39 |
Table 3
Identified DMP tools for evaluation.
| TOOL | ORIGIN/CONTEXT |
|---|---|
| Argos | OpenAIRE (Europe) |
| BioDMP | (Brazil, biology-specific) |
| Data Stewardship Wizard | Portage Network (Canada) |
| DataWiz | Leibniz Institute (Germany) |
| DMP Assistant | Portage Network (Canada; based on DMPonline) |
| DMP Opidor | France (European integration) |
| DMPonline | Digital Curation Centre (UK) |
| DMPTool | DCC/University of California (US) |
| Dmptuuli | Finland |
| DMPTY | Clarin-D (Germany) |
| Easy DMP | Norway |
| FioDMP | Fiocruz (Brazil) |
| GAMS DMP | GAMS Software (Global) |
| LabArchives (DMP module) | LabArchives (US) |
| NSD DMP | Norwegian Centre for Research Data (Norway) |
| OpenDMP | Brazil (open-source, machine-actionable focus) |
| Pagoda | Madroño Consortium (Spain, EU Horizon) |
| PARTHENOS DMP | EU PARTHENOS project (Archaeology, Europe) |
| PGD-BR | Brazil (based on DMPTool, for Brazilian agencies) |
Table 4
Detailed comparison of selected DMP tools.
| CRITERIA | DMPTOOL | DMPONLINE | PGD-BR | DS-WIZARD | OPENDMP |
|---|---|---|---|---|---|
| Supported metadata standards | Allows selection from a list (e.g., Dublin Core). Exports DMPs with basic metadata. Focuses on funder compliance over specific schemas. | Supports RDA DMP Common Standard via API. Allows linking to various standards in guidance text. | Inherits DMPTool’s functionality, with templates adapted for Brazilian standards (e.g., DC-BR). | Natively built around a knowledge model. Highly structured, can map to any standard, including RDA DMP Common Standard. | Natively uses a machine-actionable model based on the RDA DMP Common Standard. Focus is on structured metadata output. |
| Integration with data repositories | Provides guidance and links to repositories. Some institutional customizations may add deeper integrations. | API allows for integration. Some instances have direct integrations with repositories like Zenodo and Figshare. | Primarily provides guidance and links, similar to the base DMPTool. | Designed for integration. API can connect to repositories to pre-fill metadata or push/pull information. | API-first design. Enables automated deposit of DMPs and associated metadata to integrated repositories (e.g., Zenodo). |
| Support for funding agency templates | Extensive library of templates from US funders (NSF, NIH) and institutions. Core feature. | Extensive library of templates from UK/EU funders (UKRI, Horizon Europe) and institutions. Core feature. | Specialized in templates for Brazilian funding agencies (FAPESP, CNPq). | Highly customizable ‘Knowledge Models’ can be built to match any funder’s requirements, but requires initial setup. | Flexible template creation, but comes with fewer pre-built templates compared to DMPTool/DMPonline. |
| Collaboration capabilities | Allows co-ownership and read/write/admin permissions for multiple users on a single DMP. | Supports collaborative editing with different permission levels. Users can request feedback from administrators. | Inherits DMPTool’s collaboration features. | Supports real-time collaborative editing and commenting on DMPs. | Supports user roles and permissions for collaborative plan management. |
| Training and support resources | Extensive help desk, FAQs, and video tutorials provided by the California Digital Library. | Comprehensive guidance, documentation, and active user community support provided by the Digital Curation Centre. | Documentation and support provided by IBICT, focused on the Brazilian context. | Good documentation for developers and users. Support is available via GitHub and community channels. | Documentation is primarily API-focused. Support is community-driven via its open-source repository. |
| Risk assessment and compliance | Templates include questions about sensitive data, ethics, and data protection. | Guidance often includes specific advice for GDPR compliance and handling sensitive data. | Templates are adapted to include questions relevant to Brazilian data protection law (LGPD). | The question-and-answer format allows for complex branching logic to guide users through risk assessment based on their answers. | Includes fields for data sensitivity and licensing, ensuring these aspects are considered in the plan. |
| Customization and extensibility | High. Institutions can customize themes, guidance, and create their own templates via a user-friendly admin interface. | High. Open-source code allows for deep customization. Institutions can create their own templates and guidance. | Customized instance of DMPTool, demonstrating its extensibility for national contexts. | Very high. The entire questionnaire (Knowledge Model) is fully customizable without coding. Open-source. | High. As an open-source, API-driven platform, it is designed to be extended and integrated into other systems. |
| API/machine-actionable support | Provides a REST API for programmatic access to DMPs. Exports in JSON, but not fully aligned with RDA standard. | Provides a REST API. Can produce machine-actionable DMPs compliant with the RDA DMP Common Standard. | Inherits DMPTool’s API capabilities. | API-first design. Produces highly structured, machine-actionable output based on its internal knowledge model. | Core feature. Entire tool is built around a RESTful API and produces RDA-compliant machine-actionable DMPs by default. |
| Security mechanisms | Uses SSL/TLS encryption. Authentication via institutional single sign-on (SSO) (e.g., Shibboleth) or local accounts. | Uses SSL/TLS encryption. Supports institutional SSO. Robust access controls. | Implements standard security measures, including SSL/TLS and institutional authentication. | Standard web security (SSL/TLS). Authentication can be integrated with institutional systems. | Standard web security (SSL/TLS). Authentication via tokens for API access. |
| Scalability | Proven to scale for a large number of users and institutions across the US. Stable under our load tests. | Proven to scale for a large, international user base. Stable under our load tests. | Performance is dependent on its hosting infrastructure but is based on a scalable architecture. Performed well in tests. | Designed to be scalable. Performance in our tests was stable, though it has a smaller user base than DMPTool/DMPonline. | Architecture is inherently scalable. As it is often self-hosted, scalability depends on the deployment environment. |
Figure 1
This bar chart clearly shows the evaluation scores for different DMP tools based on comprehensive criteria including basic functions, technical aspects, and usability. The quantitative scores make it easy to compare the relative performance of each tool: Data Stewardship Wizard (DSW): 703.5 points, DMPTool: 615.5 points, RDMO NFDI4Ing: 549.5 points, DMPonline: Lower score (exact value not specified in source), EasyDMP: Lower score (exact value not specified in source). The visualization uses a clean, professional style with clear labels, gridlines, and distinct colors for each tool. This makes it much easier to understand the performance differences between tools compared to the original simple column representation.
Table 5
Generalizable decision matrix for DMP tool selection.
| DIMENSION | KEY QUESTION | SCORING GUIDE (0–3) |
|---|---|---|
| Funder Templates | How well does the tool support compliance with funders? | 0: No templates. 1: Few, outdated templates. 2: Good library, some customization needed. 3: Extensive, up-to-date, and easily customizable templates. |
| Usability & Support | How easy is the tool to use for non-technical researchers? | 0: No documentation, unintuitive UI. 1: Basic documentation. 2: Good documentation and intuitive UI. 3: Extensive tutorials, helpdesk, and highly intuitive UI. |
| Collaboration | How well does the tool support team-based DMP creation? | 0: No collaboration features. 1: Basic sharing (read-only). 2: Versioning and multi-user editing. 3: Real-time collaboration and granular permissions. |
| Customization | How easily can an institution adapt the tool? | 0: Not customizable. 1: Basic theming. 2: GUI-based template and guidance customization. 3: Deep extensibility via code and full template control. |
| API & Integration | How well does the tool connect with other systems? | 0: No API. 1: Limited, poorly documented API. 2: Well-documented REST API for data retrieval. 3: Comprehensive API for read/write and deep integration. |
| maDMP Support | How well does the tool support machine-actionable standards? | 0: No structured export. 1: Basic structured export (e.g., generic JSON). 2: Export partially compliant with RDA DCS. 3: Natively produces fully RDA DCS-compliant output. |
| TOOL | METADATA STANDARDS | REPOSITORY INTEGRATION | FUNDER TEMPLATES | COLLABORATION | TRAINING & SUPPORT | RISK ASSESSMENT | CUSTOMIZATION | API/MADMP | SECURITY | SCALABILITY | TOTAL SCORE |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Argos | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 10 |
| BioDMP | 1 | 0 | 1 | 0 | 0 | 1 | 0 | 0 | 1 | 0 | 4 |
| Data Stewardship Wizard | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 10 |
| DataWiz | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 10 |
| DMP Assistant | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 10 |
| DMP Opidor | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 10 |
| DMPonline | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 10 |
| DMPTool | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 8 |
| Dmptuuli | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 8 |
| DMPTY | 1 | 0 | 1 | 1 | 0 | 1 | 1 | 0 | 1 | 0 | 6 |
| Easy DMP | 0 | 0 | 1 | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 6 |
| FioDMP | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 8 |
| GAMS DMP | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 1 | 1 | 3 |
| LabArchives (DMP module) | 0 | 0 | 1 | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 6 |
| NSD DMP | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 9 |
| OpenDMP | 1 | 1 | 0 | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 8 |
| Pagoda | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 9 |
| PARTHENOS DMP | 1 | 0 | 1 | 1 | 0 | 1 | 1 | 0 | 1 | 0 | 6 |
| PGD-BR | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 8 |