A Semantic Web Model for Standardizing FAIR Metrics and Assessment Results

Introduction

Ensuring and evaluating the quality of research data is of central importance in today’s data-driven research landscape. Data quality has long been understood not only in terms of formal correctness or content accuracy, but also as ‘fitness for use’, encompassing relevance, clarity, accessibility, and interpretability (Strong et al., 2002). More recently, additional criteria such as trustworthiness, reliability, and transparency of data providers have gained prominence. In this context, the FAIR principles (Findable, Accessible, Interoperable, Reusable) were introduced as a framework for improving the management and reuse of digital research objects (Wilkinson et al., 2016).

Since their publication, the FAIR principles have been widely adopted in policies and guidelines issued by publishers, funding agencies, and research organizations (e.g., European Commission, 2018; International Science Council, 2021; Taylor & Francis, 2025). To operationalize these principles, several initiatives have proposed more fine-grained FAIR metrics and maturity models that make FAIRness measurable (e.g., Devaraju et al., 2020; RDA FAIR Data Maturity Model Working Group, 2020; European Commission, 2021).

On the basis of these metrics, a growing number of tools have been developed to assess the FAIRness of research data. While some approaches rely on self-assessment and expert judgment (e.g., ARDC FAIR Data Self-Assessment Tool or FAIRAware), others aim at fully automated evaluations, such as the FAIR Evaluator (Wilkinson et al., 2019), F-UJI (Devaraju and Huber, 2021), FAIR-Checker (Gaignard et al., 2023), and FAIRshake (Clarke et al., 2019). Although these tools share a broad, domain-agnostic interpretation of FAIR, disciplinary communities increasingly require more specialized tests that reflect community-specific practices and standards (see, e.g., European Commission, 2020; European Commission, 2021).

At the same time, the scope of FAIR has expanded beyond datasets to other types of digital objects, including semantic artifacts (Poveda-Villalón et al., 2020) and software (Barker et al., 2022). This diversification has led to additional FAIR metrics and specialized tools such as FOOPS and O’FAIRe (Garijo et al., 2021; Amdouni et al., 2022), which target ontologies and software artefacts, respectively. While this plurality enables the assessment of FAIR practices across a wide range of resources, it also introduces substantial heterogeneity in how metrics are defined, implemented, and reported.

Different tools often rely on distinct interpretations of FAIR principles and implement varying metric tests to capture specific aspects of FAIRness. For example, a metric addressing the FAIR principle F1 may include separate tests for identifier syntax and for registration with a persistent identifier authority, yet the concrete syntax patterns applied can differ across implementations. Such variation can lead to divergent assessment outcomes for the same dataset. Indeed, Wilkinson et al. (2023) have shown that even differences in metadata harvesting strategies can significantly affect FAIR evaluation results. As a consequence, comparing FAIR scores across tools—or assessing the quality and transparency of the tools themselves—becomes difficult.

A further challenge lies in the lack of standardized output formats. Although most automated FAIR assessment tools expose their results via REST APIs, their JSON responses follow different schemas and conventions. As a result, machine-actionable aggregation and comparison of FAIR assessments from multiple tools is currently not feasible. Several studies have therefore called for greater transparency and standardization of FAIR metrics and result representations (e.g., Sun, Emonet and Dumontier, 2022; Lamarre et al., 2024). At the same time, initiatives such as the EOSC FAIR Task Force have emphasized the need for governance and coordination of FAIR metrics and tools, while also recognizing that diversity in metrics offers an opportunity for more holistic FAIR assessments if appropriate technical standards are adopted (Verburg et al., 2023).

Against this background, this paper proposes a web standards-based semantic model for the representation of FAIR metrics and FAIR assessment results. Building on existing recommendations for the publication of data quality information on the Web (Lóscio, Burle and Calegari, 2017), we examine which requirements FAIR assessments impose on such models. We then propose a semantic framework that integrates established vocabularies to describe FAIR metrics, metric tests, and assessment outcomes in a harmonized, machine-actionable way. Finally, we demonstrate the practical applicability of the model through an implementation in the F-UJI assessment tool and outline how web standards can support holistic analyses of FAIR assessments across different tools and domains.

FAIR-Specific Requirements for Data Quality Vocabularies

To identify common requirements for a harmonized representation of FAIR assessment results, we analyzed the response formats of several automated FAIR assessment tools. The analysis focused on the presence and structure of properties used to express FAIRness levels, such as numerical scores, percentage values, and ordinal test outcomes (e.g., pass/fail, see also Wilkinson et al., 2023). We also examined whether tools provide information about the scale on which a score is measured and whether they include verbose output such as log or debug messages.

Further aspects of the analysis included the use of unique identifiers for FAIR metrics and their relation to the corresponding FAIR principles. Although no single property (except the indication of a metric ID) is shared by all tools, each identified feature is supported by at least three of them. This indicates that these elements should be considered essential for any harmonized output model.

Overall, the analysis shows that although the investigated FAIR tools share some commonalities, their output formats differ substantially. No single output property is unique to one tool; each is used by at least three, indicating that these elements should be considered in any harmonized model. In particular, tools commonly report multiple FAIRness indicators in parallel, such as pass/fail statuses alongside numerical or percentage scores (Table 1).

Based on this analysis, several core requirements for a harmonized FAIR assessment output format can be identified. Such a format should include unique, actionable identifiers for metrics, together with titles and descriptions. It should support numerical and percentage scores with explicit units and allow results to be expressed on different scales, including ordinal levels and test status information. Multi-level maturity scales, such as those proposed by the FAIR Data Maturity Model Working Group (2020), illustrate this need. An appropriate FAIR assessment semantic model is therefore required, which can describe FAIR metrics as well as the results of individual measurements against these metrics as part of an overall FAIR assessment result.

The semantic model was developed based on the identified requirements through an iterative process. Following the literature analysis, an initial conceptual model was designed using draw.io and subsequently translated into a Resource Description Framework (RDF). The resulting RDF representations of FAIR assessment evaluations were iteratively refined and validated using the W3C RDF Validator.

Core Model for Standardized FAIR Assessment Results

As a foundational framework for modeling FAIR metrics and assessment results, the W3C Data Quality Vocabulary (DQV; Lóscio, Burle and Calegari, 2017) provides a well-established and extensible basis. DQV allows a quality evaluation against a defined metric to be expressed as an instance of dqv:QualityMeasurement, with the metric itself represented as an instance of dqv:Metric.

To link FAIR assessment results to the assessed resource, DQV’s close integration with the Data Catalog Vocabulary (DCAT; Albertoni et al., 2024) is particularly advantageous. While individual quality measurements can be attached directly to a dcat:Dataset (Figure 1) using dqv:hasQualityMeasurement, FAIR assessments typically consist of multiple metric results. It is therefore more convenient to use dqv:hasQualityMetadata to associate the dataset with an instance of dqv:QualityMetadata, which serves as a container for a collection of related dqv:QualityMeasurement instances.

Figure 1

To characterize the type of quality being measured—here, the degree of FAIRness—an instance of dqv:Dimension can be defined. For each measurement, an appropriate unit (e.g., percentage) should be specified using the Ontology of Units of Measure (OM; Rijgersberg, van Assem and Top, 2013), as recommended by the DQV specification (e.g., om:Percentage_unit).

The assessment process itself can be described as a provenance (PROV; Moreau, Missier and W3C Provenance Working Group, 2013) activity using prov:Activity, generating the FAIR assessment result and attributed to a specific assessment service or tool represented as prov:SoftwareAgent (Figure 1). This core model thus provides a standards-compliant structure to describe what is being measured, how it is measured, and by which agent.

The Extended Model

While DQV provides a solid foundation, it does not cover all requirements identified for FAIR assessment outputs and metrics. In particular, it lacks explicit support for defining measurement scales, representing ordinal maturity levels, capturing verbose test information, and expressing constraints associated with individual metric tests.

To address these gaps, we extend the core model using additional, well-established ontologies following the proposal of Radulovic et al. (2017). The Quality Model Ontology (QMO; Radulovic and García Castro, 2015) is used to relate metrics to supported measurement scales. Scales themselves are defined as instances or subclasses of om:Scale, such as om:OrdinalScale or om:CardinalScale. The Evaluation Result Ontology (EVAL; Radulovic and García Castro, 2017) provides eval:isMeasuredOnScale, which allows a specific measurement result to reference the scale on which it is expressed (Figure 2).

Figure 2

To describe test outcomes and execution details, we incorporate the Evaluation and Report Language (EARL; Abou-Zahra, 2017). EARL provides a controlled vocabulary for test results, e.g., earl:passed, earl:failed or less binary stati, such as earl:cantTell as required by the EOSC FAIR Task Force (see Wilkinson et al., 2024). It further supports the inclusion of additional information such as logger messages via earl:info.

Within this extended model, FAIR metrics and their associated metric tests are represented as instances of dqv:Metric, qmo:QualityMeasure, and earl:TestCase. Consequently, FAIR assessment results are defined as instances of dqv:QualityMeasurement, eval:QualityValue and earl:Assertion (Figure 2). To specify test inputs, conditions, and expected results, the Test Metadata Vocabulary (TEST; Curran and Duborst, 2005) is a good choice. This would, for example, allow the specification of a test:expectedResult property thus to define benchmark conditions.

Dependencies between metrics and tests are expressed using prov:wasDerivedFrom, allowing higher-level metrics to be derived from more specific sub-metrics and enabling the definition of an aggregate FAIR metric that summarizes overall compliance.

Together, these vocabularies allow FAIR metrics, metric tests, and assessment results to be modeled in a coherent, standards-based, and machine-actionable way.

Implementation in F-UJI

The proposed model was used to define a standard-compliant FAIR assessment result output for the F-UJI FAIR assessment tools (Devaraju and Huber, 2021). F-UJI is designed as a Web API and programmed in Python, following the OpenAPI specification with Swagger. The source code is available as Open Source at Github (https://github.com/pangaea-data-publisher/fuji). A web-based user interface at https://www.f-uji.net serves as a reference implementation.

The F-UJI API offers two main methods: ‘evaluate’ for performing FAIR assessments and ‘metrics’ for displaying the metrics used. By default, F-UJI serializes FAIR metrics and assessment results as JSON documents. Internally, metric and metric-test definitions are specified in YAML files (Figure 3), which form the basis for generating machine-readable RDF representations according to the proposed model (Figure 4). The definition of the FAIR metrics used by F-UJI is explained in more detail in Devaraju et al. (2020) and Devaraju and Huber (2021). The YAML configuration also allows for community-specific extensions, refinements, or the removal of individual metrics.

Figure 3

Figure 4

To support semantic interoperability, dedicated RDF serialization routines were implemented for both metric representations and assessment results. Although OpenAPI supports multiple response media types and HTTP content negotiation, the construction and serialization of RDF graphs are handled explicitly at the application layer. RDF representations can be exposed by the API in all formats supported by the underlying RDF library (rdflib), with the concrete format selected via content negotiation.

On this basis, the following sections distinguish between (i) the semantic modeling of FAIR metrics and (ii) the representation and publication of assessment results.

FAIR metrics are identified using dereferenceable Uniform Resource Identifiers (URIs) following the pattern:

Assessment results are modeled as dqv:QualityMeasurement, eval:QualityValue, and earl:Assertion, with dqv:QualityAnnotation used to capture maturity levels. Numeric and percentage scores are recorded consistently, and EARL is used to express pass/fail outcomes and debug information. Complete FAIR assessments are published as dqv:QualityMetadata and dcat:Dataset which allows adding rich metadata to an assessment result. The F-UJI demo web client provides the option to save API assessment results (JSON) in a MySQL database, which are indexed sequentially (Figure 4). Each assessment result dataset is identified on the F-UJI website via the URI pattern https://www.f-uji.net/result/ [assessment_id], which provides both RDF and HTML representations through content negotiation. Furthermore, the RDF representation is also available via standard file extensions such as ‘.ttl’ and ‘.rdf’.

To avoid additional API calls, the web client includes a PHP-based RDF serialization component that operates on cached assessment results. For results returned directly by the API that have not yet been stored and therefore have no persistent ID, the F-UJI API assigns a UUID by default to uniquely identify them in RDF output, ensuring consistent referencing even for unsaved assessments.

A complete example of the RDF output is provided at https://f-uji.net/result/124.ttl, with an archived version available at https://web.archive.org/web/20260122094616/https://f-uji.net/result/124.ttl. The corresponding metric in RDF is provided at https://f-uji.net/metrics/0.8.ttl (archived at https://web.archive.org/web/20260122100505/https://f-uji.net/metrics/0.8.ttl).

Evaluation and Practical Applicability

The approach presented in this work was evaluated with respect to its applicability for representing FAIR assessment outputs and supporting interoperability between heterogeneous assessment services. Its practical relevance is supported by the increasing adoption of the Data Quality Vocabulary for representing and exchanging structured quality evaluation results across different domains.

A typical use case is the automatized assessment of successful implementations of Data Management Plans (DMP). For this purpose, Arnhold et al. (2025) proposed a model based on the DMP Data Vocabulary (DMPQV), an extension of DQV specifically designed to incorporate quality assessments expressed using DQV. Similarly, the aggregation of FAIR assessments from different sources based on the transformations of FAIR assessment results to DQV has been demonstrated to be useful in the agricultural, forestry, and food sector by Jung and Martini (2025). Furthermore, DQV has been adopted as a quality representation format for DCAT-AP datasets in European Open Data infrastructures where dedicated quality workflow frameworks integrate data quality and FAIR assessments (Wentzel et al., 2023).

These applications demonstrate that DQV-based evaluation outputs are already contributing to an emerging ecosystem of consumers beyond the original assessment systems.

Existing FAIR semantic models, such as FAIR-O (Tuozzo and Pellegrino, 2026) and the OSTRails FAIR Reference Model (Wilkinson and Garijo, 2026), also incorporate DQV concepts and introduce their own domain-specific concepts derived from DQV for representing FAIR-related quality information. However, these ontologies are still in a relatively early stage and have yet to be recognized by the community. Further, additional mappings may be required to align their FAIR-specific concepts with existing DQV-based data quality workflows and ensure interoperability.

The proposed model leverages the emerging DQV-based ecosystem by applying DQV to FAIR assessment results generated by FAIR assessment tools. The implementation in F-UJI demonstrates that FAIR metrics, individual tests, and complete assessment results can be represented using interoperable RDF structures facilitating their integration, comparison, and reuse by aggregators without requiring tool-specific transformations.

Conclusions and Perspectives

Exposing the capabilities and offerings of data repositories is crucial for enabling users to identify appropriate systems within a FAIR-by-design ecosystem. Data quality indicators, such as FAIR ratings of individual datasets, can be highly useful, but they must be clearly specified and made visible in metadata descriptions for both machines and humans.

The model proposed here may help harmonize the output formats of the currently numerous FAIR assessment tools. Since each tool evaluates different aspects of FAIR, aggregating the results from as many tools as possible is necessary for a holistic view. A harmonized output of data quality and FAIR assessments will thus enable the development of independent, user-friendly FAIR aggregation tools, such as the proposed ‘FAIR extension’—a browser extension (Hernandez Serrano and Emonet, 2022) designed to summarize various FAIR test results.

The choice of W3C standards, particularly the Data Quality Vocabulary (DQV), allows assessed datasets to directly reference their FAIR assessment results within their metadata. For example, repositories can use dqv:hasQualityMetadata as a property of dcat:Dataset when metadata is expressed as DCAT. Similarly, for metadata expressed as schema:Dataset appropriate rating-related properties can be used to reference the FAIR assessment. Ideally, these FAIR assessment datasets would also have persistent identifiers, a requirement for making them FAIR themselves.

The proposed model can be further extended. For instance, rather than relying solely on textual descriptions via the Test Metadata Vocabulary, future implementations could use the Shapes Constraint Language (SHACL; Knublauch and Kontokostas, 2020) to precisely define requirements and expected results, using shacl:NodeShape as an additional type for a FAIR metric or metric test.

As outlined in this paper, the proposed ontologies can model FAIR assessment results as instances of multiple classes. This approach aligns with the recommendations of the W3C ‘Data on the Web’ Working Group and supports corresponding user expectations. However, it also represents a limitation compared to a vocabulary or ontology that covers all necessary aspects comprehensively. Further limitations of the proposed model are as follows:

• Complexity due to the integration of multiple ontologies

• Redundant representation of concepts across different vocabularies

• Limited support for FAIR-specific concepts

• Limited formalization of metric constraints

One possible solution would be to refine the DQV model in a future version—for example, to allow explicit indication of scales used in dqv:Metric and dqv:QualityMeasurement. This change alone would make it unnecessary to instantiate metrics and measurements as additional classes (eval:QualityValue and qmo:QualityMeasure), thereby avoiding the need for two separate ontologies to express the same information.

Alternatively, an extension of the DQV ontology could be derived from the proposed model to produce a simplified output format fully compatible with DQV. As this is beyond the scope of this paper, we refer here to promising developments within the OsTrails project (https://ostrails.eu/). In addition, the newly established FAIR Metrics and Digital Objects Task Force within the European Open Science Cloud (EOSC) aims to standardize FAIR tests and has brought together many of the leading developers and stakeholders. The work and model proposed here should therefore be seen as a basis for discussion in this and other initiatives focused on improving data quality and FAIRness.

Data Accessibility Statement

An example RDF output complying to the model are available at https://f-uji.net/result/124.ttl, an archived version can be downloaded at https://web.archive.org/web/20260122094616/https://f-uji.net/result/124.ttl.

The corresponding metric in RDF is provided at https://f-uji.net/metrics/0.8.ttl which is archived at https://web.archive.org/web/20260122100505/https://f-uji.net/metrics/0.8.ttl.