Table 1
Assignment of the DKRZ data dissemination system to the domains as described by Treloar & Harboe-Ree (2008).
| Domain | Phase | DKRZ system |
|---|---|---|
| Research preparation phase | Concept generation | data management (DM) planning tool RDMO |
| Private Research | production/processing | DKRZ storage on hard disc and tape HPSS4 |
| Shared Research | project collaboration intended use | ESGF, globally distributed project repository |
| Public | long-term archiving impact re-use | Long-term Archive |
Figure 1
Characteristics of data and metadata Quality Assurance Maturity Levels. QMM levels corresponding a) to different steps of the data production workflow and b) to the five data production phases with their standardisation characteristics and increasing degrees of formalisation.
Table 2
Shows a comparison of SMM and QMM.
| SMM | QMM |
|---|---|
| Software Readiness | Omitted: the data object is considered as persistent. Software development would lead to new data objects except software documentation. That is part of the metadata provenance. |
| Metadata | Criterion: Completeness Aspect: Existence of Metadata |
| User Documentation | Criterion: Completeness Aspect: Existence of Metadata |
| Uncertainty Characterisation | Criterion: Accuracy |
| Public Access/Feedback/Update | Criterion: Accessibility/Criterion: Completeness Aspect: Existence of Metadata level 5: data provenance chain exists including internal and external objects e.g. software, articles, method and workflow description/Criterion: Consistency Aspect: Versioning and Controlled Vocabularies (CVs) |
| Usage | Omitted: we use the ISO19157 explanation of data usability. It depends on the ‘particular application’. From this point of view, an evaluation of usage is not possible. |
Figure 2
OAIS Reference Model Information Packages on different Phases of the QMM process, showing the submission (SIP), archival (AIP), and dissemination information packages (DIP).
Figure 3
DKRZ Long Term Archive – example of minimum metadata (PDI, following the OAIS reference model).
Table 3
Overview of the QMM quality criteria and sub-criteria (aspects).
| Criterion | Aspect |
|---|---|
| Consistency | Data Organisation and Data Object |
| Versioning and Controlled Vocabularies (CVs) | |
| Data-Metadata Consistency | |
| Completeness | Existence of Metadata |
| Existence of Data | |
| Accessibility | Metadata Access by Identifier |
| Data Access by Identifier | |
| Accuracy | Plausibility |
| Statistical Anomalies |
Table 4
QMM criterion consistency.
| Level 1 | Level 2 | Level 3 | Level 4 R1.2 | Level 5 | |
|---|---|---|---|---|---|
| Aspect: Data Organisation and Data Object | |||||
| conceptual development | data organisation is structured/conform to | ||||
| internal rules informal documented | project specification | well-defined rule e.g. discipline-specific standards and long-term archive requirements (OAIS Package Info -binds) | interdisciplinary standards | ||
| data objects (OAIS) are | |||||
| SIPs consistent to internal rules | SIPs correspond to project requirements | I1, I2 AIPs conform to well-defined rules e.g. discipline-specific standards and long-term archive requirements | AIPs conform to interdisciplinary standards up-to-date and consistent to external scientific objects if feasible | ||
| DIPs are fully machine-readable with references to sources | |||||
| I1 DIPs datasets are self-describing | |||||
| data formats – Content Data Object (OAIS) | |||||
| correspond to project requirements | I1 conform to well-defined rules e.g. discipline-specific standards and long-term archive requirements | conform to interdisciplinary standards | |||
| data sizes are consistent | |||||
| file extensions are consistent | |||||
| Aspect: Versioning and Controlled Vocabularies (CVs) | |||||
| conceptual development | versioning follows/is | ||||
| internal rules informal documented | systematic corresponds to project requirements | systematic collection including documentation of enhancement conform to well-defined rules old versions stored if feasible | |||
| In case new versions are published: documentation is consistent to previous versions | |||||
| data labelled with CVs conform to | |||||
| informal CVs if feasible | formal project defined CVs if feasible | I1, I2 discipline-specific standards | interdisciplinary standards | ||
| Aspect: Data-Metadata Consistency | |||||
| not evaluated | OAIS metadata components are consistent | ||||
| PDI components: Provenance- unsystematically documented: Reference- creators | PDI components: Provenance – basically documented: Reference –creators contact Descriptive Information -naming conventions for discovery – find and search | Complete PDI * Provenance Context Reference – cross Fixity Access Rights and Representation Information Descriptive Information Package Info | |||
| *maintenance and storage policy are not affected, since they belong to the repository certification. | I3 external metadata and data are consistent | ||||
Table 5
QMM criterion completeness.
| Level 1 | Level 2 | Level 3 | Level 4 R1.2 | Level 5 |
|---|---|---|---|---|
| Aspect: Existence of Data (Completeness and Persistence) | ||||
| not evaluated | data is in production and may be deleted or overwritten | datasets exist, not complete and may be deleted but not overwritten unless explicitly specified | data entities (conform to discipline-specific standards) are complete dynamic datasets – data stream are not affected number of datasets (aggregation) is consistent data are persistent, as long as expiration date requires | data entities (conform to interdisciplinary standards) are complete dynamic datasets – data stream are not affected number of datasets (aggregation) is consistent data are persistent, as long as expiration date requires |
| Aspect: Existence of Metadata | ||||
| not evaluated | OAIS metadata components exist | |||
| PDI components: Provenance- unsystematically documented Reference- creators | PDI components: Provenance – basically documented: Reference –creators contact Descriptive Information: naming conventions for discovery – find and search | F2, R1 Complete PDI * R1.2 Provenance Context Reference Fixity Access Rights and Representation Information R1.1 Descriptive Information F4 Package Info | ||
| metadata is conform to interdisciplinary standards data provenance chain exists including internal and external objects e.g. software, articles, method and workflow description | ||||
| *maintenance and storage policy are not affected, since they belong to the repository certification. | ||||
Table 6
QMM criterion accessibility.
| Level 1 | Level 2 | Level 3 | Level 4 R1.2 | Level 5 |
|---|---|---|---|---|
| Aspect: Data Access by Identifier | ||||
| not evaluated | data is accessible by | |||
| file names | internal unique identifier correspond to project requirements | permanent identifier (expiration is documented) (OAIS Package Info – identifies) datasets have an expiration date and are accessible for at least 10 years (conform to rules of good scientific practice) | F1, A1 global resolvable identifier (PID-persistent identifier) registered with resolving to data access including backup where it is commonly accepted that the identifier is persistently resolvable at least to information about fate of the object data is accessible within other data infrastructures including cross references | |
| checksums are correct | ||||
| checksums are accessible | ||||
| a bijective mapping between identifier and datasets is documented e.g. in data header (OAIS Package Info – binds, identifies) | ||||
| Aspect: Metadata Access by Identifier | ||||
| not evaluated | metadata is accessible by | |||
| not specified | internal unique identifier correspond to project requirements | by permanent identifier expiration is documented (F4 OAIS Package Info – identifies) complete data citation is persistent | F1, A1 global resolvable identifier including backup complete data citation is persistent | |
| I3 external PID references are supported | ||||
| a mapping between data access identifier and metadata access identifier is implemented (OAIS Package Info relates Content Info and PDI) | ||||
Table 7
QMM criterion accuracy.
| Level 1 | Level 2 | Level 3 | Level 4 R1.2 | Level 5 |
|---|---|---|---|---|
| Aspect: Plausibility | ||||
| not evaluated | R1 documented procedure about technical sources of errors and deviation/inaccuracy exists (data header and content is consistent) | |||
| R1 documented procedure about methodological sources of errors and deviation/inaccuracy documented procedure with validation against independent data R1 references to evaluation results (data) and methods exist | ||||
| Aspect: Statistical Anomalies | ||||
| not evaluated | R1 missing values are indicated e.g. with fill values | |||
| R1 documented procedure of statistical quality control is available | ||||
| scientific consistency among multiple data sets and their relationships is documented if feasible | ||||