A Structured Examination of Reproducibility: A Case Study for HTS Using the PRIMAD Model and BioCompute Object

Meznah Aloqalaa; Stian Soiland-Reyes; Carole Goble

doi:10.5334/dsj-2026-024

A Structured Examination of Reproducibility: A Case Study for HTS Using the PRIMAD Model and BioCompute Object

Data Science Journal

Volume 25 (2026): Issue 1

By: Meznah Aloqalaa , Stian Soiland-Reyes and Carole Goble

Open Access

|Jul 2026

Figures & Tables

Table 1

The PRIMAD model for reproducibility in a computational experiment; each reproducibility label is associated with its gain and the way of achieving it. What dimensions will change are represented by the dark-coloured cells, what could possibly change are represented by the light-coloured ones, and what will not change are the non-coloured cells (adapted and extended from Freire, Fuhr and Rauber (2016)). The original table has been updated with new gains and labels, which are highlighted in underlined text to clearly distinguish the additions.

Label: Repeat
Platform	Research Objective	Implementation	Method	Actor	Data (Raw)	Data (Parameter)
Gain: Consistency across Trials, Determinism
Label: Port, Relocate
Platform	Research Objective	Implementation	Method	Actor	Data (Raw)	Data (Parameter)
Gain: Cross-Platform Compatibility (Portability), Minimal Dependency (Flexibility)
Label: Reuse/Repurpose
Platform	Research Objective	Implementation	Method	Actor	Data (Raw)	Data (Parameter)
Gain: Cross-Disciplinary Application (Apply Code in Different Settings), Resource Efficiency
Label: Re-code, Reinterpret
Platform	Research Objective	Implementation	Method	Actor	Data (Raw)	Data (Parameter)
Gain: Improved Code Quality, Correctness of Implementation, Expand Adoption, Enhanced Efficiency, Flexibility
Label: Ratify, Validate
Platform	Research Objective	Implementation	Method	Actor	Data (Raw)	Data (Parameter)
Gain: Hypothesis Correctness, Validation via a Different Approach, Findings Robustness
Label: Review, Independent Verify
Platform	Research Objective	Implementation	Method	Actor	Data (Raw)	Data (Parameter)
Gain: Enhanced Transparency (Sufficiency Information), Independent Verification
Label: Resample, Generalise
Platform	Research Objective	Implementation	Method	Actor	Data (Raw)	Data (Parameter)
Gain: Applicability across Different Settings
Label: Reparameterisation, Recalibration, Parameter Sweep,
Platform	Research Objective	Implementation	Method	Actor	Data (Raw)	Data (Parameter)
Gain: Robustness, Adaptation to Other Conditions (Sensitivity), Parameters Optimisation

Table 2

Overview of the BCO framework, highlighting the three top-level fields and its eight domains. Domains marked with an asterisk (*)—Error Domain, Parametric Domain, and Extension Domain—are optional and may be included as needed.

BCO TOP-LEVEL FIELDS
Unique Identifier, Schema Version’s Reference and Lightweight Cryptographic Hash Value
BCO DOMAINS
Usability Domain: An array of free text values that should align with the terminology used in the BCO name, keywords, and external references (xref) sections. It is intended to describe the computational workflow in the following sections: the need, method, results, and how the results can be used or interpreted for the scientific use case.	Provenance Domain: History, version, contributors and review status of the BCO.
	Input/output Domain: input and output files used by the computational workflow for execution.
Description Domain: Pipeline steps, input and output file relations, and external references for review.	Execution Domain: Execution of a BCO in terms of scripts and the information required to run a pipeline.
* Error Domain: Specifies the accepted error range in an empirical and algorithmic manner.	* Parametric Domain: A set of parameters used to customise the pipeline
* Extension Domain: The part of the workflow used to encompass additional structures outside the BCO schema.

Method steps for the systematic examination between the BCO and PRIMAD frameworks, including four steps (use case examination, concept-level/general mapping, field-level mapping, and quantitative scoring) applied to the BCO_022531/2.2 use case.

Table 3

A Mapping summary between the PRIMAD model dimensions and BCO domains and fields. For clarity, some subfields have been excluded. An asterisk (*) indicates optional domains and fields.

BIOCOMPUTE OBJECT			PRIMAD MODEL
DOMAIN	FIELDS	SUB FIELDS	DIMENSION
Usability	Free Text		Research Objective	Consistency	Transparency
Usability	Free Text		Method	Consistency
Provenance	Name
	Version
	Review*	status
		reviewer comment
		date
		reviewer
		reviewer details
	Derived From*
	Obsolescence Time*
	Embargo Period*
	Created Time
	Modification Time
	Contributors	Name	Actor
		Contribution
		other details
	Licence
Description	Keywords
	External References	Namespace, Name, IDs, Access Time	Platform
	Platform/Environment		Platform
	Pipeline Steps	Step Number	Implementation
		Name
		Description
		Version*
		Prerequisites*	Platform
		Input List	Data (Input Data)
		Output List		Consistency
	Script		Implementation
Execution	Script driver		Platform
	Algorithmic tools and Software Prerequisites	Name	Platform
		Version
		URI	Platform
		Access Time
		SHA1 Checksum
	External Data Endpoints	Name	Platform
	External Data Endpoints	URI
	Environment Variables
IO	Input Subdomain	Filename*	Data (Input Data)
		URI	Data (Input Data)
		Access Time*
		SHA1 Checksum*
	Output Subdomain	Filename*		Consistency
		Media Type
		URI
		Access Time*
		SHA1 Checksum*
Parametric*	Parameter		Data (Parameter)
	Value
	Step
Error*	Algorithmic Subdomain	Inclusion Rule		Consistency
	Algorithmic Subdomain	Reference
	Empirical Error	Inclusion Rule
		Reference
		Definitions
Extension*	Free Text

Table 4

Summary of mapped fields and quantitative scores for PRIMAD dimensions in the BCO use case (BCO_022531/2.2), where field scores (Si) were assigned as 0 = missing/empty, 1 = present but insufficiently detailed or inaccessible, and 2 = present and verifiable.

PRIMAD DIMENSION		NO. OF MAPPED FIELDS IN THE DIMENSION (n_d)	SUM OF SCORES OF THE MAPPED FIELDS $(\sum s_{j})$	$Dimension Score (d) = \frac{\sum s_{i}}{2 \times n_{d}} \times 100$
Platform		7	10	71
Research Objective		1	2	100
Implementation		5	9	90
Method		1	2	100
Actor		3	6	100
Data	Data Parameter	3	6	100
Data	Data Input	3	5	83
Consistency		10	17	85
Transparency		58	78	67

Table 5

Proposed aspects for the PRIMAD model and their documentation through the BCO domains’ fields.

TIME		ADDITIONAL RESOURCES
BCO Fields	Domains	BCO Fields	Domains
Obsolescence Time	Provenance	External References	Description
EMBARGO PERIOD		FAULT TOLERANCE
Created Time		BCO Fields	Domains
Modification Time		Empirical Error Values	Error
Review Date		Algorithm Error Values	Error
Input/output Subdomain Access Time	I/O	METADATA SCHEMAS
Input/output List Access Time	Description	BCO Fields	Domains
External Reference Access Time	Description	BCO Schema version	Top-Level Fields
DATA CATEGORISATIONS		Additional Schema	Extension
BCO Fields	Domains	VERSION
Input/output Subdomains	I/O	BCO Fields	Domains
Input/output Lists	Description	BCO Version	Provenance
Parameters	Parametric	Software Prerequisites Version	Execution
HUMAN ROLES		Pipeline Step Version	Description
BCO Fields	Domains	LICENCE
Contributor	Provenance	BCO fields	Domains
Reviewer	Provenance	BCO licence	Provenance
REVIEW
BCO Fields	Domains	BCO fields	Domains
Review Status	Provenance	Review Date	Provenance
Reviewer Comment	Provenance	Reviewer
FUNCTIONALITY
BCO Fields	Domains	BCO Fields	Provenance	Domains
Platform/Environment	Description	Modification Time	Provenance
Pipeline Step’s Prerequisite	Description	BCO Version
SHA Checksum	Execution	Obsolescence Time
Environment Variable		Embargo Period
External Data Endpoints		BCO licence
Software Prerequisites		SHA Checksum		I/O
		ETag		Top-Level Fields

Table 6

The proposed depth is in the form of aspects for each PRIMAD model’s dimension.

PRIMAD DIMENSIONS	RESEARCH OBJECTIVE	METHOD	PLATFORM	IMPLEMENTATION	ACTOR	DATA
PROPOSED ASPECTS	RESEARCH OBJECTIVE	METHOD	PLATFORM	IMPLEMENTATION	ACTOR	DATA
Data Categorisations						✓
Versions	✓		✓	✓		✓
Time			✓	✓		✓
Additional Resources			✓	✓		✓
Fault Tolerance			✓	✓		✓
Licence				✓		✓
Functionality			✓	✓		✓
Human Roles					✓
Review	✓
Metadata Schemas	✓		✓	✓	✓	✓

References

Authors

Metrics

Articles in this issue

DOI: https://doi.org/10.5334/dsj-2026-024 | Journal eISSN: 1683-1470

Journal RSS Feed

Language: English

Page range: 24 - 24

Submitted on: Nov 24, 2025

Accepted on: Jun 15, 2026

Published on: Jul 14, 2026

Published by: Ubiquity Press

In partnership with: Paradigm Publishing Services

Keywords:

computational reproducibility,

PRIMAD model,

BioCompute Object

© 2026 Meznah Aloqalaa, Stian Soiland-Reyes, Carole Goble, published by Ubiquity Press
This work is licensed under the Creative Commons Attribution 4.0 License.

Volume 25 (2026): Issue 1

A Structured Examination of Reproducibility: A Case Study for HTS Using the PRIMAD Model and BioCompute Object

Figures & Tables

Table 1

Table 2

Figure 1

Table 3

Table 4

Table 5

Table 6

Paradigm

My account