MIRROR: Local Evidence Bundles for Reproducible Research Software Review, with a JANUS Case Study

Many statements in a research article depend on code or data, such as ‘the script in the supplement reproduces Table 4’. Such claims are hard to check later, because files change after the text is written. MIRROR is an open-source Python tool and workflow that keeps each claim next to the files that support it. A bundle is a local folder with the work under review, copies of its supporting files, and a claim register that links each claim to its evidence and to the wording the user judges safe to publish. The bundle records a digest (checksum) of each of these files: a short code that changes whenever the file changes. An offline verifier program reports missing or changed files, format errors, and claims that use assurance or compliance phrases such as ‘guarantees’. The worked case is JANUS, a study of how sentences from articles change when machine-translated into other languages and back. The review recorded which files were prepared for release and which stayed private, and it also led to a data availability statement that names the study’s data and code supplement. A bundle that passes the verifier shows what was kept and claimed, not that a claim is true.