metaScreener: A Plugin-Based Desktop Application for Human-in-the-Loop Systematic Literature Screening

metaScreener is an open-source, cross-platform desktop application implemented in Python/Tkinter that delivers a sequential, multi-stage pipeline for systematic literature screening. The software integrates deterministic rule-based filters with large language model (LLM) inference to automate high-volume citation screening within structured evidence synthesis workflows. Its plugin architecture exposes seven interoperable modules covering reference marker extraction, bibliographic reference resolution, eligibility criteria structuring, heuristic-based filtering, and LLM-based full-record eligibility adjudication. Each screening decision is logged within a timestamped, SHA-256-verified bundle archive that satisfies the audit and reproducibility requirements expected in rigorous evidence synthesis methodology. In a demonstration use case comprising 776 candidate records, the pipeline reduced the corpus to 73 records requiring full human review—a 90.6% reduction—with deterministic pre-filtering accounting for 98.3% of exclusions and LLM-assisted stages providing fine-grained adjudication over the residual candidate set. A complementary human validation exercise covering 254 LLM-screening decisions across three LLM-adjudicated criteria reports observed human-versus-LLM agreement of 83.5% and 87.1% on the two exclusion criteria, with the inclusion criterion showing fair agreement (Cohen’s 𝜅 = 0.28) and a marked asymmetric pattern in which the LLM hedges toward an uncertain verdict on ambiguous abstracts. Complete decision traceability was preserved across all stages. metaScreener is released under the MIT licence and is publicly available at https://github.com/lars-ulaval/metaScreener.