(1) Overview
Introduction
Internet-based experimentation has become a prominent method in behavioral and cognitive research, allowing researchers to collect data efficiently from large and geographically diverse participant samples. Advances in web technologies and experimental platforms now support complex behavioral tasks outside the laboratory environment. Recent studies have demonstrated that a wide range of cognitive and perceptual measurements can be reliably obtained online, including visual function assessments [9], and psychophysical measurements of body image perception [5]. These findings complement earlier work showing that online data can closely match laboratory results [3]. Despite these advances, implementing complex and flexible experimental designs, particularly those requiring online computational modeling or custom trial structures, remains challenging.
Moving from lab-based to online research is not always straightforward [4]. Although tools exist to support experiment implementation, deployment, and recruitment [15]. Most online frameworks rely on JavaScript (JS), whereas laboratory research is often conducted in Python, MATLAB, or C++. This mismatch complicates the transition of existing experiments, particularly when they depend on Python-based code or computational models.
Python is widely used in scientific and data-driven research and benefits from a wide range of libraries for computation, data handling, and modeling. In behavioral experiments, open-source tools such as OpenSesame [10], Experyment [7] and PsychoPy [14, 1] support laboratory experiments, with PsychoPy also enabling online deployment via the Pavlovia platform [13]. However, automatic translation from Python to JavaScript remains limited for computationally intensive or model-based paradigms, making the implementation of advanced online experiments challenging.
To overcome these limitations, we developed PyBOLE, a Python-based framework built on the Django web framework [16], which handles server-side logic including participant sessions, HTTP requests, and database interactions, as well as security through its built-in mechanisms for session management, access control, and data protection. PyBOLE allows the complete experimental workflow to be implemented in Python, with JavaScript restricted to client-side stimulus presentation and response collection, with all experimental logic and data processing handled server-side. This architecture eliminates the need to host experiments on third-party platforms or pay usage fees, while enabling integration of Python libraries, real-time updates of computational models during data collection, and automated logging of both behavioral and model data. PyBOLE outputs only raw experimental data, which should be processed, analyzed, and visualized using external tools.
PyBOLE supports both standard and complex experimental paradigms, including adaptive methods, providing full control over trial structure and computational workflows. Researchers do need some Python and basic web familiarity to implement experiments, but in return PyBOLE allows highly customized or computationally intensive studies that would be difficult to execute in standard browser-based frameworks or on hosted platforms with limited flexibility. This architecture relies on a clear separation between client-side interaction and server-side computation, requiring coordination between JavaScript-based interfaces and Python-based backend processes. It therefore entails a higher level of technical involvement than approaches based solely on client-side or fully integrated frameworks.
PyBOLE is designed as a complementary tool to existing general-purpose online experimentation platforms. Rather than replacing established solutions, it targets experiments that rely on server-side computation and tight integration with Python-based models. This positioning makes it particularly suitable for computationally demanding paradigms such as those requiring real-time processing or model updates during data collection, which are difficult to implement within existing online experimental frameworks.
Implementation and architecture
The architecture of the experiment framework relies on the python web application framework Django. Conceptually, PyBOLE is divided into entities (see Figure 1) that structure the progress of an experiment:
Experiment represents the structure of the whole experiment with the associated pages (more details below). Experimental parameters are specified using a text-based file in JavaScript Object Notation (JSON), a standard structured data format, including elements such as trial sequences and paths to image databases.
Session. An experiment can be attached to several sessions. Indeed, it can be useful to have an experiment available for different populations or to use different stopping criteria (time of the experiment, number of trials, etc.). To this end, a JSON configuration is associated with a session;
SessionProgress is the core entity of the application’s operation. SessionProgress links a session to a participant. When a participant starts an experiment, a new SessionProgress instance is created. This instance manages the exchanges with the user, sends the next stimulus to the local computer and determines when to end the experiment according to stop criteria specified in the session. A SessionProgress, includes SessionStep instances storing the participant’s answer for each presented stimulus.
Figure 1
Representation of the main entities and their relationships of the PyBOLE framework.
In practice an experiment is composed of four pages: an information page (collecting participant information), an example page (instructions and training trials), a main page (stimulus presentation), and an end page.
The SessionProgress entity manages the transition between these pages. It requires the implementation of four methods: start, next, progress and end. Figure 2 illustrates the interaction between the SessionProgress and the experiment pages.
Figure 2
Interaction of the SessionProgress entity methods for navigation within PyBOLE experiment.
The timeline of an actual experiment would be as follows:
The specific instance of SessionProgress is created when starting the session;
Once the information page is validated by the participant, the start method is called to retrieve the entered information.
The example page is then displayed. Once the instructions have been completed, the first trial starts and the stimulus is proposed to the participant (first call of the method next);
Then, after each response and as long as the end method does not meet a stopping criterion, a new stimulus (from the next method) is displayed to the participant on the main page. The progress indicator of the whole experiment (progress method) is sent to the main page;
If the end method indixates completion, the end-experiment page is displayed.
The PyBOLE framework is freely available on Github: PyBOLE GitHub. The repository includes detailed documentation PyBOLE covering framework usage, data exchanges between JavaScript and Django, and guidance for securely setting up experiments. To better understand how the proposed application interface works, it is recommended to use the example database provided in the documentation.
PyBOLE supports a broad range of psychophysics paradigms. Conventional methods, such as the method of constant stimuli, can be implemented in Python using the framework. Adaptive methods, including QUEST+ [17, 18] and other Python-based implementations (e.g. [19]) can also be incorporated to efficiently estimate psychometric function parameters. While many adaptive methods are readily available in Python, implementations for online experiments in JavaScript are still relatively limited. jsQuestPlus, a JavaScript implementation of the QUEST+ method for estimating psychometric function parameters in online studies, has been proposed [8], but many other advanced methods remain unavailable in JavaScript.
QUEST+ is a Bayesian adaptive procedure designed to efficiently estimate multiple parameters of a psychometric function. On each trial, it chooses the next stimulus from a predefined set by maximizing the expected information gain, based on the observer’s previous responses. This targeted stimulus selection allows QUEST+ to converge on accurate estimates with fewer trials than many traditional methods, making it particularly suited for experiments where trial efficiency is critical, such as online studies.
While PyBOLE offers flexible support for both conventional and adaptive psychophysics paradigms, it has some limitations. The software requires hosting a server, which stores experimental data in a database. Additionally, setting up and customizing experiments requires some programming proficiency. While this may seem challenging for users without experience, creating a functional experiment is achievable with basic guidance.
The example application illustrating the use of PyBOLE is based on the QUEST+ method to adjust the stimuli parameters throughout the experiment. Here we used QUEST+ to probe PyBOLE, to better describe its functions and to allow other users to modify the relevant parts to implement a Python based experimental procedures according to their own needs. This study was tested across several browsers (Google Chrome, Firefox, Safari) and operating systems (Microsoft Windows, Apple macOS, Linux).
Quality control
To assess the functionality and robustness of PyBOLE, we conducted a series of quality control analyses in the context of a fully online adaptive psychophysical experiment. The purpose was to evaluate the framework across the complete experimental pipeline, including stimulus presentation, behavioral data collection, real-time model updating, stopping control, and structured data recording, while ensuring consistency with established laboratory implementations.
As a case study, PyBOLE integrated a Python implementation of QUEST+ adaptive procedure. At initialization, the framework defined the parameter grid of the psychometric model, established prior distributions, and set entropy-based stopping criteria. During execution, PyBOLE controlled stimulus presentation, collected participant responses and reaction times, updated model parameters in real time, computed entropy, and evaluated stopping conditions while enforcing minimum and maximum trial constraints. All behavioral and model data were automatically recorded at each trial.
We implemented a simple two-alternative forced-choice (2AFC) discrimination task in which participants compared images generated with varying levels of sampling quality. The study received ethical approval (University of Lille, Agreement n°2019-392-S78), and participants provided informed consent online prior to participation. The stimulus set consisted of 400 images with systematically varied noise levels.
Participants began with written instructions, followed by six training trials with feedback to signal the correct answer. No feedback was provided during the main session. Stimuli remained visible until response. To ensure data quality in the online setting, attention checks and an instructional manipulation check (IMC) were included [15, 12, 2, 6]. A stopping rule based on a minimum entropy threshold was applied, with a minimum of 200 and a maximum of 250 trials.
Figure 3 illustrates representative outputs derived from PyBOLE data, with visualizations created using external plotting libraries. After the session, the trial-by-trial data and QUEST+ parameter estimates were extracted and used to generate the plots. The evolution of the perceptual threshold estimate across trials shows progressive stabilization as the adaptive procedure converges (Figure 3a). Correspondingly, Shannon entropy decreases over trials (Figure 3b), reflecting the reduction of uncertainty in parameter estimates. Convergence was further assessed by examining the stability of parameter estimates over the final trials, confirming that the adaptive procedure reached a consistent solution.
Figure 3
Example analysis for a representative participant based on PyBOLE data. (a) Evolution of the threshold estimate during the adaptive procedure. (b) Shannon entropy across trials, reflecting decreasing parameter uncertainty.
Although the entropy threshold was reached at trial 55, the experiment continued to satisfy the predefined minimum number of trials (200), ensuring sufficient data collection for stable estimation. These results demonstrate that PyBOLE correctly implements adaptive stopping rules while maintaining controlled experimental constraints.
PyBOLE records behavioral data continuously during the interaction between the participant’s browser and the server, ensuring no loss of trial information. Trial-level logs include responses, reaction times, stimulus parameters, entropy values, and model estimates, enabling detailed post hoc analyses.
Importantly, the parameter estimates and convergence patterns were comparable to laboratory implementations of QUEST+ using a Matlab version, confirming that PyBOLE reliably reproduces adaptive psychometric procedures in an online context [11].
Overall, these quality control analyses demonstrate that PyBOLE successfully integrates an adaptive algorithm within a fully automated experimental pipeline, while providing flexible, structured, and reproducible data handling suitable for diverse online research applications. The framework is fully independent, allowing users to embed any computational model of their choice locally, and is freely available, giving researchers full control over the experimental design and analysis.
(2) Availability
Operating system
GNU/Linux, Mac OS X, Windows and on any platform supported by Docker, such as Azure and AWS.
Programming language
It is principally implemented in Python and secondly in JavaScript and HTML.
Dependencies
E.g. libraries, frameworks, incl. minimum version compatibility.
Python version 3.7.1 minimum, Django ≥ 3.2.12, django-admin-interface ≥ 0.18.5, django-multiselectfield ≥ 0.1.12, django-json-widget ≥ 1.1.1, django-polymorphic ≥ 3.1.0, numpy ≥ 1.21.5, Pillow ≥ 9.0.1.
List of contributors
No further scientists/researchers contributed to the development of PyBOLE.
Software location
Archive
Name: PyBOLE
Persistent identifier: https://doi.org/10.5281/zenodo.18908779
Licence: The software is released under the MIT License, allowing reuse, modification, and redistribution with appropriate attribution.
Publisher: Jérôme Buisine
Version published: v1.0.1
Date published: 08/03/26
Code repository GitHub
Persistent identifier: https://github.com/prise-3d/PyBOLE
Date published: 16/03/22
(3) Reuse potential
PyBOLE is an open-source Python framework designed for online psychophysics and perceptual experiments. It enables the implementation of a wide range of experimental paradigms and is applicable to fields such as cognitive neuroscience, psychology, vision science, and human-computer interaction. It is particularly well suited for computationally demanding experiments, such as adaptive paradigms that require online data processing during data collection, which can be difficult to implement in JavaScript-based environments. Its modular design allows researchers to create, customize, and share reproducible experiments, as well as extend the framework for specialized paradigms. More generally, the use of Python enables integration of a wide range of scientific libraries for data processing, modeling, and analysis, allowing the implementation of complex and customized experimental workflows. The repository provides example experiments, including a Ponzo illusion protocol defined via a JSON configuration file, along with sample session data to facilitate reuse.
Author Contributions
Vasiliki Myrodia: Conceptualization, Methodology, Investigation, Writing – Original Draft.
Jérôme Buisine: Software, Investigation, Visualization, Writing – Original Draft.
Samuel Delepoulle: Funding acquisition, Validation, Writing – Review & Editing.
Laurent Madelain: Supervision, Validation, Writing – Review & Editing.