Table 1
An overview of online proctoring phase in relation to exam sessions.
| CONTEXT AND TIMING | ONLINE PROCTORING PHASES | TYPICALLY CARRIED OUT BY | |
|---|---|---|---|
| Before exam, to be used for multiple exams | Registration – Acquire identity documents or profiles to be used for authentication | Human or machine | |
| Related to specific exam | Pre-exam | Authentication – Verifying the identity of student against pre-recorded or live evidence System and room check – Verification/restriction of student’s computer system and physical environment to comply with exam conditions | Human or machine |
| During exam | Monitoring – Checking that student complies with exam rules and does not access/receive external help; capturing data and raising flags indicating potential rule violations for immediate or post-exam intervention Immediate intervention – Dialog with student to address potential rule violations; potentially termination of exam session | Human or machine or combination (e.g., machine generated flags, human intervention) | |
| Post-exam | Review – Review of data recorded during exam guided by flags raised; decisions on impact on exam outcomes | Human | |
| Longer term data collection, beyond specific exam | Profiling – Collection of data to build profile of student to be used in future exam proctoring | Machine | |
Table 2
Data used by advanced proctoring technologies compared to those traditionally used with assessment technologies.
| DATA TRADITIONALLY USED WITH LEARNING TECHNOLOGIES | DATA ADDED VIA THE USE OF ADVANCED PROCTORING TECHNOLOGIES |
|---|---|
| Personal data Name, DoB, id number, … Still images of faces, not necessarily in high resolution Data based on the assessment content Exam answers, plagiarism similarity indices | Physiological and behavioural biometrics High resolution images of faces and students’ physical environments Flags (indicating potential exam rule violations) supported by personal data (e.g., video recordings) instead of assessment data Profiles based on biometric data |
Table 3
Discussion of data privacy and security in proctoring system reviews.
| ARTICLE | FOCUS | DETAILS ON DATA HANDLING, PRIVACY AND SECURITY |
|---|---|---|
| Foster and Layman (2013) | Comparison of eight proctoring systems, looking at features built into the systems, technical requirements, and support for proctoring modes | Not addressed |
| Hussein et al. (2020) | Looked at eight systems, trialled one; costs and licencing are part of the review criteria; concerned with selecting an online proctoring system during the COVID-19 pandemic | Not addressed |
| Arnò et al. (2021) | Review of 29 proctoring systems looking at system functionality, open-source status and cost-free access; concerned with selecting an online proctoring system during the COVID-19 pandemic | Investigate compliance with data protection regulations (in particular GDPR), user-friendliness and need for client-side software installation yet provide little detail |
| Labayan et al. (2021) | Review of 16 proctoring systems, looking at service characteristics, technical features and hosting location | Address GDPR compliance, yet do not provided details |
| Nigam et al. (2021) | A literature-based review of 15 AI-based proctoring systems which provides limited system details | Not addressed No insights regarding threats of AI technologies despite specific reference to AI |
| Aurelia et al. (2023) | Provide a brief overview of features, advantages, and disadvantages of 13 proctoring systems; no details on assessment criteria are provided | Not addressed |
| Fidas et al. (2023) | Briefly introduce six proctoring systems | Not addressed |
Table 4
Systems included in and excluded from review.
| Systems included in review (from Capterra listing and other sources) |
| BRISO; CONSTRUCTOR PROCTOR; DIGIEXAM; DIGIPROCTOR; DUGGA; EVALART; EXAMITY; EXAMONLINE; HIREPRO PROCTORING; HONORLOCK; INSPERA PROCTORING; INTEGRITY ADVOCATE; IRIS INVIGILATION; METTLE; MOODLE PROCTORING AND PROCTORING PRO; PROCTOREDU; PROCTORIO; PROCTORSTONE; PROCTORTRACK; PROCTORU; PSI BRIDGE; ROSALYN; RPNOW; SMOWL; SUMADI; SURPASS; SYNAP; TALVIEW; TESTWE; UXPERTISE XP; WISE PROCTOR; WITWISER; YOUTESTME |
| Systems excluded from review (from Capterra listing) |
| CAMPUS 365 ERP PLATFORM (school management software); ELYSA GRADE (could not locate website); EVALART (test provision for candidate selection); EVERCERT (certification consulting); FORM PRESENTER (software to control time available for filling out forms); HIREPRO VIDEO INTERVIEWS (video interview software); PROCTORIZER (website in Spanish only); ULEARN (website not available) |
Table 5
Criteria used for analysis of proctoring features.
| CRITERION | EXPLANATION | |
|---|---|---|
| Timing of human proctor engagement | Human-live | Live observation by human proctor (optionally supported by machine generated flags) with the potential of live interaction with student, setting of flags for later analysis, or immediate decision making |
| Human-recorded | Post exam analysis by human proctor based on recordings (optionally supported by human set or machine generated flags) | |
| Level of machine-generated input in the proctoring process | Automated flags | Machine-generated flags to be used for immediate or post-exam attention |
| Automated alerts | Machine-generated alerts to students during exam (aimed at improving proctoring processes with minimal interruption of exam taking) | |
| Automated decisions | Machine-generated decision making that affects exam outcomes (by denying student access to the exam, by substantially interrupting the exam taking, or by ending the exam session prematurely) |
Table 6
Suggested proctoring approach based on automated monitoring and human decision-making.
| PRE-EXAM | DURING EXAM | POST-EXAM |
|---|---|---|
| Automated identity and environment (physical, student’s devices) checking based on student proctoring identity profile Automated testing of input devices (e.g., camera, microphone) for data collection | Student is automatically alerted to system issues (e.g., need to adjust lighting to improve image quality) Flags indicating potential exam rule violations are created based on automated real-time monitoring | Student and proctor review flags and associated data Communication between proctor and student to resolve issues Decision making by exam review board as required |