A Three-Year Multi-Faculty Evaluation of 16 SI-PASS Schemes: Attendance Thresholds, Performance Gains, and Standardized Metrics

SI-PASS Schemes at ULiège

The University of Liège implements a regular SI-PASS model that follows as closely as possible the recommendations outlined by the European Centre for SI-PASS in the official Supervisor Training Manual (2022, pp. 11–12). These guidelines specify the structural and procedural elements required to ensure program integrity for all stakeholders. A detailed appraisal of ULiège’s compliance with these guidelines is provided in Appendix 1. Across the faculties of Applied Sciences, Bioengineering, Psychology, and Sciences, SI-PASS is delivered weekly during the semester, typically over 13 weeks. Leaders complete a six-hour mandatory initial training and are offered refresher sessions, consistent with European Centre recommendations.

Data Sources and Methods

The data was collected from all faculties who ran an SI-PASS program during the academic years 2019–2020, 2020–2021, and 2021–2022.

The first avenue of inquiry (AOI 1: Participation) was examined through attendance reports completed by SI-PASS leaders after each session. These reports recorded the session date and the identities (later pseudonymized) of all participants. Although leaders were instructed to submit a report for every meeting, occasional omissions occurred (these gaps are indicated in Table 1). Beyond functioning as a program quality indicator, participation data is a central analytic pillar of the study, given that SI-PASS effects are known to depend on exposure (Cheng & Walter, 2009). Without reliable participation metrics, performance outcomes (AOI 3) cannot be meaningfully interpreted.

To examine student perceptions (AOI 2), the study used an online exit survey administered between the final SI-PASS session and the exam in each targeted course. The questionnaire combined Likert-type items with two open-ended questions prompting students to articulate their motives for either participating or not participating in SI-PASS. Qualitative responses underwent thematic analysis following Braun and Clarke’s (2006) reflexive approach, supported by an iterative coding process. Survey non-response was addressed by reporting response rates and conducting analyses on available data. Given the descriptive purpose of this AOI, missingness was not considered. Links between perceptions and performance were explored through the item “SI-PASS will significantly contribute to my success at the exam of the target course,” which allowed integration of AOI 2 with AOI 3.

Performance data (AOI 3) were obtained from RADIUS (ULiège’s unit in charge of administrative data management), which provided the official exam marks of all students enrolled in the targeted high-risk courses. Analyses were conducted using independent-samples t-tests to compare participants and non-participants, and Tukey HSD (honestly significant difference) post-hoc tests when multi-level attendance categories were compared. Because participation was voluntary, analyses are quasi-experimental and therefore interpreted cautiously in light of known selection biases. For this reason, results are discussed in alignment with participation intensity rather than simple binary contrasts.

Finally, the cost analyses (AOI 4) relied on aggregated financial data provided by the university’s central accounting services. These data consisted of total annual expenditures associated with SI-PASS, including leader compensation, program coordination, and administrative support. While detailed cost breakdowns were not available, the global figures enabled the calculation of efficiency ratios by relating total expenditures to participation and performance indicators from AOIs 1–3, thereby integrating the fourth dimension into a broader evaluation of program effectiveness.

Taken together, these four data streams were triangulated to produce a multimodal evaluation of SI-PASS schemes at ULiège. This structure mirrors the “3P” framework proposed by Verpoorten et al. (2021) and extends it by introducing a fourth “P” dedicated to cost-efficiency, thereby addressing a gap in SI-PASS evaluation, as economic and cost-related indicators remain rarely reported in the literature (Bengesai et al., 2023; Dawson et al., 2014).

Data Treatment and Ethical Considerations

Quantitative analyses were conducted using JAMOVI (version 2.3). Depending on the variable structure, independent-samples t-tests, one-way ANOVAs (analysis of variance), Tukey HSD post-hoc tests, and chi-square tests were used. When cell sizes were insufficient to satisfy test assumptions (e.g., expected frequencies < 5), statistical tests were not performed; this is explicitly indicated in the corresponding tables. Qualitative analyses of open-ended survey responses were conducted using reflexive thematic analysis (Braun & Clarke, 2006). Missing or incomplete data (e.g., occasional missing attendance reports, non-response to perception survey items) were not imputed. Analyses were carried out on available data, and missingness rates are reported where relevant. Potential biases due to survey non-response or incomplete attendance logs are acknowledged in the limitations.

Data collection followed approval from the Ethics Committee of the Faculty of Psychology, Speech Therapy, and Education Sciences (protocol 2122–098, amendment approved on 10/13/2023). All data were pseudonymized prior to analysis: names, student IDs, and leader identifiers were removed and replaced with numerical codes. Given the risk of reidentification through data triangulation (attendance patterns by course by year), raw data cannot be made publicly available; only aggregated results are reported.

Participation (AOI 1)

Table 1 provides a multi-year and multi-faculty overview of an SI-PASS program conducted in a French-speaking Belgian university. Bolded columns correspond to indicators already featured in the SRE and are therefore benchmark ready. Remaining columns present additional metrics that proved useful in the ULiège experience and whose relevance could be discussed within SI-PASS community. Detailed explanations and calculation procedures for each indicator reported in Table 1 are provided in Appendix 2, and course details are in Appendix 3. The second semester of 2019–2020 and the first semester of 2020–2021 coincide with COVID-19 lockdown periods (March 18–May 3, 2020 and November 2–December 13, 2020; Belga, 2021), a conceptual factor that should be taken into account when interpreting participation patterns and program delivery conditions since SI-PASS schemes were maintained but transitioned to an online format.

Perception (AOI 2)

Performance (AOI 3)

Table 7 compares the scores obtained by students who attended SI-PASS and those who did not with a t-test. Out of the 16 iterations, 10 are statistically significant, in favor of engagement in the program. Amongst the six nonsignificant ones, three schemes took place in the pandemic period. The discrepancy between the number of participating students (n SI-PASS) in Table 7 and the Actual Unique Participants shown in Table 1 (Column 4) occurs because some SI-PASS students did not sit the final exam (having withdrawn from the course, changed options, or similar) and were therefore excluded from the academic performance analysis. In Sciences, the first semesters of 20–21 (n = 13) and 21–22 (n = 30) yield no effect, whereas the second semester of 21–22 becomes significant when participation rises to n = 63, hinting at a sample-size threshold. Yet that explanation fails in Applied Sciences, where large samples (n = 108 in the second semester of 19–20; n = 190 in the first semester of 20–21) also produce null results.

Table 8 summarizes the minimum number of SI-PASS meetings needed to produce a statistically significant improvement in academic performance. Attendance thresholds at which group means begin to diverge significantly were calculated with a Tukey’s HSD. A clear pattern emerges across contexts, faculties, years, and semesters: significant grade gains appear only after five to eight meetings. Bioengineering is the most consistent, with improvement evident after five sessions in every cohort to date. Psychology is also constant but has never shown significance below eight. Sciences contributes a single significant datapoint (also at eight), while Applied Sciences oscillates between six and eight.

Price Expenditures (AOI 4)

Over the past three years, and among the four faculties concerned, the total number of leaders added up to 229 (Table 1, Column 9). Leaders are formally employed by the university on 35-hour contracts and earn €10 per hour. Their wages constitute the program’s principal expense (Table 1, Column 16), as it amounted to a total of €81,900 for the studied period and faculties. This sum only relates to the leaders’ salary.

Faculties’ Storylines

Across the four faculties, the results reveal four distinct SI-PASS “profiles” shaped by disciplinary culture, course affordances, student motivations, and local program history. These profiles help clarify patterns that remain partially concealed in aggregated multi-faculty analyses.

Applied Sciences shows the clearest pattern of academically driven participation. Students overwhelmingly cite mastery-oriented motives and attend strategically, with early-semester surges followed by steady attrition. Performance effects are present but inconsistent, appearing only when weekly attendance stabilizes above mid-semester levels. This cohort might be particularly sensitive to workload peaks, treating SI-PASS as an efficiency optimization tool: a resource to use when needed, not a community to join.

Bioengineering, by contrast, displays the most regular performance gains, with significant effects emerging after exactly five sessions in every cohort. Despite variations in participation rates, these students consistently report pragmatic motives (keeping pace, verifying understanding) that translate into stable attendance behaviors. The Bioengineering profile therefore represents the most “aligned” case, where disciplinary culture, student expectations, and SI-PASS mechanisms converge to produce predictable effects.

Psychology presents a markedly different narrative. Participation is high, but for notably different reasons: social and affective motives dominate (e.g., ambiance, emotional support, approachability of leaders). Students use SI-PASS not only for academic clarification but also as a space for belonging and reassurance. Performance gains emerge only after eight meetings, suggesting that the discipline’s motivational profile enhances engagement but requires sustained exposure to translate into measurable grade improvements.

Sciences remains the smallest and least stable cohort. Participation is moderate and performances effects appear only once, in the semester where attendance increases substantially. Motivations tend to mirror those of Applied Sciences (academic clarification) but remain weaker in intensity. The small cohort sizes and the varied history of SI-PASS implementation in this faculty likely amplify fluctuations. Sciences therefore represents a “low-signal” context where conclusions must remain tentative.

Participation

The multi-year coverage does not reveal any strong year or semester. The content-domain view is equally inconclusive: no single faculty emerges as a clear champion. Across the three-year dataset, levels of participation (Table 1, Column 5 or C5) fluctuate both within and between faculties, with one noticeable similarity: the lowest values usually occur in the second semester (except for Sciences 20–21, first semester). Two mechanisms may explain this “semester effect.” First, new students, because they are diligent, curious, or unsure, tend to sample every optional activity during their first months on campus (especially when SI-PASS sessions appear in their timetables and feel quasi-compulsory) as part of their search for academic and social footing (Byl et al., 2022). This aligns with broader findings that first-year students often “over-engage” early on as they explore available support before refining their study strategies (Kift, 2015; Tinto, 2017). However, a first semester at university can prove disillusioning. A proportion of the students remain officially enrolled in the second semester (C3) while, for many reasons, they disengage from studies, shrinking the second semester pool. Conversely, the initial semester may also boost the self-confidence of other students enough to perceive SI-PASS as less necessary in the second semester.

Table 1 shows three outliers in C5: exceptionally high overall percentages of participation for the faculties of Applied Sciences in 21–22 first semester (93.7%) and of Psychology in both 20–21 second semester (84.3%) and 21–22 first semester (83.5%). Session logs (Appendix 4) suggest that these peaks were driven by unusually high first-week attendance followed by rapid attrition. Rather than a spread out “one-time visitor” phenomenon, a phrasing that overstates the exceptional nature of the pattern, it is more accurate to interpret this as a consistent behavioral tendency across cohorts, where students “vote with their feet”: many attend the opening meeting to assess the perceived value of SI-PASS, and then they decide whether to continue. This early evaluative behavior aligns with broader findings that students strategically allocate time based on perceived usefulness, scheduling constraints, and competing demands (Entwistle & McCune, 2004). Attendance declines further across the semester (steadily, but predictably) though a small uptick around the ninth meeting appears, plausibly tied to pre-exam anxiety—an established pattern in peer learning and tutoring programs, where usage often spikes before major assessments (Arendale, 2022; Palmer et al., 2009). This pattern also intersects with the academic calendar: attendance often rises in the weeks preceding major assessments, and leaders typically adapt by shifting sessions toward revision-oriented collaborative problem-solving, revisiting past difficulties, and allowing students to steer the agenda toward exam-relevant concerns, a form of responsive facilitation also noted in large-scale PASS implementations (Chi et al., 2008). Such adjustments help preserve a key concern of SI-PASS philosophy (starting from participants’ evolving needs, including during peak-stress periods) while simultaneously highlighting the risk of sessions being reduced to last-minute cramming, which does not align with SI-PASS intentions. Because of these twin declines (early attrition and gradual fading), ULiège’s mean head count per session (C15) remains well below the SRE benchmark of 11 despite an unclear calculation process. Nonetheless, Liège matches the SRE figure of 51% (C4) of eligible students attending at least one session; a notable result given that Liège reports by semester while the SRE uses a full academic year. In short, roughly half of the cohort tries SI-PASS, and about one quarter attends half of the sessions. The sudden and sharp attendance drop in Psychology during the second semester of 21–22 may hint at a third factor influencing attendance: target course selection. That semester, SI-PASS adjusted course coverage to include one course that did not use exercises and instead favored rote learning, which may have reduced the perceived value of the SI-PASS method. This example illustrates a broader point: course selection interacts with SI-PASS mechanisms, and future work should disentangle semester effects from course effects.

A structural factor likely to reduce overall attendance figures could be the number of prescribed meetings. At ULiège, leaders were contractually asked to run 13 one-hour sessions per semester (SRE by comparison: “on average the leaders hold about 15 sessions during an academic year being 0.5–3 hours in length”), yet the declared total (C12) rarely matches the theoretical target (C11). Interviews and attendance sheets revealed that many leaders simply could not squeeze 13 meetings into 15 weeks and that some of them compensated by running longer sessions (sometimes counting for one only in the presence report). Column 11 thus raises not only a factual discrepancy but a broader feasibility issue: the alignment between institutional expectations and actual operational constraints. After the study period, ULiège reduced the requirement to 12 sessions, a more realistic figure locally. Importantly, feasibility affects not only attendance but also institutional outcomes: if session scheduling becomes unrealistic, fewer meetings occur, mechanisms weaken, and SI-PASS cannot fully contribute to early academic engagement, one of the strongest predictors of retention and progression (Kuh et al., 2006; Tinto, 1997). This link is consistent with the primary goals identified across European SI-PASS programs, where improved retention and smoother transition into higher education are consistently reported as central motivations and outcomes (e.g., Bournemouth University, Queen Margaret University, and University of Bath as displayed in the SRE). Ensuring that the program remains operationally feasible is therefore a prerequisite not only for attendance, but also for securing the very institutional benefits (progression, reduced dropout, and academic adjustment) that justify SI-PASS investment.

Another consideration about attendance figures is that they might be higher than in Table 1, when attention is given to the number of leaders (C9) and diligent leaders (C10). Taken together, these columns reflect the difficulty of gathering exact data: six of 16 iterations lack attendance sheets (C10 < C9), meaning that some leaders, despite contracts and reminders, do not provide attendance data. This mirrors a well-documented challenge in educational field research, where missing administrative records introduce uncertainty and may lead to underestimation of participation metrics (Bowen et al., 2005). When meetings are not logged in the shared file as required, it remains unclear whether they took place or were not organized. Nonetheless, most leaders are generally diligent in reporting attendance, and the number of missing records remains limited. If we go from students’ attendance to “faculty attendance” in the program, C2 shows that in Liège, the number of faculties implementing SI-PASS has grown from one to four, but that this number has not changed since.

Regarding the link between pandemic and attendance, the COVID-19 pandemic sometimes boosted participation (e.g., Faculty of Bioengineering in the first semester of 20–21 compared to the second semester) and sometimes did not (e.g., Faculty of Applied Sciences in the first semester of 20–21 compared to the second semester). These are the only two faculties for which both semesters of the same year are present and thus allow for comparison. However, this measure could be considered a confounded variable with the previously mentioned pattern of the first semester being better attended than the second regardless of the year and faculty. Therefore, conclusions regarding a possible link between the pandemic and attendance ought to be nuanced. Even Malm et al. (2022) highlight several discrepancies between COVID sessions and regular ones, such as the complexity of monitoring engagement during online meetings and the increased complexity of the leaders’ tasks. These findings align with broader research on pandemic-era learning, which shows both increased flexibility and reduced interactional quality in online peer-learning environments (Baber, 2021). Data from this specific time were presented for the sake of exactitude and relatedness to other work examining virtual meetings (Huijser et al., 2008; Watts et al., 2015).

Perception

The primary aim of SI-PASS is to help students master demanding courses, and participants’ feedback confirms that it is viewed as an academic integration program. The dominant theme (“Better understanding of course materials” and “Learn, understand, and ask questions,” Table 2) echoes the foundational SI-PASS objective of supporting mastery in high-risk courses, corresponding with the SRE’s core focus on grades and graduation. Motivations such as “Keeping pace” reflect the alignment between SI-PASS scheduling and lecture progression. Social motives, although secondary, are consistent and meaningful: “Socializing,” “Ambiance,” and “Emotional support” indicate psychosocial gains that arise organically through the model’s collaborative, conversational structure. The “Elder leader” motive bridges academic and affective domains: students appreciate both the approachability (“Helped me deal with my growing stress”) of near-peers and their ability to demystify complex material (“Easier to ask questions to someone like us”). As seen in Table 2, faculty patterns are almost caricatural: Sciences, Applied Sciences, and Bioengineering provide the largest share of performance-oriented motives, whereas Psychology contributes most of the social ones. Finally, higher attendance correlates with a much stronger appreciation of both academic and social dimensions (Table 6); occasional visitors notice the benefits but not enough to join the core group.

If students are to attend enough SI-PASS sessions to internalize the program’s benefits (Table 6), logistics become a primary enabler or impediment, as highlighted in Table 5. At the faculty level, embedding SI-PASS in undisturbed timetables prevents competition with other academic activities. At the leaders’ level, scheduling sessions well in advance, establishing routines in time and venue, and (as they are contractually expected) sending a weekly reminder to every student in their group is critical to attendance. Yet the data still contain complains about missing or late information. Until these basics are secured, overall participation (C5) and average attendance per session (C15) are unlikely to rise.

More broadly, these constraints mirror findings from large-scale peer-support programs internationally. For instance, the Queensland University of Technology’s STIMulate initiative demonstrates how robust scheduling infrastructure (e.g., daily drop-in availability, dedicated service spaces, and accessible volunteer facilitators) helps maintain accessibility and mitigate organizational friction in peer learning services (Wilson et al., 2017). Meanwhile, peer support interventions in contexts like ours have encountered repeated challenges in sustainability and scaling when operational constraints (e.g., time, supervision, administration) were not addressed, even when student demand was present (Osborn et al., 2022).

Performance

Overall, the statistical results situate the ULiège SI-PASS experience within the longstanding positive pattern well summarized in this statement from the university, which is the cradle of the program: “The data overwhelmingly show that students who attend SI on a regular basis do substantially better than students who scarcely attend or don’t attend at all. This is true of each semester, and for the last 40 years at UMKC and internationally” (European Centre for SI-PASS, 2022, p. 21). Ten of the 16 iterations tested at Liège in Table 7 confirm this statement; six, however, do not. These nonsignificant cases are scattered across faculties and semesters, suggesting multiple interacting (and still unexplained) factors: sample size, cohort engagement, exam formats, online delivery constraints, varying levels of leader facilitation fidelity, degree of academic staff endorsement, and so on. Half of the null iterations coincide with COVID-19 semesters, when sessions moved online. The data show both lower attendance and weaker appreciation of the online format, reinforcing the association developed throughout this article between participation, perception, and performance. Untangling the interplay of cohort size and GPA at baseline, delivery mode, instructional context, and program fidelity will require targeted future research. Identifying why some cohorts gain while others do not is now the key to refining SI-PASS implementation and maximizing its academic impact. However, clear a pattern emerges: across faculties, years, and semesters, significant grade gains appear only after five to eight meetings. These exposure thresholds may serve as additional meaningful standardized metrics for future cross-institutional comparison, and they align with prior work showing that deeper collaborative mechanisms require repeated participation to stabilize (Gilman, 2025; Rokusek et al., 2022).

Price Expenditures

The current scholarship encourages researchers to question the efficiency of student support programs (Kraft, 2020). The issue of cost-effectiveness is nowadays especially pressing in French-speaking Belgium, where ARES, the overarching coordination of higher education, now requires institutions not only to supply evidence of impact but also to relate that evidence to program costs. Such financial data are still unusual: besides the wage figure reported here in Column 16, one of the few detailed estimates appears in Dekker et al.’s (2023) recent study: “The time spent by the SI leaders, the involved coordinators, trainers, project manager, and scheduler, times their respective salaries totaled €98 400. Divided by the 255 students who were granted access to the SI sessions is €385.88 per student” (pp. 5–6). Applying the same formula (C16/C4) to Liège yields a cost of €42 per participant, although it covers only the leaders’ salaries and excludes overheads such as supervision and administration. Even with those omissions, the figure compares favorably with the annual salary of a full-time teaching assistant. Beyond direct costs, SI-PASS may contribute to institutional outcomes of major relevance: retention, progression, and cumulative credit accumulation. Research consistently shows that early academic engagement and belonging predict persistence into the second year (Kift, 2015; Tinto, 2012), and SI-PASS’ combination of academic support and peer integration is precisely aligned with mechanisms known to reduce dropout risk. Thus, cost-benefit analyses should consider long-term institutional savings as well as short-term instructional costs.

Perspectives: Leveraging Metrics for Action

The metric set developed offers a roadmap for action at three complementary levels. Locally, faculties can identify where attendance thresholds fall short, where perception of value declines, or where cost per participant increases. This mirrors the emphasis in implementation science on using “local evidence loops” to support continuous program refinement through context-specific data (Fixsen et al., 2010). Institutionally, shared metrics enable cross-faculty comparison, highlight disciplinary specificities, support evidence-based decision-making for resource allocation, and demonstrate accountability to ARES and other external bodies—an approach consistent with broader higher-education governance trends emphasizing transparency, equity, and data-driven quality enhancement (Tight, 2018). Internationally, aligning local metrics with the SRE strengthens global comparability and contributes to the collective development of SI-PASS research. The additional descriptors proposed here, such as the five-to-eight-session effectiveness threshold, the cost-per-participant estimates, and the cross-modal triangulation, extend the common vocabulary and provide building blocks for cross-institutional studies (e.g., participation, perception, performance, cost-benefit studies, international comparisons linking perception patterns to academic outcomes, and controlled trials that vary variables to test their causal impact). Such work would deepen theoretical understanding in joint research while feeding evidence-based refinements back into practice. Ultimately, these metrics and their triangulations are therefore not ends in themselves; they are decision tools. Used iteratively, shared and discussed to increase knowledge, they can steer continuous improvement locally, inform policy choices centrally, and enrich collective knowledge globally. As the founder of SI-PASS already anticipated: “Collect, analyze, and report data each semester. Credibility relies on careful and systematic program evaluation” (Martin & Hurley, 2005, p. 319).

Limitations

Several limitations qualify these findings. Data incompleteness is the first: six of the 16 iterations lack attendance sheets (C10 < C9). Although most leaders submitted records, a few did not, so the counts in Table 1 are almost certainly underestimates (although a slight counter effect of over-reporting cannot be totally ruled out despite trust and follow-up). Data gaps of this type are commonly reported in program-monitoring literature, where fidelity records and attendance logs are often incomplete or uneven (Carroll et al., 2007). Exit-survey participation was modest for both attendees and non-attendees, introducing possible non-response bias in the perception results (Porter & Whitcomb, 2005). Moreover, data from the most recent semester in Applied Sciences is incomplete due to unexplained lost records. Secondly, sample sizes are sometimes small, reducing statistical power despite three-year aggregation. This limitation is typical of voluntary academic support programs, where participation variability constrains effect detection (Slavin, 2008). Third, although three years exceed the span of most published SI-PASS studies, they capture only the early institutionalization phase. Ongoing longitudinal tracking is needed to determine whether attendance patterns and effects levels continue to fluctuate or reach a point at which further measurement adds little value. Fourth, SI-PASS is voluntary. Compulsory participation would solve the small-sample problem and could streamline logistics for faculties, yet it might harm the program’s ethos, which remains anchored on the side of self-directed, mastery-oriented learning. Finally, comparisons with other institutions remain constrained by variation in data availability and reporting practices—precisely the gap this study aims to help close. Cross-institutional inconsistencies in measurement and reporting have been identified as a major barrier to cumulative SI-PASS research (Zha et al., 2019), underscoring the need for harmonized metrics such as those proposed here.