Introduction
In recent years, Open Educational Resources (OER) and Open Educational Practices (OEP) have become central reference points in debates on teaching and learning in Higher Education Institutions (HEIs, Atenas et al., 2024; Bellinger & Mayrberger, 2019; Hohaus, 2025b, 2025c; Mishra, 2025). These concepts describe closely connected dimensions of Open Education, which is considered a desirable orientation for contemporary higher education (Deimann, 2019). In practice, however, Open Education often remains uneven and tentative. Research shows that there are structural and cultural constraints shaping the implementation of OER and OEP in HEIs. These include limited policy support, insufficient infrastructure, low awareness of open licenses, and missing incentive structures at the institutional level (Swain & Pathak, 2024). Empirical work from German-speaking contexts lends further support to this perspective. For example, among large universities and universities of applied sciences, only a small percentage have formally adopted OER policies, which suggests that openness has not yet become the prevailing institutional norm (Hohaus, 2024b). Related strands of research emphasize that openness rarely stabilizes through isolated initiatives. Instead, it depends on longer-term processes of organizational development and cultural change (Hohaus, 2024c). These trends are reflected in the concept of Open Educational Culture, which foregrounds shared norms and values in organizations (Mayrberger, 2024). From a sociological perspective, this also becomes visible in the growing relevance of open educational platforms, which can be understood as part of broader dynamics of platformization shaping participation and visibility in higher education (Hohaus, 2024a).
Within these institutional frameworks, openness does not remain an abstract principle. It materializes in concrete teaching situations, which can be examined through the lens of Open Pedagogy (OP). OP describes teaching approaches that emphasize participation and shared responsibility in knowledge production (Hegarty, 2015; Tietjen & Asino, 2021). In OP-based teaching, the focus shifts from resources alone to learning processes, with the traditional role divisions between educators and students becoming more flexible. In this sense, OP aligns with critical approaches to learning that foreground meaning-making through shared practice and engagement with authentic contexts. Students engaged in open pedagogical designs report collaboration and real-audience engagement as central to their learning experiences (Gilpin et al., 2025). The present study turns to the question of how such sense-making processes unfold when openness is embedded in concrete seminar designs. The central aim of this exploratory study is to examine how students evaluate selected openness-related practices under controlled yet realistic conditions. As such, it does not test effects derived from a specific theoretical model, but seeks to map patterns in how students differentiate between pedagogical configurations. The focal point of this study is the evaluation of variations in presentation format, feedback type, and sharing mode in relation to the rating items of perceived appeal, distinctiveness, and impact. Furthermore, the study investigates whether these evaluations vary across students with different individual dispositions. This acknowledges that openness is unlikely to be experienced in a uniform way, and that student responses to openness-related practices may reflect heterogeneous orientations.
Framework
Contrary to the notion put forth by Wiley and Hilton (2018), which posits that open licenses are an indispensable component of OER-enabled pedagogy, this study adopts a notion of OP that includes aspects such as trust among learners in collaborative formats or engaging in peer review activities (Hegarty, 2015). More precisely, the present study focuses on three OP-related dimensions: group presentations (as opposed to individual presentations), peer feedback (as opposed to lecturer feedback), and sharing of materials beyond the classroom (as opposed to course-internal sharing only). The first two dimensions are understood as OP-related principles in a broader sense (Brandenburger, 2022), while the sharing mode represents OP-related principles in the narrower sense, associated with students’ materials being published under an open license or in the public domain (Wiley et al., 2014).
Pedagogically speaking, group presentations have been linked to the principle of learning by teaching, as preparing to present may help the presenters’ own learning in comparison to passive study (Remsberg & Hagar, 2019). Empirical research shows positive effects of group presentations on various learning-related aspects, such as, among others, self-efficacy (Ranjbar et al., 2025), critical thinking (Nasiri & Khojasteh, 2024) and commitment (Gumede, 2025). Studies also indicate that communicative competencies can be fostered effectively in the context of collaborative presentations. Evidence comes, e.g., from language-learning contexts, where presentations constitute an effective means of fostering and assessing learners’ spoken competencies. For example, Chou (2011) examined individual and group presentations among 52 third-year university students in a Professional English course in Taiwan, using questionnaires, interviews, and oral performance assessments. The results showed that learner strategies positively affected language performance in both presentation formats. However, group presentations had the strongest effect on improving students’ speaking ability, whereas other cooperative activities mainly enhanced learning motivation. Similarly, Hartono et al. (2023) analyzed the development of English presentation self-efficacy among Indonesian ESP undergraduates by comparing individual, homogeneous-group, and heterogeneous-group presentation tasks over one semester. The study found that presentation self-efficacy increased significantly, with homogeneous-group presentations leading to greater gains than both heterogeneous-group and individual formats. The extant literature indicates that individual differences in personality (e.g., extraversion) are also likely to shape how students engage with and evaluate collaborative formats such as group presentations (Müller et al., 2024).
The second dimension is peer feedback, which is commonly contrasted with lecturer feedback (or instructor feedback) (Li & Zhang, 2021). Research has indicated that the implementation of peer feedback can result in enhancements in various domains. These domains include, but are not limited to, student engagement (Emery et al., 2024), self-efficacy (Kiselyov & Schunn, 2024), teamwork skills (Díaz-Vicario et al., 2025), critical thinking (Shao et al., 2024) and communicative competencies (Zheng et al., 2023). Peer feedback has also been demonstrated to foster students’ resilience and confidence in the context of preparing and giving oral presentations (Lim et al., 2024). Moreover, it has been shown that peer feedback tends to be accurate and consistent to a large extent, even though there is a strong bias once the peer feedback becomes part of the final grade of the course (Sridharan et al., 2019). Personality traits can also influence how students experience and interpret peer feedback because individual dispositions shape not only engagement with peers but also how credible or anxiety-provoking learners find evaluative interactions with fellow students. Recent research has shown that in particular the personality traits of agreeableness and openness systematically affect the dynamics and perceptions of peer assessment processes (Alves et al., 2024).
Thirdly, the dissemination of student-generated content beyond the immediate course context constitutes an additional OP-related dimension that has been examined in this study. Watling (2012) underscores the pedagogical potential of engaging students in OEP, articulating this perspective with the concept of a “student as producer” culture. In this paradigm, learners are not positioned merely as recipients of knowledge but as active contributors to shared intellectual resources. This orientation is particularly evident in the concept of “renewable assignments”, defined as course assignments designed in a way that allows student-created work to be reused beyond the immediate instructional context (Lazzara et al., 2024). From a student perspective, studies show positive effects of renewable assignments on motivation (Chen & Hendricks, 2023). Empirical studies such as those by Braßler (2024) and Adedoyin and Altinay (2023) further demonstrate that students’ engagement in the production and sharing of OER can foster digital competence. To the best of the author’s knowledge, research has not yet examined how personality traits shape students’ evaluations of renewable assignments. Consequently, the present study draws upon the findings from related research on knowledge sharing among university students. While this is not equivalent to renewable assignments, this approach captures a closely related decision: whether students are willing to share self-produced knowledge beyond a closed learning group. For instance, Nishanthi and Munasinghe (2020) demonstrated that Big Five personality traits are systematically related to undergraduates’ online knowledge-sharing behaviour in educational contexts, with extraversion, openness to experience, conscientiousness, and emotional stability predicting a higher willingness to share knowledge, while trust considerations played a comparatively minor role.
In addition to the three dimensions of group presentation, peer feedback and external sharing, it is necessary to examine theoretical frameworks that may affect student perceptions of the seminar designs. In this study, Self-Determination Theory (SDT) and Expectancy-Value Theory (EVT) are employed as interpretive frameworks to differentiate between affective evaluations of comfort and support and value-oriented judgments of relevance and impact. SDT conceptualizes competence, autonomy, and relatedness as basic psychological needs that shape motivation and well-being (Ryan & Deci, 2000). SDT has been used in higher education research in various ways. For example, evidence from college courses indicates that instructor autonomy support (e.g., acknowledging students’ perspectives, offering meaningful choice, minimizing controlling language) predicts more autonomous motivation, which in turn relates to better course adjustment and performance (Black & Deci, 2000). Longitudinal work with university students similarly shows that perceived teacher autonomy support is linked to increases in students’ experienced autonomy and vitality over the semester (Núñez & León, 2016). SDT can also be used as a foundation for a learner-centred teaching approach, including, e.g., challenging tasks and post-lesson reflection (Jeno, 2015). Furthermore, SDT has been employed to investigate the potential causes of student attrition and to forecast academic performance (Jeno et al., 2018). In contrast, EVT approaches evaluations from a different angle, assuming that students’ achievement-related choices, performance, effort, and persistence are primarily determined by their expectations for success and the subjective value they assign to a task (Wigfield & Eccles, 2000). For example, according to Berweger et al. (2022), higher students’ expectancy-value appraisals are linked to positive achievement-related emotions, whereas lower appraisals are related to negative emotional responses. Their hierarchical analyses further indicated that intrinsic and utility value (but not attainment value) were associated with positive emotions. Research further demonstrates that expectancy-value interactions predict students’ dropout intentions, with high study values and low costs partially compensating for low expectancies of success (Breetzke & Bohndick, 2024).
Methodology
Factorial Survey Approach
A Factorial Survey Experiment (FSE) is a methodological framework that enables the systematic variation of hypothetical situations to analyze how different attributes influence participants’ evaluations. FSEs constitute a specific subtype of vignette studies, characterized by the systematic and experimental variation of multiple vignette dimensions in a factorial design (Auspurg & Hinz, 2015). Participants are presented with concise descriptions of hypothetical scenarios or individuals, in which specific dimensions are systematically varied. Subsequently, respondents are tasked with evaluating each vignette using a predetermined set of evaluation criteria (Dülmer, 2022). One of the advantages of factorial survey research is that it allows researchers to explore how different contexts affect judgments in the light of detailed scenarios in which multiple factors are varied (Wallander, 2009). Consequently, factorial surveys are instrumental in elucidating the manner, in which individuals assimilate multiple pieces of information to formulate an integrated evaluation. By embedding experimental manipulations within a realistic scenario, factorial surveys can support causal inference about the manipulated vignette dimensions under the assumptions of the design (Petzold & Wolbring, 2019). Therefore, factorial surveys have been used to investigate various aspects in higher education, including, e.g., students’ dropout decisions (Piepenburg & Beckmann, 2022), teacher expectations (Geven et al., 2021) and attitudes among HEI employees towards students with disabilities (Goodall et al., 2024). However, a review of the literature has shown that this approach has not been applied in OP-related research before. This finding is noteworthy, as the implementation of “openness” necessitates complex value judgments that are optimally addressed through vignette-based assessment (e.g., license type). Therefore, the implementation of an FSE in this particular context represents a novel methodological contribution. It facilitates the systematic examination of how students’ evaluations fluctuate in relation to disparate seminar designs. Given that teaching is rarely encountered as a single isolated feature but rather as a constellation of design decisions, a factorial survey approach is particularly suited to disentangle how students weigh and combine these elements in their evaluations. Designed as a within-subject design (Kim, 2010), this study aims to show how the same participants respond to multiple conditions or scenarios, which allows for the examination of differences in their judgments that are not confounded by between-person variability. This approach was chosen to maximize statistical power and to control for stable individual differences in evaluative tendencies. In order to “neutralize potential effects of the dimension order” (Sauer et al., 2020), it is common in FSE research to randomize the order of the vignettes across participants.
Design of the Survey
The FSE consisted of three distinct components: In the initial phase of the study, the participants were requested to provide information about their individual demographics, including their age, gender, the institution of higher education they currently attend (university or university of applied sciences), the specific field of study, the current semester, and their general attitudes regarding their chosen academic program. The second part of the study comprised the vignette experiment itself, which was based on eight vignettes specifically developed for the present study. In the third part of the questionnaire, participants completed the Big Five Inventory-10 (BFI-10), which is a brief and validated instrument designed to assess the Big Five personality traits of Extraversion, Agreeableness, Conscientiousness, Neuroticism, and Openness (Rammstedt, 2007; Rammstedt et al., 2014).
In the vignette component, participants first read an introductory text stating that all vignettes referred to a weekly 90-minute on-campus seminar conducted over twelve sessions within a single semester and attended by approximately twenty students. The seminar was described as aiming to support students in preparing for major academic projects, such as bachelor’s or master’s theses, and in developing advanced study-related competencies in the area of academic writing, research methods, data analysis, and the use of digital or AI-based tools for information retrieval and problem solving. The introductory text also indicated that in the seminar students are required to deliver a graded presentation at the end of the semester, with the presentation topic being assigned in the first session. Next, participants were instructed to carefully read the eight vignettes, each of which was described as representing a slightly different version of this same seminar setting, and to evaluate them independently from one another. The instruction explicitly emphasized that there were no right or wrong answers, as the goal was to capture participants’ personal judgments and impressions of the various course designs. Each vignette represented a distinct combination of three experimental dimensions that systematically varied across the scenarios: 1) Presentation format (whether the presentation was prepared and delivered individually or in small groups of three to four students), 2) type of feedback (whether students received lecturer feedback or peer feedback during the semester), and 3) sharing mode (whether the final presentation materials were shared internally within the course or made externally available on an open platform). The combination of these three binary dimensions resulted in eight unique vignettes. All eight combinations were considered plausible; therefore, all vignettes were retained. Each participant evaluated all eight vignettes. In accordance with standard practices in FSE research (see previous subsection), the vignettes were shown in randomized order in order to safeguard against order effects (Sauer et al., 2020). With this full factorial design, all dimensions and levels were fully crossed, ensuring that the factors were orthogonal and that the causal effect of each dimension could be independently identified (Treischl & Wolbring, 2022).
The following two examples illustrate contrasting vignette variants, i.e. a combination of individual preparation, lecturer feedback and internal sharing (vignette A) and a combination of group presentation, peer feedback and external sharing (vignette B).
Vignette A: “In this version, the seminar is organized as follows: The presentations are prepared individually by the students and are also delivered individually on the presentation day. During the semester, students receive feedback from the instructor (e.g., professor or lecturer) to improve their presentation materials. The presentation materials created by the students (e.g., slides and examples) are shared exclusively within this seminar after the presentations have been delivered.”
Vignette B: “In this version, the seminar is organized as follows: The presentations are prepared collaboratively by small groups of three to four students and delivered jointly by the group members on the presentation day. During the semester, students receive feedback from their peers in the seminar to enhance the quality of their presentation materials. The materials created by the students (e.g., slides and examples) are made publicly available on an online platform after the presentations.”
After reading each vignette, participants were asked to evaluate the described seminar based on five rating items using five-point Likert scales (1 = strongly disagree to 5 = strongly agree). These items captured participants’ perceptions of the seminar in terms of perceived 1) overall appeal (“Overall, this seminar appears appealing and interesting to me.”), 2) distinctiveness (“The design of this seminar differs from most courses in my degree program.”), 3) broader impact (“I think this seminar could have an impact beyond its immediate participants.”), 4) motivational aspects (“I believe I would be motivated to actively participate in this seminar.”) as well as the 5) anticipated social recognition of their own work within such a course (“I can imagine that the work I do in this seminar would be positively recognized by others.”).
Sample
The factorial survey was conducted from September to October 2025. Participants were recruited through the crowdsourcing platform SurveyCircle (https://www.surveycircle.com/). The participants did not receive financial compensation. Instead, participation was incentivized through SurveyCircle’s point-based system, which increases the visibility of participants’ own surveys on the platform and represents a common form of non-monetary compensation in this context.
The final sample comprised 184 HEI students (N = 184) after excluding incomplete survey responses and respondents who failed an attention check consisting of an instructed-response item (Muszyński, 2023). The majority of the participants were enrolled at universities (n = 103, 55.9%), while the remaining students attended universities of applied sciences (n = 81, 44%). Most participants identified as female (n = 135, 73.3%); the remainder identified as male (n = 49, 26.6%). Students came from 15 different federal states in Germany, with the largest groups representing North Rhine-Westphalia (n = 50, 27.2%), Baden-Württemberg (n = 33, 17.9%), and Bavaria (n = 26, 14.1%); no students from Brandenburg took part. A small number of participants were enrolled at institutions outside Germany (Austria: n = 2; Switzerland: n = 2). The mean age of the participants was 26.30 years (SD = 4.68; range = 19–50), and the average semester of study was 9.26 (SD = 3.50; range = 2–26). The mean personality trait scores (1–5 scale), as measured by the BFI-10, were as follows: Extraversion (M = 3.13, SD = 1.02), Agreeableness (M = 3.32, SD = 0.78), Conscientiousness (M = 3.81, SD = 0.77), Openness (M = 3.68, SD = 1.07), and Neuroticism (M = 3.15, SD = 0.94). The participants’ academic disciplines were categorized into three overarching domains: a) economy and business administration (n = 86, 46.7%), b) STEM and psychology (n = 63, 34.2%), and c) humanities and social sciences (n = 35, 19%).
The study was conducted in accordance with the prevailing ethical standards for research involving human subjects. Prior to the survey, participants were informed about the purpose of the study, the use of the data for research purposes only, and their right to withdraw at any time without disadvantage. The storage and processing of data was conducted in accordance with the provisions of the General Data Protection Regulation (GDPR). At the beginning of the survey, participants received a short introductory text outlining the general aims of the study and a debriefing provided after completion offered a more detailed explanation of the study’s objectives. The data that support the findings of this study are not publicly available due to data protection and privacy considerations.
Data Analysis
With all participants evaluating the same set of eight vignettes, a hierarchical data structure with multiple evaluations nested within individuals emerged. The three rating items of 1) perceived appeal, 2) perceived distinctiveness and 3) perceived impact were analyzed. Given the ordinal nature of the response variable and this clustered design, a Cumulative Link Mixed Model (CLMM) – a subtype of mixed-effects models designed for ordinal outcomes – with a random intercept for participants was fitted for each of the three rating items (Taylor et al., 2022). Vignettes were not modeled as a random effect as they differed exclusively with regard to the experimentally varied dimensions.
Model building followed a step-wise bottom-up approach (Gülzau, 2025): A null model including only a random intercept for participants was first estimated. In order to assess the extent to which variance in the ordinal responses could be explained by differences between respondents, the intraclass correlation coefficient (ICC) was calculated for the null model (Austin & Merlo, 2017). Next, a base model was implemented, incorporating the three experimental vignette dimensions (i.e. presentation format, feedback type, and sharing mode). Then, respondent-specific variables of gender, semester of study, and personality factors were added (“person model”). Finally, cross-level interactions between the primary variable of interest (sharing mode) and person-specific variables were. The derived p-values were corrected with False Discovery Rate (FDR), following the Benjamini-Hochberg method (Haynes, 2013). Given the exploratory nature of the study, both original and adjusted p-values are taken into account and reported in this article. Model fit was assessed with log-likelihood values, Akaike Information Criterion (AIC) and significance testing of fixed effects. All models were estimated using the Laplace approximation.
All statistical analyses were conducted in RStudio (Posit team, 2025) with R version 4.5.1 (R Core Team, 2025). The ordinal regression models with random effects were estimated with the function clmm() from the ordinal package (Christensen, 2023). Data visualization was performed with tidyverse (Wickham et al., 2019), dplyr (Wickham et al., 2019), ggplot2 (Wickham, 2016), and tidyr (Wickham et al., 2014). Model summaries and significance tests were obtained through the use of summary() and odds ratios (OR) were calculated by exponentiating model coefficients.
Results
Perceived Appeal of the Seminar (Item: “Overall, This Seminar Appears Appealing and Interesting.”)
Figure 1 presents students’ ratings across the eight vignettes for rating item 1. Table 1 summarizes the model coefficients, while Table 2 reports model fit indices. The null model indicated a high between-person variance, captured by a random-intercept variance of σ2 = 2.18, corresponding to an ICC of approximately 0.40. Across subsequent model specifications, random-intercept variances remained of similar magnitude (σ2 = 2.25 in the base model; σ2 = 2.02 in the person model; σ2 = 2.04 in the interaction model). The incorporation of seminar-related predictors into the base model resulted in an enhancement of model fit (ΔAIC = 18.1). Two predictors emerged as robustly associated with perceived appeal. Group presentation was positively associated with higher ratings of appeal (OR = 1.41, adj. p < .01), whereas peer feedback predicted lower ratings compared to lecturer feedback (OR = 0.70, adj. p < .01). The sharing mode (internal vs. external) showed no association with perceived appeal. When personality and demographic covariates were added in the person model, these associations remained stable. In addition, neuroticism showed a small negative association with perceived appeal (OR = 0.75), which was statistically significant before correction (p = .041) but did not remain significant after FDR adjustment (adj. p = .136). The interaction model did not reveal any statistically significant moderation effects of individual characteristics on the association between sharing mode and perceived appeal.
Figure 1
Student Evaluations (Rating Item 1).
Table 1
Base Model, Person Model and Interaction Model (Rating Item 1).
| TERM | BASE MODEL | PERSON MODEL | INTERACTION MODEL |
|---|---|---|---|
| External Sharing | 0.98 [0.81, 1.20]; p = .865, adj. p = .865 | 0.98 [0.81, 1.20]; p = .864, adj. p = .931 | 1.65 [0.24, 11.14]; p = .610, adj. p = .648 |
| Group Presentation | 1.41 [1.15, 1.72]; p < .001, adj. p < .01 | 1.40 [1.15, 1.71]; p < .001, adj. p < .01 | 1.40 [1.15, 1.71]; p < .001, adj. p < .01 |
| Peer Feedback | 0.70 [0.58, 0.85]; p < .001, adj. p < .01 | 0.70 [0.58, 0.85]; p < .001, adj. p < .01 | 0.70 [0.57, 0.85]; p < .001, adj. p < .01 |
| Female | — | 1.27 [0.73, 2.22]; p = .403, adj. p = .539 | 1.42 [0.77, 2.61]; p = .257, adj. p = .569 |
| Semester | — | 1.05 [0.98, 1.12]; p = .156, adj. p = .390 | 1.02 [0.95, 1.10]; p = .532, adj. p = .646 |
| Extraversion | — | 1.11 [0.86, 1.43]; p = .431, adj. p = .539 | 1.16 [0.88, 1.54]; p = .279, adj. p = .569 |
| Agreeableness | — | 0.99 [0.73, 1.34]; p = .931, adj. p = .931 | 1.09 [0.78, 1.52]; p = .601, adj. p = .648 |
| Conscientiousness | — | 1.22 [0.89, 1.67]; p = .216, adj. p = .418 | 1.15 [0.82, 1.62]; p = .425, adj. p = .582 |
| Openness | — | 1.14 [0.91, 1.42]; p = .251, adj. p = .418 | 1.14 [0.90, 1.45]; p = .290, adj. p = .569 |
| Neuroticism | — | 0.75 [0.57, 0.99]; p = .041, adj. p = .136 | 0.78 [0.58, 1.06]; p = .118, adj. p = .414 |
| External Sharing × Extraversion | — | — | 0.90 [0.72, 1.12]; p = .361, adj. p = .569 |
| External Sharing × Agreeableness | — | — | 0.81 [0.62, 1.06]; p = .122, adj. p = .414 |
| External Sharing × Conscientiousness | — | — | 1.13 [0.86, 1.49]; p = .368, adj. p = .569 |
| External Sharing × Openness | — | — | 1.00 [0.83, 1.21]; p = .982, adj. p = .982 |
| External Sharing × Neuroticism | — | — | 0.91 [0.71, 1.16]; p = .445, adj. p = .582 |
| External Sharing × Semester | — | — | 1.05 [0.99, 1.12]; p = .074, adj. p = .414 |
| External Sharing × Female | — | — | 0.79 [0.49, 1.28]; p = .345, adj. p = .569 |
[i] Note. Values are odds ratios with 95% confidence intervals in brackets. p-values are reported before and after false discovery rate (FDR) adjustment. Models were estimated using cumulative link mixed models with random intercepts for participants.
Perceived Distinctiveness of the Seminar (Item: “The Design of This Seminar Differs from Most Other Courses in my Program.”)
Figure 2 displays students’ distinctiveness ratings across the eight vignettes. Table 3 provides the model coefficients, while Table 4 presents the fit statistics. The null model revealed pronounced between-person heterogeneity, reflected in a random-intercept variance of σ2 = 2.86 and an ICC of approximately 0.46. Random-intercept variances remained at a comparable level (σ2 = 3.09 in the base model; σ2 = 2.94 in the person model; σ2 = 3.03 in the interaction model). The inclusion of vignette-level predictors improved model fit (ΔAIC = 75.9). Among the pedagogical dimensions, external sharing emerged as a strong predictor of perceived distinctiveness (OR = 2.43, adj. p < .001), indicating that seminars involving public sharing of student work were judged as considerably more different from other courses. By contrast, neither presentation format nor feedback type showed robust associations with distinctiveness after FDR correction. The incorporation of personality predictors within the person model did not change this pattern. External sharing remained robust in both the base and person models. In the interaction model, the effect of external sharing at the reference levels of the moderators was not statistically significant after FDR adjustment. Conscientiousness amplified the association between external sharing and perceived distinctiveness (OR = 1.50, p = .003), but this effect did not remain significant after FDR adjustment (adj. p = .058).
Figure 2
Student Evaluations (Rating Item 2).
Table 3
Base Model, Person Model and Interaction Model (Rating Item 2).
| TERM | BASE MODEL | PERSON MODEL | INTERACTION MODEL |
|---|---|---|---|
| External Sharing | 2.43 [1.99, 2.96]; p < .001, adj. p < .001 | 2.42 [1.98, 2.96]; p < .001, adj. p < .001 | 0.20 [0.03, 1.38]; p = .103, adj. p = .427 |
| Group Presentation | 0.89 [0.73, 1.09]; p = .261, adj. p = .261 | 0.89 [0.73, 1.09]; p = .257, adj. p = .513 | 0.90 [0.74, 1.09]; p = .269, adj. p = .571 |
| Peer Feedback | 1.21 [1.00, 1.48]; p = .052, adj. p = .079 | 1.21 [1.00, 1.48]; p = .052, adj. p = .261 | 1.21 [1.00, 1.48]; p = .052, adj. p = .295 |
| Female | — | 1.27 [0.66, 2.45]; p = .470, adj. p = .784 | 1.18 [0.58, 2.39]; p = .645, adj. p = .803 |
| Semester | — | 1.00 [0.93, 1.09]; p = .947, adj. p = .947 | 0.97 [0.89, 1.06]; p = .530, adj. p = .803 |
| Extraversion | — | 1.27 [0.94, 1.71]; p = .118, adj. p = .393 | 1.28 [0.93, 1.77]; p = .125, adj. p = .427 |
| Agreeableness | — | 1.06 [0.74, 1.52]; p = .737, adj. p = .947 | 1.03 [0.70, 1.51]; p = .890, adj. p = .890 |
| Conscientiousness | — | 0.96 [0.66, 1.38]; p = .812, adj. p = .947 | 0.78 [0.52, 1.16]; p = .220, adj. p = .569 |
| Openness | — | 1.01 [0.78, 1.31]; p = .939, adj. p = .947 | 0.98 [0.74, 1.29]; p = .862, adj. p = .890 |
| Neuroticism | — | 0.79 [0.57, 1.09]; p = .158, adj. p = .395 | 0.81 [0.57, 1.15]; p = .234, adj. p = .569 |
| External Sharing × Extraversion | — | — | 0.98 [0.78, 1.22]; p = .840, adj. p = .890 |
| External Sharing × Agreeableness | — | — | 1.07 [0.82, 1.39]; p = .629, adj. p = .803 |
| External Sharing × Conscientiousness | — | — | 1.50 [1.14, 1.98]; p < .01, adj. p = .058 |
| External Sharing × Openness | — | — | 1.08 [0.89, 1.30]; p = .440, adj. p = .803 |
| External Sharing × Neuroticism | — | — | 0.95 [0.74, 1.21]; p = .662, adj. p = .803 |
| External Sharing × Semester | — | — | 1.06 [1.00, 1.13]; p = .041, adj. p = .295 |
| External Sharing × Female | — | — | 1.19 [0.74, 1.91]; p = .483, adj. p = .803 |
[i] Note. Values are odds ratios with 95% confidence intervals in brackets. p-values are reported before and after false discovery rate (FDR) adjustment. Models were estimated using cumulative link mixed models with random intercepts for participants.
Perceived Impact of the Seminar (Item: “I Think This Seminar Can Have an Impact Beyond the Immediate Group of Participants.”)
Figure 3 visualizes students’ perceptions of external impact across the vignette conditions. Table 5 presents the model coefficients, and Table 6 reports the model fit indices. The null model indicated notable between-person heterogeneity, as evidenced by a random-intercept variance of σ2 = 1.28, which corresponds to an ICC of approximately 0.28. Across subsequent model specifications, random-intercept variances increased but remained within a comparable range (σ2 = 2.03 in the base model; σ2 = 1.78 in the person model; σ2 = 2.01 in the interaction model). The incorporation of pedagogical design variables within the base model resulted in a substantial enhancement of model fit (ΔAIC = 390.0). Among the vignette-level predictors, two factors were associated with perceived impact in the base model. External sharing showed a very strong positive association (OR = 8.11, adj. p < .001), while group presentation was linked to moderately higher impact ratings (OR = 1.32, adj. p < .01). Feedback type showed no statistically significant association with perceived impact. The inclusion of personality and demographic predictors did not change the central role of external sharing, which remained by far the strongest determinant of perceived societal impact. In the person model, female gender (OR = 1.77, p = .034) and extraversion (OR = 1.28, p = .043) were positively associated with perceived impact prior to correction, but these associations did not remain statistically significant after FDR adjustment. Neuroticism showed a marginal negative tendency that did not reach statistical significance (adj. p = .114). In the interaction model, conscientiousness (OR = 2.68, adj. p < .001) and openness to experience (OR = 1.52, adj. p < .001) both amplified the positive association between external sharing and perceived impact. Consequently, students with higher scores on the conscientiousness and openness scales found outward-facing seminar designs, which involved public sharing of student work, to be particularly impactful.
Figure 3
Student Evaluations (Rating Item 3).
Table 5
Base Model, Person Model and Interaction Model (Rating Item 3).
| TERM | BASE MODEL | PERSON MODEL | INTERACTION MODEL |
|---|---|---|---|
| External Sharing | 8.11 [6.50, 10.13]; p < .001, adj. p < .001 | 8.09 [6.48, 10.11]; p < .001, adj. p < .001 | 0.09 [0.01, 0.60]; p = .013, adj. p = .043 |
| Group Presentation | 1.32 [1.09, 1.61]; p < .01, adj. p < .01 | 1.32 [1.09, 1.61]; p < .01, adj. p = .026 | 1.32 [1.09, 1.61]; p < .01, adj. p = .023 |
| Peer Feedback | 1.05 [0.86, 1.28]; p = .625, adj. p = .625 | 1.05 [0.86, 1.27]; p = .631, adj. p = .744 | 1.04 [0.85, 1.26]; p = .708, adj. p = .803 |
| Female | — | 1.77 [1.04, 3.02]; p = .034, adj. p = .108 | 2.06 [1.13, 3.75]; p = .019, adj. p = .054 |
| Semester | — | 1.04 [0.98, 1.11]; p = .197, adj. p = .281 | 1.03 [0.95, 1.10]; p = .488, adj. p = .638 |
| Extraversion | — | 1.28 [1.01, 1.63]; p = .043, adj. p = .108 | 1.31 [0.99, 1.72]; p = .056, adj. p = .129 |
| Agreeableness | — | 0.94 [0.70, 1.25]; p = .670, adj. p = .744 | 1.02 [0.73, 1.41]; p = .916, adj. p = .969 |
| Conscientiousness | — | 0.78 [0.58, 1.05]; p = .102, adj. p = .170 | 0.48 [0.34, 0.68]; p < .001, adj. p < .001 |
| Openness | — | 0.97 [0.79,1.20]; p = .797, adj. p = .797 | 0.80 [0.63, 1.01]; p = .061, adj. p = .129 |
| Neuroticism | — | 0.77 [0.60, 1.01]; p = .057, adj. p = .114 | 0.79 [0.59, 1.07]; p = .125, adj. p = .229 |
| External Sharing × Extraversion | — | — | 1.0 [0.80, 1.24]; p = .969, adj. p = .969 |
| External Sharing × Agreeableness | — | — | 0.82 [0.63, 1.07]; p = .147, adj. p = .229 |
| External Sharing × Conscientiousness | — | — | 2.68 [2.04, 3.52]; p < .001, adj. p < .001 |
| External Sharing × Openness | — | — | 1.52 [1.25, 1.84]; p < .001, adj. p < .001 |
| External Sharing × Neuroticism | — | — | 0.94 [0.74, 1.20]; p = .615, adj. p = .746 |
| External Sharing × Semester | — | — | 1.04 [0.98, 1.11]; p = .148, adj. p = .229 |
| External Sharing × Female | — | — | 0.79 [0.49, 1.27]; p = .327, adj. p = .463 |
[i] Note. Values are odds ratios with 95% confidence intervals in brackets. p-values are reported before and after false discovery rate (FDR) adjustment. Models were estimated using cumulative link mixed models with random intercepts for participants.
Discussion
Regarding perceived appeal, students rated the seminar as more interesting when tasks were prepared and presented in groups and when feedback was provided by the teacher rather than by peers. A substantial body of research has demonstrated that collaborative group activities can promote social cohesion (Johnson & Johnson, 2014; Slavin, 2014), which may foster a sense of safety and support. Similarly, instructor feedback is often perceived as more credible and authoritative than peer feedback (Chen, 2021; Strijbos et al., 2010), which may further contribute to positive affective evaluations. It is important to acknowledge that in this survey, peer feedback was presented in an abstract manner without the use of scaffolding. This may have led participants to infer unsystematic or low-credibility feedback. The results imply that, in the absence of explicit scaffolding, peer feedback may be perceived as less supportive than instructor feedback, suggesting that training, criteria, and exemplars could be important for increasing its perceived value, which is supported by prior research (Carless & Boud, 2018). As previously mentioned, the dimension of sharing did not significantly influence perceived appeal. One possible interpretation of the absence of an effect is that making work visible to others introduces epistemic risk (e.g., the potential for criticism or loss of control). A small negative effect of neuroticism (not significant after correction) may suggest that students high in emotional instability are less comfortable with the performance aspects embedded in the seminar format, perhaps due to heightened evaluation anxiety, which mirrors evidence that links neuroticism to test anxiety (Yang et al., 2025). The pattern indicates that appeal judgments are shaped by cues of competence support and interpersonal security rather than by openness or public dissemination.
With regard to perceived distinctiveness, the findings show that external sharing was by far the most influential design feature, whereas presentation format and feedback type showed no robust association with perceived distinctiveness. Seminars in which student-created materials were envisioned as publicly available were perceived as standing out more clearly from other courses in students’ degree programs. This pattern suggests that distinctiveness judgments may be less concerned with how supportive or demanding a seminar feels, and more with the question of whether it departs from established academic routines. Despite the lack of representativeness of the sample in this factorial survey experiment, the connection between external sharing and perceived distinctiveness aligns with the notion that practices oriented towards openness persist as a relatively exceptional phenomenon. From the students’ perspective, the visibility of course outputs beyond the classroom appears to signal a deviation from the normative academic script. Under this script, student work is typically produced for the purpose of assessment and remains confined to the course context. The notion that external sharing makes a seminar “different” therefore suggests that OP-related practices involving students as producers have not yet become a normalized element of everyday teaching. From an actor-network perspective, this perceived distinctiveness can be understood as an effect of reconfigured pedagogical arrangements (Hohaus, 2025a), as external sharing introduces additional actors (such as public audiences and digital platforms) into the learning context. The recognition of student-generated knowledge artifacts as valid academic outputs, such as through digital portfolios or open repositories, has the potential to contribute to the integration of openness into the established practices of higher education.
For the rating item of impact, sharing emerged as the dominant influence on whether students believed that the seminar could have relevance beyond its immediate participants. The dissemination of student work led to a heightened societal impact assumption among participants. From a motivational perspective, this pattern can be interpreted in terms of attainment value (Wigfield & Eccles, 2000), as external visibility signals that the work produced in the seminar “matters” beyond course requirements. This strong association between external sharing and perceived impact aligns with core ideas of OP that frame learners as “knowledge creators” going beyond the more traditional notion of “knowledge consumers” (Clinton-Lisell, 2021, p. 256). When students’ work is made accessible beyond the classroom (whether through open repositories, blogs, or other platforms), it acquires an audience that extends beyond the instructor. In this sense, the seminar is no longer oriented solely toward assessment, but toward contribution. Within the framework of SDT, such outward-facing designs may support experiences of competence and relatedness, as students perceive their work as socially meaningful, and autonomy, as they act as contributors to a broader knowledge space (Ryan & Deci, 2000). The present findings thus resonate with the concept of renewable assignments, which contrast traditional “disposable” coursework with assignments designed to retain value after completion (Wiley & Hilton, 2018). According to the findings of previous studies, engagement with renewable assignments has been demonstrated to be associated with more positive motivational experiences (Clinton-Lisell, 2021). Within the vignette context of this study, these dynamics are reflected in students’ judgments that externally shared seminar outputs are more likely to have impact beyond the classroom.
Beyond the main effects of seminar design, the findings point to the role of individual differences in shaping how openness-related practices are evaluated. Personality traits moderated several associations, indicating that openness is not experienced uniformly across students. Specifically, conscientiousness and openness to experience strengthened the positive association between external sharing and perceived impact, though not with appeal or distinctiveness. Students who are high in conscientiousness may be especially responsive to designs that frame coursework as purposeful and socially relevant, while openness to experience is commonly associated with receptiveness toward novel and outward-facing learning arrangements. By contrast, neuroticism showed a small negative association with perceived appeal (even though this finding was not significant after FDR correction). This pattern is consistent with the assumption that students who are more sensitive to stress may experience performative or visible seminar elements as emotionally demanding. These effects serve to reinforce the distinction between affective evaluations of pedagogical comfort and value-oriented judgments of relevance and impact introduced earlier. While openness-related practices such as external sharing tend to increase perceived value, they may simultaneously be experienced as more demanding by some students. At the same time, the substantial between-person variance observed across all three evaluation outcomes indicates that individual differences in students’ judgments cannot be fully explained by the personality traits included in this study. Despite accounting for dispositional tendencies such as conscientiousness, openness, or neuroticism, a considerable share of variability remains unexplained. This suggests that student evaluations of openness-related seminar designs are also shaped by factors such as prior educational experiences or implicit expectations about academic work, which were beyond the scope of the present study.
Conclusion
The objective of this study was to examine how students evaluate openness-related seminar designs that vary along presentation format, feedback type, and sharing mode. The utilization of a factorial survey experiment has yielded evidence that openness is not perceived in a uniform manner. Instead, the activation of distinct evaluative logics is triggered by the implementation of different pedagogical features. While group-based presentation formats and lecturer feedback primarily shaped perceived appeal, external sharing of student work strongly influenced perceptions of distinctiveness and societal impact. These findings suggest that openness beyond the classroom is recognized as meaningful and consequential, even when it does not necessarily enhance immediate affective acceptance. The results also demonstrate how specific openness-related practices are evaluated differently depending on what they demand from students and what they promise in return. Beyond these findings, the study makes a conceptual contribution by highlighting student perceptions as an important lens for understanding the implementation of OP. Methodologically, the factorial survey design allowed the three OP-related seminar features to be varied systematically while holding the seminar context constant.
While the present study offers novel insights into how students perceive key dimensions of openness in course design and how their perceptions may be linked to personality factors, several limitations should be acknowledged: First, the factorial vignette design is a methodological approach that utilizes a hypothetical scenario to assess perceived outcomes, as opposed to the actual outcomes experienced by the subject. In essence, students evaluated hypothetical scenarios rather than actual courses. This approach isolates causal perceptions but may attenuate effects that depend on lived experience such as the benefits of engaging with the production of peer feedback (Nicol et al., 2014). Another limitation is that rating items 4 and 5, which were not further discussed in this article, and to some extent rating item 1 (see Figure 1) show evidence of ceiling effects, i.e. the tendency of participants’ responses to concentrate near the upper limit or the maximum attainable score of a given measurement instrument (Garin, 2023). It is possible that this pattern reflects positive evaluations of the presented scenario as described in the introductory texts of the vignettes. Such ceiling effects, which are fairly common for factorial survey research (Sauer et al., 2020), can reduce the sensitivity of the statistical models to detect effects between experimental conditions. The study is also limited by its abstraction from specific content, adopted to isolate the effects of particular factors. Put another way, the vignettes remained silent on the substantive content of OP, particularly its potential to advance or constrain social justice (Bali et al. 2020). Moreover, a further critique has been directed towards the validity of student evaluations of teaching (SET) in general. Although anonymous SET are commonly used to assess instructors and inform key academic decisions, they have been regarded as unreliable indicators of teaching quality as SET scores reflect biases and external factors instead of actual learning outcomes (Uttl, 2021, 2024). Finally, across all three evaluation outcomes, the random intercept variances were substantial. The ICC ranged between approximately 0.28 and 0.46. This highlights that evaluative responses are not only shaped by experimental manipulations but also by further motivational orientations that were not covered by the respondent-specific instruments applied in this survey. From an exploratory perspective, this residual heterogeneity points to the complexity of student sense-making processes rather than to a shortcoming of the design.
Future research could build on these findings in several ways. A synthesis of vignette-based methodologies with field studies of authentic open practices would enable researchers to examine the translation of perceived appeal, distinctiveness and impact into engagement and learning over time. Longitudinal designs could shed light on whether initial discomfort with outward-facing practices gives way to more positive evaluations as students gain experience and confidence. Subsequent research could also investigate the potential of integrating openness-related course designs with pedagogical safety and authenticity, with the aim of accommodating diverse student needs while preserving the feasibility of openness for heterogeneous student groups.
Data Accessibility Statement
The data analyzed in this study are not publicly available for legal considerations.
Sustainable Development Goals (SDGs)
The study is linked to the SDG of Quality education (SDG 4).
Author Note
Following the Academic Integrity and Transparency in AI-assisted Research and Specification Framework (Bozkurt, 2024), the author declares that DeepL (accessed February 2026) was used for the final stylistic editing of the manuscript. The final version of the paper is the sole responsibility of the human author.
Ethics and Consent
Participation was voluntary and anonymous. Before starting the online survey, participants were informed about the study and data protection in accordance with the EU General Data Protection Regulation (GDPR). By proceeding, they provided informed consent for participation and publication of anonymized results.
Author Contributions (CRediT)
Pascal Hohaus is responsible for: Conceptualization, Data Curation, Formal Analysis, Investigation, Methodology, Visualization, Writing – original draft and Writing – review & editing.