Introduction
Developing learner-centered and high-quality open educational resources (OER) for higher education courses in online learning environments presents multiple challenges, including perceptions that OER materials are lower quality than traditional textbooks (Irvine et al., 2021). This project was designed to address these challenges through multiple phases of user research, testing, and expert peer reviews. The research supported the writing and development of a multi-media OER textbook about information retrieval systems, aimed at students in an introductory course in an online Master in Library and Information Science (MLIS) degree program. Three elements in the research design and textbook’s development were shown to be essential to ensuring high quality: multiple stages of intentionally iterative user research and testing with students, peer reviews with subject matter experts (SMEs) to fully evaluate all subtopics, and accessibility testing to ensure inclusivity so that all students could easily use its multimedia content. The multimedia content of the textbook combines written chapters, embedded slideshows and videos, and links to supplemental readings. The MLIS students are the main intended readers; however, the book is also freely available to anyone interested in the topics covered.
The learning environment is a fully online MLIS degree program at a public university in the Western United States [anonymized for peer review]. Most students complete the degree in 2.5 years while they are working full-time, and many also have family responsibilities. The flexibility of online learning is the students’ most-cited benefit of the program. The particular course used in the study covers design concepts in information retrieval systems, and it is typically taken during a student’s first semester. The course had been redesigned around threshold concepts, which are critical concepts that transform a learner’s understanding of a given subject area and become integrative to their identity (Meyer & Land, 2003; Cousin, 2006; Timmermans, 2010; Tucker, 2018). This redesign prompted the need for a new textbook.
The research domains and relevant theoretical frameworks that informed the project’s design included open educational practices (OEP), Universal Design and accessibility-first studies, culturally-relevant pedagogy, and the impact of textbook affordability on student learning experiences. In addition, we reviewed the literature specific to the economics of OER for its effects on learner inclusivity, and we analyzed the format’s accessibility to support this aim. Semi-structured interviews and questionnaires were used in data collection.
The research design had four phases that were iterative, in order to support rigor in reviewing the book’s topical content, usability, and quality. The phases were: (1) alpha test with semi-structured interviews and follow-up questionnaires (n = 4); (2) domain expert peer reviews (n = 10); (3) beta test pilot course with questionnaires (n = 58); and (4) course delivery (n = 630) with survey data collection (n = 102). The implications of the project’s outcomes and the lessons learned can inform similar efforts in higher education. The elements that ensure rigor in the writing, peer reviewing, and testing with students make our recommendations particularly helpful to courses that are required for degree completion and that have high enrollments.
Research Objectives
The research question for the study was: What critical design elements ensure quality OER and student-centered learning experiences for higher degree students?
The two main objectives of the research were: to provide a textbook to support the course learning objectives and to reflect current technologies and trends in information retrieval system design. It was also essential that the content align with the scaffolded assignments in the course that cover database design, vocabulary design, and the basics of information architecture for websites. A key priority for the book was to build into its development both expert peer reviews and user testing with students, in phases that were iterative. In addition, to equitably support diverse learning styles, including knowledge practices entwined with non-Western and decolonized pedagogies, the book was designed with multimedia content, including embedded videos, recordings, and supplemental readings that provided more depth so students could choose to explore subtopics of particular interest to them. Finally, all content was thoroughly tested to meet current accessibility standards. A key goal was that all students have at hand their own copy of the textbook, whether digital (PDF), print format, or both. Our goal was a 100% adoption rate.
Learning Environment and Learner Profile
The project was conducted at a public university in the United States [anonymized for peer review]. The MLIS and other higher education degree programs offered through the school have been fully online since 2009. Faculty members use a combination of synchronous and asynchronous technologies to create engaging learning experiences (Marghalani & York, 2021). Synchronous technologies include Zoom and other interactive applications (VoiceThread, Panopto, etc.) that evoke in-person classroom learning experiences. Coursework is accessible through the Canvas LMS, including readings, recorded lectures, and discussion forums that facilitate asynchronous and peer engagement.
The use of these learning technologies mitigates many of the common pitfalls associated with distance education, such as a sense of isolation and lack of learner engagement (Akpen et al., 2024; Culduz, 2024; Heider, 2021; Wang, 2023). Students can meaningfully connect with their peers and instructors despite not being in the same physical location (Akpen et al., 2024; Crawford & Huling, 2024; Park & Choi, 2009). As an additional benefit, they learn how to be productive in online environments, which supports them in collaborative skills and open-mindedness (Highton et al., 2011), while preparing them for today’s virtual work environments. Learners are able to complete their degree at their own pace while still experiencing rich and lasting social learning.
The learner profile is reflected in data collected in exit interviews from the MLIS program (SJSU, 2024a; 2024b). These data show the following:
Ethnic diversity (2015–2021): White 59%; Hispanic 22.8%; Asian 10.3%; Black 4%; American Indian or Alaskan Native 1.3%; other or unknown 2.6%
Gender diversity (2021): female 75%; male 17%; 8% non-binary, transgender, or other.
Residency (2021): 55% California; 45% other U.S. states and other countries.
Current employment type (2021): 44% libraries; 29% education; 27% other.
Time to complete MLIS degree (as of 2020): 2.5 years or less 45%, 3 to 3.5 years 27%; 4 to 4.5 years 19%; 5 or more years 9%.
Another aspect of the learner profile considered in the project objectives was the adoption rate for the previous textbook so that we could compare it to the new textbook. For many years prior to 2015, the course textbook had been an expensive (over US$100) traditional textbook. In 2015, this changed to a coursebook of collected writings by faculty. The cost of the coursebook was $7 for the digital version and $33 for a print copy. During the preceding four semesters, with slightly over 2,000 students, 1,384 copies were purchased (751 digital and 633 printed), for an adoption rate of 69.2%. Two notes: We were unable to track how many students procured a copy from a student who had completed the course, nor could we track the number of students who might have purchased the book in both formats. As noted, our goal for the new book was 100% adoption rate.
Literature
To create an OER textbook that would equitably serve the needs of our learners, we examined literature on four themes: the learning needs of non-traditional students, the state of textbook affordability in higher education, emerging trends within open educational resources (OER) and open educational practices (OEP) development, and the principles underpinning universal design of educational resources.
Non-Traditional Learners
The flexibility of the online learning model is essential for many degree program participants who are adult learners and/or non-traditional students (Bozzetti et al., 2024). The category of non-traditional student has been conceived in various ways. These conceptions largely overlap in terms of seven characteristics that also underpin the definition used by the U.S. Department of Education’s National Center for Education Statistics (NCES) (2015). According to the NCES, the seven characteristics are: being independent for financial aid purposes, having one or more dependents, being a single caregiver, not having a traditional high school diploma, delaying post-secondary enrollment, attending school part time, and being employed full time (p. 7). These students face unique life pressures that may impinge on their educational success (Azfal, 2020). Similarly, due to structural barriers to socioeconomic mobility experienced by groups historically excluded from and underrepresented in higher education, learners from minority ethnoracial backgrounds are also often considered non-traditional students (Sapir, 2022; Stoessel et al., 2015; Wong & Hoskins, 2022).
The program’s spring 2024 exit survey reports that 71% of students “were between the ages of 26 and 45 at the time of graduation” (SJSU, 2024a, para. 18), and 83% were both students “and employees at public libraries, K-12 schools, academic libraries, and a host of other types of career environments” (SJSU, 2024a, para. 17). Balancing familial and financial commitments with their academic course load, a hallmark of the nontraditional student (Beckwith, 2023; Bozzetti et al., 2024; Dotta et al., 2020), is especially prevalent among participants, who cite the program’s distance education model and flexibility as factors that permit them to pursue an advanced degree (SJSU, 2024a).
Textbook Affordability and Development
The steadily increasing cost of textbooks (U.S. Bureau of Labor Statistics, 2021) imposes a considerable financial burden on students pursuing higher education in the U.S., especially non-traditional students. Textbooks are the largest out-of-pocket expense for many low-income students (Martin et al., 2017; Nguyen & Herron, 2021), and as a result of consistently high textbook costs, low-income students may experience significant academic and emotional difficulties. A recent study by Nagle and Vitez (2020) sheds light on this startling phenomenon. Of the students they surveyed,
66% reported they had not purchased assigned materials for one or more courses (p. 2);
25% reported working extra hours to purchase textbooks (p. 3);
11% reported skipping meals due to the cost of course materials. (p. 3)
Of the students who reported forgoing course materials, 90% expressed concern that this choice would negatively impact their grade. (Note: Nagle and Vitez issued a report in 2021; however, the 2021 statistics are heavily focused on impacts of the COVID-19 pandemic, therefore we chose to use their 2020 report.)
Similarly stunning findings have been reported across the relevant literature:
Students will prioritize purchasing textbooks for in-major courses and forgo materials for general education courses (Wittkower & Lo, 2020);
Students’ grades are negatively impacted when they cannot afford to purchase assigned materials, and in some cases, students may fail the course (LeMire, 2024; Martin et al., 2017; Wittkower & Lo, 2020);
Textbook costs influence students’ course selection (Jenkins et al., 2020; Martin et al., 2017);
Students may take fewer courses each semester due to the cost of associated materials, delaying graduation or contributing to their choice to withdraw without obtaining a degree (Buczynski, 2007; DiSanto et al., 2019; Martin et al., 2017; Jenkins et al., 2020).
Textbook prices pose barriers that are even more significant for racial/ethnic minorities, low-income students, and/or first-generation college students (Jenkins et al., 2020, p. 4).
In addition to common metrics, such as grade point average and graduation rate, the impacts of high textbook costs are reflected in qualitative studies of student experiences. When students cannot afford to purchase required materials, they may experience anxiety and mental health struggles (Brandle et al., 2019; Nguyen & Herron, 2021). They may also experience alienation or a diminished sense of belonging in their courses and be less engaged with their peers, instructors, and the course material (Nguyen & Herron, 2021).
The experiences of instructors and authors of original textbooks should also be considered when discussing factors in OER textbook development. Pearce et al. (2022) found that “open pedagogy can be one way of building the kind of deep engagement necessary to activate a shift in identity – from novice to active member of a practitioner community” (para. 5). Designing and using OER to foster a culture of open pedagogy and support personal transformation challenges traditional norms of higher education, evoking a range of positive, negative, and conflicted emotions among students and faculty. The commitment to change and growth that this process signals nonetheless aligns with the flexible and expansive ethos of culturally responsive pedagogy. Beyond cost-effectiveness, OER development thus offers rich cognitive, affective, and cultural rewards for learning communities.
Open educational resources (OER) & practices (OEP)
The rising cost of textbooks is entwined with another trend in higher education: the emergence of OER, open educational resources, and OEP, open educational practices. OER “generally refer[s] to educational resources that are publicly shareable through multimodal data (e.g., text, audio, and visual stimuli)” (Moon & Park, 2021, p. 315), and therefore freely available to learners, and OEP refers to the teaching methods or pedagogy that facilitate learner engagement and achievement using OER. The flexible application of OER and the emerging concepts of OEP and OER-enabled pedagogy are often theorized in terms of the ‘five Rs’: retain, reuse, revise, remix, redistribute (Moon & Park, 2021; Schroeder & Donat, 2023; Wiley, 2014; Wiley & Hilton, 2018; Zaid et al., 2024). These factors represent the range of behaviors that authors can choose to make available to users, resulting in an ‘open’ resource (see Table 1). Authors can make any/all of these rights available to users, most commonly through various Creative Commons licenses (Heck et al., 2020). Instructors and learners are consequently empowered to adapt OER based on the context of instruction and learner needs to facilitate creativity, learning, enrichment, and culturally responsive pedagogy (Kurek, 2016; Wolfenden et al., 2012).
Table 1
The ‘Five Rs of Openness’ elements (Wiley, 2014; Wiley & Hilton, 2018).
| FIVE Rs OF OPENNESS | DESCRIPTION |
|---|---|
| Retain | the right to make, own, and control copies of the content |
| Reuse | the right to use the content in a wide range of ways (e.g., in a class, in a study group, on a website, in a video) |
| Revise | the right to adapt, adjust, modify, or alter the content itself (e.g., translate the content into another language) |
| Remix | the right to combine the original or revised content with other open content to create something new (e.g., incorporate the content into a mashup) |
| Redistribute | the right to share copies of the original content, your revisions, or your remixes with others (e.g., give a copy of the content to a friend) |
The relative novelty of OER materials means fewer studies have been conducted on their quality and efficacy as compared to their traditional ‘non-open’ counterparts. However, emerging research is beginning to converge on three points:
First, the introduction of OER in a higher education setting has no adverse impact on student achievement (Hunsicker-Walburn et al., 2018; Luo et al., 2020; Croteau, 2017), and in many instances is correlated with improved learner motivation and/or academic performance (Cooney, 2017; Read et al., 2020; Colvard et al., 2018; Feldstein et al., 2012; Hilton et al., 2016; Hunsicker-Walburn et al., 2018; Bliss et al., 2013; LeMire, 2024; Weller et al., 2015);
Second, students who take courses that assign OER make faster progress toward graduation than students who take courses using traditional textbooks (Fischer et al., 2020; Hilton et al., 2016);
Third, students tend to express strong satisfaction with the quality of OER used in their courses (Cooney, 2017; Hunsicker-Walburn et al., 2018; LeMire, 2024; Lin, 2019; Magro & Tabaei, 2020; Wynants & Dennis, 2022), and cite cost-effectiveness and ease of use (Cooney, 2017; Feldstein et al., 2012; Hunsicker-Walburn et al., 2018; Magro & Tabaei, 2020) as distinct advantages of these materials.
An additional benefit of OER is its capacity to facilitate more independent and self-directed learning (Lin, 2019; Olivier, 2020; Stracke et al., 2023), and thereby promote the development of a non-dogmatic cognitive orientation called ’open thinking’ (Jung & Lee, 2022), which includes openness to multiple perspectives, new learning, collaboration, sharing, and to diversity and inclusion. Lifelong and socially responsive learning (Bozkurt et al., 2025) have also emerged as a trend in open learning priorities, reflecting a convergence (and synergy) between OEP, which “strives to be antiracist, democratizing, liberatory, and decolonized” (Jhangiani, 2019, para. 6), and the mindful integration of multiple (non-Western) pedagogies into higher education (Katz & Van Allen, 2022; Olivier, 2020).
Universal design: an accessibility-first design perspective
OER materials are ‘open’ in the dual sense that they are readily accessible for digital sharing and reuse and are publicly available free of charge. Despite the elimination of certain impediments, barriers to access remain as many OER lack accessibility features for learners with disabilities (Lomellini et al., 2025; Navarrete & Luján-Mora, 2018; Weirick-Johnson & Abumeeiz, 2022; Zhang et al., 2020). Common issues include: a lack of compatibility with assistive technologies (Lomellini et al., 2025; Majeed, 2018; Moon & Park, 2021; Navarrete & Luján-Mora, 2018; Zhang et al., 2020), inaccessible interface design and navigation elements (Lomellini et al., 2025; Moon & Park, 2021), inaccurate or missing captions, transcripts, and alternative text (Lomellini et al., 2025), and complicated language and jargon without plain language alternatives (Lomellini et al., 2025; Navarrete & Luján-Mora, 2018).
An accessibility-first design movement is gaining momentum in higher education but remains underutilized and underexplored in both OER and commercial learning resources. As the number of students pursuing higher education who identify as having one or more disabilities continues to rise (Zaid et al., 2024), the need for design justice (Weirick-Johnson & Abumeeiz, 2022) and an accessibility-first design approach has come into ever sharper focus. Also referred to as ‘universal design’ or ‘universal design for learning’ (UDL), this philosophy exhorts instructional designers and their host institutions to “employ accessibility as a framework for design and not an afterthought applied to already created materials” (Weirick-Johnson & Abumeeiz, 2022, p. 17, emphasis added). Fundamentally, it is a “framework for improving instruction because it helps provide equal opportunities for all learners to succeed” (Marghalani & York, 2021, p. 4), and it achieves this by adhering to seven principles: equitable use, flexibility in use, simple and intuitive use, perceptible information, tolerance for error, low physical effort, and size and space for approach and use (University of Washington, 2022). See Table 2.
Table 2
Seven principles of UDL as defined by the University of Washington (2022).
| PRINCIPLE | DEFINITION |
|---|---|
| equitable use | The design is useful and marketable to people with diverse abilities. A website designed so that it is accessible to everyone, including people who are blind, employs this principle. |
| flexibility in use | The design accommodates a wide range of individual preferences and abilities. An example is a museum that allows a visitor to choose to read or listen to the description of the contents of a display case. |
| simple and intuitive use | Use of the design is easy to understand, regardless of the user’s experience, knowledge, language skills, or current concentration level. Science lab equipment with control buttons that are clear and intuitive is a good example of this principle. |
| perceptible information | The design communicates necessary information effectively to the user, regardless of ambient conditions or the user’s sensory abilities. An example of this principle is when a TV in noisy public areas includes captions. |
| tolerance for error | The design minimizes hazards and the adverse consequences of accidental or unintended actions. An example is an educational software program that provides guidance when the user makes an inappropriate selection. |
| low physical effort | The design can be used efficiently and comfortably and with a minimum of fatigue. Doors that are easy to open by people with a wide variety of physical characteristics demonstrate this principle. |
| size and space for approach and use | Appropriate size and space are provided for approach, reach, manipulation, and use, regardless of the user’s body size, posture, or mobility. A science lab designed for use by students with a wide variety of physical abilities is an example of this principle. |
While the seven principles are not equally applicable in every design scenario, together they offer a set of heuristics for prioritizing the access and comfort of learners with a range of diverse abilities. By designing for each learner, we are designing for all learners, thus ensuring inclusivity and equitable and universal accessibility in the learning environment.
Research Design and Methodologies
The research design involved methodologies that were applied with the relevant research literature, discussed above, guiding our decisions. Both qualitative—interviews and think-aloud protocols—and quantitative methods (surveys) were used. In addition, ten experts were recruited to conduct peer reviews of the book’s content, with each expert assigned to review topics in their specific domains. The research design had four phases: (1) alpha test with semi-structured interviews and follow-up questionnaires (n = 4); (2) domain expert peer reviews (n = 10); (3) beta test pilot course with questionnaires (n = 58); and (4) course delivery (n = 630) with survey (n = 102). All data collected and analyzed during the study are present within this article. The survey instruments used in Phases 3 and 4 are available from the corresponding author on reasonable request.
The learning environment was critically important in the research design. Since the course is offered totally online, we also conducted the research totally online. We held the interviews via Zoom, then edited the transcripts for accuracy, and we distributed the surveys via email. All of the alpha participants also sent further ideas and feedback to the lead researcher via email, further supplementing their responses to the questionnaire and the interviews.
Methodologies & Data collection Phases
Phase 1: Alpha Test Interviews and Questionnaire (n = 4)
For the alpha test, we combined three user research methods with each participant: semi-structured interviews, independent experimentation by the participants using the multimedia textbook and the course blog, and post-interview questionnaires to gather think-after reflections (Branch, 2000). Think-aloud protocols (Lewis & Rieman, 1994; Hope, 2007; Kvale & Brinkmann, 2009) were used during the interviews and experimentation to support responsive interviewing discussions with participants (Rubin & Rubin, 2005).
The participant sample was drawn from students who had taken the course six months prior to Phase 1. This meant that they were able to provide comparisons between the old textbook and the new one, as well as to evaluate the new sequencing of the course content and assignments. Their perspectives were invaluable for these purposes of evaluating the textbook content, sequencing, and alignment with course assignments.
The alpha participants were pre-selected by the lead researcher based on their demonstrated ability during the previous course, taught by her, to articulate their impressions and their willingness to be uninhibited about expressing their views and ideas. They were capable students, but not necessarily the top performers in the class. Their ability both to analyze and to speak out about their experiences and perspectives was paramount. Because of this pre-existing relationship, our ability to conduct rigorously responsive interviews (Rubin & Rubin, 2005, p. 30–38), including the think-aloud protocols (Hope, 2007; Kvale & Brinkmann, 2009) and reflective think-after post-interview (Branch, 2000), was greatly enhanced. We had established familiarity and trust as a foundation and, in following the responsive interviewing model (Rubin & Rubin, 2005), we could, as interviewers, “contribute actively to the conversation” and be “flexible and adaptable” to participant responses (p. 36).
Seven students were invited to participate, and four accepted. All four were willing to commit 2.5 to 3 hours and eager to fully participate in the research project. Their compensation was a collection of university-branded merchandise (travel mug, etc.). Their time was spent on three activities: reviewing the book prior to the interview, the interview, and completing a post-interview questionnaire. The nature of the alpha interviews was highly participative and involved a significant level of commitment from all participants (Kvale & Brinkman, 2009). In addition to the three planned activities, all of the alpha participants contributed further ideas via emails with the lead researcher. For example, adding chapter-specific visual anchors was suggested by one participant who had a graphic arts background, and the lead researcher designed this feature, then invited the participant’s review of the new page layouts.
Each alpha tester was emailed information outlining the various activities and the appropriate ways to prepare. They were instructed to spend a minimum of one hour carefully reviewing and evaluating the book and accompanying materials, including the embedded recordings and reading links and the information on the course blog about PDF tools. Alpha testers were instructed to complete these pre-interview activities during the ten days prior to their scheduled interview date.
The semi-structured interview covered nine questions and lasted an average of 45 minutes. The main topics were:
Participant’s use of support materials, including the course blog;
Issues of usability and accessibility, or universal design, the participant had encountered;
Participant’s views on the clarity, quality, and coverage of textbook content;
Participant’s perception of the book’s cultural relevance for diverse learners and engagement with social justice themes.
Participants were also informed that they would have the opportunity to provide additional feedback via a follow-up questionnaire sent via email following their interview. This helped with the well-known differences in subjects’ abilities to verbalize their thoughts on the spot (van Someren et al., 1994). Interviews were recorded (with participants’ knowledge and consent), transcribed, and analyzed by both researchers.
After the interview, testers completed a questionnaire with 20 multiple-choice and short-answer questions. Its main purpose was to provide an opportunity to express any further thoughts and reflections on their experiences with the book. Questions included prompts rephrased from the interview, as well as new questions. Participants were allotted one weekend to complete the questionnaire, and all questionnaire results were anonymized.
Phase 2: Expert Peer Reviews (n = 10)
In the second phase, ten subject domain experts reviewed the 12 chapters and four appendices in the book. The experts were faculty at the university, and the materials were distributed according to the subject areas in which they conduct research and teach. Eight were current instructors of the course, and two were former instructors, selected for their expertise specific to metadata schemas and relational database management. The lead researcher invited the ten experts to provide exhaustive peer reviews on one or two chapters/appendices. Their reviews resulted in detailed peer reviews, covering the full range of information retrieval topics, addressing any errors and any gaps in the latest technologies and trends in their areas of expertise.
Phase 3: Beta Test (n = 58)
Phase 3, beta testing, was the piloting of the course using the new textbook in summer 2024. Data were collected from the students (n = 57) and from the two instructors. A survey with 18 questions about the new book was distributed by the two instructors to their respective students, with the time required for completion of approximately 10 minutes. Feedback was gathered from the instructors in open-ended questions via email. Responding to the Phase 3 survey was optional for students enrolled in the summer course. All responses were anonymous, and thus no formal consent was required.
Phase 4: Course Delivery (n = 630) and Survey (n = 102)
Twenty course sections in the fall 2024 semester used the new textbook, a total of 630 students, taught by eight different instructors. The same survey from the summer was distributed, with 18 questions about the book and its support materials (PDF tools and low-cost online print services). Using the same survey reinforced the reliability of the instrument and the validity of the results (Babbie, 2007; Cobern & Adams, 2020). In addition, most of the survey questions had been tested the previous two years; the questions about the textbook and blog being the only new ones.
In addition, all course faculty met at the end of the semester to discuss their experiences with the new book and the changes to the sequencing of the course content. As was the case with the summer survey, it was optional for students in Phase 4 to respond to the survey, and all responses were anonymous and not connected to their enrollment.
Results
Data were collected according to the four phases described above:
Phase 1. alpha test with semi-structured interviews and follow-up questionnaires (n = 4);
Phase 2. domain expert peer reviews (n = 10);
Phase 3. beta test pilot course with questionnaires (n = 58); and
Phase 4. course delivery (n = 630) with survey (n = 102).
The results after each phase were analyzed and reflected in content revisions, to improve the quality of the textbook and the user experience, before moving on to the subsequent phase.
Results by Data Collection Phase
Phase 1: Alpha Test Interviews and Questionnaire (n = 4)
The first phase, alpha testing, involved two forms of data collection with the four participants: semi-structured interviews (SSI) and a follow-up questionnaire to elicit any further thoughts and reflections.
Data collected during alpha testing, including both the SSI and questionnaire, demonstrated clear patterns that strongly endorsed the new textbook and the use of OER generally. Four themes were evident in the alpha test results: learner preferences, usefulness of the course blog, perceived usability, and comparison to the previous textbook.
Learner Preferences
Alpha testers reported being comfortable using digital resources and indicated a preference for a balance of print and digital learning resources. When asked, ‘Do you typically prefer to use print or digital resources (e.g., textbooks) when learning new concepts and skills?’, all participants identified at least one advantage of using digital resources (e.g., the convenience of embedded hyperlinks).
The participants’ relative comfort with digital resources was further affirmed in the follow-up questionnaire, with 75% agreeing or strongly agreeing with the statement, “I usually find eBooks easy to use” (see Figure 1).

Figure 1
Responses to statement, “I usually find eBooks easy to use,” using Likert scale.
Course Blog Usefulness and One-Stop Approach
When asked during the interviews about their experience using the blog, all alpha testers indicated they believed the blog would provide “enough” or “more than enough” information to a student with limited digital fluency. One tester (‘Jackie’) expressed that they had found the information on PDF readers and digital annotation tools to be “very helpful” and had learned valuable digital skills by accessing the blog. Participant ‘Nurja’ noted, “I looked at all of the information on the blog, and I loved it. It was great. It makes me wish that all of my classes had a blog like this, with…all of the information you need right there.”
Positive Usability
Positive usability was one of the strongest themes supported by the alpha testing data. As noted above, alpha testers’ positive opinions of the blog spoke to its usability, including its capacity to help users with more limited digital literacy skills succeed in the course. These favorable views were echoed in other responses.
In response to the interview question, “Tell me about your experience using the new textbook,” alpha testers offered overwhelmingly positive feedback. Participant ‘Meg’ noted that “nothing was confusing.” Another tester, ‘Nurja’, echoed this sentiment, stating: “Everything made total sense, and nothing was unclear to me.” A closely related idea about navigability also emerged: “It was laid out consistently, which was great; it was very easy to navigate.”
The easy navigability and clarity of scaffolded content as supportive of user experience were also substantiated by the questionnaire data. In response to “Each chapter in the textbook introduced a manageable amount of information,” all respondents agreed or strongly agreed, and similarly, in response to “The textbook clearly explained concepts,” all respondents agreed or strongly agreed.
In response to the open-ended question, “What were the most successful and least successful aspects of the textbook design and user experience for you?,” participants identified the most successful aspects as 1) “its level of accessibility,” 2) “the content flow, use of links and embedded materials, and ease of use,” and 3) the fact that users could “navigate easily, the content was clear and easy to understand” making it “very accessible.” When asked during the interviews, “What changes would you make to improve the textbook user experience?” all participants responded that they had no further suggestions.
Comparison to Previous Textbook
Alpha testers were asked to evaluate the design of the new textbook as well as compare it to the previous edition. During the SSIs, the latter question was posed as: “Based on what you recall from taking the course, how does the new textbook user experience compare to the last version of the textbook?” Participants responded:
All changes were “positives, or they made [the textbook] better” (‘Meg’);
The new edition is “much more user friendly…than the previous eBook. The information…[is] super accessible…similar to that, it’s very easy to understand, and the flow is accessible…for…many different skill levels or…learning styles.…this [edition of the textbook]…is just more…accessible…it just seems easier to use” (‘Nurja’);
“…the chapter layout this time around seemed to be much more organized” (‘Gina’);
The new edition was “easier to navigate” and “more user-friendly” (‘Jackie’).
Questionnaire responses echoed these sentiments. Answers to, “How did your experience with the new textbook compare with the one you used when you took the course last fall?” included:
“The new textbook is much more engaging and easier to follow”;
“I found this new textbook to be a much improved experience from the one I used last fall”;
“This eBook is much easier to use than the previous eBook…I had no issues using the new textbook at all”;
“The layout and figures were more logical and the supplementary materials were visually pleasing and the information was well presented.”
In response to the statement, “Compared to the textbook I used in my course last fall, this textbook is better organized for the course assignments,” all alpha testers strongly agreed; and again, in response to the statement, “I would recommend the textbook to other MLIS students,” all testers strongly agreed. The relevance and high usability of the textbook coupled with the fact that it was free of charge (which ‘Nurja’ identified as “incredible, and very helpful for students”) may account in part for alpha users’ positive evaluations.
At the conclusion of Phase 1, the lead researcher revised the book to address the alpha test feedback, then readied it for Phase 2, the peer reviews by ten subject matter experts.
Phase 2: Expert Peer Reviews (n = 10)
The expert peer review phase resulted in important revisions to the book content. These changes were of two types. First, there were updates for new technologies such as AI-guided tools, particularly relevant in the search engine chapters. Appendices, aimed at reference use, with information on metadata, were updated with the latest metadata system information. Second, the reviewers recommended changes to illustrations in the introductory chapters. For example, websites that provided examples of ways to organize, label, and design vocabulary systems to improve the user experience (UX) were replaced by the author to better exemplify best practices in information architecture.
Phase 3: Beta Test (n = 58)
A pilot version of the course used the new book during the summer of 2024, a ten-week term that covers the same content as a 16-week term, but in an accelerated format. There were two sections of the course, each taught by a different instructor, with a total of 58 students. In the seventh week, we distributed a survey that asked the students about their experiences with the book. Unfortunately, the response rate was disappointingly low: five students submitted the survey, and we determined that the results were not statistically meaningful. We evaluated the qualitative comments; however, there were none that called for changes to the book. Fortunately, we did gather meaningful feedback in our reach-out to the two instructors for their reflections. Both reported that using the new book was a major improvement over the previous textbook, that the sequencing of topics fully supported the assignments, and that the coverage of current technologies was excellent.
Phase 4: Course Delivery (n = 630) and Survey (n = 102)
In the fourth and final phase, the textbook was rolled out to 630 students in the 16-week Fall 2024 term, in 20 course sections taught by eight different instructors. A few weeks before the end of the term, a questionnaire was distributed by the instructors, with 31 questions about the course materials, including the textbook, assignments, and the custom tutorials and screencasts for the database software for the design assignments. Our focus here is on the 18 questions specific to the textbook. Results from Phase 4 are reported in three parts:
student learning environments to understand how they used the textbook (quantitative data), such as the devices and format (digital and/or print);
extracts from open-ended questions (qualitative data) about their user experiences; and
instructors’ perspectives on how their students experienced the textbook, summarized from the faculty meeting held at the end of the semester.
Learning environments and perspectives (quantitative)
Understanding the environments and ways in which the students used the textbook was part of the foundation for understanding their information experiences. Several questions addressed their approaches to using the book’s multimedia content. Figure 2 shows the devices most often used with the book, and Figure 3 represents their use of the print and/or digital (PDF) versions of the book.

Figure 2
Devices used by students with the textbook PDF.

Figure 3
Students’ frequency of using print and digital (PDF) versions.
The course blog where students downloaded the textbook includes information on PDF tools for viewing and highlighting, along with recommendations for text-to-speech (read-aloud) tools. All of the tools had been tested by our researchers. Figure 4 shows the students’ choices of tools.

Figure 4
PDF software tools/apps students used with the textbook.
The survey also asked about their overall experience with understanding concepts covered in the book, with Figure 5 showing that 83.3% of students agreed or strongly agreed that the textbook clearly explained the concepts; 16.7% were neutral on the statement, and no respondents disagreed.

Figure 5
Student opinions of textbook’s clarity in explaining concepts.
Qualitative responses and thematic patterns
The resultant themes in the qualitative responses about the user learning experiences are represented by quotes in the students’ own words (in vivo), shown in Table 3.
Table 3
Phase 4: Initial themes from qualitative datasets.
| THEME | REPRESENTATIVE PARTICIPANT QUOTES |
|---|---|
| Information structures in the book affecting the user experience | This textbook is my favorite. I appreciate having chapters and supplemental readings/videos all in one place. The most successful aspects of the textbook were the content flow, use of links and embedded materials, and ease of use. |
| Immersive experience from the multimedia content, embedded videos | The most successful aspect of the textbook design was its user-friendly navigation, allowing for quick access to chapters and sections. The search function also worked well, making it easy to locate specific topics. I appreciate the clickable “contents” sections that take you to specific sections/pages. The embedded Panopto videos were a great addition, as were all other linked resources and materials. |
| Freedom from obstacles in textbook content, benefiting the user experience. | I think the textbook design is very helpful, and it helps students save on money spent on course materials. The design of the content was very easy to navigate, and I liked the table of contents in the sidebar. Impact: the “one place” approach supports accessibility, learning styles, and inclusivity. |
| Learning modes are integrated in textbook content. | I love that the videos relate directly to the text and supplement it. It’s really nice to have everything in one place. User friendly. Easier to understand. Wish all classes were set up in this way. |
| Easy to interact with the concepts, to take ownership of it, and personally connect to it. | I liked that the videos were interspersed throughout the text to break up the reading. I like highlighting phrases I feel are important and searching for key topics for review and study. Flexible, Intuitive, User-Friendly, Accessible, Accepting. |
| High quality and understandability of textbook content | I didn’t find any…of the content…difficult to understand because it seemed to kind of build itself up… as you…went through, you were able to kind of scaffold that information to get a better understanding for later chapters. |
Learning ownership and integration
As occurred during Phase 1: Alpha Test, students emailed further unsolicited comments about the book after completing the survey and the course. These comments enriched the themes in the results, many representing gratitude and appreciation, but most importantly, a deepening sense of ownership of their learning experiences. This spontaneous feedback reinforced the theme in Table 3 of learning ownership about the concepts and integration of the threshold knowledge. We found this to be the basis of a key finding from the study with potential for further exploration.
Instructors’ viewpoints of student experiences
Further results were gathered during a faculty ‘roundtable’ with the instructors who taught the course in the fall term in 20 course sections. One instructor provided a summary of five key findings based on themes from this instructors’ discussion, shown in Table 4.
Table 4
Instructors’ findings from new textbook roll-out.
| FINDING | IMPLICATION AND ILLUSTRATION |
|---|---|
| 1. Reduced financial barrier to entry | Reflects university’s commitment to meeting needs for working students. |
| 2. Reduced technical barrier to entry | Reflects understanding of variable technical expertise levels among LIS students. |
| 3. Reduced interoperability issues | Reflects awareness regarding increasing use of mobile devices—the content works the same on mobile as on desktop. |
| 4. Ensures consistent user experience | Less prone to variation due to eBook/eReader configurations. |
| 5. Reduced technical support requests on instructor | Lightens the workload for the instructor so their focus can be student learning. |
Noteworthy is that the themes from the instructors’ findings align with the learning experiences reported by the students. For example, both students and instructors reported ease of use, including for the embedded links to videos and supplemental readings.
The researcher-author needed to make only minor changes to the book at the conclusion of Phase 4. The changes were fixing embedded URLs that had moved during the semester and adding more information to the how-to page for navigating a PDF (changing the view and highlighting).
Findings and Discussion
The findings from the study underscore the importance of conducting user research and rigorous peer reviews by subject matter experts when undertaking an OER project. By the time we completed Phase 4, with 630 students having used and evaluated the new book, there were only minor modifications required and, more importantly, the students and their instructors had reported highly positive experiences. The first three phases were the main reasons for this outcome. The other reasons were that the book was designed around UDL principles, grounded in the literature, and aligned with the course content based on threshold concepts and engagement with diverse social justice themes. This foundation underpinned the comments about the positive learning experiences, such as the supportive “pacing” and “thoughtful layout.”
Alignment between Findings and the Literature
The user research findings echoed a key theme from the literature examining trends in higher education and OER, namely that OER materials have a strong potential to positively impact student learning experience and learning outcomes. The clear preference among testers for the OER textbook over the former non-open textbook confirmed that OER materials can be of very high quality and can provide users with superior learning experiences. The positive user experiences that students reported align with the positive correlation found between the use of OER materials in higher education settings and increased student achievement (Bliss et al., 2013; Colvard et al., 2018; Feldstein et al., 2012; Hilton et al., 2016; LeMire, 2024; Read et al., 2020; SPARC, 2017; Weller et al., 2015).
Alignment with UDL Principles
In our study design and user research methodologies, we harnessed learner-centered and accessibility-first principles to create a textbook that was consistent with the principles of UDL, with particular relevance to the first five of the seven principles: equitable use, flexibility in use, simple and intuitive use, perceptible information, and tolerance for error. Table 5 shows specific features of the textbook that addressed these principles, offering an overview of the ways that an accessibility-first philosophy informed the design process.
Table 5
Overview of UDL principles (University of Washington, 2022) and specific textbook features that addressed them.
| PRINCIPLE | DEFINITION | TEXTBOOK FEATURES |
|---|---|---|
| equitable use | Design is useful and marketable to people with diverse abilities. |
|
| flexibility in use | Design accommodates a wide range of individual preferences and abilities. |
|
| simple and intuitive use | Use of the design is easy to understand, regardless of the user’s experience, knowledge, language skills, or current concentration level. |
|
| perceptible information | Design communicates necessary information effectively to the user, regardless of ambient conditions or the user’s sensory abilities. |
|
| tolerance for error | Design minimizes hazards and the adverse consequences of accidental or unintended actions. |
|
The Phase 1 alpha test data reflected the successful incorporation of these principles into the design, with multiple instances of participants citing the easy navigability, accessibility, and user-friendliness of the book. Participants’ perceptions of the book’s features and culturally relevant content signal its consistency with UDL best practices and multiple (non-Western) pedagogies, indicating that OER can be of excellent quality and meet the needs of a diverse range of learners.
Iterative Design: Cycles of Improvement Based on User Feedback
The positive findings from our study and the positive correlations between 1) OER materials and user experience and 2) OER and student achievement documented in previous studies, together provide a strong justification for the use of OER materials in higher education. In addition to ensuring the high quality of the OER textbook, ensuring responsiveness to user needs and equity of access for all students remained primary objectives. Our iterative approach, guided by user feedback, centered on the learner during the design process and supported development of a textbook that was responsive to the needs and preferences expressed by real users.
OER Adoption and Quality Assurance
The textbook was completed and launched in its current form in late July 2024. In addition to the 234 pages of narrative, the book contains 12 custom videos, 20 videos curated from other sources, 32 supplemental readings, and an extensive bibliography. To date, over 4,100 copies of the book have been downloaded free of charge. The adoption rate is 100% for our own students, far exceeding the 69.2% rate for the previous book.
The research and development that resulted in the OER textbook contributes to a growing body of literature identifying the many benefits of OER in higher education. However, OER materials do not have an inherent advantage over non-open materials in terms of quality. An important lesson learned from this project is that rigorous research and testing are central to ensuring OER materials are responsive to learning objectives and individual learner needs. The project facilitated quality assurance through four controls: a structured user research protocol, technology-aided PDF accessibility checks, domain expert peer reviews, and the involvement of a graduate research assistant to represent an MLIS student’s perspective throughout the design process.
The user research phases were instrumental in eliciting user needs and perspectives. In addition to confirming findings of previous studies, which demonstrate that most learners have favorable impressions of OER materials and can benefit financially and emotionally from the provision of cost-free higher education resources, the use of semi-structured interviews and questionnaires led to important improvements, ensuring the final textbook was responsive to student needs and high quality.
In addition to representing the perspectives of discipline experts in reviewing the textbook, the graduate research assistant working on the project provided an MLIS student’s perspective on the textbook, ensuring that a key stakeholder group was centered in the design process. By offering insights into the student experience—nontraditional students, in particular—she helped ensure the creation of a learner-centered OER. Her contributions to the project included:
Becoming familiar with WCAG 2.0 accessibility standards and verifying that the style guide complied with them;
Researching and recommending a variety of no-cost external accessibility tools for users that are compatible with the eBook file format;
Evaluating the content of the eBook and supplementary materials through a lens of learner-centered design and an MLIS student’s point of view;
Providing copyediting to ensure clarity and readability;
Conducting formal accessibility testing on the alpha edition via manual checking and using Adobe Acrobat’s accessibility checker;
Participating in all phases of user research from study design and questionnaire development to the delivery of semi-structured interviews to ensure adequate representation of the student perspective.
The provision of accessibility checks in particular was instrumental in ensuring the textbook met UDL standards. Accessibility checks were conducted in an iterative manner. We submitted the alpha edition to Adobe Acrobat’s internal accessibility checker and made manual changes based on the results documented in the accessibility report it generated. We repeated this process until the PDF was fully accessible.
By making these methodologies transparent and replicable and tracing their connections to an end product that has been shown to be highly usable and accessible, our project has demonstrated that OER materials can offer the twin advantages of high quality and affordability when adequate provisions are made. Representing the perspectives of both students and domain experts in the textbook design and centering accessibility-first principles from the beginning are powerful and reproducible approaches to quality assurance that allow for a greater integration of excellent OER materials in higher education.
Benefits for Educators of Well-designed OER
OER and OEP can be transformative for educators as well. Researchers exploring threshold concepts in student learning have further asserted that the repositioning of an instructor’s own thinking about open education is in itself transformative. As this study focused on a course designed around threshold concepts for the content (Tucker & Perucci, 2025), the threshold concepts theoretical framework and its curriculum principles underpinned the writing. Indeed, the framework transforms how a researcher-educator conducts their work. Tur et al. (2020) assert that applying ‘the threshold lens to the OE conversation will add value because it provides a theoretical framework within which to assess the impact of engaging in OEP on the identity formation of the open educator’ (p. 11). This meta-experience of new thinking about OER and OEP opens up new possibilities for educators and students alike (Atkinson & Fields, 2023; Adamiak, 2024).
Next Steps: Wider Distribution and Ongoing Evaluation
The author has received requests from educators at other universities to use the textbook in their information retrieval courses, and this has led to planning for wider distribution on OER platforms for the next edition of the book. This next step in the book’s development has already begun and involves guidance from OER librarians in other countries, including Australia and Indonesia. Specific parts of the book are being modified so that it is easier for instructors to localize the content and to reflect their students’ learning environment. As an example, all mentions of the local university library and degree requirements are being rewritten with an ecumenical positioning, allowing for straightforward adoption by other universities.
For ongoing evaluation, the Phase 4 questionnaire is being modified, eliminating questions that were somewhat redundant, and this version will be used to assess the textbook content each year. The modifications are also aimed at reducing the amount of time needed from respondents in order to increase participation. We are reducing the number of questions about the software tutorials and merging several questions about assignments. In order to support longitudinal analysis of the course materials and student experiences, however, most questions will be retained in their current form.
Conclusion and Implications
This research and the successful distribution of an OER textbook to over 4,000 students in its first year yielded lessons that may be summarized in three elements that support excellence in OER:
multiple stages of user research and testing with students,
peer reviews with subject matter experts (SMEs) to fully evaluate all subtopics, and
accessibility testing to ensure inclusivity.
Prioritizing all of these elements when planning for, developing, and writing an OER textbook, a researcher-educator must ensure that the user research will be iterative and that the experts involved in the peer reviews will be rigorous in their reviews, both for subject content and instructional voice. The testing for accessibility, which supports inclusivity, must follow local and, if appropriate, based on intended distribution, international guidelines.
Perhaps a most gratifying conclusion was finding out that all students acquired a copy of the book of their own, to have at hand and to keep. Our goal for 100% adoption rate was met. Writing a textbook for a required course, covering essential (threshold) concepts in information retrieval systems, we knew it was likewise essential that all students be able to own a copy of the book—and that it be the latest edition, in hand at the start of term, and in digital or print format (or both).
Textbooks are in some situations available on reserve at university libraries, often with short time limits, but demand can exceed supply, and the online access needed for distance learning and working students may be even further limited. As decried by Jenkins et al. (2020), this gap is a significant social justice issue that begs attention from educators. The conclusions from this study provide a way forward.
Implications for further study are to explore more deeply, and in more nuanced ways, how removing the obstacles of cost, access, and distribution affects the learning experiences of students. Two areas for further study are of primary interest. First, we would like to better understand the phenomenon that emerged in the later stages of the semester, that of students expressing ownership of their learning experiences and how OER and 100% adoption rates play a role in this. We are also curious to explore the intersections of UDL and threshold concepts in course design. Designing research methods for this purpose will build upon the literature covered here and use qualitative approaches such as semi-structured interviews with follow-up questionnaires, which reaped considerable insights during this study. Second, the theme of inclusivity, of how, by designing for each learner, we are designing for all learners is of significant interest and importance. We plan to expand our understanding of how OER textbooks can further contribute to this educational goal.
Limitations
The study was conducted with post-graduate students in a totally online learning environment, taking a course on information retrieval systems design. Therefore, the findings may have reduced relevance for developing an OER textbook in learning environments that are not totally online or in other topical domains. However, it is worth noting that, because the study focus was a post-graduate course taken during the students’ first or second term, the findings may well be relevant to higher-level undergraduate programs. Note that learner demographics are presented in the Introduction section, but that the study was not designed to address cultural or gender issues within the student population, but rather to address any accessibility issues that might arise in using the textbook and its accompanying materials (videos and supplemental readings).
Data Accessibility Statement
All data analyzed during this study are included in this published article. Surveys used during the current study are available from the corresponding author on reasonable request.
Ethics and Consent
An institutional ethics review was not applicable to the project because it did not collect individually identifiable private information and was not designed to contribute generalizable knowledge.
Acknowledgements
The authors gratefully acknowledge the student participants in the research, their candor in speaking about their experiences, and their willingness to contribute their time and insights.
Competing Interests
The authors have no competing interests to declare.
Author Contributions (CRediT)
Virginia M. Tucker: conceptualization, project administration, funding acquisition, methodology, formal analysis, investigation, supervision, visualization, writing-original draft, writing-review and editing.
Camille A. Charette: formal analysis, writing-original draft, writing-review and editing.
