Introduction
Effective source evaluation teaching strategies are an essential component of information literacy instruction. This project drew on three popular source evaluation methods: Blakeslee’s CRAAP test,1 Caulfield’s SIFT framework,2 and Bizup’s BEAM rhetorical device.3 While these source evaluation frameworks have individual benefits, each has their own strengths and weaknesses when applied to scientific literature.
In preparing to teach an upper-level chemistry seminar, the author drew on the aforementioned methods to craft a source evaluation heuristic that would serve as an instructional framework and provide students with a memorable mnemonic when navigating the research process. CAAMP-PADL stands for currency, author/authority, accuracy, material type, publisher, purpose, audience, data, and lateral reading. Rather than focusing only on lateral reading (SIFT), vertical reading (CRAAP), or identifying a document’s potential research value (BEAM), the framework encourages students to view sources holistically. The goal of the CAAMP-PADL framework is to introduce students to key scientific information resources and material types, to develop skills in evaluating research data, to read sources laterally, and then to use vertical reading to narrow down to a short list of best resources. Pedagogically, the aim is to weave in source evaluation as a common thread through all information literacy instruction sessions—ranging from database basics to critically evaluating scholarly sources.
This research study report’s intent is threefold: Firstly, to describe the development of the method and the teaching strategies used when initially implementing CAAMP-PADL in a semester-long chemistry course; secondly, to provide evidence of how students engaged with the method while completing a final literature review project; and finally, to consider avenues for updating the CAAMP-PADL method and for implementing a homegrown source evaluation framework within instructional settings.
Literature Review
With the advent of personal computers in the 1990s, students transitioned from searching in brick-and-mortar libraries to accessing a library’s holdings digitally. For student researchers, the regulated library environment gave way to the unfettered web.4 As librarians grappled with students’ lack of evaluation skills, source evaluation started to gain traction in information literacy circles.5 Librarians began to shoulder the task of not only helping students find and access information, but also teaching learners to critically evaluate a material’s trustworthiness.
Metzger affirms that teaching students to select quality resources hinges on the concepts of expertise and trustworthiness.6 In an attempt to help students select trustworthy resources, librarians have created a plethora of different mnemonics, tests, checklists, habits, and heuristics.7 Even though the current challenge is to evaluate dynamic digital sources, the field’s approach to teaching source evaluation frequently employes analog methods. Furthermore, as Auberry explores, ample ink has been spilled in conveying different metrics for evaluating sources with little to show about students embracing these methods in everyday use.8 While there is little consensus on the best way to teach source evaluation or how to foster student habits, there is agreement that teaching source evaluation skills is a critical, lifelong skill.9 A judicious review of the literature reveals many tools have been used to teach source evaluation with varying degrees of success.
Just over twenty years ago, Blakeslee introduced the now ubiquitous CRAAP (currency, relevance, authority, accuracy, and purpose) test.10 The CRAAP test is an outgrowth of the metrics that librarians use to select library resources, and the test employs a checklist of guiding questions. The CRAAP test asks students to employ vertical reading, the practice of reading a source from the top down. Students might evaluate the site’s URL, authorship, and publication date, among other factors. In and of themselves, these are not bad criteria; the CRAAP test does introduce students to important concepts that can help learners narrow down to the best resource.11 However, the CRAAP test finds allegory in the myth of Narcissus. No matter how deeply you investigate the resource, researchers can become entranced with the source itself—whether or not the claims offered hold any truth. The CRAAP test can lead students to unknowingly select unreputable resources.
Swiftly after the CRAAP test’s introduction, Metzger provided a robust critique of the method.12 Metzger posits that the test lacks the dynamism needed to tackle the challenges posed by the web. Similarly, Ostenson provides a thorough critique of checklist approaches generally, as these methods incorporate leading questions that can promote shallow thinking.13 Looking to how source evaluation plays out in the classroom, Mills et al. embrace the stance that instructors and librarians can more mindfully engage with students and source evaluation by recognizing that many learners have some previous experience with checklist-based approaches.14 Acknowledging these prior experiences, instructors can encourage students to build off their prior learning and to read texts using new skills, such as lateral reading—the process of reading and fact-checking across a variety of sources.
The Stanford History Education Group’s (SHEG) Wineburg and McGrew observed how students, historians, and professional fact checkers evaluated information sources. Their work revealed that historians and students were easily fooled by a website’s appearance or “about us” section (i.e. through vertical reading).15 Conversely, fact checkers spent minimal time on the website itself; rather, they opened a new tab to search for corroborating sources.16 Lateral reading provides needed elements of quality control in the research process, by leading readers to verify claims rather than accepting a website’s information unawares.
Since 2017, Caulfield has been a leading proponent of leaving behind checklist methods in favor of habit-based, lateral reading practices.17 Caulfield’s SIFT (stop, investigate the source, find better coverage, trace claims to the original context) method provides students with actionable steps to perform. The method relies on students looking across sources to verify information and tracing claims back to an original source. In subsequent work, Caulfield and Wineburg caution against the use of checklists, because they place emphasis on researchers positioning themselves in the context of the work (i.e. Narcissus’ pool).18 SIFT asks students to acknowledge their own worldview, potential for bias, and other tacit factors that might influence decision making.
Not all instructors feel confident in SIFT. As Faix and Daniels report, “The SIFT process is not linear, so students will need to figure out which step makes the most sense to start with, [and] students… may not need to use all components of SIFT for every source that they encounter.”19 And Russo et al. argue, “[F]act checking is not the ultimate solution for student source evaluation. Professional fact-checkers delve deep into content, pull the story apart, and put it back together, using tacit knowledge to execute moves which are not appropriate for student researchers.”20 Considering Russo et al. in context, SIFT and other fact-checking methods are appropriate for college students; however, these methodologies do not intend to turn students into expert fact checkers. As with any source evaluation metric, setting realistic student learning goals is essential. The SIFT method’s strength lies in using lateral reading techniques but SIFT heavily relies on students choosing other good sources or developing their own tacit research skills. In an attempted compromise, Carey developed the SIFT & PICK model, which incorporates both lateral and vertical reading.21
Outside of checklists and lateral reading, instructors have also embraced Bizup’s BEAM (background, evidence, argument, methods) rhetorical device.22 While not a source evaluation framework per se, BEAM encourages students to view potential sources through the lens of how the resource might benefit their research. For example, one source might provide good background information, while another provides a nuanced research methodology that offers keen insights into the topic. As shown by Jastram et al., students are often uncertain of what is being asked of them in research assignments.23 Students express that they do not know what kinds of sources would best meet their information need or how to incorporate different source types into their work. Jastram et al. and Buhler et al.’s work highlight that teaching source genres remains a critical component of information literacy and source evaluation on the whole, because it provides students with greater awareness of how to substantiate their writing.24 Teaching source genre equips students to disambiguate between digital resources that may appear similar and to evaluate information considering the material’s potential usefulness.
Overall, the literature remains undecided on the best methods for teaching source evaluation. This leaves instructors with the opportunity to create discipline-specific source evaluation methods that best meet their pedological goals. If instructors choose to create their own method, they are left with the challenge of evaluating their method’s effectiveness in the classroom. This interplay between using established methods, developing and implementing a homegrown method, and assessing the method’s use are all key components of this preliminary study.
Developing and Implementing the CAAMP-PADL Method
Course Context
James Madison University (JMU) is a university with a full-time enrollment of approximately 21,000 students. With a Carnegie classification of “R2,” JMU is a doctoral university with high research activity. JMU’s College of Science and Math (CSM) is comprised of 5 departments. As a Science & Math Librarian, the author provides services to the Chemistry & Biochemistry department. CHEM 481: Literature and Seminar is taught by a chemistry faculty member and co-taught by an embedded librarian. The course is for juniors and seniors, and enrollment typically hovers around 30 students each fall semester. CHEM 481 is a 1-credit seminar-style course designed to help students build research skills. As required by the American Chemical Society (ACS) Guidelines for Undergraduate Chemistry Programs, students should have skills in retrieving, evaluating, and analyzing sources from the scientific literature.25 Students participate in five to six information literacy sessions spread across the sixteen-week semester, and even when formal library instruction is not provided, the librarian often attends class to help students navigate the databases needed for in-class assignments.
Crafting a Science Source Evaluation Framework
In the fall of 2023, the author started working within the CHEM 481 course. The librarian had several planning meetings with the course’s chemistry faculty member before the start of term. These conversations centered around the need to have students use databases effectively, find high quality literature sources, and to dive deeply into the facts and figures (i.e. data) presented in the chemical literature. Additionally, in revamping the course’s LibGuide, the author saw common threads developing. Students needed to learn the hallmarks of good sources while simultaneously learning other skills such as lateral reading, genre identification, and engaging with data sources.
With a penchant for preparation, the author crafted all the librarian-taught lecture materials and activities before the start of the semester—to create meaningful sessions that were in pedagogical concert with one another. Drawing on conversations with the course instructor, investigating legacy materials from previous librarians, and overhauling the course’s LibGuide were all fundamental in gaining an awareness of the course’s learning goals. It was in the preparation stages of this course that CAAMP-PADL was developed. Source evaluation is one of the first sessions taught by the librarian, which lays the groundwork for subsequent sessions on using databases and consulting the chemical literature. Preparing the first source evaluation session sparked the idea of creating a new source evaluation method specifically attuned to chemistry students’ needs. With course outcomes in mind, the author then considered different source evaluation frameworks and their potential helpfulness in evaluating scientific sources. The author saw that various source evaluation methods met specific learning needs, but no one method reasonably covered all the desired learning outcomes. This became the launch point for creating a homegrown method that would serve as a pedagogical path to follow throughout the course.
In pursuit of creating a holistic source evaluation narrative, the CRAAP method was first considered, as it helps students to investigate source-specific details; the components of CRAAP were retained and rearranged. As explored previously, these criteria are best taught alongside other source evaluation skills. Moving to the theoretical underpinnings of SIFT, “L” was added for “lateral reading.” As students familiarize themselves with the chemical literature, it is important for students to learn how to corroborate their findings or to seek additional resources to clarify their understanding of nuanced topics. After seeing preliminary student work and turning to BEAM, there was an apparent need to delineate source genre more clearly— a trade journal can look remarkably like a scholarly website— so “M” was added for “material type.” The chemical literature is rooted in providing well-substantiated experimental data, so “D” was added for “data.” Specifically, the class has several “facts and figures” assignments that ask students to report out on the data, tables, and images provided in scientific articles. Each of the leading letters were rearranged to form CAAMP-PADL. The CAAMP-PADL Source Evaluation guide was created to support in-course instruction and student learning. The guide served as recommended reading for students, and it provided more information than could be conveyed in class, should students need additional support or differentiated instruction.
Teaching Strategies Employed Using CAAMP-PADL
Source evaluation was introduced during the third week of class, which aligned with the librarian’s second lecture. In the first librarian-taught session, students received instruction on using databases and keywords, so the second class began by introducing primary, secondary, and tertiary sources. Because these traditional categories and their associated genre are unfamiliar to many students, this discussion was grounded in tangible examples of varying source types.26 After discussing discipline specific genera, the CAAMP-PADL method was introduced. Students received instruction on each component of the framework, and students also received a CAAMP-PADL handout with guiding information.
For their first introduction to CAAMP-PADL, students were asked to walk through each of its components, to learn more about the different criteria used in the source evaluation process. This was achieved by having students complete an in-class activity. Students were divided into groups and then tasked with evaluating an instructor-supplied resource, such as a blog, magazine, website, journal article, or something else. Students walked through the CAAMP-PADL mnemonic, and then provided a “stoplight signal” for their resource—red, yellow, or green. Red equated to “do not use,” and green provided a “good to go” signal, with yellow meaning “use with caution.” Each group then presented their resource’s information and stoplight signal to the class. Rather than discussing each of the nine components abstractly, having students engage with both the method and current scientific resources better aligned with the course’s active learning goals.
Now that students had been exposed to the entire method, CAAMP-PADL then became a common thread through subsequent classes and activities; CAAMP-PADL was used to contextualize literature searches in relationship to the documents used in chemical research. For example, students discussed the strengths of lateral reading when paired with analysis of an article’s currency and data reporting, or the benefits of opening a separate tab to investigate an author or publishing company. When discussing authority, the class used Scopus’ author and citation tracking features to learn more about an author’s publication history. Students compared sources laterally, and then leaned into vertical reading strategies to narrow down the best resources. Students directed their own learning by deciding which components were most relevant to the specific question that they were asking, but they still received instruction on various factors to consider when choosing resources—a balance between building tacit skills and directed learning. Leading up to the final literature review project, several assignments asked students to reflect on their article choices in light of the method. Considering the “PubMed search activity,” students were asked to find an article that supported their research question, and then to discuss the three most helpful CAAMP-PADL criteria when evaluating their chosen resource. The author used CAAMP-PADL throughout the semester to pedagogically guide conversations about the research process.
Assessment
Study Purpose
The purpose of this study is to understand how students enrolled in the fall 2023 cohort of CHEM 481 interacted with the CAAMP-PADL method. The research questions used to inform the study’s goals are:
How did students engage with each component of the method during their final literature review project?
In light of students’ engagement with CAAMP-PADL, how can the method be updated to better meet student and instructor goals?
Survey methods
In mid-November 2023, the CAAMP-PADL research project was introduced to the class. This timing was strategic, as students were about to begin the course’s literature review capstone project. Ideally, the pre-survey would have been administered at the start of the semester, but because the 1-credit class front-loads important course concepts at the start, and because this study did not come to fruition until late September, the pre-survey was not introduced until later in the semester. The pre- and post-literature review surveys included in the research study were mandatory for course credit, but students had the option to complete an additional consent form, thus allowing their responses to be collected for research.
The pre-literature review survey was administered in mid-November, just prior to students beginning their literature review projects. The goal of the pre-literature review survey was to garner information about students’ source evaluation instruction experiences prior to taking CHEM 481. This six-question survey was provided through the CHEM 481 Canvas course during a ninety-minute class session and before starting their literature review assignments. After completing the final project, students then took a five-question post-survey in Qualtrics. Questions included in the post-survey related specifically to students’ use and engagement with CAAMP-PADL during their final project.
In spring 2024, student responses were cleaned and organized for analysis. Of the thirty students enrolled in the course, twenty-five students indicated their consent, one student directly withheld their consent, and four students did not complete the consent waiver. Open-ended student responses were uploaded to the qualitative analysis program NVivo. The author single-coded the data and used grounded theory to categorize emergent themes. The coding sought to identify common codes related to students’ engagement with the framework; this coding process revealed 61 individual codes spread across four open-ended survey questions. These qualitative responses and their associated analysis are not presented in this paper. Quantitative responses were analyzed using the SPSS software.
Assessment Results
Pre-Literature Review Survey Results
The pre-literature review survey gauged students’ prior exposure to source evaluation techniques before taking CHEM 481. In response to the question “Have you ever received source evaluation instruction in another college level class?,” 54% (n = 13 of 24) of respondents indicated that they had received previous instruction. Students then responded to a multiple-choice question that asked them to identify the source evaluation method that they were taught, if they had received prior instruction. Options included CRAAP, SIFT, PICK, or “something else.” CRAAP was the leading method, with 69% (n = 9 of 13) indicating prior exposure to the test. Subsequently, all 24 students were asked to indicate their source evaluation habits. Responses include 8.3% (n = 2) selecting “evaluate all sources;” 58.3% (n = 14) “evaluate most sources;” 33% (n = 8) “evaluate some sources.”
Taken as a group, the pre-survey data revealed that just over half of respondents had prior exposure to source evaluation, and they were acting on the need to investigate their sources at least “most of the time.” Because the course is composed of juniors and seniors, the likelihood of students having received prior instruction is high, as the response data confirms. The challenge for CAAMP-PADL then becomes whether the method helps students meet course learning objectives.
Post-Literature Review Survey Results
The post-literature review survey explored how students used the CAAMP-PADL method during their final project. The post-survey received 25 student responses. For reference, students were asked to indicate the average number of sources they evaluated when finding the 10 articles required for the final project. The median was 41% of students (n = 10), who indicated that they evaluated most sources (6 to 8 articles). On either side of the median, 21% (n = 5) of students indicated that they evaluated some sources (5 articles), and an additional 21% (n = 5) stated that they evaluated all sources (10 articles). The remaining 17% indicated that they only evaluated a few sources (2 to 4 articles), with zero students indicating that they did not evaluate any sources (0 articles).
Students were then presented with a radio button matrix, where they indicated how frequently they used each of the criteria in CAAMP-PADL when evaluating sources. Students had these options: never (0 sources); sometimes (2–4 sources); generally (5 sources); frequently (6–8 sources); always (10 sources). To calculate the mean and standard deviation, “never” was assigned 0 and “always” was assigned 4. To conceptualize overall student use, student responses were evaluated as a matrix, where each of CAAMP-PADL’s 9 criteria were weighted against the frequency of use, from 0 to 4. Potential matrix scores range from 0 to 36. Analysis shows that the minimum matrix student score is 9 and the maximum score is 32. The mean matrix score is 23.83, with a standard deviation of 7.53. This indicates that on average, students were typically using components of CAAMP-PADL at least generally or frequently.
Responses to the matrix were further refined to provide a breakdown of how frequently students consulted each component of the framework, as provided in Table 1. Data ranked the highest, with 54.2% (n = 13) of respondents indicating that they always look at data when evaluating a source. Other high-ranking categories include currency (50%, n = 12) and material type (47.8%, n = 11).
Table 1
Frequency of consultation for each component of the CAAMP-PADL framework.
| NEVER (0) | SOMETIMES (1) | GENERALLY (2) | FREQUENTLY (3) | ALWAYS (4) | µ (σ) | |
|---|---|---|---|---|---|---|
| Currency | 1, 4.2% | 2, 8.3% | 4, 16.7% | 5, 20.8% | 12, 50% | 3.04 (1.20) |
| Authority/Author | 1, 4.2% | 8, 33.3% | 5, 20.8% | 4, 16.7% | 6, 25.0% | 2.25 (1.30) |
| Accuracy | 0, 0% | 3, 12.5% | 4, 16.7% | 9, 37.5% | 8, 33.3% | 2.92 (1.02) |
| Material Type | 1, 4.3% | 2, 8.7% | 3, 13% | 6, 26.1% | 11, 47.8% | 3.04 (1.12) |
| Publisher | 2, 8.3% | 7, 29.2% | 3, 12.5% | 9, 37.5% | 3, 12.5% | 2.17 (1.24) |
| Purpose | 1, 4.2% | 3, 12.5% | 4, 16.7% | 7, 29.2% | 9, 37.5% | 2.83 (1.20) |
| Audience | 2, 8.3% | 6, 25% | 6, 25% | 6, 25% | 4, 16.7% | 2.17 (1.24) |
| Data | 1, 4.2% | 1, 4.2% | 3, 12.5% | 6, 25% | 13, 54.2% | 3.21 (1.10) |
| Lateral Reading | 2, 8.3% | 4, 16.7% | 8, 33.3% | 6, 25% | 4, 16.7% | 2.25 (1.19) |
Students were then asked to indicate which framework components they more strongly considered when choosing materials. In other words, they were asked which factors would have served as a deciding factor if they had “weighed” the value of each criterion. This question was answered by using slider bars ranked from 0 (not important) to 5 (very important), as indicated in Table 2. Note that no respondents selected “not important,” so this column was omitted from the table. Like the results above, data (µ = 4.36) ranked as the most important characteristic that students considered for their research, followed by accuracy (µ = 4.26), currency (µ = 4.01), and material type (µ = 3.74).
Table 2
Factors more strongly considered when selecting resources.
| 1 | 2 | 3 | 4 | 5 | µ (σ) | |
|---|---|---|---|---|---|---|
| Currency | 0, 0% | 1, 4.3% | 7, 29.2% | 4, 16.7% | 11, 45.8% | 4.01 (0.99) |
| Authority/Author | 2, 8.3% | 3, 12.5% | 7, 29.2% | 7, 29.2% | 5, 20.8% | 3.41 (1.21) |
| Accuracy | 1, 4.2% | 1, 4.2% | 1, 4.2% | 8, 33.3% | 13, 54.2% | 4.26 (1.04) |
| Material Type | 0, 0% | 2, 8.7% | 7, 30.4% | 9, 39.1% | 5, 21.7% | 3.74 (0.92) |
| Publisher | 2, 8.7% | 4, 17.4% | 9, 39.1% | 5, 21.7 % | 3, 13.0% | 3.13 (1.14) |
| Purpose | 0, 0% | 5, 20.8% | 3, 12.5% | 10, 41.7% | 6, 25.0% | 3.71 (1.08) |
| Audience | 4, 16.7% | 7, 29.2% | 6, 25.0% | 6, 25.0% | 1, 4.2% | 2.71 (1.20) |
| Data | 0, 0% | 2, 8.3% | 1, 4.2% | 7, 29.2% | 14, 58.3% | 4.36 (0.92) |
| Lateral Reading | 3, 12.5% | 3, 12.5% | 11, 45.8% | 3, 12.5% | 4, 16.7% | 3.10 (1.00) |
To summarize the quantitative results, Table 3 compares the mean and standard deviation between the two previous survey questions. Analyzing the data side by side offers insights into whether there is any correlation between frequency of use and how strongly students consider the component during the evaluation process.
Table 3
Comparison between “frequency of use” and “component consideration.”
| CAAMP-PADL CRITERION | FREQUENCY OF USE µ (σ) | COMPONENT CONSIDERATION µ (σ) |
|---|---|---|
| Currency | 3.04 (1.20) | 4.1 (0.99) |
| Author/Author | 2.25 (1.30) | 3.4 (1.21) |
| Accuracy | 2.92 (1.02) | 4.3 (1.04) |
| Material Type | 3.04 (1.19) | 3.7 (0.91) |
| Publisher | 2.17 (1.24) | 3.1 (1.14) |
| Purpose | 2.83 (1.20) | 3.7 (1.08) |
| Audience | 2.17 (1.24) | 2.7 (1.16) |
| Data | 3.21 (1.10) | 4.36 (0.92) |
| Lateral Reading | 2.25 (1.19) | 3.1 (1.21) |
Across both questions, data is the component that students use and consider most frequently. Frequency of use, material type, and currency received identical means and standard deviations. For component consideration, accuracy and currency were the other highest-ranking criteria. While currency and accuracy have analogs in the CRAAP test, it is encouraging to see students using material type more frequently than other categories.
The survey concluded by asking students how they might use CAAMP-PADL moving forward. Overall, 92% (n = 23) responded that they would use portions of the framework in the future. The remaining 8% (n = 2) indicated that either they do not like source evaluation or that they found the framework time consuming.
Discussion
Overall, the pre-assignment survey shows that students came to CHEM 481 with some source evaluation experience, especially regarding the CRAAP test. The post-survey responses provide various insights. Firstly, quantitative analysis from the multiple-choice question and matrix agree that students are evaluating “most sources” at least generally (~5 of 10 sources) or frequently (~6–8 of 10 sources). To this author, the goal of teaching a classroom of students to evaluate “all of their sources, all of the time” seems lofty and perhaps unattainable, whereas the goal of teaching students to be more source-aware seems to be a more realistic and sustainable model for teaching and learning.
Secondly, CAAMP-PADL encouraged students to look at documents in new ways, with behaviors exhibited such as emphasizing data more strongly, identifying specific material types, and more deeply considering currency. Data and material types are not common criteria in other source evaluation schema, yet the survey responses show that students used these facets to help in their decision making. Chemistry students, and science students more broadly, are vested in choosing documents that have supporting data. Within CHEM 481, there is an emphasis on collecting, organizing, and communicating data, so it is promising to see that students began, after using CAAMP-PADL, to engage with data in more concrete ways. The chemical literature spans across many material types, including resources such as patents or genetic sequencing banks. Being able to identify a resource is a necessary first step in selecting meaningful materials. These survey outcomes directly mirror the course’s major learning goals, which supports the use of CAAMP-PADL as a pedagogical tool aimed at helping students engage with research sources.
Furthermore, it was helpful as an instructor to see which categories students consulted more frequently or weighed more heavily when using CAAMP-PADL during the evaluation process. Knowing that students are using data, currency, material type, and accuracy most frequently supports the teaching and learning exercises used in class, but conversely, it also shows the areas that need additional support, such as discussing publishers or source audiences. While the survey results provide prospective indicators that students can engage with the CAAMP-PADL method at various levels, this study’s conclusions point to a need for developing course materials that highlight the use of the lesser used criteria.
These conclusions point to using CAAMP-PADL in multiple ways. Teachers can use it to scaffold their lessons into a cohesive narrative, or it can serve as a talking point for students as they learn the vocabulary and skills surrounding literature searching. The CAAMP-PADL method is unique in that it brings together traditional checklists, lateral reading techniques, and rhetorical analysis to provide instructors with a robust means of teaching students about scholarly writing and evaluation. Students can also use the method to inform their own source selection process.
Limitations
As previously discussed, this research project was not introduced until November. This meant that students did not complete a pre-survey before the start of formal instruction (i.e. in August). Rather, in mid-November, students completed a pre-literature review survey before beginning their final assignments. The two surveys employed offer insights into how students used the method, but a more in-depth analysis could have been completed if students had taken an initial intake questionnaire at the start of term. Additionally, convenience sampling was used for this study. While other science courses received CAAMP-PADL instruction, CHEM 481 was selected because students worked with the method over the semester. Considering future work, if the method were evaluated for its efficacy, a validated pre- and post-course instruction survey would be necessary.
Conclusions and Recommendations
Because of the copious research related to source evaluation, librarians should feel equipped to create their own systems of source evaluation that align with their specific teaching and learning goals. In the author’s instructional context, students needed to learn more about material types, data, and lateral reading, so these core concepts were incorporated. In a spirit of creativity and transferable knowledge, librarians should consider the following when creating a homegrown source evaluation framework:
Check in with the course instructor before using your own method. Some instructors are only open to CRAAP, SIFT, or something else.
Take reflective notes when first trialing your method or framework. For example, the “M” for “material type” was originally called “medium.” After realizing that this was confusing to students, the wording was updated.
Explore different platforms for teaching source evaluation. Some students liked having a handout in class, while others preferred turning to the CAAMP-PADL LibGuide. Multimodal teaching and differentiated materials are helpful in supporting inclusive learning.
Provide students with several points of engagement with the method. Within a one-shot session, refer to the method by incorporating source evaluation throughout the lesson, or design an activity that asks students to directly engage with the method. For semester-long courses, reinforce prior teaching to foster students’ long-term memory and recall.
For librarians seeking to adapt CAAMP-PADL or to create their own homegrown source evaluation method outside of the sciences, reflect on the unique traits of your discipline and your specific learning goals. If there are specific literature search skills needed within your discipline, then these objectives are the best place to start.
Like other evaluation methods, CAAMP-PADL employs a memorable mnemonic, which proved to be a helpful teaching tool. Anecdotally, students would bring up CAAMP-PADL in conversing about their work, and CAAMP-PADL was a frequent point of discussion by students in end-of-semester course surveys. Designing and implementing a homegrown source evaluation method was a challenge, but it also came with the reward of cultivating a discipline-specific information literacy and source evaluation scaffold. From an instructional design standpoint, the acronym provides ample opportunities for creating camping-themed materials, which may make teaching and learning more pleasant for everyone involved.
Notes
[2] Mike Caulfield, “Introducing SIFT, a Four Moves Acronym,” Hapgood (blog), May 12, 2019, https://hapgood.us/2019/05/12/sift-and-a-check-please-preview/.
[3] Joseph Bizup, “BEAM: A Rhetorical Vocabulary for Teaching Research-Based Writing,” Rhetoric Review 27, no. 1 (January 3, 2008): 72–86, doi: 10.1080/07350190701738858.
[4] Nora J. Bird, Claire R. McInerney, and Stewart Mohr, “Source Evaluation and Information Literacy,” Communications in Information Literacy 4, no. 2 (September 2010): 171; Alyssa Russo, Amy Jankowski, Stephanie Beene, and Lori Townsend. “Strategic Source Evaluation: Addressing the Container Conundrum.” Reference Services Review 47, no. 3 (2019): 294–313, doi: 10.1108/RSR-04-2019-0024.
[5] Jonathan Ostenson, “Reconsidering the Checklist in Teaching Internet Source Evaluation,” Portal: Libraries and the Academy 14, no. 1 (2014): 34, doi: 10.1353/pla.2013.0045.
[6] Miriam J. Metzger, “Making Sense of Credibility on the Web: Models for Evaluating Online Information and Recommendations for Future Research,” Journal of the American Society for Information Science and Technology 58, no. 13 (2007): 2078, doi: 10.1002/asi.20672.
[8] Kendra Auberry, “Increasing Students’ Ability to Identify Fake News through Information Literacy Education and Content Management Systems,” Reference Librarian 59, no. 4 (October 2018): 182–83, doi: 10.1080/02763877.2018.1489935.
[9] Blakeslee, “The CRAAP Test,” 6; Ryan McGeough and C. Kyle Rudick, “‘It Was at the Library; Therefore It Must Be Credible’: Mapping Patterns of Undergraduate Heuristic Decision-Making,” Communication Education 67, no. 2 (April 3, 2018): 165, doi: 10.1080/03634523.2017.1409899; John McManus, “Don’t Be Fooled: Use the SMELL Test To Separate Fact from Fiction Online,” MediaShift, February 7, 2013, http://mediashift.org/2013/02/dont-be-fooled-use-the-smell-test-to-separate-fact-from-fiction-online038/.
[11] Jennifer A. Fielding, “Rethinking CRAAP: Getting Students Thinking like Fact-Checkers in Evaluating Web Sources,” College & Research Libraries News 80, no. 11 (December 2019): 262, doi: 10.5860/crln.80.11.620; Anthony Bernard Tardiff, “Have a CCOW: A CRAAP Alternative for the Internet Age,” Journal of Information Literacy 16, no. 1 (June 2022): 122, doi: 10.11645/16.1.3092.
[14] Jenny Mills et al., “Beyond the Checklist Approach: A Librarian-Faculty Collaboration to Teach the BEAM Method of Source Evaluation,” Communications in Information Literacy 15, no. 1 (March 2021): 130, doi: 10.15760/comminfolit.2021.15.1.7.
[15] Sam Wineburg and Sarah McGrew, “Lateral Reading: Reading Less and Learning More When Evaluating Digital Information,” SSRN Scholarly Paper (Rochester, NY, October 6, 2017), doi: 10.2139/ssrn.3048994.
[16] Joel Breakstone et al., “Why We Need a New Approach to Teaching Digital Literacy,” Phi Delta Kappan 99, no. 6 (March 1, 2018): 27, doi: 10.1177/0031721718762419; Wineburg and McGrew, “Lateral Reading,” 1.
[18] Mike Caulfield and Samuel S. Wineburg, Verified: How to Think Straight, Get Duped Less, and Make Better Decisions about What to Believe Online (Chicago: The University of Chicago Press, 2023), 206.
[19] Allison Faix and Tristan Daniels, “Teaching SIFT for Source Evaluation in Asynchronous One-Credit Information Literacy Courses,” Portal: Libraries and the Academy 23, no. 3 (2023): 405.
[21] Ellen Carey, “Research Guides: SIFT & PICK Fact Checking & Source Evaluation: Home,” 2023, https://libguides.sbcc.edu/siftpick/home.
[23] Iris Jastram, Claudia Peterson, and Emily Scharf, “Source Evaluation: Supporting Undergraduate Student Research Development,” In the Library with the Lead Pipe, October 13, 2021, https://www.inthelibrarywiththeleadpipe.org/2021/source-evaluation/.
[24] Ibid.; Amy G. Buhler et al., “Container Collapse and the Information Remix: Students’ Evaluations of Scientific Research Recast in Scholarly vs. Popular Sources,” Working Paper, 2019, https://alair.ala.org/handle/11213/17634.
[25] American Chemical Society. “2023 ACS Guidelines for Undergraduate Chemistry Programs,” March 12, 2024. https://drive.google.com/file/d/10HMPsGv9yHIWX_pKdomzHkAXFOK34POL/view?usp=sharing&usp=embed_facebook.
Ethics and Consent
This study was approved by James Madison University’s Institutional Review Board, protocol #24-4479.
Acknowledgements
This study only came to fruition with the support of Dr. Barbara Reisner and the fall 2023 cohort of CHEM 481 students. I am grateful to my colleague Liana Bayne-Lin for her feedback on the method and for using CAAMP-PADL in her own teaching. I would also like to thank Kelly Giles and Lara Sapp for their manuscript feedback. Finally, I would like to thank Dr. Dayna Henry for her data analysis and SPSS guidance.
Competing Interests
The author has no competing interests to declare.