Data from “A Registered Report Testing the Effect of Sleep on DRM False Memory: Greater Lure and Veridical Recall but Fewer Intrusions After Sleep”

1.1 Sleep and DRM false memory

Newly acquired declarative memories tend to be better remembered after a period of sleep compared to a similar duration of wakefulness (e.g., Ashton & Cairney, 2021; Mak et al., 2024; Plihal & Born, 1997; Potkin & Bunney, 2012). Some theories attribute this benefit to sleep-related consolidation, during which hippocampal replay reactivate the newly acquired memory, facilitating its assimilation into long-term neocortical stores (e.g., Klinzing et al., 2019; Paller et al., 2021; Stickgold, 2005). Alternatively, sleep may protect newly encoded memories from external interference, resulting in less forgetting (Jenkins & Dallenbach, 1924; Yonelinas et al., 2019).

In recent years, mounting evidence has suggested that the role of sleep extends beyond what was previously encoded (Horváth et al., 2016; Lewis & Durrant, 2011; Lutz et al., 2017). One strand of research in this area is how sleep impacts false memories, where individuals recall events or items that never actually happened (Calvillo et al., 2016; Darsaud et al., 2011; van Rijn et al., 2017). The Deese-Roediger-McDermott (DRM) paradigm is perhaps the most widely used paradigm for inducing false memories in the laboratory (Deese, 1959; Roediger & McDermott, 1995). Here, participants study lists of semantic associates (“studied list words” like nurse, hospital, sick), but not a “critical lure” that represents the gist of the list (e.g., doctor). In later memory tests (typically free recall or recognition), participants often mistakenly recall the critical lures or identify them as previously seen, despite not encountering them before. This DRM false memory effect has been widely studied and replicated across age groups (e.g., Sugrue & Hayne, 2006), speakers of different languages (e.g., Bialystok et al., 2020), presentation modalities (e.g., Cleary & Greene, 2002; Smith & Hunt, 1998), and various delay intervals between wordlist presentation and the subsequent memory test (e.g., from a few minutes to 60 days later; Seamon et al., 2002). Among these, a number of studies have examined whether a period of sleep, as opposed to wakefulness, during the delay interval affects the incidence of DRM false memory.

1.2 Theoretical significance

Understanding how sleep (vs. wake) may influence DRM false memory has substantial theoretical value, because vastly different predictions have been made by existing memory/sleep models. For example, the Information Overlap to Abstract model (iOtA; Lewis & Durrant, 2011) postulates that sleep plays a crucial role in gist abstraction, leading to the explicit prediction that sleep (vs. wake) would increase false memory for the critical lures (which represent the gist of each list). Conversely, an alternative proposition (e.g., Fenn et al., 2009; Lo et al., 2014) posits that sleep could potentially reduce DRM false memory. Specifically, it argues that sleep may facilitate the consolidation of studied list words, which may, in turn, aid the suppression of critical lures, potentially via some kind of source monitoring processes (e.g., Kensinger & Schacter, 1999).

Our study, in addition to disentangling those opposing theories, serves to refine existing theoretical frameworks, including Fuzzy-Trace Theory (Brainerd & Reyna, 1998) and the Activation/Monitoring Framework (Roediger et al., 2001). These two theories have dominated the DRM literature (see Chang & Brainerd, 2021 for a review), but they are not sleep-specific theories, and are therefore mute on the role of sleep. Therefore, understanding how sleep may influence DRM false memory will help tighten these theories.

1.3 Weak and contradictory evidence base

To-date, about 10 published studies have investigated whether a delay interval containing sleep (vs. wakefulness) affects the incidence of DRM false memories (e.g., Fenn et al., 2009; Payne et al., 2009; McKeon et al., 2012). Despite using similar experimental design and materials, these studies have yielded conflicting results: Some showed a post-sleep increase in DRM false memory (McKeon et al., 2012; Payne et al., 2009; Shaw & Monaghan, 2017), others reported no overall effect (Diekelmann et al., 2010), and some reported a reduction in false memories following sleep (Fenn et al., 2009; Lo et al., 2014). In short, the evidence base on how sleep may influence DRM false memory remains weak and inconsistent. In light of this, Newbury and Monaghan (2019) conducted a meta-analysis of 12 ‘Sleep × DRM’ experiments. They reported that while sleep did not have a consistent effect, the effect of sleep was moderated by a key factor: the number of semantic associates in a DRM list. Specifically, a consistent post-sleep increase in false memories was observed when each list contains fewer associates (N = 10 words/list), whereas no consistent sleep effect emerged when each list contained more (N = 12 or 15 words/list). Newbury and Monaghan’s (2019) meta-analysis pointed us towards the most conducive parameter for detecting a sleep effect in the DRM paradigm (i.e., short list length); however, the literature lacks a well-powered empirical study that tests the validity of this parameter. This is the backdrop against which our registered report takes a centre stage, where we used exclusively short DRM lists to investigate the effect of sleep.

1.4 Our registered report

We conducted a 2 (Interval) × 2 (Test Time) between-participant DRM experiment, which used free recall as the sole outcome measure. The first independent variable, Interval, refers to whether participants completed free recall either shortly or 12 hours after studying the DRM wordlists. The second independent variable, Test Time, refers to whether participants completed free recall either in the morning or in the evening. Our experimental design, therefore, resulted in four groups: Immediate-AM, Immediate-PM, Delay-AM (Sleep), and Delay-Wake (Wake). The inclusion of the Immediate groups helped rule out potential circadian effects on encoding/retrieval.

Our sample size was 488 young adults (after exclusion), giving us over 90% statistical power (alpha = 0.05) to detect our desired sleep effects. Our dataset, to the best of our knowledge, stands as the largest publicly accessible DRM free recall data. Regardless of whether researchers are interested in the effect of sleep, they may use our recall data for (1) modelling time-related memory decay and memory search processes, (2) exploring the relation between lexical properties and recall rates, (3) investigating how individual differences in e.g., veridical recall, contribute to false recall, and (4) using our data to benchmark memory studies conducted online.

2.1 Study design

Our experiment had a 2 (Interval: Immediate vs. Delay) × 2 (Test Time: AM vs. PM) fully between-participant design. Figure 1 summarises the experimental procedure.

Figure 1

2.2 Time of data collection

Data collection took place between 01 December 2022 and 02 April 2023.

2.3 Location of data collection

Participants were recruited online via Prolific (https://www.prolific.co/). All participants completed the experiment unsupervised and at a location of their own choosing. Access to the experiment was restricted solely to individuals possessing a British IP address, as we need to ensure participation at a certain time of day.

2.4 Sampling, sample and data collection

Following our prior sleep studies conducted via Prolific (Ball et al., 2024; Mak et al., 2023; 2024; Mak & Gaskell, 2023; Experiment 2), an ‘expression of interest’ survey (available in Appendix A) was administered on Qualtrics to recruit a pool of participants (N = 2296). The first part of the survey collected essential demographic details: gender identity, age, current country of residence, primary language, ethnicity, highest education level, and history of developmental/sleep disorders (if applicable). The second part of the survey outlined the main study, explaining that participants would be randomly allocated to one of the four groups and that no preferences would be accommodated. Participants indicated whether or not they would like to take part in the main study. Of the 2296 respondents, 1940 expressed interest in taking part, who were then screened for their eligibility:

1. Aged 18–25

2. Speaks English as (one of) their first language(s)

3. No known history of any psychiatric (e.g., schizophrenia), developmental (e.g., dyslexia) or sleep (e.g., insomnia) disorders

4. Currently resides in the UK, indexed by their IP address

5. Normal vision or corrected-to-normal vision

6. Normal hearing

7. Able to complete the study using a laptop or a desktop PC

8. Able to complete both the study and test phases

9. Has an approval rate of >96% on Prolific (This helps ensure that a participant tends to take online studies seriously).

Those who fitted the above inclusion criteria (N = 904) were randomly allocated to one of the four experimental groups. In the end, 534 participants completed both the study and test phases. All participants were reimbursed at a rate of £9.5/hr, but those in the Delay groups received a bonus of £0.2 after completion of both sessions. Of the 534 participants, 46 were excluded from our confirmatory analyses for meeting one or more of our pre-registered exclusion criteria:

• 1. (Sleep and Wake groups only) they reported consuming any alcoholic drinks between study and test. (N = 11)

• 2. (Sleep group only) they reported to have had fewer than 6 hours of overnight sleep prior to test or rated their sleep quality as poor or extremely poor. (N = 5)

• 3. (Wake group only) they reported to have had a nap between study and test. (N = 11)

• 4. they failed more than one of the four attention checks (i.e., 3 + 3 = 11) at study. (N = 6)

• 5. they failed to report more than one of the four digits in the attention check of free recall. (N = 12)

• 5. they submitted a blank response in free recall. (N = 1)

Our confirmatory analyses encompassed a total of 488 participants, with 124 in each of the Immediate groups, and 120 in each of the Delay groups (see Table 1 for group characteristics). Note that although 46 participants were excluded from our analyses, their recall and survey data are available in the published datasets.

2.5 Materials/Survey instruments

We made use of 20 DRM wordlists from Stadler et al. (1999), who normed a total 36 lists. The 20 we chose elicited the highest false recall rates in their study. We used the first 8 associates of each list, and as per the standard DRM paradigm, they were arranged in a descending order of associative strength to the critical lures (see Table 2). A participant studied all 20 lists, and therefore, a total of 160 list items.

We note that the original DRM lists by Stadler et al. (1999) were tailored for American participants, and two words (e.g., trash, Mississippi) were not immediately relatable to people in the UK. We, therefore, changed these words (e.g., trash → rubbish), as noted in Table 2.

2.6 Quality Control

To ensure the best possible data quality, we followed our prior online studies (e.g., Curtis et al., 2022; Mak, 2021) by incorporating several attention checks into the experiment, detailed below.

2.7 Positive controls

Given the online nature of our experiment, we first assessed whether our data exhibit anticipated behavioural effects. Specifically, we examined the presence of the DRM false memory effect, the serial position effect, and a potential sleep benefit in correct recall. This scrutiny is essential to establish the validity and robustness of our recall data.

2.7.2 Serial position effect

It is expected that the first (1^st) and last (8^th) words in our DRM wordlists would be better recalled than those in the middle due to primacy and recency effects, respectively (e.g., Ebbinghaus, 1885; Mak et al., 2021; Nipher, 1878; Tse & Altarriba, 2007). This was indeed the case in our data, as both the Immediate and Delay groups exhibited the well-established U-shaped serial position curve (see Figure 2). Then, I used a Wilcoxon Signed Ranked Test to compare each participant’s proportion of recall in the 1^st position against that in the middle positions (i.e., the average in the 4^th and 5^th positions), confirming a primacy effect (z = –17.09, p < .001). I also ran the same test comparing a participant’s recall proportion in the 8^th position against that in the middle positions. It confirms a recency effect (z = –4.87, p < .001).

Figure 2

2.7.3 Sleep benefit on studied word recall

Decades of evidence suggests that newly encoded declarative memories are better remembered after sleep than after an equivalent amount of wakefulness (e.g., Plihal & Born, 1997). It is, therefore, expected that the Sleep group would recall more studied list words than their Wake counterparts (e.g., Lahl et al. 2008). This was indeed the case [M_Sleep = 17.33 (SD = 16.59) vs. M_Wake = 13.41 (SD = 12.93); see Figure 3], and an independent t-test showed that this was statistically significant [t(237.63) = 2.20, p = .029].³

Figure 3

I also performed an additional independent t-test, where the dependent variable was “adjusted correct recall”, which subtracts the number of intrusions (i.e., neither the studied list words nor the lures) from the number of correct recalls (Diekelmann et al., 2010; Payne et al., 2009). In line with the previous t-test, it also revealed superior performance in the Sleep (vs. Wake) participants [M_Sleep = 10.55 (SD = 19.66) vs. M_Wake = 3.31 (SD = 17.31); t(234.26) = 3.03, p = .003]. In short, our data demonstrated a clear sleep (vs. wake) benefit in studied word recall.

In summary, our recall data unveiled the classic DRM false memory effect, the serial position curve, and a sleep benefit in studied word recall. These observations underscore the strength and dependability of our online-collected data, attesting to its solid foundation for meaningful analyses.

2.8 Main results

The data reported here were collected to shed light on the effect of sleep in the DRM paradigm. Our registered report (Mak, O’Hagan, Horner, & Gaskell, 2023) described the findings in detail, so I will only a provide a brief summary of the key findings here.

1. The sleep group produced significantly fewer intrusions than the wake group.

2. When intrusions were statistically controlled for, there was evidence for greater false recall of the critical lures after sleep (vs. wakefulness). However, when intrusions were not controlled for, this sleep-wake difference became non-significant.

3. Regardless of whether intrusions were controlled for, there was clear evidence of the sleep group outperforming the wake group in veridical recall of the studied list words.

4. There were significant Interval × Test Time interactions in all the above analyses, indicating that these findings are above and beyond any time-of-day effects.

2.9 Data anonymisation and ethical issues

This research received ethical clearance from the Department of Psychology Ethics Committee at the University of York. All participants provided informed consent before commencing the study. No personally identifiable data were collected unless participants in the delay groups voluntarily provided an email address to receive a reminder to take part in Session 2.

Each participant was associated with two distinct IDs: one was randomly generated by Gorilla (the platform in which we programmed the experiment), while the other was a participant-specific Prolific ID. While the former is untraceable to individuals, the latter holds some traceability. To maximise anonymity, Prolific IDs were removed from all the publicly shared datasets.

2.10 Existing use of data

Mak, M. H. C., O’Hagan, A., Horner, A. J., & Gaskell, M. G. (2023). A registered report testing the effect of sleep on Deese-Roediger-McDermott false memory: greater lure and veridical recall but fewer intrusions after sleep. Royal Society Open Science, 10(12). https://doi.org/10.1098/rsos.220595.

3.1 Repository location

The repository is located on Open Science Framework (doi: 10.17605/OSF.IO/P5JRH). URL Link to the repository: https://osf.io/p5jrh/.

3.2 Object/file name

See Table 3 for details.

3.3 Data type

See Table 3 above.

3.4 Format names and versions

All the data are available in comma-separated values (csv) format.

3.5 Language

British English

3.6 License

CC0.

3.7 Limits to sharing

There is no limit to sharing.

3.8 Publication date

All the datasets were published in the OSF repository on 24th/25th April 2024.

3.9 FAIR data/Codebook

1. Findability: To enhance findability, we have assigned descriptive metadata and tags to the recall data and repository.

2. Accessibility: We chose a reputable and widely recognised data repository (OSF) for hosting our datasets. Furthermore, we made our data available in csv to ensure compatibility with various data analysis tools.

3. Interoperability: To achieve interoperability, we have structured our dataset in a highly accessible and comprehensive way, enabling effective aggregation.

4. Reuse: We have taken proactive measures to facilitate dataset reuse. Comprehensive documentation, including a detailed README file, has been uploaded to OSF to help guide users through the recall data.

4.1 Strengths

First, our dataset encompasses data from 534 participants, making it a valuable resource for researchers seeking statistical power in their analyses. This large sample size increases the generalisability of findings and the likelihood of detecting subtle behavioural effects.

Second, we pre-registered our study, including the experimental procedures, exclusion criteria, pre-processing steps, and analysis plans. Our pre-registration underwent a thorough peer-review process to ensure methodological rigour and transparency, reinforcing the robust foundation upon which our study was conducted.

Third, unlike prior studies in the ‘Sleep × DRM’ literature, our sample was not restricted to undergraduate students. Instead, our diverse participant cohort encompassed a broader range of individuals, enhancing generalisability and ecological validity.

Finally, our dataset emerges as a pivotal benchmark in an era where online experiments are increasingly prominent in psychological research. Our study is poised to play a crucial role in shaping the trajectory of future memory/sleep studies conducted online. Researchers venturing into this area can look to our dataset as a reference point that embodies the standards to which they should aspire. This benchmark not only underscores the viability of online research in capturing complex cognitive processes but also encapsulates the meticulous considerations required to ensure methodological robustness.

4.2 Limitations

Our experiment was conducted online, potentially introducing variations in participant engagement, distractions, and environmental conditions compared to in-person settings. For example, online experiments had no control over participants’ immediate surroundings, potentially introducing uncontrolled variables such as noise and light that might influence behavioural performance. As a remedy, we asked participants to provide information on their surrounding environment in the test phase survey, but no formal analyses were conducted to investigate these factors. Notwithstanding, we still observed clear evidence for the classic DRM false memory effect, serial position effect, and a sleep benefit in word recall. These offer reassurance of the reliability of our data. In fact, administering the experiment online, while losing control over some extra-experimental variables, offers potential advantages in other respects. For example, it mirrors how most participants typically encode information on a daily basis. This bolsters the ecological validity of our study, enabling a closer approximation of how memory processes take place in real life.

Another key limitation of our registered report is that we recruited 18-to-25-year-olds only.⁴ Therefore, our data cannot inform false memory performance or sleep-related memory effects in different age groups (e.g., see Scullin & Bliwise, 2015 for a review).

Finally, while we randomly allocated participants to the four experimental groups, the sample might still have inherent biases due to self-selection. For example, participants with a morningness preference may be more likely to take part in the study if they were assigned to the Immediate-PM/Wake groups (e.g., Mak et al., 2023; Experiment 1). Fortunately, however, the four experimental groups were well-matched on sleepiness rating and morningness/eveningness preference (see Table 1), providing some reassurance that the effect of self-selection was minimal.

4.3 Potential Reuse Scenarios