Picturing Stories: A Citizen-Science Annotated Dataset of Children’s Book Illustrations

Context

Illustrated books for children have played a fundamental role in children’s learning and narrative absorption for over three centuries. As complex multimodal artifacts, they transmit cultural values through both their textual and visual design (Grenby & Immel, 2009; Kümmerling-Meibauer, 2023; Nikolajeva, 2002; Nodelman, 1988). These materials have long been studied across literary studies, education, childhood studies, book history, and psychology (Kümmerling-Meibauer & Meibauer, 2017; Montag et al., 2015; Wang & Shao, 2025). However, existing studies of children’s literature have been notably limited by their small-scale investigations of individual works, especially with respect to visual content (Stover, 2023).

Scaling quantitative visual research requires abstracting visual features into analytical units (Arnold & Tilton, 2024b; Smits & Wevers, 2023). Historical book illustrations also pose unique challenges for visual understanding (Mak, 2011). Unlike paintings and photography, book illustrations derive meaning from surrounding text and often rely on sparse visual cues. Historical examples create further challenges through low resolution, monochrome reproduction, period-specific styles, and degradation (e.g., foxing and bleed through). Because such annotation is difficult to automate yet too time-consuming for small expert teams, our citizen-science approach produces large-scale, human-grounded data while preserving auditable patterns of consensus and disagreement.

To support these aims, we introduce a new dataset of 44,043 manually annotated illustrations from historical children’s books created through a large-scale citizen science project called “Picturing Children’s Stories” hosted on Zooniverse.org (Figure 1). The images are drawn from the Internet Archive’s Children’s Library (IA),¹ which contains 2,506 digitized books published between 1731 and 1930 largely in English and the International Children’s Digital Library (ICDL),² which contains 882 books published between 1954 and 2011 in 35 languages (Figures 2 and 3). The project was live for a total of 42 weeks, during which time 10,647 volunteers contributed over 1.3 million total annotations (see Table 1 for a full overview of project statistics). The resulting dataset provides structured labels and free-text captions for a large collection of children’s book illustrations alongside the ability to understand ambiguity and disagreement in collective annotations.

Figure 1

Figure 2

Figure 3

Drawing on recent work on narrative theory for large-scale text analysis (Piper et al., 2021), our annotation framework was designed to capture the fundamental components of visual narrative (who, what, where, and how): who (co-)appears in an image, where the scene takes place, what material objects structure the scene, what actions characters undertake, how emotionally intense the moment appears to be (i.e., emotional arousal), and the identification of social units such as families and related social actions around caregiving, teaching, and play (see Table 2 for a list of all nine separate tasks). Our annotation framework is thus grounded in an actantial theory of narrative representation that focuses on agents undertaking actions in a social context imbued with emotional valence (Greimas, 1984). Our framework also aligns with existing annotation practices for cultural heritage imagery, which often focus on figures, objects, settings, and poses, while extending them toward more explicitly narrative and childhood-centered questions (Garcia & Vogiatzis, 2018; Reshetnikov et al., 2022; Stefanini et al., 2019).

By combining elementary narrative annotations alongside natural-language captions, Picturing Stories is intended to serve three related purposes. First, it provides a new empirical resource for studying the visual culture of childhood across a broad historical span, including depictions of gender, family, agency, emotion, setting, and material culture. Second, it offers a benchmark for evaluating the ability of multimodal large language models (MLLMs) to interpret historical illustrations in ways that approximate human judgment. Finally, this project helps demonstrate the value of citizen science as a scalable method for producing interpretable, publicly grounded annotations for cultural heritage collections.

Steps

To prepare our data for annotation, we downloaded all books available from the IA and those that were published after 1949 from the ICDL. Because of the larger size of the IA data, we performed a first filtering pass using PyMuPDF to analyze gray-scaled page images and identify potential illustration-containing pages, resulting in a total of 31,040 pages for further annotation. For ICDL, we kept all pages. Over 96.6% of IA books are English-language books, with the next most represented language being French at 2.7% and German at 0.6%. Based on an analysis of available publishing information, we estimate that Anglophone IA books originate equally from Britain (49.2%) and North America (46.4%). For ICDL, we find that two thirds of the illustrations are in four languages, with the remaining third spanning 31 further languages, provided in the supplementary material (Figure 3).

For the manual annotation phase, we utilized the platform Zooniverse.org, which is the largest citizen science platform in the world. Our project was launched in May 2025 and lasted for a total of 42 weeks. We utilized a series of dedicated “workflows” on Zooniverse to organize and assist volunteers in their annotation tasks (Table 2, Figure 4). For all tasks, we required three annotators per illustration before it was retired.

Figure 4

Quality control

We undertake a number of quality control assessments in our project. First, Zooniverse projects contain About pages, tutorials, field guides, and talk pages where participants can gain more information about specific tasks and the larger research questions being asked. We had four undergraduate student moderators participating for the duration of the project to answer volunteer questions in the talk pages.

Second, we report inter-annotator agreement separately for the two collections (Table 3), giving users insight into the subjectivity of each task. Reliability varies with task complexity and with the material itself. Illustration, setting, and character detection exhibited the highest agreement, while emotional intensity, object detection, family behavior, and character actions showed lower agreement.

Third, we release consensus labels non-destructively. Consensus is taken by majority — by mean for the ordinal intensity rating — over illustrations with at least three annotators. Rather than collapse disagreement, each row records the degree of agreement (unanimous, majority, or none) together with the complete per-label vote counts, and two further readings of each answer set: the union of all labels any annotator chose, and those corroborated by at least two. Because the votes are retained, users can recompute the union, the intersection, or any m-of-n threshold, and the individual annotations remain the source of truth.

Finally, caption quality is assessed in two steps. We retain captions of more than four words and no longer than four standard deviations above the mean (~80 words). The floor is deliberately low, because inspection showed word count to be a weak proxy for quality: five- and six-word captions are usually adequate, while low-effort responses occur at every length. We then embed each caption with all-MiniLM-L6-v2 and compare captions of the same illustration against randomly paired captions. Median within-image similarity is 0.52 (IA) and 0.54 (ICDL) against 0.11 and 0.12 for random pairs — roughly a fivefold separation — and the median pairwise difference in caption length is 10 and 9 words. Around 9% of illustrations fall below the 90th percentile of the random distribution, marking candidates for misaligned or sparse captions. We publish this alignment score for every caption as a diagnostic: low similarity conflates genuinely poor captions with valid non-English ones (the model is English-centric), with richer captions whose counterparts are vague, and with legitimate alternative readings of the same image. Users may filter at a threshold of their choosing.

Limitations

Our dataset contains several limitations. First, the selection criteria behind the two collections (IA and ICDL) are not commensurable and thus users should be cautious about using the combined set for diachronic analysis. While the ICDL dataset contains a wide spread of languages, these are not evenly distributed nor is the selection of children’s books representative of their respective language cultures. This set is more useful for assessing interlingual differences of illustration practices rather than large-scale cultural analysis of different world literatures. Finally, we do not provide the image files as part of our dataset but do include links to all books in the dataset in the metadata for researchers to reproduce.

Repository name

Borealis

Object name

DATA_PicturingStories_Annotations.csv

Format names and versions

CSV

Creation dates

2025-05-18 to 2026-03-07

Dataset creators

Andrew Piper (McGill University): conceptualization; methodology, Robert Budac (University of Alberta): data curation; project administration

Language

English

License

CC BY-NC-SA 4.0

Publication date

2026-09-17