KNoTE: A TEI Encoded XML Dataset of Modern Korean Novels

Repository location

The KNoTE (Korean Novel TEI Encoded) dataset is publicly accessible through the Zenodo repository: https://doi.org/10.5281/zenodo.18679321.

Context

The dataset was produced as part of a structured training programme in humanities data design at the Academy of Korean Studies (AKS). While TEI-based corpora have been developed for various European literary traditions, notably eighteenth-century French novels (Röttgermann, 2024) and early Holocaust testimony texts and camp literature (Entrup et al., 2013), and while recent data papers have begun to address non-Western modernist literatures in Asian languages (Wong, 2026), modern Korean fiction of 1906–1954 has so far lacked a structured TEI representation. Written across the transitional Hangul–Hanja mixed script and shaped by the colonial-modern experience, these texts present distinctive encoding challenges that existing frameworks do not address. Meanwhile, natural language processing research on Korean novels has advanced through character-centric annotated datasets (Kim et al., 2024), yet no TEI-encoded corpus has been made available for computational analysis of the broader literary structure. KNoTE addresses this gap by providing the first open, TEI-conformant XML dataset of modern Korean novels.

Steps

The dataset production involved two main procedures. First, in the data collection phase, source texts were collected from Korean Wikisource using a hybrid approach¹ combining the Wikisource XML dump file with the Wikisource and Wikidata APIs. A custom Python-based scraping and parsing script was developed to take a pre-compiled CSV of target works as input and retrieve each work’s wikitext and metadata. For works split across multiple chapters, the script parsed the full dump file, collected and sorted all related chapters, and reassembled them into a single complete text. All wiki markup was automatically stripped, with outputs generated in XML format. Second, in the semi-automatic encoding phase, initial TEI markup was generated using a web application (“Conversion of Korean Modern Literature xml/tei”²) built on Google AI Studio (Gemini 2.5 Flash, Figure 1). This methodology builds on the capacity of large language models (LLMs) to handle TEI annotation tasks; as recent evaluations suggest, these models can now automate labour-intensive markup steps such as structural tagging and entity disambiguation (Strutz, 2026).

Figure 1

Utilising prompt engineering tailored to modern Korean fiction, the application performed automated construction of the TEI header, generation of a character ontology (<listPerson>), named entity tagging (<persName>, <placeName>, <orgName>), distinction between direct speech and interior monologue (<said>), Hanja annotation (<foreign xml:lang="zh">), and Korean postpositional particle separation in entity tagging. Encoding was based on the TEI ALL schema of the TEI P5 Guidelines. Tailored prompt engineering is necessary because, without strict guidance, LLMs frequently produce XML validation errors or bypass external reference data during complex tasks (De Cristofaro & Zilio, 2025). Structuring the prompts in this way reduced such errors and improved markup accuracy.

Table 2 lists representative custom values from this process, including political ideologies, social classes, modern Korean spaces, and a project-specific virtual speaker edcmt for narratorial commentary. Many named entities also carry an external authority link: Korean places, organisations, events, and concepts to the Encyclopedia of Korean Culture (encykorea.aks.ac.kr), and persons to Wikipedia (ko.wikipedia.org) or Wiktionary (ko.wiktionary.org). A complete inventory of all TEI elements and attributes in the corpus is provided in the repository (KNoTE_TEI_tags_full.md) and the Zenodo record.

Sampling strategy

The KNoTE dataset aims to present a reproducible TEI encoding workflow and provide a seed dataset for evaluating automated TEI tagging systems. Texts were therefore deliberately sourced from Korean Wikisource—a freely accessible, public-domain repository—to ensure that the dataset can be independently verified and extended without access restrictions. The dataset comprises 33 works of modern Korean fiction by 22 authors, spanning from Yi In-jik’s Tears of Blood (Hyeol-ui Nu, 1906) to Kang So-cheon’s The Photo Studio that Takes Pictures of Dreams (1954), covering the formative decades of modern Korean literature. Following the corpus-balancing framework of Röttgermann (2024), works were selected by four explicit criteria rather than canonical reputation alone: (i) literary-historical canonicity, operationalised through inclusion in standard literary histories and reference works (e.g., the Encyclopedia of Korean Culture) and the availability of Wikidata/Wikisource records; (ii) representative or “first-of-its-kind” status (e.g., the first sinsoseol, the first original Korean science fiction, and the debut work of the first modern woman novelist); (iii) balance across publication period, narrative form, and literary movement; and (iv) a verifiable public-domain source text suitable for annotation. Table 3 lists each work with its publication year, narrative form, and selection rationale; the resulting corpus ranges from sinsoseol (新小說) and short fiction to early modern science fiction. Women authors remain a minority (6 of 33 works), reflecting the historical scarcity of published women writers; we accordingly included all major available women writers of the period (Kim Myeong-sun, Na Hye-seok, Kang Kyeong-ae, and Ji Ha-ryeon).

Quality control

To ensure encoding consistency and accuracy, multiple layers of manual validation were implemented. Following the initial artificial intelligence (AI)-assisted markup, seven annotators reviewed, corrected, and supplemented the tags based on internally developed encoding guidelines. Subsequently, the curator and data manager conducted a full-coverage cross-review of all files, checking <persName>, <ref> consistency, <said> attribute integrity, and <foreign xml:lang> accuracy. Finally, the project supervisor performed a final scholarly review. Each file’s <revisionDesc> records the dates of encoding completion and verification. This encoding and quality control process was carried out as part of a structured training programme in humanities data design at the Academy of Korean Studies (AKS). A preliminary version of this workflow will be presented at a workshop accepted for DH2026 (Daejeon, South Korea): “Building a Korean Literature TEI/XML Database: From Data Acquisition to AI-Enhanced Reading.”

Repository name

Zenodo

Object name

KNoTE dataset (on Zenodo: https://doi.org/10.5281/zenodo.18679321)

Format names and versions

XML, conformant to TEI P5 (TEI ALL schema)

Creation dates

2025-09-01–2026-04-27

Dataset creators

Kim, Gayeon (Data manager, Academy of Korean Studies); Park, Seonyeong (Data curator, Academy of Korean Studies); Ji, Haein (Data collector, Academy of Korean Studies); Lee, Hagyeong (Annotator, Sogang University); Lee, Byeongjoo (Annotator, Republic of Korea Air Force Academy); Jeong, Chaeyeon (Annotator, Korea University); Lee, Jae-yeol (Annotator, Academy of Korean Studies); Jo, Gyungmin (Annotator, Academy of Korean Studies); Lim, Iro (Annotator, Academy of Korean Studies); Ismayilov, Orkhan (Annotator, Academy of Korean Studies); Kim, Byungjun (Supervisor, Academy of Korean Studies).

Language

Korean, with Hanja (Chinese character) annotations marked via xml:lang="zh".

License

Creative Commons Attribution 4.0 International (CC-BY 4.0).

Publication date

2026-04-27