(1) Overview
Repository location
Context
Contemporary studies of Indonesian literature increasingly employ computational approaches, reflecting a shift from interpretative reading toward large-scale corpus-based analysis (Jockers, 2017; Moretti, 2000; Underwood, 2019). These approaches facilitate the investigation of lexical patterns, stylistic variation, language change, and literary dynamics through structured textual data. In corpus linguistics, diachronic corpora have become an important resource for examining long-term linguistic evolution and literary practices (Hilpert & Gries, 2016; Michel et al., 2011). However, such developments remain concentrated on languages supported by well-established digital infrastructures and openly accessible corpora (Biber et al., 1998). By contrast, Indonesian-language corpus infrastructure, particularly diachronic literary corpora, remains limited. As a result, research on Indonesian literature continues to rely largely on qualitative analyses of a relatively small number of printed texts, while opportunities for large-scale quantitative investigation remain underutilized (Auer et al., 2005).
Despite ongoing efforts to digitize Indonesian literary texts, the lack of a corpus that systematically integrates historical, linguistic, and orthographic dimensions remains a significant gap, particularly given major spelling transitions from Van Ophuijsen to Soewandi and Ejaan Yang Disempurnakan (EYD) (Indonesia, 2000). To address this gap, we developed the Diachronic Orthography of Indonesian Literature (DOIL) dataset, a diachronic corpus of 58 historically significant literary works (1920–2000) representing major literary periods, genres, and the three official Indonesian orthographic systems, enabling the study of language change as a reflection of Indonesia’s cultural and political transformations.
(2) Method
Data collection began with the identification of representative works associated with major periods in Indonesian literary history, including Balai Poestaka, Poejangga Baroe, Angkatan 45, Angkatan 50, Angkatan 66, Angkatan 70, Angkatan 80, and Reformasi (Rokib, 2024). The selection of works followed a purposive sampling strategy considering three main criteria: (1) historical significance within the Indonesian literary canon, (2) the availability of printed sources suitable for digitization, and (3) representation across literary genres (novel, short story, poetry, and drama) to ensure formal diversity within the corpus.
Data acquisition
All texts in the corpus were collected from printed sources and digitized using the free version of CamScanner with built-in Optical Character Recognition (OCR). The OCR output produced machine-readable text files containing character recognition errors, page remnants, and inconsistent spacing. Data cleaning followed a semi-manual workflow in which automated scripts removed repetitive structural noise and normalized spacing and punctuation, while OCR errors, paragraph boundaries, and remaining textual inconsistencies were manually corrected through iterative quality inspection. Paratextual elements, including front and back matter, page numbers, headers, and footers, were also removed.
Text Processing and Annotation
To support computational analysis and data interoperability, each cleaned text underwent tokenization, orthographic standardization, lemmatization, and annotation. Tokenization was performed using Python 3 scripts in Google Colab, preserving the original historical spellings, while orthographic standardization generated normalized forms based on modern Indonesian spelling (EYD) for comparison with historical variants. Lemmatization mapped inflected and derived word forms to their base forms using Indonesian morphological rules (Badan Pengembangan dan Pembinaan Bahasa, 2022), supplemented by manual verification. Annotation files record the original token, standardized form, lemma, and orthographic system for each token. For example, the historical token mendjadi is preserved during tokenization, standardized as menjadi, lemmatized as jadi, and recorded with its associated Van Ophuijsen orthographic classification in the annotation layer.
Orthographic annotation
A distinctive feature of this corpus is the explicit modelling of Indonesian orthographic change as part of its diachronic framework. Each work is first orthographically standardized while preserving the original text, and each token is accompanied by annotations recording its original form, standardized form, lemma, and the orthographic system associated with the source text. The corpus represents the three official Indonesian orthographic systems: Van Ophuijsen (1901–1947), Soewandi or Republican Spelling (1947–1972), and Ejaan Yang Disempurnakan (EYD) (1972–2000). Orthographic classification is determined through publication year and verification of historical spelling features, including forms such as oe, tj, dj, nj, and sj. Orthographic information is distributed through token-level annotation files accompanying each work, enabling research on spelling reform, language standardization, and historical language change.
Metadata construction
Metadata follow a modified Dublin Core schema (Dublin Core Metadata Initiative, 2012) and are stored in metadata.csv. Fields include title, author, publisher, publication year, literary period, genre, orthography system, word count, source type, copyright status, and annotation level. Copyright information is recorded as ‘Uncertain copyright status’ when ownership or legal protection could not be verified with sufficient confidence, providing a transparent indication of potential legal ambiguity for data reuse.
Data organization
The repository comprises five corpus layers (cleaned, tokenized, standardized, lemmatized, and annotated texts), accompanied by metadata.csv, README.md, and DOIL_processing_workflow.pdf. To accommodate copyright constraints, the repository provides open corpus extracts together with their corresponding linguistic annotations and metadata for eligible works. The extracts preserve the linguistic and structural characteristics of each work while excluding final chapters when necessary.
Quality control
Bibliographic records were verified against authoritative sources. OCR outputs and all corpus layers underwent automated checks and manual inspection. Metadata were validated for consistency, while duplication and integrity checks ensured correspondence between repository files and documentation.
(3) Dataset Description
Dataset information
Repository name: Zenodo
Object name: Diachronic Orthography of Indonesian Literature (DOIL)
Repository location: https://doi.org/10.5281/zenodo.20728073
Formats: ZIP (cleaned, tokenized, standardized, lemmatized, and annotated corpus files), CSV (metadata), Markdown (README), and PDF (processing workflow)
Language: Indonesian
License: CC BY 4.0 (metadata, documentation, and open corpus extracts only)
(4) Reuse Potential
The Diachronic Orthography of Indonesian Literature (DOIL) provides a reusable resource for corpus linguistics, historical linguistics, Indonesian literary studies, digital humanities, and Indonesian Natural Language Processing (NLP). By integrating diachronic literary texts with standardized metadata and five linguistic processing layers, the corpus supports research on lexical change, orthographic variation, spelling reform, language standardization, and literary history. To accommodate copyright considerations, the repository distributes only open corpus extracts together with metadata and annotations under a CC BY 4.0 license, while the original literary works remain under their respective copyright holders and are not redistributed.
AI Declaration
Generative AI tools were used in a limited manner for grammar checking during manuscript preparation.
Acknowledgements
The authors would like to thank Universitas Negeri Surabaya and Kementerian Pendidikan Tinggi, Sains, dan Teknologi (Kemdiktisaintek) and Universiti Malaya for providing academic support and a conducive research environment for this study. We also acknowledge the National University of Singapore (NUS) for providing the first author with a conducive environment for writing this paper through the Visiting Scholar scheme in 2026.
Author Contributions
Mohammad Rokib: Conceptualization; Data Curation; Formal Analysis; Investigation; Methodology; Project Administration; Visualization; Writing – Original Draft; Writing – Review & Editing.
Moh. Mudzakkir: Conceptualization; Supervision; Methodology; Validation.
Noorhidawati Abdullah: Data Curation; Metadata Design; Validation; Supervision.
Lutfiyah Alindah: Data Curation; Investigation; OCR Processing; Text Cleaning; Tokenization; Lemmatization; Writing – Review & Editing.