Construction of Consensus-Labeled Data for the Study of Emotions in Korean Full-Length Novels

This study proposes a consensus-labeling methodology for constructing emotion data for Korean full-length novels by integrating multiple annotations, model predictions, and researcher verification. Emotion interpretation in literary texts may vary across researchers, while sentence-level emotion prediction models may not fully capture emotions that depend on broader narrative context. First, for 5,282 sentences from Sohyeonseongrok, one set of deep learning model predictions and two sets of human annotations were combined. Provisional labels were derived through majority rule and then subjected to researcher verification based on surrounding narrative context. Next, eight emotion prediction models trained under different training-data configurations were applied to 15,426 sentences from Sossi-Samdaerok. When five or more models predicted the same emotion, the shared label was treated as a provisional model consensus and subsequently reviewed by the researcher. Model comparisons showed that predictions varied according to training-data configuration, while the model trained on the Sohyeonseongrok consensus-labeled data achieved agreement comparable to a model trained on a substantially larger dataset formed by concatenating three annotation datasets. These findings suggest that consensus labeling can provide practical shared reference labels for computational analysis while preserving information about differing annotation perspectives, rather than establishing a single objectively correct emotion label. The resulting datasets are publicly available on Zenodo under a CC-BY 4.0 license for reuse in Digital Humanities and NLP research on Korean classical literature.