Knowledge Distribution in German Drama

Repository location

The dataset can be found on GitHub at https://github.com/quadrama/knowledge-annotation as well as on Zenodo at https://doi.org/10.5281/zenodo.8319261.

Context

This dataset was collected in the project Q:TRACK – Quantitative Drama Analytics: Tracking Character Knowledge. Q:TRACK targets the fact that a play’s dramatic characters can have different levels of awareness of certain information. Hence, the transmission and distribution of knowledge is one central object of study for drama analysis. In his Poetics, Aristotle emphasizes the importance of so-called anagnorisis. Aristotle’s concept of anagnorisis refers to recognition scenes, where a character, for instance, recognizes a long-lost relative and all previous events appear in a new light (Aristotle, 1995). The “discrepant awareness” (Evans, 1960, p. VIII) of different characters and/or characters and the audience can propel the plot of a play, creates suspense and thus greatly contributes to the play’s effect (Anz, 1998; Cave, 1988; Pfister, 1988). Therefore, the project aims to systematically model and track the distribution of knowledge in plays through annotation.

This adds to existing research in the field of computational literary studies where characters and their relationships in plays have recently gained attention (Fischer, Trilcke, Kittel, Milling, & Skorinkin, 2018; Krautter & Vauth, 2023; Lee & Lee, 2017; Trilcke, 2022). The knowledge distribution can also be used to specify the character interactions with regard to network analysis (Krautter, 2023). Character relationships are also covered in a dataset by Massey, Xia, Bamman, and Smith (2015) that is based on English narratives. However, they do not distinguish between the diverging and developing knowledge of individual characters and work with text summaries only.

We restricted the annotation to the domain of knowledge about character relations, as this domain is key in many plays. In Johann Gottlob Benjamin Pfeil’s tragedy Lucie Woodvil (1756), for instance, the main character Lucie learns too late that her lover and father of her unborn child is also her brother. The annotated relations in our dataset include family relations (parent_of(A, B), child_of(B, A), siblings(B, C) …), love relations (in_love_with(B, D), engaged(B, D), spouses(B, D) …) questions of identity (identity (A, E), has_name(A, ‘name’)) and death (dead(A), murderer_of(B, A)). In addition to the knowledge itself, the annotations contain information about the source and target of each knowledge transfer. This results in the following tag structure:

transfer(SOURCE, TARGET, KNOWLEDGE, ATTRIBUTES)

SOURCE is the character that passes on the knowledge, TARGET is one or several characters that receive the knowledge, and KNOWLEDGE specifies the knowledge itself as one of the character relations described above. Optional attributes allow to include additional information, e. g. if SOURCE is lying or if the information is still uncertain (see details in Andresen, Krautter, Pagel, & Reiter, 2021, in German).

Steps

The dataset comprises the 30 German plays listed in Table 1, with a total size of 736,808 tokens (including all utterances as well as stage directions). The plays were retrieved in the TEI-XML format from the Drama Corpora Project (Fischer et al., 2019) and imported into CorefAnnotator. The data were annotated in three rounds:

1. In the initial round, 16 plays were annotated by two annotators following a preliminary guideline. Issues were discussed with one of the authors and, where necessary, with the whole team. This process resulted in the final annotation guideline (Andresen et al., 2021, in German).

2. In the second round, the other 14 plays were annotated independently following the guideline. These plays were used to calculate the inter-annotator agreement using the measure gamma by Mathet, Widlöcher, and Métivier (2015), as presented (and discussed critically) in Andresen, Krautter, Pagel, and Reiter (2022b).

3. In a final round, every play was discussed and double checked by at least one annotator. In this round, three more relations were added for murder, death and pregnancy.

The final version of the corpus (round 3) comprises 37 files, as for seven plays, both annotators performed the last step of finalizing the annotations, resulting in two final versions for these plays. We decided to keep two versions instead of creating a single gold standard, because in many cases more than one way of annotating the play was justified (see below). In total, there are 1277 annotated text passages, which corresponds to an average number of 34.5 annotations per text, with a considerable standard deviation of 18.8.

Sampling strategy

The plays were manually selected to cover

• plays of which we knew that knowledge about character relations is important for the plot, (?) as well as plays where this was not the case,

• tragedies as well as comedies,

• plays from different literary epochs (1740–1900).

Accordingly, the dataset is not designed to be representative of a specific group of texts, but to cover a wide range of relevant phenomena.

Quality control

All plays were annotated by two people independently, making it possible to calculate the inter-annotator agreement. The agreement is rather low for many of the plays, see Table 2. This is due to the high complexity and interpretation dependency of the task. In many cases more than one way of modeling the data is plausible. Also, measuring inter-annotator agreement in a way that makes the scores comparable to other studies is challenging for annotations without predefined annotation spans. See Andresen et al. (2022b) for a more in-depth discussion and the repository for more detailed scores. We publish several versions of each annotation as well as the annotation guidelines (Andresen et al., 2021, in German) for comparability and transparency.