Implicit Messages of Narratives and Evaluative Text Structures: A Network-Based Approach

1. Introduction

When people want to convince us of normative matters, they often tell a story. Only in rare cases, however, is what they want to convince us of explicitly formulated in the story, like the proverbial ‘moral of the story’ in a fable. Often the assertion remains implicit; it is therefore an implicit message or, as some researchers have it, an “implicit assertion” (Mikkonen 2010; Gittel 2023; for discussion and delimitation: García-Carpintero 2020). That such implicit assertions exist in fictional narrative texts is undisputed. However, there has been little research into the textual properties that make this the case in certain texts and not in others. This paper argues that evaluative text structures, i.e. complex patterns of evaluations and oppositions which emerge from a focused text analysis, play a central role in the generation of implicit messages in literary texts.

Beside several non-terminologized terms like “moral statement” (moralische Aussage, Müller 2010: 468), “message” (Botschaft, Andriga 1994: 105), “moral of the story” (‘Moral von der Geschicht’, Gabriel 2010: 254), several technical terms have been proposed for the implicit messages of literary texts: Locus classicus is Hospers’ (1960) “implied truths”. There are also “implied […] thesis”/“implicit thematic statement” (Lamarque and Olsen 1994: 324), “implicit reflections” or “theses” (Beardsley 1958: 415), “implicit assertions” (Mikkonen 2010: 312–30), and “implicit […] propositions” (Petraschka 2014: 100). All these concepts refer to a special kind of content-transcending text meanings that go beyond what is the case in the fictional world (e.g. character X is jealous of character Y).¹ The differentia specifica of this kind of content-transcending text meaning is that it can be expressed as a proposition which, although not present in the fictional work itself, is suggested or implied by it. This distinguishes it from other content-transcending interpretations, such as Goethe’s “Elective Affinities” being a novel about the tragedy of understanding signs.

While, under the heading of “narrative persuasion” (Bilandzic and Busselle 2013), psychologists studied effects of reader-bound factors like transportation or identification (Green and Brock 2002; Slater and Rouner 2002), a reader’s prior knowledge and attitudes as well as the perceived similarity to central characters (Appel 2022; Sukalla 2023), research, in general, rarely addresses the text structures and contextual properties on which implicit assertions are based. Literary theory refers to this as the “problem of implication” (cf. Sirridge 1980). Lamarque and Olsen (1994) have described how implicit thematic assertions are made in literary criticism in the context of thematic appreciative interpretations but have identified the problem of how exactly the relevant elements of a work are identified and conveyed in the context of such an interpretation with the “problem of interpretation itself” (cf. esp. 326–8, here: 328). Other researchers have hypothesized that various text-internal and text-external factors may be relevant: Possible work-related factors include the tone of the utterance (Mikkonen 2010: 325), the style, treatment of the theme, the plot, sympathy for the characters (Hospers 1960: 41), while literary conventions, general convictions of the contemporary audience, knowledge about the author are brought into play as possible contextual factors (Mikkonen 2010: 326). Further hypotheses can be found in research on relevant genres such as exemplum, fable, or parable (see Stierle 1973, esp. 351–7; Zymner 1991) and in the field of “narrative ethics” (Booth 1961 [1983]: 377–98; Phelan 2005). Finally, research on literary evaluations (Hunt and Vipond 1986; Winko 1991; Prinz and Winko 2013; Prinz 2020) has suggested that implicit assertions result from the complex interplay of inner-literary evaluations on different text levels and can be described as evaluations at the level of the “text organization” (Prinz and Winko 2013: 405).

This paper takes up the last idea by a) introducing an annotation scheme for literary evaluations (i.e., the attribution of a positive or negative or positively or negatively connoted characteristic within a literary text), b) examining their distribution in a corpus of German-language fictional narratives, c) introducing the concept of “evaluative text structure” and visualizing such evaluative text structures in the form of networks, and d) qualitatively and quantitatively examining their connection with implicit assertions. As will be shown, even short literary texts usually contain a large number of literary evaluations, the detailed analysis of which represents a challenge. It is therefore not surprising that narratology has so far only taken a rudimentary look at the interplay of evaluations and the evaluative text structures resulting from them, and mainly in case studies (e.g. Phelan and Martin 1999; Dimpel 2018). In contrast, digital annotation combined with network analysis offers a more adequate possibility that complements a recent thought-provoking study of “story morals” on the basis of summaries/stories using Large Language Models (Hobson et al. 2024). A major difference between this approach and the approach taken here, beside their partial reliance on summaries, is that Hobson et al. (2024) seem to presuppose that every story has an implicit message (“moral”).

The reader may wonder why from the many postulated possible factors in research the paper just picks one. There are two reasons: First, one may observe that many implicit assertions from literary texts as ascribed by literary critics have evaluative or normative content themselves (see the examples in Gittel 2023). If many implicit assertions are themselves evaluations, and if fictional-literary texts typically contain evaluations, then the idea suggests itself that the evaluative content of literary evaluations is somehow transferred to real-world entities. The second reason is pragmatic in nature. What I call evaluative structure is in itself a very complex higher-order property of texts. It would therefore be presumptuous to investigate other potentially relevant factors in addition to this factor.

The paper is structured as follows: The next section gives a short overview of a digital annotation of literary evaluations in a corpus of thirty-five fictional German-language narrative texts and a document-level annotation of implicit assertions is presented (Section 2). In the next section, I clarify what I mean by evaluative structures and propose different network visualizations for them (Section 3). Based on first insights from these visualizations, I propose a narratologically motivated typology for the texts of the annotation corpus and contrast it with a typology obtained via a quantitative comparison of their network structure (Portrait Divergence) (Section 4). Finally, I analyze whether it is possible to predict whether a text has a (clear) implicit message, both from these two typologies as well as from annotation and structural network features (Section 5). The final section summarizes the results and outlines open questions for further research.

2. The Manual Annotation

The annotation corpus consists of thirty-five fictional narrative texts in German published after 1800. The texts were selected from different sources (Gutenberg, a short story anthology, etc.) in order to cover a wide variety of themes and time periods while at the same time aiming at corpus diversity in terms of authors and authors’ sex. The length of the texts varies between 138 and 12435 tokens, nine and 334 sentences.² For all texts except for one, which is the beginning of a novel, the complete text was annotated.

The annotation was performed collaboratively by pairs of two annotators per text for three main phenomena: literary evaluations, literary encodings, and oppositions between entities. To ensure the quality of the data, a consensus-based gold-standard (discussion of cases of disagreement among annotators and principal investigator) was created and inter-annotator agreement calculated. The iterative process of gold-standard evaluation helped to repeatedly improve the annotation guidelines (Gius and Jacke, 2017). The following analyses are based on the gold-standard data which includes the unanimous annotations (on which the annotators agreed upon) plus the annotations for which the annotation team (two annotators plus PI) reached a consensus.

The following description of the annotation categories is limited to the most important categories and aspects of the annotation scheme for the purpose of this article. The annotation guidelines are available online (Gittel [forthcoming]) as is the annotation data (Gittel 2025).

Literary evaluations are a narratological phenomenon that has been researched relatively seldomly (see however: Hunt and Vipond 1986; Winko 1991; Prinz 2020). They can be understood as an act of evaluation³ consisting of the attribution of a positive or negative or positively or negatively connoted characteristic (Prinz and Winko 2013: 402; Winko 1991: 54–6). Literary evaluations can be verbal or nonverbal (e.g. spitting in front of someone’s feet), explicit or implicit. They can be understood as a relation with four relata: Who values whom or what in what respect and how? This can be expressed more technically as follows:

where X represents the evaluator (in computational linguistics often called “holder”/“source”), Y the evaluated entity (target), A the evaluation aspect and P the evaluation polarity (Benamara et al. 2017: 218). Evaluators can be: a narrator (the default assumption), a character, or the author (e.g. for evaluations in titles or pronounced happy/tragic endings).⁴ The following types of evaluated entities are distinguished: character, group of characters, organization (character group with proper name and common interests), space, area, boundary, time period, thing, abstract, other. Frequent evaluation aspects are aesthetic (pleasure, germ. Wohlgefallen), eudaemonistic (well-being), epistemic (knowledge), moral (morality), social status and other. Polarity has two characteristics: positive or negative. In addition, triggers for recognizing the evaluation were annotated where available.

Entities were annotated as nominal phrases, evaluations as noun phrases or clauses, and evaluation triggers as individual words. The annotation was carried out using the INCEpTION software (Klie et al. 2018). Figure 1 shows an example of a nominal phrase and a clause evaluation. In both cases, the evaluator is the narrator, which is not shown as relation, as this is the default assumption of the annotation guidelines.⁵

Figure 1

All evaluated entities were given an identifier (their proper name or – for entities without a proper name – the wording of the first mention) so that multiple evaluations of the same entity could be easily identified. Furthermore, the semantic trigger of the evaluation (e.g. “beautiful” in “a beautiful girl”), if present, was annotated by a separate tag. This allows to distinguish explicit evaluations (with trigger) from implicit evaluations (without trigger) later on.⁶

Literary encodings are text passages that lead to semantic loadings of specific entities, for example a character or a fictive location being described as traditional, morbid or, in contrast as modern and vital. As these examples already suggest, literary encodings may include evaluative aspects and have been researched much less in literary theory than in specific branches of research on literary modernism (esp. Lindner 1994). For literary texts, five dimensions of literary encodings are especially important:

• – natural vs. cultural (Nabholz 2004: 51–60; Grätz 2016; Brasch 2017: esp. 222f.)

• – healthy vs. sick (King 2012)

• – profound vs. superficial (Lindner 1994)

• – traditional vs. modern (von Graevenitz 1998: 10)

• – harmonious vs. disharmonious (Thesing 2021: 236, Ft. 59)

The reason why these particular categories were selected is related to the project context in which the annotation was carried out: a project on literary-fictional forms of modernity critique.⁷ Literary encodings were annotated in an analogous way to literary evaluations, meaning that for each encoding the data contains the ‘encoder’ (source), the encoded entity (target), the encoding dimension (e.g. traditional vs. modern), and the encoding polarity (e.g. traditional). Figure 2 shows three examples for literary encodings.

Figure 2

Furthermore, oppositions between entities of the fictional world have been annotated. Especially literary structuralists considered oppositions as important for the construction of meaning in literary texts (Greimas 1983 [1966]: ch. 12; Lotman 1977 [1970]; Titzmann 2003; Krah 2015: ch. 5). Three types of oppositions between entities were annotated:

1. value oppositions,

2. encoding oppositions, and

3. other feature oppositions.

Value oppositions exist between good and bad entities. Encoding oppositions exist between entities that are opposite regarding at least one of the foremost mentioned dimensions, e.g. between natural or cultural encoded entities. Other feature oppositions are oppositions that exist between entities with other opposing properties. All opposition types are only annotated between entities of the same type (e.g. characters, locations, time periods).

Figure 3 shows an example of a value opposition.

Figure 3

There are no restrictions regarding the distance between the relata in the text; many of the annotated oppositions (contrary to the one in Figure 3) are so-called long-distance relations (one part of the opposition may appear early in the text and the other part several sentences or pages later). Since entities that are important for interpretation are usually mentioned several times in the text, the guidelines demand to annotate the opposition at the text position where it becomes clear that the opposition in question exists.⁸

Table 1 provides an overview of the annotation corpus.

Table 2 shows the annotator agreement for each category, except for oppositions for which no standard measure is available.⁹ I computed inter-annotator agreement using Mathet’s Gamma (γ, Mathet et al. 2015). γ respects the individual annotated passages as units in a continuum, and also partially overlapping passages are compared as units instead of disjointed NPs/clauses.

The inter-annotator agreement for evaluations is still within a range that is considered acceptable by researchers in Computational Literary Studies dealing with highly complex and context-sensitive phenomena. The low inter-annotator agreement for literary encodings is partly due to the limited number of annotators for the study who had to split their attention and were primarily focused on the more frequent phenomenon of evaluations. A future project may improve in this regard by training specialized annotators for each task.

Implicit assertions are theses about the real world that a fictional text suggests or implies without formulating them explicitly (Mikkonen 2010; Gittel 2023). It was annotated at document level a) how certain the annotator is that the text contains an implicit assertion (assertive force), and b) how certain the annotator is about what exactly the implicit assertion is (assertive clarity). This metadata was annotated by three annotators due to its central importance for the research question. The questionnaire and the corresponding inter-annotator agreement (Krippendorff’s alpha, Krippendorff 2013) is shown in Table 3.

Please note that the moderate agreement between annotators for assertive force and assertive clarity has to be expected for this task, because it involves a very high degree of interpretation at different levels. Before answering the question whether a text has an implicit message one needs to understand: what is the case in the imaginary world the text devises, what is the plot, what certain symbols and/or metaphors mean etc. What the agreement scores nevertheless show is that the assessment whether there is a (clear) implicit assertion or not is not arbitrary either.

3. The Visualization of Evaluative Text Structures

On the basis of the annotated data, literary evaluations can now, as promised at the beginning, be analyzed not only selectively, as has been customary in research up to now, but systematically. The importance of empiricization at this point becomes clear when one considers the complexity of evaluative text structures with the help of multi-layer network representations. Figure 4 illustrates only the character-related part of the evaluative structure of Kleist’s story “The Earthquake in Chili” (“Das Erdbeben in Chili”), i.e. by showing evaluations and literary encodings at character level (blue), narrator level (orange), and oppositions at the text organization level (green), for the distinction of these three levels of analysis see Prinz and Winko (2013).

Figure 4

Evaluations are represented by simple arrows in grey. Characters with an overall negative evaluation are shown with a minus, characters with an overall positive evaluation with a plus, whereby all the evaluations of a character on a level are offset against each other regardless of their evaluation aspect. Thick arrows indicate that several evaluations are superimposed here, for example in the case of Josephe Asteron and Don Fernando Ormez at the narrator level. Literary encodings are visualized in different colors (see legend). On the text organization level, oppositions are visualized as double-sided arrows in grey, for example between the noble Don Fernando and the leader of the mob, Master Pedrillo. Analogous representations can be created for locations or other entity types.

Although multilayer networks are useful to illustrate the complexity of evaluative structures, the remainder of this paper will rely on directed single-layer networks, i.e. networks with one layer on which connections (“edges”) between nodes have a specific direction. This is for two reasons. First, multi-layer networks are less apt to visually compare networks of different texts because, at least for longer texts, they quickly get confusing. Second, the task of computationally comparing directed multi-layer networks is, to my knowledge, an unsolved problem in network science. The single-layer networks used in the following (see the example in Figure 5) are designed as a trade-off between readability and informational content. The chosen form of presentation is guided by the hypothesis that implicit assertions are often generated via certain forms of contrastivity. Accordingly, not only the evaluations (positive ones in green and negative ones in blue) but also the oppositions between entities (orange) are shown. Node size has been set to out-degrees (number of outgoing edges) and label size to degrees (number of edges the node is connected to). Eudaemonistic evaluations regarding characters, i.e. evaluations regarding their well-being, are omitted because they are usually subject to change in the course of the narrative and their integration into the networks would potentially distort what one may call the basic evaluative structure of a narrative. However, it could be particularly interesting to integrate them into a dynamic network analysis for longer fictional texts like whole novels (see Coll Ardanuy and Sporleder 2015; Park et al. 2022). Furthermore, I decided to leave out literary encodings and focus on evaluations and oppositions only. What such a directed single-layer network resulting from these decisions looks like can be seen in Figure 5, that is based on the annotation of Kleist’s story again. Evaluations are represented as unidirectional arrows in blue (negative) or green (positive), oppositions are represented as two-directional edges in orange. This kind of network representation is the basis for all further analysis in the remainder of the paper. The caption of each network figure contains a link to the complete, zoomable visualization.

Figure 5

Side note: Some readers may wonder why the focus of the representation is on the evaluations and not on the evaluative properties of the entities that result from them. The question of the evaluative properties of the entities is of course interesting, but more relevant to the question of implicit assertions are the acts of evaluation and thus the question of who evaluates whom and how. As Prinz (2019) shows using the example of Schiller’s story “Criminal out of Lost Honour” (“Verbrecher aus verlorener Ehre”), the difference between extradiegetic and intradiegetic evaluations is of great importance for the implicit message of the story, since intradiegetic evaluations, i.e. evaluations of characters, are often interest-driven and can in turn be relativized/debunked by the narrator or even the author. In addition, the determination of the evaluative characteristics of the entities is associated with various methodological and literary-theoretical problems, the solution of which requires research of its own. These include an assessment of the credibility of characters and narrative instances and the question of whether there are evaluative fictional facts (for discussion see Walton and Tanner 1994 and the debate about the puzzle of imaginative resistance, Tuna 2024). Fictional facts generally refer to sentences that indicate what is the case in the fictional world of a fiction, for example that the protagonist of Schiller’s story is called Christian Wolf and commits murder. But is it also a fictional fact that this Christian Wolf is morally reprehensible or that he is not? Even if we ignore the extensive philosophical debate on “moral realism” (Sayre-McCord 2023), this is not a trivial question to answer, as it is not immediately clear whose standard of evaluation should be decisive in answering it, that of the narrator, that of the author, or that of the (contemporary?) reader.

4. Typologies of Evaluative Text Structures

Comparing thirty-five directed single-layer network visualizations manually, quickly raises the question whether there are certain types of evaluative structures that can be identified. This section deals with this question from two angles, using a qualitative narratological approach (4.1.) and a quantitative approach that clusters the networks according to their structural similarities and dissimilarities among each other (4.2).

4.1 A Narratologically Motivated Typology

If one visually compares the network graphs, it quickly becomes clear that the evaluative attention in texts, i.e. the number of evaluations received per entity can be distributed very differently and that the evaluative power can be centralized (usually to the narrator) or distributed among different entities (e.g. Brecht’s “Müllers natürliche Haltung” [“Müller’s Natural Attitude”] with three main evaluators). Furthermore, the relationship between evaluations and feature oppositions can be very different. Thea Dorn’s story “Vorsicht Steinschlag” (“Danger Falling Stones”) is illustrative here, in which the narrator only evaluates one of the two main characters, but there are numerous feature and encoding oppositions between the two characters (see Figure 6).

Figure 6

Based on the visual comparison and narratological knowledge, I propose the following tentative typology. There are three types of texts that are characterized by contrastivity:

• A) Texts with feature contrasts (n = 7): This category includes texts with feature contrasts like young vs. old or man vs. woman and/or encoding contrasts like cultural vs. natural or other. Robert Walser’s story “Der Nachen” (“The Bark”), which describes the rendezvous between a man and a woman on a rowing boat, would be paradigmatic for this category (see Figure 7).

• B) Texts with feature contrasts and value contrasts (n = 15): This category includes texts that contain value contrasts in addition to feature contrasts (e.g. interesting vs. boring).¹⁰ A paradigmatic text of this category would be Hollenstein’s “Gelb wie eine Zitrone” (“Yellow Like a Lemon”) (see Figure 8), which depicts the boring reality of a mine in contrast to an imagined world behind a yellow door.

• C) Texts with subverted contrasts (n = 3): These are texts in which there is a contrast between the same entity at different points in the “histoire”. An example of this is Kafka’s story “Bericht für eine Akademie” (“Report for an Academy”), in which the protagonist, the monkey Rotpeter, attains “the average education of a European” at the end, or Bachmann’s “Auch ich habe in Arkadien gelebt” (“I Too Have Lived in Arcadia”), where Arcadia is initially portrayed (contrary to its cultural-historical coding) as a decaying place, but at the end of the story is once again a place of longing. Such subverted contrasts are visualized in the network as circular “self-oppositions”, i.e. as oppositions whose start and end points represent the same entity (see Figure 9).

Figure 7

Figure 8

Figure 9

In addition, there are two other types that appear relevant for the generation of implicit messages and are not directly related to visually striking features of the networks:

• D) Texts with the narrative schema “Disposition-Outcome” (n = 6): This schema has two typical manifestations, good deed-reward and offense-punishment; in Grimms’ fairy tale “Frau Holle” (“Mother Holle”) both schemata are found, the industrious daughter is rewarded, the lazy daughter is punished. In the context of literary evaluation analysis, the outcome can be understood as evaluation at the text-organization level (see Prinz and Winko 2013: 406): The text is constructed in such a way that something specific is rewarded or punished. The outcome does not always have to be related to an action. In Hermann Löns’ “Die beiden Höfe” (“The Two Farms”), the tradition-conscious farming family is rewarded, while the family that believes in progress is punished. In Böll’s “Wanderer, kommst du nach Spa…” (“Stranger, Bear Word to the Spartans We…”), the outcome (the death of the protagonist) is the result of a world view, the war-glorifying ideology of his school days. Frame narratives in particular can dress up corresponding ‘example stories’ (Stierle 1973) in such a way that the implicit assertion of the internal narrative is made abundantly clear (e.g. Auerbach’s “Der Kindesmord” [“The Child Murder”]).

• E) Satirical texts (n = 4): Very briefly, specific text-world relations which the reader creates using specific knowledge are constitutive for texts of this type. These are often relations between fictional entities and real-world entities or relations between fictional entities and classes of entities (exemplification). Thus, Thomas Mann’s story “Beim Propheten” (“With the Prophet”) can and has been understood as a satirical critique of the poet Ludwig Derleth and the class of self-proclaimed poet-prophets. Characteristic of the evaluation structure of this type is that entities that refer to real-world-entities are evaluated negatively.

• F) Other (n = 7): These are texts that do not fall into any of the above categories, because they do not exhibit the relevant characteristics mentioned above.

This inductively derived typology is non-exhaustive and non-exclusive, i.e. one text can belong to more than one category.

4.2 A Typology Based on Quantitative Network Comparisons

Tantardini et al. (2019) compared quantitative methods for the comparison of networks. For unknown node-correspondence and weighted directed networks as in the present case, there are only three methods available. I selected Portrait Divergence (Bagrow and Bollt 2019), which quantifies the structural dissimilarity between two networks by evaluating the differences in their ‘network portraits’ – matrix representations that capture the distribution of shortest path lengths across the nodes.¹¹ Unlike traditional measures that focus on specific topological features (e.g. degree distribution or clustering coefficients), portrait divergence offers a holistic approach by incorporating multiscale information about node-to-node connectivity patterns. By calculating the Jensen-Shannon divergence between the normalized portraits of two networks, the method provides a robust and interpretable metric for comparative analysis.

Concretely, for each text I created three separate directed weighted networks: an opposition network (containing every opposition as bidirectional edge), a positive evaluation network, and a negative evaluation network. Then I compared the same types of networks, resulting in three network portrait matrices. These portrait matrices have been merged into a 3D-tensor representing the similarities between the networks regarding the three aspects (positive and negative evaluations, oppositions), because Portrait Divergence does not accept negative weights.¹² In the last step I determined the optimal number of clusters by comparing silhouette scores. The result for two clusters (average silhouette score = 0.24) is shown in Figure 10.

Figure 10

In addition to the improper clustering, one observes that one cluster is much bigger than the other (7 vs. 26) and that Hesse’s “Der Steppenwolf” (“Steppenwolf”), which is the longest text in the corpus, is a clear outlier. The two texts gravitating far to the upper right, the ones from Heckmann and Dorn, are both texts which are strongly dominated by oppositions. In contrast, Kafka and Hesse (bottom right) are essentially star shape-like networks. Far to the left side, one finds texts that have more than one evaluating entity (e.g. Storm, Kleist, Grimm).

5. Predicting Implicit Assertions from …

It is reasonable to assume that central macro-characteristics of fictional texts are related to evaluative text structures. This includes literary themes (certain themes could be associated with the frequent occurrence of certain evaluative aspects, for example justice with moral evaluations), aesthetic value of the texts (very schematic/didactic evaluative structures may hinder aesthetic appreciation), and implicit messages of texts. As Hunt and Vipond formulated in their seminal essay as early as 1986: “evaluations are, in fact, invitations to construct and share a version of what a teller might be ‘getting at’” (Hunt and Vipond 1986: 68).

In order to explore whether there is an empirical connection between the annotated metadata regarding implicit assertions on the one hand (cf. Section 2) and the annotated evaluative text structures on the other, I follow three approaches. First, I assess whether the narratologically motivated typology presented in 5.1 correlates with assertive clarity or assertive force. Second, I test whether the Portrait Divergence based typology presented in 4.2 allows to predict assertive clarity or assertive force. Third, I use a couple of annotation features as well as network-based features to create alternative typologies and test whether these can be used to predict assertive clarity or assertive force. The first approach is supervised; the other two are unsupervised.

5.3 Annotation Based and Global Network Features

This section focuses on annotation-based features (e.g. number of positive evaluations, number of encoding oppositions, see Appendix A2) as well as on standard network features (e.g. in-degree variance, see A2) in order to predict assertive clarity and assertive force. Network features were extracted with Python’s NetworkX library and, wherever it makes sense, normalized to network size (number of nodes). To annotation features I applied length normalization. All features were rescaled to a range between 0 and 1 using min-max scaling.

For both target variables I followed the same procedure. First, I conducted a Lasso regression using L1 regularization to enforce sparsity. The optimal regularization strength (alpha) was selected using 5-fold cross-validation. In a second step, I applied hierarchical clustering to group the documents based on these informative features. I used agglomerative hierarchical clustering with Ward’s linkage determining the optimal number of clusters by silhouette scores and visual inspection. Then, I used the same method to calculate Precision, Recall and F1-score (assigning cluster labels by majority vote) as explained in section 4.2.

The results for assertive clarity are shown in Figure 11 and Table 6.

Figure 11

Table 6

Evaluation of Hierarchical Clustering based on Annotation Based and Global Network Features, “LOW” and “HIGH” refer to assertive clarity. ). P – Precision, R – Recall, F1 – F1-Score.

	P	R	F1	SUPPORT
LOW	0.81	0.94	0.87	18
HIGH	0.93	0.76	0.84	17
accuracy	0.86	0.86	0.86	0.86
macro avg	0.87	0.85	0.86	35
weighted avg	0.87	0.86	0.86	35

As we can see, the clusters identified by the clustering reflect the two types of texts – those with low assertive clarity and those with high assertive clarity – very well. There are four clusters, three of which are homogeneous or nearly homogeneous. The third cluster (in red) is a mixed cluster consisting of ten texts with low and four texts with high assertive clarity. Only five texts out of thirty-five are misclassified, based on majority vote cluster labels.

Figure 12 shows the average feature values by cluster. Since there are three features that count the appearance of a certain type of triad (a group of three nodes and their linkage pattern, Snijders 2012), these three types are represented in Figure 13.

Figure 12

Figure 13

One observes several interesting things: First, certain features are almost exclusive to certain clusters (e.g. for cluster 1, triad 021C, which corresponds to an evaluated entity which itself evaluates another entity). Second, the profile of cluster 2 (high assertive clarity) stands out very clearly. It is similar to type B in the narratological typology (feature contrast and value contrast), because this cluster has high average values on positive and negative evaluations and shows the highest evaluative difference between contrastive entities (i.e. between entities with overlapping oppositions). The triad 120D, generally speaking, corresponding to an entity which evaluates two opposed entities and the triad 021U corresponding to an entity which is evaluated by two other entities are most relevant for cluster 2. Additionally, the text from Hollenstein paradigmatic for type B is in cluster 2. Third, Cluster 4 (low assertive clarity) scores highest on feature oppositions (in contrast to encoding or value oppositions, which are not among the relevant features). Finally, the third mixed cluster seems also to be indicated through a diverse mixture of features never scoring highest for single feature.

The results for assertive force are shown in Figure 14 and Table 7.

Figure 14

Table 7

Evaluation of Hierarchical Clustering based on Annotation Based and Global Network Features, “LOW” and “HIGH” refer to assertive force. P – Precision, R – Recall, F1 – F1-Score.

	P	R	F1	SUPPORT
LOW	1.00	0.50	0.67	12
HIGH	0.79	1.00	0.88	23
accuracy	0.83	0.83	0.83	0.83
macro avg	0.90	0.75	0.78	35
weighted avg	0.86	0.83	0.81	35

The seven clusters identified reflect the division between texts with low vs. high assertive force well. Again, we have several clusters that are homogenous and one bigger mixed cluster, cluster 5 (brown), in which six texts are misclassified according to majority vote. Although the dataset is clearly disbalanced (12 LOW vs. 23 HIGH), a pure LOW-cluster, cluster 3 (red), has been identified.

Looking at the feature importance in Figure 15, several things stand out. Again, there is a cluster including the Hollenstein text which is strongly marked through evaluative difference between contrastive entities (cluster 4) and a high proportion of triad 120D. Cluster 6 is marked through implicit evaluations. As in the clustering before, the mixed cluster seems to unite ‘unobtrusive’ texts that do not stand out regarding a single feature.

Figure 15

The LOW-cluster (cluster 3) is marked through feature oppositions and the proportion of weakly connected components, i.e. subgraphs where one can reach any node from any other node by ignoring edge direction. This suggests that a less connected evaluative structure with a number of subgraphs in this sense – oppositions whose nodes are not themselves evaluated or evaluators that are not evaluated themselves – is indicative of low assertive force.

6. Summary

The paper started from the assumption that evaluative text structures, i.e. complex patterns of evaluations and oppositions which emerge from evaluations within the text, play a central role in the generation of implicit messages in literary texts. This assumption turned out to be very fruitful. Based on the annotation of literary evaluations and oppositions in thirty-five narrative fictional texts, the paper showed that representations of evaluative structures in the form of directed networks allow to already visually discern certain text structure types. Moreover, the paper showed that quantitative methods can be used to compare and cluster these networks in meaningful ways, which allows to predict to a certain degree whether a narrative has a (clear) implicit message. An approach using a variety of annotation and network-based features proved in this regard more successful than a conventional method of network comparisons like Portrait Divergence. The following features or combinations of features were particularly important for predicting assertive clarity based on annotation and network-based features: evaluated entities that themselves evaluate other entities (triad 021C), an entity which evaluates two opposed entities (triad 120D), an entity which is evaluated by two other entities (triad 21U), evaluative difference of contrastive entities, the frequency of negative, positive, and implicit evaluations. Predicting assertive force involves for the most part the same features but also includes feature oppositions and the number of weakly connected components, i.e. the number of subgraphs of connected nodes when edge directions are ignored.

Of course, these results are only a first promising step of an analysis of evaluative text structures. Further research should primarily overcome two limitations of the present paper. On the one hand, it would of course be desirable to work with much larger data sets, for which automatic recognition of the phenomena in question, for example with the help of LLMs, would be a prerequisite. On the other hand, the analysis here has focused on static networks, which seem appropriate, especially for shorter texts. However, if the analysis is to be extended to longer texts such as novels, it will be advisable to consider dynamic network models that can capture significant changes in the evaluation structure as the plot progresses. The narratological typology sketched above has pointed to a case with the ‘disposition-outcome’ type that could also be relevant for longer texts. A further interesting topic for research would of course be to predict not only which texts have an implicit message, but also to predict the topic of this message or ultimately the implicit message itself. The findings of this paper suggest that certain types of oppositions may be highly indicative in this regard. However, at this point at the latest, we enter an area where different interpretations of the texts have a significant impact. It is not without reason that literary theorists Lamarque and Olsen (1994) coined the term ‘implicit thematic statement’ for the implicit message of a literary text. This makes it clear that the boundaries of literary analysis, but also the boundaries of the problems involved, are likely to be fluid here.

Data Accessibility Statement

Data and code supporting this article are available in a dedicated publicly accessible github repository: https://doi.org/10.5281/zenodo.17591665. The annotation data is (due to copyright restrictions partly) available here: https://doi.org/10.5281/zenodo.16570556.

Competing Interests

The author has no competing interests to declare.

Author Contributions

Benjamin Gittel is the sole author.