Making Sense of Italian Legacy Resources: Integrating ItalWordNet and PAROLE-SIMPLE-CLIPS for Sense Annotation

The ELEXIS Infrastructure and the Italian Contribution

In recent years, the European Commission invested in a multi-national project dedicated to bridging the gap between traditional lexicography and modern language technology, by fostering cooperation and data exchange among major European lexicographic initiatives: the European Lexicographic Infrastructure (ELEXIS).² One of its primary goals was to overcome the fragmentation of the European lexicographic landscape by establishing common standards and shared infrastructure (Krek et al., 2018). Within this framework, a particularly active research thread was dedicated to the creation of word sense resources and technologies aimed at improving multilingual Word Sense Disambiguation (WSD) methods and/or at providing new benchmarking data.

Recognizing that small-to-medium scale, highly curated datasets are of immense value for model fine-tuning, evaluation, and tasks requiring explicit replication of human knowledge, the project set out to create a 10-language parallel corpus, annotated up to the sense level and correlated with sense inventories derived from expert-compiled lexicographic dictionaries (Martelli et al., 2021). This paper discusses the experience of creating the Italian sub-corpus of this initiative. Going beyond the project’s original scope, it proposes a methodology that is transferable to other languages and contexts. In particular, it demonstrates how existing expert-curated lexicographic resources, rather than being superseded by modern approaches, can be systematically reused to produce high-quality sense-annotated data for languages where such resources are scarce.

High-quality semantic resources for Italian

Despite the prestige of Italian lexicography, there is a scarcity of open-access, digital lexical resources encoding sense-level information that offer both broad coverage and high-quality human curation. For contemporary standard Italian, several major legacy and reference dictionaries, such as Salvatore Battaglia’s Grande Dizionario della Lingua Italiana (GDLI, Battaglia 1961–2002) and modern large‑scale works such as Tullio De Mauro’s Grande dizionario italiano dell’uso (De Mauro, 1999–2000, 2016–), are typically accessible through online search interfaces but, despite their scholarly authority, are not released for direct reuse in NLP or AI applications. This is mostly due to technical and representational complexities, restrictive copyright policies and a protective stance among Italian publishers. As a result, to the best of our knowledge, no legacy Italian dictionary is currently available in an open, machine-readable, and actionable format.³ Consequently, existing resources are often proprietary, domain-specific, or lack the granularity required for sophisticated semantic analysis.

Unlike other languages represented in the parallel corpus, for which “traditional” dictionaries were available in machine-readable and actionable formats, no legacy Italian dictionary is currently accessible for open research or integration into AI. While BabelNet can be considered as the most extensive sense-level resource for Italian, it cannot be considered a lexicographic-level lexical resource since it was created automatically by merging various WordNet-like resources, by linking to Wikipedia, and by machine translating lexical gaps (cf. Navigli and Ponzetto 2012, Navigli et al. 2021). Though invaluable for scale, it is generally considered not a “gold standard” and presents several practical challenges, such as: sense redundancy (i.e. duplication of entries where mapping between resources failed), lack of a clear human-verified footprint for every entry, uneven sense granularity. Unlike English, however, which also benefits from very large semantic resources like the English WordNet, aka Princeton WordNet (Fellbaum, 1998), Italian lacks a unified, large-coverage sense inventory.

The sense-annotated corpus discussed in this paper was meant to be a step towards closing the gap in semantic resource availability through a bottom-up approach, by reusing and integrating two complementary, highly curated resources (described in sections 2.1.1 and 2.1.2 below).

Although originally conceived to support multilingual WSD within the ELEXIS project, this richly annotated linguistic corpus is suitable for reuse by multiple research communities, particularly as it comes in community shared formats and tagsets (CoNLL-U) and in open licenses. For NLP practitioners, for instance, the dataset provides high-quality, expert-curated annotations that can be exploited in several tasks for various purposes, for example for supervised fine-tuning, benchmarking, and probing the semantic competence of language models. At the same time, the resource is equally relevant for linguists interested in various approaches: translation studies, comparative linguistics, lexical semantics, syntax. It can be reused to add other levels of linguistic annotation that may be used in conjunction with the pre-existing ones for unforeseen studies, as in (Čibej, 2025), which used a version enriched with dependency relations for syntactic profiling and cross-linguistic comparison, and thus demonstrated its potential for reuse beyond its original scope. This confirms that richly annotated linguistic datasets support a wide range of analytical perspectives, many of which emerge only after data publication, and reinforces the importance of designing resources with openness, interoperability, and long-term reuse in mind.

2.1 Legacy resources

2.2 Newly created resources

3.1 Corpus creation and refinement

The parallel data for the ten languages were originally derived centrally from WikiMatrix (Schwenk et al., 2021), a pre-existing resource of parallel sentences extracted from Wikipedia and widely used in the NLP community, especially for machine translation. Although WikiMatrix is not deposited within the CLARIN network, it served as the source from which the set of parallel sentences was derived. Ideally, parallel sentences were collected for all ten languages; in practice, priority was given to sentences for which translations in as many languages as possible were available. The remainder, in the case of the Italian sub-corpus, was supplemented with manual translations by one of the authors. After all revisions, the parallel raw corpus consists of 2,024 sentences and approximately 35,000 tokens per language.

The preparation of the dataset followed a multi-layered approach to ensure high-quality data and cross-lingual consistency, in accordance with the Universal Dependencies (UD) standard. The raw dataset underwent an extensive phase of manual correction and linguistic enhancement according to specific formal and lexical-semantic criteria, in order to ensure high-quality translation data. As these preliminary phases followed the general project workflow, we refer readers to Martelli et al. for a comprehensive account of the common data preparation workflow (2021).

We briefly review the process here, focusing on issues specific to the Italian sub-corpus. First, the quality of the translations was checked, and revisions or additions were made by one of the linguist authors where needed. Most of the work consisted of adding missing translations, correcting punctuation and morphological errors, removing notes in square brackets and other non-essential content typical of Wikipedia pages, and marking for exclusion sentences that contained too many Named Entities (NEs), that is, more than three, as established by the project-level guidelines. This refinement was essential to ensure translation quality, as well as grammatical and orthographic accuracy, in preparation for subsequent annotation.

Subsequently, the corpus was automatically annotated for four linguistic layers with UDPipe (Straka et al., 2016), following UD guidelines and recommendations: tokenization and subtokenization, lemmatization, part-of-speech (POS) tagging, and morphological analysis. Following the Italian UD guidelines, subtokenization was applied to complex forms such as articulated prepositions (e.g., nella ‘in the.DET.f.s’ decomposed into in ‘in’+ la ‘the DET.f.s’) and verbal forms with enclitic pronouns (e.g., mangiandolo ‘eating-it.DET.m.s’ decomposed into mangiando ‘eating’ + lo ‘it.DET.m.s’). The resulting Italian sub-corpus contains 39,036 tokens, and 6,380 lemmas (see the breakdown statistics for content words in Table 1 below).

To ensure accuracy, these automatically annotated morphosyntactic layers were then manually validated and corrected where necessary. Manual validation was carried out by expert linguists to ensure adherence to both project-level guidelines and Italian-specific UD recommendations. Notable deviations from the UD specifications concerned abbreviations and acronyms. Acronyms such as U.S.A. or UE were treated as single words, whereas units of measurement were tagged as symbols (SYM). Foreign words were tagged as such primarily when they appeared in titles or long incidental expressions; otherwise, they were assigned the appropriate POS, especially when they were commonly used in everyday or technical language. NEs also required special treatment. Unlike in many standard WSD datasets, both general concepts and NEs were targeted for annotation in order to provide a more comprehensive semantic representation of the text. This choice required further correction of the morphosyntactic annotation, especially in the case of multi-word NEs containing non-content words, adjectives, and adverbs, since each component of the NE had to be tagged as PROPN in order to facilitate its identification during the subsequent semantic annotation and its separate treatment according to the guidelines. Most errors in the Italian dataset involved lemmatization, especially in the case of homographs, OOV items, and infrequent irregular forms; consequently, lemmatization required substantial manual effort.

As regards the correction of part-of-speech tagging, specific UD Italian practices were applied, such as tagging possessive adjectives as determiners and using the AUX tag for copulas, which in practice meant correcting the verb essere ‘to be’ to AUX in most cases. Adjectival participles also required careful manual revision, as they were tagged as verbs when used predicatively and as adjectives when used attributively.

Finally, each content word in the corpus (adjectives, adverbs, verbs, and nouns) was annotated with a corresponding sense specifying its meaning in context, taken from the previously compiled companion sense inventory (see Section 3.2 below). Table 1 reports UPOS statistics, including sense coverage. Table 2 shows the statistic for MWEs.

3.2 Constructing the Italian Sense Inventory

A critical step in the workflow was the creation of a sense inventory for all lemmas of content words identified in the corpus. Whereas other language teams could rely on a single comprehensive legacy dictionary, the Italian context required a different approach. Due to the lack of an open-access, machine-readable traditional dictionary, the inventory was derived from the two most authoritative computational lexical resources available for Italian: IWN and PSC.

A first draft of the inventory was created by semi-automatically merging the senses of the lemmas occurring in the corpus as encoded in these two resources. The technical procedure for accessing the original databases, using the MySQLdb Python module, and the subsequent merging logic were initially tested in two undergraduate dissertation projects supervised by the authors. This merging process relied on pre-existing manual mappings of sense subsets produced by the original compilers of the resources (Roventini & Ruimy, 2008; Roventini et al., 2007), as well as on an automated alignment procedure based on similarity measures between PSC definitions and IWN glosses, together with the exploitation of their ontological type systems. Details of these procedures are described in the above-mentioned dissertations (Pisani, 2021; Poli, 2021). However, this version was incomplete because it relied on an initial noisy and linguistically unannotated version of the ELEXIS dataset, due to the time constraints of the students involved.

Before starting the annotation process, once the raw sentence dataset had been finalized and manually validated, the mapping process was repeated and the mappings were manually checked by the authors of this paper. Even so, not all lemmas occurring in the corpus were covered by the sense inventory. This was due partly to the limited coverage of the original resources, PSC and IWN, and partly to automatic annotation errors in lemmatization and POS tagging.

Instead of completing the inventory and supplying senses for missing lemmas by resorting to other open resources such as Wiktionary, which would have introduced problematic issues of consistency and sense granularity in relation to the existing entries and would have required further manual revision, it was decided to use BabelNet senses where these were suitable and of sufficient quality, or to allow annotators to create new senses when needed (see Section 3.3 below for details).

The initial sense inventory uploaded into the chosen annotation tool contained about 11,000 senses drawn from merging IWN and SIMPLE. The inventory is formatted as a TSV file and covers mainly nouns and verbs, with a smaller number of adjectives and adverbs, reflecting the composition of the original resources. Each enumerated sense was assigned a unique ID, together with its lemma and POS (UPOS, according to the UD tagset), and a definition or gloss. In addition, where available, each sense is linked to the original ID or IDs in PSC and IWN.

As a result of the annotation process, the sense inventory was enriched with 307 new senses (cf. Section 4 for some examples and Tables 3, 4, 5, and 6 below for overview statistics).

3.3 Annotation of the parallel Italian sub-corpus

Semantic annotation was conducted using LexTag,⁵ a web-based tool adopted for the annotation of all ten languages. Annotation guidelines were developed collegially at project level by expert linguists and lexicographers from the ten language teams. These were subsequently tested, discussed, and refined across the language groups during a first pilot annotation phase, which, for Italian, was carried out by the authors (cf. Pedersen et al., 2023, p. 14 for Danish). For the Italian component, some additional language-specific guidelines were defined to help annotators in unclear cases. This was particularly relevant for past participles used adjectivally, MWEs, and the procedures for entering new senses into the tool in order to enrich the sense inventory. More specifically, clearer criteria were established for several critical areas. First, drawing from past initiatives, such as PARSEME (Monti et al., 2017) cited above, guidelines for MWE identification were provided to help determine when a sequence should be annotated as a multi-word.

Second, specific instructions were given on whether metaphorical uses should be mapped onto existing literal senses or assigned a new dedicated sense entry, i.e., in cases of non-lexicalized metaphorical extensions. Third, guidance was given on selecting the most appropriate sense when the integrated inventory presented partially overlapping senses.⁶

As for sense selection, annotators were instructed to follow a fixed hierarchy in order to ensure consistency. First they should use the ELEXIS-IT sense inventory. If the relevant sense was not available there, they should fall back on BabelNet. Only as a last resort were they instructed to create a new sense and provide a definition. This hierarchy was designed to maximize the reuse of existing curated resources and to minimize unnecessary sense creation, both for preventing duplication and noise within the broader ecosystem of Italian online semantic resources, and for reducing delays in the annotation process.

The Italian annotation team consisted of four student annotators and three senior linguists. The annotators were advanced students in linguistics undertaking internships at the Istituto di Linguistica Computazionale (ILC-CNR). The senior annotators, who are also the authors of this article, carried out a pilot annotation to test both the guidelines and the tool. During the main phase, the four annotators worked in pairs, with each pair annotating half of the dataset so that every sentence was processed by two individuals. One of the authors acted as adjudicator, resolving disagreements and monitoring consistency.

This organization enabled internal discussion and allowed the calculation of Inter-Annotator Agreement (IAA). Following the standard metric for such tasks, a Cohen’s Kappa score of .72 was calculated for the Italian team, which is generally considered a good level of agreement for semantic annotation tasks. Agreement was higher for nouns, approximately .78, than for verbs, approximately .65. These results were achieved through an iterative process. Initial pilot annotations yielded lower agreement, approximately .55, which prompted the team to refine the language-specific guidelines, as briefly described above.