Machine Learning Assisted Conversion of Biographical Records Into Wikidata Triples

2.1 Machine Learning Tools

During the in-depth digitisation process, it was necessary not only to read the scans and create full texts, but also to recognise the individual dictionary entries and their subcomponents (the biographical entry itself, the person’s works, and the sources). Despite the printed nature of the dictionary, OCR methods proved insufficient for such advanced text segmentation. Therefore, the handwritten text recognition (HTR) method was used. The use of Transkribus⁴ for digitising the BLGBL was deemed unsuitable for two primary reasons. Firstly, the cost of digitising hundreds of pages would be prohibitive. Secondly, models created in Transkribus can only be used on that platform and cannot be exported and transferred elsewhere, resulting in a form of vendor lock-in. Therefore, the open-source software eScriptorium⁵ was selected for the digitisation.

Two models were developed to convert scans into full text. The layout segmentation model (Baránek, 2024b) was designed to recognise regions such as headers, page numbers, and text columns, as well as lines that mark the beginning of individual biographical entries and their subsequent paragraphs (Figure 1). This model displays an accuracy of 61.3%. The recognition model, developed to process mixed Czech-German texts, achieves 99.8% accuracy.

Figure 1

The main innovative tool is biography2wikidata (Baránek, 2024a), an NLP model for transforming acquired full texts into structured data (triples). Jaskulski et al., 2025, sought to extract such data from the Polski Słownik Biograficzny [Polish Biographical Dictionary] using LLM prompts, achieving a relatively high success rate. However, the objective of the Wikidata Triples project was not only to extract text strings, but also to assign Wikidata identifiers to them. Therefore, a custom machine learning model was created.

The biography2wikidata model is a text-to-text transformer based on Google’s mT5 small model. Prior to manual annotation, a pre-trained model was developed. Specifically, the pre-training involved converting authentic biographical data from Wikidata into text strings that adhered to the format employed in the BLGBL.⁶ Consequently, the pre-trained model successfully structured the introductory text of the biographical entries into meaningful units (claims), identifying elements such as name, occupation, birth and death dates and places, and assigning Wikidata identifiers to the most common entities. This approach eliminated the need to begin annotation entirely from scratch, as initial biographical records were annotated with the assistance of the pre-trained model.

Subsequent fine-tuning of the model was conducted using verified annotated texts from individual issues of the first BLGBL volume. The resulting model is based on a dataset containing more than 5,000 dictionary entries. The model annotates the full text of biographical entries (Figure 2) using Wikidata identifiers, thereby ensuring that the extracted data aligns structurally with Wikidata as closely as possible. This enables a direct comparison between the content of a dictionary entry and its corresponding Wikidata entity. According to the transformer, the model displays a loss value of 0.39.

Figure 2

However, of greater relevance is the number of valid statements that the model is capable of extracting from the input. The evaluation test was performed on 100 unseen entries from the second BLGBL volume (Table 1). The data indicate that the model accurately retrieves approximately 60% of basic statements and 20% of qualifiers, resulting in an overall retrieval rate of 50% for both basic and qualifier statements. While these figures fall short of 100% accuracy, the model segments the text effectively and significantly accelerates data annotation. When using the model, manual work is confined to verifying annotations and validating the correct assignment of identifiers to entities with the rare occurrence. The presence of such rare entities also precludes the possibility of achieving 100% model accuracy.

2.2 Data Modelling

The primary concern in annotating the texts of the biographical entries was data modelling. For each entry, three interrelated questions were addressed: what information should be modelled, how it should be modelled, and how much detail should be included. The following examples illustrate each of these questions.

Firstly, what information should be modelled? The biographies in the BLGBL are concise and frequently composed of incomplete sentences. This is a significant advantage, as it allows complete information to be modelled for a substantial proportion of the entries. However, it is important to note that the BLGBL also contains entries that include text deviating from the subject and not directly related to the individual. This phenomenon is particularly evident in entries concerning entrepreneurs, which may include information about the factories they founded, the locations of factory branches, and the number of employees. Such information relating to factories rather than individuals was deliberately excluded from the modelling and annotation.

Secondly, how should the information be modelled? Simple statements, such as date and place of birth, or the names of parents or descendants, are straightforward to model. Appropriate properties for such claims exist on Wikidata, and qualifiers can be used to express nuances. However, more complex information can often be modelled in multiple ways. The intention was to follow the logic of Wikidata. Yet this is complicated by inconsistent data modelling practices within Wikidata itself. For instance, in order to identify a school where a teacher worked, three different properties are used as qualifiers: employer (P108, 799 statements), workplace (P937, 352 statements), and location (P276, 328 statements).⁷ It is not possible to simply choose the most commonly used qualifier, as historically university teachers often held unpaid positions as Privatdozent (Durdík, 1893). For this reason, the workplace property was considered the more reliable option when annotating BLGBL entries.

The situation is further complicated by the fact that Wikidata is a dynamic, evolving system. For example, during the course of the project, the community decided to remove the ‘of’ property (P642) and replace it with more specific properties.⁸ To maintain consistency with Wikidata, the annotation of the BLGBL full text has been adjusted accordingly.

Thirdly, how much detail should be modelled? A typical example concerns information about a person studying a specific faculty at a university. Modelling requires a balanced approach between two fundamental objectives: on the one hand, achieving the highest possible fidelity to the source data, and on the other hand, ensuring the efficiency and functionality of the model. Modelling at the university level enables faster machine learning than modelling specific faculties. It should also be noted that highly specific items are frequently absent from Wikidata. Consequently, the level of detail (granularity) was determined by the principle of efficiency relative to the time invested.

3.1 Items Not Existing on Wikidata

The basic matching revealed that a corresponding entity on Wikidata existed for only 73% (3,693 of 5,065) of biographical entries, despite the fact that half a century has passed since the publication of the initial BLGBL volume. This finding raises a crucial question: Do the 27% of personalities “lost in the gap” share any common characteristics? In other words, is there any definable gap on Wikidata?

A detailed analysis, encompassing life dates and occupations, supports this supposition. The majority (75%) of these individuals were born during the 19^th century, and nearly a quarter (23%) of all of them were employed in education as teachers, university professors or school principals – most of whom were active around the turn of the 19^th and 20^th centuries. Other significant professions (each representing 3–5%) included factory owners, actors, doctors, regional writers, and regional researchers.

These data reveal a significant bias in the reflection of historical reality. Over the course of the 20^th and 21^st centuries, the perceived notability of some of these professions has declined, meaning they may no longer be considered notable enough from a contemporary perspective. However, during the period of nationalization and intensifying Czech-German national tensions at the turn of the 19^th and 20^th centuries, teachers, school principals, and regional cultural figures played a pivotal role in disseminating ideas and shaping socio-cultural identities and boundaries (Zahra, 2008). Moreover, during the Industrial Revolution, the direct influence of individual industrialists on society was far greater than in contemporary times, when the decisions of large corporations are made not by individuals but rather by boards of directors.

In conclusion, the data demonstrate that the evaluation of notability on Wikidata is characterised by a predominantly contemporary (ahistorical) perspective, rather than by an accurate reflection of the historical reality of who was truly notable during the period in question. This conclusion aligns with the findings of other studies analysing content gaps (Das et al., 2023). Baker and Mahal (2024) even argue that ‘the Wikidata model favours a presentist and event-based temporality’. As a free knowledge base, the content of Wikidata is largely based on the current and more or less random interests of its editors. In contrast, academic institutions produce biographical dictionaries according to predetermined criteria. These are certainly not free of bias, but still represent a more systematic and objective approach. Academic biographical dictionaries can therefore help to at least partially eliminate the biases and historical content gaps present in Wikidata.

3.2 Contents of Existing Items

A comparison of the 73% of dictionary entries that have corresponding entities on Wikidata enables the determination of the completeness of the latter. From another perspective, the analysis addresses the extent to which academic dictionaries (the BLGBL in this case) still possess untapped potential for enriching Wikidata. An examination of the fundamental data (Tables 2 and 3), encompassing birth and death dates and places, reveals that this information is consistent between the dictionary and Wikidata in 75% of cases.

However, between 5% and 6% of entries in the BLGBL contain more precise information about the date of birth or death than that provided by Wikidata – that is, the BLGBL contains the full date, whereas Wikidata records only the year. The BLGBL has even greater untapped potential with regard to places of birth and death: 16% and 21% of its entries, respectively, contain information on the place of birth and death, while this type of information is entirely absent from the corresponding Wikidata entities. Finally, a notable proportion of entries contain information that contradicts the information on Wikidata; the accuracy of these claims should be verified through further historical research.

The potential of the BLGBL remains even more underutilised in relation to other properties. For instance, the first BLGBL volume contains 1,968 entries that correspond to Wikidata entities and together provide in total 2,626 claims about the schools attended. Of these claims, 1,999 (76%) are entirely absent from Wikidata – that is, there is neither a more nor a less precise value. A similar situation is evident in properties related to academic degrees, function or position, membership in religious orders or any organisations (such as associations), and awards received (Table 4).

In conclusion, the collected data indicates that, despite Wikidata’s 13-year existence, the potential of academic dictionaries that have not yet been digitised as full texts remains largely untapped. A case study of the first BLGBL volume reveals that Wikidata entities are missing for 27% of dictionary entries; that existing Wikidata entities lack 14% of the birth and death data available in the BLGBL; and that, for other types of information, the rate of absence ranges between 80–90%.

In the case of the BLGBL, the main obstacle to data transfer was the fact that it was available only as scanned images. For other biographical dictionaries, the barrier may lie in restrictive copyright policies that prevent the dissemination of data. Some dictionaries may have already been digitised as full texts, but their content remains behind a paywall (e.g. Grove Music Online¹⁰ with Wikidata ID P8591).

3.3 Data Publishing

Following a thorough analysis, the BLGBL dataset was published in October 2025 in collaboration with Collegium Carolinum as a MediaWiki-based online encyclopaedia under a Creative Commons BY-NC-SA 4.0 licence (https://blgbl.de). All BLGBL entries were linked to Wikidata. In order to facilitate the comprehensive integration of the online version of the BLGBL into the LOD ecosystem, new entities were created on Wikidata where none previously existed, and existing Wikidata entities were enriched with the BLGBL identifier (P13876) and at least basic biographical data (birth and death details).

Furthermore, a SPARQL endpoint was established (https://blgbl.de/sparql),¹¹ enabling the public to construct their own queries on the digitised structured data.