Wikibase Solutions for African Literary Metadata

(1.1) Invisibility of African literary and oral materials

The reasons for the exclusion of certain kinds of African literature from library and archive catalogues are complex and multiple. Primarily, this is a consequence of the ways cataloguing systems, which owe their designs largely to 19th century American and European library science, were biased towards the book object as the unit from which metadata was generated (Holden, 2020, p. 84). Given the fact that until the early 19th century most sub-Saharan African cultural expression was orally transmitted, the consequence of this bias has been extensive. Most obviously, the oral expressive forms that dominated the continent before the onset of book culture have been relegated to the fields of sociology, anthropology or ethnography, rather than to literary studies. Secondly, the fact that cataloguing standards were developed to describe collections of books has created a problem for African expressive culture, much of which has been circulated orally or in a variety of print forms that do not easily conform to the book-driven conception of the ‘work’ – one which is determined by ‘Author-Title’ metadata employed by most libraries and cataloguing standards.

The consequence of this has been detrimental to the archiving and visibility of African literatures, both written and oral. How, for example, does one catalogue the metadata of oral literatures that are not authored by any one person or by an identifiable collective? The colonial archive dealt with this by ascribing authorship to the transcribers of oral works, or to the ethnographers that later recorded or filmed their performance – a practice of attribution that has contributed to the erasure of African authorship in the formal archive and to a clear practice of European folklorists appropriating African materials. Moreover, lack of linguistic expertise has resulted in African cultural forms being erased as they are catalogued under English terms and ontological structures. For example, the Zimbabwean language, Shona, has a strong tradition of oral praise poetry and this tradition has a clear structure of praises, including clan praises, personal praises and boasts. Under personal praises is a form known as nhetemo dzokunyaradza mwana, which translates directly as ‘phrases of comfort for a baby’, but which is sung, and thus can be translated into English as ‘lullaby’. A lullaby might be categorised as a song or a children’s song, but this would miss the fact that these soothing songs to babies are part of the praise tradition in Shona (Hodza and Fortune 1979).

A further factor contributing to the invisibility of metadata about African print literature in the formal archive and catalogue has to do with the relationship between mode of publication or distribution of a work and the metadata it accrues. From the late 19th century through to the late 20th century, the dominant form through which African literature was published and circulated was in newspapers, magazines and pamphlets. Libraries seldom catalogue the content of such ephemeral print materials, thereby obscuring the literary content that is embedded in these forms. An illustrative example can be found in the publication of 300 serialised Swahili novels and novellas in two Kenyan-Tanzanian newspapers between 1960 and 1980 (Taifa Weekly and Baraza Weekly).³ Of these (exactly) 300 serialisations only 57 appear in the World Catalogue (WorldCat), which is the largest platform for library metadata from thousands of libraries across the globe.

(1.2) A Wikibase for African Literary Metadata

Because of the diversity of the materials that ALMEDA is cataloguing – from print ephemera (such as market pamphlets, newspapers, and magazines) to performance-based works (theatrical works, street performance), to online materials (social media written or video works), and potentially in any number of African languages – the project needed to work with a flexible linked open data solution. The data model had to allow for the creation of metadata descriptions of various literary forms generally not captured in bibliographic databases and must have the capacity for multilingual functionality. Since Wikibase met these requirements, thereby offering the possibility to build a flexible and durable knowledge base whilst also allowing for easy data management for researchers with no coding skills, the decision was made to use this software for the ALMEDA repository.

The project made the further choice to use a private Wikibase instance for its repository rather than to create, more simply, a project within Wikidata itself (Rossenova et al., 2022). While Wikidata’s extensive reach and pre-existing modelling would have allowed the project to begin entering collected metadata on literary materials easily and quickly, there were several reasons as to why the decision was made to operate ALMEDA as a separate Wikibase project.

First and foremost, ALMEDA is an academic project that must be held accountable for the accuracy and veracity of its data: by allowing data to be editable by any Wikidata community member, such veracity could not be guaranteed. Wikidata addresses this problem with their notability criteria by requiring entities to be verifiable with legitimate and publicly available references, such as university-level textbooks, reference books, academic journals, or newspapers.⁴ However, the informal nature of the materials that ALMEDA is collecting and the fact that these materials do not as yet exist in verifiable databases make it difficult to provide such proof. This highlights a problem that emerges when creating structured and verifiable data on highly informal and as yet uncatalogued material: when trying to correct absences or data inequality in global databases, the burden of proof remains on existing formalised data, in the form of external identifiers or references, to prove the veracity of an entity.⁵

A further problem with Wikidata is that the knowledge schema behind many of its classes for literature and culture has inherited the structural and classificatory problems of the very standards that the ALMEDA project wishes to interrogate. One example of this in Wikidata is the genre ‘oral literature’: though a sub-class of ‘literature’, Wikidata nevertheless connects this to the Library of Congress external identifier ‘Folk Literature’, which is a classification that the ALMEDA project rejects.⁶ By using a private Wikibase instance, instead, ALMEDA has been able to determine the configuration of the knowledge base itself and to build a data model based on the inclusive ontology for African expressive forms developed in the project.

(1.2) Wikibase for FAIR and CARE data

Despite having made the choice to work with what might be perceived as a siloed, private Wikibase, as opposed to the community-driven Wikidata, the ALMEDA project is committed to investing in its data being as interoperable as possible. By aligning the structure of the ALMEDA Wikibase with Wikidata when possible, we aim to enable federated searches in the future across the repositories and to share our data, where possible, with Wikidata.⁷ This is part of the project’s commitment to implement FAIR (Findable, Accessible, Interoperable and Reusable) principles of data management.

The CARE principles⁸ (Collective Benefit, Authority to Control, Responsibility, Ethics) of data management are also core to the ALMEDA project, given that a significant amount of data relevant to this project relates to cultural heritage materials. To ensure that data is for the ‘Collective Benefit’ of the communities it represents or emerges from, ALMEDA works with researchers from within relevant communities, guaranteeing that the kinds of data captured on the materials are curated with attention to the inherent cultural knowledge models and linguistic logics of the material.⁹

Furthermore, Wikibase enables the perpetual editability of all data collected in ALMEDA: this allows for continued conversations between researchers, communities and other stakeholders, which in turns means that by seeking feedback and collaboration with communities in the structuring and presentation of data, ALMEDA will enable communities to have the ‘Authority to Control’ the data collected. Through the structure of an appointed and named advisory editorial board, ALMEDA will review all data with the expertise necessary to take ‘Responsibility’ in transparent and ‘Ethical’ ways for how that data is presented and for what purposes. The largest contribution that ALMEDA is making towards the CARE principles is to ensure that African language forms are included in the ontology in the languages that these forms are created in.

(2.1) Modelling

ALMEDA is developing an ontology that models African literary expressions in ways that both resist the impact of colonial library science on the knowledge schema for these works and that allows African language terms their own space in the knowledge model without subsuming them under an English (or other colonial language) term. At the same time, this ontology must operate multilingually, to enable accessibility. To achieve this, the project team has worked extensively with researchers on creating a structure for a wide range of ‘Forms of the Creative Work’ relevant to the project. We are also working with African language experts who are compiling translated lexicons of these forms of literary expression and adding new forms from their language-context. It is important to note that this is a fluid ontology, which Srinivasan and Huang define as “flexible knowledge structures that evolve and adapt to communities’ interest based on contextual information articulated by human contributors, curators, and viewers…” (2005, p. 193). By virtue of our approach, the inclusion of as-yet-uncatalogued forms in the ALMEDA database requires us to keep our ontology open to review as new forms may challenge the structure we are working with.

The basis for ALMEDA’s ontology is the Library Reference Model, object-oriented version (LRMoo), “intended to capture and represent the underlying semantics of bibliographic information and to facilitate the integration, mediation, and interchange of bibliographic and museum information.”¹⁰ ALMEDA’s domain of African literary and cultural materials in print, audio, video or digital formats is of relevance to both the library and museum worlds, and the decision to work in alignment with the LRMoo formal ontology ensures future interoperability with these sectors. However, where LRMoo allows for the theoretical modelling of literary works on four levels (Work, Expression, Manifestation, and Item), many of the materials ALMEDA is modelling are only available as single expressions, manifestations, and even items. As such, and in the interests of best use of resources, ALMEDA has adopted Wikidata’s two-level model for ‘works’: that being, ‘work’ and ‘version, edition, translation’.¹¹ This simplifies the modelling and data upload processes without losing the quality of the data.

Besides drawing on formal ontologies such as LRMoo and pre-existing Wikidata models such as WikiProject Books, ALMEDA is also introducing project-specific elements to the ontology. The most significant of these being the aforementioned types we include under ‘Form of the Creative Work’, which, while fluid and open for further inclusion of new forms, is a domain-specific list that is unique to the ALMEDA project. These types and their language-independent object properties have an instrumental role in the ontology, allowing discoverability of literary expressions across language and cultural contexts.¹²

While ALMEDA’s model for ‘Forms of the Creative Work’ constitutes a new way of structuring data in the domain of African literature, we nevertheless have designed it to maximize interoperability with existing databases and sectors. This is important for the sustainability and usability of ALMEDA’s data, and also because a significant amount of our data, while on informal and ephemeral materials, is likely to intersect with more formal literary data. Some published authors, for example, published works in small magazines, while others have written plays, performed in street environments, based on their published novels. The ALMEDA project highlights the fact that the intersections between formal and informal data are themselves porous and messy. We seek to link our data to existing formal data – such as that found in library catalogues – rather than to create a database disconnected from the formal literary field.

(2.2) Wikibase implementation

ALMEDA uses the open-source software Wikibase Suite Deploy to self-host its linked open knowledge base.¹³ The software is divided into smaller units in the form of Docker containers, self-contained packages that include the code, settings, and libraries needed to run a piece of software. Each container fulfils a specific function within the complex structure of Wikibase, such as functions related to the interface, data repository, data entry, search functionality, query services, and web traffic. Compared to Wikibase Cloud,¹⁴ a ready-to-use installation of Wikibase hosted by Wikimedia Germany, Wikibase Suite Deploy requires more technical expertise and computational resources to implement but offers greater control over the instance and customization of its functionalities.

One example of this customization is the possibility to install MediaWiki extensions.¹⁵ These extensions, developed by members of the Wikibase community, provide additional functionalities and even new data types. For example, the WikibaseQualityConstraint extension can be used to define constraints on the properties and check whether they are met in statements, informing the editors of items where properties are incorrectly used.¹⁶ Thanks to Docker, the main components of the Wikibase instance can be deployed with the predefined system configuration and environment, but for the installation of extensions a lot of testing work is required.

A newly created Wikibase instance is a blank canvas that requires extensive set-up and configuration to fit a project’s needs. After the first launch of the instance, the environment variables need to be specified in order for the software to work correctly. These settings specify among other things the communication between the interface and the database, the web address where the Wikibase can be accessed, and the location of services such as the SPARQL query service for the retrieval of data. Other configuration steps include setting up user groups and rights as well as adding support for languages. The majority of the languages relevant to ALMEDA were already provided for in Wikibase. Additional languages can be added as long as the language has an IETF language tag.¹⁷ For ALMEDA, this meant that the Bantu language Lulogooli could be added, but not informal urban vernacular languages that do not have such a tag, one example being Sheng, a mix of English, Swahili and other Kenyan languages spoken in Nairobi.

The next step involves translating the ontology into the Wikibase data model¹⁸ and populating the instance with items and properties. In Wikibase, items are uniquely identified entities that, in the context of ALMEDA, could represent a creative form, an author, a country, or an event. Each item has a unique identifier, starting with the letter ‘Q’ followed by a number, as well as human readable labels, descriptions, and aliases in different languages. Properties define specific characteristics or relationships of an item. They also have a unique identifier (starting with the letter ‘P’ followed by a number), as well as multilingual labels, descriptions, and aliases. Knowledge about an item is expressed through statements, which follow a subject-predicate-object structure: the subject is the item, the predicate a property, and the object is a value, which could for instance be another item, a date, or string. To illustrate this, we can use an example from the ALMEDA Wikibase: the Namibian writer Ndawedwa Denga Hanghuwo who received an award called the Bank Windhoek Doek Literary Award For Fiction (Figure 1). To the item ‘Ndawedwa Denga Hanghuwo’, a statement is added using the property ‘award received’ and linking it to the item for the award.

Figure 1

Furthermore, the Wikibase model allows adding contextual information to statements through qualifiers and references. Qualifiers can be used to further specify a statement (e.g. using the qualifiers ‘for work’ and ‘point in time’ to express for which work the author received the award and the time of the rewarding as seen in the example presented in Figure 1). References refer to the source of the information. They can take different shapes and refer to academic articles, datasets, encyclopedias, as well as web pages. In the example given here, the webpage of the literary magazine is provided as a reference. Here the work by Ndawedwa Denga Hanghuwo is published alongside the author’s biographical information.¹⁹

A major challenge when setting up a Wikibase instance, is the time-investment required in implementing the data model based on the project’s ontology and in populating the instance with properties and data items, such as countries and languages. To facilitate this process, the project reused properties and basic metadata items from the public Wikidata, though this was not always straightforward. In Wikidata, for example, ‘human settlement’ items often include varied and sometimes haphazard definitions of ‘megacities’, ‘cities’, ‘big cities’ and ‘towns’. Johannesburg,²⁰ for example, is an instance of both a ‘city’ and a ‘big city’, while Nairobi²¹ is (in addition to being an ‘administrative territorial entity of Kenya’) a ‘big city’, but not a ‘city’ – both cities have similar central populations of around 6 million inhabitants.²² The messiness of data in the hands of an ever-changing community of editors is a problem for research-based data repositories. As such, ALMEDA has decided to only use one term, ‘human settlement’, to determine all urban and rural locations and we are populating these items with data from GeoNames rather than Wikidata because of GeoNames’ standardization and stability of data.²³

Other data was used to ‘kick-start’ the ALMEDA project, such as literary or related occupations, existing African author data, and data on existing festivals, events, awards and venues. Once the project members agreed on which data could be reused, two programming packages – WikibaseIntegrator²⁴ and Pywikibot²⁵ – were used to retrieve these properties and items and insert them into the ALMEDA instance. The imported data was then examined for their qualities, and subsequently simplified, cleaned and standardised for ALMEDA’s purposes. Wikidata had much non-essential information for ALMEDA, which was filtered out during the reuse process. The basic information imported from Wikidata items are Labels, Descriptions and Aliases in seven languages. We also kept the links to Wikidata for provenance and reuse purposes. For countries reused, two properties ‘coordinate location’ and ‘inception’ are kept. For languages reused, we keep the property ‘country’ and three external identifiers, ‘Ethnologue.com language code’, ‘ISO 639-3 code’ and ‘Glottolog code’.²⁶

(2.3) Workflow

With the implementation of the ALMEDA Wikibase nearing completion, the project is moving forward with consolidating the workflow for uploading collected metadata to the instance. Metadata is collected in several ways, through case studies by affiliated researchers, data harvesting from available physical and digitized materials at archives and libraries, and dataset donations from collaborating researchers and organizations. New metadata will be collected and added continuously throughout the project. After the project has formally ended, an advisory editorial board will be established to review metadata submissions before they are uploaded.

A question that impinges on the project as a digital humanities initiative is the problem that domain experts from literary studies and related fields have little to no experience of working with datasets, since this is seldom required in the field. This meant that basic training was required to bring researcher expertise in alignment with how data needed to be structured. ALMEDA has addressed this problem by tasking our information specialist with preparing templates for data capture for researchers. These templates facilitate the data-uploading process and ensure that the necessary data fields are present for the data uploader to proceed.

When a new dataset comes in, the data is first checked against the appropriate template. This includes checking whether the necessary data fields are present and the values in the columns align with the formatting standards decided upon. Any inconsistencies in the dataset are corrected, such as the incorrect formatting of dates, page numbers, and author names. If any major problems are encountered that cannot be resolved by the data uploader, the dataset is sent back to the Principal Investigator for referral to the researcher or data collector. Once all the requirements are fulfilled, the cleaned dataset is published on Zenodo, generating the digital object identifier (DOI) used for reference in the items in the Wikibase instance. The data cleaning steps are performed using basic functionalities of spreadsheet editors and the data wrangling tool OpenRefine. This tool offers a range of useful methods for cleaning and transforming data, for example harmonizing spelling variations of author names through clustering.²⁷

For the data upload, there are several options to add data to the Wikibase instance. Data can be added through manual entry of individual items, but also in bulk using the previously mentioned python packages. In the case of ALMEDA, OpenRefine has so far been the preferred tool, due to its capacity to clean, reconcile, and upload the data in one place. From the single dataset different items need to be created in the instance, related to the source of publication or venue, the author/creator, the creative work itself, and finally the ‘version, edition, or translation’ of the work. For each of these a list of required and optional properties is defined.

Reconciliation in OpenRefine is used to match the values in the dataset with already-existing records on the ALMEDA instance.²⁸ This is done to check whether the work or author already has a presence in our instance and link the values related to ‘country of citizenship’ or ‘form of creative work’ with the appropriate item on the Wikibase. Additionally, reconciliation of data about people takes place against Wikidata to see if relevant information regarding e.g., sex assigned at birth, date of birth, awards received is available. This information, if available, is then added to the record alongside the metadata collected by ALMEDA. The link to the Wikidata item is added to the record using the ALMEDA property ‘Wikidata link’. Reconciliation is also a required step before data can be uploaded to the Wikibase instance. If the value in the cell cannot be matched to an existing item on the Wikibase instance, a new item is created alongside the other statements.

In order to reconcile and upload data, a Wikibase manifest is needed, a configuration file that specifies how OpenRefine should communicate with the instance. Additionally, the data uploader should be added to the user group of bots to be able to create and edit items in bulk. Furthermore, a Wikibase schema needs to be defined. This schema defines how the columns and values in the dataset are to be transformed into statements.²⁹ When uploading a dataset, OpenRefine provides the possibility to inspect future edits on the instance in preview. The tool also reports issues with the schema and values, reducing the chance of incorrect uploads.

An overall challenge for the project concerns aligning its different elements, from the data modelling and the implementation of the ontology in Wikibase, to the data upload workflow. The ontology is developed from an ideal theoretical scenario, where all the metadata related to entities is relatively clear and available. The actual datasets collected by researchers or donated by collaborating partners, however, do not mirror the ideal data model as defined in the ontological structure. The datasets include valuable but fragmented and often patchy data that need to be restructured to fit the data model. Because of the informal circumstances in which much of our data is collected, these datasets often miss basic metadata that is normally expected in library catalogues, such as date of birth or affiliation. As a result, the ontology and level of detail of the metadata need to be weighed against the workload for the data uploader.

At the same time, ALMEDA has made some choices that, whilst adding extra work for a data uploader, have been retained for the purposes of theoretical clarity and future interoperability of the data with other repositories. One example of this is our two-level modelling of a creative work at both the ‘work’ and ‘versions, edition, and translation’ level. Whilst this two-level structure is not strictly relevant for single issue works (which we anticipate will be the vast majority of ALMEDA’s content), and while it would be easier to remove this distinction and upload one item for each work rather than two, this could affect the searchability and interoperability of the whole repository.