A Part-of-Speech Tagger for Older Scots: A spaCy-Based Model Using the CLAWS7 Tagset

Repository location

The dataset is available at Dataverse: https://doi.org/10.7910/DVN/8VLSEK (Beattie (2026))

Context

This resource was created as part of a doctoral research project using corpus methods to explore the language of the Scottish and English Reformations. The corpus contains Early Modern English and Older Scots (OSc) texts¹ —related languages but different enough to require separate annotation approaches. While automated taggers are well-established for modern and well-resourced languages, including CLAWS for Early Modern English (Garside & Smith, 1997), significant challenges remain when developing tools for historical varieties and low-resource languages, particularly where training data is limited and there is a large amount of linguistic variation.

Recent projects have developed annotation tools for other low-resource languages. Lameris and Stymne (2021) found that a tagger trained only on English performed poorly when applied to Scots without any Scots-specific training (42–56% accuracy), showing that Scots cannot simply be treated as a variant of English. Adding a small amount of manually annotated Modern Scots data improved accuracy to 67.4%, though this remained limited by the quality of available word embeddings—numerical representations of word meaning used by the model—and the coarse granularity of the Universal Part of Speech (UPOS) tagset used (Petrov et al., 2012). Millour et al. (2024) applied a similar approach to Corsican, improving an Italian-trained tagger from 62.84% to 93.38% accuracy through fine-tuning on manually annotated Corsican material, with spelling variation and rare tags being the primary challenges. OSc faces similar challenges, with the added difficulty of limited surviving material typical of historical languages.

No PoS tagger has previously been trained specifically on OSc using machine learning methods. While some existing work has applied rule-based approaches to tagging OSc material—Gotthard’s PhD research involved tagging OSc material using Python scripts alongside dictionaries from the Linguistic Atlas of Older Scots and the Penn Treebank (Gotthard, 2023, pp. 38–40)—such methods are time-consuming, require exhaustive manual specification of linguistic rules, and rely on the quality of the dictionaries to capture orthographic variation. The tagger described here addresses this gap: a spaCy-based model built using the spaCy natural language processing library trained on pre-tagged OSc literary texts (Bushnell, 2021), achieving 89.83% accuracy using pre-trained weights from a high-resource English model. The tagger uses the CLAWS7 tagset (UCREL, 1996), which offers considerably finer morphosyntactic granularity than the UPOS tagset.

Steps

The tagger was built using spaCy (Honnibal et al., 2020), an open-source Python natural language processing library that supports tasks including tokenisation, lemmatisation, and PoS tagging, allows custom taggers to be trained. spaCy’s neural tagging approach was suited to OSc because it predicts tag sequences by representing words as token embeddings and incorporating character-level information to handle the orthographic variation present in OSc.

The spaCy model for the tagger was trained using two tagsets: UPOS, required by spaCy, and CLAWS7, the primary output tagset. CLAWS7 contains 152 tags which can be grouped into larger macro-categories (for example, all noun tags begin with N, proper nouns NP, etc.), allowing for both broad and narrow searches of tagged data. The training data was a ca.100,000-word corpus of pre-tagged OSc literary texts compiled by Bushnell (2021), covering OSc literature written before 1600. The files were pre-processed to remove extraneous etymological and semantic information, retaining the original word forms, lemmas, and CLAWS7 tags. A conversion script was used to map CLAWS7 tags to their UPOS equivalents, and the files were converted from JSON to the .spacy format required for training. The corpus was divided into three sets: ‘train’ (two thirds of the data, used to train the model), ‘dev’ (used by the model to check itself during training), and ‘test’ (used for formal accuracy evaluation afterwards).

Three model variants were compared, differing in their use of pre-trained resources: training data only; pre-trained word vectors from the English-language spaCy model en_core_web_sm; and pre-trained weights from en_core_web_lg. After initial training on the Bushnell corpus, an iterative process was used, whereby individual OSc texts outside the original training set were tagged, reviewed, and added to the training data. This allowed the tagger to develop familiarity with vocabulary outside of the literary training data, broadening the tagger’s applicability to other OSc research.

Sampling strategy

The choice of training data was constrained by the availability of existing annotated OSc material. The corpus of OSc literature selected as the primary training source was chosen because the size of the corpus—approximately 100,000 words—provided a substantial body of OSc material for the model to work with, and the texts were already tagged using the CLAWS7 tagset. While normalised spellings of the training data were available in the dataset, these were normalised according to English orthographic conventions. Therefore, original OSc spellings were retained in the training data rather than normalised forms to provide the model with as broad a range of OSc orthographic forms as possible.

Quality control

Model accuracy was evaluated using two methods. First, spaCy’s built-in evaluate command was used to produce an overall accuracy score for each of the three model variants. A second bespoke evaluation script was also written to compare model output against the ‘test’ set, generating precision (proportion of positive predictions correctly identified), recall (proportion of all positive predictions identified), and F1-scores (harmonic mean of precision and recall) for each of the 118 tags present in the corpus. This allowed for a more granular assessment of performance across tags, distinguishing high-frequency tags (F1 over 0.74) from low-frequency tags, whose performance was more variable. On the basis of these evaluations, the model using pre-trained weights was determined to be the most accurate, with an overall accuracy of 89.83%.

A limitation of the CLAWS7 tagset is the absence of a dedicated punctuation tag, which resulted in punctuation tokens being assigned arbitrary tags during the tagging process. This was resolved post-hoc using a Python script to replace incorrectly assigned punctuation tags with the correct punctuation symbol. Manual review of tagger outputs during the iterative tagging process also identified recurring error types, particularly around the disambiguation of OSc morphosyntactic features such as the <-is> suffix, which can represent plural nouns, verb endings, and genitive markers. These errors were documented and will inform future development of the tagger.

Repository name

Dataverse

Object name

‘A Part-of-Speech Tagger for Older Scots: A spaCy-Based Model Using the CLAWS7 Tagset’

Format names and versions

Training data: .spacy

Tagging script: .py

Accepted input: .txt

Outputs: .csv and .xml

Creation dates

2024-08-10–2026-06-24

Dataset creators

Beth Beattie, University of Glasgow

Language

Older Scots

License

CC0 1.0

Publication date

2026-06-24