Evaluating UDPipe 2 and Stanza for Morphosyntactic Annotation of Late Latin: A Pilot Study on Sixth-Century Texts

PERIOD	AUTHOR	WORK AND LOCI	TOKENS	TREEBANK
Early Latin (240 BC–90 BC)
Classical Latin (90 BC–14 AD)	Caesar	Commentarii de Bello Gallico (1. 1–54; 2. 1–35; 3: 1–27; 4: 1–9, 11–38; 5. 1–7, 49; 6. 1–44; 7.3, 5, 34, 77)	28,608	PROIEL
	Caesar	Commentarii de Bello Gallico (2.1–2.3; 2.5; 2.7; 2.9; 2.14–2.18; 2.32–2.33)	352	Perseus
	Cicero	In Catilinam (1.1–2.11)	1,915	Perseus
		Epistulae ad Atticum (letters 1.1–20; 2.1–25; 3.1–27; 4.1–19, 4A and 8A; 5.1–6 and 8–21; 6.1–3 and 9; 7.1–4, 7, 10, 12–13, and 19–26; 8.1–16, 11A, 11B, 11D, 12A–C, and 15A)	42,855	PROIEL
		De officiis (1–39, 41-113, 116, 133–137, 139–161)	10,644	PROIEL
	Propertius	Elegiae (1.1–1.22)	2,801	Perseus
	Vergil	Aeneid (6, 1–336)	779	Perseus
	Ovid	Metamorphoses (1.1–1.778)	2,655	Perseus
	Augustus	Res Gestae	713	Perseus
Silver Latin (14 AD–200 AD)	Phaedrus	Fabulae (1–3)	4,118	Perseus
	Sallust	Bellum Catilinae	4,999	Perseus
	Seneca	Hercules Furens	7,714	CIRCSE
		Agamemnon	5,580	CIRCSE
		Oedipus	5,931	CIRCSE
	Petronius	Satyricon (26–78)	5,955	Perseus
	Tacitus	Germania	5,674	CIRCSE
	Tacitus	Historiae (1.1.21)	867	Perseus
	Suetonius	Life of Augustus (1–55)	2,066	Perseus
Late Latin (200 AD–600 AD)	Palladius	Opus Agriculturae (Ch. I-XXXXII)	12,148	PROIEL
	Jerome	Vulgate’s Revelation	2,003	Perseus
	Jerome	Vulgate New Testament (Matthew 1–28; Mark 1–16; Luke 1–24; John 1–21; Acts 1–28; Romans 1–16; 1 Corinthians 1–16; 2 Corinthians 1–13; Galatians 1–6; Ephesians 1–6; Philippians 1–4; Colossians 1–2; 1 Thessalonians 1–5; 2 Thessalonians 1–3; 1 Timothy 4 and 6; 2 Timothy 1–4; Titus 1 and 3; Philemon 1; Hebrews 1–2, 4–5, 7, 10–11, and 13; James 2–5; 1 Peter 2–3 and 5; 2 Peter 2–3; 1 John 1–2 and 4–5; 2 John 1; 3 John 1; Jude 1; Revelation 1–22)	112,454	PROIEL
Medieval Latin (600 AD–1492 AD)	521 legal charters from Tuscia (774–897 CE)⁵		242,391	LLCT
	Thomas Aquinas and related authors	Summa contra Gentiles (Books 1–4)	450,480	ITTB
	Thomas Aquinas and related authors	Concordances of the lemma forma	450,480	ITTB
	Dante Alighieri	De vulgari eloquentia	55,286	UDante
		De monarchia
		Letters
		Quaestio de aqua et terra
		Eclogues

Table 2

Dataset description.

*Repository location*	https://doi.org/10.7910/DVN/QHGETA https://github.com/annachiaraclementelli/6th-century-gold-standard-UD
*Repository name*	Annotated gold standard of selections of four sixth century Late Latin texts
*Objects name*	beneditctus.conllu; cassiodorus.conllu; eugippius.conllu; gregorius.conllu
*Format names and versions*	CoNLL-U
*Creation dates*	2026-02-01–2026-07-31
*Dataset creators*	Annachiara Clementelli (Università per Stranieri di Perugia & Ghent University; role: annotator)
*Language*	Late Latin
*Publication date*	2026-07-31
*Content*	Author	Work	Period	Genre	Tokens
	Benedict	Regula	534 AD	Monastic rule	684
	Cassiodorus	Variae	527–540 AD	Administrative epistolary	684
	Eugippius	Vita sancti Severini	ca. 526 AD	Hagiography	684
	Pope Gregory I	Regula pastoralis	ca. 590 AD	Pastoral treatise	684
				Total	2,736

Example of manual POS-annotation with nominal ellipsis.

Table 3

Evaluation scores of the UDPipe 2 and Stanza models, in random order.

	UDPipe 2				STANZA
Model	UPOS	UAS	LAS	CLAS	UPOS	UAS	LAS	CLAS
Perseus	93.41	73.76	66.22	67.74	89.86	63.62	53.51	52.53
PROIEL	90.34	72.91	63.43	65.78	75.88	70.06	54.90	55.95
ITTB	92.75	72.40	64.71	66.81	90.23	69.22	60.43	61.92
LLCT	83.93	54.94	42.20	39.66	85.43	57.54	45.72	44.52
UDante	92.83	73.21	65.63	67.36	90.15	65.56	55.31	55.21
CIRCSE	92.79	73.43	65.85	67.08
EvaLatin 2024	94.33	80.34	74.08	78.64