Machine Learning Applications in Archaeological Practices: A Review

3.3.1. Statistical overview

Among the different subfields of archaeology present in our review, surveying (and site prospection), conservation and cataloguing, as well as classification and typology, are the most represented in our dataset and together account for 49% of all study cases (Figure 5A). Landscape archaeology and geoarchaeology are also well represented with 20 and 17 attributed study cases respectively. The oldest applications (1997 – 2016) of machine learning in archaeology are linked with the subfields of classification and typology as well as conservation and cataloguing. In recent years, there has been a general increase in all topics with no clear trend visible, except for the subfields of surveying as well as conservation and cataloguing, which have considerably increased since 2021 (Figure 5A). Only the subfields of zooarchaeology, archaeobotany, and archaeological excavation remain nearly constant through time.

Figure 5

Examining the different architecture of machine learning methods used, we observed that artificial neural networks (ANNs) and ensemble learning represent 62% of all study cases, with 110 and 70 occurrences respectively (Figure 2). Linear classifiers, decision trees, rule induction, and nearest neighbour classifiers were also well-represented, accounting for 25% of total applications. An increase of the use of ANNs in our list of reviewed articles from 2021 onwards could be observed, with the number of applications multiplying four-fold compared to the number of applications in the years 2019 and 2020 (Figure 5B). At a more granular scale, we found a total of 70 machine learning models (Figure 6, Annexe). In addition, we found a particularly high diversity in the unsupervised learning and clustering category of model families, with a unique model for each of the twelve recorded applications for the category (Figure 6, Annexe). Despite the dominance of ANNs (Figure 6), random forest was the most common individual model (n = 54). We found that across all articles reviewed, the mean number of models used per study case was 2.12, with a high disparity across different publications, with some applying only one model, while articles such as Bataille et al. (2018) and Courtenay et al. (2019) tested six different models at once. Classification was the most common form of data evaluation, applied in more than 75% of study cases (n = 112). On the other hand, only 19% (n = 29) were regression models, and clustering was only represented with six uses (6%, Figures 2, 8 and 9).

Figure 6

It was more difficult to observe patterns for the other features we collected, as the attribution to one or several categories for an article was more subjective to the person reviewing. Two of the a posteriori study tasks are highly prominent: automatic structure detection and artefact classification. These two tasks account for 45% (n = 67) of all study case tasks (Figure 2). Tasks such as taphonomic classification, archaeological predictive models, and architectural element classification are well represented with 5 to13 study cases each, whilst nine task categories (e.g. sourcing, species classification, movement recognition) were only relevant for around 5 study cases.

For the different types of input data observed in all study cases we reviewed, remote sensing images were the most represented, with around 40% (n = 58) of the total. Four other input types (small-scale images, artefact measurements, spectra, and 3D models) were also well represented (Figure 7A), being present in between 6 and 16% of all study cases reviewed. The remaining nine input types were poorly represented, being present in only 14% of all study cases.

Figure 7

Finally, according to the assessment of article authors or (if not discussed) the results metrics, we could observe that most applications (79) had reported “successful” outcomes (Figure 7B), with an additional 40 mixed results (27.2%). Only 15 applications (10.2%) were reported to have been “unsuccessful”, whilst we noted 9 (6.1%) study cases whose application had important methodological issues and classified them as such even if the authors reported a successful outcome. Finally, four (2.7%) study cases did not have a defined outcome. These results, however, could be affected by the well-reported issue of publication bias against the reporting of negative results (Song et al. 2013), as well as the related file-drawer effect (Rosenthal 1979; Sparks 2009), which can strongly affect the perception and production of scientific results (Song et al. 2010).

3.3.2. Aggregated information

Through the aggregation of the data annotated for the study tasks, method families, archaeological subfields, and input types we obtained an overview of the relationship between the objectives of the study cases and the means used to access them.

Correlations between tasks and model families are highly heterogeneous (Figure 8). Whilst the highest correspondence rate between taphonomic classification and unsupervised learning and clustering methods is 26%, the correspondence between artefact classification and ANNs is much higher, at 49%. Even stronger correlation rates could be observed between archaeological predictive models and ensemble learning methods at 66%, and between automatic structure detection and ANNs, at 60%.

Figure 8

The correlations between archaeological subfields and study tasks are lower than the correlations visible for architecture (Figure 9). Articles dealing with environmental reconstruction had their highest correlation at only 33% with the subfield of geoarchaeology. Studies treating artefact classification problematics have a higher 45% correspondence rate with subfields of classification and typology. The highest correlation rate, however, is present between the task of automatic structure detection and the subfield of surveying, at 80% correlation.

Figure 9

Finally, and perhaps expectedly, several of the most represented tasks had the largest number of study cases with reported unsuccessful or mixed results (Figure 10). Studies dealing with automatic structure detection (25.85% of total) and artefact classification (19.73% of total) accounted for 60% and 33% (respectively) of studies that reported mixed or unsuccessful results. Other less represented tasks have higher success rates. Articles dealing with artefact prediction are successful in 80% of cases, whilst articles dealing with environmental reconstruction all report successful results.

Figure 10

3.3.3. Classification of artefacts and animal remains

Whilst classification tasks were mainly performed on ceramic material (31%), as in Hörr et al. (2014) or Anichini et al. (2021), a wide range of different materials were used in our dataset of reviewed articles. Studies on coins (Boon et al. 2009), stone tools (MacLeod 2018; Pargeter et al. 2019; Emmitt et al. 2022; as well as Orellana Figueroa et al. 2021, excluded from our review to avoid possible conflicts of interest, though still relevant), archaeobotanical seed remains (Landa et al. 2021), phytoliths (Berganzo-Besga et al. 2022), ivory figurines (Gansell et al. 2014), were observed. Moreover, 17% of study cases were conducted on cave art images, such as in Kogou et al. (2020), or Horn et al. (2022b, a). On all classification study cases 17% (n = 5) dealt with the bioarchaeological classification of human or animal remains. Due to the variety of materials studied, the input data for models were very diverse. Small-scale images accounted for 48% (n = 14) of all input data for classification task, metric data for 27% (n = 8), as well as less frequent input data types such as multilayer images of artefact (e.g. depth maps, 3D model layers), spectra and remote sensing images all together accounted for 23% (n = 7).

In our corpus, two machine learning method families emerge as the foremost choices for artefact classification, ANNs represent 50% of the models used, followed by ensemble learning, accounting for 31% of the models (Figure 8). Bayesian classifiers, as well as decision tree and rule induction are also present, but represent less than 15% of the model architectures for the artefact and animal remains classification task.

3.3.4. Archaeological predictive models (APMs)

Ten studies in our review focused on archaeological predictive models (APMs), dealing with either human landscape occupation patterns, or human-environment interactions. Input data were in 77% (n = 7) of all study case larger-scale raster images from a diverse range of natural covariates or remote sensing images.

The most common method family applied was ensemble learning, representing 62.5% (n = 10) of the entries for this task. A unique model entry exclusively applied to this task was Maximum Entropy (MaxEnt) (Benner et al. 2019; Yaworsky et al. 2020). Regarding the results from APMs, 66% (n = 6) were classified as successful, with the remaining 44% (n = 4) classified as partially successful or unsuccessful, such as in Hansen and Nebel (2020) or Miera et al. (2022).

3.3.5. Automatic structure detection

Representing 25% (n = 38) of all study cases, automatic structures detection, also known as geographical object-based image analysis (GEOBIA), is the most prominent task category in our review. Input data came from a wide range of remote sensing images, mainly LiDAR or airborne laser scanning (ALS), used in 50% of all study cases in this task, or various satellite image collections (23%), such as in Menze et al. (2006) or Orengo et al. (2020). The remaining 27% input data used were either UAV ortho-images, such as in Orengo and Garcia-Molsosa (2019), Monna et al. (2020), Agapiou et al. (2021), Altaweel et al. (2022), and Fisher et al. (2022), historical maps, such as Garcia-Molsosa et al. (2021), and since 2020, ground-penetrating radar (GPR), or even bathymetric data (Febriawan et al. 2020; Bordon et al. 2021).

ANNs were the most prominent family of methods for automatic structure detection, accounting for 62% of all models applied for this task. Different ANN models were used, the most popular being mask region-based convolutional neural networks (MR-CNN), which accounted for 13% of all models applied for automatic structure detection, followed by faster region-based convolutional neural networks (FR-CNN), and U-Net, which each accounted for 8% and 7% of the models used. However, the most popular model was random forest, which accounted for 16% of all models used for this task. The results of applications in this task were mainly reported as unsuccessful or partially successful (59%, n = 31).

3.3.6. Digital heritage

Articles focused on this task were mainly represented by those dealing with the classification and reconstruction of architectural elements, with a total of twelve study cases (8% of all 147 study cases reviewed). The models in this task category mainly used small-scale images from architectural photos as input (58%, n = 7), but also point clouds and 3D models in 33% of the revived study cases, such as in Grilli and Remondino (2019) or in Matrone and Martini (2021).

The models applied in this task are somewhat equally divided between two families of models: ensemble learning with 35% of all study cases, and decision trees and rule induction models accounting for 25% of total models applied. Machine learning applications for this task were generally negatively assessed, in comparison to applications for other tasks, with only 40% of study cases reporting successful results.

3.3.7. Text analysis

We counted five study cases that applied machine learning methods to text analysis tasks (3% of all 147 study cases), and all except one (Dhivya and Devi 2021) used text as input data for their model. Almost all applied models were based on ANNs (83%, n = 5), such as in Dhivya and Devi (2021) and Brandsen and Lippok (2021), only Boon et al. (2009) used a memory-based learning (MBL) model, part of the category of unsupervised learning and clustering algorithms. The results of the applications for this task were overwhelmingly negatively assessed, with four out of the five studies reporting mixed or unsuccessful results.

3.3.8. Taphonomic classification

We reviewed a total of twelve study cases categorised as dealing with the classification of taphonomic features on archaeological finds (8% of all case studies). All of these articles were applied to osseous material, and all but two of them deal with bone surface modifications (BSMs), with the remaining two dealing with bone breakage. Bone surface modifications are marks left on the surface of bones by agents such as hominin butchery (e.g. using stone tools), animal carnivory, or trampling. The goal of all the articles under discussion was to uncover what agent caused the specific taphonomic process (BSM or breakage). Input data were primarily metric and categorical variables of bone marks or breaks in six cases (50%), or microscope images in three cases (25%), with the remaining three (25%) using PCA scores from the analysis of geometric morphometric landmarks.

Many of the articles reviewed applied a large number of machine learning methods as well as methods that we do not consider machine learning (such as PLSDA and MDA; e.g. Domínguez-Rodrigo 2018; Abellán et al. 2022) to compare their performance on the specific task, whilst others (e.g. Byeon et al. 2019; Cifuentes-Alcobendas and Domínguez-Rodrigo 2019) were more restrained. This meant that there was no clear machine learning method or family of methods that was used much more frequently than others, although neural networks were applied in all but one of the articles reviewed (see “Results”), which is not the case for other methods also commonly used in these articles, such as SVMs or random forests.

ANNs were applied in all the reviewed study cases except for one (Aramendi et al. 2019) and represent 28.3% (n = 15) of all models applied for taphonomic classification. Ensemble learning, linear classifiers, decision trees and rule induction, nearest neighbour classifiers, and Bayesian classifiers all represented between 17% and 11% of the referenced architectures for this task category.

However, many of the articles found were deemed through our reviewing process to have had important methodological issues, especially due to under-reporting or under-detailing of the specific methodology used (see Figure 10; see also 4.2.6).

4.2.1. Classification of artefacts and animal remains

Classifying artefacts and animal (including human) remains into categories is a central aspect of archaeological work (Read 2018). Archaeologists have looked for solutions in computer science since the second half of the twentieth century (cf. 1.; Binford and Binford 1966; Orton 1980, pp. 156–178). It is therefore not surprising to see machine learning applications developed for this purpose already very early (Figure 5A). Earlier papers generally applied methods such as decision trees, ensemble learning, Bayesian classifiers, as well as linear classifiers combined with nearest neighbour classifiers (Nguifo et al. 1997; Hörr et al. 2014; Mircea et al. 2015a; MacLeod 2018; Canul-Ku et al. 2019; Pargeter et al. 2019; González-Molina et al. 2020). Since 2020, however, ANNs have gained more traction (e.g. Ushizima et al. 2020; Graham et al. 2020), and now represent the most used family of methods applied for this task (Figures 6, 8; Ling et al. 2024). ANNs, especially CNNs, are standard for computer vision tasks like object classification (Guo et al. 2016; Chai et al. 2021). Dimensionality reduction methods (e.g. PCA and MDA) are often performed alongside machine learning algorithms and can be an effective way to improve model accuracy when applied correctly (e.g. Muzzall 2021). However, their ability to distinguish categories of artefacts seems limited compared to machine learning methods such as random forest (Cole et al. 2022).

Concerns about the validity of certain forms of typological classification, both human-made and computer-based, as a practice have been previously raised, especially from North American researchers (Adams and Adams 1991; Hörr et al. 2014). Other researchers, however, do not hesitate in using computational approaches for classification (Pargeter et al. 2019), and even creating new classification methodologies with the help of machine learning. Notably, Hörr and others, seeking a less subjective approach to typological classification, developed a three-layer method for this purpose (Hörr 2011, pp. 136–137; Hörr et al. 2014), successively incorporating unsupervised, semi-supervised, and supervised classification algorithms within a framework Fayyad et al. (1996) termed the Knowledge Discovery Process. This approach is promising for the analysis of archaeological material, and could prove useful for many archaeological studies. Such an approach invites critical questions on the artificiality of typological classifications and opens new avenues for future studies such as those dealing with ceramic material or stone tools. We can also mention Martín-Perea et al. (2020), who used a method similar to that of (Hörr et al. 2014) with a three-level pipeline to detect fossiliferous levels of an archaeological or palaeontological site.

Assessing and comparing the outcomes of these diverse classifications is challenging, given the inherent variations in datasets, modelling techniques, and evaluation metrics employed across different studies. Nonetheless, various opinions have already emerged on the issue. Emmit et al. (2022) and Jalandoni et al. (2022) are optimistic about using machine learning for the classification of archaeological material, though we could also add Hörr et al. (2014) or Bickler (2018), who are more cautious but remain hopeful for the future of the practice. Others are less enthusiastic, however, and point to mixed results (Demján et al. 2022b; Lyons et al. 2022). Overall, it seems likely that machine learning applications for the classification of archaeological material will spread not only within the scientific community but also among professionals of archaeology, conservation institutions, and a wider public. The ArchAIDE project is already a noteworthy example of an application of open-access machine learning tools for archaeology to serve the public at large (Gualandi et al. 2021; Anichini et al. 2021, Anichini and Gattiglia, 2022). Another example of a mobile application for ceramics classification is developed in Santos et al. (2024), and a broader project of automation and digitalisation tools for archaeological artefacts has been developed in the Automata initiative (Naso and Sciuto 2025). The reconstruction of ceramics based only on few pottery sherds has also been achieved with machine learning methods (Stamatopoulos and Anagnostopoulos 2016; Cardarelli 2024). Another promising development has been made in Ruschioni et al. (2023), where they classified ceramics based not on their typology but rather on their chemical properties, which is more data-compatible with a machine learning process.

In summary, as one of the core elements of archaeological work, the use of machine learning for the classification of artefacts has been debated theoretically and tested since the early 2000s. While ceramics are the most represented type of artefact used for machine learning-based classification in our corpus, a wide variety of materials have also been used (e.g., ivory, stone tools, etc.). With the majority (62%, n = 18) of study cases using image data as inputs, the recent increase in performance of neural network models could spur further advances for artefact classification. The future of this discipline will also likely be linked with more data-driven material, standardised pre-processing, and user-friendly classification software, be it portable or not.

4.2.2. Archaeological predictive models (APMs)

Already present since the development of settlement patterns analysis (Willey 1953), the tradition of APMs rose alongside the development of modelling and prediction theory in archaeology (Judge and Sebastian 1988). It is therefore surprising to see little interest in applying machine learning methods to this approach in recent years even down to 2022 (Figure 5A). APMs focus on predicting unknown or undiscovered sites so that they can be discovered and documented based on previous field observations and variable predictors, for example climate, topography and anthropic features (Brandt et al. 1992; Kvamme 2006). Critics can point to the relative reluctance of researchers towards APMs (Kvamme 2006, pp. 7, table 1.1; Lock and Harris 2006, pp. 42–45). One main criticism about APMs is that “… predictive modelling as it is presently practiced is fundamentally about environmental determinism.” (Kohler 1988, p. 19). This statement is particularly true when we consider the strong influence of environmental and ecological studies in the field of landscape archaeology (cf. 3.3.4). However, even if this criticism has neither recent origins, nor is it confined to the past (Wheatley 2004; Arponen et al. 2019; Kristiansen 2019), responses to it exist. Coombes and Barber emphasise the interest of models despite their incorrectness, stating that ”A simple model cannot hope to replicate all the complexities of environment–culture relationships across a civilization, but one basic approach that can provide valuable insights is to treat human populations in ecological terms, with their ranges shifting in response to changing conditions” (Coombes and Barber 2005, p. 305). Furthermore, new applications of APMs dedicated to cultural heritage preservation against looting (El-Hajj 2021) can prove to be a promising solution to further aid in the protection of cultural heritage alongside more intensive public funding.

The frequent use of the MaxEnt model for APMs, initially designed by ecologists (Phillips et al. 2006; Sillero et al. 2021), shows the influence of environmental niche concepts in archaeology (Demján et al. 2022a; Vernon et al. 2022; Lundström et al. 2024; Yaworsky et al. 2024a, b). MaxEnt is well suited to examining questions of dispersion and provides more robust and deterministic archaeological predictions due to it being a presence-only model, using background points rather than true absence or pseudo-absence data (used by other methods, including machine learning) into its modelling: “As a result, MaxEnt is more suitable for archaeological data than the other predictive modelling approaches” (Yaworsky et al. 2020, p. 14). Furthermore, when compared to other models such as random forest, which can be prone to overfitting, MaxEnt appears as the best option for species dispersion problems (Valavi et al. 2022).

An essential consideration in APMs is the number of covariates, predictors, or variables required for the building of the models. In Yaworsky et al. (2020), 55 predictors were used, Hansen and Nebel (2020) employed 26 predictors, Friggens et al. (2021) used 25 predictors, Castiello and Tonini (2021) used 13 predictors, and Benner et al. (2019) relied on 8 predictors. However, if data transformation techniques such as PCA are applied, which seek to reduce the number of covariates whilst maximising the impact of those that remain, a smaller number of predictors can be used for the final model instead (Hansen and Nebel 2020; Yaworsky et al. 2020; El-Hajj 2021). Research on species distribution models assess the maximum number of predictors at $k =\frac{n - 50}{8}$ (Field et al. 2012), k being the number of predictors and n the number of occurrences of the species.

The large number of variables required, the complex preprocessing steps needed, and the difficulty in selecting and interpreting the influence of the model variables, make APMs a time-consuming and less attractive method than approaches based on image recognition, which can be simpler to apply. Furthermore, the theoretical criticisms discussed above must also be added to the methodological difficulty of applying APMs, which could lead to a desire to avoid further complexity in the methods in the form of machine learning methods, which could explain their low numbers compared to applications for other tasks.

4.2.3. Automatic structure detection

Over the past decade, there has been an explosion in machine learning methods applied to automatic structure detection tasks (Figure 5A). Though earlier applications were mostly based on satellite STRM or ASTER images (Menze et al. 2006; Menze and Ur 2012), the recent explosion of light detection and ranging (LiDAR) and airborne laser scanning (ALS) data (Wurzer et al. 2015; Gillings et al. 2020), has led to an increase in interest for this field. While earlier studies employed random forest methods (Menze et al. 2006; Guyot et al. 2018; Stott et al. 2019), recent papers mainly applied ANNs, which aligns with the predominance of image data and neural network architectures that excel at image classification tasks. The recent multi-scale region-based convolutional neural network (MR-CNN) model, adapted from convolutional neural networks (CNNs), has been eagerly applied by archaeologists, as its raison d’être is well aligned with the task of detecting archaeological structures from image data (Bundzel et al. 2020; Bonhage et al. 2021; Carter et al. 2021; Davis et al. 2021; Guyot et al. 2021; Altaweel et al. 2022; Banasiak et al. 2022; Fisher et al. 2022). MR-CNNs segment images via masked regions, aiding structure identification and validation. Furthermore, the emergence of transfer learning has amplified the use of CNNs in archaeology (Gallwey et al. 2019; Soroush et al. 2020; Herrault et al. 2021). By using a neural network model trained for a different but similar type of image data (e.g. to detect bicycles in an image), and training it on the actual desired data (e.g. detect motorcycles) but with a much smaller dataset, compared to the dataset needed without transfer learning, transfer learning can be helpful where only small datasets are available, reducing thus also the time and cost of data acquisition and preparation. In our dataset of articles, the use of transfer learning – counted at 41% (n = 16) of all study cases reviewed on automatic structure detection – were generally based on models pre-trained with image datasets such as ImageNet (Russakovsky et al. 2015) or COCO (Lin et al. 2014), though in one unique case authors also relied on specialised datasets (Silburt et al. 2019).

However, as Herrault et al. (2021) has highlighted, CNNs have limitations in their interpretability and are also too sensitive for unbalanced classes (i.e. the categories available for prediction), a common issue for archaeological data. A deeper exploration of models, as done by Monna et al. (2020), exploring six distinct families of methods (ANNs, Bayesian classifiers, linear classifiers, ensemble learning, unsupervised learning and nearest neighbour classifiers), allows for a better understanding of the data, despite the increase in complexity and time. Following the training of the models, Monna et al. (2020) aggregate predictions through a hard voting mechanism, itself ensemble learning, ultimately identifying random forest (an ensemble learning method) as the best model.

To evaluate the success of machine learning archaeological structure detection models, authors have access to a wide range of metrics. The F1-score is the most widely used with 20 accounted uses (e.g. Character et al. 2021; Altaweel et al. 2022; Banasiak et al. 2022), but prediction accuracy is also common, with 16 applications in the study cases on automatic structure detection (e.g. Monna et al. 2020; Davis and Lundin 2021). While not always directly discussed, precision, average precision, and recall can nevertheless be found in the results in most of the study cases reviewed here (Bonhage et al. 2021; Berganzo-Besga et al. 2021). Intersect over union (IoU) is only present in five study cases (Bundzel et al. 2020; Bordon et al. 2021; Banasiak et al. 2022; Trotter et al. 2022; Yang et al. 2022), despite revealing valuable information on a model accuracy and being particularly well-suited for image segmentation tasks. The newly developed “boundary IoU” (Cheng et al. 2021) was specifically created for evaluating image segmentation models, and could become an important addition to future studies. New metrics have even been developed explicitly for archaeological image segmentation. Fiorucci et al. (2022) introduced centroid-based and pixel-based measures, as they found IoU not ideal for evaluating discrete archaeological objects.

The significant popularity of machine learning applications for automatic structure detection can be further found in its perceived suitability to answer the research question of whether a certain feature is an archaeological site or structure. Machine learning methods seem to be particularly effective when applied to massive structures such as burial mounds (Guyot et al. 2018; Caspari and Crespo 2019; Monna et al. 2020; Berganzo-Besga et al. 2021) or large-size archaeological sites (Menze et al. 2006; Menze and Ur 2012; Stott et al. 2019).

To summarise, following the explosion of big data such as the use of archaeological remote sensing imagery (especially LiDAR), combined with the development of powerful deep learning models, new possibilities for the identification of settlements and archaeological structures have appeared. Leaning massively on transfer learning and pre-trained models, the aforementioned studies applying these newly developed models provide encouraging results to archaeologists. However, the diversity of applications, metrics, pre-processing, and the very nature of the neural network models all lead to a heterogeneity of methodologies available for this task.

4.2.4. Digital heritage

In the articles applying machine learning methods to digital heritage, we identified two different approaches. On the one hand, there were the study cases with a complete workflow, with a first step of semantic image segmentation preprocessing performed via machine learning before a second step of classification based on machine learning (Grilli and Remondino 2019; Nogales et al. 2021). These successive steps follow a gradual and systematic process, which meets the requirements for the Knowledge Discovery Process (cf. 4.2.1, Fayyad et al. 1996). On the other hand, there were works that focused either only on the semantic segmentation step (Felicetti et al. 2021; Matrone and Martini 2021) or only on the classification step (Toler-Franklin et al. 2010; Mesanza-Moraza et al. 2021; Prasomphan 2022; Pavan Kumar et al. 2022; Pepe et al. 2022).

The high number of ANN models for semantic image segmentation adapted to cultural heritage tasks confirms a tendency (Sultana et al. 2020), with ANNs generally obtaining better results compared to decision trees (Boston et al. 2022). Non-standardised datasets and the diversity in the preprocessing stages made for a heterogeneous assemblage of reviewed study cases, making them difficult to adequately compare. The lack of standard protocols (see part. 7 of Fiorucci et al. 2020) has led to the continued use of older models (e.g. Pepe et al. 2022). Limiting the use of colour images or reducing the number of classes has been suggested to improve future models (Grilli and Remondino 2019).

Two different approaches are currently used in machine learning techniques for digital heritage problems: one emphasises a more progressive and integrative workflow with, successive detailed pre-processing steps, while the other focuses on semantic segmentation. Although deep learning models, in particular ANNs, appear to be the most used models, results are still hardly trustworthy in most cases, due to the lack of comparative datasets and standardised metrics.

4.2.5. Text analysis

Text analysis is one of the earliest topics to benefit from machine learning applications (Boon et al. 2009), as the potential benefits were quickly identified (Richards et al. 2015). The quantity of information and data collected in the archaeological field rose exponentially following the development of new technologies, systematic recording processes, and the development of rescue archaeology (Brandsen 2023, p. 229). This veritable deluge of records (Bevan 2015) has made analysing the tremendous number of archaeological texts difficult. It is therefore unsurprising that machine learning methods already developed for text extraction in other fields found adoption in archaeology here.

Two approaches have emerged from this new demand for parsing through large numbers of archaeological reports, along with a third unrelated one. Firstly, machine learning methods can quickly find similarity between two texts where a human may only find heterogeneity (Boon et al. 2009). Secondly, machine learning can also be useful for highlighting the underrepresentation of certain archaeological findings in existing literature (e.g. cremation: Brandsen and Lippok 2021, p. 6). Lastly, researchers have also used machine learning to extract and interpret graphemes or other signs from archaeological material. This last application focuses on extracting glyphs from images (Dhivya and Devi 2021) or finding matching pieces from document fragments (Abitbol et al. 2021). For these image-based tasks, ANNs are the family of models used (Dhivya and Devi 2021; Abitbol et al. 2021) likely due to their effectiveness in computer vision tasks (Cf. 4.2.2). However, we consider the results to still be too limited as of now.

All study cases that used natural language processing, required for analysing content similarity in a text document, all relied on ANNs, except for the early application of Boon et al. (2009), which used instead an unsupervised memory-based-learning model (MBL). Large language models, having been introduced only recently (Vaswani et al. 2017), did not feature in our dataset due to our cutoff point of 2022, though there have already been some applications more recently (Chang et al. 2024). Despite their youth, the impact of large language models across modern society has been large (Tamkin et al. 2021; Eloundou et al. 2023; Clusmann et al. 2023; Tayan et al. 2024) and the archaeological field will undoubtedly implement these new models for either research purposes or education (Agapiou and Lysandrou 2023; Cobb 2023).

Despite its essential role in archaeological research, machine learning applications for text analysis are poorly represented in our corpus. Almost only relying on ANN models in our corpus, these applications face strong methodological and technical issues during the pre-processing of the textual data. However, the recent development of large language models, absent in our corpus, is likely to drastically change the automated analysis of archaeological textual data in the future.

4.2.6. Taphonomic classification

Despite the wide range of methods applied, many of the articles reviewed were found to contain important methodological issues, both from unclear and insufficient reporting of the methods applied, to important considerations of the data used that were seemingly eschewed.

Some of these concerns have already been highlighted (McPherron et al. 2022; but see the reply by Abellán et al. 2022, including its Supplementary Texts 1 and 2; Moclán and Domínguez-Rodrigo 2023; see also Courtenay et al. 2024), specifically the unclear methodology for the process of “bootstrapping” described in one publication in our reviewed dataset (Domínguez-Rodrigo 2018; but also present in our dataset in Domínguez-Rodrigo and Baquedano 2018; Aramendi et al. 2019; Moclán et al. 2019, 2020; and Courtenay et al. 2019). The unclear methodology reported in Domínguez-Rodrigo (2018) could be interpreted as stating that bootstrapping was applied to create a larger overall dataset which was then split for training and testing (as seems to also be explicitly stated in Domínguez-Rodrigo and Baquedano 2018; but less so in Aramendi et al. 2019 or Courtenay et al. 2019). This constitutes a methodological failure, as the models will share the same data for across both training and testing datasets, rendering any results unusable in the same way it would be to have an answer sheet when taking an exam. Although we do not wish to reiterate the “bootstrapping” discussion, it is still important to highlight how imprecise reporting of methodology and code unavailability can cast doubt of the validity of the methods applied.

When the data and code are made available, such as in Abellán et al. 2022, concerns are easier to both discard and confirm. For instance, the Ensemble.ipynb file provided in the Supplementary Data (henceforth SD) 2 contains an off-by-one error, which has the effect of excluding the last variable when training the models used. If this is indeed the code used for the main article (it is at least for the Supplementary Text 4), it would have considerable implications for the reported results and the accuracy of the reported methods. Moreover, the R file croc paper 2022 code.r contains additional problems for the “correct” bootstrapping section (module 4 in the file), and we could not run the code for this module without errors (with R 4.3.2, and caret 7.0–1).

On the other hand, the code suggests that the definition of “bootstrapping” used by the authors is to resample input data with replacement to obtain a larger synthetic input dataset and then performing k-folds when training the model (perhaps explaining the reported use of both bootstrapping and k-folds in Dominguez-Rodrigo 2018 and Domínguez-Rodrigo and Baquedano 2018). This is contrary to the ideal implementation of bootstrapping, where the resampling is done by each classifier in the ensemble learning model, thus creating multiple bootstrapped datasets (one per classifier), as well as providing an out-of-bag error to verify the accuracy of each classifier when aggregating the models (Breiman 1996). The code also implies that their “proper bootstrapping” process for creating a larger dataset may also be applied to the testing data after splitting as well, but this would severely skew model accuracy metrics, since the same data could be correctly or incorrectly predicted multiple times.

Other issues may require even more technical knowledge to identify, such as those in Moclán et al. (2019), who evaluated their models with both a bootstrapped and non-bootstrapped experimental sample, seeking to predict the agent (anthropic direct percussion, spotted hyena (C. crocuta) carnivory and wolf (C. lupus) carnivory) of bone fracture planes in bone fragments (Moclán et al. 2019). However, the dataset provided in the Electronic supplementary material 1 (ESM 1) also contains information on the bone fragment the fracture planes come from (e.g. epiphysis presence, length, and number of planes), rather than only the information of the plane itself (Moclán et al. 2019, app. ESM 1). In some cases a single bone fragment could have six fracture planes in the dataset, so when splitting the dataset, some input data may have come from the same bone as some testing data, and thus have this shared bone metadata, likely leading to overfitting as all bone fracture planes from the same bone fragment were caused by the same agent (see Varoquaux and Cheplygina 2022). Though no code was made available, the authors do state that these metadata variables were indeed used (Moclán et al. 2019, p. 4668). In addition, the number of entries in ESM 1 do not match the number of fragments reported for any of the three classes. Even when counting only unique bone fragments instead of planes using ESM 1, the numbers still do not match for any class. Furthermore, for the hyena class, it is also clear that some fragments did not have all their fracture planes added to the ESM 1 dataset, since there are an odd number of entries that state the planes per bone fragment is 2, thus implying there is at least one plane that was not included (Moclán et al. 2019).

The issues in Moclán et al. (2019) are likely also present in another article (Moclán et al. 2020; and perhaps by Abellán et al. 2022, at least for the tramp_cm_croc_MDR_EB.txt dataset, which contains a ‘number of grooves’ variable). Although the data and code used by Moclán et al. 2020 was not made public, the authors state that the work was a continuation of Moclán et al. (2019), repeatedly citing it as their methodological starting point, and likely using its trained model, especially as the variables described were identical. Moclán et al. (2020) focused on classifying bone fracture agency in archaeological – as opposed to experimental – material, They provide strong conclusions based on the results of a model with considerable unaddressed (and unacknowledged) problems. A similar issue could be raised for Abellán et al. (2022), though they are far more couched in their language. Nevertheless, Fig. 4 in Abellán et al. (2022) states that for some of the archaeological BSMs analysed, the models gave a classification probability of 99% for being crocodile-made, but at the same time also a 99% probability for being a cut mark. Oddly, the probabilities do not add up to 1 except for the last BSM shown (BOU-VP-11/12), the only one classified as a cut mark. Whether this was a transcribing error or a deeper issue is unclear, as detailed results are not available in the SD. The vast class imbalance of input data is another issue, despite the authors suggesting otherwise. Even classifying no crocodile BSMs correctly would provide an F1 score of 0.92 if only 11 out of 146 testing cut marks were misclassified, thus making the lack of confusion matrix or detailed results reporting concerning, even if no strong conclusion was put forth (see Courtenay et al. 2024).

The use of machine learning methods to draw any conclusion from its own predictions of archaeological data must be very carefully done, the limitations of the methods must be clearly and transparently examined, and the strength of the conclusions must be scrupulously tempered. Anything else should not be considered good scientific practice.

However, despite the lengthy discussion above, not all study cases reviewed had methodological shortcomings. The methods used in other articles in this category such as Byeon et al. (2019) and Cifuentes-Alcobendas and Domínguez-Rodrigo (2019) are promising and well-grounded (with only a few caveats, such as the latter’s omission on whether BSMs from the same bone were split into both training and testing, which could have unduly increased validation accuracy; see also Courtenay et al. 2024) and should definitely be the basis of further studies on the subject, including those seeking to independently verify and replicate the results of the numerous BSM agent classification studies, especially as recent attempts have not obtained the same optimistic results, despite employing robust methodology (Courtenay et al. 2024).