Archaeological Classification of Small Datasets Using Meta- and Transfer Learning Methods: A Case Study on Hittite Stele Fragments

2.1 Data Selection

The Hittite Empire was a major Bronze Age power centered in Anatolia (modern-day Turkey) from approximately 1600–1200 BCE. It reached its height under King Suppiluliuma I (14th century BCE), when it rivaled Egypt for control over the Near East. The Hittites were pioneers in iron processing and chariot warfare, giving them significant military advantages. Their capital was Hattusa (near modern Boğazkale, Turkey), featuring impressive architectural achievements including massive stone walls and gates (Bryce 2002; Genz & Mielke 2011; Gilibert 2011; Van den Hout 2020).

Among the material culture, we focus here on the stone relief stelae (Hawkins 2014; Hundley 2014; Cammarosano 2015). These stelae, originating from ancient Hittite centers such as Alacahöyük, Arslantepe, Karkamış, and Sakçagözü (Figure 1), offer invaluable insights into the Hittite cultural and historical milieu, depicting military triumphs, religious ceremonies, and daily life. Despite their significance, many stelae remain unidentified due to fragmentation and uncertain provenance, often resulting from illicit trade or the lack of information in private collections. Our study addresses the challenge of classifying these artifacts using visual attributes not biased by any verbal description or prior identification and employing advanced machine learning techniques. The goal is to advance in the prediction of the original provenience of fragmented stelae, based on the visual information we may learn and generalize from well-preserved items, whose archaeological context and provenience are well known. The direct objective of the present research is to determine to which of the four candidate cities the examples in the test dataset (specifically, the side with relief art) belong, based on stylistic differences and stone surface textures using gradient optimization-based models. Stelae from regions like Karkamış and Sakçagözü in southern Anatolia and northern Syria show a blend of Hittite and local styles, incorporating motifs and techniques from neighboring cultures. If we can distinguish these elements, it would be possible to infer the provenience of a fragment preserved in a museum or a private collection in the absence of contextual information for its origin.

Figure 1

To achieve these objectives, we have compared the performance of diverse machine learning methods, specifically meta-learning and transfer learning models, as well as hybrids of them, which have proven to be successful even with limited datasets. Additionally, a Hittite art expert conducted a classification to provide insights into the differences between AI and human expertise.

The image dataset utilized in this investigation was meticulously acquired by the first author (DK) under controlled conditions within the Ankara Anatolian Civilizations Museum’s Hittite Reliefs Exhibition Hall, where all photographic documentation was executed with consistent equipment parameters—identical camera model, uniform settings, and standardized ambient illumination conditions. We have detected minor variations in illumination intensity across the exhibition space and subtle angular deviations during photographic documentation. No effort has been made to eliminate those variations because the way to deal with the presence of noise and spurious variation aligns with the central research question underpinning our methodological approach.

For dataset preparation, we followed this procedure:

• Data Collection: The raw dataset comprised 298 JPEG images with a resolution of 4928 × 3264 pixels of stelae from the four previously mentioned Hittite centers (Alacahöyük, Arslantepe, Karkamış, and Sakçagözü).

• Data Cleaning: Duplicate images and those unrelated to the thematic focus were removed, resulting in a refined dataset of 88 images organized into separate folders based on the geographical location of the original stelae.

• Image Preprocessing: The RGB color channel information was preserved throughout our computational processing pipeline to maintain rich visual information requisite for nuanced stylistic classification. The 88 images were resized to 1500 × 1000 pixels. Some regions of these images were cropped to generate 412 images simulating “fragmented stelae”. Cropped patches were chosen deliberately, emphasizing those parts of the stelae particularly hard to discern due to degradation and alteration (Figure 2). This approach aims to simulate real-world scenarios where archaeologists face challenges in classifying highly deformed fragments that cannot be easily identified.

• Sample Selection: Of the 412 samples, we have processed only 136 to avoid differences in sample size for each provenience. Only 34 images are available from the Sakçagözü site. Our database contains the same number of stelae from each site.

• Training/Testing Split and Final Decision: We ended up with 28 training (support) and 6 testing (query) samples in each of the 4 classes, totaling 112 samples for training and 24 samples for testing (approximately 82% to 18%). Therefore, after all these steps, the initial dataset of this study consisted of 136 samples. The small number of testing cases has been compensated for using an explicit evaluation procedure (as discussed further in the paper).

Figure 2

Cropped images were standardized to a resolution of 224 × 224 pixels (Figure 3). This resolution balances the risks of overfitting, which can occur at higher resolutions, and the loss of critical feature information at lower resolutions. This level of image preprocessing is a fundamental step in computer vision pipelines, ensuring raw image data is properly formatted for deep learning architectures. Standardization of pixel values to a consistent range, resizing, and cropping have been considered essential, particularly for constrained datasets like ours, to optimize computational efficiency and model performance (Bengio, Goodfellow & Courville 2017). Spatial resizing supports mini-batch learning, mitigates memory limitations, and accelerates training and inference by reducing computational overhead. Trade-offs between resolution and batch size directly impact recognition accuracy, as recent advancements in recognition-aware preprocessing demonstrate. However, the goal of such operations is not perceptual quality but rather the accurate performance of the recognition model (Talebi & Milanfar 2018).

Figure 3

This initial dataset has been further divided into training and testing subsets for each Hittite city, with 28 training (support) and 6 testing (query) samples per provenience set, maintaining an approximate 82% to 18% split (Figure 4). We are conscious that this may seem very small for testing the predictive capabilities of a machine learning model; however, we could not obtain more viable specimens. We have prioritized a balanced dataset with 6 test samples per city, directly addressing the conspicuous lack of stelae from Sakçagözü in the Ankara Anatolian Civilizations Museum’s Hittite Reliefs Exhibition Hall. Small test set size is not a methodological oversight, but a deliberate choice to simulate authentic archaeological conditions. No specific strategy for data augmentation has been considered here, given the implicit problems it may cause. Artificially increasing the size of the dataset through augmentation can give a false sense of security. While the dataset appears larger, it may not actually contain more diverse or informative examples. This can lead to overconfidence in the model’s performance (Wong et al. 2016).

Figure 4

2.2 Enhanced Dataset Implementation

To investigate the impact of increased data volume on model performance, an expanded version of the original dataset was created. The enhanced dataset featured a substantial increase in both training and testing instances: evolving from the original 136 instances (112 for training, 24 for testing; an 82.35%–17.65% split) to 208 total instances (152 for training, 56 for testing; a 73.08%–26.92% split). This expansion more than doubled the test set size, increasing it from 24 to 56 instances. A balanced distribution across the four archaeological classes was maintained, with the enhanced dataset providing 38 training samples per class (compared to 28 in the initial dataset) and 14 testing samples per class (compared to 6 previously).

The samples used for this expansion were selected from a reserve collection of digital photographs, featuring additional Stelae from the Ankara Museum of Anatolian Civilizations. These images were captured using identical equipment and under standardized conditions, consistent with the original dataset. All new samples underwent the same preprocessing, transformation, and cropping procedures as outlined in the methodology. Furthermore, model training parameters remained identical to those employed with the original dataset. This methodological consistency ensures direct comparability of results and allows for the isolation of data volume effects on classification performance. The considerably larger test set provides a more robust foundation for evaluating model generalization.

It is crucial to differentiate this dataset enhancement from traditional data augmentation techniques common in machine learning. The expansion was achieved by incorporating entirely novel, genuine samples from the reserve collection, rather than by applying synthetic manipulations (such as rotation or grayscale conversion) to existing images. The scope of this expansion was ultimately determined by the limited availability of artifacts from the Sakçagözü site, which dictated an upper boundary of 14 test samples per class. This constraint resulted in a balanced test dataset of 56 unique instances (14 samples × 4 classes), composed of authentic archaeological artifacts.

The dataset was expanded by 18 new samples per class, totaling 72 additional examples across the four classes. The allocation of these new instances was systematic: 10 examples per class (40 total) were added to the training subset, and 8 examples per class (32 total) were added to the testing subset. This differential proportional allocation, leading to the revised 73.08%–26,92% training–testing split, was a deliberate methodological choice. The primary aim was to significantly enlarge the test set, thereby providing a more rigorous framework for assessing the generalization capabilities of the models. The final dataset dimensions of 152 training samples and 56 testing samples represent a considerable volumetric increase. This approach ensured that while the training/testing proportions were adjusted, class balance was maintained, and the expanded dataset offered substantially improved conditions for robust model evaluation and the assessment of generalization.

2.3 Methodological Overview

We have compared the results of four different supervised learning approaches for predicting the provenience of our small set of Hittite stelae:

• Classical Machine Learning Algorithms: Traditional approaches, including Support Vector Machines (SVM), Decision Trees (DT), Random Forests (RF), Logistic Regression (LR), and Naive Bayes (NB).

• Transfer learning: A pre-trained deep learning model, ResNet18, was utilized to leverage transfer learning for this task.

• Simple CNN with FSL: A straightforward convolutional neural network (CNN) architecture was employed in conjunction with FSL. As it will be argued later, without the integration of FSL, the simple CNN architecture underperformed, failing to generalize and resulting in underfitting. Therefore, cross-validation has not been implemented in this model.

• Hybrid Approach: A combination of the ResNet18 pre-trained network architecture and a gradient-based meta-learning algorithm (model-agnostic meta-learning, MAML) was applied, incorporating few-shot learning (FSL) techniques.

The performance of these four approaches was evaluated against the classification accuracy rate achieved by a human expert as a reference value, providing a benchmark for comparison and highlighting the relative efficacy of advanced ML methodologies in this context.

Convolutional neural networks (CNNs) are designed specifically to preserve spatial autocorrelation through convolutional filters that detect patterns across neighboring pixels. They allow hierarchical feature extraction, which captures both low-level textures and high-level shapes. Furthermore, they offer translation invariance that allows them to recognize features regardless of their position. This makes CNNs dramatically more effective for distinguishing the intricate carved details, weathering patterns, and stylistic elements that characterize different periods and regions of Hittite stone monuments.

Transfer learning (pretrained models) is an approach where knowledge gained from training a model on one task is applied to improve learning on a related task. A CNN is “pre-trained” on a large dataset (such as ImageNet; Deng et al. 2009) with millions of diverse images across thousands of categories. This teaches the network to recognize fundamental visual features like edges, textures, shapes, and complex patterns. The pre-trained network is then repurposed for a new, specialized task (such as Hittite stelae classification) by keeping the early and middle convolutional layers (which capture universal visual features) and replacing and retraining the final layers to recognize specific features relevant to the new task (Weiss, Khoshgoftaar & Wang 2016; Zhuang et al. 2020). The goal is to adjust the model’s weights to better suit the new task. In fine-tuning, some or all of the pre-trained layers are unfrozen, and the entire model is trained on the new dataset. Fine-tuning is a specific form of transfer learning where the pre-trained model’s layers are unfrozen and allowed to be updated during training on the new task. This allows the model to learn more task-specific features while still benefiting from the pre-trained weights (Schmidhuber 1987; Pan & Yang 2009; Vanschoren 2018; Hospedales et al. 2021).

ResNet18 is a CNN architecture with 18 layers, and it has been trained on over a million images from the ImageNet database (Stanford Vision Lab 2020). It can classify images into 1000 object categories and has learned rich feature representations for a wide range of images. It is an example of a residual neural network in which the layers learn residual functions with reference to the layer inputs (He et al. 2016). In this paper, we started with the pre-trained ResNet18 model; we removed the final classification layer, substituted it with our Hittite stelae classification task (four neurons, one for each of the four candidate Hittite centers: Alacahöyük, Arslantepe, Karkamış, and Sakçagözü), and fine-tuned the model with our image dataset from the Ankara Museum. We opted for the ResNet18 deep learning architecture because of its computational efficiency, attributable to its relatively low parameter count. Although the specific content of ancient engraved stelae may differ from the natural images in ImageNet, the pre-trained model can still provide a strong starting point. The model can be fine-tuned to adapt to the specific characteristics of the stelae images, such as the depiction of human figures holding objects. Given the limited size of our dataset, ResNet18 offers an optimal balance between model complexity and adaptability, facilitating effective learning while minimizing computational overhead (Liu et al. 2023).

For ResNet18 processing, we used standard convolutional filtering. In other models, to reduce the impact of small size and simplify the models, we used the scikit-learn library (Pedregosa et al. 2011) to transform multi-dimensional input tensors into flattened representations where image data with dimensions characterized by height (h), width (w), and channels (c) undergo transformation into single-dimensional vectors of length h × w × c.

Meta-learning, also known as “learning to learn,” is a subfield of machine learning where an algorithm gains experience over multiple learning episodes, using this experience to improve its learning performance over time. It is particularly useful when dealing with extremely limited datasets, as it allows the model to generalize from a few examples by leveraging knowledge acquired from previous tasks. In the meta-learning literature, the terms “training dataset” and “test dataset” are conceptually referred to as the “support dataset” and “query dataset”, respectively. Few-shot learning (FSL) is a special case of meta-learning designed to train models that can recognize new patterns or classes after seeing only a few examples, rather than requiring thousands of labeled samples. Beginning with a pre-trained CNN (transfer learning) to start the classification task with a strong foundation of general features and patterns, FSL allows a model to adapt its knowledge to a new domain with minimal examples (typically 1–5 per class). This is where the “few-shot” name comes from (Schmidhuber 1987; Naik & Mammone 1992; Thrun & Pratt 1998; Ravi & Larochelle 2017; Wang et al. 2020; Hospedales et al. 2021; Song et al. 2023).

Many few-shot techniques work by learning a similarity metric or embedding space where similar items cluster together. New examples are classified based on their proximity to the few support examples in this space. Techniques like model-agnostic meta-learning (MAML) train the model to be easily adaptable to new tasks with minimal gradient updates, essentially “learning how to learn” from a few examples. Its main goal is to find an optimal set of initial parameters that may serve as a strong starting point for quick adaptation across a wide range of tasks. During training, MAML uses a nested optimization procedure with two levels: a) In the inner loop, the model temporarily adapts its parameters to specific tasks using just a few examples (the “support set”) and a small number of gradient steps; b) in the outer loop, these adapted parameters are evaluated on new examples from the same tasks (the “query set”), and the loss is used to update the original initialization parameters. By repeating this process across many different tasks, MAML learns initial parameters that are positioned in parameter space such that small changes can produce good task-specific models. This makes the model inherently flexible and quick to adapt to previously unseen tasks during testing, requiring only a handful of examples to achieve good performance (Finn, Abbeel & Levine 2017; Vanschoren 2018; Hospedales et al. 2021).

MAML is particularly valuable because of its “model-agnostic” nature, which allows for its application to any architecture optimizable through gradient descent, including neural networks for classification, regression, and reinforcement learning. As archaeological datasets typically consist of limited samples, the meta-learning approach offers a more efficient and adaptable solution for artifact identification and classification. It can bridge the gap between AI capabilities and archaeological necessities. More details in Appendix 1, with an exhaustive mathematical background of some of these methods.

2.4 Implementation and parameter setting

For classical machine learning methods, 2D images of 224 × 224 pixels were flattened into 1D vectors of 50,176 components. This transformation of high-dimensional image data into vectorized numericals implies a necessary dimensionality reduction to allow for reasonable computer processing times (Chychkarov et al. 2021; Géron 2022). Details appear in Table 1.

In our CNN models, the output layer is defined by four units corresponding to four categories: Alacahöyük, Arslantepe, Karkamış, and Sakçagözü, the four areas of provenience of the stelae according to museum documentation and annotated in each image. Each neuron outputs a probability via the softmax function, indicating the probability that a fragment originates from each city. This setup provides a probabilistic assessment across all classes, enabling accurate classification and aiding in the archaeological interpretation of these ancient artifacts. Table 2 highlights the different hyperparameter configurations used across the four model approaches in the study. It shows the progression from simple conventional machine learning models to more sophisticated neural network architectures with advanced optimization techniques and regularization strategies. For hybrid and transfer models, the same hyperparameters were used.

2.5 Validation procedure

Training relied on a small “support set” of labeled examples for the four classes (112 images from stelae of known origin, 28 images per class: Hittite city). For preliminary validation, we implemented 6 known examples from each of the four proveniences, with a total of 24 images. A secondary validation with an enhanced dataset was also carried out with 14 samples for each class.

An initial procedure involved dividing the 112 training instances into three roughly equal subsets, or folds, each comprising approximately 37–38 samples. The cross-validation then proceeded iteratively: in each of the three iterations, two folds (around 74–75 samples) served as the training data for that specific iteration, while the third, remaining fold (around 37–38 samples) was utilized as the validation set. Key performance metrics such as accuracy, precision, recall, and F1-score were computed on this validation set in each iteration. The average of these metrics over the three folds yielded a more reliable estimate of the model’s generalization capability on data not seen during that iteration’s training. Crucially, the 24 test samples (6 per class) of the first dataset were held out entirely during this cross-validation process and were only engaged after model training and validation were complete, ensuring a final, unbiased assessment of the chosen model’s predictive accuracy on new unseen data (Burkov 2019; Alpaydin 2021; Géron 2022). For the enhanced dataset implementation, the models included 152 training samples and 56 testing samples, totaling 208 samples. In this case, the task size is 4-way 38-shot for support (training) and 4-way 14-shot for query (test).

3.1 Preliminary results with the initial training and testing data set

The evaluation of machine learning approaches to Hittite stele classification was conducted across two distinct experimental phases to assess both baseline performance and scalability. The initial experimental phase utilized 136 samples (112 training, 24 testing) to establish proof-of-concept effectiveness under severely constrained data conditions. Subsequently, an enhanced dataset phase employed 208 samples (152 training, 56 testing) to evaluate performance improvements with increased data availability. Throughout both phases, human expert classification was conducted by an associate professor specializing in Hittite art, providing authoritative benchmarks for model validation. In the initial dataset phase, the human expert achieved 62.5% accuracy on the 24-sample test set, correctly classifying 15 of 24 fragments. This performance established our baseline reference for evaluating machine learning effectiveness under challenging conditions where even expert human classification faces significant difficulties due to fragment degradation and limited diagnostic features. The enhanced dataset phase revealed substantially improved human expert performance, achieving 85.7% accuracy on the expanded 56-sample test set, correctly classifying 48 of 56 fragments. This dramatic improvement (23.2 percentage points) demonstrates the critical importance of reference material availability for both human and machine-based archaeological classification.

The different methods of classifying Hittite stelae according to their provenance have been evaluated in terms of different parameters:

• Precision: This measures the accuracy of the positive predictions made by the model. It is the ratio of correctly predicted positive observations to the total predicted positives. High precision indicates that the model has a low false positive rate, meaning that when it predicts a positive outcome, it is likely to be correct.

• Accuracy: This measures the overall correctness of the model. It is the ratio of correctly predicted observations (both positive and negative) to the total observations. High accuracy indicates that the model is making correct predictions for a large portion of the dataset. However, accuracy can be misleading if the dataset is imbalanced.

• Recall (Sensitivity): This is the ratio of correctly predicted positive observations to all observations in the actual class.

• F1 Score: This is the harmonic mean of precision and recall (sensitivity), providing a single metric that balances both concerns. A high F1 score indicates that the model has a good balance between precision and recall.

As expected, our results indicate that statistical learning models, although they exhibit very high accuracy and precision (higher than 0.9) on training data, show very low performance scores (below 0.5) on test data. This is a clear indication of overfitting. Such models have classified the training data too well, including its noise and peculiarities, rather than learning the underlying patterns that may generalize to new, unseen data. This is a clear consequence of limiting the comparison to pixel values and not to the spatial organization of those same pixels. The reasons for this may be:

• SVM operates in a high-dimensional feature space but treats each pixel as an independent dimension without considering its spatial relationships.

• Decision Trees make binary splits on individual feature values, losing the ability to recognize patterns that depend on pixel neighborhoods.

• Random Forests build multiple decision trees that make splits based on individual pixel values, but none of these decision trees can efficiently capture patterns that span multiple adjacent pixels.

• Logistic Regression uses linear combinations of individual features, making it unsuitable for capturing complex, non-linear spatial patterns in archaeological imagery.

Transfer and meta-learning methods show far better results in testing generalizations and predictive power, arriving at 81–83% of correctly classified items in the test stage, using, however, very small datasets for this testing. Although the small size of the testing database may compromise the results, we should consider that machine learning results are better than those obtained by a voluntary human expert, an associate professor in Hittite Art from a university, who was able to correctly classify the same test items in 62.5% of cases. Precision, recall, and F1-scores are consistently higher than those obtained with statistical learning methods (Tables 3 and 4). Respective confusion matrices are presented in Appendix 2.

Table 4 provides comprehensive performance metrics for traditional machine learning approaches, revealing consistently poor performance with test accuracies uniformly below 50% despite achieving near-perfect training accuracies in several cases. Support Vector Machines attained 39.29% test accuracy, Random Forests reached 42.86%, and Logistic Regression achieved 50.00%—the highest among traditional methods. These results indicate severe overfitting, where models learned training data specificities rather than generalizable patterns, a predictable consequence of high-dimensional feature spaces combined with limited training examples.

The transfer learning model based on ResNet18 achieved 72.22% test accuracy with 83.93% validation accuracy, demonstrating effective adaptation of pretrained features to archaeological classification tasks. While lower than the hybrid approach, this performance still exceeded human expert classification by 9.72 percentage points, validating the effectiveness of transfer learning for archaeological applications even without meta-learning enhancements.

Three-fold cross-validation (as detailed in the Methods section; Figure 5 and Table 5) enabled the comprehensive utilization of the dataset for both training and validation, mitigating the limitations of a single train-test split. The confusion matrices across folds indicated consistent performance, with total elements summing to approximately 34, closely aligning with the expected average values derived from the dataset’s size and folds. The model’s performance metrics averaged across folds demonstrated a high degree of reliability: accuracy (86.9%), precision (79.6%), recall (80.3%), and F1 score (79.9%). The consistent results across folds highlight the model’s stability and generalizability when applied to diverse data splits, underscoring its potential for accurately classifying archaeological artifacts (Bengio, Goodfellow & Courville 2017; Alpaydin 2021).

Figure 5

A three-fold cross-validation approach was selected to balance data utilization and evaluation robustness, given the small dataset size. While fewer folds (e.g., 2) may lead to high variance due to larger test sets, more folds (e.g., 4) could result in smaller training sets and less reliable metrics. Three folds allow training the model on approximately 67% of the data, which is enough to learn meaningful patterns while still reserving substantial data for validation. This choice enables meaningful performance assessment while mitigating the limitations of small datasets (Bengio, Goodfellow & Courville 2017; Alpaydin 2021).

Despite the inherent limitations of our small testing dataset, our cross-validation results demonstrate consistency across folds, achieving an average accuracy of 86.9%, higher than the results obtained by the human expert. This high score supports the idea that the optimal three folds achieve an optimal trade-off between bias and variance. This consistent performance across different data splits suggests robust generalization capabilities, indicating the model’s ability to learn underlying patterns even with limited and poor-quality examples. Importantly, the observed misclassification patterns align with established archaeological knowledge regarding artistic similarities between certain cities, providing further evidence that the model is capturing meaningful relationships within the data. While a larger test set might offer some advantages, our current methodology offers a more authentic representation of the challenges inherent in archaeological classification.

Table 6 presents disaggregated results for the hybrid model, revealing class-dependent variations in performance. Classification of Sakçagözü fragments demonstrated a strong balance between precision (0.811) and recall (0.748), leading to a robust F1-score of 0.770. However, Alacahöyük fragments exhibited the lowest performance among all categories, with a precision of 0.586 and an F1-score of 0.604, indicating challenges in distinguishing fragments from this geographical provenance. These variations suggest that different regional styles present varying levels of classification difficulty, with some possessing more distinctive visual characteristics than others.

Table 7 reveals distinct performance patterns across Hittite archaeological sites that illuminate transfer learning strengths and limitations. Sakçagözü demonstrates exceptional performance with perfect test precision (1.0) but reduced recall (0.7778), indicating confident but occasionally incomplete identification. Arslantepe exhibits the most dramatic performance decline, with recall dropping from 0.8630 to 0.50, suggesting significant challenges in recognizing this regional style in unseen examples. Karkamış consistently shows the lowest performance across validation and test phases (F1-scores of 0.7079 and 0.6253), indicating that this production center presents the greatest classification challenges. These differential patterns suggest varying levels of visual distinctiveness among Hittite regional styles, with important implications for archaeological interpretation.

Upon the analysis of the simple CNN+FSL classification metrics in Table 8, the model has different accuracy and precision, depending on the output. In the case of Sakçagözü stelae, it shows robust recall coefficients (0.93 and 0.92 in classifying training data and predicting test instances, respectively) and commendable precision metrics (0.63 and 0.79), culminating in optimal F1-scores (0.75 and 0.85). Conversely, Alacahöyük Stelae are the most difficult to predict given the evidence of overfitting, as reflected in the precipitous decline in F1-scores from 0.50 to 0.16, as it is the case with Stelae interpreted as produced in Karkamış (F1-scores decreasing from 0.55 to 0.40). In this simple CNN+FSL model, Arslantepe stelae, which are very well classified and predicted with other models, show suboptimal classification and prediction efficacy (F1-scores of 0.47 and 0.32).

The performance metrics reveal significant challenges in classifying Hittite stelae from the four provenience sites, with clear evidence of overfitting and highly variable performance across different archaeological sites. This preliminary model demonstrates poor generalization, with average test performance (43% F1-score) substantially lower than training performance (56% F1-score). This 13-point drop indicates the model is overfitting to the limited training data rather than learning generalizable features that distinguish stelae from different archaeological sites. Except for the classification of Sakçagözü samples, whose high test recall rate (0.92) suggests the model rarely misses Sakçagözü stelae, the remaining are not well classified, with Alacahöyük as the worst performing classifier. The poor precision (0.15) and recall (0.17) suggest the model struggles to distinguish Alacahöyük stelae from other sites, possibly due to subtle stylistic differences or insufficient training examples. The great variation in performance between sites suggests uneven representation in the training data and that the model is memorizing training-specific features rather than learning generalizable archaeological patterns. With only 6 test images per site (24 total ÷ 4 sites), these metrics have extremely high variance. A single misclassification represents a 16.7% error rate, making the results unreliable. Results improve dramatically when we add transfer learning to the equation (Table 7).

The average performance is fairly good. The classification of training data is excellent across all sites, with F1-scores ranging from 0.88 to 0.95, indicating the model successfully learns to distinguish between different provenience sites during training. Although the testing set is small, its classification is also very good, with F1-scores above 0.80, showing the model has learned transferable features rather than memorizing training-specific patterns. The classification of samples from Sakçagözü is excellent, with a 0.92 testing F1-score, showing perfect recall (1.00) and strong precision (0.86). This consistency suggests robust feature learning for this site’s distinctive characteristics. Alacahöyük samples seem to be the worst classified, suggesting inherent classification challenges for this site’s stelae. Karkamış data achieves a solid 0.83 F1-score with balanced precision and recall. The Arslantepe samples classification shows perfect precision (1.00) but lower recall (0.83), indicating the model is conservative in its predictions but accurate when it does classify stelae from this site. These results suggest that low-level visual features learned on ImageNet (edges, textures, shapes) are highly relevant for distinguishing archaeological artifacts, even across vastly different domains. The smaller gap between training and testing data classification (0.92 vs 0.83) indicates good generalization, likely due to the pre-trained weights providing a strong initialization. Despite significant improvement, Alacahöyük remains problematic with a 0.67 F1-score, suggesting this site’s stelae may share visual characteristics with other sites or require domain-specific feature engineering.

The transfer+meta-learning consisting hybrid model (Table 6) demonstrates exceptional performance and represents a significant advancement, achieving higher and more consistent results across all Hittite stelae provenience sites. Test performance (0.821) exceeds training performance (0.725), indicating excellent generalization capabilities rather than overfitting. This counterintuitive improvement from training to testing suggests the meta-learning approach has successfully learned to adapt and generalize to new examples. Arslantepe classifications achieve near-perfect performance with a 0.933 F1-score, perfect test precision (1.00), and excellent recall (0.889). This represents the best single-site performance across all tested models. Alacahöyük classifications show dramatic improvement, achieving an 0.797 F1-score. The balanced precision (0.804) and recall (0.833) indicate the meta-learning approach has successfully addressed the classification challenges for this site. Sakçagözü results suggest the strong performance of the classificatory model, with a 0.797 F1-score, showing the model’s robustness across different site characteristics. The same can be said about Karkamış results, achieving a 0.755 F1-score with balanced metrics, representing consistent improvement over previous approaches. The model’s ability to improve from training to testing exemplifies meta-learning’s core strength: learning to learn quickly from limited examples, which is ideal for archaeological datasets with scarce labeled data. Unlike the simple pre-trained transfer model that showed variation between sites, the meta-learning model achieves more uniform performance (F1-scores ranging from 0.755 to 0.933), indicating robust feature learning. Compared to plain ResNet18 results, meta-learning shows marginal improvement in average performance (0.821 vs 0.83) but with superior consistency across sites and better generalization characteristics.

To sum up, our results prove that in Hittite art and historical studies, it is feasible to accurately predict and trace the production centers of decorated stone fragments, even when these artifacts are heavily eroded and deteriorated. The hybrid model, which integrates MAML with FSL and transfer learning techniques using the ResNet18 architecture, performs well in classifying stele fragments from four significant archaeological sites: Alacahöyük, Arslantepe, Karkamış, and Sakçagözü. Achieving an impressive classification accuracy of approximately 83%, these machine learning models effectively discern subtle stylistic and iconographic elements that reflect the distinct production techniques of each region, even with a tiny dataset.

The model’s analytical capabilities extend to the morphological and decorative motifs of the stele fragments, successfully identifying specific cultural signatures unique to each site. Although stelae from Alacahöyük, with their distinctive designs and unique stylistic attributes, may seem easier to classify and predict, we have also obtained very good results with Sakçagözü, which exhibits considerable variability in design. We have discovered that difficulties in classification do not prevent obtaining good prediction results with testing data. The performance of the different machine learning models we have compared not only advances our understanding of Hittite artistic production but also enriches provenance studies, providing a robust framework for associating fragmented artifacts with their production centers. This innovative approach marks a promising advancement in AI-based archaeological methodology, harnessing artificial intelligence to complement and extend traditional analytical techniques, thus providing opportunities for novel insights into Hittite society.

In this study, we did not investigate in detail the differences in performance of the different tools with the target types. This part of the research is still ongoing. We know that in our case study, as in others, different algorithms are more or less satisfactory depending on the specific characteristics of the input. The misclassification rate is different in each class because the stylistic features defining each provenience were different. Our results are still a black box: we know that stelae from different regions are different, and the machine knows how to distinguish them, but we researchers still do not know why. We need a deeper stylistic analysis that will go far beyond the limits of this paper, focusing on the ability of the algorithms to correctly classify and predict. We have checked that the different state of preservation of the reliefs does not affect the results, and that neither resizing nor cropping has “penalized” some images more than others. Classification results are a consequence of the intrinsic homogeneity of images related to each particular site and the heterogeneity between them.

3.2 Testing the classificatory model with additional data

To investigate how increasing the dataset size affects classification performance, particularly testing, a parallel set of experiments was conducted with our enhanced dataset (208 instances). With 14 test examples per class (56 total), these results provide more statistically robust comparisons than previous smaller datasets, increasing confidence in the observed performance patterns. Basic parameters are the same as in Table 2, to ensure comparability of results. It should be remarked that the human expert predictions were far better with this additional data, with an average of 85.7% success.

Table 9 indicates the improvement rates after enhancing the dataset’s test samples. Results from classical statistical learning are still poor, although they have improved with additional data, especially in the case of K-nearest neighbors. It is interesting to note the impossibility of generalizing a classificatory rule for Random Forest, Logistic Regression, and Decision Trees: after classifying correctly all training data, they fail when applying the learned rule to new data. results from CNN models are far better, although fairly similar to results with the smaller database, as if the new evidence has not improved results. Again, meta-learning shows better results than simple transfer learning. Results from non-pre-trained CNNs are bad, even with meta-learning. Results of transfer learning (without and with meta-learning) are far more consistent (Table 10).

While the expanded dataset shows improvement, the gain is surprisingly modest considering the additional training data, suggesting the meta-learning approach was already performing near its optimal level with the limited initial dataset. However, in the case of the classification of Alacahöyük samples, the classifier benefited most from additional training examples, going from an initial F1-score of 0.797 to a 0.88 F1-score with the additional data (+10.4%). In contrast, classifications for samples coming from Arslantepe show a slight decline, suggesting the additional data may have introduced more challenging examples. The modest overall improvement (+4.7%) despite additional training data demonstrates meta-learning’s exceptional data efficiency. The initial small dataset was sufficient for the algorithm to learn effective adaptation strategies. These results suggest meta-learning approaches may experience diminishing returns from additional data, as they’re designed to learn quickly from limited examples rather than requiring large datasets. It is important to take into account:

• The expanded dataset likely provides more statistically reliable results due to larger sample sizes, even if absolute performance improvements are modest.

• The additional data should result in narrower confidence intervals around the performance estimates, increasing the reliability of the reported metrics.

• The fact that some sites showed performance decreases with additional data suggests the meta-learning model had achieved stable generalization with the initial dataset, and additional examples may have introduced noise rather than signal.

• The results emphasize that for meta-learning approaches, data quality and diversity may be more important than raw quantity, as the algorithm is designed to extract maximum learning from minimal examples.

This comparison provides strong empirical support for meta-learning’s core value proposition: achieving high performance with limited training data. The ability to reach an 82% F1-score with the initial small dataset, with only modest improvements from additional data, validates meta-learning as the optimal approach for archaeological computer vision, where data scarcity is inherent to the domain.

Table 11 presents a detailed comparative analysis for enhanced dataset predictions across all classes, showing that while the human expert achieved slightly better overall accuracy (85.7%) compared to the hybrid model (82.74%), the patterns of errors differed meaningfully between human and machine classification. Both approaches achieved perfect or near-perfect performance on Sakçagözü samples, but the hybrid model excelled with Aslantepe artifacts (92.9% accuracy) while the human expert performed better with Alacahöyük samples (85.7% accuracy).

3.3. Human-based learning and Machine Deep Learning

In a preliminary experiment, with the initial 136 images, the human expert assigned 24 test samples to each of the 4 centers, according to his previous experience. In contrast, the different machine learning models required indexing testing data differently to perform predictions. Each model predicted the provenance of the 24 test samples by referencing 4 training folders (6 samples in each folder) with matching names.

Results are clearly unbalanced, in favor of AI-based models, which perform better than the human expert: 85.7% vs. 62.50% of precision. The highly degraded nature of some of the samples may explain the fair results obtained by the human expert.

In the case of the enhanced dataset, with 14 test examples per class (56 total), the influence of very degraded samples diminishes, and therefore, the results are better. The human expert has obtained an 85.7% accuracy. These results do not show any superiority of human over AI-based learning, and vice versa. Furthermore, the idea of consulting a human expert is to obtain some insights into the machine’s performance. The human expert was merely requested to classify the same dataset so that some insights could be gained. The expert evaluation protocol maintained experimental rigor by presenting samples in randomized order without indication of their actual provenance, preventing confirmation bias or other systematic errors that might artificially inflate performance metrics. This blind evaluation approach ensured that human expert results provide legitimate benchmarks for assessing machine learning effectiveness rather than serving merely as validation of known classifications.

Stelae from Sakçagözü, the human expert, the naked transfer model, and the transfer+meta-learning model achieve perfect or near-perfect performance (100%, 92.9%, 100%). This suggests Sakçagözü stelae possess highly distinctive visual characteristics that are easily recognizable by both human experts and AI systems. The Alacahöyük stelae have been the most difficult to identify for AI models. Human expertise provides a 7.1% advantage, suggesting subtle stylistic features that require archaeological knowledge to distinguish, and not only image features. The same can be said for Karkamış, although AI-based results are only moderately worse. In the Arslantepe case, the hybrid transfer+meta-learning model outperforms the human expert by 14.3%, suggesting the AI has learned visual patterns that even experts find challenging to consistently identify.

In the initial dataset phase, utilizing 136 samples (112 training, 24 testing), we established proof-of-concept effectiveness under the most challenging conditions archaeologists typically encounter. The human expert achieved 62.5% accuracy on the 24-sample test set, correctly classifying 15 of 24 fragments. This performance established our baseline reference for evaluating machine learning effectiveness under conditions where even expert human classification faces significant difficulties due to fragment degradation and limited diagnostic features. The enhanced dataset phase, employing 208 samples (152 training, 56 testing), revealed substantially improved human expert performance, achieving 85.7% accuracy on the expanded test set by correctly classifying 48 of 56 fragments. This dramatic improvement of 23.2 percentage points demonstrates the critical importance of comprehensive reference material availability for both human and machine-based archaeological classification, establishing a fundamental principle that applies across both cognitive and computational approaches to pattern recognition in archaeological contexts.

3.4 Additional validation

Gradient-weighted Class Activation Mapping (Grad-CAM) utilizes the gradients of any target concept (such as a specific class score) flowing into the final convolutional layer to produce visualizations. These gradients indicate the importance of each feature map for a particular decision. The technique calculates the derivative of the output of the reduction layer for a given class with respect to a convolutional feature map. Regions where this gradient is large correspond to areas where the final score depends most on the input data. The gradients are then combined and processed to create a heatmap that can be overlaid on the original image with transparency (typically 50% alpha), thereby simultaneously displaying both the original image and the areas of focus (Berganzo-Besga et al. 2022).

We employed this approach to ensure that our best-performing AI-based model focuses on genuine stylistic elements rather than background stone textures or irrelevant image artifacts. We relied on the PyTorch Grad-CAM package (https://github.com/jacobgil/pytorch-grad-cam) for AI explainability to verify that the algorithm learned correctly and was free of biases related to data acquisition. The method was implemented with a Grad-CAM threshold of 0.50 to facilitate visual interpretation (ranging from 0 to 1, where 0 indicates insignificance and 1 represents decisive influence in determining the image’s class). This visual explanation helps to elucidate the patterns used by the deep learning model to classify settlements and, consequently, to better understand Hittite stelae.

To ensure that our models classify stelae based on iconographic content rather than irrelevant background features, we defined a new model using five input classes, adding a new category labeled “background.” The training dataset contained samples in which the material characteristics of the stone support or color/brightness aberrations interfered with stylistic features. Once the best model was selected, another model was trained with the additional background class, which concentrated potential misclassifications caused by background relevance over stylistic motifs. The new training dataset contained 155 images, with 31 per class (including the background class).

In a preliminary experiment, using data trained without the additional fifth class (Figure 6), the heatmaps revealed that the model primarily focused on specific engraved features and relief patterns rather than background areas or stone texture alone. The red/orange activation zones consistently highlighted carved figures and reliefs. Where these features were visible, the model showed strong activation on human figures, animals, and decorative motifs, capturing unique stylistic features that likely distinguish different cultural origins. The model appeared to be learning culturally specific artistic styles rather than relying on irrelevant features such as lighting or background. Similar types of engravings (e.g., human figures, geometric patterns) received comparable attention across samples. Nevertheless, some heatmaps displayed scattered activation rather than coherent focus on complete motifs. In certain images, edge effects were observed, where activations appeared concentrated on artifact boundaries, which may indicate that the model was learning shape rather than content. Better-preserved areas received more attention, potentially overlooking diagnostic features in damaged regions.

Figure 6

When re-training data with the fifth class, the addition of a stone background class appears to have significantly improved the model’s ability to distinguish between carved content and substrate material. Several improvements are evident (Figure 7 and Table 12).

Figure 7

The heatmaps show:

• More focused activations on engraved features rather than stone texture variations.

• Reduced noise from natural stone patterns and surface irregularities that previously may have confused the classifier.

• Cleaner attention maps with less scattered activation across non-diagnostic areas

• Sharper boundaries around carved elements.

• Better figure-ground separation. The model now more effectively ignores background stone variations while highlighting cultural artifacts.

• Consistent attention patterns across different preservation states and lighting conditions.

While improved, some challenges persist:

• Fragmented activations still occur in heavily weathered areas.

• Edge effects remain visible where carved and uncarved regions meet.

• Variable attention intensity across different artifact preservation states.

The comparison between four-class and five-class model attention patterns proved particularly revealing. While the four-class model occasionally exhibited attention to stone surface characteristics alongside iconographic elements, the five-class model with background class showed markedly more focused attention on carved depictions. This pattern suggests that the inclusion of negative examples through the background class, despite reducing overall accuracy, produced more archaeologically valid classification strategies that better align with human expert methodology.

The developed pre-trained ResNet18 model was evaluated with a nested cross-validation approach (outer and inner k = 3) due to the small number of samples instead of using the traditional hold-out method where the dataset is divided into training, validation and testing datasets. This approach, although more computationally expensive, guarantees greater reliability and stability of accuracy. Therefore, it is optimal for limited datasets (Vabalas et al. 2019).

Grad-CAM analysis suggests that the observed performance reduction may result from the integration of additional features and, consequently, greater original variation. The background class effectively functioned as a negative control, compelling models to develop precise attention mechanisms focused on iconographic content rather than relying on broader visual patterns that might include non-diagnostic features. Grad-CAM visualizations across all four archaeological site classes confirmed that our models successfully learned to attend to carved depictions and iconographic elements rather than irrelevant photographic or stone surface characteristics. Heatmap analysis revealed concentrated attention on areas containing human figures, symbolic representations, and decorative motifs—precisely the features that human experts examine when determining stele provenance. Importantly, visualizations demonstrated minimal activation in background regions, photographic edges, or plain stone surfaces, confirming that models avoided learning spurious correlations based on imaging conditions or non-cultural features.

The incorporation of a fifth output class (“background”) provides an additional validation of the method’s potential for achieving more accurate classifications. Current models may confuse stelae from different sites that share similar stone materials, leading to inflated error rates that could be corrected by explicitly incorporating material variation effects. Sites that previously exhibited lower precision scores (such as Alacahöyük in earlier results) may show substantial improvements as the model learns to distinguish stylistic features from geological characteristics. This directly addresses a core challenge in archaeological classification, where material properties can obscure cultural patterns.

By explicitly accounting for stone background effects, the model’s decisions become more archaeologically meaningful, focusing on genuine cultural and stylistic differences rather than geological coincidences.