A Benchmark Dilemma: Between Representativeness and Domain-Specificity in Visual Arts Domains

NAME	DATASET	TASK	METADATA FIELDS
D1	Cleaned WikiArt	WikiArt label classifications	image, artist, genre, style, index old, genre str, style str, artist str, idx after filter
D2	5 artists in 11 variants	WikiArt sub tasks	qid, title, image url raw, image filename, inception, collection, collection qid, location, location qid, material, genre, height cm, width cm, depicts, inventory number, Wikidata url, artist, image, license short name, license url, artist credit, usage terms
D3	2x 5 artists in 11 variants	WikiArt/Data comparison	filename, image, eva02 backbone embedding, qid, title, image url, image filename, date, collection, location, inventory number, wikidata url, artist, artist-nationality, image rights, dinov2-giant projection-head-embedding, dinov2-giant backbone embedding, eva02 projection-head-embedding, siglip so400m projection-head-embedding, siglip so400m backbone embedding, class
D4	22 artists à 35x16–20 segments	Projection head finetuning	filename, image bytes, qid, title, image url raw, inception, collection, collection qid, location, location qid, material, genre, height cm, width cm, depicts, inventory number, wikidata url, artist, license short name, license url, usage terms, copyright status, credit
D5	200 images, 1035 tree segments	Object detection	image, artist, main class, id, filename, + ablation scores from a classification task

Comparison of class sizes between D2 (WikiArt) and D3 (Wikidata).

Table 2

WikiArt classificatoin: Macro-averaged precision (P), recall (R), and F1 (F1) for genre, style, and artist classification across evaluated models. Based on WikiArt and its labeling.

	GENRE			STYLE			ARTIST
MODEL	P	R	F1	P	R	F1	P	R	F1
CLIP-ViT-B-16-DataComp.XL-s13B-b90K	0.75	0.84	0.78	0.59	0.73	0.63	0.57	0.60	0.56
CLIP-ViT-H-14-laion2B-s32B-b79K	0.78	0.86	0.81	0.70	0.80	0.74	0.73	0.75	0.72
CLIP-ViT-L-14-DataComp.XL-s13B-b90K	0.77	0.86	0.81	0.66	0.76	0.70	0.67	0.70	0.67
CLIP-ViT-L-14-laion2B-s32B-b82K	0.78	0.86	0.81	0.69	0.79	0.72	0.69	0.72	0.69
CLIP-ViT-bigG-14-laion2B-39B-b160k	0.81	0.88	0.84	0.75	0.80	0.77	0.81	0.84	0.82
CLIP-ViT-g-14-laion2B-s34B-b88K	0.79	0.87	0.82	0.71	0.79	0.74	0.76	0.79	0.77
clip-vit-large-patch14	0.78	0.86	0.81	0.68	0.78	0.71	0.68	0.71	0.68
dinov2-base	0.73	0.81	0.76	0.60	0.69	0.63	0.59	0.62	0.59
dinov2-large	0.76	0.83	0.79	0.62	0.69	0.65	0.59	0.62	0.59
dinov2-giant	0.77	0.83	0.79	0.65	0.73	0.68	0.63	0.66	0.63
eva02_clip_336	0.80	0.87	0.83	0.72	0.79	0.73	0.72	0.75	0.73
siglip-base-patch16-224	0.74	0.83	0.78	0.58	0.71	0.62	0.54	0.57	0.53
siglip-base-patch16-256	0.76	0.84	0.79	0.62	0.72	0.65	0.56	0.58	0.55
siglip-base-patch16-256-multilingual	0.76	0.84	0.79	0.61	0.72	0.64	0.54	0.58	0.54
siglip-base-patch16-384	0.77	0.85	0.80	0.60	0.72	0.64	0.58	0.61	0.57
siglip-base-patch16-512	0.77	0.85	0.80	0.63	0.74	0.66	0.61	0.62	0.59
siglip-large-patch16-256	0.78	0.85	0.81	0.67	0.76	0.70	0.65	0.68	0.65
siglip-large-patch16-384	0.79	0.87	0.82	0.69	0.77	0.72	0.70	0.72	0.69
siglip-so400m-patch14-384	0.81	0.88	0.84	0.73	0.82	0.75	0.77	0.79	0.77

Table 3

WikiArt Classification: Hyperparameter settings.

HYPERPARAMETER	GENRE	STYLE	ARTIST
Hidden layer size	500	1200	2000
Activation	ReLU	ReLU	ReLU
Dropout	0.40	0.30	0.30
Optimizer	Adam	Adam	Adam
Learning rate	1 × 10^–4	3 × 10^–4	1 × 10^–3
Weight decay	0.01	0.01	0.01
LR scheduler	ExponentialLR	ExponentialLR	ExponentialLR
Scheduler decay (γ)	0.9	0.9	0.9
Loss	Weighted Cross-Entropy	Weighted Cross-Entropy	Weighted Cross-Entropy
Early stopping patience	5	5	5

Confusion matrix of the genre classification by WikiArt labels.

Table 4

WikiArt classification: Religious vs. genre paintings performance.

MODEL	PRECISION	RECALL	F1
CLIP-ViT-B-16-DataComp.XL-s13B-b90K	0.913 (0.005)	0.928 (0.003)	0.919 (0.004)
CLIP-ViT-L-14-DataComp.XL-s13B-b90K	0.926 (0.013)	0.942 (0.007)	0.932 (0.011)
CLIP-ViT-bigG-14-laion2B-39B-b160k	0.950 (0.016)	0.957 (0.007)	0.953 (0.013)
dinov2-base	0.892 (0.009)	0.910 (0.005)	0.898 (0.007)
dinov2-giant	0.904 (0.009)	0.923 (0.007)	0.911 (0.009)
eva02_clip_336	0.948 (0.006)	0.956 (0.004)	0.952 (0.005)
siglip-base-patch16-224	0.908 (0.008)	0.924 (0.003)	0.914 (0.005)
siglip-large-patch16-384	0.936 (0.003)	0.948 (0.003)	0.941 (0.002)
siglip-so400m-patch14-384	0.952 (0.011)	0.962 (0.004)	0.956 (0.008)

Top misclassifications in a task of genre against religious painting classification.

Different image modifications to account for varying types of image quality in art historical data.

The same painting (Gateway in the Via Sepulcralis in Pompeii by Christen Købke, 1846) from two different datasets, left from WikiArt, right from the museum website. This image demonstrates the difference in color grading and graininess between different image sources.

Paintings with frames from the Statens Museum for Kunst, Kopenhagen. These examples demonstrate how elaborate frames can take up a lot of space and information in comparison to the painting itself. If a museum had specific framing policies, these might reflect as unwanted biases in the data. Inventory numbers from left to right: KMS8840, KMS3991, KMS870, KMSst194.

Table 5

Image quality task: F1 scores (mean with standard deviation).

MODEL	strong_blur	grayscale	contrast	frame	jpeg_compr	no_aug
CLIP-ViT-B-16-DataComp.XL-s13B-b90K	0.786 (0.020)	0.835 (0.015)	0.624 (0.019)	0.845 (0.017)	0.800 (0.021)	0.892 (0.019)
CLIP-ViT-L-14-DataComp.XL-s13B-b90K	0.780 (0.020)	0.860 (0.026)	0.671 (0.049)	0.876 (0.014)	0.843 (0.018)	0.906 (0.017)
CLIP-ViT-bigG-14-laion2B-39B-b160k	0.902 (0.030)	0.904 (0.016)	0.818 (0.020)	0.946 (0.010)	0.929 (0.015)	0.955 (0.005)
dinov2-base	0.833 (0.006)	0.835 (0.013)	0.816 (0.016)	0.842 (0.012)	0.825 (0.013)	0.847 (0.016)
dinov2-giant	0.865 (0.013)	0.869 (0.010)	0.842 (0.010)	0.867 (0.011)	0.862 (0.013)	0.878 (0.013)
eva02_clip_336	0.858 (0.028)	0.868 (0.028)	0.781 (0.019)	0.910 (0.016)	0.879 (0.022)	0.924 (0.007)
siglip-base-patch16-224	0.729 (0.041)	0.841 (0.008)	0.567 (0.057)	0.775 (0.040)	0.697 (0.057)	0.887 (0.006)
siglip-large-patch16-384	0.847 (0.024)	0.904 (0.011)	0.643 (0.045)	0.830 (0.055)	0.825 (0.030)	0.938 (0.012)
siglip-so400m-patch14-384	0.846 (0.024)	0.917 (0.013)	0.755 (0.032)	0.915 (0.017)	0.880 (0.012)	0.958 (0.013)

Table 6

Image quality task : F1 scores (mean with standard deviation).

MODEL	vignette	weak_grain	light_artifact	canny_sketch	pencil_sketch	no_aug
CLIP-ViT-B-16-DataComp.XL-s13B-b90K	0.872 (0.019)	0.832 (0.022)	0.885 (0.009)	0.321 (0.061)	0.267 (0.022)	0.892 (0.019)
CLIP-ViT-L-14-DataComp.XL-s13B-b90K	0.892 (0.013)	0.851 (0.016)	0.900 (0.012)	0.364 (0.056)	0.406 (0.051)	0.906 (0.017)
CLIP-ViT-bigG-14-laion2B-39B-b160k	0.944 (0.017)	0.909 (0.016)	0.938 (0.015)	0.321 (0.013)	0.507 (0.063)	0.955 (0.005)
dinov2-base	0.834 (0.014)	0.833 (0.018)	0.848 (0.014)	0.497 (0.025)	0.498 (0.034)	0.847 (0.016)
dinov2-giant	0.871 (0.010)	0.867 (0.013)	0.877 (0.014)	0.520 (0.023)	0.563 (0.036)	0.878 (0.013)
eva02_clip_336	0.910 (0.018)	0.857 (0.028)	0.916 (0.015)	0.396 (0.044)	0.479 (0.040)	0.924 (0.007)
siglip-base-patch16-224	0.840 (0.017)	0.829 (0.024)	0.866 (0.009)	0.363 (0.024)	0.347 (0.018)	0.887 (0.006)
siglip-large-patch16-384	0.880 (0.026)	0.897 (0.018)	0.926 (0.020)	0.310 (0.030)	0.413 (0.052)	0.938 (0.012)
siglip-so400m-patch14-384	0.928 (0.012)	0.896 (0.013)	0.940 (0.019)	0.346 (0.078)	0.465 (0.102)	0.958 (0.013)

Table 7

Image quality task: Difference in F1 score relative to no_aug (augmentation minus no_aug). Largest absolute drop per augmentation is underlined (ties included).

MODEL	vignette	weak_grain	light_artifact	canny_sketch	pencil_sketch
CLIP-ViT-B-16-DataComp.XL-s13B-b90K	–0.02 (0.02)	–0.06 (0.02)	–0.01 (0.01)	–0.57 (0.06)	–0.63 (0.02)
CLIP-ViT-L-14-DataComp.XL-s13B-b90K	–0.01 (0.01)	–0.06 (0.02)	–0.01 (0.01)	–0.54 (0.06)	–0.50 (0.05)
CLIP-ViT-bigG-14-laion2B-39B-b160k	–0.01 (0.02)	–0.05 (0.02)	–0.02 (0.02)	–0.63 (0.01)	–0.45 (0.06)
dinov2-base	–0.01 (0.01)	–0.01 (0.02)	0.00 (0.01)	–0.35 (0.03)	–0.35 (0.03)
dinov2-giant	–0.01 (0.01)	–0.01 (0.01)	–0.00 (0.01)	–0.36 (0.02)	–0.32 (0.04)
eva02_clip_336	–0.01 (0.02)	–0.07 (0.03)	–0.01 (0.02)	–0.53 (0.04)	–0.45 (0.04)
siglip-base-patch16-224	–0.05 (0.02)	–0.06 (0.02)	–0.02 (0.01)	–0.52 (0.02)	–0.54 (0.02)
siglip-large-patch16-384	–0.06 (0.03)	–0.04 (0.02)	–0.01 (0.02)	–0.63 (0.03)	–0.53 (0.05)
siglip-so400m-patch14-384	–0.03 (0.01)	–0.06 (0.01)	–0.02 (0.02)	–0.61 (0.08)	–0.49 (0.10)

Table 8

Image quality task: Difference in F1 score relative to no_aug (augmentation minus no_aug).

MODEL	strong_blur	grayscale	contrast	frame	jpeg_compr
CLIP-ViT-B-16-DataComp.XL-s13B-b90K	–0.11 (0.02)	–0.06 (0.02)	–0.27 (0.02)	–0.05 (0.02)	–0.09 (0.02)
CLIP-ViT-L-14-DataComp.XL-s13B-b90K	–0.13 (0.02)	–0.05 (0.03)	–0.24 (0.05)	–0.03 (0.01)	–0.06 (0.02)
CLIP-ViT-bigG-14-laion2B-39B-b160k	–0.05 (0.03)	–0.05 (0.02)	–0.14 (0.02)	–0.01 (0.01)	–0.03 (0.02)
dinov2-base	–0.01 (0.01)	–0.01 (0.01)	–0.03 (0.02)	–0.01 (0.01)	–0.02 (0.01)
dinov2-giant	–0.01 (0.01)	–0.01 (0.01)	–0.04 (0.01)	–0.01 (0.01)	–0.02 (0.01)
eva02_clip_336	–0.07 (0.03)	–0.06 (0.03)	–0.14 (0.02)	–0.01 (0.02)	–0.05 (0.02)
siglip-base-patch16-224	–0.16 (0.04)	–0.05 (0.01)	–0.32 (0.06)	–0.11 (0.04)	–0.19 (0.06)
siglip-large-patch16-384	–0.09 (0.02)	–0.03 (0.01)	–0.30 (0.05)	–0.11 (0.06)	–0.11 (0.03)
siglip-so400m-patch14-384	–0.11 (0.02)	–0.04 (0.01)	–0.20 (0.03)	–0.04 (0.02)	–0.08 (0.01)

Table 9

WikiArt and Wikidata classification: Comparison F1 scores and standard deviation.

MODEL	WIKIDATA	WIKIART	DIFFERENCE
CLIP-ViT-B-16-DataComp.XL-s13B-b90K	0.892 (0.017)	0.892 (0.019)	0.000
CLIP-ViT-L-14-DataComp.XL-s13B-b90K	0.918 (0.017)	0.906 (0.017)	0.012
CLIP-ViT-bigG-14-laion2B-39B-b160k	0.955 (0.006)	0.955 (0.005)	0.000
dinov2-base	0.878 (0.008)	0.847 (0.016)	0.031
dinov2-giant	0.888 (0.015)	0.878 (0.013)	0.010
eva02_clip_336	0.927 (0.008)	0.924 (0.007)	0.003
siglip-base-patch16-224	0.903 (0.014)	0.887 (0.006)	0.016
siglip-large-patch16-384	0.942 (0.015)	0.938 (0.012)	0.004
siglip-so400m-patch14-384	0.959 (0.017)	0.958 (0.013)	0.001

Table 10

Finetuning a projection head: Painter selection. Training classes include 35 paintings, probe 1 consists of 7 paintings and probe 2 of 15.

PAINTER	TRAINING	PROBE 1	PROBE 2
Andreas Achenbach	x
Ivan Aivazovsky	x
Knud Baade			x
Albert Bierstadt	x
Eugene Louis Boudin	x
August Cappelen			x
Carl Gustav Carus	x
F. E. Church	x	x
Thomas Cole	x
John Constable	x
J.-B. Camille Corot	x	x
C.-F. Daubigny	x	x
Joseph Wright of Derby	x
Narcisse Virgilio Díaz	x	x
R. S. Duncanson	x
Caspar David Friedrich	x	x
Carlos de Haes	x
B. C. Koekkoek	x
Isaac Levitan	x
Johan Lundbye			x
Anton Mauve	x	x
Thomas Moran	x
Théodore Rousseau	x
Ivan Shishkin	x	x
P. C. Skovgaard			x
J. M. W. Turner	x

Table 11

Finetuning a projection head: Training parameters per model. Additionally, the SigLIP-SO400M model was trained with one warm-up epoch for both runs.

MODEL	DURATION	EPOCHS	LR	PAIRS	BATCH
DINOv2-giant	55.6	20	3e-5	2000	32
SigLIP-SO400M-patch14-384 I	37.5	20	5e-5	1500	32
SigLIP-SO400M-patch14-384 II	–	20	2e-5	1500	32
DINOv2-base I	4.7	10	2e-5	1200	32
DINOv2-base II	3.4	20	2e-5	1500	32
EVA02-CLIP-L-14-336 I	22.7	20	5e-5	1500	32
SigLIP-B-patch16-224 II	4.2	20	7e-5	1500	32
EVA02-CLIP-L-14-336 II	44.3	20	2e-5	1500	32
SigLIP-L-patch16-384	39.1	20	5e-5	1500	32
SigLIP-B-patch16-224 I	2.5	10	5e-5	1200	32
CLIP-ViT-bigG-14-laion2B	71.1	30	1e-4	1500	32
CLIP-ViT-B-16-DataComp.XL	4.3	30	1e-4	1500	32
CLIP-ViT-L-14-DataComp	9.8	30	5e-5	1500	32

Table 12

Finetuning a projection head: best checkpoint selected by training R@1.

	TRAINING				TEST PERFORMANCE
MODEL	ACC	R@1	R@5	R@10	PROBE 1	DIFF.	PROBE 2	DIFF.
SigLIP-SO400M-patch14-384 I	0.96	0.90	0.97	0.98	0.72	0.18	0.63	0.27
EVA02-CLIP-L-14-336 I	0.94	0.81	0.94	0.96	0.86	–0.05	0.71	0.09
SigLIP-B-patch16-224 II	0.94	0.80	0.93	0.96	0.64	0.16	0.67	0.13
SigLIP-L-patch16-384	0.91	0.80	0.93	0.96	0.62	0.18	0.72	0.08
SigLIP-SO400M-patch14-384 II	0.91	0.78	0.93	0.96	0.71	0.07	0.83	–0.05
DINOv2-giant	0.90	0.78	0.92	0.95	0.74	0.03	0.77	0.01
DINOv2-base II	0.89	0.72	0.90	0.94	0.68	0.04	0.86	–0.14
EVA02-CLIP-L-14-336	0.89	0.69	0.88	0.93	0.62	0.07	0.67	0.02
CLIP-ViT-bigG-14-laion2B	0.86	0.64	0.89	0.94	0.40	0.23	0.75	–0.12
DINOv2-base I	0.85	0.61	0.82	0.91	0.70	–0.09	0.84	–0.23
SigLIP-B-patch16-224 I	0.85	0.50	0.80	0.88	0.45	0.05	0.67	–0.17
CLIP-ViT-B-16-DataComp.XL	0.72	0.29	0.64	0.77	0.34	–0.05	0.72	–0.43
CLIP-ViT-L-14-DataComp	0.60	0.14	0.40	0.59	0.13	0.01	0.54	–0.41

Table 13

Finetuning a projection head: best checkpoint selected by probe 1 (known probe).

	TRAINING				TEST PERFORMANCE
MODEL	ACC	R@1	R@5	R@10	PROBE 1	DIFF.	PROBE 2	DIFF.
EVA02-CLIP-L-14-336 I	0.94	0.81	0.94	0.96	0.86	–0.05	0.71	0.09
DINOv2-giant	0.90	0.77	0.90	0.93	0.79	–0.02	0.63	0.13
SigLIP-SO400M-patch14-384 II	0.89	0.76	0.90	0.93	0.78	–0.01	0.81	–0.05
DINOv2-base I	0.84	0.60	0.82	0.90	0.74	–0.14	0.86	–0.26
DINOv2-base II	0.85	0.64	0.82	0.89	0.72	–0.09	0.82	–0.19
SigLIP-SO400M-patch14-384 I	0.96	0.90	0.97	0.98	0.72	0.18	0.63	0.27
EVA02-CLIP-L-14-336	0.88	0.66	0.87	0.91	0.70	–0.04	0.74	–0.07
SigLIP-B-patch16-224 II	0.89	0.68	0.89	0.93	0.66	0.02	0.70	–0.02
SigLIP-L-patch16-384	0.92	0.78	0.92	0.96	0.64	0.14	0.75	0.03
SigLIP-B-patch16-224 I	0.82	0.47	0.76	0.85	0.60	–0.12	0.61	–0.14
CLIP-ViT-bigG-14-laion2B	0.86	0.64	0.89	0.94	0.40	0.23	0.75	–0.12
CLIP-ViT-B-16-DataComp.XL	0.72	0.29	0.64	0.77	0.34	–0.05	0.72	–0.43
CLIP-ViT-L-14-DataComp	0.61	0.14	0.41	0.59	0.13	0.01	0.54	–0.41

Table 14

Finetuning a projection head: best checkpoint selected by probe 2.

	TRAINING				TEST PERFORMANCE
MODEL	ACC	R@1	R@5	R@10	PROBE 1	DIFF.	PROBE 2	DIFF.
DINOv2-base II	0.89	0.72	0.90	0.94	0.68	0.04	0.86	–0.14
DINOv2-base I	0.84	0.60	0.82	0.90	0.74	–0.14	0.86	–0.26
SigLIP-SO400M-patch14-384 II	0.91	0.78	0.93	0.96	0.71	0.07	0.83	–0.05
DINOv2-giant	0.79	0.46	0.76	0.86	0.55	–0.10	0.82	–0.37
SigLIP-L-patch16-384	0.76	0.42	0.72	0.82	0.32	0.11	0.81	–0.38
EVA02-CLIP-L-14-336 I	0.92	0.77	0.92	0.95	0.82	–0.05	0.80	–0.03
EVA02-CLIP-L-14-336	0.87	0.60	0.83	0.90	0.68	–0.08	0.79	–0.19
CLIP-ViT-bigG-14-laion2B	0.72	0.30	0.63	0.78	0.26	0.04	0.77	–0.47
SigLIP-B-patch16-224 I	0.80	0.47	0.76	0.86	0.49	–0.02	0.75	–0.28
CLIP-ViT-B-16-DataComp.XL	0.69	0.26	0.60	0.74	0.32	–0.06	0.74	–0.48
SigLIP-SO400M-patch14-384 I	0.92	0.75	0.92	0.95	0.51	0.24	0.74	0.02
SigLIP-B-patch16-224 II	0.92	0.79	0.92	0.95	0.64	0.15	0.70	0.09
CLIP-ViT-L-14-DataComp	0.61	0.14	0.41	0.59	0.13	0.01	0.54	–0.41

Table 15

Finetuning a projection head: Standard deviation across repeats for each model (hyperparameter-variants not included).

MODEL	ACC	R@1	R@5	R@10	PROBE 1	PROBE 2
SigLIP-SO400M-patch14-384	0.0062	0.0108	0.0025	0.0035	0.0269	0.0279
DINOv2-giant	0.0111	0.0131	0.0102	0.0094	0.0223	0.0518
SigLIP-L-patch16-384 I	0.0136	0.0120	0.0119	0.0079	0.0343	0.0258
EVA02-CLIP-L-14-336 I	0.0085	0.0070	0.0051	0.0057	0.0520	0.0390
DINOv2-base II	0.0104	0.0069	0.0014	0.0041	0.0233	0.0217
SigLIP-B-patch16-224	0.0221	0.0105	0.0140	0.0115	0.0744	0.0194
CLIP-ViT-bigG-14-laion2B	0.0212	0.0302	0.0143	0.0183	0.0771	0.0992
CLIP-ViT-B-16-DataComp.XL	0.0267	0.0608	0.0563	0.0304	0.0736	0.0416
CLIP-ViT-L-14-DataComp.XL	0.0200	0.0095	0.0173	0.0129	0.0655	0.0491

Table 16

Evaluating object detection: Evaluation metrics and comparison between SAM 3 and expert annotation.

GENERAL
Images in subset		200
Both agree: 0 trees		54
Expert annotated trees		1035 boxes/140 images
SAM 03 predicted trees		1236 boxes/134 images
IOU-BASED		COUNT-BASED		DETECTION-LEVEL
METRIC	VALUE	METRIC	VALUE	METRIC	VALUE
TP @ IoU ≥ 0.5	688	Precision	0.697	TP	128
Precision	0.557	Recall	0.832	FP	6
Recall	0.665	F1 score	0.758	FN	12
F1 score	0.606			Precision	0.955
Mean IoU (matched pairs)	0.664			Recall	0.914
Median IoU	0.699			F1 score	0.934

Comparison of expert vs. SAM 3 bounding boxes on the worst performing paintings.

Table 17

Evaluating object detection: Distribution of detected trees across image augmentations.

SOURCE	MEAN	MEDIAN	STD	MAX
Expert	5.18	3.0	6.00	32
Unaltered	6.18	3.0	7.64	50
Contrast	3.56	2.0	4.95	40
Grayscale	5.64	3.0	7.11	47
Strong blur	2.52	1.0	3.70	22
Strong grain	1.60	0.0	2.42	19
Weak grain	2.85	1.0	3.76	21
JPEG compression	4.47	2.0	5.89	33
Vignette	5.90	3.0	7.21	50
Frame	5.35	2.0	6.82	43
Light artifact	6.06	3.0	7.47	47
Pencil sketch	1.44	0.0	2.65	17
Canny sketch	0.42	0.0	1.01	6

Table 18

Evaluating object detection: IoU performance under augmentations relative to the original model.

AUGMENTATION	VS. ORIGINAL DETECTION
AUGMENTATION	PREC.	REC.	F1	IOU
Unaltered	1.000	1.000	1.000	1.000
Contrast	0.722	0.416	0.528	0.731
Grayscale	0.875	0.798	0.835	0.889
Strong blur	0.822	0.336	0.477	0.768
Strong grain	0.608	0.157	0.250	0.634
Weak grain	0.726	0.335	0.458	0.715
JPEG compression	0.676	0.489	0.567	0.692
Vignette	0.911	0.871	0.890	0.950
Frame	0.898	0.778	0.833	0.865
Light artifact	0.957	0.939	0.948	0.958
Pencil sketch	0.622	0.145	0.235	0.614
Canny sketch	0.747	0.050	0.094	0.722

Table 19

Evaluating object detection: Effect of augmentations on mean tree detections relative to original model.

AUGMENTATION	MEAN (AUG)	MEAN (ORIG)	RATIO	DELTA	DEVIATION
Unaltered	6.180	6.18	1.000	0.000	0.000
Light artifact	6.065	6.18	0.981	–0.115	0.019
Vignette	5.905	6.18	0.956	–0.275	0.044
Grayscale	5.635	6.18	0.912	–0.545	0.088
Frame	5.350	6.18	0.866	–0.830	0.134
JPEG compression	4.470	6.18	0.723	–1.710	0.277
Contrast	3.560	6.18	0.576	–2.620	0.424
Weak grain	2.850	6.18	0.461	–3.330	0.539
Strong blur	2.525	6.18	0.409	–3.655	0.591
Strong grain	1.595	6.18	0.258	–4.585	0.742
Pencil sketch	1.440	6.18	0.233	–4.740	0.767
Canny sketch	0.415	6.18	0.067	–5.765	0.933

References

Authors

Metrics

Articles in this issue

DOI: https://doi.org/10.5334/johd.623 | Journal eISSN: 2059-481X

Journal RSS Feed

Language: English

Page range: 133 - 133

Submitted on: Jul 27, 2026

Accepted on: Sep 8, 2026

Published on: Oct 13, 2026

Published by: Ubiquity Press

In partnership with: Paradigm Publishing Services

Keywords:

benchmark,

visual arts,

image classification

© 2026 Marta Kipke, Louise Brix Pilegaard Hansen, Rie Schmidt Eriksen, Katrine Baunvig, Kristoffer Nielbo, published by Ubiquity Press
This work is licensed under the Creative Commons Attribution 4.0 License.

Volume 12 (2026): Issue 1

A Benchmark Dilemma: Between Representativeness and Domain-Specificity in Visual Arts Domains

Figures & Tables

Table 1

Figure 1

Table 2

Table 3

Figure 2

Table 4

Figure 3

Figure 4

Figure 5

Figure 6