A Benchmark Dilemma: Between Representativeness and Domain-Specificity in Visual Arts Domains
References
- Canny, J. (2009). A computational approach to edge detection. IEEE Transactions on pattern analysis and machine intelligence (6), 679–698. 10.1109/TPAMI.1986.4767851
- Carion, N., Gustafson, L., Hu, Y.-T., Debnath, S., Hu, R., Suris, D., …, Feichtenhofer, C. (2026). Sam 3: Segment anything with concepts. Retrieved from
https://arxiv.org/abs/2511.16719 . - Cetinic, E. (2021). Towards generating and evaluating iconographic image captions of artworks. Journal of Imaging, 7(8). Retrieved from
https://www.mdpi.com/2313-433X/7/8/123 - Choudhury, M. D. (2021). Automated identification of painters over wikiart image data using machine learning algorithms (Unpublished doctoral dissertation). Dublin, National College of Ireland.
- de Beyssat, C. D. (2025). Victims of Posterity. Identifying Gaps on 19th-Century French Art History with Wikidata. Journal of Open Humanities Data, 11(1). 10.5334/johd.399
- Dodge, S., & Karam, L. (2016). Understanding how image quality affects deep neural networks. In 2016 eighth international conference on quality of multimedia experience (qomex) (pp. 1–6). 10.1109/QoMEX.2016.7498955
- Fang, Y., Sun, Q., Wang, X., Huang, T., Wang, X., & Cao, Y. (2024). EVA-02: A visual representation for neon genesis. Image and Vision Computing, 149, 105171. Retrieved from
https://www.sciencedirect.com/science/article/pii/S0262885624002762 - Gadre, S. Y., Ilharco, G., Fang, A., Hayase, J., Smyrnis, G., Nguyen, T., …, Schmidt, L. (2023).
DataComp: In search of the next generation of multimodal datasets . In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, & S. Levine (Eds.), Advances in neural information processing systems vol. 36, 27092–27112. Curran Associates, Inc. Retrieved fromhttps://proceedings.neurips.cc/paper_files/paper/2023/file/56332d41d55ad7ad8024aac625881be7-Paper-Datasets_and_Benchmarks.pdf - Garcia, N., & Vogiatzis, G. (2018). How to read paintings: semantic art understanding with multi-modal retrieval. In Proceedings of the European Conference on Computer Vision (ECCV) Workshops. 10.1007/978-3-030-11012-3_52
- Hansen, L. B. P., Eriksen, R. S., Feldkamp, P., Lassche, A., Nielbo, K., Baunvig, K., & Bizzoni, Y. (2025a). Framing the Canon: A Computational Study of Canonicity in Danish Golden Age Paintings (1750–1870). Anthology of Computers and the Humanities, 3, 339–356. 10.63744/KTLpQIY247dD
- Hansen, L. B. P., Kostkan, J., Rocca, R., & Nielbo, K. L. (2025b). Multimodal Pre-Training of Vision Models Yields Better Embeddings for Visual Art. In Digital Humanities in the Nordic and Baltic Countries Publications. 10.5617/dhnbpub.12286
- Hermann, K., Chen, T., & Kornblith, S. (2020). The origins and prevalence of texture bias in convolutional neural networks. Advances in neural information processing systems, 33, 19000–19015.
- Hudson, G., Léger, A., Niss, B., Sebestyén, I., & Vaaben, J. (2018). JPEG-1 standard 25 years: past, present, and future reasons for a success. Journal of Electronic Imaging, 27(4), 040901. 10.1117/1.JEI.27.4.040901
- Khan, F. S., Beigpour, S., Van de Weijer, J., & Felsberg, M. (2014). Painting-91: a large scale database for computational painting categorization. Machine vision and applications, 25(6), 1385–1397. 10.1007/s00138-014-0621-6
- Kipke, M., Eriksen, R., Nielbo, K., & Baunvig, K. (2026). The narrative of national romanticism in 19th- Century Painting: The danish golden age and beyond. 10.5281/zenodo.19678439
- Li, Q., Xu, C., & Wang, Q. (2026). Art painting classification based on efficient convolutional neural networks. In International conference on computer vision and digital art (iccvda 2025), vol. 14117, pp. 109–114.
SPIE . 10.1117/12.3109873 - Liao, P., Li, X., Liu, X., & Keutzer, K. (2022). The ArtBench Dataset: Benchmarking Generative Models with Artworks. arXiv preprint arXiv:2206.11404.
- Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., …, others (2023). Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193.
- Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., …, Sutskever, I. (2021). Learning Transferable Visual Models From Natural Language Supervision. Retrieved from
https://arxiv.org/abs/2103.00020 - Sant, T., & Tabone, E. (2024). Art through Wikidata: A Digital Curation Perspective. Mimesis Journal, 13(2), 379–389.
- Schneider, S., Springstein, M., Stalter, J., Ritter, D., Ewerth, R., & Müller-Budack, E. (2026). Uncovering Biases Through Faceted Image Retrieval: Wikidata as a Data Provider for Art-Historical Research. Journal of Open Humanities Data, 12(1). 10.5334/johd.436
- Schroff, F., Kalenichenko, D., & Philbin, J. (2015). Facenet: A unified embedding for face recognition and clustering. In Proceedings of the ieee conference on computer vision and pattern recognition, (pp. 815–823). 10.1109/CVPR.2015.7298682
- Schuhmann, C., Vencu, R., Beaumont, R., Kaczmarczyk, R., Mullis, C., Katta, A., Coombes, T., Jitsev, J., & Komatsuzaki, A. (2021). LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs. Retrieved from
https://arxiv.org/abs/2111.02114 - Spinaci, G., Klic, L., Colavizza, G., et al. (2025). Benchmarking Multimodal Large Language Models in Zero-shot and Few-shot Scenarios: Preliminary Results on Studying Christian Iconography. In Anthology of computers and the humanities (vol. 3, pp. 1198–1210). 10.63744/oxWtm5MhhwBH
- Springstein, M., Schneider, S., Rahnama, J., Stalter, J., Kristen, M., Müller-Budack, E., & Ewerth, R. (2024). Visual narratives: Large-scale hierarchical classification of art-historical images. In Proceedings of the ieee/cvf winter conference on applications of computer vision (pp. 7220–7230). 10.1109/WACV57701.2024.00705
- Strezoski, G., & Worring, M. (2017). OmniArt: Multi-task Deep Learning for Artistic Data Analysis. Retrieved from
https://arxiv.org/abs/1708.00684 - Wasielewski, A. (2023). Computational formalism: Art history and machine learning. MIT Press. 10.7551/mitpress/14268.001.0001
- Xiao, C., Chung, I., Kerboua, I., Stirling, J., Zhang, X., Kardos, M., Solomatin, R., Al Moubayed, N., Enevoldsen, K., & Muennighoff, N. (2025). Mieb: Massive image embedding benchmark. In Proceedings of the ieee/cvf international conference on computer vision (pp. 22187–22198). 10.1109/ICCV51701.2025.02060
- Zhai, X., Mustafa, B., Kolesnikov, A., & Beyer, L. (2023). Sigmoid Loss for language image pre-training. 10.1109/ICCV51070.2023.01100
- Zhao, W., Zhou, D., Qiu, X., & Jiang, W. (2021). Compare the performance of the models in art classification. Plos one, 16(3),
e0248414 . 10.1371/journal.pone.0248414 - Zweig, B. (2024).
Hyperformalism: Notes on Machine Vision and Art Historical Method . In Critical Digital Art History, (pp. 146–166). Intellect. 10.1386/9781789389760_9
DOI: https://doi.org/10.5334/johd.623 | Journal eISSN: 2059-481X
Language: English
Page range: 133 - 133
Submitted on: Jul 27, 2026
Accepted on: Sep 8, 2026
Published on: Oct 13, 2026
Published by: Ubiquity Press
In partnership with: Paradigm Publishing Services
Keywords:
© 2026 Marta Kipke, Louise Brix Pilegaard Hansen, Rie Schmidt Eriksen, Katrine Baunvig, Kristoffer Nielbo, published by Ubiquity Press
This work is licensed under the Creative Commons Attribution 4.0 License.