Implementing State-of-the-Art Deep Learning Approaches for Archaeological Object Detection in Remotely-Sensed Data: The Results of Cross-Domain Collaboration

Martin Olivier; Wouter Verschoof-van der Vaart

doi:10.5334/jcaa.78

Abstract

The ever-increasing amount of remotely-sensed data pertaining to archaeology renders human-based analysis unfeasible, especially considering the expert knowledge required to correctly identify structures and objects in these type of data. Therefore, robust and reliable computer-based object detectors are needed, which can deal with the unique challenges of not only remotely-sensed data, but also of the archaeological detection task.

In this research – across-domain collaboration between archaeology and computer science — the latest developments in object detection and Deep Learning — for both natural and satellite imagery — are used to develop an object detection approach, based on the YOLOv4 framework, and modified to the specific task of detecting archaeology in remotely-sensed LiDAR data from theVeluwe(the Netherlands). Experiments show that a general version of the YOLOv4 architecture outperforms current object detection workflows used in archaeology, while the modified version of YOLOv4, geared towards the archaeological task, reaches even higher performance. The research shows the potential and benefit of cross-domain collaboration, where expert knowledge from different research fields is used to create a more reliable detector.

References

Arnoldussen, S. 2018. The fields that outlived the Celts: The use-histories of later prehistoric field systems (Celtic Fields or Raatakkers) in The Netherlands. Proceedings of the Prehistoric Society, 84: 303–327. DOI: 10.1017/ppr.2018.5
Open DOI Search in Google Scholar Back to article
Ball, JE, Anderson, DT and Chan, CS. 2017. Comprehensive survey of deep learning in remote sensing: Theories, tools, and challenges for the community. Journal of Applied Remote Sensing, 11(4): 042609. DOI: 10.1117/1.JRS.11.042609
Open DOI Search in Google Scholar Back to article
Bennett, R, Cowley, D and De Laet, V. 2014. The data explosion: Tackling the taboo of automatic feature recognition in airborne survey data. Antiquity, 88(341): 896–905. DOI: 10.1017/S0003598X00050766
Open DOI Search in Google Scholar Back to article
Bevan, A. 2015. The data deluge. Antiquity, 89(348): 1473–1484. DOI: 10.15184/aqy.2015.102
Open DOI Search in Google Scholar Back to article
Bochkovskiy, A, Wang, C-Y and Liao, H-YM. 2020. YOLOv4: Optimal Speed and Accuracy of Object Detection, 28 April 2020. Available at https://arxiv.org/abs/2004.10934 [Last accessed 2 November 2021].
Search in Google Scholar Back to article
Bonhage, A, Raab, A, Eltaher, M, Raab, T, Breuß, M and Schneider, A. 2021. A modified Mask region-based convolutional neural network approach for the automated detection of archaeological sites on high-resolution light detection and ranging-derived digital elevation models in the North German Lowland. Archaeological Prospection, 1–10. DOI: 10.1002/arp.1806
Open DOI Search in Google Scholar Back to article
Bourgeois, QPJ. 2013. Monuments on the Horizon. The Formation of the Barrow Landscape throughout the 3rd and 2nd Millennium BC. Leiden: Sidestone Press.
Search in Google Scholar Back to article
Bundzel, M, Jǎšcur, M, Kováč, M, Lieskovskỳ, T, Siňćak, P and Tkáčik, T. 2020. Semantic Segmentation of Airborne LiDAR Data in Maya Archaeology. Remote Sensing, 12(22): 3685. DOI: 10.3390/rs12223685
Open DOI Search in Google Scholar Back to article
Casana, J. 2020. Global-Scale Archaeological Prospection using CORONA Satellite Imagery: Automated, Crowd-Sourced, and Expert-led Approaches. Journal of Field Archaeology, 45: 89–100. DOI: 10.1080/00934690.2020.1713285
Open DOI Search in Google Scholar Back to article
Cowley, D. 2012. In with the new, out with the old? Auto-extraction for remote sensing archaeology. Proceedings of SPIE, 8532: 853206. DOI: 10.1117/12.981758
Open DOI Search in Google Scholar Back to article
Cowley, D, Verhoeven, GJ and Traviglia, A. 2021. Editorial for Special Issue: Archaeological Remote Sensing in the 21st Century: (Re)Defining Practice and Theory. Remote Sensing, 13(8): 1431. DOI: 10.3390/rs13081431
Open DOI Search in Google Scholar Back to article
Deforce, K, Groenewoudt, B and Haneca, K. 2020. 2500 years of charcoal production in the Low Countries: The chronology and typology of charcoal kilns and their relation with early iron production. Quaternary International. DOI: 10.1016/j.quaint.2020.10.020
Open DOI Search in Google Scholar Back to article
Everingham, M, Van Gool, L, Williams, CKI, Winn, J and Zisserman, A. 2010. The PASCAL Visual Object Classes (VOC) Challenge. Internattional Journal of Computer Vision, 88: 303–338. DOI: 10.1007/s11263-009-0275-4
Open DOI Search in Google Scholar Back to article
Gao, L, He, Y, Sun, X, Jia, X and Zhang, B. 2019. Incorporating negative sample training for ship detection based on deep learning. Sensors, 19(3): 684. DOI: 10.3390/s19030684
Open DOI Search in Google Scholar Back to article
Gattiglia, G. 2015. Think big about data: Archaeology and the Big Data challenge. Archëaologische Informationen, 38(1): 113–124. DOI: 10.11588/ai.2015.1.26155
Open DOI Search in Google Scholar Back to article
GDAL/OGR contributors. 2021. GDAL/OGR Geospatial Data Abstraction software Library. Available at https://gdal.org. [Last accessed 2 November 2021].
Search in Google Scholar Back to article
Ghiasi, G, Lin, TY and Le, QV. 2018. Dropblock: A regularization method for convolutional networks. Advances in Neural Information Processing Systems, 12: 10727–10737.
Search in Google Scholar Back to article
Gillings, M, Hacıigüzeller, P and Lock, G. 2020. Archaeology and spatial analysis. In: Gillings, M, Hacıigüzeller, P and Lock, G (eds.), Archaeological Spatial Analysis: A Methodological Guide, 1–16. Oxon/New York: Routledge. DOI: 10.4324/9781351243858-1
Open DOI Search in Google Scholar Back to article
Gong, M. 2021. A Novel Performance Measure for Machine Learning Classification. International Journal of Managing Information Technology, 13(1): 1–19. DOI: 10.5121/ijmit.2021.13101
Open DOI Search in Google Scholar Back to article
Goodfellow, I, Bengio, Y and Courville, A. 2016. Deep Learning. Cambridge, MA: The MIT Press.
Search in Google Scholar Back to article
Guo, Y, Liu, Y, Oerlemans, A, Lao, S, Wu, S and Lew, MS. 2016. Deep learning for visual understanding: A review. Neurocomputing, 187: 27–48. DOI: 10.1016/j.neucom.2015.09.116
Open DOI Search in Google Scholar Back to article
He, K, Gkioxari, G, Dollár, P and Girshick, R. 2018. Mask R-CNN, 13 April 2017. Available at http://arxiv.org/abs/1703.06870 [Last accessed 2 November 2021].
Search in Google Scholar Back to article
Hesse, R. 2010. LiDAR-derived Local Relief Models – a new tool for archaeological prospection. Archaeological Prospection, 17(2): 67–72. DOI: 10.1002/arp.374
Open DOI Search in Google Scholar Back to article
Hirsch, F, Schneider, A, Bonhage, A, Raab, A, Drohan, PJ and Raab, T. 2020. An initiative for a morphologic-genetic catalog of relict charcoal hearths from Central Europe. Geoarchaeology, 35(6): 1974–1983. DOI: 10.1002/gea.21799
Open DOI Search in Google Scholar Back to article
Huggett, J. 2020. Is Big Digital Data Different? Towards a New Archaeological Paradigm. Journal of Field Archaeology, 45: S17. DOI: 10.1080/00934690.2020.1713281
Open DOI Search in Google Scholar Back to article
Ju, M, Luo, J, Zhang, P, He, M and Luo, H. 2019. A Simple and Efficient Network for Small Target Detection. IEEE Access, 7: 85771–85781. DOI: 10.1109/ACCESS.2019.2924960
Open DOI Search in Google Scholar Back to article
Kazimi, B, Thiemann, F and Sester, M. 2019. Semantic Segmentation of Manmade Landscape Structures in Digital Terrain Models. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 4(2): 87–94. DOI: 10.5194/isprs-annals-IV-2-W7-87-2019
Open DOI Search in Google Scholar Back to article
Kokalj, Ž and Hesse, R. 2017. Airborne Laser Scanning Raster Data Visualization: A Guide to Good Practice. Ljubljana: Založba ZRC. DOI: 10.3986/9789612549848
Open DOI Search in Google Scholar Back to article
Krizhevsky, A, Sutskever, I and Hinton, GE. 2012. ImageNet classification with deep convolutional neural networks. Advances In Neural Information Processing Systems, 25: 1106–1114. DOI: 10.1016/j.protcy.2014.09.007
Open DOI Search in Google Scholar Back to article
Kumar, R and Manash, EB. 2019. Deep learning: A branch of machine learning. Journal of Physics: Conference Series, 1228: 12045. DOI: 10.1088/1742-6596/1228/1/012045
Open DOI Search in Google Scholar Back to article
Lambers, K, Verschoof-van der Vaart, WB and Bourgeois, QPJ. 2019. Integrating remote sensing, machine learning, and citizen science in Dutch archaeological prospection. Remote Sensing, 11(7): 794. DOI: 10.3390/rs11070794
Open DOI Search in Google Scholar Back to article
LeCun, Y, Bengio, Y and Hinton, G. 2015. Deep learning. Nature, 521: 436–444. DOI: 10.1038/nature14539
Open DOI Search in Google Scholar Back to article
Liashchynskyi, P and Liashchynskyi, P. 2019. Grid Search, Random Search, Genetic Algorithm: A Big Comparison for NAS, 12 December 2019. Available at http://arxiv.org/abs/1912.06059 [Last accessed 2 November 2021].
Search in Google Scholar Back to article
Lin, T-Y, Maire, M, Belongie, S, Hays, J, Perona, P, Ramanan, D, Dollar, P and Zitnick, C. 2014. Microsoft COCO: common objects in context. Lecture Notes in Computer Science, 8693: 740–755. DOI: 10.1007/978-3-319-10602-1_48
Open DOI Search in Google Scholar Back to article
Liu, L, Pan, Z and Lei, B. 2017. Learning a Rotation Invariant Detector with Rotatable Bounding Box, 26 November 2017. Available at http://arxiv.org/abs/1711.09405 [Last accessed 2 November 2021].
Search in Google Scholar Back to article
McCoy, MD. 2017. Geospatial Big Data and archaeology: Prospects and problems too great to ignore. Journal of Archaeological Science, 84: 74–94. DOI: 10.1016/j.jas.2017.06.003
Open DOI Search in Google Scholar Back to article
Nationaal Georegister. 2021. Publieke Dienstverlening Op de Kaart (PDOK). Available at https://www.pdok.nl/ [Last accessed 2 November 2021].
Search in Google Scholar Back to article
Opitz, R and Herrmann, J. 2018. Recent trends and long-standing problems in archaeological remote sensing. Journal of Computer Applications in Archaeology, 1(1): 19–41. DOI: 10.5334/jcaa.11
Open DOI Search in Google Scholar Back to article
Perrault, R, Shoham, Y, Brynjolfsson, E, Clark, J, Etchemendy, J, Grosz, B, Lyons, T, Manyika, J, Mishra, S and Niebles, JC. 2019. The AI Index 2019 Annual Report. Standford, CA: Human-Centered AI Institute, Stanford University.
Search in Google Scholar Back to article
QGIS Development Team. 2017. QGIS Geographic Information System. Available at http://qgis.org [Last accessed 2 November 2021].
Search in Google Scholar Back to article
Qian, X, Lin, S, Cheng, G, Yao, X, Ren, H and Wang, W. 2020. Object Detection in Remote Sensing Images Based on Improved Bounding Box Re- gression and Multi-Level Features Fusion. Remote Sensing, 12(1): 143. DOI: 10.3390/rs12010143
Open DOI Search in Google Scholar Back to article
Raab, T, Hirsch, F, Ouimet, W, Johnson, KM, Dethier, D and Raab, A. 2017. Architecture of relict charcoal hearths in northwestern Connecticut, USA. Geoarchaeology, 32(4): 502–510. DOI: 10.1002/gea.21614
Open DOI Search in Google Scholar Back to article
Razavian, AS, Azizpour, H, Sullivan, J and Carlsson, S. 2014. CNN features off-the-shelf: An astounding baseline for recognition. IEEE CVPRWorkshop, 806–813. DOI: 10.1109/CVPRW.2014.131
Open DOI Search in Google Scholar Back to article
Redmon, J, Divvala, SK, Girshick, RB and Farhadi, A. 2015. You Only Look Once: Unified, Real-Time Object Detection, 9 May 2016. Available at http://arxiv.org/abs/1506.02640 [Last accessed 2 November 2021]. DOI: 10.1109/CVPR.2016.91
Open DOI Search in Google Scholar Back to article
Redmon, J and Farhadi, A. 2016. YOLO9000: Better, Faster, Stronger, 25 December 2016. Available at http://arxiv.org/abs/1612.08242 [Last accessed 2 November 2021]. DOI: 10.1109/CVPR.2017.690
Open DOI Search in Google Scholar Back to article
Redmon, J and Farad, A. 2018. YOLOv3: An Incremental Improvement, 8 April 2018. Available at http://arxiv.org/abs/1804.02767 [Last accessed 2 November 2021].
Search in Google Scholar Back to article
Ren, S, He, K, Girshick, R and Sun, J. 2017. Faster R-CNN: Towards real- time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6): 1137–1149. DOI: 10.1109/TPAMI.2016.2577031
Open DOI Search in Google Scholar Back to article
Rezatofighi, SH, Tsoi, N, Gwak, J, Sadeghian, A, Reid, ID and Savarese, S. 2019. Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression, 15 April 2019. Available at http://arxiv.org/abs/1902.09630 [Last accessed 2 November 2021]. DOI: 10.1109/CVPR.2019.00075
Open DOI Search in Google Scholar Back to article
Sammut, C and Webb, GI. 2010. Encyclopaedia of Machine Learning. Boston, MA: Springer. DOI: 10.1007/978-0-387-30164-8
Open DOI Search in Google Scholar Back to article
Soroush, M, Mehrtash, A, Khazraee, E and Ur, JA. 2020. Deep Learning in Archaeological Remote Sensing: Automated Qanat Detection in the Kurdistan Region of Iraq. Remote Sensing, 12(3): 500. DOI: 10.3390/rs12030500
Open DOI Search in Google Scholar Back to article
Srivastava, N, Hinton, G, Krizhevsky, A and Salakhutdinov, R. 2014. Dropout: A Simple Way to Prevent Neural Networks from Overfitting. JMLR, 15(1): 1929–1958.
Search in Google Scholar Back to article
Sumbul, G, Charfuelan, M, Demir, B and Markl, V. 2019. Bigearthnet: A Large-Scale Benchmark Archive for Remote Sensing Image Understanding. IGARSS 2019–2019 IEEE International Geoscience and Remote Sensing Symposium, 5901–5904. Yokohama: IEEE. DOI: 10.1109/IGARSS.2019.8900532
Open DOI Search in Google Scholar Back to article
Trier, ØD, Cowley, D and Waldeland, AU. 2019. Using deep neural networks on airborne laser scanning data: Results from a case study of semi-automatic mapping of archaeological topography on Arran, Scotland. Archaeological Prospection, 26(2): 165–175. DOI: 10.1002/arp.1731
Open DOI Search in Google Scholar Back to article
Trier, ØD, Reksten, JH and Løseth, K. 2021. Automated mapping of cultural heritage in Norway from airborne lidar data using faster R-CNN. International Journal of Applied Earth Observation and Geoinformation, 95: 102241. DOI: 10.1016/j.jag.2020.102241
Open DOI Search in Google Scholar Back to article
Trier, ØD, Salberg, A-B and Pilø, LH. 2018. Semi automatic mapping of charcoal kilns from airborne laser scanning data using deep learning. In: Matsumoto, M and Uleberg, E (eds.). CAA 2016: Oceans of Data. Proceedings of the 44th Conference on Computer Applications and Quantitative Methods in Archaeology, 219–231. Oxford: Archaeopress.
Search in Google Scholar Back to article
Van der Schriek, M and Beex, W. 2017. The application of LiDAR-based DEMs on WWII conflict sites in the Netherlands. Journal of Conflict Archaeology, 12(2): 94–114. DOI: 10.1080/15740773.2017.1440960
Open DOI Search in Google Scholar Back to article
Van Etten, A. 2018. You Only Look Twice: Rapid Multi-Scale Object Detection In Satellite Imagery, 24 May 2018. Available at https://arxiv.org/pdf/1805.09512.pdf [Last accessed 2 November 2021].
Search in Google Scholar Back to article
Van Rossum, G and Drake, F. 2009. Python 3 Reference Manual. Scotts Valley, CA: CreateSpace.
Search in Google Scholar Back to article
Verhoeven, GJ. 2017. Are we there yet? A review and assessment of archaeological passive airborne optical imaging approaches in the light of landscape archaeology. Geosciences, 7(3): 86. DOI: 10.3390/geosciences7030086
Open DOI Search in Google Scholar Back to article
Verschoof-van der Vaart, WB. In press. Learning to Look at LiDAR. Combining CNN-based object detection and GIS for archaeological prospection in remotely-sensed data. Unpublished thesis (PhD), Leiden University.
Search in Google Scholar Back to article
Verschoof-van der Vaart, WB and Lambers, K. 2019. Learning to look at LiDAR: The use of R-CNN in the automated detection of archaeological objects in LiDAR data from the Netherlands. Journal of Computer Applications in Archaeology, 2(1): 31–40. DOI: 10.5334/jcaa.32
Open DOI Search in Google Scholar Back to article
Verschoof-van der Vaart, WB, Lambers, K, Kowalczyk, W and Bourgeois, QPJ. 2020. Combining Deep Learning and Location-Based Ranking for Large-Scale Archaeological Prospection of LiDAR Data from The Netherlands. ISPRS International Journal of Geo-Information, 9(5): 293. DOI: 10.3390/ijgi9050293
Open DOI Search in Google Scholar Back to article
Verschoof-van der Vaart, WB and Landauer, J. 2021. Using CarcassonNet to automatically detect and trace hollow roads in LiDAR data from the Netherlands. Journal of Cultural Heritage, 47: 143–154. DOI: 10.1016/j.culher.2020.10.009
Open DOI Search in Google Scholar Back to article
Vletter, WF and van Lanen, RJ. 2018. Finding vanished routes: Applying a multi-modelling approach on lost route and path networks in the Veluwe Region, The Netherlands. Rural Landscapes: Society, Environment, History, 5(2): 1–19. DOI: 10.16993/rl.35
Open DOI Search in Google Scholar Back to article
Yun, S, Han, D, Oh, SJ, Chun, S, Choe, J and Yoo, Y. 2019. CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features, 28 May 2019. Available at http://arxiv.org/abs/1905.04899 [Last accessed 2 November 2021]. DOI: 10.1109/ICCV.2019.00612
Open DOI Search in Google Scholar Back to article
Zheng, Z, Wang, P, Liu, W, Li, J, Ye, R and Ren, D. 2019. Distance-IoU Loss: Faster and Better Learning for Bounding Box Regression, 19 November 2019. Available at http://arxiv.org/abs/1911.08287 [Last accessed 2 November 2021].
Search in Google Scholar Back to article