Building a Database with Thermal Images for the Classification of Emotional States

Pavel, Marius Sorin; Aiordăchioaie, Dorel

doi:10.2478/bipie-2023-0013

Abstract

This research paper aims primarily to highlight the stages regarding the development of a database with facial images in the thermal spectrum, presenting various emotional states (for example, neutral, happy and sad). A series of exploratory results was presented, carried out on five databases with thermal images (DBold-2019, DBnew-2023, DBnew-2023_Crop, DBnew- 2023_Noise and DBnew-2023_ROI), belonging to the CCETIC research center (CCETIC, 2023). The database development process includes both the thermal image acquisition part and the pre-processing steps (cutting, alignment, selection of areas of interest, removal of annotations/marking elements, etc). And secondly, it aims to present an analysis regarding the general structure of the thermal image processing application, together with a series of image processing methods and techniques, addressed in certain research works. Some experimental results obtained with the help of the five databases are presented, as well as a series of general conclusions regarding the development of a database with facial thermal images.

References

Ahlberg J., Berg A., Markuš N., Multi-person fever screening using a thermal and a visual camera, Linkoping: Swedish Symposium on Image Analysis (SSBA), (2015).
Search in Google Scholar Back to article
Aiordachioaie D., Culea-Florescu A., Pavel M.S., On Human Faces Thermal Image Processing for Classification Purposes, 6th International Symposium on Electrical and Electronics Engineering (ISEEE), pp. 1-6, (2019), Galați, Romania, doi: 10.1109/ISEEE48094.2019.9136103.
Search in Google Scholar Back to article
Anbarjafari G., University of Tartu, Digital Image Processing. Introduction to image processing, Retrieved 1 21, 2023, https://sisu.ut.ee/imageprocessing/book/1.
Search in Google Scholar Back to article
Arbeláez P., Maire M., Fowlkes C., Malik J., Contour Detection and Hierarchical Image Segmentation, IEEE Transactions on Pattern Analysis and Machine Intelligence, (2010), pp. 898-916, 10.1109/TPAMI.2010.161.
Search in Google Scholar Back to article
Arnoux C. (n.d.). CA 1950 Thermal camera, https://www.arc.ro/chauvin-arnoux-ca-1950-diacam2#tab-2, Accessed April 2023.
Search in Google Scholar Back to article
Balaras C., Argiriou A. (n.d.), Infrared thermography for building diagnostics, Energy and Buildings, Volume 34, Issue 2, February 2002, pp. 171-183, https://doi.org/10.1016/S0378-7788(01)00105-0.
Search in Google Scholar Back to article
Berg A., Ahlberg J., Felsberg M., Enhanced analysis of thermographic images for monitoring of district heat pipe networks, Linköping: Submitted to Pattern Recognition Letters (PRL), (2016).
Search in Google Scholar Back to article
Byung M., Dong G.J., Soon J.B., Fabrication of Silicon Window for Low-price Thermal Imaging System, Journal of Sensor Science and Technology, Volume 24 Issue 4, pp. 264-269, (2015).
Search in Google Scholar Back to article
Baral S., Miandashti R.A., Richardson H., Near-field thermal imaging of optically excited gold nanostructures: scaling principles for collective heating with heat dissipation into the surrounding medium, Nanoscale, (2018).
Search in Google Scholar Back to article
Berg A., Detection and Tracking in Thermal Infrared Imagery, Linköping: Linköping University, (2016), Linköping studies in science and technology.
Search in Google Scholar Back to article
Berg A., Ahlberg J., Felsberg M., A thermal object tracking benchmark, Linkoping: Advanced Video and Signal Based Surveillance (AVSS), (2015a), 12th IEEE International Conference.
Search in Google Scholar Back to article
Berg A., Öfjäll K., Ahlberg J., Felsberg M., Detecting rails. Lecture Notes in Computer Science, Springer International Publishing, pp. 492-503, Copenhagen, (2015b).
Search in Google Scholar Back to article
Brouant L., Pham H., Foucher É., Picard A., VIS-NIR multispectral for camouflage detectio, Proceedings Volume 12740, Emerging Imaging and Sensing Technologies for Security and Defence VIII, 1274004, Amsterdam (2023), https://doi.org/10.1117/12.2678661.
Search in Google Scholar Back to article
Caimi F.M., Technical challenges and recent developments in underwater imaging, Society of Photo-Optical Instrumentation Engineers (SPIE), (1995), Micro-Optics/Micromechanics and Laser Scanning and Shaping.
Search in Google Scholar Back to article
CCETIC, Laboratorului de Semnale și Prelucrarea Informațiilor, Centru de cercetare în electronică, Galați: ETTI (2023), http://www.etc.ugal.ro/ccetic/eng/database2.html.
Search in Google Scholar Back to article
Chandel R., Gupta G., Image Filtering Algorithms and Techniques: A Review, International Journal of Advanced Research in Computer Science and Software Engineering, Computer Science Department, (2013).
Search in Google Scholar Back to article
Chen J., Norman J., Nam Y., (2020, 03 15), American Multiracial Face Database, (CCBy Attribution 4.0 International ) Retrieved 2023, from https://osf.io/qsdrp/.
Search in Google Scholar Back to article
Group T.S., Measurement Parameters, (2020), https://www.thesnellgroup.com/featured-tips/ir-camera-measurement-tools-and-parameters-what-do-they-mean.
Search in Google Scholar Back to article
Haider M., Doegar A., Verma K.R., Fault Identification in Electrical Equipment using Thermal Image Processing, 2018 International Conference on Computing, Power and Communication Technologies (GUCON), (2018), Greater Noida, India.
Search in Google Scholar Back to article
Hung N., Kazunori K., Fan C., Bac L., A Thermal Facial Emotion Database and Its Analysis, Ho Chi Minh: Japan Advanced Institute of Science and Technology, (2014).
Search in Google Scholar Back to article
Ilango G., Marudhachalam R., New Hybrid Filtering Techniques for Removal of Gaussian Noise from Medical Images, ARPN Journal of Engineering and Applied Sciences (2011).
Search in Google Scholar Back to article
Jeong S., Kim M.G., Oh K.Y., Thermal monitoring of live-line power transmission lines with an infrared camera mounted on an unmanned aerial vehicle, SageJournals, Structural Health Monitoring, Volume 22, Issue 6, (2023), https://doi.org/10.1177/14759217231156359.
Search in Google Scholar Back to article
Jędrasiak K., Nawrat A., The Comparison of Capabilities of Low Light Camera, Thermal Imaging Camera and Depth Map Camera for Night Time Surveillance Applications, Advanced Technologies for Intelligent Systems of National Border Security, pp. 117-128, Part of the Studies in Computational Intelligence book series (SCI,volume 440), (2013).
Search in Google Scholar Back to article
Jonathan T., Xiaopeng H., Svetlana B., James L., Yanfei X., Gang C., Infrared-Transparent Visible-Opaque Fabrics for Wearable Personal Thermal Management, ACS Photonics, American Chemical Society, pp. 769-778, (2015), https://doi.org/10.1021/acsphotonics.5b00140.
Search in Google Scholar Back to article
Jones B., Plassmann P., Digital infrared thermal imaging of human skin, IEEE Engineering in Medicine and Biology Magazine, pp. 41-48 (2002).
Search in Google Scholar Back to article
Karen P., Tufts Face Database Thermal Cropped. Panetta’s Vision thermal face images, (2019), http://tdface.ece.tufts.edu/.
Search in Google Scholar Back to article
Kölzer J., Oesterschulze E., Deboy G., Thermal imaging and measurement techniques for electronic materials and devices, Microelectronic Engineering, Volume 31, Issues 1–4, pp 251-270, February 1996.
Search in Google Scholar Back to article
Kim H.S., Lim C.S., Kim Y.D., Intelligent intrusion detection system featuring a virtual fence, active intruder detection, classification, tracking, and action recognition, (2017), Annals of Nuclear Energy Volume 112, February 2018, pp. 845-855.
Search in Google Scholar Back to article
Liu Z., Wang S., Wu G., Peng P., A Natural Visible and Infrared Facial Expression Database for Expression Recognition and Emotion Inference, (2010), IEEE Transactiond on Multimedia, pp. 682-691.
Search in Google Scholar Back to article
Marius P., Codreanu N.D., Termoviziune şi termografie în industrie, (2019), https://www.electronica-azi.ro/print.php?id=1247.
Search in Google Scholar Back to article
Moldovanu S., Pavel M.S., Aiordachioaie D., First Order Features Extraction from Thermal Images for Human State Recognition, The 8th International Symposium on Electrical and Electronics Engineering (ISEEE), (2023), Galați.
Search in Google Scholar Back to article
Nguyen H., Kazunori K., Fan C., Bac L., A Thermal Facial Emotion Database and its Analysis, (2013), Pacific-Rim Symposium on image and video Technology, pp. 397-408, Guanajuato.
Search in Google Scholar Back to article
Paugam R., Wooster M., Roberts G., Use of handheld thermal imager data, Geoscience and Remote Sensing, IEEE Transactions on, (2013), 51(6):3385-3399, IEEE.
Search in Google Scholar Back to article
Pavel M.S., Aiordachioaie D., On Change Detection in the State of the Human Faces by Thermal Images and Renyi Information Measures, 2021 International Symposium on Signals, Circuits and Systems (ISSCS), (2021), Iași, Romania, DOI: 10.1109/ISSCS52333.2021.9497373.
Search in Google Scholar Back to article
Pavel M.S., Moldovanu S., Aiordachioaie D., On the use of the Ensemble Learning Method for the Classification of the Human Emotions, (2024), the 28th International Conference on System Theory, Control and Computing (ICSTCC 2024), Sinaia.
Search in Google Scholar Back to article
Pavel M.S., Moldovanu S., Aiordachioaie D., Emotion Recognition in Human Thermal Images with Artificial Intelligence Technology, (2023), IEEE International Conference on Emerging Technologies and Factory Automation (ETFA), Sinaia.
Search in Google Scholar Back to article
Pere M., Ismael C., Teo V., Drones to the Rescue! Unmanned Aerial Search Missions Based on Thermal Imaging and Reliable Navigation, (2012), InsideGNSS.
Search in Google Scholar Back to article
Preda A., Scurtu I., Thermal image building inspection for heat loss diagnosis, (2019), 5th International Scientific Conference SEA-CONF 2019.
Search in Google Scholar Back to article
Rai M., Maity T., Yadav R.K., Thermal imaging system and its real time applications: a survey, American Society for Engineering Education, Journal of Engineering Technology (ISSN: 0747-9964), Volume 6, Issue 2, July, pp. 290-303 (2017).
Search in Google Scholar Back to article
Roselyne I., Faruk A., Khaled A., Applications of Thermal Imaging in Agriculture - A Review, Scientific Research Publishing Inc. Johannesburg (2014).
Search in Google Scholar Back to article
Saini S., Arora K., A Study Analysis on the Different Image Segmentation Techniques, International Journal of Information & Computation Technology, ISSN 0974-2239, Volume 4, Number 14, pp. 1445-1452 (2014).
Search in Google Scholar Back to article
Sebastian B., Human detection in thermal images using low-level features, https://www.researchgate.net/publication/282328996_Human_detection_in_thermal_images_using_low-level_features, (2015), Gliwice.
Search in Google Scholar Back to article
Shima T., Sugimoto S., Okutomi M., Comparison of image alignment on hexagonal and square lattices, 2010 IEEE International Conference on Image Processing, (2010), 10.1109/ICIP.2010.5654351.
Search in Google Scholar Back to article
Siegel R., Land mine detection, Instrumentation Measurement Magazine, IEEE, (2002), 5(4):22-28.
Search in Google Scholar Back to article
Sikora T., MPEG Digital Video-Coding Standards, (1997), IEEE SP Magazine, pp. 82-100.
Search in Google Scholar Back to article
Suryani E., Asmari E.I., Harjito B., Image Segmentation of Acute Myeloid Leukemia Using Multi Otsu Thresholding, Journal of Physics: Conference Series, Volume 1803, 2nd International Conference on Enhanced research and Industrial Application 2020, 8-9 October 2020, Yogyakarta, Indonesia.
Search in Google Scholar Back to article
Tian D.P., A Review on Image Feature Extraction and Representation Techniques, (2013), International Journal of Multimedia and Ubiquitous Engineering.
Search in Google Scholar Back to article
Virginia E.D., Marcos F.Z., Jiří M., Carl Database, (2013), Signal Processing Laboratory.
Search in Google Scholar Back to article
Vollmer M., Infrared Thermal Imaging, (2021), Computer Vision pp. 666-670.
Search in Google Scholar Back to article
Wang S., Liu Z., Lv S., Lv Y., Wu G., Peng P., A Natural Visible and Infrared Facial Expression Database for Expression Recognition and Emotion Inference, IEEE transactiond on Multimedia, (2010), pp. 682-691.
Search in Google Scholar Back to article
Watson A., Image Compression Using the Discrete Cosine Transform, Math. Journal, (1994), 4(1).
Search in Google Scholar Back to article
Wolberg G., Geometric Transformation Techniques for Digital Images: A Survey, Columbia University Computer Science Technical Reports, Department of Computer Science, (2011), Columbia University, https://doi.org/10.7916/D8TH8VRW.
Search in Google Scholar Back to article
Zhang Y., Wang Z., Fu X., An experimental method for improving temperature measurement accuracy of infrared thermal imager, Infrared Physics & Technology, Volume 102, November 2019, 103020, https://doi.org/10.1016/j.infrared.2019.103020.
Search in Google Scholar Back to article

Building a Database with Thermal Images for the Classification of Emotional States

Abstract

Paradigm

My account