Research on the Gaze Direction of Head-Eye Data Fusion

Xu, Xin; Wang, Changyuan

doi:10.21307/ijanmc-2021-031

Abstract

The line of sight refers to the gaze direction of the human eye and reflects the focus of human attention. Head movement is an important accompanying behavior in the process of human gaze, and it is of great significance to human visual attention. This paper intends to combine the gaze focus calculation modeling of head movement and eye movement combined with deep learning data fusion. By combining the gaze direction calculation model of data fusion and neural network algorithm, deep learning technology is used to reveal the relationship between head movement and eye movement, and the data of head movement and eye movement are merged to realize accurate and fast real-time gaze spatial direction calculation. New ideas for improving the efficiency, reliability, usability and functionality of the gaze tracking system. In this paper, the convolutional neural network method is used, and the classification accuracy of the line of sight direction reaches 99% when the head posture is free.

References

Fuhl W, Santini T, Kasneci G. Pupilnet: Convolutional neural networks for robust pupil detection[J]. arXiv preprint arXiv:1601.04902, 2016.
Search in Google Scholar Back to article
Eivazi S, Santini T, Keshavarzi A. Improving real-time CNN-based pupil detection through domain-specific data augmentation[C]. Proceedings of the 11th ACM Symposium on Eye Tracking Research & Applications, 2019: 1–6.
Search in Google Scholar Back to article
Fuhl W, Geisler D, Rosenstiel W. The applicability of Cycle GANs for pupil and eyelid segmentation, data generation and image refinement[C]. Proceedings of the IEEE International Conference on Computer Vision Workshops, 2019: 0–0.
Search in Google Scholar Back to article
Zhu J-Y, Park T, Isola P. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]. Proceedings of the IEEE international conference on computer vision, 2017: 2223–2232.
Search in Google Scholar Back to article
Tsironi E, Barros P, Weber C. An analysis of convolutional long short-term memory recurrent neural networks for gesture recognition[J]. Neurocomputing, 2017, 268: 76–86.
Search in Google Scholar Back to article
Liu Z, Li J, Shen Z. Learning efficient convolutional networks through network slimming[C]. Proceedings of the IEEE International Conference on Computer Vision, 2017: 2736–2744.
Search in Google Scholar Back to article
Bao J, Ye M. Head pose estimation based on robust convolutional neural network[J]. Cybernetics and Information Technologies, 2016, 16(6): 133–145.
Search in Google Scholar Back to article
Patacchiola M, Cangelosi A. Head pose estimation in the wild using convolutional neural networks and adaptive gradient methods[J]. Pattern Recognition, 2017, 71: 132–143.
Search in Google Scholar Back to article
Santini T, Niehorster D C, Kasneci E. Get a grip: slippage-robust and glint-free gaze estimation for real-time pervasive head-mounted eye tracking[C]. Proceedings of the 11th ACM symposium on eye tracking research & applications, 2019: 1–10.
Search in Google Scholar Back to article
Su Haiming, Hou Zhenjie, Liang Jiuzhen. A gaze tracking method using geometric features of human eyes [J]. Journal of Image and Graphics, 2019(201906): 914–923.
Search in Google Scholar Back to article
Park S, Spurr A, Hilliges O. Deep pictorial gaze estimation[C]. Proceedings of the European Conference on Computer Vision (ECCV), 2018: 721–738.
Search in Google Scholar Back to article
Zhang X, Sugano Y, Fritz M. Mpiigaze: Real-world dataset and deep appearance-based gaze estimation[J]. IEEE transactions on pattern analysis and machine intelligence, 2017, 41(1): 162–175.
Search in Google Scholar Back to article
Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition [J]. arXiv preprint arXiv:1409.1556, 2014.
Search in Google Scholar Back to article
Mase K, Watanabe Y, Suenaga Y. Real-time head motion detection system. Sensing and Reconstruction of Three-Dimensional Objects and Scenes: International Society for Optics and Photonics; 1990. p. 262–8.
Search in Google Scholar Back to article
Rolland JP, Davis LD, Baillot Y. A survey of tracking technologies for virtual environments. Fundamentals of wearable computers and augmented reality: CRC Press; 2001. p. 83–128.
Search in Google Scholar Back to article
Zhou H, Hu HJBsp, control. Human motion tracking for rehabilitation—A survey. 2008;3:1–18.
Search in Google Scholar Back to article
Al-Rahayfeh A, Faezipour MJIjoteih, medicine. Eye tracking and head movement detection: A state-of-art survey. 2013; 1:2100212.
Search in Google Scholar Back to article
Von Lüdinghausen MJCATOJotAAoCA, Anatomists tBAoC. Bilateral supernumerary rectus muscles of the orbit. 1998; 11:271–7.
Search in Google Scholar Back to article
Raudonis V, Simutis R, Narvydas G. Discrete eye tracking for medical applications. 2009 2nd International Symposium on Applied Sciences in Biomedical and Communication Technologies: IEEE; 2009. p. 1–6.
Search in Google Scholar Back to article
Botha CP, de Graaf T, Schutte S, Root R, Wielopolski P, van der Helm FC, et al. MRI-based visualisation of orbital fat deformation during eye motion. Visualization in medicine and life sciences: Springer; 2008. p. 221–33.
Search in Google Scholar Back to article
Glarin RK, Nguyen BN, Cleary JO, Kolbe SC, Ordidge RJ, Bui BV, et al. Mr-eye: high-resolution mri of the human eye and orbit at ultrahigh field (7t). 2021; 29:103–16.
Search in Google Scholar Back to article
Uhl A, Wild P. Multi-stage visible wavelength and near infrared iris segmentation framework. International Conference Image Analysis and Recognition: Springer; 2012. p. 1–10.
Search in Google Scholar Back to article
Loskutova E, Butler JS, Hernandez Martinez G, Flitcroft I, Loughman JJCER. Macular Pigment Optical Density Fluctuation as a Function of Pupillary Mydriasis: Methodological Considerations for Dual-Wavelength Autofluorescence. 2021; 46:532–8.
Search in Google Scholar Back to article
Jan F, Usman IJO. Iris segmentation for visible wavelength and near infrared eye images. 2014; 125:4274–82.
Search in Google Scholar Back to article
Chen Y, Davoine F. Simultaneous Tracking of Rigid Head Motion and Non-rigid Facial Animation by Analyzing Local Features Statistically. BMVC2006.
Search in Google Scholar Back to article

Research on the Gaze Direction of Head-Eye Data Fusion

Abstract

Paradigm

My account