Image Processing Methods for Gesture-Based Robot Control

Abdelouahab Zaatri; Hamama Aboud

doi:10.2478/auseme-2021-0008

Abstract

In this paper we discuss some image processing methods that can be used for motion recognition of human body parts such as hands or arms in order to interact with robots. This interaction is usually associated to gesture-based control. The considered image processing methods have been experienced for feature recognition in applications involving human robot interaction. They are namely: Sequential Similarity Detection Algorithm (SSDA), an appearance-based approach that uses image databases to model objects, and Kanade-Lucas-Tomasi (KLT) algorithm which is usually used for feature tracking. We illustrate the gesture-based interaction by using KLT algorithm. We discuss the adaptation of each of these methods to the context of gesture-based robot interaction and some of their related issues.

References

[1] Zaatri, A., and Ousalah, M., “Integration and design of multi-modal interfaces for supervisory control systems”, Journal of information fusion, vol.4, pp. 135–150, 2003.10.1016/S1566-2535(03)00018-6
Search in Google Scholar Back to article
[2] Abiri, A., Heise, G., Zhao,X., Jiang,Y., and Abiri, F., “Brain Computer Interface for Gesture Control of a Social Robot: an Offline Study”, in Proc. 25th Iranian Conference on Electrical Engineering (ICEE), Tehran, Iran, May 2-4, 2017, pp. 113–117.10.1109/IranianCEE.2017.7985231
Search in Google Scholar Back to article
[3] Mitra, S., and Acharya,T., “Gesture Recognition: A Survey”, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 37, no. 3, pp-311-324, May 2007.10.1109/TSMCC.2007.893280
Search in Google Scholar Back to article
[4] Sigalas, M, Baltzakis, H., and Trahanias, P., “Gesture recognition based on arm tracking for human-robot interaction”, in Proceedings IEEE/RSJ International Conference on Intelligent Robots and Systems, October 2010, pp. 5424–5429.10.1109/IROS.2010.5648870
Search in Google Scholar Back to article
[5] Mathe, E., Mylonas, P., Spyrou, E., and Mylonas, P., “Arm Gesture Recognition using a Convolutional Neural Network”, in Proc. 13th International Workshop on Semantic and Social Media Adaptation and Personalization (SMAP), 2018, pp. 37–42.10.1109/SMAP.2018.8501886
Search in Google Scholar Back to article
[6] Luo, J., Liu,Y.,and Ju. Z., “An Interactive Astronaut-Robot System with Gesture Control”, Comput. Intell. Neurosci. vol.2016 (1), pp. 1-11, 2016.10.1155/2016/7845102
Search in Google Scholar Back to article
[7] Staub, C., Can, C., Knoll, A., Nitsch,V., Karl, I., and Färber, B., “Implementation and Evaluation of a gesture-based Input Method in Robotic Surgery”, in Proc. IEEE International Workshop on Haptic Audio Visual Environments and Games (HAVE), Qinghuandao, Hebei, China, October 2011, pp.1–7.10.1109/HAVE.2011.6088384
Search in Google Scholar Back to article
[8] Elliott, R, E., Hill,S. G., and Barnes, M., “Gesture-Based Controls for Robots: Overview and Implications for Use by Soldiers”, Human Research and Engineering Directorate, ARL, US Army Research Laboratory, ARL-TR-7715, July 2016.10.21236/AD1011904
Search in Google Scholar Back to article
[9] Kewley-Port, D., and Nearey, T. M., “Speech synthesizer produced voices for disabled, including Stephen Hawking”, The Journal of the Acoustical Society of America, vol. 148, pp. R1-R2, 2020.10.1121/10.000149032752746
Search in Google Scholar Back to article
[10] Galván-Ruiz, J., Travieso-González, C.M., Tejera-Fettmilch, A., Pinan-Roescher, A., Esteban-Hernández, L., and Domínguez-Quintana, L., “Perspective and Evolution of Gesture Recognition for Sign Language: A Review”, Sensors (Basel). vol. 20, no. 12, pp. 3571, June 2020.10.3390/s20123571734970332599793
Search in Google Scholar Back to article
[11] Zaatri, A., and Bouchemal, B., “Some interactive control modes for planar cable-driven robots”, World Journal of Engineering, vol. 10, no. 5, pp. 485–490, 2013.10.1260/1708-5284.10.5.485
Search in Google Scholar Back to article
[12] Zaatri, A., Tuytelaars, T., Waarsing, R., Van Brussel, H., and Van Gool, L., “Design and Implementation of a Supervised Intelligent Function”, in Proc. International Conference on Artificial Intelligence, IC-AI00, Las Vegas, USA, 2000, pp. 799–805.
Search in Google Scholar Back to article
[13] Aboud, H., and Zaatri, A., “Suivi de gestes pour la commande des robots”, in Proc. Congres Algerien de Mécanique (CAM-2011), Guelma, November 2011, pp. 14–17.
Search in Google Scholar Back to article
[14] Shirai, Y., “Three-Dimensional Computer Vision”, Springer.1987.10.1007/978-3-642-82429-6
Search in Google Scholar Back to article
[15] Tuytelaars, T., Zaatri, A., Van Gool, L., and Van Brussel, “Automatic Object Recognition as Part of an Integrated Supervisory Control System”, in Proc. ICRA, San Francisco, USA; 2000, pp. 3707–3712.
Search in Google Scholar Back to article
[16] Tuytelaars, T., and Mikolajczyk, K., “Local Invariant Feature Detectors: A Survey, Foundations and Trends”, Computer Graphics and Vision., vol. 3, no. 3, pp. 177–280, 2007.10.1561/0600000017
Search in Google Scholar Back to article
[17] Baker, S., and Mattews, I., “Lucas-Kanade 20 Years On: A Unifying Framework”, International Journal of Computer Vision., Kluwer Academic Publishers, vol. 56, no. 3, pp. 221–255, 2004.10.1023/B:VISI.0000011205.11775.fd
Search in Google Scholar Back to article
[18] Besl, P. J., and Jain, R. C., “Three-dimensional object recognition”, ACM Comput. Surv., (CSUR)17, pp.75–145, 1985.10.1145/4078.4081
Search in Google Scholar Back to article
[19] Lucas, B., and Kanade, T., “An iterative image registration technique with an application to stereo vision”, in Proceedings of the International Joint Conference on Artificial Intelligence, 1981, pp. 674–679.
Search in Google Scholar Back to article
[20] Tomasi, C., and Kanade, T., “Shape and Motion from Image Streams: a Factorization Method—Part 3 -Detection and Tracking of Point Features”, Technical Report CMU-CS-91–132, April 1991.
Search in Google Scholar Back to article
[21] https://cecas.clemson.edu/~stb/klt/
Search in Google Scholar Back to article
[22] http://www.java2s.com/ref/jar/download-klttracker131jar-file.html
Search in Google Scholar Back to article
[23] Fassold, H., et al., “Realtime KLT Feature Point Tracking for High Definition Video”, Proceedings, https://www.yumpu.com/en/document/view/42917243/realtime-klt-feature-point-tracking-for-high-definition-semedia, 2009.
Search in Google Scholar Back to article
[24] Barnes, B., et al., “Evaluation of Feature Detectors for KLT based Feature Tracking using the Ondroid U3”, in Proceedings of Australasian Conference on Robotics and Automation, The University of Melbourne, Melbourne, Australia, 2014, pp. 1–9.
Search in Google Scholar Back to article

Image Processing Methods for Gesture-Based Robot Control

Abstract

Paradigm

My account