Communication with the Manipulator Using Gestures to Enhance the Manipulability of Persons with Reduced Mobility
References
- H. F. M. Van der Loos and D. J. Reinkensmeyer. Springer Handbook of Robotics, chapter “Rehabilitation and Health Care Robotics”, 1223–1251. Springer Berlin Heidelberg, Berlin, Heidelberg, 2008.
- A. Nanavati, V. Ranganeni, and M. Cakmak, “Physically Assistive Robots: A Systematic Review of Mobile and Manipulator Robots that Physically Assist People with Disabilities”, Annual Review of Control, Robotics, and Autonomous Systems, vol. 7, no. Volume 7, 2024, 2024, 123–147, https://doi.org/10.1146/annurev-control-062823-024352
- R. Soyama, S. Ishii, and A. Fukase, “8 Selectable Operating Interfaces of the Meal-Assistance Device “My Spoon”. In: Advances in Rehabilitation Robotics: Human-Friendly Technologies on Movement Assistance and Restoration for People with Disabilities, Springer Berlin Heidelberg, 2004, 155–163, 10.1007/10946978_8
- A. Campeau-Lecours, H. Lamontagne, S. Latour, P. Fauteux, V. Maheu, F. Boucher, C. Deguire, and L.-J. C. L’Ecuyer, “Kinova Modular Robot Arms for Service Robotics Applications”, International Journal of Robotics Applications and Technologies, vol. 5, no. 2, 2017, 10.4018/IJRAT.2017070104
- “iARM”. Assistive Innovations, 2026. https://ww w.assistive-innovations.com/robotarm
- “BATEO”. ACCREA, 2026. https://accrea.com/products/bateo/
- S. Jain, A. Farshchiansadegh, A. Broad, F. Abdollahi, F. Mussa-Ivaldi, and B. Argall, “Assistive Robotic Manipulation through Shared Autonomy and a Body-Machine Interface”. In: 2015 IEEE International Conference on Rehabilitation Robotics (ICORR), 2015, 526–531, 10.1109/ICORR.2015.7281253
- A. Vijay Joseph, A. Mathur, J. Verma, and A. Singh, “Gesture Based Wireless Control for a Robotic Manipulator”, International Journal of Engineering & Technology, vol. 7, no. 2.31, 2018, 231–234, 10.14419/ijet.v7i2.31.13449
- P. Sihombing, R. B. Muhammad, H. Herriyance, and E. Elviwani, “Robotic Arm Controlling Based on Fingers and Hand Gesture”. In: 2020 3rd International Conference on Mechanical, Electronics, Computer, and Industrial Technology (MECnIT), 2020, 40–45, 10.1109/MECnIT48290.2020.9166592
- X. Li, “Human-Robot Interaction Based on Gesture and Movement Recognition”, Signal Processing: Image Communication, vol. 81, 2020, 115686, https://doi.org/10.1016/j.image.2019.115686
- W. Rhee, Y. G. Kim, J. H. Lee, J. W. Shim, B. S. Kim, D. Yoon, M. Cho, and S. Kim, “Unconstrained Lightweight Control Interface for Robot-Assisted Minimally Invasive Surgery Using MediaPipe Framework and Head-Mounted Display”, Virtual Reality, vol. 28, no. 114, 2024, https://doi.org/10.1007/s10055-024-00986-1
- A. R. Khan, A. Khan, and A. Faisal, “Gesture-Controlled Robotic Arm for Real-Time Surgical Tool Assistance”. In: 2025 Fourth International Conference on Power, Control and Computing Technologies (ICPC2T), 2025, 886–891, 10.1109/ICPC2T63847.2025.10958621
- J. Xie, Z. Xu, J. Zeng, Y. Gao and K. Hashimoto, “Human-Robot Interaction Using Dynamic Hand Gesture for Teleoperation of Quadruped Robots with a Robotic Arm”, Electronics, vol. 14, no. 5, 2025, 10.3390/electronics14050860
- C. Lugaresi, J. Tang, H. Nash, C. McClanahan, E. Uboweja, M. Hays, F. Zhang, C.-L. Chang, M. G. Yong, J. Lee, W.-T. Chang, W. Hua, M. Georg, and M. Grundmann. “MediaPipe: A Framework for Building Perception Pipelines”, 2019.
- F.-J. Chu, R. Xu, Z. Zhang, P. A. Vela, and M. Ghovanloo, “Hands-Free Assistive Manipulator Using Augmented Reality and Tongue Drive System”. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2018, 5463–5468, 10.1109/IR0S.2018.8594508
- P. D. Lillo, F. Arrichiello, D. D. Vito, and G. Antonelli, “BCI-Controlled Assistive Manipulator: Developed Architecture and Experimental results”, IEEE Transactions on Cognitive and Developmental Systems, vol. 13, no. 1, 2021, 91–104, 10.1109/TCDS.2020.2979375
- K. P. Hawkins, P. M. Grice, T. L. Chen, C.-H. King, and C. C. Kemp, “Assistive Mobile Manipulation for Self-Care Tasks Around the Head”. In: 2014 IEEE Symposium on Computational Intelligence in Robotic Rehabilitation and Assistive Technologies (CIR2AT), 2014, 16–25, 10.1109/CIRAT.2014.7009736
- B. Kulecki, “Multimodal Robot Programming Interface Based On RGB-D Perception and Neural Scene Understanding Modules”, Journal of Automation, Mobile Robotics and Intelligent Systems, vol. 17, no. 3, 2024, 29–37, 10.14313/JAMRIS/3-2023/20
- C.-S. Chung, K. Williams, S. M. Eckstein, I. Eckstein, R. Cooper, and R. A. Cooper, “Manipulation Performance of an Assistive Robotic Arm Control Interface for Individuals with Cerebral Palsy - A Case Study”. In: Proceedings of the RESNA Annual Conference, 2022.
- K. Arent, J. Jakubiak, M. Drwięga, M. Cholewinski, G. Stollnberger, M. Giuliani, M. Tscheligi, D. Szczęśniak-Stańczyk, M. Janowski, W. Brzozowski, and A. Wysokinski, “Control of Mobile Robot for Remote Medical Examination: Design Concepts and Users’ Feedback from Experimental Studies”. In: 2016 9th International Conference on Human System Interactions (HSI), 2016, 76–82, 10.1109/HSI.2016.7529612
- A. Lebrasseur, J. Lettre, F. Routhier, P. S. Archambault, and A. Campeau-Lecours, “Assistive Robotic Arm: Evaluation of the Performance of Intelligent Algorithms”, Assistive Technology, vol. 33, no. 2, 2021, 95–104, 10.1080/10400435.2019.1601649, PMID: 31070524.
- S. Koryl. “Technology for Human-Manipulator Communication Using Gestures”. Master’s thesis, Wroclaw University of Science and Technology, 2024.
- Google. MediaPipe Solutions Guide, 2024. https://ai.google.dev/edge/mediapipe/solutions/guide
- G. Amprimo, G. Masi, G. Pettiti, G. Olmo, L. Priano, and C. Ferraris, “Hand Tracking for Clinical Applications: Validation of the Google MediaPipe Hand (GMH) and the Depth-Enhanced GMH-D Frameworks”, Biomedical Signal Processing and Control, vol. 96, 2024, 106508, https://doi.org/10.1016/j.bspc.2024.106508
- Y. Alemu. “Robotic System Technology to Support the Manipulability of a Person with Motor Disabilities”. Master’s thesis, Wroclaw University of Science and Technology, 2024.
- G. Niemeyer, C. Preusche, S. Stramigioli, and D. Lee, “Telerobotics”. In: Springer Handbook of Robotics, 2016.
- R. A. Grier, A. Bangor, P. Kortum, and S. C. Peres, “The System Usability Scale: Beyond Standard Usability Testing”, Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 57, no. 1, 2013, 187–191, 10.1177/1541931213571042
- R. Szczepanowski, T. Niemiec, E. Cichon, K. Arent, M. Florkowski, and J. Sobecki, “Factor Analysis of the Polish Version of Godspeed Questionnaire (Gqs)”, Journal of Automation, Mobile Robotics and Intelligent Systems, vol. 16, no. 2, 2023, 30–35, 10.14313/JAMRIS/2-2022/13
- G. Pochwatko, J.-C. Giger, M. Rozanska-Walczuk, J. Swidrak, K. Kukielka, J. Mozaryn, and N. Piçarra, “Polish Version of the Negative Attitude Toward Robots Scale (NARS-PL)”, Journal of Automation, Mobile Robotics and Intelligent Systems, vol. 9, no. 3, 2015, 65–72, 10.14313/JAMRIS_2-2015/25
- A. Bangor, P. Kortum, and J. Miller, “Determining What Individual SUS Scores Mean: Adding an Adjective Rating Scale”, Journal of Usability Studies, vol. 4, no. 3, 2009, 114–123.
Language: English
Page range: 1 - 8
Submitted on: Jan 20, 2025
Accepted on: May 13, 2025
Published on: Jun 22, 2026
In partnership with: Paradigm Publishing Services
Publication frequency: 4 issues per year
Keywords:
Related subjects:
© 2026 Sebastian Koryl, Katarzyna Zadarnowska, Krzysztof Arent, published by Łukasiewicz Research Network – Industrial Research Institute for Automation and Measurements PIAP
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.