Augmented reality visualization for aiding biopsy procedure according to computed tomography based virtual plan

Majak, Marcin; Żuk, Magdalena; Świątek-Najwer, Ewelina; Popek, Michał; Pietruski, Piotr

doi:10.37190/abb-01811-2021-02

Abstract

Purpose: The purpose of this study was to develop and verify an intraoperative module for supporting navigated biopsy procedures using optical see-through head-mounted display (HMD).

Methods: Biopsy procedure including entry and endpoints of needle insertion was planned preoperatively having regard to the resection region segmentation and safety margin definition. Biopsy procedures were performed by two users using an intraoperative optical navigation module on a specially prepared brain phantom. Two visualization techniques were compared: an accurate augmented reality one, where a virtual plan is superimposed onto surgical field by using optical see-through HMD together with personalized calibration method and visualization on the external display.

Results: Averaged errors from 24 trials using external display were 2.04 ± 0.83 mm for the first user and 2.69 ± 1.11 mm for the second one, while applying HMD 2.50 ± 0.93 mm (the first user) and 2.17 ± 0.82 mm (the second user), respectively.

Conclusions: Proper usage of HMD visualization preceded by the personalized calibration allows the user to perform navigated biopsy procedure with comparable accuracy to its equivalent with the external display. Additionally, augmented reality visualization improves ergonomics and enables focusing on the surgical field without losing a direct line of sight with the field of view as it happens for external displays. However, ensuring high accuracy of augmented reality visualization still requires proper calibration and some user experience, which is challenging.

References

Andrews C.M., Henry A.B., Soriano I.M., Southworth M.K., Silva J.R., Registration Techniques for Clinical Applications of Three-Dimensional Augmented Reality Devices, IEEE Journal of Translational Engineering in Health and Medicine, 2020, 9, DOI: https://doi.org/10.1109/JTEHM.2020.3045642.
Search in Google Scholar Back to article
Badiali G., Cutolo F., Cercenelli L., M. Carbone M., D’Amato R., Ferrari V., Marchetti C., The VOSTARS project: A new wearable hybrid video and optical see-through augmented reality surgical system for maxillofacial surgery, Int. J. Maxillofacial Surg., 2019, 48, DOI: https://doi.org/10.1016/j.ijom.2019.03.472.
Search in Google Scholar Back to article
Badiali G. et al., Review on Augmented Reality in Oral and Cranio-Maxillofacial Surgery: Toward “Surgery-Specific” Head-Up Displays, IEEE Access, 2020, 8, 59015–59028, DOI: 10.1109/ACCESS.2020.2973298.
Search in Google Scholar Back to article
Bosc R., Fitoussi A., Hersant B., Dao T.H., Meningaud J.P, Intraoperative augmented reality with heads-up displays in maxillofacial surgery: A systematic review of the literature and a classification of relevant technologies, Int. J. Oral Maxillofacial Surg., 2019, 48 (1), 132–139, DOI: 10.1016/j.ijom.2018.09.010.
Search in Google Scholar Back to article
Carse B., Meadows B., Bowers R., Rowe P., Affordable clinical gait analysis: An assessment of the marker tracking accuracy of a new low-cost optical 3d motion analysis system, Physiotherapy, 2013, 99 (4), 347–351.
Search in Google Scholar Back to article
Chen X., Xu L., Wang Y., Wang H., Wang F., Zeng X., Wang Q., Egger J., Development of a surgical navigation system based on augmented reality using an optical see-through headmounted display, Journal of Biomedical Informatics, 2015, 55, DOI: https://doi.org/10.1016/j.jbi.2015.04.003.
Search in Google Scholar Back to article
Cutolo F. et al., Ambiguity-Free Optical–Inertial Tracking for Augmented Reality Headsets, Sensors, 2020, 20 (5), DOI: 10.3390/s20051444.
Search in Google Scholar Back to article
De Amici S., Sanna A., Lamberti F., Pralio B., A wii remote-based infrared-optical tracking system, Entertainment Computing, 2010, 1 (3–4), 119.
Search in Google Scholar Back to article
Garrido-Jurado S., Munoz-Salinas R., Madrid-Cuevas F., Marin-Jimenez M., Automatic generation and detection of highly reliable fiducial markers under occlusion, Pattern Recognition, 2014, 47 (6), 2280–2292.
Search in Google Scholar Back to article
García-Vázquez V., Von Haxthausen F., Jäckle S., Schumann C., Kuhlemann I., Bouchagiar J., Höfer A.C., Matysiak F., Hüttmann G., Goltz J.P., Kleemann M., Ernst F., Horn M., Navigation and visualization with Holo-Lens in endovascular aortic repair, Innov. Surg. Sci., 2018, 3 (3), 167–177, DOI: 10.1515/iss-2018-2001.
Search in Google Scholar Back to article
Gil J.J., Díaz I., Accini F., Inferring material properties in robotic bone drilling processes, Acta Bioeng. Biomech., 2019, 21 (3), 109–118, DOI: 10.5277/ABB-01386-2019-02.
Search in Google Scholar Back to article
Grubert J., Itoh Y., Moser K., Swan J.E., A Survey of Calibration Methods for Optical See-Through Head-Mounted Displays, IEEE Transactions on Visualization and Computer Graphics, 2018, 24 (9), 2649–2662, DOI: 10.1109/TVCG.2017.2754257.
Search in Google Scholar Back to article
Itoh Y., Klinker G., Interaction-free calibration for optical seethrough head-mounted displays based on 3D eye localization, Proc. IEEE Symp. 3D User Interfaces, 2014, 75–82.
Search in Google Scholar Back to article
Kunz C., Genten V., Meißner P., Hein B., Metric-based evaluation of fiducial markers for medical procedures, Proc. SPIE 10951, Medical Imaging 2019: Image-Guided Procedures, Robotic Interventions, and Modeling, 109512O, 2019, DOI: https://doi.org/10.1117/12.2511720.
Search in Google Scholar Back to article
Lin M.A., Siu A.F., Bae J.H., Cutkosky M.R., Daniel B.L., HoloNeedle: Augmented reality guidance system for needle placement investigating the advantages of three-dimensional needle shape reconstruction, IEEE Robot. Autom. Lett., 2018, 3 (4), 4156–4162, DOI: 10.1109/LRA.2018.2863381
Search in Google Scholar Back to article
Majak M., Żuk M., Świątek-Najwer E., Popek M., Pietruski P., Biopsy procedure applied in MentorEye molecular surgical navigation system, Lecture Notes in Computational Vision and Biomechanics, 2018, 27, 338–344.
Search in Google Scholar Back to article
Makibuchi N., Kato H., Yoneyama A., Vision-based robust calibration for optical see-through head-mounted displays, Proc. IEEE Int. Conf. Image Process., 2013, 2177–2181.
Search in Google Scholar Back to article
Mcknight R.R., Pean C.A., Buck J.S. et al., Virtual Reality and Augmented Reality – Translating Surgical Training into Surgical Technique, Curr. Rev. Musculoskelet. Med., 2020, 13, 663–674, https://doi.org/10.1007/s12178-020-09667-3
Search in Google Scholar Back to article
De Oliveira M.E., Debarba H.G., Lädermann A., Chagué S., Charbonnier C., A hand-eye calibration method for augmented reality applied to computer-assisted orthopedic surgery, Int. J. Med. Robot., 2019, 15 (2), DOI: 10.1002/rcs.1969.
Search in Google Scholar Back to article
Pietruski P., Majak M., Świątek-Najwer E., Żuk M., Popek M., Mazurek M., Świecka M., Jaworowski J., Navigation-guided fibula free flap for mandibular reconstruction: A proof of concept study, Journal of Plastic, Reconstructive and Aesthetic Surgery, 2019, 72 (4), DOI: 10.1016/j.bjps.2019.01.026.
Search in Google Scholar Back to article
Pietruski P., Majak M., Świątek-Najwer E., Żuk M., Popek M., Jaworowski J., Mazurek M., Supporting fibula free flap harvest with augmented reality: A proof-of-concept study, The Laryngoscope, 2019, 130 (5), 1173–1179, https://doi.org/10.1002/lary.28090
Search in Google Scholar Back to article
Pietruski P. et. al., Supporting mandibular resection with intraoperative navigation utilizing augmented reality technology – A proof of concept study, Journal of Cranio-Maxillofacial Surgery, 2019, 47 (6), DOI: https://doi.org/10.1016/j.jcms.2019.03.004.
Search in Google Scholar Back to article
Qian L., Barthel A., Johnson A., Osgood G., Kazanzides P., Navab N., Fuerst B., Comparison of optical see-through head-mounted displays for surgical interventions with objectanchored 2D-display, Int. J. Comput. Assist. Radiol. Surg., 2017, 12 (6), DOI: 10.1007/s11548-017-1564-y.
Search in Google Scholar Back to article
Rahman R., Wood M.E., Qian L., Price C.L., Johnson A.A., Osgood G.M., Head-Mounted Display Use in Surgery: A Systematic Review, Surgical Innovation, 2020, 27 (1), 88–100, DOI: 10.1177/1553350619871787.
Search in Google Scholar Back to article
Sakai D., Joyce K., Sugimoto M. et al., Augmented, virtual and mixed reality in spinal surgery: A real-world experience, Journal of Orthopaedic Surgery, 2020, DOI: 10.1177/2309499020952698.
Search in Google Scholar Back to article
Tuceryan M., Genc Y., Navab N., Single-Point Active Alignment Method (SPAAM) for Optical See-Through HMD Calibration for Augmented Reality, Teleoperators and Virtual Environments, 2002, 11, 259–276.
Search in Google Scholar Back to article
Wacker F.K., Vogt S.K., Khamene A., Jesberger J.A., Nour S.G., Elgort D.R., Sauer F., Duerk J.L., Lewin J.S., An augmented reality system for MR image-guided needle biopsy: initial results in a swine model, Radiology, 2006, 238 (2), 497–504.
Search in Google Scholar Back to article
Wang H., Wang F., Xu L., Chen X., Wang Q., Precision insertion of percutaneous sacroiliac screws using a novel augmented reality-based navigation system: a pilot study, International Orthopaedics, 2016, 40, 1941–1947.
Search in Google Scholar Back to article
Wang J., Shen Y., Yang S., A practical marker-less image registration method for augmented reality oral and maxillofacial surgery, Int. J. Comput. Assist. Radiol. Surg., 2019, 14 (5), 763–773, DOI: 10.1007/s11548-019-01921-5.
Search in Google Scholar Back to article
Żuk M., Majak M., Świątek-Najwer E., Popek M., Kulas Z., Evaluation of calibration procedure for stereoscopic visualization using optical See-Through Head Mounted Displays for a complex oncological treatment, Lecture Notes in Computational Vision and Biomechanics, 2018, 27, 354–359.
Search in Google Scholar Back to article

Augmented reality visualization for aiding biopsy procedure according to computed tomography based virtual plan

Abstract

Paradigm

My account