Augmented reality in a multidisciplinary surgical clinic: application experience and prospects for technology development

The article presents the results of an analysis of the experience of using augmented reality (AR technology) by the staff of a multidisciplinary surgical clinic in various branches of surgery. 151 patients were included in the study, which was conducted from 2021 to 2025 at the Department of Naval Surgery of the Kirov Military Medical Academy and at its clinical base at the I.I. Dzhanelidze Research Institute of Emergency Medicine. All patients underwent surgery for various diseases of the chest, abdomen, pelvis, and extremities, the structure of which was represented by malignant locally spread tumors of the abdomen and pelvis (n=31), benign diseases of the biliopancreatoduodenal zone and chest (n=14), as well as foreign bodies (fragments) of the chest, abdomen, and extremities (n=106). The perioperative application of augmented reality technology included five main stages: patient selection, spiral computed tomography, the construction of a topographic and anatomical model (TAM) of the patient, the choice of a method for combining (linking) the model and the area of surgical intervention, and the intraoperative use of technology. The medical and technical support of AR technology is represented by a hardware and software complex – a navigation system based on augmented reality "Medical Vision", including a personal computer and augmented reality glasses "Microsoft Hololens II". The evaluation of AR technology was carried out in two directions. The first direction is an objective and subjective study of the accuracy of intraoperative comparison of TAM, the second is a comprehensive assessment of the degree of satisfaction of the surgical team with various aspects of the technology used. The average deviation of the topographic and anatomical model was the lowest when using AR technology for surgical removal of foreign bodies (fragments), unlike other types of surgical interventions, the same dynamics was demonstrated by the results of a subjective assessment. A comprehensive subjective assessment of AR technology satisfaction among surgeons, in general, indicates its positive perception. It was found that experienced surgeons (with more than 15 years of experience) attach more importance to the role of augmented reality technology at the planning stage, in contrast to less experienced surgeons who rated its intraoperative use higher. In addition, less experienced specialists speak better about the ergonomics of using AR technology during surgical interventions. The study shows that augmented reality technology in surgery is undoubtedly necessary in preoperative planning to increase the confidence of the surgical team and prepare it for the upcoming surgical intervention. A direct comparison TAM with the field of surgical intervention at this stage of the development of engineering support for AR technology is justified when removing foreign bodies (fragments). The results obtained in the study made it possible to formulate and substantiate a multimodal concept of planning and perioperative safety of surgical intervention based on augmented reality.

1. Konovalov A.N., Okishev D.N., Pilipenko Ju.V., et al. Dopolnennaja real'nost' kak metod nejronavigacii pri mikrohirurgicheskom lechenii cerebrovaskuljarnoj patologii: opisanie metoda i opyta primenenija. Zhurnal Voprosy Neirokhirurgii Imeni N.N. Burdenko. 2025;89(1):37–45. DOI: 10.17116/neiro20258901137. (In Russ.)

2. Konovalov A.N., Pilipenko Ju.V., Okishev D.N., et al. Ispol'zovanie dopolnennoj real'nosti kak metoda nejronavigacii pri vypolnenii jekstra-intrakranial'nogo mikroanastomoza. Operativnaya khirurgiya i klinicheskaya anatomiya. 2024;8(3):28–34. DOI: 10.17116/operhirurg2024803128. (In Russ.)].

3. Chegini S., Edwards E., McGurk M., et al. Systematic review of techniques used to validate the registration of augmented-reality images using a head-mounted device to navigate surgery. Br J Oral Maxillofac Surg. 2023;61(1):19–27. DOI: 10.1016/j.bjoms.2022.08.007.

4. Baashar Y., Alkawsi G., Ahmad W.N.W, et al. Effectiveness of using augmented reality for training in the medical professions: meta-analysis. JMIR Serious Games. 2022;10(3):e32715. DOI: 10.2196/32715.

5. Greben'kov V.G., Rumjancev V.N., Ivanov V.M., et al. Opyt primenenija tehnologii dopolnennoj real'nosti v hirurgicheskom lechenii bol'nyh pervichnymi i recidivnymi opuholjami organov malogo taza. Bulletin of the Russian Military Medical Academy. 2022;24(4):697–706. DOI: 10.17816/brmma112505. (In Russ.)

6. Agahanova M.D., Greben'kov V.G., Rumjancev V.N., et al. Opyt primenenija tehnologii dopolnennoj real'nosti v hirurgicheskom lechenii bol'nogo s inkapsulirovannymi metallicheskimi inorodnymi telami nizhnih konechnostej. Bulletin of the Russian Military Medical Academy. 2023;25(2):261–268. DOI: 10.17816/brmma321172. (In Russ.)

7. Ivanov V.M., Krivtsov A.M., Strelkov S.V., et al. Practical application of augmented/mixed reality technologies in surgery of abdominal cancer patients. Journal of Imaging. 2022;8(7):1–16. DOI: 10.3390/jimaging8070183.

8. Dai J., Qi W., Qiu Z., Li C. The application and prospection of augmented reality in hepato-pancreato-biliary surgery. Biosci Trends. 2023; 17(3):193–202. DOI: 10.5582/bst.2023.01086.

9. Ma L., Huang T., Wang J., Liao H. Visualization, registration and tracking techniques for augmented reality guided surgery: a review. Phys Med Biol. 2023;68(4). DOI: 10.1088/1361-6560/acaf23.

10. Javaheri H., Ghamarnejad O., Lukowicz P., et al. Enhancing perioperative outcomes of pancreatic surgery with wearable augmented reality assistance system: a matched-pair analysis. Ann. Surg. Open. 2024;(4):e516. DOI: 10.1097/AS9.0000000000000516.