AI-Based System for Assistance in Minimally Invasive Renal Procedures Using Mixed Reality. First Steps

Abstract

The main objective of this study is the implementation and configuration of an assistance system for minimally invasive renal surgeries, incorporating an automatic segmentation module for the renal anatomy based on Artificial Intelligence (AI) and using computed tomography (CT) image studies, and its integration into an immersive and interactive mixed reality (MR) system. The goal is to enrich surgical planning, ensuring greater accuracy and safety to improve patient outcomes. The imaging dataset for this study was obtained from the KITS23 challenge, with 20 CT imaging studies randomly selected from the 489 available with ground truth annotations. The interactive interface using MR was developed using Unity in conjunction with the Microsoft HoloLens v2 device. For medical image segmentation, the Vista3D AI model was employed due to its versatility and high performance. All studies were successfully segmented, demonstrating a Dice score distribution with a high concentration of values above 0.8 for renal anatomy segmentation, indicating robust and consistent performance. However, for cyst segmentation, the Dice score distribution revealed a significant proportion of lower values, reflecting the complexity of this type of anatomical structures. In addition, an application for MR visualization of 3D renal anatomical models was developed to facilitate surgical planning. This application allows clinicians to better identify the renal anatomy in order to enhance traditional surgical planning methods. The development of this assistive system lays the foundation for increased accuracy, reduced errors and improved surgical outcomes, contributing to safer and more efficient procedures.