Augmented reality for medical system Research- Biorobotics

Robot-assisted minimally invasive surgeries (RMIS) are becoming increasingly popular as they provide increased dexterity and control to the surgeon while also reducing trauma, blood loss and hospital stays for the patient. These devices are typically teleoperated by the surgeons using visual feedback from stereo-cameras, but without any haptic feedback. This can result in the surgeon relying only on vision to identify tumors by mentally forming the correspondence between intraoperative view and pre-operative images such as CT scans/MRI, which can be cognitively demanding.
Our group has been actively pursuing research in developing augmented reality (AR) for surgical systems. Our aim is to autonomously searches for tumors and dynamically displays a computer graphic model of them superimposed on the organ. Once localized, we aim to develop algorithms that can update the model based on the tissue deformation and physiological motions, while overlaying the stiffness information in real time. We believe that such a system has the potential to quickly reveal the location and shape of tumors, and the visual overlay will reduce the cognitive overload of the surgeon.
The AR system can also be used to overlay information annotated by the surgeon prior to the surgery on CT/MRI scans as well as any other preoperative surgical plans made by the surgeon. Further the surgeon would be able to interact with the system and provide information such as ‘no-fly’ zones, ablation trajectory, annotate important anatomical landmarks during the surgery which can be viewed and updated using the AR system.

Papers
ARforMedicalSystemF1 ARforMedicalSystemF2