Augmented reality for medical system Research- Biorobotics
Robot-assisted minimally invasive surgeries (RMIS) are becoming increasingly popular as they provide increased dexterity and control to the surgeon while also reducing trauma, blood loss and hospital stays for the patient. These devices are typically teleoperated by the surgeons using visual feedback from stereo-cameras, but without any haptic feedback. This can result in the surgeon relying only on vision to identify tumors by mentally forming the correspondence between intraoperative view and pre-operative images such as CT scans/MRI, which can be cognitively demanding.
Our group has been actively pursuing research in developing augmented reality (AR) for surgical systems. Our aim is to autonomously searches for tumors and dynamically displays a computer graphic model of them superimposed on the organ. Once localized, we aim to develop algorithms that can update the model based on the tissue deformation and physiological motions, while overlaying the stiffness information in real time. We believe that such a system has the potential to quickly reveal the location and shape of tumors, and the visual overlay will reduce the cognitive overload of the surgeon.
The AR system can also be used to overlay information annotated by the surgeon prior to the surgery on CT/MRI scans as well as any other preoperative surgical plans made by the surgeon. Further the surgeon would be able to interact with the system and provide information such as ‘no-fly’ zones, ablation trajectory, annotate important anatomical landmarks during the surgery which can be viewed and updated using the AR system.
- “A Real-time Augmented Reality Surgical System for Overlaying Stiffness Information”, in proceedings of RSS 2018.
- “A surgical system for automatic registration, stiffness mapping and dynamic image overlay”, in proceedings of ISMR 2018 (Best paper finalist)
- Development of an Inexpensive Tri-axial Force Sensor for Minimally Invasive Surgery”, In proceedings of IROS 2017.
- “Trajectory-Optimized Sensing for Active Search of Tissue Abnormalities in Robotic Surgery”, In proceedings of ICRA 2018.
- “Probabilistic pose estimation using a Bingham distribution-based linear filter ”, IJRR 2018.
- “Multimodal Registration Using Stereo Imaging and Contact Sensing”, RSS workshop 2017.
- “Using Bayesian Optimization to Guide Probing of a Flexible Environment for Simultaneous Registration and Stiffness Mapping”, ICRA 2016.
- “Complementary Model Update: A Method for Simultaneous Registration and Stiffness Mapping in Flexible Environments",ICRA 2016.
- “Sparse Point Registration”, ISRR 2017.