A Gesture-Based Interface for Remote Surgery

Abstract

There has been a great deal of research activity in computer- and robot-assisted surgeries in recent years. Some of the advances have included robotic hip surgery, image-guided endoscopic surgery, and the use of intra-operative MRI to assist in neurosurgery. However, most of the work in the literature assumes that all of the expert surgeons are physically present close to the location of a surgery. A new direction that is now worth investigating is assisting in performing surgeries remotely. As a first step in this direction, this chapter presents a system that can detect movement of hands and fingers, and thereby detect gestures, which can be used to control a catheter remotely. Our development is aimed at performing remote endovascular surgery by controlling the movement of a catheter through blood vessels. Our hand movement detection is facilitated by sensors, like LEAP, which can track the position of fingertips and the palm. In order to make the system robust to occlusions, we have improved the implementation by optimally integrating the input from two different sensors. Following this step, we identify high-level gestures, like push and turn, to enable remote catheter movements. To simulate a realistic environment we have fabricated a flexible endovascular mold, and also a phantom of the abdominal region with the endovascular mold integrated inside. A mechanical device that can remotely control a catheter based on movement primitives extracted from gestures has been built. Experimental results are shown demonstrating the accuracy of the system.