Loading [a11y]/accessibility-menu.js
Deep Learning Assisted Robotic Magnetic Anchored and Guided Endoscope for Real-Time Instrument Tracking | IEEE Journals & Magazine | IEEE Xplore

Deep Learning Assisted Robotic Magnetic Anchored and Guided Endoscope for Real-Time Instrument Tracking


Abstract:

This letter presents the first case of implementing deep-learning based instrument tracking on a magnetic anchored surgical endoscope. The compact magnetic actuated endos...Show More

Abstract:

This letter presents the first case of implementing deep-learning based instrument tracking on a magnetic anchored surgical endoscope. The compact magnetic actuated endoscope has a unique structure that allows operations near the anchor surface, ideal for video assisted thoracoscopic surgery (VATS). Autonomous tool tracking alleviates surgeon's burden and prevents human errors from muscle fatigues or miscommunication. However, conventional methods rely on color labels or require modification to instrument, and has risk of failure due to occlusion of marker. In this letter, we combine deep-learning instrument detection with visual servoing control. This allows the magnetic endoscope to track surgical tools automatically, without color markers or instrument modification. We used a modified TernausNet-16 network that can detect surgical instrument in real time, with a small training dataset of 1846 images. Experiments show that the magnetic endoscope can effectively track a marker-less instrument. It can also track continuous motions of a target traveling at 40 mm/s. The performance was also verified by completing mock-up surgical task in a simulated thoracic cavity.
Published in: IEEE Robotics and Automation Letters ( Volume: 6, Issue: 2, April 2021)
Page(s): 3979 - 3986
Date of Publication: 17 March 2021

ISSN Information:

Funding Agency:


Contact IEEE to Subscribe

References

References is not available for this document.