Abstract
A microsurgery system that achieves real-time enhanced micrometer scale positioning accuracy by fusing visual information from a high speed monovision camera mounted on an optical surgical microscope and acceleration measurements from an intelligent handheld instrument, ITrem2, is presented. The high speed camera captures images of the tool tip of ITrem2 to track its position in real-time. The focus value of the tool tip in the acquired image is used to locate the tool tip along the principal axis of the objective lens of the microscope and edge based geometric template matching gives the position in pixel coordinates. ITrem2 utilizes four dual-axis miniature digital MEMS accelerometers to sense and update the motion information. The system has a first in, first out (FIFO) queue to track the recent history of the slow non-drifting position estimation from the vision system and acceleration readings from the inertial sensors together with their respective time stamps. In the proposed method, real-time visual servoing of micrometer scale motion is achieved by taking into account the dynamic behavior of the vision feedback and incorporating synchronized fusion of these complementary sensors.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Rehbinder, H., Ghosh, B.K.: Multi-rate fusion of visual and inertial data. In: International Conference on Multisensor Fusion and Integration for Intelligent Systems, MFI 2001, pp. 97–102 (2001)
Armesto, L., Chroust, S., Vincze, M., Tornero, J.: Multi-rate fusion with vision and inertial sensors. In: Proceedings of Robotics and Automation, ICRA 2004, April 26 -May 1, vol. 1, pp. 193–199 (2004)
Hol, J.D., et al.: Robust real-time tracking by fusing measurements from inertial and vision sensors. Journal of Real-Time Image Processing 2, 149–160 (2007)
Huster, A., Frew, E.W., Rock, S.M.: Relative position estimation for AUVs by fusing bearing and inertial rate sensor measurements. In: MTS/IEEE OCEANS 2002, October 29-31, vol. 3, pp. 1863–1870 (2002)
Parnian, N., Golnaraghi, M.F.: Integration of vision and inertial sensors for industrial tools tracking. Sensor Review 27(2), 132–141 (2007)
Tan, U.-X., Veluvolu, K., Latt, W.T., Shee, C.Y., Riviere, C., Ang, W.T.: Estimating displacement of periodic motion with inertial sensors. IEEE Sensors Journal 8, 1385–1388 (2008)
Veluvolu, K.C., Ang, W.T.: Estimation of physiological tremor from accelerometers for real-time applications. Sensors 11(3), 3020–3036 (2011)
Latt, W.T., Veluvolu, K.C., Ang, W.T.: Drift-free position estimation of periodic or quasi-periodic motion using inertial sensors. Sensors 11(6), 5931–5951 (2011)
Karras, G.C., Loizou, S.G., Kyriakopoulos, K.J.: A visual-servoing scheme for semi-autonomous operation of an underwater robotic vehicle using an IMU and a Laser Vision System. In: Robotics and Automation (ICRA), May 3-7, pp. 5262–5267 (2010)
Zhou, Y., Nelson, B.J., Vikramaditya, B.: Fusing force and vision feedback for micromanipulation. In: Proceedings of 1998 IEEE International Conference on Robotics and Automation, May 16-20, vol. 2, pp. 1220–1225 (1998)
Sun, Y., Duthaler, S., Nelson, B.J.: Autofocusing Algorithm Selection in Computer Microscopy. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, August 2-6, pp. 70–76 (2005)
Wu, Q., Merchant, F.A., Castleman, K.R.: Microscope Image Processing. Academic Press (2008)
Ang, W.T., Riviere, C.N., Khosla, P.K.: Nonlinear regression model of a low-g MEMS accelerometer. IEEE Sensors J. 7(1&2), 81–88 (2007)
Latt, W.T., Tan, U.-X., Shee, C.Y., Riviere, C.N., Ang, W.T.: Compact Sensing Design of a Hand-held Active Tremor Compensation Instrument. IEEE Sensors Journal 9, 1864–1871 (2009)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Aye, Y.N., Zhao, S., Shee, C.Y., Ang, W.T. (2013). Fusion of Inertial Measurements and Vision Feedback for Microsurgery. In: Lee, S., Yoon, KJ., Lee, J. (eds) Frontiers of Intelligent Autonomous Systems. Studies in Computational Intelligence, vol 466. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-35485-4_29
Download citation
DOI: https://doi.org/10.1007/978-3-642-35485-4_29
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-35484-7
Online ISBN: 978-3-642-35485-4
eBook Packages: EngineeringEngineering (R0)