Abstract
Coronary heart disease preoperative diagnosis plays an important role in the treatment of vascular interventional surgery. Actually, most doctors are used to diagnosing the position of the vascular stenosis and then empirically estimating vascular stenosis by selective coronary angiography images instead of using mouse, keyboard and computer during preoperative diagnosis. The invasive diagnostic modality is short of intuitive and natural interaction and the results are not accurate enough. Aiming at above problems, the coronary heart disease preoperative gesture interactive diagnostic system based on Augmented Reality is proposed. The system uses Leap Motion Controller to capture hand gesture video sequences and extract the features which that are the position and orientation vector of the gesture motion trajectory and the change of the hand shape. The training planet is determined by K-means algorithm and then the effect of gesture training is improved by multi-features and multi-observation sequences for gesture training. The reusability of gesture is improved by establishing the state transition model. The algorithm efficiency is improved by gesture prejudgment which is used by threshold discriminating before recognition. The integrity of the trajectory is preserved and the gesture motion space is extended by employing space rotation transformation of gesture manipulation plane. Ultimately, the gesture recognition based on SRT-HMM is realized. The diagnosis and measurement of the vascular stenosis are intuitively and naturally realized by operating and measuring the coronary artery model with augmented reality and gesture interaction techniques. All of the gesture recognition experiments show the distinguish ability and generalization ability of the algorithm and gesture interaction experiments prove the availability and reliability of the system.















Similar content being viewed by others
References
World Health Organization., Global status report on noncommunicable diseases 2014: Geneva, World Health Organization, pp 29–30, 2015.
Miller, J.M., Rochitte, C.E., Dewey, M., Arbabzadeh, A., Niinuma, H., Gottlieb, I., et al., Diagnostic performance of coronary angiography by 64-row ct. New England Journal of Medicine. 359(22):2324–2336, 2008. doi:10.1056/nejmoa0806576.
Fine, J.J., Hopkins, C.B., Ruff, N., and Newton, F.C., Comparison of accuracy of 64-slice cardiovascular computed tomography with coronary angiography in patients with suspected coronary artery disease. The American journal of cardiology. 97(2):173–174, 2006. doi:10.1016/j.amjcard.2005.08.021.
Kukar, M., and Grošelj, C., Transductive machine learning for reliable medical diagnostics. Journal of Medical Systems. 29(1):13–32, 2005. doi:10.1007/s10916-005-1101-3.
Mandal, I., and Sairam, N., Accurate prediction of coronary artery disease using reliable diagnosis system. Journal of Medical Systems. 36(5):3353–3373, 2012. doi:10.1007/s10916-012-9828-0.
Lahsasna, A., Ainon, R.N., Zainuddin, R., and Bulgiba, A., Design of a fuzzy-based decision support system for coronary heart disease diagnosis. Journal of Medical Systems. 36(5):3293–3306, 2012. doi:10.1007/s10916-012-9821-7.
Karabulut, E.M., and İbrikçi, T., Effective diagnosis of coronary artery disease using the rotation forest ensemble method. Journal of Medical Systems. 36(5):3011–3018, 2011. doi:10.1007/s10916-011-9778-y.
Gao, M.K., Chen, Y.M., Liu, Q., Huang, C., Li, Z.Y., and Zhang, D.H., Three-dimensional path planning and guidance of leg vascular based on improved ant colony algorithm in augmented reality. Journal of Medical Systems. 39(11):1–11, 2015. doi:10.1007/s10916-015-0315-2.
Adhami, L., Jacobs, S., and Thiele, H., Cardio navigation: planning, simulation, and augmented reality in robotic assisted endoscopic bypass grafting. Annals of Thoracic Surgery. 79(6):2040–2048, 2005. doi:10.1016/j.athoracsur.2004.11.060.
Figl, M., Rueckert, D., Hawkes, D., Casula, R., Hu, M., and Pedro, O., et al., Coronary Motion Modeling for Augmented Reality Guidance of Endoscopic Coronary Artery Bypass. Biomedical Simulation, International Symposium, Isbms 2008, London, Uk, July 7–8, 2008, Proceedings. Vol. 5104, pp. 197–202. 2008. doi: 10.1007/978–3–540-70521-5_22.
Chen, E.C.S., Wedlake, C., and Peters, T.M., An augmented reality platform for planning of minimally invasive cardiac surgeries. Proceedings of SPIE - The International Society for Optical Engineering. 8316(4):563–567, 2012. doi:10.1117/12.911998.
Szabó, Z., Berg, S., Sjökvist, S., Gustafsson, T., Carleberg, P., Uppsäll, M., et al., Real-time intraoperative visualization of myocardial circulation using augmented reality temperature display. International Journal of Cardiovascular Imaging. 29(2):521–528, 2013. doi:10.1007/s10554-012-0094-5.
Weichert, F., Bachmann, D., Rudak, B., and Fisseler, D., Analysis of the accuracy and robustness of the leap motion controller. Sensors. 13(5):6380–6393, 2013. doi:10.3390/s130506380.
Guna, J., Jakus, G., Pogačnik, M., Tomažič, S., and Sodnik, J., An analysis of the precision and reliability of the leap motion sensor and its suitability for static and dynamic tracking. Sensors. 14(2):3702–3720, 2014. doi:10.3390/s140203702.
Regenbrecht, H., Collins, J., and Hoermann, S., A leap-supported, hybrid AR interface approach. Australian Computer-Human Interaction Conference: Augmentation, Application, Innovation, Collaboration. pp. 281–284, 2013. doi:10.1145/2541016.2541053.
Khattak, S., Cowan, B., Chepurna, I., and Hogue, A., A real-time reconstructed 3D environment augmented with virtual objects rendered with correct occlusion. IEEE Games, Entertainment, and Media Conference. pp. 1–8, 2014. doi: 10.1109/GEM.2014.7048102.
Vargas, Hermes Fabian, and Vivas, O. A., Gesture recognition system for surgical robot's manipulation. 2014 XIX Symposium on IEEE Image, Signal Processing and Artificial Vision (STSIVA). pp. 1–5, 2014. doi: 10.1109/STSIVA.2014.7010172.
Shen, J., Luo, Y., Wang, X., Wu, Z., and Zhou, M., GPU-Based Realtime Hand Gesture Interaction and Rendering for Volume Datasets Using Leap Motion. International Conference on Cyberworlds. IEEE Computer Society. pp. 85–92, 2014. doi: 10.1109/CW.2014.20.
Rabiner, L.R., A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE. 77(2):257–286, 1989. doi:10.1016/B978-0-08-051584-7.50027-9.
Rose, R. C., Discriminant wordspotting techniques for rejecting non-vocabulary utterances in unconstrained speech, ICASSP, 1992, Acoustics, Speech, and Signal Processing, IEEE International Conference on, Acoustics, Speech, and Signal Processing, IEEE International Conference on 1992. pp. 105–108, 1992. doi:10.1109/ICASSP.1992.226109.
Lee, S.H., and Jin, H.K., Augmenting the discrimination power of hmm by nn for on-line cursive script recognition. Applied Intelligence. 7(7):305–314, 1997. doi:10.1023/A:1008261419981.
Khorasani, A., and Daliri, M.R., Hmm for classification of parkinson’s disease based on the raw gait data. Journal of Medical Systems. 38(12):1–6, 2014. doi:10.1007/s10916-014-0147-5.
Li, Z., Wei, Z., Yue, Y., Wang, H., Jia, W., Burke, L.E., et al., An adaptive hidden markov model for activity recognition based on a wearable multi-sensor device. Journal of Medical Systems. 39(5):1–10, 2015. doi:10.1007/s10916-015-0239-x.
Bilal, S., Akmeliawati, R., Shafie, A.A., and Salami, M.J.E., Hidden markov model for human to computer interaction: a study on human hand gesture recognition. Artificial Intelligence Review. 40(4):495–516, 2013. doi:10.1007/s10462-011-9292-0.
Yoon, H.S., Soh, J., Bae, Y.J., and Yang, H.S., Hand gesture recognition using combined features of location, angle and velocity. Pattern Recognition. 34(7):1491–1501, 2001. doi:10.1016/S0031-3203(00)00096-0.
Chen, Y.M., and Zhang, Y.H., Research on human-robot interaction technique based on hand gesture recognition. Robot. 31(4):351–356, 2009. doi:10.3321/j.issn:1002-0446.2009.04.009.
Li, T.H.S., Kao, M.C., and Kuo, P.H., Recognition system for home-service-related sign language using entropy-based k-means algorithm and abc-based hmm. IEEE Transactions on Systems Man & Cybernetics Systems. 64(5):1–1, 2015. doi:10.1109/TSMC.2015.2435702.
Liu, N., Lovell, B. C., Kootsookos, P. J., and Davis, R. I. A., Model Structure Selection and Training Algorithms for an HMM Gesture Recognition System. International Workshop on Frontiers in Handwriting Recognition. Vol. 1, pp. 100–105. IEEE Computer Society, 2004. doi:10.1109/IWFHR.2004.68.
Lee, J.H., Delbruck, T., Pfeiffer, M., et al., Real-time gesture interface based on event-driven processing from stereo silicon retinas. IEEE Transactions on Neural Networks & Learning Systems. 25(12):2250, 2014. doi:10.1109/TNNLS.2014.2308551.
Preim, B., and Oeltze, S., 3d visualization of vasculature: an overview. Mathematics & Visualization. 296(1–4):39–59, 2008. doi:10.1007/978-3-540-72630-2_3.
Antiga, L., Eneiordache, B., and Remuzzi, A., Computational geometry for patient-specific reconstruction and meshing of blood vessels from mr and ct angiography. IEEE Transactions on Medical Imaging. 22(5):674–684, 2003. doi:10.1109/TMI.2003.812261.
Acknowledgements
This work was supported by Shanghai International Cooperation Fund Project(No.12510708400) and Shanghai Innovation Action Plan Project(No.16511101200) of Science and Technology Committee of Shanghai Municipality.
Author information
Authors and Affiliations
Corresponding author
Additional information
This article is part of the Topical Collection on Education & Training
Rights and permissions
About this article
Cite this article
Zou, Yb., Chen, Ym., Gao, Mk. et al. Coronary Heart Disease Preoperative Gesture Interactive Diagnostic System Based on Augmented Reality. J Med Syst 41, 126 (2017). https://doi.org/10.1007/s10916-017-0768-6
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10916-017-0768-6