Skip to main content

Advertisement

SpringerLink
Log in

Coronary Heart Disease Preoperative Gesture Interactive Diagnostic System Based on Augmented Reality

  • Education & Training
  • Published:
Journal of Medical Systems Aims and scope Submit manuscript

Abstract

Coronary heart disease preoperative diagnosis plays an important role in the treatment of vascular interventional surgery. Actually, most doctors are used to diagnosing the position of the vascular stenosis and then empirically estimating vascular stenosis by selective coronary angiography images instead of using mouse, keyboard and computer during preoperative diagnosis. The invasive diagnostic modality is short of intuitive and natural interaction and the results are not accurate enough. Aiming at above problems, the coronary heart disease preoperative gesture interactive diagnostic system based on Augmented Reality is proposed. The system uses Leap Motion Controller to capture hand gesture video sequences and extract the features which that are the position and orientation vector of the gesture motion trajectory and the change of the hand shape. The training planet is determined by K-means algorithm and then the effect of gesture training is improved by multi-features and multi-observation sequences for gesture training. The reusability of gesture is improved by establishing the state transition model. The algorithm efficiency is improved by gesture prejudgment which is used by threshold discriminating before recognition. The integrity of the trajectory is preserved and the gesture motion space is extended by employing space rotation transformation of gesture manipulation plane. Ultimately, the gesture recognition based on SRT-HMM is realized. The diagnosis and measurement of the vascular stenosis are intuitively and naturally realized by operating and measuring the coronary artery model with augmented reality and gesture interaction techniques. All of the gesture recognition experiments show the distinguish ability and generalization ability of the algorithm and gesture interaction experiments prove the availability and reliability of the system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. World Health Organization., Global status report on noncommunicable diseases 2014: Geneva, World Health Organization, pp 29–30, 2015.

  2. Miller, J.M., Rochitte, C.E., Dewey, M., Arbabzadeh, A., Niinuma, H., Gottlieb, I., et al., Diagnostic performance of coronary angiography by 64-row ct. New England Journal of Medicine. 359(22):2324–2336, 2008. doi:10.1056/nejmoa0806576.

    Article  CAS  PubMed  Google Scholar 

  3. Fine, J.J., Hopkins, C.B., Ruff, N., and Newton, F.C., Comparison of accuracy of 64-slice cardiovascular computed tomography with coronary angiography in patients with suspected coronary artery disease. The American journal of cardiology. 97(2):173–174, 2006. doi:10.1016/j.amjcard.2005.08.021.

    Article  PubMed  Google Scholar 

  4. Kukar, M., and Grošelj, C., Transductive machine learning for reliable medical diagnostics. Journal of Medical Systems. 29(1):13–32, 2005. doi:10.1007/s10916-005-1101-3.

    Article  PubMed  Google Scholar 

  5. Mandal, I., and Sairam, N., Accurate prediction of coronary artery disease using reliable diagnosis system. Journal of Medical Systems. 36(5):3353–3373, 2012. doi:10.1007/s10916-012-9828-0.

    Article  PubMed  Google Scholar 

  6. Lahsasna, A., Ainon, R.N., Zainuddin, R., and Bulgiba, A., Design of a fuzzy-based decision support system for coronary heart disease diagnosis. Journal of Medical Systems. 36(5):3293–3306, 2012. doi:10.1007/s10916-012-9821-7.

    Article  PubMed  Google Scholar 

  7. Karabulut, E.M., and İbrikçi, T., Effective diagnosis of coronary artery disease using the rotation forest ensemble method. Journal of Medical Systems. 36(5):3011–3018, 2011. doi:10.1007/s10916-011-9778-y.

    Article  PubMed  Google Scholar 

  8. Gao, M.K., Chen, Y.M., Liu, Q., Huang, C., Li, Z.Y., and Zhang, D.H., Three-dimensional path planning and guidance of leg vascular based on improved ant colony algorithm in augmented reality. Journal of Medical Systems. 39(11):1–11, 2015. doi:10.1007/s10916-015-0315-2.

    Article  CAS  Google Scholar 

  9. Adhami, L., Jacobs, S., and Thiele, H., Cardio navigation: planning, simulation, and augmented reality in robotic assisted endoscopic bypass grafting. Annals of Thoracic Surgery. 79(6):2040–2048, 2005. doi:10.1016/j.athoracsur.2004.11.060.

    Article  PubMed  Google Scholar 

  10. Figl, M., Rueckert, D., Hawkes, D., Casula, R., Hu, M., and Pedro, O., et al., Coronary Motion Modeling for Augmented Reality Guidance of Endoscopic Coronary Artery Bypass. Biomedical Simulation, International Symposium, Isbms 2008, London, Uk, July 7–8, 2008, Proceedings. Vol. 5104, pp. 197–202. 2008. doi: 10.1007/978–3–540-70521-5_22.

  11. Chen, E.C.S., Wedlake, C., and Peters, T.M., An augmented reality platform for planning of minimally invasive cardiac surgeries. Proceedings of SPIE - The International Society for Optical Engineering. 8316(4):563–567, 2012. doi:10.1117/12.911998.

    Google Scholar 

  12. Szabó, Z., Berg, S., Sjökvist, S., Gustafsson, T., Carleberg, P., Uppsäll, M., et al., Real-time intraoperative visualization of myocardial circulation using augmented reality temperature display. International Journal of Cardiovascular Imaging. 29(2):521–528, 2013. doi:10.1007/s10554-012-0094-5.

    Article  PubMed  Google Scholar 

  13. Weichert, F., Bachmann, D., Rudak, B., and Fisseler, D., Analysis of the accuracy and robustness of the leap motion controller. Sensors. 13(5):6380–6393, 2013. doi:10.3390/s130506380.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Guna, J., Jakus, G., Pogačnik, M., Tomažič, S., and Sodnik, J., An analysis of the precision and reliability of the leap motion sensor and its suitability for static and dynamic tracking. Sensors. 14(2):3702–3720, 2014. doi:10.3390/s140203702.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Regenbrecht, H., Collins, J., and Hoermann, S., A leap-supported, hybrid AR interface approach. Australian Computer-Human Interaction Conference: Augmentation, Application, Innovation, Collaboration. pp. 281–284, 2013. doi:10.1145/2541016.2541053.

  16. Khattak, S., Cowan, B., Chepurna, I., and Hogue, A., A real-time reconstructed 3D environment augmented with virtual objects rendered with correct occlusion. IEEE Games, Entertainment, and Media Conference. pp. 1–8, 2014. doi: 10.1109/GEM.2014.7048102.

  17. Vargas, Hermes Fabian, and Vivas, O. A., Gesture recognition system for surgical robot's manipulation. 2014 XIX Symposium on IEEE Image, Signal Processing and Artificial Vision (STSIVA). pp. 1–5, 2014. doi: 10.1109/STSIVA.2014.7010172.

  18. Shen, J., Luo, Y., Wang, X., Wu, Z., and Zhou, M., GPU-Based Realtime Hand Gesture Interaction and Rendering for Volume Datasets Using Leap Motion. International Conference on Cyberworlds. IEEE Computer Society. pp. 85–92, 2014. doi: 10.1109/CW.2014.20.

  19. Rabiner, L.R., A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE. 77(2):257–286, 1989. doi:10.1016/B978-0-08-051584-7.50027-9.

    Article  Google Scholar 

  20. Rose, R. C., Discriminant wordspotting techniques for rejecting non-vocabulary utterances in unconstrained speech, ICASSP, 1992, Acoustics, Speech, and Signal Processing, IEEE International Conference on, Acoustics, Speech, and Signal Processing, IEEE International Conference on 1992. pp. 105–108, 1992. doi:10.1109/ICASSP.1992.226109.

  21. Lee, S.H., and Jin, H.K., Augmenting the discrimination power of hmm by nn for on-line cursive script recognition. Applied Intelligence. 7(7):305–314, 1997. doi:10.1023/A:1008261419981.

    Article  Google Scholar 

  22. Khorasani, A., and Daliri, M.R., Hmm for classification of parkinson’s disease based on the raw gait data. Journal of Medical Systems. 38(12):1–6, 2014. doi:10.1007/s10916-014-0147-5.

    Article  Google Scholar 

  23. Li, Z., Wei, Z., Yue, Y., Wang, H., Jia, W., Burke, L.E., et al., An adaptive hidden markov model for activity recognition based on a wearable multi-sensor device. Journal of Medical Systems. 39(5):1–10, 2015. doi:10.1007/s10916-015-0239-x.

    Article  Google Scholar 

  24. Bilal, S., Akmeliawati, R., Shafie, A.A., and Salami, M.J.E., Hidden markov model for human to computer interaction: a study on human hand gesture recognition. Artificial Intelligence Review. 40(4):495–516, 2013. doi:10.1007/s10462-011-9292-0.

    Article  Google Scholar 

  25. Yoon, H.S., Soh, J., Bae, Y.J., and Yang, H.S., Hand gesture recognition using combined features of location, angle and velocity. Pattern Recognition. 34(7):1491–1501, 2001. doi:10.1016/S0031-3203(00)00096-0.

    Article  Google Scholar 

  26. Chen, Y.M., and Zhang, Y.H., Research on human-robot interaction technique based on hand gesture recognition. Robot. 31(4):351–356, 2009. doi:10.3321/j.issn:1002-0446.2009.04.009.

    Google Scholar 

  27. Li, T.H.S., Kao, M.C., and Kuo, P.H., Recognition system for home-service-related sign language using entropy-based k-means algorithm and abc-based hmm. IEEE Transactions on Systems Man & Cybernetics Systems. 64(5):1–1, 2015. doi:10.1109/TSMC.2015.2435702.

    Article  Google Scholar 

  28. Liu, N., Lovell, B. C., Kootsookos, P. J., and Davis, R. I. A., Model Structure Selection and Training Algorithms for an HMM Gesture Recognition System. International Workshop on Frontiers in Handwriting Recognition. Vol. 1, pp. 100–105. IEEE Computer Society, 2004. doi:10.1109/IWFHR.2004.68.

  29. Lee, J.H., Delbruck, T., Pfeiffer, M., et al., Real-time gesture interface based on event-driven processing from stereo silicon retinas. IEEE Transactions on Neural Networks & Learning Systems. 25(12):2250, 2014. doi:10.1109/TNNLS.2014.2308551.

    Article  Google Scholar 

  30. Preim, B., and Oeltze, S., 3d visualization of vasculature: an overview. Mathematics & Visualization. 296(1–4):39–59, 2008. doi:10.1007/978-3-540-72630-2_3.

    Article  Google Scholar 

  31. Antiga, L., Eneiordache, B., and Remuzzi, A., Computational geometry for patient-specific reconstruction and meshing of blood vessels from mr and ct angiography. IEEE Transactions on Medical Imaging. 22(5):674–684, 2003. doi:10.1109/TMI.2003.812261.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

This work was supported by Shanghai International Cooperation Fund Project(No.12510708400) and Shanghai Innovation Action Plan Project(No.16511101200) of Science and Technology Committee of Shanghai Municipality.

Author information

Authors and Affiliations

  1. College of Information Technology, Shanghai Ocean University, No.999 Huchenghuan Road , Pudong New District, Shanghai City, China

    Yi-bo Zou

  2. School of Computer Engineering and Science, Shanghai University, No.99 Shangda Road, Baoshan District, Shanghai City, China

    Yi-min Chen, Ming-ke Gao, Quan Liu, Si-yu Jiang, Jia-hui Lu, Chen Huang, Ze-yu Li & Dian-hua Zhang

  3. The 32nd Research Institute, China Electronics Technology Group Corporation, No.63 Chengliugong Road, Jiading District, Shanghai City, China

    Ming-ke Gao

  4. Ruijin Hospital, School of Medicine, Shanghai Jiao Tong University, No2 Ruijin Road, Shanghai City, China

    Ze-yu Li

Authors
  1. Yi-bo Zou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yi-min Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ming-ke Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Quan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Si-yu Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jia-hui Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Chen Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Ze-yu Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Dian-hua Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yi-min Chen.

Additional information

This article is part of the Topical Collection on Education & Training

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zou, Yb., Chen, Ym., Gao, Mk. et al. Coronary Heart Disease Preoperative Gesture Interactive Diagnostic System Based on Augmented Reality. J Med Syst 41, 126 (2017). https://doi.org/10.1007/s10916-017-0768-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10916-017-0768-6

Keywords

Search

Navigation