Skip to main content
SpringerLink
Log in

3D Interaction Through a Real-Time Gesture Search Engine

  • Conference paper
  • First Online:
Computer Vision - ACCV 2014 Workshops (ACCV 2014)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 9009))

Included in the following conference series:

Abstract

3D gesture recognition and tracking are highly desired features of interaction design in future mobile and smart environments. Specifically, in virtual/augmented reality applications, intuitive interaction with the physical space seems unavoidable and 3D gestural interaction might be the most effective alternative for the current input facilities such as touchscreens. In this paper, we introduce a novel solution for real-time 3D gesture-based interaction by finding the best match from an extremely large gesture database. This database includes the images of various articulated hand gestures with the annotated 3D position/orientation parameters of the hand joints. Our unique matching algorithm is based on the hierarchical scoring of the low-level edge-orientation features between the query frames and database and retrieving the best match. Once the best match is found from the database in each moment, the pre-recorded 3D motion parameters can instantly be used for natural interaction. The proposed bare-hand interaction technology performs in real-time with high accuracy using an ordinary camera.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Yousefi, S.: 3D Photo Browsing for Future Mobile Devices. In: Proceedings of the ACMMM12, Nara, Japan, 29 October–2 November, 2012

    Google Scholar 

  2. Yousefi, S.: Enabling media technologies for mobile photo browsing. Licentiate thesis. Umea University, Department of Applied Physics and Electronics. Umea, Sweden (2012). ISBN 978-91-7459-426-3

    Google Scholar 

  3. Dorfmueller-Ulhaas, K., Schmalstieg, D.: Finger tracking for interaction in augmented environments. In: 2nd ACM/IEEE Int’l Symposium on Augmented Reality (2001)

    Google Scholar 

  4. Maggioni, C.: A novel gestural input device for virtual reality. In: Virtual Reality Annual International Symposium, pp. 118–124. IEEE (1993)

    Google Scholar 

  5. Hardenberg, C.V., Berard, F.: Bare-hand human-computer interaction. In: Proceedings of the 2001 Workshop on Perceptive User Interfaces. ACM International Conference Proceeding Series, Orlando, Florida, vol. 15, pp. 1–8 (2001)

    Google Scholar 

  6. Iwai, D., Sato, K.: Heat sensation in image creation with thermal vision. In: ACM SIGCHI International Conference on Advances in Computer Entertainment Technology (2005)

    Google Scholar 

  7. Kolsch, M., Turk, M.: Fast 2D hand tracking with flocks of features and multi-cue integration. In: Proceedings of the Computer Vision and Pattern Recognition Workshop (2004)

    Google Scholar 

  8. Ren, Z., Meng, J., Yuan, J., Zhang, Z.: Robust hand gesture recognition with kinect sensor. In: ACM Multimedia, pp. 759–760 (2011)

    Google Scholar 

  9. Erol, A., Bebis, G., Nicolescu, M., Boyle, R., Twombly, X.: Vision-based hand pose estimation: a review. Comput. Vis. Image Underst. 108, 52–73 (2007)

    Article  Google Scholar 

  10. Stenger, B., Thayananthan, A., Torr, P., Cipolla, R.: Model-based hand tracking using a hierarchical Bayesian filter. IEEE Trans. Pattern Anal. Mach. Intell. 28(9), 1372–1384 (2006)

    Article  Google Scholar 

  11. Yang, R., Sarkar, S.: Gesture recognition using hidden markov models from fragmented observations. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2006)

    Google Scholar 

  12. Bencheikh, M., Bouzenada, M., Batouche, M.: A new method of finger tracking applied to the magic board. In: Conference on Industrial Technology (2004)

    Google Scholar 

  13. Zhou, H., Ruan, Q.: Finger countour tracking based on model. In: Conference on Computers, Comunications, Control and Power Engineering, p. 503 (2002)

    Google Scholar 

  14. Arce, F., Valdez, J.: Accelerometer-based hand gesture recognition using artificial neural networks. In: Castillo, O., Kacprzyk, J., Pedrycz, W. (eds.) Soft Computing for Intelligent Control and Mobile Robotics. SCI, pp. 67–77. Springer, Heidelberg (2011)

    Google Scholar 

  15. Choi, J., Song, K., Lee, S.: Enabling a gesture-based numeric input on mobile phones. In: IEEE International Conference on Consumer Electronics (ICCE), pp. 151–152 (2011)

    Google Scholar 

  16. Hrst, W., Wezel, C.: Gesture-based interaction via finger tracking for mobile augmented reality. Multimedia Tools Appl. 62, 1–26 (2012)

    Google Scholar 

  17. Baldauf, M., Zambanini, S., Fröhlich, P., Reichl, P.: Markerless visual fingertip detection for natural mobile device interaction. Mobile HCI, pp. 539–544 (2011)

    Google Scholar 

  18. Lee, D., Lee, S.: Vision-based finger action recognition by angle detection and contour analysis. ETRI J. 33(3), 415–422 (2011)

    Article  Google Scholar 

  19. Hannuksela, J., Barnard, M., Sangi, P., Heikkil, J.: Camera-Based Motion Recognition for Mobile Interaction. ISRN Signal Processing (2011)

    Google Scholar 

  20. Hagbi, N., Bergig, O., El-Sana, J., Billinghurst, M.: Shape recognition and pose estimation for mobile augmented reality. In: 8th IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2009), IEEE Computer Press (2009)

    Google Scholar 

  21. Cao, Y., Wang, C., Zhang, L., Zhang, L.: Edgel index for large-scale sketch-based image search. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Colorado Springs, USA, pp. 761-768 (2011). ISBN 978-1-4577-0394-2

    Google Scholar 

  22. Eitz, M., Hildebrand, K., Boubekeur, T., Alexa, M.: Sketch-based image retrieval: benchmark and bag-of-features descriptors. IEEE Trans. Vis. Comput. Graphics 17(11), 1624–1636 (2011)

    Article  Google Scholar 

  23. Ikizler, N., Duygulu, P.: Human action recognition using distribution of oriented rectangular patches. In: Workshop on Human Motion, pp. 271–284 (2007)

    Google Scholar 

  24. Kondori, F.A., Liu, L.: 3D Active human motion estimation for biomedical applications. In: World congress on Medical Physics and Biomedical Engineering (WC2012). Beijing, China (2012)

    Google Scholar 

  25. Kondori, F.A.: Human motion analysis for creating immersive experience. Licentiate thesis, Department of Applied Physics and Electronics, Umea University, Sweden (2012). ISBN 9789174594164

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. KTH Royal Institute of Technology, 100 44, Stockholm, Sweden

    Shahrouz Yousefi & Haibo Li

Authors
  1. Shahrouz Yousefi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Haibo Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shahrouz Yousefi .

Editor information

Editors and Affiliations

  1. Center for Visual Information Technology, International Institute of Information Technology, Hyderabad, India

    C.V. Jawahar

  2. Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China

    Shiguang Shan

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Yousefi, S., Li, H. (2015). 3D Interaction Through a Real-Time Gesture Search Engine. In: Jawahar, C., Shan, S. (eds) Computer Vision - ACCV 2014 Workshops. ACCV 2014. Lecture Notes in Computer Science(), vol 9009. Springer, Cham. https://doi.org/10.1007/978-3-319-16631-5_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-16631-5_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-16630-8

  • Online ISBN: 978-3-319-16631-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation