Skip to main content
SpringerLink
Log in

Personalized gesture interactions for cyber-physical smart-home environments

智能家庭环境中的个性化手势交互

  • Research Paper
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

Abstract

A gesture-based interaction system for smart homes is a part of a complex cyber-physical environment, for which researchers and developers need to address major challenges in providing personalized gesture interactions. However, current research efforts have not tackled the problem of personalized gesture recognition that often involves user identification. To address this problem, we propose in this work a new event-driven service-oriented framework called gesture services for cyber-physical environments (GS-CPE) that extends the architecture of our previous work gesture profile for web services (GPWS). To provide user identification functionality, GS-CPE introduces a two-phase cascading gesture password recognition algorithm for gesture-based user identification using a two-phase cascading classifier with the hidden Markov model and the Golden Section Search, which achieves an accuracy rate of 96.2% with a small training dataset. To support personalized gesture interaction, an enhanced version of the Dynamic Time Warping algorithm with multiple gestural input sources and dynamic template adaptation support is implemented. Our experimental results demonstrate the performance of the algorithm can achieve an average accuracy rate of 98.5% in practical scenarios. Comparison results reveal that GS-CPE has faster response time and higher accuracy rate than other gesture interaction systems designed for smart-home environments.

创新点

1. 论文设计了一个支持智能家庭环境中个性化手势交互的集成框架GS-CPE,该框架同时集成了用户身份识别与个性化手势识别,通过服务的形式提供了一组简单易用的API。 2. 论文提出了一种可用于用户身份识别的手势识别算法。通过结合基于统计模型的HMM分类器及基于模板匹配的GSS分类器,该算法可在训练数据集有限的条件下较大幅度地提高手势密码的识别准确率。 3. 论文提出了一种融合多数据源的个性化手势识别算法,通过对比来自不同数据源的手势识别结果来消除对拒绝似然比的依赖,并针对实际环境设计了手势模板动态调节方法,从而提高了该算法的识别准确率。

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bernhaupt R, Obrist M, Weiss A, et al. Trends in the living room and beyond: results from ethnographic studies using creative and playful probing. ACM CIE, 2008, 6: 5

    Google Scholar 

  2. Panger G. Kinect in the kitchen: testing depth camera interactions in practical home environments. In: Proceedings of the CHI Extended Abstracts on Human Factors in Computing Systems. New York: ACM, 2012. 1985–1990

    Google Scholar 

  3. Pan G, Wu J H, Zhang D Q, et al. GeeAir: a universal multimodal remote control device for home appliances. Pers Ubiquitous Comput, 2010, 14: 723–735

    Article  Google Scholar 

  4. Vatavu R D. Point & click mediated interactions for large home entertainment displays. Multimed Tools Appl, 2012, 59: 113–128

    Article  Google Scholar 

  5. Kühnel C, Westermann T, Hemmert F, et al. I’m home: defining and evaluating a gesture set for smart-home control. Int J Hum-Comput Stud, 2011, 69: 693–704

    Article  Google Scholar 

  6. Vatavu R D. A comparative study of user-defined handheld vs. freehand gestures for home entertainment environments. J Ambient Intell Smart Environ, 2013, 5: 187–211

    Google Scholar 

  7. Vatavu R D, Zaiti I A. Leap gestures for TV: insights from an elicitation study. In: Proceedings of the ACM International Conference on Interactive Experiences for TV and Online Video. New York: ACM, 2014. 131–138

    Google Scholar 

  8. Li W, Lee Y H, Tsai W T, et al. Service-oriented smart home applications: composition, code generation, deployment, and execution. Serv Oriented Comput Appl, 2012, 6: 65–79

    Article  Google Scholar 

  9. Vatavu R D, Chera C M, Tsai W T. Gesture profile for web services: an event-driven architecture to support gestural interfaces for smart environments. In: Ambient Intelligence. Berlin: Springer-Verlag, 2012. 161–176

    Chapter  Google Scholar 

  10. Lou Y H, Wu W J. A real-time personalized gesture interaction system using Wii remote and Kinect for tiled-display environment. In: Proceedings of the International Conference on Software Engineering and Knowledge Engineering. Skokie: KSI, 2013. 131–136

    Google Scholar 

  11. Zhang H K, Wu W J, Lou Y H. A personalized gesture interaction system with user identification using Kinect. In: PRICAI 2014: Trends in Artificial Intelligence. Berlin: Springer, 2014. 614–626

    Google Scholar 

  12. Vatavu R D. User-defined gestures for free-hand TV control. In: Proceedings of the 10th European Conference on Interactive TV and Video. New York: ACM, 2012. 45–48

    Chapter  Google Scholar 

  13. Wobbrock J O, Morris M R, Wilson A D, et al. User-defined gestures for surface computing. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. New York: ACM, 2009. 1083–1092

    Google Scholar 

  14. Vatavu R D, Wobbrock J O. Formalizing agreement analysis for elicitation studies: new measures, significance test, and toolkit. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. New York: ACM, 2015. 1325–1334

    Google Scholar 

  15. Wobbrock J O, Aung H H, Brandon R, et al. Maximizing the guessability of symbolic input. In: Proceedings of the CHI Extended Abstracts on Human Factors in Computing Systems. New York: ACM, 2005. 1869–1872

    Google Scholar 

  16. Lou Y H, Yao T, Chen Y Q, et al. A novel scheme of ROI detection and transcoding for mobile devices in high-definition videoconferencing. In: Proceedings of the 5th Workshop on Mobile Video. New York: ACM, 2013. 31–36

    Google Scholar 

  17. Wang Y W, Yang C, Wu X, et al. Kinect based dynamic hand gesture recognition algorithm research. In: Proceedings of the 4th International Conference on Intelligent Human-Machine Systems and Cybernetics, Nanchang, 2012. 274–279

    Google Scholar 

  18. Zhu H M, Pun C M. Real-time hand gesture recognition form depth image sequences. In: Proceedings of the 9th International Conference on Computer Graphics, Imaging and Visualization, Hsinchu, 2012. 49–52

    Google Scholar 

  19. Moni M A, Shawkat Ali A B M. HMM based hand gesture recognition: a review on techniques and approaches. In: Proceedings of the 2nd IEEE International Conference on Computer Science and Information Technology, Beijing, 2009. 433–437

    Google Scholar 

  20. Kiefer J. Sequential minimax search for a maximum. Proc American Math Soc, 1953, 4: 502–506

    Article  MathSciNet  MATH  Google Scholar 

  21. Vatavu R D, Anthony L, Wobbrock J O. Gestures as point clouds: a $P recognizer for user interface prototypes. In: Proceedings of the 14th ACM International Conference on Multimodal Interaction. New York: ACM, 2012. 273–280

    Google Scholar 

  22. Myers C S, Rabiner L R. A comparative study of several dynamic time-warping algorithms for connected word recognition. Bell Syst Tech J, 1981, 60: 1389–1409

    Article  Google Scholar 

  23. Wobbrock J O, Wilson A D, Li Y. Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes. In: Proceedings of the 20th Annual ACM Symposium on User Interface Software and Technology. New York: ACM, 2007. 159–168

    Chapter  Google Scholar 

  24. Carmona J M, Climent J. A performance evaluation of HMM and DTW for gesture recognition. In: Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications. Berlin: Springer-Verlag, 2012. 236–243

    Chapter  Google Scholar 

  25. Liu J Y, Zhong L, Wickramasuriya J, et al. uWave: accelerometer-based personalized gesture recognition and its applications. Pervasive Mob Comput, 2009, 5: 657–675

    Article  Google Scholar 

  26. Reyes M, Dominguez G, Escalera S. Feature weighting in dynamic time warping for gesture recognition in depth data. In: Proceedings of the IEEE International Conference on Computer Vision Workshops. Piscataway: IEEE, 2011. 1182–1188

    Google Scholar 

  27. Liu X, Mu Y, Zhang D, et al. Large-scale unsupervised hashing with shared structure learning. IEEE Trans Cybern, 2015, 45: 1811–1822

    Article  Google Scholar 

  28. Liu X, Deng C, Lang B, et al. Query-adaptive reciprocal hash tables for nearest neighbor search. IEEE Trans Image Process, 2015, 25: 907–919

    Article  MathSciNet  Google Scholar 

  29. Chen M Y, AlRegib G, Juang B H. 6DMG: a new 6D motion gesture database. In: Proceedings of the 3rd Multimedia Systems Conference. New York: ACM, 2012. 83–88

    Google Scholar 

  30. Mitra S, Acharya T. Gesture recognition: a survey. IEEE Trans Syst Man Cybern Part C-Appl Rev, 2007, 37: 311–324

    Article  Google Scholar 

  31. Ruiz J, Li Y, Lank E. User-defined motion gestures for mobile interaction. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. New York: ACM, 2011. 197–206

    Google Scholar 

  32. Schlömer T, Poppinga B, Henze N, et al. Gesture recognition with a Wii controller. In: Proceedings of the 2nd International Conference on Tangible and Embedded Interaction. New York: ACM, 2008. 11–14

    Chapter  Google Scholar 

  33. Vatavu R D. Nomadic gestures: a technique for reusing gesture commands for frequent ambient interactions. J Ambient Intell Smart Environ, 2012, 4: 79–93

    Google Scholar 

  34. Pu Q F, Gupta S, Gollakota S, et al. Whole-home gesture recognition using wireless signals. In: Proceedings of the 19th Annual International Conference on Mobile Computing & Networking. New York: ACM, 2013. 27–38

    Chapter  Google Scholar 

  35. van Seghbroeck G, Verstichel S, de Truck F, et al. WS-Gesture: a gesture-based state-aware control framework. In: Proceedings of the IEEE International Conference on Service-Oriented Computing and Applications. Piscataway: IEEE, 2010. 1–8

    Google Scholar 

  36. Zheng Y W, Sheng H, Zhang B C, et al. Weight-based sparse coding for multi-shot person re-identification. Sci China Inf Sci, 2015, 58: 100104

    Google Scholar 

  37. Hayashi E, Maas M, Hong J I. Wave to me: user identification using body lengths and natural gestures. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. New York: ACM, 2014. 3453–3462

    Google Scholar 

  38. Wang D X, Xiong Z H, Zhang M. An application oriented and shape feature based multi-touch gesture description and recognition method. Multimed Tools Appl, 2012, 58: 497–519

    Article  Google Scholar 

  39. Chera C M, Tsai W T, Vatavu R D. Gesture ontology for informing Service-oriented architecture. In: Proceedings of IEEE International Symposium on Intelligent Control. Piscataway: IEEE, 2012. 1184–1189

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. State Key Laboratory of Software Development Environment, Beihang University, Beijing, 100191, China

    Yihua Lou, Wenjun Wu & Wei-Tek Tsai

  2. MintViz Research Lab, Integrated Center for Research, Development, and Innovation in Advanced Materials, Nanotechnologies, and Distributed Systems for Fabrication and Control (MANSiD) & Department of Computer Science, University Stefan cel Mare of Suceava, Suceava, 720229, Romania

    Radu-Daniel Vatavu

Authors
  1. Yihua Lou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wenjun Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Radu-Daniel Vatavu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wei-Tek Tsai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wenjun Wu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lou, Y., Wu, W., Vatavu, RD. et al. Personalized gesture interactions for cyber-physical smart-home environments. Sci. China Inf. Sci. 60, 072104 (2017). https://doi.org/10.1007/s11432-015-1014-7

Download citation

  • Received:

  • Revised:

  • Published:

  • DOI: https://doi.org/10.1007/s11432-015-1014-7

Keywords

关键词

--

Search

Navigation