Abstract
In this paper, we present a handheld device called GeeAir for remotely controlling home appliances via a mixed modality of speech, gesture, joystick, button, and light. This solution is superior to the existing universal remote controllers in that it can be used by the users with physical and vision impairments in a natural manner. By combining diverse interaction techniques in a single device, the GeeAir enables different user groups to control home appliances effectively, satisfying even the unmet needs of physically and vision-impaired users while maintaining high usability and reliability. The experiments demonstrate that the GeeAir prototype achieves prominent performance through standardizing a small set of verbal and gesture commands and introducing the feedback mechanisms.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Campbell LW (1997) A more ‘universal’ remote control. http://web.media.mit.edu/~lieber/Teaching/Collab97/Collab-Projects/remote.html
http://www.oneforall.co.uk/en_UK/product/1/universal-remotes/3/advanced/25/digital-12
http://www.logitech.com/index.cfm/remotes/universal_remotes/devices/3898&cl=us,en
Lee L, Johnson T (2006) URCousin: universal remote control user interface. In: Proceedings of the Human Interface Technologies Conference, April 2006
Niezen G, Hancke GP (2008) Gesture recognition as ubiquitous input for mobile phones. International Workshop on Devices that Alter Perception (DAP08), conjunction with Ubicomp08, 2008
Bolt RA (1980) Put-that-there: voice and gesture at the graphics interface, SIGGRAPH’80, pp 262–270
Machate J, Burmester M, Bekiaris E (1997) Towards an intelligent multimodal and multimedia user interface providing a new dimension of natural HMI in the teleoperation of all home appliances by E&D users, 6th International Conference Man–Machine Interactions Intelligent Systems in Business, Montpellier, May 1997, pp 226–229
Machate J (1999) Being natural—on the use of multimodal interaction concepts in smart homes. In: Proceedings of the HCI International ‘99, pp 937–941
Wilson A, Oliver N (2003) Gwindows: robust stereo vision for gesture-. based control of windows. In: Proceedings of the 5th international conference on multimodal interfaces, New York, NY, USA, pp 211–218
Krum DM, Omoteso O, Ribarsky W, Starner T, Hodges LF (2002) Speech and Gesture Multimodal Control of a Whole Earth 3D Visualization Environment. In: Proceedings of Symposium on Data Visualization, Barcelona, Spain, pp 195–200
Starner T, Auxier J, Ashbrook D, Gandy M (2000) The gesture pendant: a self-illuminating, wearable, infrared computer vision system for home automation control and medical monitoring. International Symposium on Wearable Computers (ISWC00), pp 87–95
Kela J, Korpipaa P, Mantyjarvi J, Kallio S, Savino G, Jozzo L, Marca D (2006) Accelerometer-based gesture control for a design environment, Personal Ubiquitous Computing, 10:285–299
Wu J, Pan G, Li S, Zhang D (2009) Gesture Recognition with a 3D Accelerometer. The Sixth International Conference on Ubiquitous Intelligence and Computing (UIC-09), Brisbane, Australia, 7–9 July, 2009
Rabiner L, Levinson L (1981) Isolated and connected word recognition—theory and selected applications. IEEE Trans Commun 29(5):621–659
Rabiner LR (1989) A tutorial on hidden markov models and selected applications in speech recognition. Proc IEEE 77:257–286
Lee C-H, Lin C-H, Juang B-H (1991) A study on speaker adaptation of the parameters of continuous density hidden Markov models. IEEE Trans Signal Process 39(4):806–814
Davis SB, Mermelstein P (1980) Comparison of parametric representation for monosyllabic word recognition in continuously spoken sentences. IEEE Trans Acoust Speech Signal Process 28:357–366
Mitra S, Acharya T (2007) Acharya: gesture recognition: a survey. IEEE Trans Syst Man Cybern Part C 37(3):311–324
Schlömer T, Poppinga B, Henze N, Boll S (2008) Gesture Recognition with a Wii Controller. International Conference on Tangible and Embedded Interaction (TEI’08), pp 11–14, Bonn Germany, Feb. 18–20, 2008
Mäntylä V-M (2001) Discrete hidden markov models with application to isolated user-dependent hand gesture recognition. VTT publications
Liu J, Wang Z, Zhong L, Wickramasuriya J, Vasudevan V (2009) uWave: accelerometer-based personalized gesture recognition and its applications. IEEE PerCom’09, 2009
Mäntyjärvi J, Kela J, Korpipää P, Kallio S (2004) Enabling fast and effortless customization in accelerometer based gesture interaction. Proceedings of the 3rd International Conference on Mobile and Ubiquitous Multimedia (MUM’04), ACM Press, 25–31, October 27–29
Christanini J, Taylor JS (2000) An introduction to support vector machines and other kernel-based methods. Cambridge University Press, Cambridge
Frigo M, Johnson SG (2005) The design and implementation of FFTW3. Proc IEEE 93(2)
Joachims T (1999) Making large-scale SVM learning practical. Advances in kernel methods—support vector learning. In: Schöllkopf B, Burges C, Smola A (ed) MIT-Press
Quinlan JR (1996) Improved use of continuous attributes in c4.5. J Artif Intell Res 4:77–90
Acknowledgments
The authors would like to thank the comments and suggestions from the anonymous reviewers. The laboratory students’ participation in the experiments is greatly appreciated. This work is supported in part by the National High-Tech Research and Development (863) Program of China (No. 2008AA01Z132, 2009AA011900), the Natural Science Fund of China (No. 60525202, 60533040), and the France ICT-Asia I-CROSS program. Dr. Shijian Li is corresponding author.
Author information
Authors and Affiliations
Corresponding authors
Rights and permissions
About this article
Cite this article
Pan, G., Wu, J., Zhang, D. et al. GeeAir: a universal multimodal remote control device for home appliances. Pers Ubiquit Comput 14, 723–735 (2010). https://doi.org/10.1007/s00779-010-0287-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00779-010-0287-7