Skip to main content
SpringerLink
Log in

Low complexity head tracking on portable android devices for real time message composition

  • Original Paper
  • Published:
Journal on Multimodal User Interfaces Aims and scope Submit manuscript

Abstract

For the people who are totally or partially unable to move or control their limbs and cannot rely on verbal communication, it is very important to obtain an interface capable of interpreting their limited voluntary movements, in order to allow communications with friends, relatives and care providers, or to send commands to a system. This paper presents a real time software application for disabled subjects, suffering from both motor and speech impairments, that provides message composition and speech synthesis functionalities based on face detection and head tracking. The proposed application runs on portable devices equipped with Android Operating System, and relies upon the O.S.’s native computer vision primitives, without resorting to any external software library. This way, the available camera sensors are exploited, and the device computational requirements accomplished. Experimental results show the effectiveness of the application in recognizing the user’s movements, and the reliability of the message composition and speech synthesis functionalities.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Mads B, Gimpel G, Hedman J (2009) The user experience of smart phones: a consumption values approach. In: Proc. 8th Global Mobility Roundtable, GMR, Cairo

  2. World Health Organization (2009) Dept. of Violence and Injury Prevention, Global status report on road safety: time for action, World Health Organization

  3. Wood E, Willoughby T, Rushing A, Bechtel L, Gilbert J (2005) Use of computer input devices by older adults. J Appl Gerontol 24(5):419–438

    Article  Google Scholar 

  4. Spinsante S, Gambi E (2012) Remote health monitoring by OSGi technology and digital TV integration. IEEE Trans Consum Electron 58(4):1434–1441

    Article  Google Scholar 

  5. Mertens A, Koch-Korfges D, Jochems N, Schlick CM (2010) Touchscreen-based input technique for people with intention tremor. In: Proc. of 3rd Conference on Human System Interactions (HSI). pp 236–240

  6. Spinsante S, Gambi E (2012) Home automation systems control by head tracking in AAL applications. In: Proc. of IEEE 1st ESTEL Conference, Rome, April 2012

  7. Ren J, Rahman M, Kehtarnavaz N, Estevez L (2010) Real-time head pose estimation on mobile platforms. J Syst, Cybern Inf 8(3):56–62

  8. Lupu RG, Ungureanu F, Bozomitu RG (2012) Mobile embedded system for human computer communication in assistive technology. In: Proc. IEEE International Conference on Intelligent Computer Communication and Processing (ICCP), 2012, pp 209–212, Aug 30 2012–Sept

  9. Takahashi K, Mitsukura Y (2013) Head pose tracking system using a mobile device. In: Proc. 2013 IEEE Int. Symposium on Robot and Human Interactive Communication, pp 461–466, 26–29 Aug 2013

  10. Montanini L, Cippitelli E, Gambi E, Spinsante S (2014) Real time message composition through head movements on portable Android devices. In: Proc. IEEE 2014 Int. Conf. on Consumer Electronics, pp 526–527, Jan 10–13, 2014, Las Vegas, USA

  11. Haraikawa T, Oyamada A, Ito H, Oikawa S, Fukui Y (2014) A cost-effective solution for realizing talking appliances for the visually impaired. In: 2014 IEEE International Conference on Consumer Electronics (ICCE), pp 317–318, 10–13 Jan 2014

  12. Ivanov R (2014) Blind-environment interaction through voice augmented objects. J Multimodal User Interfaces 8(4):345–365

    Article  Google Scholar 

  13. Batliner A, Hacker C, Nth E (2008) To talk or not to talk with a computer. J Multimodal User Interfaces 2(3–4):171–186

    Article  Google Scholar 

  14. Chandramouli C, Agarwal V (2009) Speech Recognition based Computer Keyboard Replacement for the Quadriplegics, Paraplegics, Paralytics and Amputees, IEEE International Workshop on Medical Measurements and Applications, pp 241–245, 29–30 May 2009

  15. Kathirvelan J, Anilkumar R, Alex ZC, Fazul A (2012) Development of low cost automatic wheelchair controlled by oral commands using standalone controlling system, IEEE International Conference on Computational Intelligence & Computing Research (ICCIC), pp 1–4, 18–20 Dec 2012

  16. McFarland DJ, Wolpaw JR (2011) Brain-computer interfaces for communication and control. Commun ACM 54(5):60–66

    Article  Google Scholar 

  17. Donegan M, Cotmore S, Holmqvist E, Buchholz M, Lundalv M, Pasian V, Farinetti L, Corno F (2009) Deliverable 3.6: Final User Trials Report. Communication by Gaze Interaction (COGAIN) IST-2003-511598. http://wiki.cogain.org/index.php/File:COGAIN-D3.6

  18. Beukelman DR, Yorkston KM, Reichle J (2000) Augmentative and alternative communication for adults with acquired neurologic disorders. Paul H Brookes, Baltimore, MD

  19. Kumar N, Kohlbecher S, Schneider E (2009) A novel approach to video-based pupil tracking. In: Proc. IEEE International Conference on Systems, Man and Cybernetics, 2009. pp 1255–1262, 11–14 Oct 2009

  20. Rantanen V, Vanhala T, Tuisku O, Niemenlehto P-H, Verho J, Surakka V, Juhola M, Lekkala J (2011) A wearable, wireless gaze tracker with integrated selection command source for human-computer interaction, IEEE Trans. On Inf Tech Biomed 15(5):795–801

    Article  Google Scholar 

  21. Lupu RG, Ungureanu F, Siriteanu V (2013) Eye tracking mouse for human computer interaction. In: Proc. 2013 E-Health and Bioengineering Conference, pp 1–4, 21–23 Nov 2013

  22. La Cascia M, Sclaroff S (1999) Fast, reliable head tracking under varying illumination. In: Proc. of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol 1

  23. Malassiotis S, Strintzis MG (2003) Real-time head tracking and 3D pose estimation from range data. In: Proc. of International Conference on Image Processing, vol 2, pp 859–862

  24. Yun F, Huang TS (2007) hMouse: head tracking driven virtual computer mouse. In: Proc. of IEEE Workshop on Applications of Computer Vision

  25. Morency LP, Sidner C, Lee C, Darrell T (2005) Contextual recognition of head gestures. In: Proceedings of the International Conference on Multimodal Interactions, Oct 46, 2005, Trento, Italy

  26. Song Y, Luo Y, Lin J (2011) Detection of movements of head and mouth to provide computer access for disabled, 2011 International Conference on Technologies and Applications of Artificial Intelligence (TAAI), pp 223–226, 11–13 Nov 2011

  27. Bastos-Filho T, Ferreira A, Cavalieri D, Silva R, Muller S, Perez E (2013) Multi-modal interface for communication operated by eye blinks, eye movements, head movements, blowing/sucking and brain waves, 2013 ISSNIP Biosignals and Biorobotics Conference (BRC), pp 1–6, 18–20 Feb 2013

  28. Morency LP, Rahimi A, Darrell T (2003) Adaptive view-based appearance model. In: Proceedings IEEE Conf. on Computer Vision and Pattern Recognition

  29. Face Tracking on Android O.S. demo video clip, available at: http://youtu.be/racDJvUJKL4

Download references

Author information

Authors and Affiliations

  1. Dipartimento di Ingegneria dell’Informazione, Università Politecnica delle Marche, Via Brecce Bianche 12, Ancona, Italy

    Laura Montanini, Enea Cippitelli, Ennio Gambi & Susanna Spinsante

Authors
  1. Laura Montanini
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Enea Cippitelli
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ennio Gambi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Susanna Spinsante
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Laura Montanini.

Electronic supplementary material

Below is the link to the electronic supplementary material.

12193_2015_174_MOESM1_ESM.mpg

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Montanini, L., Cippitelli, E., Gambi, E. et al. Low complexity head tracking on portable android devices for real time message composition. J Multimodal User Interfaces 9, 141–151 (2015). https://doi.org/10.1007/s12193-015-0174-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12193-015-0174-7

Keywords

Search

Navigation