Skip to main content
SpringerLink
Log in

User experience design for a smart-mirror-based personalized training system

  • 1173: Interaction in Immersive Experiences
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This paper describes the user experience (UX) design for a smart-mirror-based personalized training system which aims to help people live a healthy life. A number of researchers and companies have developed fitness systems that use a virtual coach which shows the user with actions they should perform. However such systems can be difficult to accurately follow the virtual guide’s motions and there are also limitations in the feedback provided to inform users of their correct body posture. This is because most systems are designed for users to simply watch and follow a character’s motions (poses) from a third person perspective. In our smart mirror-based system, users are able to follow the exercise-postures of a virtual professional trainer shown in a first person viewpoint and receive coaching through a real-time motion correction. This is based on a predefined database of the trainer’s postures gained from motion-capture technology, and it is personalized to the user’s body 3D model acquired through an instant one-time scanning process. In this paper, we report on the UX design of our system, mainly focusing on understandable visualization, intuitive interaction, attractive information representation and easily acceptable user scenarios. Through a series of user studies, we analyze and discuss user friendliness, information comprehension, and user satisfaction as they relate to our design. In addition, we also assess the similarity and effectiveness of the proposed system compared to traditional personalized training (PT) at a gym. Based on the implications, we discuss future research directions for improving the user experience of the smart-mirror-based PT system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Anderson F, Grossman T, Matejka J, Fitzmaurice G (2013) Youmove: enhancing movement training with an augmented reality mirror. In: Proceedings of the 26th annual ACM symposium on user interface software and technology, pp 311–320

  2. Beck M, Crié D (2018) I virtually try it... I want it! Virtual fitting room: a tool to increase on-line and off-line exploratory behavior, patronage and purchase intentions. J Retail Consum Serv 40:279–286

    Article  Google Scholar 

  3. Besserer D, Bäurle J, Nikic A, Honold F, Schüssel F, Weber M (2016) Fitmirror: a smart mirror for positive affect in everyday user morning routines. In: Proceedings of the workshop on multimodal analyses enabling artificial agents in human-machine interaction, pp 48–55

  4. Brooke J (1996) Sus: a quick and dirty usability scale. In: Jordan P, Thomas B, Weerdmeester B, McClelland I (eds) Usability evaluation in industry, lecture notes in computer science. Taylor and Francis, London, pp 189–194

  5. CBS This Morning (2018) This smart mirror puts a personal trainer in your reflection. https://www.youtube.com/watch?v=nSmTTZcpVGg. Accessed 2020

  6. Colantonio S, Coppini G, Germanese D, Giorgi D, Magrini M, Marraccini P, Martinelli M, Morales MA, Pascali MA, Raccichini G, Righi M, Salvetti O (2015) A smart mirror to promote a healthy lifestyle. Biosyst Eng 138:33–43

    Article  Google Scholar 

  7. Ehara J, Saito H (2006) Texture overlay for virtual clothing based on pca of silhouettes. In: Proceedings of the 5th IEEE and ACM international symposium on mixed and augmented reality, pp 139–142

  8. Eisert P, Fechteler P, Rurainsky J (2008) 3-d tracking of shoes for virtual mirror applications. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–6

  9. Fitts PM (1954) The information capacity of the human motor system in controlling the amplitude of movement. J Exp Psychol 47(6):381–391

    Article  Google Scholar 

  10. Hunter College Center for Health Technology (2019) https://nychealthtech.org/mirror-the-newest-trend-in-technology-based-home-fitness/. Accessed 2020

  11. Interface design rules, lecture hci, lecture-2 informatics, University of Edinburgh, 23rd (October 2007). https://www.interaction-design.org/literature. Accessed 2020

  12. Kinect for windows retail clothing scenario (2013) https://www.youtube.com/watch?v=Mr71jrkzWq8. Accessed 2020

  13. Kaiser E, Olwal A, McGee D, Benko H, Corradini A, Li X, Cohen P, Feiner S (2003) Mutual disambiguation of 3d multimodal interaction in augmented and virtual reality. In: Proceedings of the 5th international conference on multimodal interfaces, pp 12–19

  14. Kurtenbach G, Hulteen E (1990) Gestures in human-computer communications. Addison-Wesley Longman Publishing Co. Inc, Reading, pp 309–317

    Google Scholar 

  15. Lee M, Billinghurst M, Baek W, Green R, Woo W (2013) A usability study of multimodal input in an augmented reality environment. Virtual Real 17(4):293–305

    Article  Google Scholar 

  16. Lee GA, Wong J, Park HS, Choi JS, Park CJ, Billinghurst M (2015) User defined gestures for augmented virtual mirrors: a guessability study. In: Proceedings of the 33rd annual ACM conference extended abstracts on human factors in computing systems, pp 959–964

  17. Lee G A, Park HS, Billinghurst M (2019) Optical-reflection type 3d augmented reality mirrors. In: Proceedings of the 25th ACM symposium on virtual reality software and technology, pp 1–2

  18. Mango Mirror (2018) https://www.mangomirror.com. Accessed 2020

  19. Marquardt Z, Beira JA, Em N, Paiva I, Kox S (2012) Super mirror: a kinect interface for ballet dancers. In: Proceedings of the CHI ’12 extended abstracts on human factors in computing systems, pp 1619–1624

  20. Miotto R, Danieletto M, Scelza JR, Kidd BA, Dudley JT (2018) Reflecting healtH: SMART Mirrors for personalized medicine. NPJ Digit Med 1:62

    Article  Google Scholar 

  21. Mottura S, Greci L, Travaini E, Viganò G, Sacco M (2007) Magicmirror & footglove: a new system for the customized shoe try-on. In: The future of product development, pp 441–450

  22. Ng KC, Weyde T, Larkin O, Neubarth K, Koerselman T, Ong B (2007) 3d augmented mirror: a multimodal interface for string instrument learning and teaching with gesture support. In: Proceedings of the 9th international conference on multimodal interfaces, pp 339–345

  23. Nike+ Training (2012) https://news.nike.com/news/introducing-nike-kinect-training. Accessed 2020

  24. Nintendo Wii Fit (2008) https://www.youtube.com/watch?v=-Taruqvk30E. Accessed 2020

  25. PerfectGym (2018) Iot trends: how will they affect the future of gyms? https://www.perfectgym.com/en/blog/business/iot-trends-how-will-they-affect-future-gyms. Accessed 2020

  26. PhysioRoom (2018) Can poor posture make you more susceptible to injuries? https://www.physioroom.com/info/poor-posture-susceptible-injuries/. Accessed 2020

  27. Piumsomboon T, Clark A, Billinghurst M, Cockburn A (2013) User-defined gestures for augmented reality. In: Proceedings of the CHI ’13 extended abstracts on human factors in computing systems, pp 955–960

  28. Piumsomboon T, Altimira D, Kim H, Clark A, Lee G, Billinghurst M (2014) Grasp-shell vs gesture-speech: a comparison of direct and indirect natural interaction techniques in augmented reality. In: Proceedings of the IEEE international symposium on mixed and augmented reality, pp 73–82

  29. PlayStation®;Move (2010) https://www.youtube.com/watch?v=7Wuvia1-Mw8. Accessed 2020

  30. Poh MZ, McDuff D, Picard R (2011) A medical mirror for non-contact health monitoring. In: Proceedings of ACM SIGGRAPH 2011 emerging technologies

  31. Poupyrev I, Billinghurst M, Weghorst S, Ichikawa T (1996) The go-go interaction technique: non-linear mapping for direct manipulation in vr. In: Proceedings of the 9th annual ACM symposium on user interface software and technology, pp 79–80

  32. Saakes D, Yeo HS, Noh ST, Han G, Woo W (2016) Mirror mirror: an on-body t-shirt design system. In: Proceedings of the 2016 CHI conference on human factors in computing systems, pp 6058–6063

  33. Thompson WR (2016) Worldwide survey of fitness trends for 2017. ACSM’s Health Fitness J 20(6):8–17

    Article  Google Scholar 

  34. Tonal (2018) Intelligent home gym. https://www.tonal.com/ Accessed 2020

  35. Trajkova M, Ferati M (2015) Usability evaluation of kinect-based system for ballet movements. In: Proceedings, Part II, of the 4th international conference on design, user experience, and usability: users and interactions, vol 9187, pp 464–472

  36. Tullis T, Albert B (2013) Measuring the user experience: collecting, analyzing, and presenting usability metrics, 2nd edn. Elsevier Inc, Amsterdam

    Google Scholar 

  37. Vera L, Gimeno J, Coma I, Fernández M (2011) Augmented mirror: interactive augmented reality system based on kinect. In: Human-computer interaction—INTERACT 2011, pp 483–486

  38. Webb J, Ashley J (2012) Beginning kinect programming with the microsoft kinect SDK, 1 edn. Apress

  39. Wobbrock JO, Aung HH, Rothrock B, Myers BA (2005) Maximizing the guessability of symbolic input. In: Proceedings of the CHI ’05 extended abstracts on human factors in computing systems, pp 1869–1872

  40. Wobbrock JO, Morris MR, Wilson AD (2009) User-defined gestures for surface computing. In: Proceedings of the SIGCHI conference on human factors in computing systems, pp 1083–1092

  41. Yong B, Xu Z, Wang X, Cheng L, Li X, Wu X, Zhou Q (2018) Iot-based intelligent fitness system. J Parallel Distrib Comput 118:14–21

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No.15501-14-1016, Instant 3D object based Join & Joy content technology supporting simultaneous participation of users in remote places and enabling realistic experience and No.2017-0-01849, Development of Core Technology for Real-Time Image Composition in Unstructured In-outdoor Environment). We thank all those who helped setting up and running the user study. Special thanks should be given to Dr. Ho Won Kim, Dr. Kyu Sung Jo, Dr. Tae Joon Kim and Mrs. Ki Nam Kim for support of 3D motion tracking and 3D modeling generation.

Author information

Authors and Affiliations

  1. Electronics and Telecommunications Research Institute, Daejeon, Korea

    Hye Sun Park & Byung-Kuk Seo

  2. University of South Australia, Adelaide, Australia

    Gun A. Lee & Mark Billinghurst

Authors
  1. Hye Sun Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gun A. Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Byung-Kuk Seo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mark Billinghurst
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hye Sun Park.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Park, H.S., Lee, G.A., Seo, BK. et al. User experience design for a smart-mirror-based personalized training system. Multimed Tools Appl 80, 31159–31181 (2021). https://doi.org/10.1007/s11042-020-10148-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-10148-5

Keywords

Search

Navigation