Skip to main content
SpringerLink
Log in

How Companion-Technology can Enhance a Multi-Screen Television Experience: A Test Bed for Adaptive Multimodal Interaction in Domestic Environments

  • Technical Contribution
  • Published:
KI - Künstliche Intelligenz Aims and scope Submit manuscript

Abstract

This article deals with a novel multi-screen interactive TV setup (smarTVision) and its enhancement through Companion-Technology. Due to their flexibility and the variety of interaction options, such multi-screen scenarios are hardly intuitive for the user. While research known so far focuses on technology and features, the user itself is often not considered adequately. Companion-Technology has the potential of making such interfaces really user-friendly. Building upon smarTVision, it’s extension via concepts of Companion-Technology is envisioned. This combination represents a versatile test bed that not only can be used for evaluating usefulness of Companion-Technology in a TV scenario, but can also serve to evaluate Companion-Systems in general.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Basapur S, Harboe G, Mandalia H, Novak A, Vuong V, Metcalf C (2011) Field Trial of a Dual Device User Experience for iTV. In: Proceedings of the EuroITV ’11. doi:10.1145/2000119.2000145

  2. Basapur S, Mandalia H, Chaysinh S, Lee Y, Venkitaraman N, Metcalf C (20102) FANFEEDS: Evaluation of Socially Generated Information Feed on Second Screen As a TV Show Companion. In: Proceedings of the EuroiTV ’12. doi:10.1145/2325616.2325636

  3. Cauchard JR, Fraser M, Han T, Subramanian S (2012) Steerable projection: exploring alignment in interactive mobile displays. Pers Ubiquitous Comput. doi:10.1007/s00779-011-0375-3

  4. Cesar P, Bulterman DCA, Jansen AJ (2008) Usages of the secondary screen in an interactive television environment: control, enrich, share, and transfer television content. In: Changing television environments. LNCS. Springer, Heidelberg . DOI 10.1007/978-3-540-69478-6\_22

  5. Courtois C, D’heer E (2012) Second screen applications and tablet users: constellation, awareness, experience, and interest. In: Proceedings of the EuroiTV ’12. doi:10.1145/2325616.2325646

  6. Courtois C, Schuurman D, De Marez L (2011) Triple screen viewing practices: diversification or compartmentalization? In: Proceedings of the EuroITV ’11. ACM, New York, pp 75–78. doi:10.1145/2000119.2000132

  7. Coutaz J (2007) Meta-user interfaces for ambient spaces. In: Coninx K, Luyten K, Schneider K (eds) Task models and diagrams for UI design. LNCS, vol 4385. Springer, Heidelberg, pp 1–15. doi:10.1007/978-3-540-70816-2_1

  8. Coutaz J, Nigay L, Salber D, Blandford A, May J, Young RM (1995) Four easy pieces for assessing the usability of multimodal interaction: the CARE properties. In: Proceedings of INTERACT95, pp 115–120

  9. Dey AK, Abowd GD (1999) Towards a better understanding of context and context-awareness. In: HUC ’99: Proceedings of the 1st int. symposium on Handheld and Ubiquitous Computing. Springer, Heidelberg, pp 304–307

  10. Glodek M, Honold F, Geier T, Krell G, Nothdurft F, Reuter S, Schüssel F, Hörnle T, Dietmayer K, Minker W, Biundo S, Weber M, Palm G, Schwenker F (2015) Fusion paradigms in cognitive technical systems for human-computer interaction. Neurocomputing 161:17–37. doi:10.1016/j.neucom.2015.01.076

    Article  Google Scholar 

  11. Gugenheimer J, Knierim P, Seifert J, Rukzio E (2014) Ubibeam: An interactive projector-camera system for domestic deployment. In: Proceedings of the ITS ’14. ACM, New York, pp 305–310. doi:10.1145/2669485.2669537

  12. Häkkilä JR, Posti M, Schneegass S, Alt F, Gultekin K, Schmidt A (2014) Let me catch this! Experiencing interactive 3D cinema through collecting content with a mobile phone. In: Proceedings of the CHI ’14). doi:10.1145/2556288.2557187

  13. Harboe G, Metcalf CJ, Bentley F, Tullio J, Massey N, Romano G (2008) Ambient social Tv: drawing people into a shared experience. In: Proceedings of the CHI ’08. doi:10.1145/1357054.1357056

  14. Hardy J, Alexander J (2012) Toolkit support for interactive projected displays. In: Proceedings of the MUM ’12. doi:10.1145/2406367.2406419

  15. Honold F, Bercher P, Richter F, Nothdurft F, Geier T, Barth R, Hörnle T, Schüssel F, Reuter S, Rau M, Bertrand G, Seegebarth B, Kurzok P, Schattenberg B, Minker W, Weber M, Biundo S (2014) Companion-technology: towards user- and situation-adaptive functionality of technical systems. In: IE’14: Proceedings of the 10th international conference on intelligent environments, pp 378–381. IEEE (2014). doi:10.1109/IE.2014.60. Video: http://companion.informatik.uni-ulm.de/ie2014/companion-system.mp4

  16. Honold F, Schüssel F, Weber M (2012) Adaptive probabilistic fission for multimodal systems. In: Proceedings of the 24th Australian computer-human interaction conference, OzCHI ’12. ACM, New York, pp 222–231. doi:10.1145/2414536.2414575

  17. Honold F, Schüssel F, Weber M (2014) The automated interplay of multimodal fission and fusion in adaptive HCI. In: IE’14: Proceedings of the 10th international conf.erenceon intelligent environments, pp 170–177. doi:10.1109/IE.2014.32

  18. Jones B, Sodhi R, Murdock M, Mehra R, Benko H, Wilson A, Ofek E, MacIntyre B, Raghuvanshi N, Shapira L (2014) Roomalive: magical experiences enabled by scalable, adaptive projector-camera units. In: Proceedings of the UIST ’14. doi:10.1145/2642918.2647383

  19. Jones BR, Benko H, Ofek E, Wilson AD (2013) IllumiRoom: peripheral projected illusions for interactive experiences. In: ACM SIGGRAPH 2013 emerging technologies. doi:10.1145/2503368.2503375

  20. Joyent Inc (2014) Node js. http://nodejs.org/. Access:ed 29 May 2015

  21. Kane SK, Avrahami D, Wobbrock JO, Harrison B, Rea AD, Philipose M, LaMarca A (2009) Bonfire: a nomadic system for hybrid laptop-tabletop interaction. In: Proceedings of the UIST ’09. doi:10.1145/1622176.1622202

  22. Lalanne D, Nigay L, Palanque P, Robinson P, Vanderdonckt J, Ladry JF (2009) Fusion engines for multimodal input: a survey. In: Proceedings of the 2009 international conference on multimodal interfaces, ICMI-MLMI ’09. ACM, New York, pp.153–160. doi:10.1145/1647314.1647343

  23. aViola JJ Jr, Buchanan S, Pittman C (2014) Multimodal input for perceptual user interfaces, chap. 9. Wiley, New York, pp 285–312. doi:10.1002/9781118706237.ch9

    Google Scholar 

  24. Miller LJ (1995) Family togetherness and the suburban ideal. Sociol Forum 10(3):393–418

    Article  Google Scholar 

  25. Myers BA (2001) Using handhelds and pcs together. Commun ACM. doi:10.1145/384150.384159

  26. Nigay L, Coutaz J (1995) A generic platform for addressing the multimodal challenge. In: CHI ’95: Proceedings of the SIGCHI conference on human factors in computing systems. ACM Press/Addison-Wesley Publishing Co., New York, pp 98–105. doi:10.1145/223904.223917

  27. Pinhanez C (2001) The everywhere displays projector: a device to create ubiquitous graphical interfaces. In: Ubicomp 2001: ubiquitous computing. LNCS. Springer, New York

  28. Robertson S, Wharton C, Ashworth C, Franzke M (1996) Dual Device User Interface Design: PDAs and Interactive Television. In: Proceedings of the CHI ’96. doi:10.1145/238386.238408

  29. Roscher D, Blumendorf M, Albayrak S (2009) Using meta user interfaces to control multimodal interaction in smart environments. In: Meixner G, Görlich D, Breiner K, Hußmann H, Pleuß A, Sauer S, den Bergh JV (eds) Proceedings of the IUI’09 workshop on model driven development of advanced user interfaces, CEUR Workshop Proceedings ISSN 1613-0073, vol 439

  30. Schmidt D, Ramakers R, Pedersen EW, Jasper J, Köhler S, Pohl A., Rantzsch H, Rau A, Schmidt P, Sterz C, Yurchenko Y, Baudisch P (2014) Kickables: tangibles for feet. In: Proceedings of the CHI ’14. doi:10.1145/2556288.2557016

  31. Schüssel F, Honold F, Weber M (2013) Using the transferable belief model for multimodal input fusion in companion systems. In: Schwenker F, Scherer S, Morency LP (eds) Multimodal pattern recognition of social signals in HCI. LNCS, vol 7742. Springer, New York, pp 100–115. doi:10.1007/978-3-642-37081-6_12

  32. Schüssel F, Honold F, Weber M, Schmidt M, Bubalo N, Huckauf A (2014) Multimodal interaction history and its use in error detection and recovery. In: Proceedings of the 16th ACM international conference on multimodal interaction, ICMI ’14. ACM, New York, pp 164–171. doi:10.1145/2663204.2663255

  33. Seifert J (2015) Mobile mediated interaction with pervasive displays. Ph.D. thesis, Ulm University,Ulm

  34. Sugimoto M, Hosoi K, Hashizume H (2004) Caretta: a system for supporting face-to-face collaboration by integrating personal and shared spaces. In: Proceedings of the CHI ’04. doi10.1145/985692.985698

  35. Vatavu RD (2013) Designing Gestural Interfaces for the Interactive TV. In: Proceedings of the EuroITV ’13. doi:10.1145/2465958.2465981

  36. Wendemuth A, Biundo S (2012) A companion technology for cognitive technical systems. In: Esposito A, Esposito AM, Vinciarelli A, Hoffmann R, Müller VC (eds) Cognitive behavioural systems. LNCS, vol 7403. Springer, New York, pp 89–103. doi:10.1007/978-3-642-34584-5_7

  37. Winkler C, Löchtefeld M, Dobbelstein D, Krüger A, Rukzio E (2014) SurfacePhone: a mobile projection device for single- and multiuser everywhere tabletop interaction. In: Proceedings of the CHI ’14. doi:10.1145/2556288.2557075

Download references

Acknowledgments

This work was supported by the Transregional Collaborative Research Center SFB/TRR 62 “Companion-Technology for Cognitive Technical Systems”, which is funded by the German Research Foundation (DFG).

Author information

Authors and Affiliations

  1. Institute of Media Informatics, Ulm University, James-Franck-Ring, 89081, Ulm, Germany

    Jan Gugenheimer, Frank Honold, Dennis Wolf, Felix Schüssel, Julian Seifert, Michael Weber & Enrico Rukzio

Authors
  1. Jan Gugenheimer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Frank Honold
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dennis Wolf
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Felix Schüssel
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Julian Seifert
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Michael Weber
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Enrico Rukzio
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jan Gugenheimer.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gugenheimer, J., Honold, F., Wolf, D. et al. How Companion-Technology can Enhance a Multi-Screen Television Experience: A Test Bed for Adaptive Multimodal Interaction in Domestic Environments. Künstl Intell 30, 37–44 (2016). https://doi.org/10.1007/s13218-015-0395-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13218-015-0395-7

Keywords

Search

Navigation