Skip to main content
SpringerLink
Log in

Lifelogging meets alternate and cross-realities: an investigation into broadcasting personal visual realities to remote audiences

  • 1161: Multimedia Alternate Realities
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

We address in this work the topic of broadcasting one’s visual reality, captured by the video cameras embedded in mobile and wearable devices, to a remote audience. We discuss several designs of such life broadcasting systems, which we position at the intersection of lifelogging, Alternate Reality, and Cross-Reality technology. To this end, we introduce the “Alternate Reality Broadcast-Time” matrix for the broadcasting and consumption of alternate realities, in which designs of systems that implement sharing and consumption of personal visual realities can be positioned, characterized, and compared. These design options range from simple video streaming over the web using conventional video protocols to mediated and augmented reality, to audio narration and vibrotactile rendering of concepts automatically detected from video captured by wearable cameras. To demonstrate the usefulness of our broadcast-time matrix, we describe three prototypes implementing lifelogging, concept recognition from video, augmented and mediated vision, and vibrotactile feedback. Our contributions open the way toward new applications that blend lifelogging, consumption of multimedia alternate realities, and XR technology to empower users with richer opportunities for self-expression and new means to connect with the followers of events in their lives.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. In this work, we use the acronym AltR to refer to Alternate Reality, since AR is the conventional acronym used for Augmented Reality.

  2. https://support.parrot.com/us/support/products/parrot-mambo

References

  1. Abowd GD, Gauger M, Lachenmann A (2003) The family video archive: An annotation and browsing environment for home movies. In: Proceedings of the 5th ACM SIGMM International Workshop on Multimedia Information Retrieval, MIR -03. ACM, New York, NY, USA, pp 1–8. https://doi.org/10.1145/973264.973266

  2. Adobe HTTP Dynamic Streaming (HDS) Technology Center | Adobe Developer Connection. https://www.adobe.com/devnet/hds.html. Accessed 20 July 2021

  3. Aiordăchioae A (2019) Eyewear-based system for sharing first-person video to remote viewers. In: Proceedings of the 2019 E-Health and Bioengineering Conference (EHB), pp 1–4. https://doi.org/10.1109/EHB47216.2019.8969871

  4. Aiordăchioae A, Vatavu RD (2019) Life-tags: a smartglasses-based system for recording and abstracting life with tag clouds. Proc ACM Hum Comput Interact 3(EICS):15:1–15:22. http://doi.acm.org/10.1145/3331157. Accessed 20 July 2021

  5. Aiordăchioae A, Vatavu RD, Popovici DM (2019) A design space for vehicular lifelogging to support creation of digital content in connected cars. In: Proceedings of the ACM SIGCHI Symposium on Engineering Interactive Computing Systems, EICS 19. ACM, New York, NY, USA. https://doi.org/10.1145/3319499.3328234

  6. Al Mahmud A, Braun J, Martens JB (2010) Designing to capture and share life experiences for persons with Aphasia. In: Proc. of the 12th Int. Conference on Human Computer Interaction with Mobile Devices and Services, Mobile HCI ’10. ACM, New York, NY, USA, pp 391–392. http://doi.acm.org/10.1145/1851600.1851680. Accessed 20 July 2021

  7. Amazon.com: Wi-Fi Video Camera on Glasses Wearable Camera Sport DV DVR Dash Camera (WiFi-B, Black): Home Audio & Theater. https://www.amazon.com/Action-Wearable-Streaming-Glasses-Recorder/dp/B01M0EPUBT. Accessed 20 July 2021

  8. Aoki PM, Grinter RE, Hurst A, Szymanski MH, Thornton JD, Woodruff A (2002) Sotto Voce: Exploring the interplay of conversation and mobile audio spaces. In: Proc. of the SIGCHI Conf. on Human Factors in Computing Systems, CHI 02. ACM, New York, NY, USA, pp 431–438. https://doi.org/10.1145/503376.503454

  9. Azuma RT (1997) A survey of augmented reality. Presence: Teleoper Virtual Environ 6(4):355–385. https://doi.org/10.1162/pres.1997.6.4.355

  10. Azuma RT (2016) The most important challenge facing augmented reality. Presence: Teleoper. Virtual Environ 25(3):234–238. https://doi.org/10.1162/PRES_a_00264

  11. Barakabitze AA, Barman N, Ahmad A, Zadtootaghaj S, Sun L, Martini MG, Atzori L (2020) QoE Management of Multimedia Streaming Services in Future Networks: A Tutorial and Survey. IEEE Communications Surveys Tutorials 22(1):526–565. https://doi.org/10.1109/COMST.2019.2958784

    Article  Google Scholar 

  12. Barman N, Martini MG (2019) QoE modeling for HTTP adaptive video streaming a survey and open challenges. IEEE Access 7:30831–30859. https://doi.org/10.1109/ACCESS.2019.2901778

  13. Bederson BB (1995) Audio Augmented Reality: a prototype automated tour guide. In: Conference Companion on Human Factors in Computing Systems, CHI 95. ACM, New York, NY, USA, pp 210-211. https://doi.org/10.1145/223355.223526

  14. Berry E, Kapur N, Williams L, Hodges S, Watson P, Smyth G, Srinivasan J, Smith R, Wilson B, Wood K (2007) The use of a wearable camera, sensecam, as a pictorial diary to improve autobiographical memory in a patient with limbic encephalitis: A preliminary report. Neuropsychol Rehabil 17(4–5):582–601. https://doi.org/10.1080/09602010601029780

  15. Billinghurst M, Clark A, Lee G (2015) A survey of Augmented Reality. Found Trends Hum Comput Interact 8(2–3):73–272. http://dx.doi.org/10.1561/1100000049. Accessed 20 July 2021

  16. Blum M, Pentland A, Troster G (2006) InSense: Interest-based life logging. IEEE MultiMedia 13(4):40–48. https://doi.org/10.1109/MMUL.2006.87

  17. Brewster S, Brown LM (2004) Tactons: Structured tactile messages for non-visual information display. In: Proceedings of the Fifth Conference on Australasian User Interface - vol. 28, AUIC 04. Australian Computer Society, Inc., AUS, pp 15–23

  18. Buschek D, Spitzer M, Alt F (2015) Video-recording your life: User perception and experiences. In: Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems, CHI EA 15. ACM, New York, NY, USA, pp 2223–2228. https://doi.org/10.1145/2702613.2732743

  19. Chambel T, Kaiser R, Niamut OA, Ooi WT, Redi JA (2016) AltMM ’16: Proceedings of the 1st International Workshop on Multimedia Alternate Realities. ACM, New York, NY, USA. https://dl.acm.org/doi/proceedings/10.1145/2983298. Accessed 20 July 2021

  20. Choi J, Jeong S, Ko J (2019) Xremul: An emulation environment for XR application development. In: Proceedings of the 20th International Workshop on Mobile Computing Systems and Applications, HotMobile ’19. ACM, New York, NY, USA, pp 187. https://doi.org/10.1145/3301293.3309569

  21. Chowdhury S, Ferdous MS, Jose JM (2016) Bystander privacy in lifelogging. In: Proceedings of the 30th International BCS Human Computer Interaction Conference: Companion Volume, HCI ’16. BCS Learning & Development Ltd., Swindon, UK, pp 15:1–15:3. https://doi.org/10.14236/ewic/HCI2016.62

  22. Cisco: Cisco Annual Internet Report (2018-2023) White Paper. https://www.cisco.com/c/en/us/solutions/collateral/executive-perspectives/annual-internet-report/white-paper-c11-741490.html. [Accessed March 2020]

  23. Clarifai (2019) About Clarifai. Transforming how we see the world. https://clarifai.com/about. Accessed 20 July 2021

  24. Cullen B (2015) WordCloud. https://www.nuget.org/packages/WordCloud. Accessed 20 July 2021

  25. dash.js (2020). https://github.com/Dash-Industry-Forum/dash.js. Accessed 20 July 2021

  26. Davies CJ, Miller A, Allison C (2014) A view from the hill: Where cross reality meets virtual worlds. In: Proceedings of the 20th ACM Symposium on Virtual Reality Software and Technology, VRST ’14. ACM, New York, NY, USA, p 213. https://doi.org/10.1145/2671015.2671138

  27. Denning T, Dehlawi Z, Kohno T (2014) In Situ with bystanders of Augmented Reality glasses: Perspectives on recording and privacy-mediating technologies. In: Proc. of the SIGCHI Conf. on Human Factors in Computing Systems, CHI 14. ACM, New York, NY, USA, pp 2377–2386. https://doi.org/10.1145/2556288.2557352

  28. de Jager D, Wood AL, Merrett GV, Al-Hashimi BM, OHara K, Shadbolt NR, Hall, W (2011) A low-power, distributed, pervasive healthcare system for supporting memory. In: Proc. of the 1st ACM MobiHoc Workshop on Pervasive Wireless Healthcare, MobileHealth 11. ACM. https://doi.org/10.1145/2007036.2007043

  29. Everingham M, Thomas B, Troscianko T (1998) Head-mounted mobility aid for low vision using scene classification techniques. Int J Virtual Real 3(4):1–10. http://dx.doi.org/10.20870/IJVR.1998.3.4.2629. Accessed 20 July 2021

  30. Eyeglasses Hidden WiFi camera. https://spyshoproundrock.com/product/cameras/eyeglasses-hidden-wifi-camera/. Accessed 20 July 2021

  31. Faklaris C, Cafaro F, Hook SA, Blevins A, OHaver M, Singhal N (2016) Legal and ethical implications of mobile live-streaming video apps. In: Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct, MobileHCI 16. ACM, New York, NY, USA, pp 722–729. https://doi.org/10.1145/2957265.2961845

  32. Fan C, Crandall DJ (2016) DeepDiary: Automatically captioning lifeloggingiImage Streams. In: ECCV 2016 Workshops. Springer. https://doi.org/10.1007/978-3-319-46604-0_33

  33. FFmpeg (2019). https://www.ffmpeg.org/. Accessed 20 July 2021

  34. Fleischman E (1998) Advanced Streaming Format (ASF) Specification. https://tools.ietf.org/html/draft-fleischman-asf-01. Accessed 20 July 2021

  35. Gartner (2018) Gartner says Worldwide Wearable Device sales to grow 26 percent in 2019. https://www.gartner.com/en/newsroom/press-releases/2018-11-29-gartner-says-worldwide-wearable-device-sales-to-grow-. Accessed 20 July 2021

  36. Gemmell J (2014) Life-logging, Thing-logging and the Internet of Things. In: Proceedings of the 2014 Workshop on Physical Analytics, WPA ’14. ACM, New York, NY, USA, pp 17–17. http://doi.acm.org/10.1145/2611264.2611276. Accessed 20 July 2021

  37. Google. Cloud Text-to-Speech. Speech Synthesis. Google Cloud. https://cloud.google.com/text-to-speech. Accessed 20 July 2021

  38. GoogleClips (2019) Google Clips Specifications. https://support.google.com/googleclips/answer/7545447?hl=en. Accessed 20 July 2021

  39. Gurrin C, Smeaton AF, Doherty AR (2014) LifeLogging: Personal big data. Found Trends Inf Retr 8(1):1–125. https://doi.org/10.1561/1500000033

  40. Gurrin C, Smeaton AF, Doherty AR (2014) LifeLogging: Personal big data. Found Trends Inf Retr 8(1):1–125. http://dx.doi.org/10.1561/1500000033. Accessed 20 July 2021

  41. Hart H (2010) 1996: JenniCam starts lifecasting. https://www.wired.com/2010/04/0414jennicam-launches/. Accessed 20 July 2021

  42. Hu C, Zhai G, Li (2015) An augmented-reality night vision enhancement application for see-through glasses. In: Proc. of the 2015 IEEE Int. Conf. on Multimedia Expo Workshops, pp 1–6. https://doi.org/10.1109/ICMEW.2015.7169860

  43. Hinbarji Z, Albatal R, O’Connor N, Gurrin C (2016) LoggerMan, a Comprehensive Logging and Visualization Tool to Capture Computer Usage, vol. 9517. Springer, Cham. https://doi.org/10.1007/978-3-319-27674-8_31

  44. Hodges S, Williams L, Berry E, Izadi S, Srinivasan J, Butler A, Smyth G, Kapur N, Wood K (2006) SenseCam: A retrospective memory aid. In: Proceedings of the 8th International Conference on Ubiquitous Computing, UbiComp’06. Springer-Verlag, Berlin, Heidelberg, pp 177–193. http://dx.doi.org/10.1007/11853565_11. Accessed 20 July 2021

  45. Hoyle R, Templeman R, Anthony D, Crandall D, Kapadia A (2015) Sensitive Lifelogs: A privacy analysis of photos from wearable cameras. In: Proc. of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI ’15. ACM, New York, NY, USA, pp 1645–1648. http://doi.acm.org/10.1145/2702123.2702183. Accessed 20 July 2021

  46. Hoyle R, Templeman R, Armes S, Anthony D, Crandall D, Kapadia A (2014) Privacy behaviors of lifeloggers using wearable cameras. In: Proc. of the 2014 ACM Int. Joint Conference on Pervasive and Ubiquitous Computing, UbiComp ’14. ACM, New York, NY, USA, pp 571–582. http://doi.acm.org/10.1145/2632048.2632079. Accessed 20 July 2021

  47. HTML5 FLV Player (2018). https://github.com/Bilibili/flv.js/. Accessed 20 July 2021

  48. Huh S, Muralidharan S, Ko H, Yoo B (2019) XR collaboration architecture based on decentralized web. In: The 24th International Conference on 3D Web Technology, Web3D ’19. ACM, pp 1–9. https://doi.org/10.1145/3329714.3338137

  49. Jacobson V, Frederick R, Casner S, Schulzrinne H (2003) RTP: A transport protocol for real-time applications. https://tools.ietf.org/html/rfc3550. Accessed 20 July 2021

  50. Janzen R, Yasrebi SN, Bose AJ, Subramanian A, Mann S (2014) Walking Through Sight: Seeing the Ability to See, in a 3-D Augmediated Reality Environment. In: Proc. of the 2014 IEEE Games Media Entertainment. pp 1–2. http://dx.doi.org/10.1109/GEM.2014.7048124. Accessed 20 July 2021

  51. Jenkins A (2019) The fall and rise of VR: The struggle to make virtual reality get real. https://fortune.com/longform/virtual-reality-struggle-hope-vr/. Accessed 20 July 2021

  52. Jiang N, Swaminathan V, Wei S (2017) Power evaluation of 360 VR video streaming on head mounted display devices. In: Proceedings of the 27th Workshop on Network and Operating Systems Support for Digital Audio and Video, NOSSDAV17. ACM, New York, NY, USA, pp 55–60. https://doi.org/10.1145/3083165.3083173

  53. Johansen R (1988) GroupWare: Computer Support for Business Teams. The Free Press, USA

    Google Scholar 

  54. Juhlin O, Reponen E, Bentley F, Kirk D (2011) Video interaction - making broadcasting a successful social media. In: CHI 11 Extended Abstracts on Human Factors in Computing Systems, CHI EA 11. ACM, New York, NY, USA, pp 2437–2440. https://doi.org/10.1145/1979742.1979578

  55. Kekulluoglu D, Kokciyan N, Yolum P (2018) Preserving privacy as social responsibility in online social networks. ACM Trans Internet Technol 18(4). https://doi.org/10.1145/3158373

  56. Kitamura K, Yamasaki T, Aizawa K (2008) Food log by analyzing food images. In: Proc. of the 16th ACM International Conference on Multimedia, MM ’08. ACM, New York, NY, USA, pp 999–1000. http://doi.acm.org/10.1145/1459359.1459548. Accessed 20 July 2021

  57. Koelle M, Ananthanarayan S, Czupalla S, Heuten W, Boll S (2018) Your smart glasses’ camera bothers me! Exploring opt-in and opt-out gestures for privacy mediation. In: Proc. of the 10th Nordic Conf. on Human-Computer Interaction, NordiCHI 18. ACM, pp 473–481. https://doi.org/10.1145/3240167.3240174

  58. Koelle M, Heuten W, Boll S (2017) Are you hiding it?: Usage habits of lifelogging camera wearers. In: Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI ’17. ACM, New York, NY, USA, pp 80:1–80:8. http://doi.acm.org/10.1145/3098279.3122123. Accessed 20 July 2021

  59. Koelle M, Kranz M, Möller A (2015) Don’t look at me that way!: Understanding user attitudes towards data glasses usage. In: Proc. of the 17th Int. Conf. on Human-Computer Interaction with Mobile Devices and Services, MobileHCI ’15. ACM, pp 362–372. http://doi.acm.org/10.1145/2785830.2785842. Accessed 20 July 2021

  60. Kruitbosch G, Nack F (2008) Broadcast yourself on Youtube: Really? In: Proceedings of the 3rd ACM International Workshop on Human-Centered Computing, HCC 08. ACM, New York, NY, USA, pp 7–10. https://doi.org/10.1145/1462027.1462029

  61. Kurita Y, Ikeda A, Tamaki T, Ogasawara T, Nagata K (2009) Haptic Augmented Reality interface using the real force response of an object. In: Proceedings of the 16th ACM Symposium on Virtual Reality Software and Technology, VRST 09. ACM, New York, NY, USA, pp 83–86. https://doi.org/10.1145/1643928.1643948

  62. Langlotz T, Sutton J, Zollmann S, Itoh Y, Regenbrecht H (2018) ChromaGlasses: Computational glasses for compensating colour blindness. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, CHI 18. ACM, New York, NY, USA. https://doi.org/10.1145/3173574.3173964

  63. Liotou E, Sfikopoulos A, Kaltzias P, Tsolkas V (2017) An evaluation of buffer- and rate-based HTTP adaptive streaming strategies. In: Proc. of the 2017 IEEE 22nd International Workshop on Computer Aided Modeling and Design of Communication Links and Networks, pp 1–6. https://doi.org/10.1109/CAMAD.2017.8031637

  64. Liu C, Bouazizi I, Gabbouj M (2011) Rate adaptation for adaptive HTTP streaming. In: Proc. of the 2nd Annual ACM Conference on Multimedia Systems, MMSys 11. ACM, pp 169–174. https://doi.org/10.1145/1943552.1943575

  65. Liu C, Bouazizi I, Hannuksela MM, Gabbouj M (2012) Rate adaptation for dynamic adaptive streaming over HTTP in content distribution network. Signal Process Image Commun 27(4):288–31. http://www.sciencedirect.com/science/article/pii/S0923596511001135. Accessed 20 July 2021

  66. Live stream WiFi Glasses. https://www.24kupi.com/live-stream-wifi-glasses.html. Accessed 20 July 2021

  67. Mann S (1999) Mediated reality. Linux J 1999(59es). https://www.linuxjournal.com/article/3265. Accessed 20 July 2021

  68. Mann S, Furness T, Yuan Y, Iorio J, Wang Z (2018) All reality: Virtual, augmented, mixed (X), mediated (X, Y), and multimediated reality. CoRR abs/1804.08386. http://arxiv.org/abs/1804.08386. Accessed 20 July 2021

  69. Marr B (2019) What is extended reality technology? A simple explanation for anyone. https://www.forbes.com/sites/bernardmarr/2019/08/12/what-is-extended-reality-technology-a-simple-explanation-for-anyone/#6ffa4ba37249. Accessed 20 July 2021

  70. Marr B (2020) The 5 biggest virtual and augmented reality trends in 2020 everyone should know about. https://www.forbes.com/sites/bernardmarr/2020/01/24/the-5-biggest-virtual-and-augmented-reality-trends-in-2020-everyone-should-know-about/#69f94c3824a8. Accessed 20 July 2021

  71. Matthias P, Billinghurst M, Siang See Z (2019) This land AR: an Australian Music and Sound XR installation. In: Proc. of the 17th Int. Conf. on Virtual-Reality Continuum and Its Applications in Industry, VRCAI ’19. ACM, New York, NY, USA. https://doi.org/10.1145/3359997.3365740

  72. McCullough M, Kulber L, Sammons P, Santos P, Kulber D (2018) Google Glass for remote surgical tele-proctoring in low- and middle-income countries: A feasibility study from Mozambique. Plast Reconstr Surg Glob Open 1. https://doi.org/10.1097/GOX.0000000000001999

  73. McVeigh-Schultz J, Stein J, Boyle J, Duff E, Watson J, Syam, A., Tasse, A., Wiscombe, S., Fisher S (2012) Vehicular lifelogging: New contexts and methodologies for human-car interaction. In: CHI 12 Extended Abstracts on Human Factors in Computing Systems, CHI EA 12. ACM, New York, NY, USA, pp 221–230. https://doi.org/10.1145/2212776.2212800

  74. MeCam (2019) High definition video camera | Best life logging device | Mini video camera - MeCam. https://mecam.me/products/mecam-hd. Accessed 20 July 2021

  75. Metsis V, Kosmopoulos D, Athitsos V, Makedon F (2014) Non-invasive analysis of sleep patterns via multimodal sensor input. Pers Ubiquit Comput 18(1):19–26. http://dx.doi.org/10.1007/s00779-012-0623-1. Accessed 20 July 2021

  76. Microsoft. Soundscape. https://www.microsoft.com/en-us/research/product/soundscape/. Accessed 20 July 2021

  77. Milgram P, Colquhoun HJ (1999) A taxonomy of real and virtual world display integration. Springer-Verlag. https://www.researchgate.net/publication/2440732_A_Taxonomy_of_Real_and_Virtual_World_Display_Integration. Accessed 20 July 2021

  78. Milgram P, Kishino F (1994) A taxonomy of mixed reality visual displays. IEICE Trans Inf Syst E77-D(12):1321–1329. https://search.ieice.org/bin/summary.php?id=e77-d_12_1321. Accessed 20 July 2021

  79. Miller K, Quacchio E, Gennari G, Wolisz A (2012) Adaptation algorithm for adaptive streaming over HTTP. In: Proc. of the 19th International Packet Video Workshop, pp 173–178. https://doi.org/10.1109/PV.2012.6229732

  80. Mingliang C (2020) illuspas/Node-Media-Server. https://github.com/illuspas/Node-Media-Server. Accessed 20 July 2021

  81. Mori S, Ikeda S, Saito H (2017) A survey of diminished reality: Techniques for visually concealing, eliminating, and seeing through real objects. IPSJ Transactions on Computer Vision and Applications 9:1–14. https://doi.org/10.1186/s41074-017-0028-1

  82. MPEG | The Moving Picture Experts Group website. https://mpeg.chiariglione.org/. Accessed 20 July 2021

  83. Muzaffar R, Yanmaz E, Raffelsberger C, Bettstetter C, Cavallaro A (2020) Live multicast video streaming from drones: an experimental study. Auton Robot 44. https://doi.org/10.1007/s10514-019-09851-6

  84. Nakagawa R, Sonobe K (2019) Encounters: A multiparticipant audiovisual art experience with XR. In: SIGGRAPH Asia 2019 XR, SA ’19. ACM New York, NY, USA, pp 6–8. https://doi.org/10.1145/3355355.3361886

  85. Nakagawa R, Sonobe K (2020) Encounters 2.0: A multiparticipant audiovisual art experience with XR. In: ACM SIGGRAPH 2020 Immersive Pavilion, SIGGRAPH ’20. ACM, New York, NY, USA. https://doi.org/10.1145/3388536.3407882

  86. Narrative (2019) The world’s most wearable HD video camera | Narrative Clip 2. http://getnarrative.com/. Accessed 20 July 2021

  87. North: Getting started with your Myo Armband. https://support.getmyo.com/hc/en-us/articles/203398347-Getting-started-with-your-Myo-armband. Accessed 20 July 2021

  88. Obrist M, Velasco C, Vi CT, Ranasinghe N, Israr A, Cheok AD, Spence C, Gopalakrishnakone P (2016) Touch, taste, & smell user interfaces: The future of multisensory HCI. In: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, CHI EA 16. ACM, New York, NY, USA, pp 3285–3292. https://doi.org/10.1145/2851581.2856462

  89. Ochi D, Kunita Y, Fujii K, Kojima A, Iwaki S, Hirose J (2014) HMD Viewing Spherical Video Streaming System. In: Proceedings of the 22nd ACM International Conference on Multimedia, MM 14. ACM, New York, NY, USA, pp 763–764. https://doi.org/10.1145/2647868.2654870

  90. Ochi D, Niwa K, Kameda A, Kunita Y, Kojima A (2015) Dive into remote events: Omnidirectional video streaming with acoustic immersion. In: Proceedings of the 23rd ACM International Conference on Multimedia, MM 15. ACM, New York, NY, USA, pp 737–738. https://doi.org/10.1145/2733373.2807963

  91. Pai YS, Hajika R, Gupta K, Sasikumar P, Billinghurst M (2020) NeuralDrum: Perceiving brain synchronicity in XR drumming. In: SIGGRAPH Asia 2020 Technical Communications, SA ’20. ACM. https://doi.org/10.1145/3410700.3425434

  92. Pamparău C, Vatavu RD (2021) Flexisee: Flexible configuration, customization, and control of mediated and augmented vision for users of smart eyewear devices. Multimed Tools Appl. https://doi.org/10.1007/s11042-020-10164-5

  93. Pantos R (2017) RFC 8216 - HTTP Live Streaming. https://datatracker.ietf.org/doc/rfc8216. Accessed 20 July 2021

  94. Paradiso JA, Landay JA (2009) Guest editors’ introduction: Cross-reality environments. IEEE Pervasive Comput. 8(3):14–15. https://doi.org/10.1109/MPRV.2009.47

  95. Parmar HMT (2012) Adobes real time messaging protocol. https://wwwimages2.adobe.com/content/dam/acom/en/devnet/rtmp/pdf/rtmp_specification_1.0.pdf. Accessed 20 July 2021

  96. Pausch R, Proffitt D, Williams G (1997) Quantifying immersion in virtual reality. In: Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 97. ACM Press/Addison-Wesley Publishing Co., USA, pp 13–18. https://doi.org/10.1145/258734.258744

  97. Peli E (1999) Simple 1-D image enhancement for head-mounted low vision aid. Vis Impair Res 1(1):3–10. https://doi.org/10.1076/vimr.1.1.3.4449

  98. Peli E, Luo G, Bowers A, Rensing N (2007) Applications of augmented-vision head-mounted systems in vision rehabilitation. J Soc Inf Disp 15(12):1037–1045. https://onlinelibrary.wiley.com/doi/abs/10.1889/1.2825088. Accessed 20 July 2021

  99. Philo A, Preethi C, Prasad K (2019) Analysis of vehicle activities and live streaming using IOT. In: Proceedings of the 2019 International Conference on Communication and Signal Processing (ICCSP), pp. 754–757. https://doi.org/10.1109/ICCSP.2019.8698001

  100. Postel J (1980) User datagram protocol. https://tools.ietf.org/html/rfc768. Accessed 20 July 2021

  101. Procyk J, Neustaedter C, Pang C, Tang A, Judge TK (2014) Exploring video streaming in public settings: Shared geocaching over distance using mobile video chat. In: Proc. of the SIGCHI Conference on Human Factors in Computing Systems, CHI 14. ACM, pp 2163–2172. https://doi.org/10.1145/2556288.2557198

  102. Protocol Extension for Low-Latency HLS (Preliminary Specification) | Apple Developer Documentation (2019). https://developer.apple.com/documentation/http_live_streaming/protocol_extension_for_low-latency_hls_preliminary_specification. Accessed 20 July 2021

  103. Rao A, Lanphier R, Stiemerling M, Schulzrinne H, Westerlund M (2016) Real-Time Streaming Protocol Version 2.0. https://tools.ietf.org/html/rfc7826. Accessed 20 July 2021

  104. Reschke JF, Fielding RT (2014) Hypertext Transfer Protocol (HTTP/1.1): Message Syntax and Routing. https://tools.ietf.org/html/rfc7230. Accessed 20 July 2021

  105. Rio MD, Meloni V, Frexia F, Cabras F, Tumbarello R, Montis S, Marini A, Zanetti G (2018) Augmented Reality for supporting real time telementoring: an exploratory study applied to ultrasonography. In: Proceedings of the 2nd International Conference on Medical and Health Informatics, ICMHI 18. ACM, New York, NY, USA, pp 218–222. https://doi.org/10.1145/3239438.3239444

  106. Roach AB (2016) WebRTC Video Processing and Codec Requirements. https://tools.ietf.org/html/rfc7742

  107. Rodríguez-Gil L, Orduña P, García-Zubía J, de Ipiña DL (2017) Interactive live-streaming technologies and approaches for web-based applications. Multimed Tools Appl 77:6471–6502

    Article  Google Scholar 

  108. Rothe S, Kegeles B, Hussmann H (2019) Camera heights in cinematic virtual reality: How viewers perceive mismatches between camera and eye height. In: Proc. of the 2019 ACM Int. Conference on Interactive Experiences for TV and Online Video, TVX 19. ACM, pp 25–34. https://doi.org/10.1145/3317697.3323362. Accessed 20 July 2021

  109. Ruether T (2019) Low-latency CMAF for Live Streaming at scale. https://www.wowza.com/blog/low-latency-cmaf-chunked-transfer-encoding. Accessed 20 July 2021

  110. Ruminski J, Smiatacz M, Bujnowski A, Andrushevich A, Biallas M, Kistler R (2015) Interactions with recognized patients using smart glasses. In: Proc. of the 8th International Conference on Human System Interaction (HSI), pp 187–194

  111. Saito K, Masuda N (2013) Two types of Twitter users with equally many followers. In: Proceedings of the 2013 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM 13. ACM, New York, NY, USA, pp 1425–1426. https://doi.org/10.1145/2492517.2492575

  112. Sani Y, Mauthe A, Edwards C  (2017) Adaptive bitrate selection: a survey. IEEE Commun Surv Tutorials 19(4):2985–3014. https://doi.org/10.1109/COMST.2017.2725241

  113. Santos-Gonzalez I, Rivero-Garcia A, Molina-Gil J, Caballero-Gil P (2017) Implementation and analysis of real-time streaming protocols 17(4):846. https://doi.org/10.3390/s17040846

  114. Schönauer C, Mossel A, Zai IA, Vatavu RD (2015) Touch, movement and vibration: User perception of vibrotactile feedback for touch and mid-air gestures. In: Human-Computer Interaction – INTERACT 2015. Springer, pp 165–172. https://doi.org/10.1007/978-3-319-22723-8_14

  115. Scourboutakos P, Lu MH, Nerker S, Mann S (2017) Phenomenologically Augmented Reality with new wearable LED sequential wave imprinting machines. In: Proceedings of the Eleventh International Conference on Tangible, Embedded, and Embodied Interaction, TEI 17. ACM, New York, NY, USA, pp 751–755. https://doi.org/10.1145/3024969.3035534

  116. Sellen AJ, Whittaker S (2010) Beyond total capture: A constructive critique of lifelogging. Commun ACM 53(5):70–77. https://doi.org/10.1145/1735223.1735243

  117. Seo D, Yoo B, Ko H (2017) Webized 3D content streaming system for autostereoscopic 3D displays. In: Proc. of the 22nd Int. Conference on 3D Web Technology, Web3D 17. ACM, New York, NY, USA. https://doi.org/10.1145/3055624.3075940

  118. Seo D, Yoo B, Ko H (2018) Webizing collaborative interaction space for cross reality with various human interface devices. In: Proc. of the 23rd Int. ACM Conf. on 3D Web Technology, Web3D ’18. ACM. https://doi.org/10.1145/3208806.3208808

  119. Seufert M, Egger S, Slanina M, Zinner T, Hofeld T, Tran-Gia P (2015) A survey on quality of experience of HTTP adaptive streaming. IEEE Commun Surv Tutorials 17(1):469–492. https://doi.org/10.1109/COMST.2014.2360940

  120. Silverlight 4 Launch Home | Microsoft Silverlight. https://www.microsoft.com/silverlight/smoothstreaming/. [Accessed March 2020]

  121. Singhal S, Neustaedter C, Ooi YL, Antle AN, Matkin B (2017) Flex-N-Feel: The design and evaluation of emotive gloves for couples to support touch over distance. In: Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, CSCW 17. ACM, New York, NY, USA, pp 98–110.  https://doi.org/10.1145/2998181.2998247

  122. Singhal S, Neustaedter C, Schiphorst T, Tang A, Patra A, Pan R (2016) You are being watched: Bystanders perspective on the use of camera devices in public spaces. In: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, CHI EA 16. ACM, New York, NY, USA, pp 3197–3203. https://doi.org/10.1145/2851581.2892522

  123. Smith K (2020) 57 fascinating and incredible YouTube statistics. https://www.brandwatch.com/blog/youtube-stats/. Accessed 20 July 2021

  124. Snap Inc. (2020) Spectacles by Snapchat. Capture Your World in (2020). https://www.spectacles.com/. Accessed 20 July 2021

  125. SnapCam (2019) iON USA | SnapCam. https://usa.ioncamera.com/snapcam/. Accessed 20 July 2021

  126. Sobel K, Takeuchi L, Castaneda LM, Bindman SW (2019) Immersive media design and children. IDC ’19. ACM, New York, NY, USA, pp 689–696. https://doi.org/10.1145/3311927.3325163

  127. SOCKET.IO 2.0 IS HERE. https://socket.io. Accessed 20 July 2021

  128. Socket.IO Client Library for Java (2015). https://mvnrepository.com/artifact/com.github.nkzawa/socket.io-client/0.3.0. Accessed 20 July 2021

  129. Software OB. Open Broadcast Software. https://obsproject.com/. Accessed 20 July 2021

  130. Speicher M, Hall BD, Nebeling M (2019) What is mixed reality? In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI 19. ACM, New York, NY, USA. https://doi.org/10.1145/3290605.3300767

  131. Stearns L, Findlater L, Froehlich JE (2018) Design of an Augmented Reality magnification aid for low vision users. In: Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility, ASSETS 18. ACM, New York, NY, USA, pp 28–39. https://doi.org/10.1145/3234695.3236361

  132. Stix G (2011) Photographic memory: Wearable cam Could help patients stave off effects of impaired recall. Scientific American. https://www.scientificamerican.com/article/photographic-memory-wearable/. Accessed 20 July 2021

  133. Taha M, Lloret J, Canovas A (2017) Survey of transportation of adaptive multimedia streaming service in internet. Network Protocols and Algorithms 9:85. https://doi.org/10.5296/npa.v9i1-2.12412

  134. Tanaka A, Takemura H, Tajima K (2014) Why you follow: A classification scheme for Twitter follow links. In: Proc. of the 25th ACM Conf. on Hypertext and Social Media, HT 14, pp 324–326. https://doi.org/10.1145/2631775.2631790

  135. Tanuwidjaja E, Huynh D, Koa K, Nguyen C, Shao C, Torbett P, Emmenegger C, Weibel N (2014) Chroma: A wearable Augmented-Reality solution for color blindness. In: Proc. of the 2014 ACM Int. Joint Conf. on Pervasive and Ubiquitous Computing, UbiComp 14, pp 799–810. https://doi.org/10.1145/2632048.2632091

  136. Tian G, Liu Y (2012) Towards agile and smooth video adaptation in dynamic HTTP streaming. In: Proceedings of the 8th International Conference on Emerging Networking Experiments and Technologies, CoNEXT 12. ACM, New York, NY, USA, pp 109–120. https://doi.org/10.1145/2413176.2413190

  137. Vatavu RD (2015) Audience silhouettes: Peripheral awareness of synchronous audience kinesics for social television. In: Proceedings of the ACM International Conference on Interactive Experiences for TV and Online Video, TVX 15. ACM, New York, NY, USA, pp 13–22. https://doi.org/10.1145/2745197.2745207

  138. Vatavu RD, Mossel A, Schönauer C (2016) Digital Vibrons: Understanding users perceptions of interacting with invisible, zero-weight matter. In: Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI 16. ACM, New York, NY, USA, pp 217–226. https://doi.org/10.1145/2935334.2935364

  139. Vatavu RD, Saeghe P, Chambel T, Vinayagamoorthy V, Ursu MF (2020) Conceptualizing augmented reality television for the living room. In: ACM International Conference on Interactive Media Experiences, IMX ’20. ACM, New York, NY, USA, pp 1–12. https://doi.org/10.1145/3391614.3393660

  140. video-dev/hls.js (2020). https://github.com/video-dev/hls.js. Accessed 20 July 2021

  141. Wowza Media Systems. 2019 Video Streaming Latency Report. https://www.wowza.com/wp-content/uploads/Streaming-Video-Latency-Report-Interactive-2019.pdf. Accessed 20 July 2021

  142. Yang J, Mattern F (2019) Audio Augmented Reality for human-object interactions. In: Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers, UbiComp/ISWC 19 Adjunct. ACM, New York, NY, USA, pp 408–412. https://doi.org/10.1145/3341162.3349302

  143. Yavoruk O (2020) The study of observation in physics classes through XR technologies. In: Proc. of the 4th Int. Conf. on Digital Technology in Education, ICDTE 2020. ACM, New York, NY, USA, pp 58–62. https://doi.org/10.1145/3429630.3429637

  144. Zhao Y, Cutrell E, Holz C, Morris MR, Ofek E, Wilson AD (2019) SeeingVR: A set of tools to make virtual reality more accessible to people with low vision. In: Proc. of the 2019 CHI Conf. on Human Factors in Computing Systems, CHI ’19. ACM. https://doi.org/10.1145/3290605.3300341

  145. Zhao Y, Szpiro S, Azenkot S (2015) ForeSee: A customizable head-mounted vision enhancement system for people with low vision. In: Proc. of the 17th Int. ACM SIGACCESS Conference on Computers & Accessibility, ASSETS 15. ACM, New York, NY, USA, pp 239–249. http://dx.doi.org/10.1145/2700648.2809865. Accessed 20 July 2021

  146. Zhao Y, Szpiro S, Shi L, Azenkot S (2019) Designing and evaluating a customizable head-mounted vision enhancement system for people with low vision. ACM Trans Access Comput 12(4). http://dx.doi.org/10.1145/3361866. Accessed 20 July 2021

  147. Zhou C, Zhang X, Huo L, Guo Z (2012) A control-theoretic approach to rate adaptation for dynamic HTTP streaming. In: Proc. of the 2012 Visual Communications and Image Processing, pp 1–6. https://doi.org/10.1109/VCIP.2012.6410740

  148. Zini F, Reinstadler M, Ricci F (2015) Life-logs aggregation for quality of life monitoring. In: Proc. of the 5th Int. Conf. on Digital Health 2015, DH ’15. ACM, pp 131–132. http://doi.acm.org/10.1145/2750511.2750531. Accessed 20 July 2021

  149. Zokai S, Esteve J, Genc Y, Navab N (2003) Multiview paraperspective projection model for diminished reality. In: Proceedings of the 2nd IEEE/ACM International Symposium on Mixed and Augmented Reality, ISMAR 03. IEEE Computer Society, USA, p 217. https://dl.acm.org/doi/10.5555/946248.946801. Accessed 20 July 2021

Download references

Acknowledgements

Original versions of the icons employed in Figs. 1, 2, 4–6 were made by Freepik (http://www.freepik.com, Miscellaneous Elements pack) from Flaticon (http://www.flaticon.com), licensed under Creative Commons BY 3.0 (http://creativecommons.org/licenses/by/3.0). The HoloLens device used in this work was kindly provided by OSF Global Services, the Mobile Division, Suceava.

Funding

This work was supported by a grant of the Ministry of Research and Innovation, CNCS-UEFISCDI, project no. PN-III-P1-1.1-TE-2016-2173 (TE141/2018), within PNCDI III. The work was carried out in the Machine Intelligence and Information Visualization Lab (MintViz) of the MANSiD Research Center. The infrastructure was provided by the University of Suceava and was partially supported from the project “Integrated center for research, development and innovation in Advanced Materials, Nanotechnologies, and Distributed Systems for fabrication and control”, No. 671/09.04.2015, Sectoral Operational Program for Increase of the Economic Competitiveness, co-funded from the European Regional Development Fund.

Author information

Authors and Affiliations

  1. MintViz Lab, MANSiD Research Center, University Ştefan cel Mare of Suceava, Suceava, 720229, Romania

    Adrian Aiordăchioae, Cristian Pamparău & Radu-Daniel Vatavu

Authors
  1. Adrian Aiordăchioae
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Cristian Pamparău
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Radu-Daniel Vatavu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Radu-Daniel Vatavu.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was supported by a grant of the Ministry of Research and Innovation, CNCS-UEFISCDI, project no. PN-III-P1-1.1-TE-2016-2173 (TE141/2018), within PNCDI III.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Aiordăchioae, A., Pamparău, C. & Vatavu, RD. Lifelogging meets alternate and cross-realities: an investigation into broadcasting personal visual realities to remote audiences. Multimed Tools Appl 83, 46707–46730 (2024). https://doi.org/10.1007/s11042-021-11310-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11310-3

Keywords

Search

Navigation