Adrian Aiordăchioae
Radu-Daniel Vatavu
ABSTRACT
We discuss in this work the opportunity of employing lifelogging devices, applications, and systems, such as systems that collect, process, and store video using mobile and wearable cameras, in order to run queries about objects and concepts of interest from everyday life. The outcome is an instance of "opportunistic mobile crowdsensing," which we implement with lifelogging technology, mobile video cameras, and camera glasses. We describe the implementation of our concept that builds on top of Life-Tags, a wearable system for abstracting life in the form of clouds of concepts automatically extracted from videos captured by lifeloggers. We show how Life-Tags can be extended with a mobile application and cloud-based services, the Firebase Realtime Database and Cloud Storage, toward integrated lifelogging and mobile crowdsensing, where the life tags of mobile and wearable users are queries for potential matches regarding specific objects and concepts of interest. We conclude with implications for the future integration of lifelogging technology, mobile and wearable computing, and crowdsensing.
- Hinbarji Z., Albatal R., O'Connor N., and Gurrin C. 2016. LoggerMan, a Comprehensive Logging and Visualization Tool to Capture Computer Usage, 9517. Springer, Cham. https://doi.org/10.1007/978-3-319-27674-8_31Google Scholar
- Abdullah Al Mahmud, Jeffrey Braun, and Jean-Bernard Martens. 2010. Designing to Capture and Share Life Experiences for Persons with Aphasia. In Proceedings of the 12th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI '10). ACM, New York, NY, USA, 391--392. https://doi.org/10.1145/1851600.1851680Google ScholarDigital Library
- E. Berry, N. Kapur, L. Williams, S. Hodges, P. Watson, G. Smyth, J. Srinivasan, R. Smith, B. Wilson, and K. Wood. 2007. The use of a wearable camera, SenseCam, as a pictorial diary to improve autobiographical memory in a patient with limbic encephalitis: A preliminary report. Neuropsychol. Rehabil. 17, 4--5, 582--601. https://doi.org/10.1080/09602010601029780Google ScholarCross Ref
- Gary Stix. 2011. Photographic Memory: Wearable Cam Could Help Patients Stave Off Effects of Impaired Recall. Scientific American. https://www.scientificamerican.com/article/photographic-memory-wearable/Google Scholar
- Phil Hall. 2019. Best action camera 2019: 10 cameras for the GoPro generation. https://www.techradar.com/news/best-actioncameraGoogle Scholar
- MeCam. 2019. High definition video camera | Best life logging device | Mini video camera - MeCam. Retrieved March, 2020 from https://mecam.me/products/mecam-hdGoogle Scholar
- Narrative. 2019. The World's Most Wearable HD Video Camera | Narrative Clip 2. http://getnarrative.com/Google Scholar
- SnapCam. 2019. iON USA | SnapCam. Retrieved March, 2020 from https://usa.ioncamera.com/snapcam/Google Scholar
- Google Clips. 2019. Google Clips Specifications. https://support.google.com/googleclips/answer/7545447?hl=enGoogle Scholar
- RedShed. 2018. The 8 Best Spy Glasses of 2018. Retrieved March, 2020 from https://redshed.co.uk/tech-reviews/best-spyglasses/Google Scholar
- Snap. 2019. Spectacles by Snapchat | Your HandsFree Camera. Retrieved March, 2020 from https://www.spectacles.comGoogle Scholar
- Adrian Aiordachioae, Radu-Daniel Vatavu. 2019. Life-Tags: A Smartglasses-based System for Recording and Abstracting Life with Tag Clouds. Proc. ACM Hum.-Comput. Interact. 3, EICS, Article 15 (June 2019), 22 pages. https://doi.org/10.1145/3331157Google ScholarDigital Library
- Marion Koelle, Swamy Ananthanarayan, Simon Czupalla, Wilko Heuten, and Susanne Boll. 2018. Your smart glasses' camera bothers me! exploring opt-in and opt-out gestures for privacy mediation. In Proceedings of the 10th Nordic Conference on Human-Computer Interaction (NordiCHI '18). ACM, New York, NY, USA, 473--481. https://doi.org/10.1145/3240167.3240174Google ScholarDigital Library
- Shilin Zhu, Chi Zhang, and Xinyu Zhang. 2017. Automating Visual Privacy Protection Using a Smart LED. In Proceedings of the 23rd Annual International Conference on Mobile Computing and Networking (MobiCom '17). ACM, New York, NY, USA, 329--342. https://doi.org/10.1145/3117811.3117820Google ScholarDigital Library
- Marion Koelle, Swamy Ananthanarayan, Simon Czupalla, Wilko Heuten, and Susanne Boll. 2018. Your smart glasses' camera bothers me! exploring opt-in and opt-out gestures for privacy mediation. In Proceedings of the 10th Nordic Conference on Human-Computer Interaction (NordiCHI '18). ACM, New York, NY, USA, 473--481. https://doi.org/10.1145/3240167.3240174Google ScholarDigital Library
- Yuhang Zhao, Sarit Szpiro, Jonathan Knighten, Shiri Azenkot. 2016. CueSee: exploring visual cues for people with low vision to facilitate a visual search task. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '16), ACM, New York, NY, USA, 73--84. https://doi.org/10.1145/2971648.2971730Google ScholarDigital Library
- R.K. Ganti, F. Ye, H. Lei. 2011. Mobile crowdsensing: current state and future challenges. IEEE Communications Magazine, 49(11), 32--39. http://dx.doi.org/10.1109/MCOM.2011.6069707Google ScholarCross Ref
- Haggai Roitman, Jonathan Mamou, Sameep Mehta, Aharon Satt, L.V. Subramaniam. 2012. Harnessing the crowds for smart city sensing. In Proceedings of the 1st International Workshop on Multimodal crowd sensing (CrowdSens '12), 17--18. https://doi.org/10.1145/2390034.2390043Google ScholarDigital Library
- Ruipeng Gao, Mingmin Zhao, Tao Ye, Fan Ye, Yizhou Wang, Kaigui Bian, Tao Wang, Xiaoming Li. 2014. Jigsaw: indoor floor plan reconstruction via mobile crowdsensing. In Proc. of the 20th Annual International Conference on Mobile Computing and Networking, 249--260. https://doi.org/10.1145/2639108.2639134Google ScholarDigital Library
- Leelou Private Eye - Home. Retrieved March, 2020 from https://leelouprivateeye.com/index.php/en/Google Scholar
- Jeffrey P. Bigham, Chandrika Jayant, Hanjie Ji, Greg Little, Andrew Miller, Robert C. Miller, Robin Miller, Aubrey Tatarowicz, Brandyn White, Samual White, Tom Yeh. 2010. VizWiz: nearly real-time answers to visual questions. In Proceedings of the 23nd annual ACM symposium on User interface software and technology (UIST '10), 333--342. https://doi.org/10.1145/1866029.1866080Google ScholarDigital Library
- TapTapSee - Blind and Visually Impaired Assistive Technology -powered by CloudSight.ai Image Recognition API. Retrieved March, 2020 from https://taptapseeapp.com/Google Scholar
- Cole Gleason, Anhong Guo, Gierad Laput, Kris Kitani, Jeffrey P. Bigham. 2016. VizMap: Accessible Visual Information Through Crowdsourced Map Reconstruction. In Proc. of the 18th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '16), 273--274. https://doi.org/10.1145/2982142.2982200Google ScholarDigital Library
- Firebase Realtime Database. Retrieved from https://firebase.google.com/docs/databaseGoogle Scholar
- Cloud Storage: Object Storage. Google Cloud. Retrieved March, 2020 from https://cloud.google.com/storageGoogle Scholar
- Vision AI | Derive Image Insights via ML | Cloud Vision API. Retrieved March, 2020 from https://cloud.google.com/visionGoogle Scholar
- Gartner Says Global Smartphone Sales Will Grow 3% in 2020. https://www.gartner.com/en/newsroom/press-releases/2020-01-28-gartner-says-worldwide-smartphone-sales-will-grow-3--Google Scholar
- Gartner Says Global End-User Spending on Wearable Devices to Total $52 Billion in 2020. Retrieved March, 2020 https://www.gartner.com/en/newsroom/press-releases/2019-10-30-gartner-says-global-end-user-spending-on-wearable-devGoogle Scholar
- Cathal Gurrin, Alan F. Smeaton, and Aiden R. Doherty. 2014. LifeLogging: Personal Big Data. Found. Trends Inf. Retr. 8, 1 (June 2014), 1--125. https://doi.org/10.1561/1500000033Google ScholarDigital Library
- Steve Hodges, Lyndsay Williams, Emma Berry, Shahram Izadi, James Srinivasan, Alex Butler, Gavin Smyth, Narinder Kapur, and Ken Wood. 2006. SenseCam: A Retrospective Memory Aid. In Proceedings of the 8th International Conference on Ubiquitous Computing (UbiComp'06). Springer-Verlag, Berlin, Heidelberg, 177--193. https://doi.org/10.1007/11853565_11Google ScholarDigital Library
- Steve Mann, James Fung, and Eric Moncrieff. 1999. EyeTap Technology for Wireless Electronic News Gathering. SIGMOBILE Mob. Comput. Commun. Rev. 3, 4 (Oct. 1999), 19--26. https://doi.org/10.1145/584039.584044Google ScholarDigital Library
- Dirk de Jager, Alex L. Wood, Geoff V. Merrett, Bashir M. Al-Hashimi, Kieron O'Hara, Nigel R. Shadbolt, and Wendy Hall. 2011. A Low-power, Distributed, Pervasive Healthcare System for Supporting Memory. In Proceedings of the First ACM MobiHoc Workshop on Pervasive Wireless Healthcare (MobileHealth '11). https://doi.org/10.1145/2007036.2007043Google ScholarDigital Library
- Mark Blum, Alex (Sandy) Pentland, and Gerhard Troster. 2006. InSense: Interest-Based Life Logging. IEEE MultiMedia 13, 4 (Oct. 2006), 40--48. https://doi.org/10.1109/MMUL.2006.87Google ScholarDigital Library
- Keigo Kitamura, Toshihiko Yamasaki, and Kiyoharu Aizawa. 2008. Food Log by Analyzing Food Images. In Proceedings of the 16th ACM International Conference on Multimedia (MM '08). ACM, New York, NY, USA, 999--1000. https://doi.org/10.1145/1459359.1459548Google ScholarDigital Library
- Vangelis Metsis, Dimitrios Kosmopoulos, Vassilis Athitsos, and Fillia Makedon. 2014. Non-invasive Analysis of Sleep Patterns via Multimodal Sensor Input. Personal and Ubiquitous Computing 18, 1 (Jan. 2014), 19--26. https: //doi.org/10.1007/s00779-012-0623-1Google ScholarDigital Library
- Olivier Augereau, Charles Lima Sanches, Koichi Kise, and Kai Kunze. 2018. Wordometer Systems for Everyday Life. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 1, 4, Article 123 (Jan. 2018), 21 pages. https://doi.org/10.1145/3161601Google ScholarDigital Library
- Adrian Aiordăchioae, Radu-Daniel Vatavu, and Dorin-Mircea Popovici. 2019. A Design Space for Vehicular LifeLogging to Support Creation of Digital Content in Connected Cars. In Proceedings of the 11th ACM SIGCHI Symposium on Engineering Interactive Computing Systems (EICS '19), Article 9, 1--6. https://doi.org/10.1145/3319499.3328234Google ScholarDigital Library
- Joshua McVeigh-Schultz, Jennifer Stein, Jacob Boyle, Emily Duff, Jeff Watson, Avimaan Syam, Amanda Tasse, Simon Wiscombe, and Scott Fisher. 2012. Vehicular Lifelogging: New Contexts and Methodologies for Human-car Interaction. In CHI '12 Extended Abstracts on Human Factors in Computing Systems (CHI EA '12). https://doi.org/10.1145/2212776.2212800Google Scholar
- Jim Gemmell. 2014. Life-logging, Thing-logging and the Internet of Things. In Proceedings of the 2014 Workshop on Physical Analytics (WPA '14). ACM, New York, NY, USA, 17--17. https://doi.org/10.1145/2611264.2611276Google ScholarDigital Library
- Floriano Zini, Martin Reinstadler, and Francesco Ricci. 2015. Life-logs Aggregation for Quality of Life Monitoring. In Proceedings of the 5th International Conference on Digital Health 2015 (DH '15). ACM, New York, NY, USA, 131--132. https://doi.org/10.1145/2750511.2750531Google ScholarDigital Library
- CloudSight Image Recognition API. https://cloudsight.aiGoogle Scholar
- Bogdan Pogorelc, Artur Lugmayr, Bjorn Stockleben, Radu-Daniel Vatavu, Nina Tahmasebi, Estefania Serral, Emilija Stojmenova, Bojan Imperl, Thomas Risse, Gideon Zenz, Matjaz Gams. 2013. Ambient Bloom: New Business, Content, Design and Models to Increase the Semantic Ambient Media Experience. Multimedia Tools and Appl. 66, 7--32. https://doi.org/10.1007/s11042-012-1228-4Google ScholarDigital Library
- Radu-Daniel Vatavu. 2012. Presence bubbles: supporting and enhancing human-human interaction with ambient media. Multimedia Tools and Applications 58, 2 (May 2012), 371--383. https://doi.org/10.1007/s11042-010-0674-0Google ScholarDigital Library
- Eyeglasses Hidden WiFi camera. Spy Shop Round Rock. https://spyshoproundrock.com/product/cameras/eyeglasses-hidden-wifi-cameraGoogle Scholar
- Clarifai. Enterprise AI Powered Computer Vision Technology Solutions | Clarifai. https://www.clarifai.comGoogle Scholar
- WordCloud 1.0.0.3. https://nuget.org/packages/WordCloud/Google Scholar
- Liting Zhou, Zaher Hinbarji, Duc-Tien Dang-Nguyen, Cathal Gurrin. 2018. LIFER: An Interactive Lifelog Retrieval System. In Proceedings of the 2018 ACM Workshop on The Lifelog Search Challenge (LSC '18). ACM, New York, NY, USA, 9--14. https://doi.org/10.1145/3210539.3210542Google ScholarDigital Library
- Google Maps Platform Documentation. Retrieved March, 2020 from https://developers.google.com/maps/documentationGoogle Scholar
- Abdullah X. Ali, Meredith Ringel Morris, Jacob O. Wobbrock. 2018. Crowdsourcing Similarity Judgments for Agreement Analysis in End-User Elicitation Studies. UIST '18, 177--188. https://doi.org/10.1145/3242587.3242621Google Scholar
- Ovidiu-Andrei Schipor, Radu-Daniel Vatavu, Wenjun Wu. 2019. SAPIENS: Towards Software Architecture to Support Peripheral Interaction in Smart Environments. Proc. ACM Hum.-Comput. Interact. 3, EICS, Article 11 (June 2019), 24 pages. https://doi.org/10.1145/3331153Google ScholarDigital Library
- Ovidiu-Andrei Schipor, Radu-Daniel Vatavu, Jean Vanderdonckt. 2019. Euphoria: A Scalable, Event-Driven Architecture for Designing Interactions Across Heterogeneous Devices in Smart Environments Information and Software Technology 109. Elsevier, 43--59. https://doi.org/10.1016/j.infsof.2019.01.006Google Scholar
- Jean-Yves Lionel Lawson, Jean Vanderdonckt, Radu-Daniel Vatavu. 2018. Mass-Computer Interaction for Thousands of Users and Beyond. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (CHI EA '18). Association for Computing Machinery, New York, NY, USA, Paper LBW032, 1--6. https://doi.org/10.1145/3170427.3188465Google Scholar
- Marion Koelle, Swamy Ananthanarayan, Susanne Boll. 2020. Social Acceptability in HCI: A Survey of Methods, Measures, and Design Strategies. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI '20). Association for Computing Machinery, New York, NY, USA, 1 -19. https://doi.org/10.1145/3313831.3376162Google ScholarDigital Library
Index Terms
- Aggregating Life Tags for Opportunistic Crowdsensing with Mobile and Smartglasses Users
Recommendations
Life-Tags: A Smartglasses-based System for Recording and Abstracting Life with Tag Clouds
We introduce "Life-Tags," a wearable, smartglasses-based system for abstracting life in the form of clouds of tags and concepts automatically extracted from snapshots of the visual reality recorded by wearable video cameras. Life-Tags summarizes users' ...
Wearables can help me learn: A survey of user perception of wearable technologies for learning in everyday life
AbstractWearable devices are a popular class of portable ubiquitous technology. These devices are available in a variety of forms, ranging from smart glasses to smart rings. The fact that smart wearable devices are attached to the body makes them ...
Inferring Multiple Activities of Mobile Users with Activity Algebra
MDM '11: Proceedings of the 2011 IEEE 12th International Conference on Mobile Data Management - Volume 02This paper proposes a novel multiple activity recognition and reasoning approach where we use on-body sensor information along with other context information to infer mobile user activities which are both concurrent and interleaved. We develop and ...
Comments