Thanh-Dat Truong
ABSTRACT
Lifelogging data provides useful insight understanding about our lives during daily activities. Thus, it is essential to develop a system to assist users to retrieve events or memories from lifelogging data from ad-hoc text queries. In this paper, we first propose a method to process lifelogging data by grouping images into visual shots and clusters, then extract semantic concepts on scene category and attributes, entities, and actions. We then develop a query system that supports 4 main types of query conditions: temporal, spatial, entity and action, and extra data criteria. Our system is expected to efficiently assist users to search for past moments in daily logs.
- R. Arandjelović and A. Zisserman. 2012. Three Things Everyone Should Know to Improve Object Retrieval. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (CVPR '12). Washington, DC, USA, 2911-- 2918. Google Scholar
Digital Library
- Duc-Tien Dang-Nguyen, Minh-Son Dao, Michael Riegler, and Cathal Gurrin. 2017. Smart Lifelogging: Recognizing Human Activities using PHASOR, Maria De Marsico, Gabriella Sanniti di Baja, and Ana Fred (Eds.). Proceedings of the 6th International Conference on Pattern Recognition Applications and Methods, 761--768.Google Scholar
- Tung Duy Dinh, Dinh-Hieu Nguyen, and Minh-Triet Tran. 2018. Social Relation Trait Discovery from Visual LifeLog Data with Facial Multi-Attribute Framework. In Proceedings of the 7th International Conference on Pattern Recognition Applications and Methods, ICPRAM 2018, Funchal, Madeira - Portugal, January 16--18, 2018. 665--674.Google ScholarCross Ref
- Aiden R. Doherty, Niamh Caprani, Ciar Conaire, Vaiva Kalnikaite, Cathal Gurrin, Alan F. Smeaton, and Noel E. O'Connor. 2011. Passively recognising human activities through lifelogging. Computers in Human Behavior 27, 5 (2011), 1948 -- 1958. 2009 Fifth International Conference on Intelligent Computing. Google ScholarCross Ref
- Aiden R. Doherty, Katalin Pauly-Takacs, Niamh Caprani, Cathal Gurrin, Chris J. A. Moulin, Noel E. O'Connor, and Alan F. Smeaton. 2012. Experiences of Aiding Autobiographical Memory Using the SenseCam. Human Computer Interaction 27, 1--2 (2012), 151--174. arXiv:https://www.tandfonline.com/doi/pdf/10.1080/07370024.2012.656050Google ScholarCross Ref
- Cathal Gurrin, Alan F. Smeaton, and Aiden R. Doherty. 2014. LifeLogging: Personal Big Data. Found. Trends Inf. Retr. 8, 1 (June 2014), 1--125. Google ScholarDigital Library
- Eddy Ilg, Nikolaus Mayer, Tonmoy Saikia, Margret Keuper, Alexey Dosovitskiy, and Thomas Brox. 2017. FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21--26, 2017. 1647--1655.Google ScholarCross Ref
- A. Karpathy and L. Fei-Fei. 2017. Deep Visual-Semantic Alignments for Generating Image Descriptions. IEEE Transactions on Pattern Analysis and Machine Intelligence 39, 4 (April 2017), 664--676. Google ScholarDigital Library
- Duy Dinh Le, Cai-Zhi Zhu, Sang Phan, Sabastien Poullot, Duc Anh Duong, and Shin'ichi Satoh. 2013. National Institute of Informatics, Japan at TRECVID 2013. In TRECVID. Orlando, Florida, USA.Google Scholar
- Vinh-Tiep Nguyen, Thanh Duc Ngo, Minh-Triet Tran, Duy-Dinh Le, and Duc Anh Duong. 2015. A Combination of Spatial Pyramid and Inverted Index for LargeScale Image Retrieval. IJMDEM 6, 2 (2015), 37--51. Google ScholarDigital Library
- Vinh-Tiep Nguyen, Khanh-Duy Le, Minh-Triet Tran, and Morten Fjeld. 2016. NowAndThen: A Social Network-based Photo Recommendation Tool Supporting Reminiscence. In Proceedings of the 15th International Conference on Mobile and Ubiquitous Multimedia (MUM '16). ACM, New York, NY, USA, 159--168. Google ScholarDigital Library
- J. Philbin, O. Chum, M. Isard, J. Sivic, and A. Zisserman. 2007. Object Retrieval with Large Vocabularies and Fast Spatial Matching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Google Scholar
- James Philbin, Michael Isard, Josef Sivic, and Andrew Zisserman. 2008. Lost in quantization: Improving particular object retrieval in large scale image databases. In In CVPR.Google Scholar
- Joseph Redmon and Ali Farhadi. 2016. YOLO9000: Better, Faster, Stronger. arXiv preprint arXiv:1612.08242 (2016).Google Scholar
- Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. In Advances in Neural Information Processing Systems (NIPS). Google ScholarDigital Library
- Josef Sivic and Andrew Zisserman. 2003. Video Google: A Text Retrieval Approach to Object Matching in Videos. In Proceedings of the Ninth IEEE International Conference on Computer Vision - Volume 2 (ICCV '03). IEEE Computer Society, Washington, DC, USA, 1470--. Google ScholarDigital Library
- Thanh-Dat Truong, Vinh-Tiep Nguyen, Minh-Triet Tran, Trang-Vinh Trieu, Tien Do, Thanh Duc Ngo, and Dinh-Duy Le. 2018. Video Search Based on Semantic Extraction and Locally Regional Object Proposal. In MultiMedia Modeling - 24th International Conference, MMM 2018, Bangkok, Thailand, February 5--7, 2018, Proceedings, Part II. 451--456.Google Scholar
- Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. 2017. Places: A 10 million Image Database for Scene Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence (2017).Google Scholar
- Bolei Zhou, Agata Lapedriza, Jianxiong Xiao, Antonio Torralba, and Aude Oliva. 2014. Learning Deep Features for Scene Recognition using Places Database. In Advances in Neural Information Processing Systems 27, Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger (Eds.). Curran Associates, Inc., 487--495. http://papers.nips.cc/paper/ 5349-learning-deep-features-for-scene-recognition-using-places-database.pdf Google ScholarDigital Library
- Cai-Zhi Zhu, Herve Jegou, and Shin'ichi Satoh. 2013. Query-Adaptive Asymmetrical Dissimilarities for Visual Object Retrieval. In IEEE International Conference on Computer Vision, ICCV 2013, Sydney, Australia, December 1--8, 2013. IEEE, 1705-- 1712. Google ScholarDigital Library
Index Terms
- Lifelogging Retrieval based on Semantic Concepts Fusion
Recommendations
Lifelogging memory appliance for people with episodic memory impairment
UbiComp '08: Proceedings of the 10th international conference on Ubiquitous computingLifelogging technologies have the potential to provide memory cues for people who struggle with episodic memory impairment (EMI). These memory cues enable the recollection of significant experiences, which is important for people with EMI to regain a ...
A review on video-based active and assisted living technologies for automated lifelogging
Highlights- Explores a topic of social interest: developing technologies for ageing populations.
AbstractProviding support for ageing and frail populations to extend their personal autonomy is desirable for their well-being as it is for the society at large, since it can ease the economic and social challenges caused by ever-ageing ...
Analysing privacy in visual lifelogging
The visual lifelogging activity enables a user, the lifelogger, to passively capture images from a first-person perspective and ultimately create a visual diary encoding every possible aspect of her life with unprecedented details. In recent years, it ...
Comments