Abstract
With lifelogging devices; such as wearable camera, smart watches, audio recorder or standalone smartphone applications; capturing daily moments becomes easier. In recent years, many workshops and panels have emerged and proposed benchmarks to face challenges in organizing, analyzing, managing, indexing and retrieving specific moments in the huge amount of multi-modal lifelog dataset. Recent advances in deep neural networks have given rise to new approaches to deep learning-based image retrieval. However, using deep neural networks in lifelog context systems is continuously rising challenges: relying on a convolutional neural network which is trained on images not related to the retrieval dataset reduced the performance to extract features. In this paper, we propose a novel fine-tuned Convolutional Neural Network approach based on a Long Short Term Memory processing for improving lifelog image retrieval. The experimental results show the feasibility and effectiveness of our approach with encouraging performance by reaching third place in the ImageCLEF Lifelog Moment Retrieval Task 2018.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ben Abdallah, F., Feki, G., Ben Ammar, A., Ben Amar, C.: A new model driven architecture for deep learning-based multimodal lifelog retrieval. In: Poster Proceedings: International Conference WSCG, CSRN 2803, Czech Republic, pp. 8–17 (2018)
Ben Abdallah, F., Feki, G., Ezzarka, M., Ben Ammar, A., Ben Amar, C.: Regim lab team at ImageCLEF lifelog moment retrieval task 2018. In: Working Notes of CLEF 2018, France (2018)
Ben Abdallah, F., Feki, G., Ben Ammar, A., Ben Amar, C.: Multilevel deep learning-based processing for lifelog image retrieval enhancement. In: Proceedings of the IEEE International Conference SMC, Japan, pp. 1348–1354 (2018)
Ai, Q., Yang, L., Guo, J., Croft, W.B.: Improving language estimation with the paragraph vector model for ad-hoc retrieval. In: Proceedings of the 39th International ACM SIGIR, USA, pp. 869–872 (2016)
Babenko, A., Slesarev, A., Chigorin, A., Lempitsky, V.: Neural codes for image retrieval. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 584–599. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10590-1_38
Browne, G., et al.: SenseCam improves memory for recent events and quality of life in a patient with memory retrieval difficulties. Memory 19, 713–722 (2011)
Crawford, K., Calo, R.: There is a blind spot in AI research. Nature 538(7625), 311 (2016)
Dang-Nguyen, D.-T., Piras, L., Riegler, M., Zhou, L., Lux, M., Gurrin, C.: Overview of ImageCLEFlifelog 2018: daily living understanding and lifelog moment retrieval. In: CLEF2018 Working Notes, CEUR-WS.org (2018)
Datta, R., Li, J., Wang, J.: Content-based image retrieval: approaches and trends of the new age. In: Multimedia Information Retrieval, pp. 253–262 (2005)
Dogariu, M., Ionescu, B.: Multimedia lab@ ImageCLEF 2018 lifelog moment retrieval task. In: Working Notes of CLEF 2018 - Conference and Labs of the Evaluation Forum, Avignon, France (2018)
Ganguly, D., Roy, D., Mitra, M., Jones, G.J.: Word embedding based generalized language model for information retrieval. In: Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 795–798. ACM (2015)
Girshick, R.: Fast R-CNN. In: Proceedings of the 2015 IEEE ICCV, Chile, pp. 1440–1448 (2015)
Gurrin, C., et al.: Overview of NTCIR-13 Lifelog-2 task. In: Proceedings of the Thirteenth NTCIR conference, Japan (2017)
Kavallieratou, E., del-Blanco, C.-R., Cuevas, C., Garcia, N.: Retrieving events in life logging. In: CLEF (Working Notes), CEUR-WS.org, France, vol. 2125 (2018)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Proceedings of the 25th International Conference on Neural Information Processing Systems, NIPS 2012, USA, vol. 1, pp. 1097–1105 (2012)
Mégret, R., et al.: The IMMED project: wearable video monitoring of people with age dementia. In: Proceedings of the 18th ACM International Conference on Multimedia, pp. 1299–1302 (2010)
Mikolov, T., Yih, W.-T., Zweig, G.: Linguistic regularities in continuous space word representations. In: Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 746–751 (2013)
Lin, J., et al.: VCI2R at the NTCIR-13 Lifelog-2 lifelog semantic access task. In: Proceedings 13th NTCIR Conference, Japan (2017)
O’Loughlin, G., et al.: Using a wearable camera to increase the accuracy of dietary analysis. Am. J. Prev. Med. 44, 297–301 (2013)
Oliveira Barra, G., Ayala, A.C., Bolanos, M., Dimiccoli, M., Giro i Nieto, X., Radeva P.: LEMoRe: a lifelog engine for moments retrieval at the NTCIR-lifelog LSAT task. In: Proceedings of the 12th NTCIR Conference, Japan (2016)
Oliveira-Barra, G., Dimiccoli, M., Radeva, P.: Leveraging activity indexing for egocentric image retrieval. In: Alexandre, L., Salvador Sánchez, J., Rodrigues, J. (eds.) IbPRIA 2017. LNCS, vol. 10255, pp. 295–303. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-58838-4_33
Redmon, J., Farhadi, A.: YOLO9000: better, faster, stronger. In: Proceedings of the IEEE Conference on CVPR, pp. 7263–7271 (2017)
Safadi, B., Mulhem, P., Quénot, G., Chevallet, J.-P.: LIG-MRIM at NTCIR-12 lifelog semantic access task. In: Proceedings of 12th NTCIR Conference, Japan (2016)
Tang, T., Fu, M., Huang, H., Chen, K., Chen, H.: Visual concept selection with textual knowledge for understanding activities of daily living and life moment retrieval. In: Working Notes of CLEF 2018, France (2018)
Tran, M., Truong, T., Duy, T.D., Vo-Ho, V., Luong, Q., Nguyen, V.: Lifelog moment retrieval with visual concept fusion and text-based query expansion. Working Notes of CLEF 2018, France (2018)
Yamamoto, S., Nishimura, T., Akagi, Y., Takimoto, Y., Inoue, T., Toda, H.: PBG at the NTCIR-13 lifelog-2 LAT, LSAT, and LEST tasks. In: Proceedings of the 13th NTCIR Conference, Japan (2017)
Zamani, H., Croft, W.B.: Estimating embedding vectors for queries. In: Proceedings of the 2016 ACM International Conference on the Theory of Information Retrieval, ICTIR 2016, New York, USA, pp. 123–132 (2016)
Zare, M.R., Woo, C.S., Ismail, J.N.: Comparative analysis of image retrieval approaches. In: Abu Osman, N.A., Ibrahim, F., Wan Abas, W.A.B., Abdul Rahman, H.S., Ting, H.N. (eds.) 4th Kuala Lumpur International Conference on Biomedical Engineering 2008. IFMBE Proceedings, vol. 21, pp. 847–850. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-69139-6_209
Zheng, L., Zhao, Y., Wang, S., Wang, J., Tian, Q.: Good Practice in CNN Feature Transfer, CoRR, abs/1604.00133 (2016)
Zhi, W., Chen, Z., Yueng, H.W.F., Lu, Z., Zandavi, S.M., Chung, Y.Y.: Layer removal for transfer learning with deep convolutional neural networks. In: Liu, D., Xie, S., Li, Y., Zhao, D., El-Alfy, E.S. (eds.) ICONIP 2017. LNCS, vol. 10635, pp. 460–469. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-70096-0_48
Zhou, L., Piras, L., Riegler, M., Boato, G., Dang-Nguyen, D., Gurrin, C.: Organizer team at ImageCLEFlifelog 2017: baseline approaches for lifelog retrieval and summarization. In: Working Notes of CLEF 2017, Ireland (2017)
Zhou, L., Piras, L., Riegler, M., Lux, M., Dang-Nguyen1, D.T., Gurrin, C.: An interactive lifelog retrieval system for activities of daily living understanding. In: Working Notes of CLEF 2018, France (2018)
Acknowledgments
The research leading to these results has received funding from the Ministry of Higher Education and Scientific Research of Tunisia under the grant agreement number LR11ES48.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Ben Abdallah, F., Feki, G., Ben Ammar, A., Ben Amar, C. (2019). Transfer Learning for Improving Lifelog Image Retrieval. In: Vento, M., Percannella, G. (eds) Computer Analysis of Images and Patterns. CAIP 2019. Lecture Notes in Computer Science(), vol 11678. Springer, Cham. https://doi.org/10.1007/978-3-030-29888-3_23
Download citation
DOI: https://doi.org/10.1007/978-3-030-29888-3_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-29887-6
Online ISBN: 978-3-030-29888-3
eBook Packages: Computer ScienceComputer Science (R0)