Skip to main content
SpringerLink
Log in

Instance based personalized multi-form image browsing and retrieval

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

It is important to adapt and personalize image browsing and retrieval systems based on users’ preferences for improved user experience and satisfaction. In this paper, we present a novel instance based personalized multi-form image representation with implicit relevance feedback and adaptive weighting approach for image browsing and retrieval systems. In the proposed system, images are grouped into forms, which represent different information on images such as location, content etc. We conducted user interviews on image browsing, sharing and retrieval systems for understanding image browsing and searching behaviors of users. Based on the insights gained from the user interview study we propose an adaptive weighting method and implicit relevance feedback for multi-form structures that aim to improve the efficiency and accuracy of the system. Statistics of the past actions are considered for modeling the target of the users. Thus, on each iteration weights of the forms are updated adaptively. Moreover, retrieval results are modified according to the users’ preferences on iterations in order to improve personalized user experience. The proposed method has been evaluated and results are illustrated in the paper. It is shown that, satisfactory improvements can be achieved with proposed approaches in the multi-form scheme.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Benbunan-Fich R, Benbunan A (2007) Understanding user behavior with new mobile applications. J Strateg Inf Syst 16(4):393–412

    Article  Google Scholar 

  2. Bockting S, Ooms M, Hiemstra D, Vet PVD, Huibers T (2008) Evaluating relevance feedback: an image retrieval interface for children. In: Proceedings of the Dutch-Belgian Information Retrieval Workshop, 14–15 Apr, pp 15–20

  3. Covey DT (2002) Usage and usability assessment: library practices and concerns. Digital Library Federation, Council on Library and Information Resources reports, January

  4. Djordjevic D, Izquierdo E (2007) An object- and user-driven system for semantic-based image annotation and retrieval. IEEE Trans Circuits Syst Video Technol 17(3):313–323

    Article  Google Scholar 

  5. Eakins JP, Briggs P, Burford B (2004) Image retrieval interfaces: a user perspective. In: Proceedings of Third International Conference on Image and Video Retrieval, CIVR 2004, Proceedings of Lecture Notes in Computer Science 3115, Dublin, Ireland, July 21–23, pp 628–637

  6. Fei-Fei L, Fergus R, Perona P (2004) Learning generative visual models from few training examples: an incremental Bayesian approach tested on 101 object categories. IEEE. CVPR 2004, Workshop on Generative-Model Based Vision

  7. Gray WD, Altmann EM (2001) Cognitive modeling and human-computer interaction. In: Karwowski W (ed) International encyclopedia of ergonomics and human factors, vol 1, pp 387–391

  8. Guldogan E, Gabbouj M (2010) Adaptive image classification based on folksonomy. Proceedings of the International Workshop on Image Analysis for Multimedia Interactive Services, WIAMIS 2010, Italy, 12–14 April, pp 1–4

  9. Guldogan E, Lagerstam E, Olsson T, Gabbouj M (2010) Multi-form hierarchical representation of image categories for browsing and retrieval. Proceedings of the SMAP 2010, 5th International Workshop on Semantic Media Adaptation and Personalization, Cyprus, December, pp 64–69

  10. Huiskes MJ, Lew MS (2008) The MIR Flickr Retrieval Evaluation. ACM International Conference on Multimedia Information Retrieval (MIR‘08), Vancouver, Canada, pp 39–43

  11. Jaimes A (2006) Human factors in automatic image retrieval system design and evaluation. Invited paper, IS&T/SPIE Internet Imaging 2006, San Jose, CA, SPIE 6061, 606103, January

  12. Jing F, Li M, Zhang H-J, Zhang B (2004) Relevance feedback in region-based image retrieval. IEEE Trans Circuits Syst Video Technol 14:672

    Article  Google Scholar 

  13. Kelly D, Belkin NJ (2001) Reading time, scrolling and interaction: Exploring implicit sources of user preference for relevance feedback. In: Proceedings of the 24th Annual International ACM Conference on Research and Development in Information Retrieval (SIGIR ‘01), USA, pp 408–409

  14. Kim YH, Rhee PK. Automatic adaptation method in intelligent image retrieval system. Proceedings of the IEEE Region 10 Conference TENCON 99. South Korea, vol 1, pp 439–442

  15. Kosch H, Döller M (2005) Multimedia database systems: where are we now? In: Proceedings of Int. Assoc. of Science and Technology for Development—Databases and Applications (IASTED-DBA), Innsbruck, Austria

  16. Kuniavsky M (2003) Observing the user experience: a practitioner’s guide to user research. Published by Morgan Kaufmann 560 p, pp 129–155

  17. Laaksonen J, Koskela M, Laakso S, Oja E (2000) PicSOM—content-based image retrieval with self-organizing maps. Pattern Recogn Lett 21:1199–1207

    Article  MATH  Google Scholar 

  18. Liu Y, Zhang D, Lu G, Ma W-Y (2007) A survey of content-based image retrieval with high-level semantics. Pattern Recognit 40(1):262–282

    Article  MATH  Google Scholar 

  19. Manavoglu E, Pavlov D, Lee Giles C (2003) Probabilistic user behavior models. IEEE International Conference On Data Mining, pp 203–210

  20. Moghaddam B, Tian Q, Lesh N, Shen C, Huang TS (2004) Visualization and user-modeling for browsing personal photo libraries. Int J Comput Vision 56(1–2):109–130

    Article  Google Scholar 

  21. Piras L, Giacinto G (2009) Neighborhood-based feature weighting for relevance feedback in content-based retrieval. Workshop on Image Analysis for Multimedia Interactive Services, London, UK, May 6–8, pp 238–241

  22. Rao Y, Mundur P, Yesha Y (2006) Fuzzy SVM ensembles for relevance feedback in image retrieval. Learning 350–359

  23. Robertson S (2001) Evaluation in information retrieval. Lecture Notes in Computer Science 1980, USA, pp 81–92

  24. Sandhaus P, Boll S (2011) Semantic analysis and retrieval in personal and social photo collections. Multimed Tools Appl 51:5–33

    Article  Google Scholar 

  25. Shen X, Tan B, Zhai C. Context-sensitive information retrieval using implicit feedback. Proceedings of The 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ‘05, 43, USA, pp 43–50

  26. Smeulders AWM, Worring M, Santini S, Gupta A, Jain R (2000) Content-based image retrieval at the end of the early years. IEEE Trans Pattern Anal Mach Intell 22(12):1349–1380

    Article  Google Scholar 

  27. Torres JM, Parkes A (2000) User modeling and adaptivity in visual information retrieval systems. In: Proceedings of the Workshop on Computational Semiotics for New Media

  28. Weiss D, Scheuerer J, Wenleder M, Erk A, Gülbahar M, Linnhoff-Popien C (2008) A user profile-based personalization system for digital multimedia content. In: Proceedings of the 3rd International Conference On Digital Interactive Media In Entertainment And Arts, DIMEA ‘08, Athens, Greece, September, vol 349, pp 281–288

  29. Zhou XS, Huang TS (2000) CBIR: from low-level features to high-level semantics. In: Proceedings of SPIE Image and Video Communication and Processing, pp 24–28

  30. Zhou X, Huang TS (2003) Relevance feedback for image retrieval: a comprehensive review. ACM Multimed Syst 8:536–554

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Signal Processing, Tampere University of Technology, Tampere, Finland

    Esin Guldogan & Moncef Gabbouj

  2. Unit of Human-Centered Technology, Tampere University of Technology, Tampere, Finland

    Thomas Olsson & Else Lagerstam

Authors
  1. Esin Guldogan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thomas Olsson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Else Lagerstam
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Moncef Gabbouj
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Esin Guldogan.

Additional information

This work is supported by Devices and Interoperability Ecosystem—DIEM—project is part of the Finnish ICT SHOK program coordinated by TIVIT and funded by Finnish Funding Agency for Technology and Innovation.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Guldogan, E., Olsson, T., Lagerstam, E. et al. Instance based personalized multi-form image browsing and retrieval. Multimed Tools Appl 71, 1087–1104 (2014). https://doi.org/10.1007/s11042-012-1249-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-012-1249-z

Keywords

Search

Navigation