Skip to main content
SpringerLink
Log in

Visualization and User-Modeling for Browsing Personal Photo Libraries

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

We present a user-centric system for visualization and layout for content-based image retrieval. Image features (visual and/or semantic) are used to display retrievals as thumbnails in a 2-D spatial layout or “configuration” which conveys all pair-wise mutual similarities. A graphical optimization technique is used to provide maximally uncluttered and informative layouts. Moreover, a novel subspace feature weighting technique can be used to modify 2-D layouts in a variety of context-dependent ways. An efficient computational technique for subspace weighting and re-estimation leads to a simple user-modeling framework whereby the system can learn to display query results based on layout examples (or relevance feedback) provided by the user. The resulting retrieval, browsing and visualization can adapt to the user's (time-varying) notions of content, context and preferences in style and interactive navigation. Monte Carlo simulations with machine-generated layouts as well as pilot user studies have demonstrated the ability of this framework to model or “mimic” users, by automatically generating layouts according to their preferences.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Balabanovic, M., Chu, L., and Wolff, G. 2000. Storytelling with digital photographs. In Proceedings of CHI2000, April. The Hague, The Netherlands.

  • Dietz, P. and Leigh, D. 2001. DiamondTouch: A multi-user touch technology. In Proceedings of ACM UIST'01, Nov., Orlando, FL.

  • Horoike, A. and Musha, Y. 2000. Similarity-based image retrieval system with 3-D visualization, In Proceedings of IEEE Int'l Conf. on Multimedia and Expo, New York, vol. 2, pp. 769–772.

    Google Scholar 

  • Jolliffe, I.T., Principal Component Analysis, Springer-Verlag, New York, 1986.

    Google Scholar 

  • Kang, H. and Shneiderman, B. 2000. Visualization methods for personal photo collections: Browsing and searching in the photofinder. In Proceedings of IEEE International Conference on Multimedia and Expo (ICME2000), New York City, New York.

  • Mimio, www.mimio.com/meet/mimiomouse.

  • Moghaddam, B., Tian, Q., and Huang, T.S. 2001. Spatial visualization for content-based image retrieval. In Proceedings of IEEE International Conference on Multimedia and Expo (ICME01), August 22–25, Tokyo, Japan.

  • Moghaddam, B., Tian, Q., Lesh, N., Shen, C., and Huang, T.S. 2002. PDH: A human-centric interface for image libraries. In Proceedings of IEEE International Conference on Multimedia and Expo (ICME02), Aug., vol. 1, pp. 901–904.

    Google Scholar 

  • Nakazato, M. and Huang, T.S. 2001. 3D MARS: Immersive virtual reality for content-based image retrieval. In Proceedings of IEEE Int'l Conf. on Multimedia and Expo, Aug. 22–25, Tokyo, Japan.

  • Popescu, M. and Gader, P. 1998. Image content retrieval from image databases using feature integration by choquet integral. SPIE Conference Storage and Retrieval for Image and Video Databases VII, San Jose, CA.

  • Rubner, Y. 1999. Perceptual metrics for image database navigation. Ph.D. dissertation, Stanford University.

  • Santini, S., Gupta, A., and Jain, R. 2001. “Emergent semantics through interaction in image databases,” IEEE Transactions on Knowledge and Data Engineering, 13(3):337–351.

    Google Scholar 

  • Santini, S. and Jain, R. 1999. Similarity measures. IEEE Transactions on PAMI, 21(9):871–883.

    Google Scholar 

  • Santini, S. and Jain, R. 2000. Integrated browsing and querying for image databases. IEEE Multimedia Magazine, July-September Issue, pp. 26–39.

  • Shen, C., Lesh, N., Moghaddam, B., Beardsley, P., and Bardsley, R. 2001. Personal digital historian: User interface design. In Proceedings of Extended Abstract of CHI 2001. April, Seattle WA, pp. 29–30.

  • Shen, C., Lesh, N., Vernier, F., Forlines, C., and Frost, J. 2002. Sharing and Building Digital Group Histories. In ACM Conference on Computer Supported Cooperative Work (CSCW), Nov.

  • Shen, C., Lesh, N., and Vernier, F. 2003. Personal digital historian: Story sharing around the table. ACM Interactions, March/April 2003 (also MERL TR2003–04).

  • Smeulders, A.W.M., Worring, M., Santini, S., Gupta, A., and Jain, R. 2000. Content-based image retrieval at the end of the early years. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(12):1349–1380.

    Google Scholar 

  • Smith, J.R. and Chang, S.F. 1994. Transform features for texture classification and discrimination in large image database. In Proc. IEEE Intl. Conf. on Image Proc., pp. 407–411.

  • Squire, D.M., Müller, H., and Müller, W. 1999. Improving response time by search pruning in a content-based image retrieval system, using inverted file techniques. In Proc. of IEEE Wrkshop on CBAIVL, June, pp. 45–49.

  • Stricker, M. and Orengo, M. 1995. Similarity of color Images. In Proc. SPIE Storage and Retrieval for Image and Video Databases.

  • Swets, D. and Weng, J. 1999. Hierarchical discriminant analysis for image retrieval. IEEE Transactions on PAMI, 21(5):386–401.

    Google Scholar 

  • Tian, Q., Moghaddam, B., and Huang, T.S. 2001. Display optimization for image browsing. In 2nd InternationalWorkshop on Multimedia Databases and Image Communications (MDIC'01), Sept. 17–18, 2001, Amalfi, Italy, pp. 167–173.

  • Tian, Q., Moghaddam, B., and Huang, T.S. 2002. Visualization, estimation and user-modeling for interactive browsing of image libraries. International Conference on Image and Video Retrieval (CIVR'02), July 18–19, London, UK, pp. 7–16

  • Torgeson, W.S. 1958. Theory and Methods of Scaling. John Wiley and Sons: New York, NY.

    Google Scholar 

  • Vernier, F., Lesh, N., and Shen, C. 2002 Visualization techniques for circular tabletop interface. In Proceedings of AVI'02, May, Trento, Italy, pp. 257–266.

  • Zhou, S.X., Rui, Y., and Huang, T.S. 1999. Water-filling algorithm: A novel way for image feature extraction based on edge maps. In Proc. IEEE Int'l. Conf. On Image Proc., Japan.

  • Zwillinger, D. (Ed.). 1995. Affine Transformations, Section 4.3.2 in CRC Standard Mathematical Tables and Formulae. CRC Press: Boca Raton, FL, pp. 265–266.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Mitsubishi Electric Research Laboratories, Cambridge, MA, 02139, USA

    Baback Moghaddam, Neal Lesh & Chia Shen

  2. Beckman Institute, University of Illinois, Urbana-Champaign, IL, 61801, USA

    Qi Tian & Thomas S. Huang

Authors
  1. Baback Moghaddam
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qi Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Neal Lesh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chia Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Thomas S. Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Moghaddam, B., Tian, Q., Lesh, N. et al. Visualization and User-Modeling for Browsing Personal Photo Libraries. International Journal of Computer Vision 56, 109–130 (2004). https://doi.org/10.1023/B:VISI.0000004834.62090.74

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/B:VISI.0000004834.62090.74

Search

Navigation