Skip to main content
SpringerLink
Log in

A data association approach to detect and organize people in personal photo collections

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In this paper we present a method to automatically segment a photo sequence in groups containing the same persons. Many methods in literature accomplish to this task by adopting clustering techniques. We model the problem as the search for probable associations between faces detected in subsequent photos considering the mutual exclusivity constraint: a person can not be in a photo two times, nor two faces in the same photo can be assigned to the same group. Associations have been found considering face and clothing descriptions. In particular, a two level architecture has been adopted: at the first level, associations are computed within meaningful temporal windows (situations); at the second level, the resulting clusters are re-processed to find associations across situations. Experiments confirm our technique generally outperforms clustering methods. We present an analysis of the results on a public dataset, enabling future comparison, and on private collections.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. In Matlab this computation can be easily performed by means of the gamfit function.

References

  1. Ahonen T, Hadid A, Pietikäinen M (2004) Face recognition with local binary patterns. In: Proc. of European Conference on Computer Vision (ECCV 2004), pp 469–481

  2. Androutsos D, Plataniotiss K, Venetsanopoulos A (1998) Distance measures for color image retrieval. In: Proc. of International Conference on Image Processing (ICIP 98), vol 2. IEEE, pp 770–774

  3. Ardizzone E, Cascia ML, Morana M, Vella F (2009) Clustering techniques for personal photo album management. J Electron Imaging 18(4):043014

    Google Scholar 

  4. Choi J, Yang S, Ro Y, Plataniotis K (2008) Face annotation for personal photos using context-assisted face recognition. In: Proc. of international conference on Multimedia Information Retrieval (MIR 2008), pp 44–51

  5. Chu WT, Lee YL, Yu JY (2009) Using context information and local feature points in face clustering for consumer photos. In: Proc. of International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2009), pp 1141–1144

  6. Davis M, Smith M, Stentiford F, Bamidele A, Canny J, Good N, King S, Janakiraman R (2006) Using context and similarity for face and location identification. In: Proceedings of the IS&T/SPIE 18th annual symposium on electronic imaging science and technology, vol 6061. pp 119–127

  7. Edmonds J, Karp R (1972) Theoretical improvements in algorithmic efficiency for network flow problems. JACM 19(2):248–264

    Article  MATH  Google Scholar 

  8. El-Khoury E, Senac C, Joly P (2010) Face-and-clothing based people clustering in video content. In: Proc. of international conference on Multimedia Information Retrieval (MIR 2010), ACM, pp 1–10

  9. Gallagher A, Chen T (2008) Clothing cosegmentation for recognizing people. In: Proc. of Computer Vision and Pattern Recognition (CVPR 2008). IEEE

  10. Gross J, Yellen J (2006) Graph theory and its applications. CRC press

  11. Heyer LJ, Kruglyak S, Yooseph S (1999) Exploring expression data: Identification and analysis of coexpressed genes. Genome Res 9(11):1106–1115. doi:10.1101/gr.9.11.1106

    Article  Google Scholar 

  12. Huang GB, Jain V, Learned-Miller E (2007) Unsupervised joint alignment of complex images. In: Proc. of international conference on computer vision (ICCV 2007)

  13. Huang T, Russell S (1997) Object identification in a bayesian context. Int Joint Conf on Artificial Intel 15:1276–1283

    Google Scholar 

  14. iPhoto (2002) http://www.apple.com/ilife/iphoto

  15. Jain A, Murty M, Flynn P (1999) Data clustering: a review. ACM Comput Surv (CSUR) 31(3):264–323

    Article  Google Scholar 

  16. Jain R, Sinha P (2010) Content without context is meaningless. In: Proc. of conference on Multimedia (MM 2010)

  17. Kakumanu P, Makrogiannis S, Bourbakis N (2007) A survey of skin-color modeling and detection methods. Pattern Recogn 40(3):1106–1122

    Article  MATH  Google Scholar 

  18. Kang H, Shneiderman B (2000) Visualization methods for personal photo collections: Browsing and searching in the photofinder. In: Proc. of International Conference on Multimedia & Expo (ICME 2000)

  19. Kuhn H (1955) The Hungarian method for the assignment problem. Nav Res Logist Q 2(1–2):83–97

    Article  Google Scholar 

  20. Lawless J (1982) Statistical models and methods for lifetime data. Wiley, New York

    MATH  Google Scholar 

  21. Lo Presti L, Morana M, La Cascia M (2010) A data association algorithm for people re-identification in photo sequences. In: Proc. of IEEE International Symposium on Multimedia (ISM 2010). IEEE

  22. Ojala T, Pietikäinen M, Mäenpää T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24(7):971–987

    Google Scholar 

  23. Phillips P, Wechsler H, Huang J, Rauss P (1998) The FERET database and evaluation procedure for face-recognition algorithms. Image and Vision Computing 16(5):295–306

    Article  Google Scholar 

  24. Phillips P, Moon H, Rizvi S, Rauss P (2002) The FERET evaluation methodology for face-recognition algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(10):1090–1104

    Article  Google Scholar 

  25. Picasa (2004) http://picasa.google.com

  26. Rubner Y, Tomasi C, Guibas L (1998) A metric for distributions with applications to image databases. Proc of International Conference on Computer Vision, (ICCV 1998)

  27. Sivic J, Zitnick C, Szeliski R (2006) Finding people in repeated shots of the same scene. In: Proc. of British Machine Vision Conference (BMVC 2006) 3:909–918

  28. Song Y, Leung T (2006) Context-aided human recognition—clustering. In: Computer Vision—ECCV 2006, Lecture Notes in Computer Science, vol 3953. Springer, Berlin, Heidelberg, pp 382–395

    Chapter  Google Scholar 

  29. Tan X, Triggs B (2007) Enhanced local texture feature sets for face recognition under difficult lighting conditions. In: Proc. of international conference on analysis and modeling of faces and gestures. Springer, pp 168–182

  30. Turk M, Pentland A (1991) Eigenfaces for recognition. J Cogn Neurosci 3(1):71–86

    Article  Google Scholar 

  31. Viola P, Jones MJ (2004) Robust real-time face detection. Int J Comput Vis 57(2):137–154

    Article  Google Scholar 

  32. Zhang L, Chen L, Li M, Zhang H (2003) Automated annotation of human faces in family albums. In: Proc. of Conference on Multimedia (MM 2003), pp 355–358

  33. Zhang L, Hu Y, Li M, Ma W, Zhang H (2004) Efficient propagation for face annotation in family albums. In: Proc of Conference on Multimedia (MM 2004), pp 716–723

  34. Zhao M, Teo Y, Liu S, Chua TS, Jain R (2006) Automatic person annotation of family photo album. In: Image and video retrieval. Lecture notes in computer science, vol 4071. Springer, Berlin, Heidelberg, pp 163–172

    Chapter  Google Scholar 

Download references

Acknowledgement

We thank all the anonymous reviewers whose insightful comments led to significant improvements of the manuscript.

Author information

Authors and Affiliations

  1. University of Palermo, Palermo, Italy

    Liliana Lo Presti, Marco Morana & Marco La Cascia

Authors
  1. Liliana Lo Presti
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marco Morana
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marco La Cascia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Liliana Lo Presti.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lo Presti, L., Morana, M. & La Cascia, M. A data association approach to detect and organize people in personal photo collections. Multimed Tools Appl 61, 321–352 (2012). https://doi.org/10.1007/s11042-011-0839-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-011-0839-5

Keywords

Search

Navigation