Article

EasyAlbum: an interactive photo annotation system based on face clustering and re-ranking

Authors:

Xiaoou TangAuthors Info & Claims

CHI '07: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Pages 367 - 376

https://doi.org/10.1145/1240624.1240684

Published: 29 April 2007 Publication History

Abstract

Digital photo management is becoming indispensable for the explosively growing family photo albums due to the rapid popularization of digital cameras and mobile phone cameras. In an effective photo management system photo annotation is the most challenging task. In this paper, we develop several innovative interaction techniques for semi-automatic photo annotation. Compared with traditional annotation systems, our approach provides the following new features: "cluster annotation" puts similar faces or photos with similar scene together, and enables user label them in one operation; "contextual re-ranking" boosts the labeling productivity by guessing the user intention; "ad hoc annotation" allows user label photos while they are browsing or searching, and improves system performance progressively through learning propagation. Our results show that these technologies provide a more user friendly interface for the annotation of person name, location, and event, and thus substantially improve the annotation performance especially for a large photo album.

References

[1]

iView Media Pro. http://www.iview-multimedia.com.

[2]

ACDSee. http://www.acdsee.com.

[3]

Microsoft Diginal Image Suite. http://www.microsoft.com/products/imaging.

[4]

Picasa. http://picasa.google.com.

[5]

Photoshop Elements. http://www.adobe.com/products/photoshopelwin.

[6]

Flickr. http://www.flickr.com.

[7]

ESP Game. http://espgame.org.

[8]

Google Image Labeler. http://images.google.com/imagelabeler.

[9]

Riya. http://www.riya.com.

[10]

T. Ahonen, A. Hadid, and M. Pietikainen. Face recognition with local binary patterns. In Proc. the 8th European Conference on Computer Vision, 2004.

[11]

F. Bach and M. Jordan. Learning spectral clustering. In Proc. of Neural Info. Processing Systems, 2003.

[12]

L. Chen, B. Hu, L. Zhang, M. Li, and H. Zhang. Face annotation for family photo album management. International Journal of Image and Graphics (IJIG), Special Issue on Multimedia Data Storage and Management, 3, 2003.

[13]

A. Girgensohn, J. Adcock, and L. Wilcox. Leveraging face recognition technology to find and organize photos. In MIR '04: Proceedings of the 6th ACM SIGMM international workshop on Multimedia information retrieval, pages 99--106, New York, NY, USA, 2004. ACM Press.

Digital Library

[14]

A. Graham, H. Garcia-Molina, A. Paepcke, and T. Winograd. Time as essence for photo browsing through personal digital libraries. In Joint Conference on Digital Libraries, pages 326--335, New York, NY, USA, 2002. ACM Press.

Digital Library

[15]

A. Kuchinsky, C. Pering, M. L. Creech, D. Freeze, B. Serra, and J. Gwizdka. Fotofile: a consumer multimedia organization and retrieval system. In CHI '99: Proceedings of the SIGCHI conference on Human factors in computing systems, pages 496--503, New York, NY, USA, 1999. ACM Press.

Digital Library

[16]

J. Platt. Autoalbum: Clustering digital photographs using probabalistic model merging, 2000.

[17]

J. C. Platt, M. Czerwinski, and B. A. Field. Phototoc: Automatic clustering for browsing personal photographs, 2002.

[18]

K. Rodden and K. R. Wood. How do people manage their digital photographs? In CHI '03: Proceedings of the SIGCHI conference on Human factors in computing systems, pages 409--416, New York, NY, USA, 2003. ACM Press.

Digital Library

[19]

B. Shneiderman and H. Kang. Direct annotation: A drag-and-drop strategy for labeling photos. In Proc. of IEEE Conf. on Information Visualization, 2000.

Digital Library

[20]

B. Suh and B. Bederson. Semi-automatic image annotation using event and torso identification. Technical report, Computer Science Department, University of Maryland, College Park, MD, 2004.

[21]

P. Viola and M. Jones. Rapid object detection using a boosted cascade of simple features. In Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, volume 1, pages 511--518, 2001.

[22]

L. von Ahn and L. Dabbish. Labeling images with a computer game. In CHI '04: Proceedings of the SIGCHI conference on Human factors in computing systems, pages 319--326, New York, NY, USA, 2004. ACM Press.

Digital Library

[23]

X. Wang and X. Tang. A unified framework for subspace face recognition. IEEE Trans. Pattern Anal. Mach. Intell., 26(9):1222--1228, 2004.

Digital Library

[24]

X. Wang and X. Tang. Random sampling for subspace face recognition. Int. J. Comput. Vision, 70(1):91--104, 2006.

Digital Library

[25]

L. Zhang, L. Chen, M. Li, and H. Zhang. Automated annotation of human faces in family albums. In Proc. of ACM Multimedia, 2003.

Digital Library

[26]

L. Zhang, Q. Yang, T. Bao, D. Vronay, and X. Tang. Imlooking: image-based face retrieval in online dating profile search. In CHI '06: CHI '06 extended abstracts on Human factors in computing systems, pages 1577--1582, New York, NY, USA, 2006. ACM Press.

Digital Library

[27]

W. Zhao, R. Chellappa, P. J. Phillips, and A. Rosenfeld. Face recognition: A literature survey. ACM Comput. Surv., 35(4):399--458, 2003.

Digital Library

[28]

Y. Zhou, L. Gu, and H.-J. Zhang. Bayesian tangent shape model: Estimating shape and pose parameters via bayesian inference. In Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, 2003.

Digital Library

[29]

Y. Zhou, W. Zhang, X. Tang, and H. Shum. A bayesian mixture model for multi-view face alignment. In CVPR '05: Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) -- Volume 2, pages 741--746, Washington, DC, USA, 2005. IEEE Computer Society.

Digital Library

Cited By

Shao HYuan X(2025)CataAnno: An Ancient Catalog Annotator for Annotation Cleaning by RecommendationIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.345637931:1(404-414)Online publication date: Jan-2025
https://doi.org/10.1109/TVCG.2024.3456379
Lyu HBai YLiang XDas UShi CGong LLi YSun MGe MMa X(2024)FARPLS: A Feature-Augmented Robot Trajectory Preference Labeling System to Assist Human Labelers’ Preference ElicitationProceedings of the 29th International Conference on Intelligent User Interfaces10.1145/3640543.3645145(344-369)Online publication date: 18-Mar-2024
https://dl.acm.org/doi/10.1145/3640543.3645145
Appiah OMartey E(2024)Concentric Blocking Techniques for Improved Feature Extraction in Local Binary Pattern (LBP) Systems2024 IEEE 9th International Conference on Adaptive Science and Technology (ICAST)10.1109/ICAST61769.2024.10856506(1-8)Online publication date: 24-Oct-2024
https://doi.org/10.1109/ICAST61769.2024.10856506
Show More Cited By

Index Terms

EasyAlbum: an interactive photo annotation system based on face clustering and re-ranking
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Graphical user interfaces
  2. Interaction design
    1. Interaction design process and methods
      1. Interface design prototyping
2. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
  2. Information systems applications
    1. Data mining
      1. Clustering

Recommendations

AutoTag 'n Search My Photos: Leveraging the Social Graph for Photo Tagging
WWW '15 Companion: Proceedings of the 24th International Conference on World Wide Web

Personal photo collections are large and growing rapidly. Today, it is difficult to search such a photo collection for people who occur in them since it is tedious to manually associate face tags in photos. The key idea is to learn face models for ...
Why we tag: motivations for annotation in mobile and online media
CHI '07: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Why do people tag? Users have mostly avoided annotating media such as photos -- both in desktop and mobile environments -- despite the many potential uses for annotations, including recall and retrieval. We investigate the incentives for annotation in ...
Photo-TaPE: user privacy preferences in photo tagging
WWW '12 Companion: Proceedings of the 21st International Conference on World Wide Web

Although they are used to expose pictures on the Web, users may not want to have a link between their identity and pictures without being able to modify them or control who accesses them. Photo tagging --- and more broadly face-recognition algorithms ---...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '07: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

April 2007

1654 pages

ISBN:9781595935939

DOI:10.1145/1240624

General Chair:
Mary Beth Rosson
The Pennsylvania State University, USA
,
Program Chair:
David Gilmore
Intel, USA

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 April 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

CHI07

Sponsor:

CHI07: CHI Conference on Human Factors in Computing Systems

April 28 - May 3, 2007

California, San Jose, USA

Acceptance Rates

CHI '07 Paper Acceptance Rate 182 of 840 submissions, 22%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

78
Total Citations
View Citations
1,573
Total Downloads

Downloads (Last 12 months)19
Downloads (Last 6 weeks)0

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Shao HYuan X(2025)CataAnno: An Ancient Catalog Annotator for Annotation Cleaning by RecommendationIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.345637931:1(404-414)Online publication date: Jan-2025
https://doi.org/10.1109/TVCG.2024.3456379
Lyu HBai YLiang XDas UShi CGong LLi YSun MGe MMa X(2024)FARPLS: A Feature-Augmented Robot Trajectory Preference Labeling System to Assist Human Labelers’ Preference ElicitationProceedings of the 29th International Conference on Intelligent User Interfaces10.1145/3640543.3645145(344-369)Online publication date: 18-Mar-2024
https://dl.acm.org/doi/10.1145/3640543.3645145
Appiah OMartey E(2024)Concentric Blocking Techniques for Improved Feature Extraction in Local Binary Pattern (LBP) Systems2024 IEEE 9th International Conference on Adaptive Science and Technology (ICAST)10.1109/ICAST61769.2024.10856506(1-8)Online publication date: 24-Oct-2024
https://doi.org/10.1109/ICAST61769.2024.10856506
Zhang YTennekes Mde Jong TCurier LCoecke BChen M(2023)Simulation-Based Optimization of User Interfaces for Quality-Assuring Machine Learning Model PredictionsACM Transactions on Interactive Intelligent Systems10.1145/3594552Online publication date: 17-May-2023
https://dl.acm.org/doi/10.1145/3594552
Shibu Kumar KSamuel P(2023)An Efficient Method to Accurately Cluster Large Number of High Dimensional Facial ImagesIEEE Access10.1109/ACCESS.2023.326886211(39934-39949)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3268862
Aggarwal APandya YRavindranathan LAhire LSethu MNandy KMagalhães Jdel Bimbo ASatoh SSebe NAlameda-Pineda XJin QOria VToni L(2022)Robust Actor Recognition in Entertainment Multimedia at ScaleProceedings of the 30th ACM International Conference on Multimedia10.1145/3503161.3548408(2079-2087)Online publication date: 10-Oct-2022
https://dl.acm.org/doi/10.1145/3503161.3548408
Zhang YWang YZhang HZhu BChen SZhang D(2022)OneLabeler: A Flexible System for Building Data Labeling ToolsProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3517612(1-22)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3517612
Liu TSugano Y(2022)Interactive Machine Learning on Edge Devices With User-in-the-Loop Sample RecommendationIEEE Access10.1109/ACCESS.2022.321207710(107346-107360)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3212077
Sun DYang KDing Z(2022)Confidence-Based Simple Graph Convolutional Networks for Face ClusteringIEEE Access10.1109/ACCESS.2022.314292210(6459-6469)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3142922
Rao BKini GAithal PVaishnavi KKamath U(2022)Parallel Local Tridirectional Feature Extraction Using GPUProceedings of the International Conference on Paradigms of Communication, Computing and Data Sciences10.1007/978-981-16-5747-4_37(437-443)Online publication date: 1-Jan-2022
https://doi.org/10.1007/978-981-16-5747-4_37
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten