Abstract
The people in an image are generally not strangers, but instead often share social relationships such as husband-wife, siblings, grandparent-child, father-child, or mother-child. Further, the social relationship between a pair of people influences the relative position and appearance of the people in the image. This chapter explores using familial social relationships as context, recognizing people and recognizing the social relationships between pairs of people. We introduce a model for representing the interaction between social relationship, facial appearance, and identity. The experiments on a set of personal collections show that significant improvement in people recognition is achieved by modeling social relationships, even in a weak label setting that is attractive in practical applications. Furthermore, we show that social relationships are effectively recognized in images from a separate test image collection.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Barnard, K., Duygulu, P., Forsyth, D., de Freitas, N., Blei, D.M., Jordan, M.I.: Matching words and pictures. JMLR 3, 1107–1135 (2003)
Berg, T., Berg, A., Edwards, J., Maire, M., White, R., Teh, Y.W., Learned-Miller, E., Forsyth, D.: Names and faces in the news. In Proceedings of the CVPR (2004)
Chen, L., Hu, B., Zhang, L., Li, M., Zhang, H.J.: Face annotation for family photo album management. IJIG 3(1), 81–94 (2003)
Chen, Y.-Y., Hsu, W.H., Mark Liao H.-Y.: Discovering informative social subgraphs and predicting pairwise relationships from group photos. In: Proceedings of the 20th ACM International Conference on Multimedia, pp. 669–678. ACM (2012)
Ding, L., Yilmaz, A.: Inferring social relations from visual concepts. In IEEE International Conference on Computer Vision, pp. 699–706. IEEE (2011)
Divvala, S.K., Hoiem, D., Hays, J., Efros, A., Hebert, M.: An empirical study of context in object detection. In: Proceedings of the CVPR (2009)
Fang, R., Gallagher, A., Chen, T., Loui, A.: Kinship classification by modeling facial feature heredity. In: Proceedings of the ICIP (2013)
Fang, R., Tang, K.D., Snavely, N., Chen, T.: Towards computational models of kinship verification. In: 17th IEEE International Conference on Image Processing, pp. 1577–1580. IEEE (2010)
Gallagher, A., Chen, T.: Clothing cosegmentation for recognizing people. In: Proceedings of the CVPR (2008)
Gallagher, A., Chen, T.: Estimating age, gender, and identity using first name priors. In: Proceedings of the CVPR (2008)
Gallagher, A., Chen, T.: Understanding images of groups of people. In: Proceedings of the CVPR (2009)
Guo, G., Fu, Y., Dyer, C., Huang, T.: Image-based human age estimation by manifold learning and locally adjusted robust regression. In IEEE Trans. Image Process. 17(7), 1111178–1111199 (2008)
Gupta, A., Davis, L.S.: Beyond nouns: Exploiting prepositions and comparative adjectives for learning visual classifiers. In: Proceedings of the ECCV (2008)
Hoiem, D., Efros, A.A., Hebert, M.: Putting objects in perspective. IJCV 80(1), 3–15 (2008)
Kapoor, A., Hua, G., Akbarzadeh, A., Baker, S.: Which faces to tag: Adding prior constraints into active learning. In: Proceedings of the ICCV (2009)
Kumar, N., Berg, A., Belhumeur, P., Nayar, S.: Attribute and simile classifiers for face verification. In: Proceedings of the ICCV (2009)
Naaman, M., Yeh, R., Garcia-Molina, H., Paepcke, A.: Leveraging context to resolve identity in photo albums. In: Proceedings of the JCDL (2005)
National Center for Health Statistics. CDC growth charts, United States. http://www.cdc.gov/nchs/data/nhanes/growthcharts/zscore/statage.xls (2007)
O’Hare, N., Smeaton, A.: Context-aware person identification in personal photo collections. IEEE Trans MM (2009)
Parikh, D., Zitnick, L., Chen, T.: From appearance to context-based recognition: Dense labeling in small images. In: Proceedings of the CVPR (2008)
Pellegrini, S., Ess, A., Schindler, K., van Gool, L.: You’ll never walk alone: Modeling social behavior for multi-target tracking. In: Proceedings of the ICCV (2009)
Shao, M., Xia, S., Fu. Kinship verification through transfer learning. In: IJCAI (2011)
Shao, M., Xia, S., Luo, J., Fu, Y.: Understanding kin relationships in a photo. In: IEEE Transactions on Multimedia (T-MM) (2012)
Song, Y., Leung, T.: Context-aided human recognition- clustering. In: Proceedings of the ECCV (2006)
Stone, Z., Zickler, T., Darrell, T.: Autotagging facebook: Social network context improves photo annotation. In: Proceedings of the CVPR Internet Vision Workshop (2008)
Tian, Y., Liu, W., Xian, R., Wen, F., Tang, X.: A face annotation framework with partial clustering and interactive labeling. In: Proceedings of the CVPR (2007)
Wang, G., Gallagher, A., Luo, J., Forsyth, D.: Seeing people in social context: Recognizing people and social relationship. In: Proceedings of the ECCV, pp. 169–182. Springer (2010)
Wang, G., Forsyth, D.: Joint learning of visual attributes, object classes and visual saliency. In: IEEE 12th International Conference on Computer Vision, 2009, pp. 537–544. IEEE (2009)
Wang, G., Hoiem, D., Forsyth, D.: Learning image similarity from flickr groups using fast kernel machines. IEEE Trans. Pattern Anal. Mach. Intell. 34(11), 2177–2188 (2012)
Xia, Siyu, Shao, Ming, Luo, Jiebo: Understanding kin relationships in a photo. IEEE Trans. Multimedia 14(4), 1046–1056 (2012)
Zhao, M., Teo, Y.W., Liu, S., Chua, T., Jain, R.: Automatic person annotation of family photo album. Lect. Notes Comput. Sci. 4071, 163 (2006)
Zhou, X., Hu, J., Lu, J., Shang, Y., Guan, Y.: Kinship verification from facial images under uncontrolled conditions. In: Proceedings of the 19th ACM International Conference on Multimedia, pp. 953–956. ACM (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Wang, G., Gallagher, A., Luo, J., Forsyth, D. (2014). Recognizing People in Social Context. In: Fu, Y. (eds) Human-Centered Social Media Analytics. Springer, Cham. https://doi.org/10.1007/978-3-319-05491-9_6
Download citation
DOI: https://doi.org/10.1007/978-3-319-05491-9_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-05490-2
Online ISBN: 978-3-319-05491-9
eBook Packages: Computer ScienceComputer Science (R0)