Skip to main content
SpringerLink
Log in

Sub-event recognition and summarization for structured scenario photos

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Structured scenario photos, referring to the images which capture important events that usually follow specific routines/structures (such as wedding ceremonies, graduation ceremonies, etc.), account for a significant proportion in personal photo collections. Conventional image analysis techniques without considering the event routines/structures are not sufficient to handle these photos. In this paper, we explore the appropriate framework to learn and utilize the specific routines for understanding these structure scenario photos. Specifically, we propose a novel framework which can systematically integrate Hidden Markov Model and Gaussian Mixture Model to recognize sub-events from structured scenario photos. Then we present a comprehensive criterion to select representative images to summarize the whole photo collection. Experimental results conducted on the real-world datasets demonstrate the superiority of our framework in both of sub-event recognition and photo summarization tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Cheng W, Chuang Y, Chen B, Wu J, Fang S, Lin Y, Hsieh C, Pan C, Chu W, Tien M (2007) Semantic-event based analysis and segmentation of wedding ceremony videos. In: Proceedings of the 9th ACM SIGMM international workshop on multimedia information retrieval, MIR 2007, Augsburg, Bavaria, Germany, 24–29 September 2007, pp 95–104

  2. Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the em algorithm. J R Stat Soc Ser B 39(1):1–38

    MathSciNet  MATH  Google Scholar 

  3. Denney B, Balan AK (2014) Selecting representative images for display. USPTO application US 12/906,107

  4. Gatica-Perez D, Loui A, Sun M-T (2003) Finding structure in home videos by probabilistic hierarchical clustering

  5. Ghahramani Z, Jordan MI (1997) Factorial hidden markov models. Mach Learn 29(2–3):245–273

    Article  MATH  Google Scholar 

  6. Hua X, Lu L, Zhang H (2003) AVE: automated home video editing. In: Proceedings of the 11th ACM international conference on multimedia, Berkeley, CA, USA, 2–8 November 2003, pp 490–497

  7. Jiang Y, Dai Q, Mei T, Rui Y, Chang S (2015) Super fast event recognition in internet videos. IEEE Trans Multimedia 17(8):1174–1186

    Article  Google Scholar 

  8. Kennedy LS, Naaman M (2008) Generating diverse and representative image search results for landmarks. In: Proceedings of the 17th international conference on world wide web, WWW 2008, Beijing, China, 21–25 April 2008, pp 297–306

  9. Pearson K (1905) The problem of the random walk. Nature

  10. Shaked D, Tastl I (2005) Sharpness measure: towards automatic image enhancement. In: Proceedings of the 2005 international conference on image processing, ICIP 2005, Genoa, Italy, 11–14 September 2005, pp 937–940

  11. Sinha P, Pirsiavash H, Jain R (2009) Personal photo album summarization. In: Proceedings of the 17th international conference on multimedia 2009, Vancouver, British Columbia, Canada, 19–24 October 2009, pp 1131–1132

  12. Sinha P, Mehrotra S, Jain R (2011) Effective summarization of large collections of personal photos. In: Proceedings of the 20th international conference on world wide web, WWW 2011, Hyderabad, India, 28 March - 1 April 2011 (Companion Volume), pp 127–128

  13. Tang J, Hua X (2014) Typicality ranking: beyond accuracy for video semantic annotation. Multimedia Tools Appl 70(2):647–660

    Article  Google Scholar 

  14. Tang J, Yan S, Hong R, Qi G, Chua T (2009) Inferring semantic concepts from community-contributed images and noisy tags. In: ACM multimedia

  15. Tang J, Wang M, Hua X, Chua T (2012) Social media mining and search. Multimedia Tools Appl 56(1):1–7

    Article  Google Scholar 

  16. Zhang L, Vaisenberg R, Mehrotra S, Kalashnikov DV (2011) Video entity resolution: applying er techniques for smart video surveillance. In: PerCom workshops

  17. Zhang L, Kalashnikov DV, Mehrotra S (2013) A unified framework for context assisted face clustering. In: ACM international conference on multimedia retrieval (ACM ICMR 2013), Dallas, Texas, USA, 16–19 April 2013

  18. Zhang L, Xu J, Li C (2013) A random-walk based recommendation algorithm considering item categories. Neurocomputing 120:391–396

    Article  Google Scholar 

  19. Zhang L, Denney B, Lu J (2014) Systems and methods for image management. USPTO application US 13/639,948

Download references

Acknowledgments

This work was supported in part by the National Science Foundation of China under Grants No. 61572252, and National Science Foundation of Jiangsu Province under Grants No. BK20150755.

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing, Jiangsu, China

    Liyan Zhang

  2. Canon Information and Imaging Solutions, Inc., Irvine, CA, USA

    Bradley Denney

  3. Nymi Inc., Toronto, ON, Canada

    Juwei Lu

Authors
  1. Liyan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bradley Denney
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Juwei Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Liyan Zhang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, L., Denney, B. & Lu, J. Sub-event recognition and summarization for structured scenario photos. Multimed Tools Appl 75, 9295–9314 (2016). https://doi.org/10.1007/s11042-016-3346-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-016-3346-x

Keywords

Search

Navigation