Skip to main content
SpringerLink
Log in

Shot Boundary Detection for Automatic Video Analysis of Historical Films

  • Conference paper
  • First Online:
New Trends in Image Analysis and Processing – ICIAP 2019 (ICIAP 2019)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 11808))

Included in the following conference series:

Abstract

In automatic video content analysis and film preservation, Shot Boundary Detection (SBD) is a fundamental pre-processing step. While previous research focuses on detecting Abrupt Transitions (AT) as well as Gradual Transitions (GT) in different video genres such as sports movies or news clips only few studies investigate in the detection of shot transitions in historical footage. The main aim of this paper is to create an SBD mechanism inspired by state-of-the-art algorithms which is applied and evaluated on a self-generated historical dataset as well as a publicly available dataset called Clipshots. Therefore, a three-stage pipeline is introduced consisting of a Candidate Frame Range Selection based on the network DeepSBD, Extraction of Convolutional Neural Network (CNN) Features and Similarity Calculation. A combination of pre-trained backbone CNNs such as ResNet, VGG19 and SqueezeNet with different similarity metrics like Cosine Similarity and Euclidean Distance are used and evaluated. The outcome of this paper displays that the proposed algorithm reaches promising results on detecting ATs in historical videos without the need of complex optimization and re-training processes. Furthermore, it points out the main challenges concerning historical footage such as damaged film reels, scratches or splices. The results of this paper contribute a significant base for future research on automatic video analysis of historical videos.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 74.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://www.vhh-project.eu/en/summary/ - last visit: 2019/05/29.

  2. 2.

    http://efilms.ushmm.org/ - last visited: 2019/05/30.

  3. 3.

    https://github.com/Tangshitao/ClipShots_basline - last visit: 2019/06/09.

  4. 4.

    https://github.com/owenzlz/Shot_Boundary_Detection_Using_CNN_features.git-last visit: 2019/06/11.

References

  1. Adjeroh, D., Lee, M.C., Banda, N., Kandaswamy, U.: Adaptive edge-oriented shot boundary detection. EURASIP J. Image Video Process. 2009(1), 859371 (2009). https://doi.org/10.1155/2009/859371

    Article  Google Scholar 

  2. Baraldi, L., Grana, C., Cucchiara, R.: Shot and scene detection via hierarchical clustering for re-using broadcast video. In: Azzopardi, G., Petkov, N. (eds.) CAIP 2015. LNCS, vol. 9256, pp. 801–811. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-23192-1_67. https://www.ebook.de/de/product/25073344/computer_analysis_of_images_and_patterns.html

    Chapter  Google Scholar 

  3. Bolei, Z., Agata, L., Aditya, K., Aude, O., Antonio, T.: Places: a 10 million image database for scene recognition. IEEE Trans. Pattern Anal. Mach. Intell. 40(6), 1452–1464 (2018). https://doi.org/10.1109/TPAMI.2017.2723009

    Article  Google Scholar 

  4. Bouthemy, P., Gelgon, M., Ganansia, F.: A unified approach to shot change detection and camera motion characterization. IEEE Trans. Circ. Syst. Video Technol. 9(7), 1030–1044 (1999). https://doi.org/10.1109/76.795057

    Article  Google Scholar 

  5. Jiang, D., Kim, J.: Video searching and fingerprint detection by using the image query and PlaceNet-based shot boundary detection method. Appl. Sci. 8(10), 1735 (2018). https://doi.org/10.3390/app8101735

    Article  Google Scholar 

  6. Gygli, M.: Ridiculously fast shot boundary detection with fully convolutional neural networks. Biochimica et Biophysica Acta 89, 95–108 (2018). https://doi.org/10.1109/RoEduNet.2013.6511763

    Article  Google Scholar 

  7. Hassanien, A., Elgharib, M.A., Selim, A., Hefeeda, M., Matusik, W.: Large-scale, fast and accurate shot boundary detection through spatio-temporal convolutional neural networks. CoRR abs/1705.03281 (2017). http://arxiv.org/abs/1705.03281

  8. Xu, J., Song, L., Xie, R.: Shot boundary detection using convolutional neural networks. In: VCIP 2016–30th Anniversary of Visual Communication and Image Processing, pp. 1–4 (2017)

    Google Scholar 

  9. Küçüktunç, O., Gudukbay, U., Ulusoy, Ö.: Fuzzy color histogram-based video segmentation. Comput. Vis. Image Underst. 114, 125–134 (2010). https://doi.org/10.1016/j.cviu.2009.09.008

    Article  Google Scholar 

  10. Porter, S., Mirmehdi, M., Thomas, B.: Temporal video segmentation and classification of edit effects. Image Vis. Comput. 21(13–14), 1097–1106 (2003). https://doi.org/10.1016/j.imavis.2003.08.014

    Article  Google Scholar 

  11. Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211–252 (2015). https://doi.org/10.1007/s11263-015-0816-y

    Article  MathSciNet  Google Scholar 

  12. Seidl, M., Zeppelzauer, M., Mitrović, D., Breiteneder, C.: Gradual transition detection in historic film material - a systematic study. J. Comput. Cult. Herit. 4(3), 10:1–10:18 (2011). https://doi.org/10.1145/2069276.2069279

    Article  Google Scholar 

  13. Smeaton, A.F., Over, P., Doherty, A.R.: Video shot boundary detection: seven years of TRECVid activity. Comput. Vis. Image Underst. 114(4), 411–418 (2010). https://doi.org/10.1016/j.cviu.2009.03.011

    Article  Google Scholar 

  14. Tang, S., Feng, L., Kuang, Z., Chen, Y., Zhang, W.: Fast video shot transition localization with deep structured models. CoRR abs/1808.04234 (2018). http://arxiv.org/abs/1808.04234

  15. Tong, W., Song, L., Yang, X., Qu, H., Xie, R.: CNN-based shot boundary detection and video annotation. In: 2015 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting, pp. 1–5 (2015). https://doi.org/10.1109/BMSB.2015.7177222

  16. Zeppelzauer, M., Mitrović, D., Breiteneder, C.: Archive film material - a novel challenge for automated film analysis. Frames Cin. J. 1(1) (2012). http://publik.tuwien.ac.at/files/PubDat_216640.pdf

  17. Li, Z., Liu, X., Zhang, S.: Shot boundary detection based on multilevel difference of colour histograms. In: Proceedings - 2016 1st International Conference on Multimedia and Image Processing, ICMIP 2016, pp. 15–22 (2016). https://doi.org/10.1109/ICMIP.2016.24

Download references

Acknowledgement

This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Grant Agreement No. 822670.

Author information

Authors and Affiliations

  1. Institute of Visual Computing and Human-Centered Technology, TU Wien, Vienna, Austria

    Daniel Helm & Martin Kampel

Authors
  1. Daniel Helm
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Martin Kampel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Daniel Helm .

Editor information

Editors and Affiliations

  1. University of Verona, Verona, Italy

    Marco Cristani

  2. University of Parma, Parma, Italy

    Andrea Prati

  3. Fondazione Bruno Kessler, Povo, Italy

    Oswald Lanz

  4. Fondazione Bruno Kessler, Povo, Italy

    Stefano Messelodi

  5. University of Trento, Povo, Italy

    Nicu Sebe

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Helm, D., Kampel, M. (2019). Shot Boundary Detection for Automatic Video Analysis of Historical Films. In: Cristani, M., Prati, A., Lanz, O., Messelodi, S., Sebe, N. (eds) New Trends in Image Analysis and Processing – ICIAP 2019. ICIAP 2019. Lecture Notes in Computer Science(), vol 11808. Springer, Cham. https://doi.org/10.1007/978-3-030-30754-7_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-30754-7_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-30753-0

  • Online ISBN: 978-3-030-30754-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation