Skip to main content
SpringerLink
Log in

Preserving Temporal Consistency in Videos Through Adaptive SLIC

  • Conference paper
  • First Online:
Advances in Computer Graphics (CGI 2020)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12221))

Included in the following conference series:

  • 1917 Accesses

Abstract

The application of image processing techniques to individual frames of video often results in temporal inconsistency. Conventional approaches used for preserving the temporal consistency in videos have shortcomings as they are used for only particular jobs. Our work presents a multipurpose video temporal consistency preservation method that utilizes an adaptive simple linear iterative clustering (SLIC) algorithm. First, we locate the inter-frame correspondent pixels through the SIFT Flow and use them to find the respective regions. Then, we apply a multiframe matching statistical method to get the spatially or temporally correspondent frames. Besides, we devise a least-squares energy-based flickering-removing objective function by taking into account the inter-frame temporal consistency and inter-region spatial consistency jointly. The obtained results demonstrate the potential of the proposed method.

H. Zhang and R. Ali—Contributed equally to this work.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bonneel, N., Sunkavalli, K., Paris, S., Pfister, H.: Example-based video color grading. ACM Trans. Graph. 32(4), 39:1–39:12 (2013)

    Article  Google Scholar 

  2. Bonneel, N., Sunkavalli, K., Tompkin, J., Sun, D., Paris, S., Pfister, H.: Interactive intrinsic video editing. ACM Trans. Graph. 33(6), 197:1–197:10 (2014)

    Article  Google Scholar 

  3. Bonneel, N., Tompkin, J., Sunkavalli, K., Sun, D., Paris, S., Pfister, H.: Blind video temporal consistency. ACM Trans. Graph. 34(6), 196:1–196:9 (2015)

    Article  Google Scholar 

  4. Chen, A.Y.C., Corso, J.J.: Propagating multi-class pixel labels throughout video frames. In: Western New York Image Processing Workshop, pp. 14–17 (2010)

    Google Scholar 

  5. Dong, X., Bonev, B., Zhu, Y., Yuille, A.L.: Region-based temporally consistent video post-processing. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 714–722 (2015)

    Google Scholar 

  6. Farbman, Z., Lischinski, D.: Tonal stabilization of video. ACM Trans. Graph. 30(4), 89:1–89:10 (2011)

    Article  Google Scholar 

  7. Hsu, C.Y., Lu, C.S., Pei, S.C.: Video halftoning preserving temporal consistency. In: 2007 IEEE International Conference on Multimedia and Expo, pp. 1938–1941 (2007)

    Google Scholar 

  8. Kamel, A., Sheng, B., Yang, P., Li, P., Shen, R., Feng, D.D.: Deep convolutional neural networks for human action recognition using depth maps and postures. IEEE Trans. Syst. Man Cybern. Syst. 49(9), 1806–1819 (2019)

    Article  Google Scholar 

  9. Karambakhsh, A., Kamel, A., Sheng, B., Li, P., Yang, P., Feng, D.D.: Deep gesture interaction for augmented anatomy learning. Int. J. Inf. Manag. 45, 328–336 (2019). https://doi.org/10.1016/j.ijinfomgt.2018.03.004. http://www.sciencedirect.com/science/article/pii/S0268401217308678

    Article  Google Scholar 

  10. Lang, M., Wang, O., Aydin, T., Smolic, A., Gross, M.: Practical temporal consistency for image-based graphics applications. ACM Trans. Graph. 31(4), 34:1–34:8 (2012)

    Article  Google Scholar 

  11. Li, C., Chen, Z., Sheng, B., Li, P., He, G.: Video flickering removal using temporal reconstruction optimization. Multimedia Tools Appl. 79, 4661–4679 (2019)

    Article  Google Scholar 

  12. Liu, C., Yuen, J., Torralba, A.: SIFT flow: dense correspondence across scenes and its applications. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 978–994 (2011)

    Article  Google Scholar 

  13. Liu, C.: Beyond pixels: exploring new representations and applications for motion analysis. Ph.D. thesis, Massachusetts Institute of Technology, Cambridge, MA, USA (2009)

    Google Scholar 

  14. Mantiuk, R., Daly, S., Kerofsky, L.: Display adaptive tone mapping. ACM Trans. Graph. 27(3), 1–10 (2008)

    Article  Google Scholar 

  15. Meng, X., et al.: A video information driven football recommendation system. Comput. Electr. Eng. 85, 106699 (2020). https://doi.org/10.1016/j.compeleceng.2020.106699

    Article  Google Scholar 

  16. Müller, M., Zilly, F., Riechert, C., Kauff, P.: Spatio-temporal consistent depth maps from multi-view video. In: 2011 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON), pp. 1–4 (2011)

    Google Scholar 

  17. Reso, M., Jachalsky, J., Rosenhahn, B., Ostermann, J.: Occlusion-aware method for temporally consistent superpixels. IEEE Trans. Pattern Anal. Mach. Intell. 41, 1441–1454 (2019)

    Article  Google Scholar 

  18. Sheng, B., Li, P., Zhang, Y., Mao, L.: GreenSea: visual soccer analysis using broad learning system. IEEE Trans. Cybern. 1–15 (2020). https://doi.org/10.1109/TCYB.2020.2988792

  19. Shin, D.K., Kim, Y.M., Park, K.T., Lee, D.S., Choi, W., Moon, Y.S.: Video dehazing without flicker artifacts using adaptive temporal average. In: The 18th IEEE International Symposium on Consumer Electronics (ISCE 2014), pp. 1–2 (2014)

    Google Scholar 

  20. Tsai, D., Flagg, M., Nakazawa, A., Rehg, J.M.: Motion coherent tracking using multi-label MRF optimization. Int. J. Comput. Vision 100(2), 190–202 (2012)

    Article  MathSciNet  Google Scholar 

  21. Wang, C.M., Huang, Y.H., Huang, M.L.: An effective algorithm for image sequence color transfer. Math. Comput. Model. 44, 608–627 (2006)

    Article  Google Scholar 

  22. Wang, Z., Chen, X., Zou, D.: Copy and paste: temporally consistent stereoscopic video blending. IEEE Trans. Circuits Syst. Video Technol. 28(10), 3053–3065 (2018)

    Article  Google Scholar 

  23. Zhang, P., Zheng, L., Jiang, Y., Mao, L., Li, Z., Sheng, B.: Tracking soccer players using spatio-temporal context learning under multiple views. Multimedia Tools Appl. 77(15), 18935–18955 (2017). https://doi.org/10.1007/s11042-017-5316-3

    Article  Google Scholar 

Download references

Acknowledgement

This work was supported in part by the National Key Research and Development Program of China under Grant 2018YFF0300903, in part by the National Natural Science Foundation of China under Grant 61872241 and Grant 61572316, and in part by the Science and Technology Commission of Shanghai Municipality under Grant 15490503200, Grant 18410750700, Grant 17411952600, and Grant 16DZ0501100.

Author information

Authors and Affiliations

  1. Nanjing University of Aeronautics and Astronautics, Nanjing, People’s Republic of China

    Han Zhang

  2. Shanghai Jiao Tong University, Shanghai, People’s Republic of China

    Riaz Ali & Bin Sheng

  3. The Hong Kong Polytechnic University, Hong Kong, People’s Republic of China

    Ping Li

  4. The University of Sydney, Sydney, Australia

    Jinman Kim

  5. Shanghai University of Sport, Shanghai, People’s Republic of China

    Jihong Wang

Authors
  1. Han Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Riaz Ali
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bin Sheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ping Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jinman Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jihong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Bin Sheng or Jihong Wang .

Editor information

Editors and Affiliations

  1. University of Geneva, Geneva, Switzerland

    Nadia Magnenat-Thalmann

  2. University of Crete, Heraklion, Greece

    Constantine Stephanidis

  3. University of Macau, Macau, China

    Enhua Wu

  4. Swiss Federal Institute of Technology, Lausanne, Switzerland

    Daniel Thalmann

  5. Shanghai Jiao Tong University, Shanghai, China

    Bin Sheng

  6. University of Sydney, Sydney, Australia

    Jinman Kim

  7. University of Crete, Heraklion, Greece

    George Papagiannakis

  8. University of Calgary, Calgary, AB, Canada

    Marina Gavrilova

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhang, H., Ali, R., Sheng, B., Li, P., Kim, J., Wang, J. (2020). Preserving Temporal Consistency in Videos Through Adaptive SLIC. In: Magnenat-Thalmann, N., et al. Advances in Computer Graphics. CGI 2020. Lecture Notes in Computer Science(), vol 12221. Springer, Cham. https://doi.org/10.1007/978-3-030-61864-3_34

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-61864-3_34

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-61863-6

  • Online ISBN: 978-3-030-61864-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation