Skip to main content
SpringerLink
Log in

Temporal-Consistency-Aware Video Color Transfer

  • Conference paper
  • First Online:
Advances in Computer Graphics (CGI 2021)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 13002))

Included in the following conference series:

Abstract

This paper proposes a new temporal-consistency-aware color transfer method based on quaternion distance metric. Compared with the state-of-the-art methods, our method can keep the temporal consistency and better reduce the artifacts. Firstly, keyframes are extracted from the source video and transfer the color from the reference image through soft segmentation based on Gaussian Mixture Models (GMM). Then a quaternion-based method is proposed to transfer color from keyframes to the other frames iteratively. Specifically, this method analyses the color information of each pixel along five directions to detect its best matching pixel through a quaternion-based distance metric. Additionally, considering the accumulating errors in frame sequences, an effective abnormal color correction mechanism is designed to improve the color transfer quality. A quantitative evaluation metric is further proposed to measure the temporal consistency in the output video. Various experimental results validate the effectiveness of our method.

S. Liu—This work was partly supported by the Natural Science Foundation of China under grant nos. 62072328 and 61672375.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Reinhard, E., Ashikhmin, M., Gooch, B., Shirley, P.: Color transfer between images. IEEE Comput. Graph. Appl. 21(5), 34–41 (2001)

    Article  Google Scholar 

  2. Song, Z., Liu, S.: Sufficient image appearance transfer combining color and texture. IEEE Trans. Multimedia 19(4), 702–711 (2017)

    Article  Google Scholar 

  3. Liu, S., Sun, H., Zhang, X.: Selective color transferring via ellipsoid color mixture map. J. Vis. Commun. Image R. 23(1), 173–181 (2012)

    Article  Google Scholar 

  4. Faridul, H.S., et al.: Colour mapping: a review of recent methods, extensions and applications. Comput. Graph. Forum. 35(1), 59–88 (2005)

    Article  Google Scholar 

  5. Song, C., Zhao, H., Jing, W.: Robust video stabilization based on particle filtering with weighted feature points. IEEE Trans. Consum. Electron. 58(2), 570–577 (2012)

    Article  Google Scholar 

  6. Bonneel, N., Sunkavalli, K., Paris, S., Pfister, H.: Example-based video color grading. ACM Trans. Graph. 32(4), 1–12 (2013)

    Article  Google Scholar 

  7. Vazquezcorral, J., Bertalmio, M.: Color stabilization along time and across shots of the same scene, for one or several cameras of unknown specifications. IEEE Trans. Image Process. 23(10), 4564–4575 (2014)

    Article  MathSciNet  Google Scholar 

  8. Pei, S., Hsiao, Y.: Simple effective image and video color correction using quaternion distance metric. In: Proceedings of IEEE International Conference Image Process, pp. 2920–2924 (2015)

    Google Scholar 

  9. Jin, L., Liu, H., Xu, X., Song, E.: Quaternion-based impulse noise removal from color video sequences. IEEE Trans. Circuits Syst. Video Technol. 23(5), 741–755 (2013)

    Article  Google Scholar 

  10. Xiao, X., Ma, L.: Gradient-preserving color transfer. Comput. Graph Forum. 28(7), 1879–1886 (2009)

    Article  Google Scholar 

  11. Papadakis, N., Provenzi, E., Caselles, V.: A variational model for histogram transfer of color images. IEEE Trans. Image Process. 20(6), 1682–1695 (2011)

    Article  MathSciNet  Google Scholar 

  12. Niu, Y., Zheng, X., Zhao, T., Chen, J.: Visually consistent color correction for stereoscopic images and videos. IEEE Trans. Circuits Syst. Video Technol. 30(3), 697–710 (2020)

    Article  Google Scholar 

  13. Liu, S., Song, Z., Zhang, X., Zhu, T.: Progressive complex illumination image appearance transfer based on CNN. J. Vis. Commun. Image R. 64, 1–11 (2019)

    Article  Google Scholar 

  14. Liao, J., Yao, Y., Lu, Y., Hua, G., Kang, S.B.: Visual attribute transfer through deep image analogy. ACM Trans. Graph. 36(4), article no. 120 (2017)

    Google Scholar 

  15. Zhu, T., Liu, S.: Detail-preserving arbitrary style transfer. In: Proceedings of the IEEE International Conference on Multimedia and Expo, pp. 1–6 (2020)

    Google Scholar 

  16. He, M., Liao, J., Yuan, L., Sander, P.V.: Neural color transfer between images. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–14 (2017)

    Google Scholar 

  17. Luan, F., Paris, S., Shechtman, E., Bala, K.: Deep photo style transfer. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6997–7005 (2017)

    Google Scholar 

  18. Zabaleta, I., Bertalmío, M.: Photorealistic style transfer for video. Sig. Process. Image Commun. 95, 116240 (2021)

    Google Scholar 

  19. Liu, S, Zhu, T.: Structure-guided arbitrary style transfer for artistic image and video. IEEE Trans. Multimedia 23 (2021, early access). https://doi.org/10.1109/TMM.2021.3063605

  20. Hogervorst, M.A., Toet, A.: Method for applying daytime colors to nighttime imagery in realtime. In: Proceedings of SPIE, pp. 6974–6984 (2013)

    Google Scholar 

  21. Hogervorst, M.A., Toet, A.: Fast natural color mapping for night-time imagery. Inf. Fusion 11(2), 69–77 (2010)

    Article  Google Scholar 

  22. Xue, S., Agarwala, A., Dorsey, J., Rushmeier, H.: Learning and applying color styles from feature films. Comput. Graph. Forum 32(7), 255–264 (2013)

    Article  Google Scholar 

  23. Wang, C.M., Huang, Y.H., Huang, M.L.: An effective algorithm for image sequence color transfer. Math. Comput. Model. 44(7), 608–627 (2006)

    Article  Google Scholar 

  24. Yao, C.H., Chang, C.Y., Chien, S.Y.: Example-based video color transfer. In: Proceedings of IEEE International Conference Multimedia Expo, pp. 1–6 (2015)

    Google Scholar 

  25. Jeong, J.Y., Kim, H.J., Wang, T.S.: Real-time video re-coloring algorithm considering the temporal color consistency for the color-blind. IEEE Trans. Consum. Electron. 58(2), 721–729 (2012)

    Article  Google Scholar 

  26. Gu, X., He, M., Leung, H., Gu, X.: Fast colorization for single-band thermal video sequences. Neurocomputing 17(1), 1146–1157 (2016)

    Article  Google Scholar 

  27. Liu, C., Freeman, W.T.: A high-quality video denoising algorithm based on reliable motion estimation. In: Proceedings of European Conference on Computer Vision, pp. 706–719 (2010)

    Google Scholar 

  28. Nguyen, R.M.H., Kim, S.J., Brown, M.S.: Illuminant aware gamut-based color transfer. Comput. Graph. Forum 33(7), 319–328 (2014)

    Article  Google Scholar 

  29. Chen, D., Liao, J., Yuan, L., Yu, N., Hua, G.: Coherent online video style transfer. In: Proceedings of IEEE Computer Vision and Pattern Recognition, pp. 1114–1123 (2017)

    Google Scholar 

  30. Aouaidjia, K., Sheng, B., Li, P., Kim, J., Feng, D.: Efficient body motion quantification and similarity evaluation using 3-D joints skeleton coordinates. IEEE Trans. Syst. Man Cybern. Syst. 51(5), 2774–2788 (2021)

    Article  Google Scholar 

  31. Lai, W.S., Huang, J.B., Wang, O., Shechtman, E., Yumer, E., Yang, M.H.: Learning blind video temporal consistency. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 170–185 (2018)

    Google Scholar 

  32. Wen, Y., Sheng, B., Li, P., Lin, W., Feng, D.: Deep color guided coarse-to-fine convolutional network cascade for depth image super-resolution. IEEE Trans. Image Process. 28(2), 994–1006 (2019)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Intelligence and Computing, Tianjin University, Tianjin, 300350, China

    Shiguang Liu & Yu Zhang

Authors
  1. Shiguang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yu Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shiguang Liu .

Editor information

Editors and Affiliations

  1. University of Geneva, Carouge, Switzerland

    Nadia Magnenat-Thalmann

  2. University of Minnesota, Minneapolis, MN, USA

    Victoria Interrante

  3. EPFL, Lausanne, Switzerland

    Daniel Thalmann

  4. University of Crete, Heraklion, Crete, Greece

    George Papagiannakis

  5. Shanghai Jiao Tong University, Shanghai, China

    Bin Sheng

  6. University of Sydney, Sydney, NSW, Australia

    Jinman Kim

  7. University of Calgary, Calgary, AB, Canada

    Marina Gavrilova

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Liu, S., Zhang, Y. (2021). Temporal-Consistency-Aware Video Color Transfer. In: Magnenat-Thalmann, N., et al. Advances in Computer Graphics. CGI 2021. Lecture Notes in Computer Science(), vol 13002. Springer, Cham. https://doi.org/10.1007/978-3-030-89029-2_36

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-89029-2_36

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-89028-5

  • Online ISBN: 978-3-030-89029-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation