Skip to main content
SpringerLink
Log in

Towards PDE-Based Video Compression with Optimal Masks and Optic Flow

  • Conference paper
  • First Online:
Scale Space and Variational Methods in Computer Vision (SSVM 2019)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 11603))

Abstract

Lossy image compression methods based on partial differential equations have received much attention in recent years. They may yield high quality results but rely on the computationally expensive task of finding optimal data.

For the possible extension to video compression, the data selection is a crucial issue. In this context one could either analyse the video sequence as a whole or perform a frame-by-frame optimisation strategy. Both approaches are prohibitive in terms of memory and run time.

In this work we propose to restrict the expensive computation of optimal data to a single frame and to approximate the optimal reconstruction data for the remaining frames by prolongating it by means of an optic flow field. We achieve a notable decrease in the computational complexity. As a proof-of-concept, we evaluate the proposed approach for multiple sequences with different characteristics. We show that the method preserves a reasonable quality in the reconstruction, and is very robust against errors in the flow field.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. The USC-SIPI image database (2014). http://sipi.usc.edu/database/

  2. Andris, S., Peter, P., Weickert, J.: A proof-of-concept framework for PDE-based video compression. In: Proceedings of 32nd Picture Coding Symposium, IEEE (2016)

    Google Scholar 

  3. Belhachmi, Z., Bucur, D., Burgeth, B., Weickert, J.: How to choose interpolation data in images. SIAM J. Appl. Math. 70(1), 333–352 (2009)

    Article  MathSciNet  Google Scholar 

  4. Bertalmío, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: Proceedings of 27th Annual Conference on Computer Graphics and Interactive Techniques, pp. 417–424. ACM Press/Addison-Wesley Publishing Company (2000)

    Google Scholar 

  5. Black, M.J.: Image sequences (2018). http://cs.brown.edu/people/mjblack/images.html

  6. Bloor, M., Wilson, M.: Generating blend surfaces using partial differential equations. Comput. Aided Des. 21(3), 165–171 (1989)

    Article  Google Scholar 

  7. Brinkmann, E.-M., Burger, M., Grah, J.: Regularization with sparse vector fields: from image compression to TV-type reconstruction. In: Aujol, J.-F., Nikolova, M., Papadakis, N. (eds.) SSVM 2015. LNCS, vol. 9087, pp. 191–202. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-18461-6_16

    Chapter  Google Scholar 

  8. Bruhn, A., Weickert, J., Feddern, C., Kohlberger, T., Schnörr, C.: Variational optical flow computation in real time. IEEE Trans. Image Process. 14(5), 608–615 (2003)

    Article  MathSciNet  Google Scholar 

  9. Carlsson, S.: Sketch based coding of grey level images. Signal Process. 15, 57–83 (1988)

    Article  Google Scholar 

  10. Chambolle, A., Pock, T.: A first order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40(1), 120–145 (2011)

    Article  MathSciNet  Google Scholar 

  11. Chen, Y., Ranftl, R., Pock, T.: A bi-level view of inpainting-based image compression. In: Kúkelová, Z., Heller, J. (eds.) Computer Vision Winter Workshop (2014)

    Google Scholar 

  12. Demaret, L., Iske, A., Khachabi, W.: Contextual image compression from adaptive sparse data representations. In: Gribonval, R. (ed.) Proceedings of SPARS 2009, Signal Processing with Adaptive Sparse Structured Representations Workshop (2009)

    Google Scholar 

  13. Facciolo, G., Arias, P., Caselles, V., Sapiro, G.: Exemplar-based interpolation of sparsely sampled images. In: Cremers, D., Boykov, Y., Blake, A., Schmidt, F.R. (eds.) EMMCVPR 2009. LNCS, vol. 5681, pp. 331–344. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-03641-5_25

    Chapter  Google Scholar 

  14. Galić, I., Weickert, J., Welk, M., Bruhn, A., Belyaev, A., Seidel, H.-P.: Towards PDE-based image compression. In: Paragios, N., Faugeras, O., Chan, T., Schnörr, C. (eds.) VLSM 2005. LNCS, vol. 3752, pp. 37–48. Springer, Heidelberg (2005). https://doi.org/10.1007/11567646_4

    Chapter  Google Scholar 

  15. Guillemot, C., Meur, O.L.: Image inpainting: overview and recent advances. IEEE Signal Process. Mag. 31(1), 127–144 (2014)

    Article  Google Scholar 

  16. Hoeltgen, L., et al.: Optimising spatial and tonal data for PDE-based inpainting. In: Bergounioux, M., Peyré, G., Schnörr, C., Caillau, J.B., Haberkorn, T. (eds.) Variational Methods, pp. 35–83. No. 18 in Radon Series on Computational and Applied Mathematics, De Gruyter (2016)

    Google Scholar 

  17. Hoeltgen, L., Peter, P., Breuß, M.: Clustering-based quantisation for PDE-based image compression. Signal Image Video Process. 12(3), 411–419 (2018)

    Article  Google Scholar 

  18. Hoeltgen, L., Setzer, S., Weickert, J.: An optimal control approach to find sparse data for laplace interpolation. In: Heyden, A., Kahl, F., Olsson, C., Oskarsson, M., Tai, X.-C. (eds.) EMMCVPR 2013. LNCS, vol. 8081, pp. 151–164. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40395-8_12

    Chapter  Google Scholar 

  19. Hoeltgen, L., Weickert, J.: Why does non-binary mask optimisation work for diffusion-based image compression? In: Tai, X.-C., Bae, E., Chan, T.F., Lysaker, M. (eds.) EMMCVPR 2015. LNCS, vol. 8932, pp. 85–98. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-14612-6_7

    Chapter  Google Scholar 

  20. Horn, B.K., Schunck, B.G.: Determining optical flow. Artif. Intell. 17(1–3), 185–203 (1981)

    Article  Google Scholar 

  21. Mainberger, M., Bruhn, A., Weickert, J., Forchhammer, S.: Edge-based compression of cartoon-like images with homogeneous diffusion. Pattern Recogn. 44(9), 1859–1873 (2011)

    Article  Google Scholar 

  22. Mainberger, M., Hoffmann, S., Weickert, J., Tang, C.H., Johannsen, D., Neumann, F., Doerr, B.: Optimising spatial and tonal data for homogeneous diffusion inpainting. In: Bruckstein, A.M., ter Haar Romeny, B.M., Bronstein, A.M., Bronstein, M.M. (eds.) SSVM 2011. LNCS, vol. 6667, pp. 26–37. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-24785-9_3

    Chapter  Google Scholar 

  23. Masnou, S., Morel, J.M.: Level lines based disocclusion. In: Proceedings of 1998 IEEE International Conference on Image Processing, vol. 3, pp. 259–263. IEEE (1998)

    Google Scholar 

  24. Ochs, P., Chen, Y., Brox, T., Pock, T.: iPiano: inertial proximal algorithm for nonconvex optimization. SIAM J. Imaging Sci. 7(2), 1388–1419 (2014)

    Article  MathSciNet  Google Scholar 

  25. Papenberg, N., Bruhn, A., Brox, T., Didas, S., Weickert, J.: Highly accurate optic flow computation with theoretically justified warping. Int. J. Comput. Vis. 67(2), 141–158 (2006)

    Article  Google Scholar 

  26. Peter, P., Hoffmann, S., Nedwed, F., Hoeltgen, L., Weickert, J.: From optimised inpainting with linear PDEs towards competitive image compression codecs. In: Bräunl, T., McCane, B., Rivera, M., Yu, X. (eds.) PSIVT 2015. LNCS, vol. 9431, pp. 63–74. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-29451-3_6

    Chapter  Google Scholar 

  27. Schmaltz, C., Peter, P., Mainberger, M., Ebel, F., Weickert, J., Bruhn, A.: Understanding, optimising, and extending data compression with anisotropic diffusion. Int. J. Comput. Vis. 108(3), 222–240 (2014)

    Article  MathSciNet  Google Scholar 

  28. Shen, J., Chan, T.F.: Mathematical models for local nontexture inpaintings. SIAM J. Appl. Math. 62(3), 1019–1043 (2002)

    Article  MathSciNet  Google Scholar 

  29. Strutz, T.: Bilddatenkompression. Vieweg (2002)

    Google Scholar 

  30. Sullivan, G.J., Wiegand, T.: Video compression - from concepts to the H. 264/AVC standard. Proc. IEEE. 93, 18–31 (2005)

    Article  Google Scholar 

  31. Sun, D.: (2018). http://research.nvidia.com/person/deqing-sun

  32. The Mathworks Inc.: Compute optical flow using Horn-Schunck method (2018). https://de.mathworks.com/help/vision/ug/compute-optical-flow-using-horn-schunck-method.html

  33. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  34. Weinzaepfel, P., Jégou, H., Pérez, P.: Reconstructing an image from its local descriptors. In: Proceedings of 2011 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 337–344. IEEE Computer Society Press (2011)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Brandenburg University of Technology Cottbus-Senftenberg, 03046, Cottbus, Germany

    Laurent Hoeltgen, Michael Breuß & Georg Radow

Authors
  1. Laurent Hoeltgen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael Breuß
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Georg Radow
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael Breuß .

Editor information

Editors and Affiliations

  1. University of Lübeck, Lübeck, Germany

    Jan Lellmann

  2. University of Erlangen-Nuremberg (FAU), Erlangen, Germany

    Martin Burger

  3. University of Lübeck, Lübeck, Germany

    Jan Modersitzki

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Hoeltgen, L., Breuß, M., Radow, G. (2019). Towards PDE-Based Video Compression with Optimal Masks and Optic Flow. In: Lellmann, J., Burger, M., Modersitzki, J. (eds) Scale Space and Variational Methods in Computer Vision. SSVM 2019. Lecture Notes in Computer Science(), vol 11603. Springer, Cham. https://doi.org/10.1007/978-3-030-22368-7_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-22368-7_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-22367-0

  • Online ISBN: 978-3-030-22368-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation