Skip to main content
SpringerLink
Log in

Multiple description coding network based on semantic segmentation

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Considering semantic information in the image compression can prominently improve the quality of synthesized image. In this paper, we propose a multiple description coding network based on semantic segmentation. In the proposed scheme, the semantic segmentation map of input image is encoded as side information to improve the coding efficiency. Firstly, multiple description feature generator network is used to produce multiple description information. Secondly, the produced multiple description information and the semantic segmentation map are fed into the semantic segmentation encoder network to obtain encoded information. Thirdly, we propose side decoder networks and central decoder network, which are used to decode the image. In the proposed architecture, the semantic information is auxiliary information, which is used to compensate the difference between the input image and generated image. After testing the two datasets, it can be seen that when the bit rate is greater than 1BPP, the PSNR can exceed 40. Therefore, the proposed method is feasible.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Agustsson E (2017) Soft-to-hard vector quantization for end-to-end learned compression of images and neural networks

  2. Agustsson E, Tschannen M, Mentzer F, Timofte R, Gool LV (2018) Generative adversarial networks for extreme learned image compression

  3. Akbari M, Liang J, Han J (2018) Dsslic: Deep semantic segmentation-based layered image compression. arXiv:1806.03348

  4. Ballé J, Laparra V, Simoncelli EP (2016) End-to-end optimized image compression. arXiv:1611.01704

  5. Bellard F Bpg image format

  6. Cao S, Wu CY, Krhenbühl P (2020) Lossless image compression through super-resolution

  7. Christopoulos CA, Ebrahimi T, Skodras AN (2000) Jpeg2000: the new still picture compression standard. In: Proceedings of the ACM Multimedia 2000 Workshops, Los Angeles, CA, USA, October 30 - November 3, 2000

  8. Feng J, Wen T, Liu S, Jie R, Xun G, Zhao D (2017) An end-to-end compression framework based on convolutional neural networks. IEEE Trans Circuits & Systems for Video Tech PP(99):1–1. https://doi.org/10.1109/DCC.2017.54

    Google Scholar 

  9. Goodfellow IJ, Pouget-Abadie J, Mirza M, Bing X, Bengio Y (2014) Generative adversarial nets. MIT Press

  10. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  11. Herranz L, Jiang S, Li X (2018) Scene recognition with cnns: Objects, scales and dataset bias. IEEE

  12. Kai Z, Zuo W, Gu S, Lei Z (2017) Learning deep cnn denoiser prior for image restoration. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR)

  13. Lili M, Jie L, Upul S, Yao Z, Huihui B, Kaup A (2014) Multiple description coding with randomly and uniformly offset quantizers. IEEE Trans Image Process 23(2):582–595

    Article  MathSciNet  Google Scholar 

  14. Liu M, Zhu C (2009) Enhancing two-stage multiple description scalar quantization. IEEE Signal Process Letters 16 (4):253–256. https://doi.org/10.1109/LSP.2009.2014104

    Article  MathSciNet  Google Scholar 

  15. Lu X, Wang W, Danelljan M, Zhou T, Gool LV (2020) Video object segmentation with episodic graph memory networks

  16. Lu X, Wang W, Ma C, Shen J, Porikli F (2020) See more, know more: Unsupervised video object segmentation with co-attention siamese networks. IEEE Trans Pattern Anal Mach Intell PP(99):1–1

    Google Scholar 

  17. Lu X, Ma C, Shen J, Yang X, Reid I, Yang M-H (2020) Deep object tracking with shrinkage loss. IEEE transactions on pattern analysis and machine intelligence

  18. Lu X, Wang W, Shen J, Crandall D, Luo J (2020) Zero-shot video object segmentation with co-attention siamese networks. IEEE transactions on pattern analysis and machine intelligence

  19. Mentzer F, Toderici G, Tschannen M, Agustsson E (2020) High-fidelity generative image compression

  20. Minnen D, Ballé J, Toderici G (2018) Joint autoregressive and hierarchical priors for learned image compression. CoRR abs/1809.02736, 1809.02736

  21. Ollivier Y (2014) Auto-encoders: reconstruction versus compression. Computer ence

  22. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. Computer Science

  23. Sutskever I, Krizhevsky A, Hinton GE Imagenet classification with deep convolutional neural networks

  24. Theis L, Shi W, Cunningham A, Husz?r F (2017) Lossy image compression with compressive autoencoders

  25. Toderici G, O’Malley SM, Hwang SJ, Vincent D, Minnen D, Baluja S, Covell M, Sukthankar R (2015) Variable rate image compression with recurrent neural networks. Computer Science

  26. Toderici G, Vincent D, Johnston N, Jin Hwang S, Minnen D, Shor J, Covell M (2017) Full resolution image compression with recurrent neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5306–5314

  27. Vaishampayan VA (1993) Design of multiple description scalar quantizers. IEEE Trans Inf Theory 39(3):821–834. https://doi.org/10.1109/18.256491

    Article  MathSciNet  Google Scholar 

  28. Van De Sande KEA, Gevers T, Snoek C (2010) Evaluating color descriptors for object and scene recognition. IEEE Trans Pattern Analysis and Machine Intell 32:1582–1596

  29. Wallace GK (1992) The jpeg still picture compression standard. Communications of the Acm 38(1):xviii–xxxiv

    Google Scholar 

  30. Wang T-C, Liu M-Y, Zhu J-Y, Tao A, Kautz J, Catanzaro B (2018) High-resolution image synthesis and semantic manipulation with conditional gans. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8798–8807

  31. Zhao H, Gallo O, Frosio I, Kautz J (2017) Loss functions for image restoration with neural networks. IEEE Trans Comput Imaging 3(1):47–57. https://doi.org/10.1109/TCI.2016.2644865

    Article  Google Scholar 

  32. Zhao L, Bai H, Wang A, Zhao Y (2018) Deep multiple description coding by learning scalar quantization

  33. Zhao L, Bai H, Wang A, Zhao Y (2018) Multiple description convolutional neural networks for image compression. IEEE Trans Circuits and Systems for Video Tech, p 1–1. https://doi.org/10.1109/tcsvt.2018.2867067

  34. Zhou B, Garcia AL, Xiao J, Torralba A, Oliva A (2015) Learning deep features for scene recognition using places database. Advances in neural information processing systems, 1

  35. Zhou B, Hang Z, Puig X, Fidler S, Barriuso A, Torralba A (2017) Scene parsing through ade20k dataset. In: Computer vision & pattern recognition

  36. Zong J, Meng L, Zhang H, Wan W (2017) Jnd-based multiple description image coding. Ksii Trans Internet & Info Systems 11(8):3935–3949

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, Shandong Normal University, Jinan, 250014, China

    Xue Li, Lili Meng, Yanyan Tan, Jia Zhang, Wenbo Wan & Huaxiang Zhang

  2. Institute of Data Science and Technology, Shandong Normal University, Jinan, 250014, China

    Xue Li, Lili Meng, Yanyan Tan, Jia Zhang, Wenbo Wan & Huaxiang Zhang

Authors
  1. Xue Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lili Meng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yanyan Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jia Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wenbo Wan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Huaxiang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lili Meng.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, X., Meng, L., Tan, Y. et al. Multiple description coding network based on semantic segmentation. Multimed Tools Appl 81, 29075–29091 (2022). https://doi.org/10.1007/s11042-022-12654-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12654-0

Keywords

Search

Navigation