Skip to main content
SpringerLink
Log in

Rate Control and Error Resilience for Object-Based Video Coding

  • Chapter
Intelligent Multimedia Communication: Techniques and Applications

Part of the book series: Studies in Computational Intelligence ((SCI,volume 280))

  • 756 Accesses

Abstract

The MPEG-4 audiovisual coding standard introduced the object-based video data representation model where video data is no longer seen as a sequence of frames or fields, but consists of independent (semantically) relevant video objects that together build the video scene. This representation approach allows new and improved functionalities, but it has also created new relevant problems in terms of typical non-normative parts of the standard, such as rate control and error resilience, which need to be solved in order to successfully transmit object-based video with an acceptable quality over networks that have critical bandwidth and channel error characteristics, such as mobile networks and the Internet. To deal with the specific problems of object-based video coding, rate control demands two levels of action: 1) the scene-level, which is responsible for dynamically allocating the available resources between the various objects in the scene (i.e., between the different encoding time instants and the different video objects to encode in each time instant), aiming at minimizing quality variations along time and between the various objects in the scene; and 2) the object-level, which is responsible for allocating the resources attributed to each object among the various types of data to code (for that object), notably texture and shape, and for computing the best encoding parameters to achieve the target bit allocations while maintaining smooth quality fluctuations. In terms of error resilience techniques, the object-based coding approach means that shape and composition information also have to be taken into account for error resilience purposes, in addition to motion and texture data. To do this, at the encoder side, the coding of video objects is typically supervised by a resilience configuration module, which is responsible for choosing the most adequate coding parameters in terms of resilience for each video object. This is important because the decoding performance will much depend on the protective actions the encoder has taken. At the decoder side, defensive actions have to be taken. This includes error detection and error localization for each decoded video object, followed by independent object-level error concealment. Afterwards, advanced scene-level error concealment is also performed, which has access to all the video objects in the scene and is used immediately before the final concealed video scene is presented to the user. In this chapter, the most recent basics, advances and trends in terms of rate control and error resilience for object-based video coding will be described.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Aign, S., Fazel, K.: Temporal & spatial error concealment techniques for hierarchical MPEG 2 video codec. In: Proc of the IEEE International Conference on Communications (ICC), Seattle, WA, USA, vol. 3, pp. 1778–1783 (1995)

    Google Scholar 

  • Åström, K., Wittenmark, B.: Adaptive control, 2nd edn. Addison-Wesley, Reading (1995)

    MATH  Google Scholar 

  • Brady, N., Soares, L.D.: Error resilience of arbitrarily shaped VOs (CE E14). ISO/IEC JTC1/SC29/WG11 M2370. Stockholm MPEG meeting (1997)

    Google Scholar 

  • CCITT SGXV, Description of reference model 8 (RM8). Doc. 525 (1989)

    Google Scholar 

  • Chen, Z., Han, J., Ngan, K.N.: Dynamic bit allocation for multiple video object coding. IEEE Transactions on Multimedia 8, 1117–1124 (2006)

    Article  Google Scholar 

  • Chiang, T., Zhang, Y.-Q.: A new rate control scheme using quadratic rate distortion model. IEEE Transactions on Circuits and Systems for Video Technology 7, 246–250 (1997)

    Article  Google Scholar 

  • Cover, T., Thomas, J.: Elements of Information Theory. John Wiley & Sons, New York (1991)

    Book  MATH  Google Scholar 

  • Frater, M.R., Lee, W.S., Pickering, M., Arnold, J.F.: Error concealment of arbitrarily shaped video objects. In: Proc. of the IEEE International Conference on Image Processing (ICIP), Chicago, IL, USA, vol. 3, pp. 507–511 (1998)

    Google Scholar 

  • Girod, B., Aaron, A., Rane, S., Rebollo-Monedero, D.: Distributed video coding. Proceedings of the IEEE 93, 71–83 (2005)

    Article  Google Scholar 

  • Gormish, M., Gill, J.: Computation-rate-distortion in transform coders for image compression. In: Proc. of the SPIE: Image and Video Processing, San Jose, CA, USA, vol. 1903, pp. 146–152 (1993)

    Google Scholar 

  • He, Z., Kim, Y.K., Mitra, S.: ρ-domain source modeling and rate control for video coding and transmission. In: Proc. of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Salt Lake City, UT, USA, vol. 3, pp. 1773–1776 (2001)

    Google Scholar 

  • He, Z., Mitra, S.: A unified rate-distortion analysis framework for transform coding. IEEE Transactions on Circuits and Systems for Video Technology 11, 1221–1236 (2001)

    Article  Google Scholar 

  • He, Z., Mitra, S.: A linear source model and a unified rate control algorithm for DCT video coding. IEEE Transactions on Circuits and Systems for Video Technology 12, 970–982 (2002)

    Article  Google Scholar 

  • He, Z., Kim, Y.K., Mitra, S.: Object-level bit allocation and scalable rate control for MPEG-4 video coding. In: Proc. of the Workshop and Exhibition on MPEG-4 (WEMP), San Jose, CA, USA, pp. 63–66 (2001)

    Google Scholar 

  • He, Z., Mitra, S.: Optimum bit allocation and accurate rate control for video coding via-domain source modeling. IEEE Transactions on Circuits and Systems for Video Technology 12, 840–849 (2002)

    Article  Google Scholar 

  • Hemami, S., Meng, T.: Transform coded image reconstruction exploiting interblock correlation. IEEE Transactions on Image Processing 4, 1023–1027 (1995)

    Article  Google Scholar 

  • ISO/IEC 11172-2, Information technology coding of moving pictures and associated audio for digital storage media at up to about 1,5 Mbit/s, part 2: video (1993)

    Google Scholar 

  • ISO/IEC 13818-2, Information technology generic coding of moving pictures and associated audio information – part 2: video (1996)

    Google Scholar 

  • ISO/IEC 14496, Information technology – coding of audio-visual objects (1999)

    Google Scholar 

  • ISO/IEC 14496-2, Information technology – coding of audio-visual objects – part 2: visual, 3rd edn. (2004)

    Google Scholar 

  • ITU-R BT.601-1, Encoding parameters of digital television for studios (1986)

    Google Scholar 

  • ITU-T H.261, Video codec for audiovisual services at p 64 kbit/s (1993)

    Google Scholar 

  • Keesman, G., Shah, I., Klein-Gunnewiek, R.: Bit-rate-control for MPEG encoders. Signal Processing: Image Communication 6, 545–560 (1995)

    Article  Google Scholar 

  • Kumar, S., Xu, L., Mandal, M.K., Panchanathan, S.: Error resiliency schemes in H.264/AVC standard. Journal of Visual Communication and Image Representation 17, 425–450 (2006)

    Article  Google Scholar 

  • Lee, H.-J., Chiang, T., Zhang, Y.-Q.: Scalable rate control for very low bit rate (VLBR) video. In: Proc. of the International Conference on Image Processing (ICIP), Santa Barbara, CA, USA, vol. 2, pp. 768–771 (1997)

    Google Scholar 

  • Lee, H.-J., Chiang, T., Zhang, Y.-Q.: Scalable rate control for MPEG-4 video. IEEE Transactions on Circuits and Systems for Video Technology 10, 878–894 (2000)

    Article  Google Scholar 

  • Lee, J.-W., Vetro, A., Wang, Y., Ho, Y.-S.: Bit allocation for MPEG-4 video coding with spatio-temporal tradeoffs. IEEE Transactions on Circuits and Systems for Video Technology 13, 488–502 (2003)

    Article  Google Scholar 

  • MPEG Test Model Editing Committee, MPEG-2 test model 5 (TM5). ISO/IEC JTC1/SC29/WG11 N400, Sydney MPEG meeting (1993)

    Google Scholar 

  • MPEG Video Group, MPEG-4 video verification model 4.0 (VM4), ISO/IEC JTC1/SC29/WG11 N1380, Chicago MPEG meeting (1996)

    Google Scholar 

  • MPEG Video Group, MPEG-4 video verification model 5.0 (VM5), ISO/IEC JTC1/SC29/WG11 N1469, Maceió MPEG meeting (1996)

    Google Scholar 

  • MPEG Video Group, MPEG-4 video verification model 8.0 (VM8). ISO/IEC JTC1/SC29/WG11 N1796, Stockholm MPEG meeting (1997)

    Google Scholar 

  • Nunes, P.: Rate control for object-based video coding. Ph.D. Thesis. Instituto Superior Técnico, Lisboa, Portugal (2007)

    Google Scholar 

  • Nunes, P., Pereira, F.: Rate control for scenes with multiple arbitrarily shaped video objects. In: Proc. of the Picture Coding Symposium (PCS), Berlin, Germany, pp. 303–308 (1997)

    Google Scholar 

  • Nunes, P., Pereira, F.: MPEG-4 compliant video encoding: analysis and rate control strategies. In: Proc. of the 34th ASILOMAR Conference, Pacific Grove, CA, USA (2000)

    Google Scholar 

  • Nunes, P., Pereira, F.: Scene level rate control algorithm for MPEG-4 video encoding. In: Proc. of the SPIE Visual Communications and Image Processing (VCIP), San Jose, CA, USA, vol. 4310, pp. 194–205 (2001)

    Google Scholar 

  • Nunes, P., Pereira, F.: Rate and distortion modeling analysis for MPEG-4 video intra coding. In: Proc. of the Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS), Lisboa, Portugal (2004)

    Google Scholar 

  • Nunes, P., Pereira, F.: Rate and distortion models for MPEG-4 video encoding. In: Proc. of the SPIE 49th Annual Meeting: Applications and Digital Image Processing XXVII, Denver, CO, USA, vol. 5558, pp. 382–394 (2004)

    Google Scholar 

  • Nunes, P., Pereira, F.: Improved feedback compensation mechanisms for multiple video object encoding rate control. In: Proc. of IEEE International Conference on Image Processing (ICIP), San Antonio, TX, USA (2007)

    Google Scholar 

  • Nunes, P., Pereira, F.: Joint rate control algorithm for low-delay MPEG-4 object-based video encoding. IEEE Transactions on Circuits and Systems for Video Technology 19, 1274–1288 (2009)

    Article  Google Scholar 

  • Pereira, F., Ebrahimi, T. (eds.): The MPEG-4 book. Prentice-Hall, Upper Saddle River (2002)

    Google Scholar 

  • Reibman, A., Haskell, B.: Constraints on variable bit-rate video for ATM networks. IEEE Transactions on Circuits and Systems for Video Technology 2, 361–372 (1992)

    Article  Google Scholar 

  • Ribas-Corbera, J., Lei, S.: Rate control in DCT video coding for low-delay communications. IEEE Transactions on Circuits and Systems for Video Technology 9, 172–185 (1999)

    Article  Google Scholar 

  • Richardson, I., Zhao, Y.: Adaptive algorithms for variable-complexity video coding. In: Proc. of the International Conference on Image Processing (ICIP), Thessaloniki, Greece, vol. 1, pp. 457–460 (2001)

    Google Scholar 

  • Ronda, J., Eckert, M., Jaureguizar, F., García, N.: Rate control and bit allocation for MPEG-4. IEEE Transactions on Circuits and Systems for Video Technology 9, 1243–1258 (1999)

    Article  Google Scholar 

  • Salama, P., Huang, C.: Error concealment for shape coding. In: Proc. of the IEEE International Conference on Image Processing (ICIP), Rochester, NY, USA, vol. 2, pp. 701–704 (2002)

    Google Scholar 

  • Schuster, G.M., Li, X., Katsaggelos, A.K.: Shape error concealment using Hermite splines. IEEE Transactions on Image Processing 13, 808–820 (2004)

    Article  MathSciNet  Google Scholar 

  • Schuster, G.M., Katsaggelos, A.K.: Motion compensated shape error concealment. IEEE Transactions on Image Processing 15, 501–510 (2006)

    Article  Google Scholar 

  • Shirani, S., Erol, B., Kossentini, F.: A concealment method for shape information in MPEG-4 coded video sequences. IEEE Transactions on Multimedia 2, 185–190 (2000)

    Article  Google Scholar 

  • Soares, L.D., Pereira, F.: Error resilience and concealment performance for MPEG-4 frame-based video coding. Signal Processing: Image Communication 14, 447–472 (1999)

    Article  Google Scholar 

  • Soares, L.D., Pereira, F.: Refreshment need metrics for improved shape and texture object-based resilient video coding. IEEE Transactions on Image Processing 12, 328–340 (2003)

    Article  Google Scholar 

  • Soares, L.D., Pereira, F.: Adaptive shape and texture intra refreshment schemes for improved error resilience in object-based video. IEEE Transactions on Image Processing 13, 662–676 (2004)

    Article  Google Scholar 

  • Soares, L.D., Pereira, F.: Combining space and time processing for shape error concealment. In: Proc. of the Picture Coding Symposium (PCS), San Francisco, CA, USA (2004)

    Google Scholar 

  • Soares, L.D., Pereira, F.: Spatial shape error concealment for object-based image and video coding. IEEE Transactions on Image Processing 13, 586–599 (2004)

    Article  Google Scholar 

  • Soares, L.D., Pereira, F.: Spatial texture error concealment for object-based image and video coding. In: Proc. of the EURASIP Conference focused on Signal and Image Processing, Multimedia Communications and Services (ECSIPM), Smolenice, Slovak Republic (2005)

    Google Scholar 

  • Soares, L.D., Pereira, F.: Spatial scene level shape error concealment for segmented video. In: Proc. of the Picture Coding Symposium (PCS), Beijing, China (2006)

    Google Scholar 

  • Soares, L.D., Pereira, F.: Temporal shape error concealment by global motion compensation with local refinement. IEEE Transactions on Image Processing 15, 1331–1348 (2006)

    Article  Google Scholar 

  • Soares, L.D., Pereira, F.: Spatio-temporal scene level error concealment for shape and texture data in segmented video content. In: Proc. of the IEEE International Conference on Image Processing (ICIP), Atlanta, GA, USA, pp. 2242–2252 (2006)

    Google Scholar 

  • Sun, Y., Ahmad, I.: A robust and adaptive rate control algorithm for object-based video coding. IEEE Transactions on Circuits and Systems for Video Technology 14, 1167–1182 (2004)

    Article  Google Scholar 

  • Sun, Y., Ahmad, I.: Asynchronous rate control for multi-object videos. IEEE Transactions on Circuits and Systems for Video Technology 15, 1007–1018 (2005)

    Article  Google Scholar 

  • Unicode Consortium, The Unicode standard, Version 3.0. Addison-Wesley (2000)

    Google Scholar 

  • Valentim, J., Nunes, P., Pereira, F.: An alternative complexity model for the MPEG-4 video verifier mechanism. In: Proc. of the International Conference on Image Processing (ICIP), Thessaloniki, Greece, vol. 1, pp. 461–464 (2001)

    Google Scholar 

  • Valentim, J., Nunes, P., Pereira, F.: Evaluating MPEG-4 video decoding complexity for an alternative video complexity verifier model. IEEE Transactions on Circuits and Systems for Video Technology 12, 1034–1044 (2002)

    Article  Google Scholar 

  • Vetro, A., Sun, H., Wang, Y.: Joint shape and texture rate control for MPEG-4 encoders. In: Proc. of the IEEE International Symposium on Circuits and Systems (ISCAS), Monterey, CA, USA, vol. 5, pp. 285–288 (1998)

    Google Scholar 

  • Vetro, A., Sun, H., Wang, Y.: MPEG-4 rate control for multiple video objects. IEEE Transactions on Circuits and Systems for Video Technology 9, 186–199 (1999)

    Article  Google Scholar 

  • Wang, Y., Zhu, Q.-F., Shaw, L.: Maximally smooth image recovery in transform coding. IEEE Transactions on Communications 41, 1544–1551 (1993)

    Article  MATH  Google Scholar 

  • Wang, Y., Zhu, Q.-F.: Error control and concealment for video communications: a review. Proceedings of the IEEE 86, 974–997 (1998)

    Article  Google Scholar 

  • Wang, Y., Wenger, S., Wen, J., Katsaggelos, A.K.: Error resilient video coding techniques. IEEE Signal Processing Magazine 17, 61–82 (2000)

    Article  Google Scholar 

  • Zhao, Y., Richardson, I.: Complexity management of video encoders. In: Proc. of the 10th ACM International Conference on Multimedia, Juan-les-Pins, France, pp. 647–649 (2002)

    Google Scholar 

Download references

Authors
  1. Paulo Nunes
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Luis Ducla Soares
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. University at Buffalo, Buffalo, NY, USA

    Chang Wen Chen

  2. Hong Kong Polytechnic University, China

    Zhu Li

  3. France Telecom R&D, Beijing, China

    Shiguo Lian

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Nunes, P., Soares, L.D. (2010). Rate Control and Error Resilience for Object-Based Video Coding. In: Chen, C.W., Li, Z., Lian, S. (eds) Intelligent Multimedia Communication: Techniques and Applications. Studies in Computational Intelligence, vol 280. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-11686-5_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-11686-5_1

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-11685-8

  • Online ISBN: 978-3-642-11686-5

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation