Abstract
We present a baseline MPEG-4 Advanced Video Coding (AVC) decoder based on the methodology of joint optimization of software and hardware. The software is first optimized with algorithm improvements for frame buffer management, boundary padding, content-aware inverse transform and context-based entropy decoding. The overall decoding throughput is further enhanced by pipelining the software and the dedicated hardware at macroblock level. The decoder is partitioned into the software and hardware modules according to the target frame rate and complexity profiles. The hardware acceleration modules include motion compensation, inverse transform and loop filtering. By comparing the optimized decoder with the committee reference decoder of Joint Video Team (JVT), the experimental results show improvement on the decoding throughput by 7 to 8 times. On an ARM966 board, the optimized software without hardware acceleration can achieve a decoding rate up to 5.9 frames per second (fps) for QCIF video source. The overall throughput is improved by another 27% to 7.4 fps on the average and up to 11.5 fps for slow motion video sequences. Finally, we provide a theoretical analysis of the ideal performance of the proposed decoder.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Joint Video Team (JVT) of ISO/IEC MPEG&ITU-T VCEG, “Draft ITU-T Recommendation and Final Draft International Standard of Joint Video Specification,” JVT-G050, ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6, Mar. 2003.
J. Zhang, Y. He, S. Yangm, and Y. Zhong, “Performance and Complexity Join Optimization for H.264 Video Coding,” in Proc. IEEE International Symposium on Circuit and System, May 2003, pp. 888–891.
V. Lappalainen, A. Hallapuro and T. D. Hamalainen, “Performance of H.26L Video Encoder on General Purpose Processor,” Journal of VLSI Signal Processing System for Signal, Image and Video Technology, pp. 239–249, 2003.
V. Lappalainen, A. Hallapuro, and T. D. Hamalainen, “Optimized Implementation of Emerging H.26L Video Decoder on Pentium III,” in Proc. 5th WSES, July 2001.
K. Ramkishor and V. Gunashree, “Real Time Implementation of MPEG-4 Video Decoder on ARM7TDMI,” in Proc. IEEE International Symposium on Intelligent Multimedia, Video, and Speech Processing, May 2001, pp. 522–526.
M.-H. Zhou and R. Tallurt, “DSP-Based Real Time Video Decoding,” in Proc. IEEE International Conference on Consumer Electronics, pp. 296–297, 1999.
W. Lin, et al., “Real Time H.263 Video Codec Using Parallel DSP,” Proc. IEEE International Conference on Image Processing, pp. 586–589, 1997.
H. Fujiwara, et al., “An All-ASIC Implementation of a Low Bit-Rate Video Codec,” IEEE Trans. on Circuit and System for Video Technology, vol. 2, pp. 123–134, June 1992.
M. Harrand, et al., “A Single Chip CIF 30-Hz, H.261, H.263, and H.263+ Video Encoder\Decoder with Embedded Display Controller,” IEEE Journal of Solid State Circuits, vol. 34, pp. 1627–1633, Nov. 1998.
M. Ikeda, et al., “SuperEnc: MPEG-2 Video Encoder Chip,” IEEE Micro magazine, July 1999.
M. Ikeda, et al., “SuperEnc: MPEG-2 Video Encoder Chip,” IEEE Micro magazine, July 1999.
S.-K. Jang, et al., “Hardware-Software Co-Implementation of a H.263 Video Codec,” IEEE Trans. on Consumer Electronics, vol. 46, no. 1, pp. 191–200, 2000.
L. D. Soares, S. Adachi, and F. Pereira, “Influence of Encoder Parameter on the Decoded Video Quality for MPEG-4 over W-CDMA Mobile Network,” in Proc. IEEE International Conference on Image Processing, 2000, pp. 148–151.
S.-H. Wang, C.-N. Wang, G.-Y. Lin, T. Chiang, and H.-F. Sun, “AHG Report on Editorial Convergence of MPEG-4 Reference Software,” ISO/IEC JTC1/SC29/WG11, M9073, Dec. 2002.
Telenor Research, “Video Codec Test Model (TMN5),” ITU-T SG 15, WP 15/1, Experts Group on Very Low Bit Rate Visual Technology, Jan 31, 1995. (Internet: ftp://bonde.na.no/pub/tmn/software/tmn-2.0.tar.gz, tmndec–2.0.tar.gz)
Joint Video Team (JVT) of ISO/IEC MPEG&ITU-T VCEG, “Context-adaptive VLC (CVLC) Coding of Coefficients,” JVT-C028, ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6, May 2002.
S.-H. Wang, et al., “A Platform-Based MPEG-4 Advanced Video Coding Decoder with Block Level Pipelining,” in Proc. IEEE ICICS-PCM, Singapore, Dec. 2003.
Author information
Authors and Affiliations
Corresponding author
Additional information
Shih-Hao Wang was born in Tainan, Taiwan, R.O.C. in 1977. He received the M.S. degree in Electrical and Control Engineering from National Chiao Tung University, Hsinchu, Taiwan, in 2001, where he is currently working toward the Ph.D. degree in the Institute of Electronics.
His research interests are video compression and VLSI implementation.
Wen-Hsiao Peng was born in Hsin-Chu, Taiwan, Republic of China, in 1975. He received the B.S. and the M.S. degrees in Electrics Engineering from National Chiao-Tung University, Hsin-Chu, Taiwan, in 1997 and 1999respectively. During 2000–2001, he was an intern in Intel Microprocessor Research Lab, U.S.A. In 2002, he joined the Institute of Electronics of National Chiao-Tung University, where he is currently a Ph.D candidate. His major research interests include scalable video coding, video codec optimization and platform based architecture design for video compression applications. Since 2000, he has been working with video coding development and implementation. He has actively contributed to the development of MPEG-4 Fine Granularity Scalability (FGS) and MPEG-21 Scalable Video Coding (Now, MPEG-4 Part 10 AVC Amd.1).
Yu-Wen Hereceived his Ph.D. degree in computer application from Tsinghua University in 2002. He was a lecture of the Department of Computer Science and Technology from 2002 to 2003 in Tsinghua University. In 2004, he joined Internet Media group of Microsoft Research Asia.
His research interests include video coding, transmission and embedded multimedia application systems.
Guan-yi Lin was born in Kaohsiung, Taiwan in 1981. He received the B.S. degree in Electronics Engineering from National Chiao Tung University, Hsinchu, Taiwan, in 2003, where he is currently working toward the M.S. degree in the Institute of Electronics.
His research interests are video compression and communication systems design.
Cheng-Yi Lin was born in Tainan, Taiwan in 1981. He received the B.S. degree in Electronics Engineering from National Chiao Tung University, Hsinchu, Taiwan, in 2003, where he is currently working toward the M.S. degree in the Institute of Electronics.
His research interests are on-chip communication and testing.
Shih-Chien Chang was born in Taichung, Taiwan in 1981. He received the B.S. degree in Electronics Engineering from National Chiao Tung University, Hsinchu, Taiwan, in 2003, where he is currently working toward the M.S. degree in the Institute of Electronics.
His research interests are video compression and VLSI implementation.
Chung-Neng Wang was born in PingTung, Taiwan, in 1972. He received the B.S. degree and Ph.D degree in computer science and information engineering from National Chiao-Tung University (NCTU), HsinChu, Taiwan in 1994 and 2003, respectively. He joined the faculty at National Chiao-Tung University in Taiwan, R.O.C in January 2003.
Since 2001 he has actively participated in ISO’s Moving Picture Experts Group (MPEG) digital video coding standardization process. He has made more than 18 contributions to the MPEG committee over the past 4 years. He published over 23 technical journal and conference papers in the field of video and signal processing. His current research interests are video/image compression, motion estimation, video transcoding, and streaming.
Tihao Chiangwas born in Cha-Yi, Taiwan, Republic of China, 1965. He received the B.S. degree in electrical engineering from the National Taiwan University, Taipei, Taiwan, in 1987, and the M.S. degree in electrical engineering from Columbia University in 1991. He received his Ph.D. degree in electrical engineering from Columbia University in 1995. In 1995, he joined David Sarnoff Research Center as a Member of Technical Staff. Later, he was promoted as a technology leader and a program manager at Sarnoff. While at Sarnoff, he led a team of researchers and developed an optimized MPEG-2 software encoder. For his work in the encoder and MPEG-4 areas, he received two Sarnoff achievement awards and three Sarnoff team awards.
Since 1992 he has actively participated in ISO’s Moving Picture Experts Group (MPEG) digital video coding standardization process with particular focus on the scalability/compatibility issue. He is currently the co-editor of the part 7 on the MPEG-4 committee. He has made more than 90 contributions to the MPEG committee over the past 10 years. His main research interests are compatible/scalable video compression, stereoscopic video coding, and motion estimation. In September 1999, he joined the faculty at National Chiao-Tung University in Taiwan, R.O.C. Dr. Chiang is currently a senior member of IEEE and holder of 13 US patents and 30 European and worldwide patents. He was a co-recipient of the 2001 best paper award from the IEEE Transactions on Circuits and Systems for Video Technology. He published over 50 technical journal and conference papers in the field of video and signal processing.
Rights and permissions
About this article
Cite this article
Wang, SH., Peng, WH., He, Y. et al. A Software-Hardware Co-Implementation of MPEG-4 Advanced Video Coding (AVC) Decoder with Block Level Pipelining. J VLSI Sign Process Syst Sign Image Video Technol 41, 93–110 (2005). https://doi.org/10.1007/s11265-005-6253-3
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11265-005-6253-3