Skip to main content
SpringerLink
Log in
Book cover

Pacific Rim Conference on Multimedia

PCM 2015: Advances in Multimedia Information Processing -- PCM 2015 pp 598–607Cite as

Hole Filling Algorithm Using Spatial-Temporal Background Depth Map for View Synthesis in Free View Point Television

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 9315))

Abstract

This paper presents a new hole filling algorithm based on background modeling and texture synthesis. Depth information of hole areas is computed by using a local background estimation method. In order to exploit the correlation between frames in temporal domain, a background modeling method is used to extract reliable background reference scenes. The holes are then filled by combining the temporal background information and the estimated depth information. A modified exemplar-based inpainting method is used to fill remaining hole pixels. Experimental results demonstrate the capability of the proposed algorithm.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Smolic, A., Kauff, P., Knorr, S., Hornung, A., Kunter, M., Muller, M., Lang, M.: Three-dimensional video postproduction and processing. Proc. IEEE 99(4), 607–625 (2011)

    Article  Google Scholar 

  2. Muller, K., Merkle, P., Tech, G., Wiegand, T.: 3D video formats and coding methods. In: IEEE International ConferenceImage Processing, pp. 2339–2392 (2010)

    Google Scholar 

  3. Dodgson, N.A.: Autostereoscopic 3D displays. Computer 38(8), 31–36 (2005)

    Article  Google Scholar 

  4. Fehn, C.: Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3DTV. In: SPIE Proceedings of the Stereoscopic Image Process and Rendering, pp. 93–104 (2004)

    Google Scholar 

  5. Zinger, S., Ruijters, D., Do, L., de With, P.H.N.: View interpolation for medical images on autostereoscopic displays. IEEE Trans. Circuits Syst. Video Technol. 22(1), 128–137 (2012)

    Article  Google Scholar 

  6. Zhang, L., Tam, W.J.: Stereoscopic image generation based on depth images for 3D TV. IEEE Trans. Broadcast. 51(2), 191–199 (2005)

    Article  Google Scholar 

  7. Lee, P.-J.: Effendi: nongeometric distortion smoothing approach for depth map preprocessing. IEEE Trans. Multimedia 13(2), 246–254 (2011)

    Article  Google Scholar 

  8. Zinger, S., Do, L., de With, P.H.N.: Free-viewpoint depth image based rendering. J. Vis. Commun. Image Represent. 21(5–6), 533–541 (2010)

    Article  Google Scholar 

  9. Doan, H.N., Nguyen, T.A., Kim, B., Hong, M.-C.: Directional hole filling algorithm in new view synthesis for 3D video using local segmentation. In: ACM International Conference Research in Adaptive and Convergent Systems, pp. 100–104, MD, USA (2014)

    Google Scholar 

  10. Solh, M., Alregib, G.: Hierarchical hole-filling for depth-based view synthesis in FTV and 3D video. IEEE. J. Sel. Top. Signal 6(5), 495–504 (2012)

    Article  Google Scholar 

  11. Oh, K., Yea, S., Ho, Y.-S.: Hole filling method using depth based in-painting for view synthesis in free viewpoint television and 3-D video. In: Picture Coding Symposium, pp. 1–4 (2009)

    Google Scholar 

  12. Marcelo, B., Guillermo, S., Vincent, C., Coloma, B.: Image inpainting. In: The 27th Annual Conference on Computer Graphics and Interactive Techniques, pp. 417–424 (2000)

    Google Scholar 

  13. Criminisi, A., Perez, P., Toyama, K.: Object removal by exemplar-based inpainting. In: IEEE International Conference Computer Vision and Pattern Recognition, vol. 2, pp. 721–728 (2003)

    Google Scholar 

  14. Ming, X., Wang, L.-H., Yang, Q.-Q., Li, D.-X., Zhang, M.: Depth-image-based rendering with spatial and temporal texture synthesis for 3DTV. EURASIP J. Image Video Process. 2013(1), 1–18 (2013)

    Article  Google Scholar 

  15. Koppel, M., Wang, W., Doshkov, D., Wiegand, T., Ndjiki-Nya, P.: Consistent spatio-temporal filling of disocclusions in the multiview-video-plus-depth format. In: IEEE International Workshop Multimedia Signal Processing, pp. 25–30 (2012)

    Google Scholar 

  16. Yao, C., Tillo, T., Jimin, X.: Depth map driven hole filling algorithm exploiting temporal correlation information. IEEE Trans. Broadcast. 60(2), 394–404 (2014)

    Article  Google Scholar 

  17. Kim, K., Chalidabhongse, T.H., Harwood, D., Davis, L.: Background modeling and subtraction by codebook construction. IEEE Int. Conf. Image Process. 5, 3061–3064 (2004)

    Google Scholar 

  18. Fernandez-Sanchez, E.J., Javier, D., Eduardo, R.: Background subtraction based on color and depth using active sensors. Sensors 13(7), 8895–8915 (2013)

    Article  Google Scholar 

  19. Glassner, A.S. (ed.): Graphics Gems I. Academic Press, Cambridge (1990)

    MATH  Google Scholar 

Download references

Acknowledgment

This research was supported by Next-Generation Information Computing Development Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT & Future Planning (No. 2012M3C4A7032182).

Author information

Authors and Affiliations

  1. Soongsil University, Sangdo-Dong, Dongjak-Gu, Seoul, South Korea

    Huu Noi Doan, Beomsu Kim & Min-Cheol Hong

Authors
  1. Huu Noi Doan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Beomsu Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Min-Cheol Hong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Huu Noi Doan .

Editor information

Editors and Affiliations

  1. Gwangju Institute of Science and Technology, Gwangju, Korea (Republic of)

    Yo-Sung Ho

  2. Chinese Academy of Sciences, Institute of Automation, Beijing, China

    Jitao Sang

  3. KAIST, Daejeon, Korea (Republic of)

    Yong Man Ro

  4. KAIST, Daejeon, Korea (Republic of)

    Junmo Kim

  5. College of Computer Science, Zhejiang University, Hangzhou, China

    Fei Wu

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Doan, H.N., Kim, B., Hong, MC. (2015). Hole Filling Algorithm Using Spatial-Temporal Background Depth Map for View Synthesis in Free View Point Television. In: Ho, YS., Sang, J., Ro, Y., Kim, J., Wu, F. (eds) Advances in Multimedia Information Processing -- PCM 2015. PCM 2015. Lecture Notes in Computer Science(), vol 9315. Springer, Cham. https://doi.org/10.1007/978-3-319-24078-7_61

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-24078-7_61

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-24077-0

  • Online ISBN: 978-3-319-24078-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation