Skip to main content
SpringerLink
Log in
Book cover

German Conference on Pattern Recognition

GCPR 2014: Pattern Recognition pp 227–238Cite as

Guided Image Super-Resolution: A New Technique for Photogeometric Super-Resolution in Hybrid 3-D Range Imaging

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 8753))

Abstract

In this paper, we augment multi-frame super-resolution with the concept of guided filtering for simultaneous upsampling of 3-D range data and complementary photometric information in hybrid range imaging. Our guided super-resolution algorithm is formulated as joint maximum a-posteriori estimation to reconstruct high-resolution range and photometric data. In order to exploit local correlations between both modalities, guided filtering is employed for regularization of the proposed joint energy function. For fast and robust image reconstruction, we employ iteratively re-weighted least square minimization embedded into a cyclic coordinate descent scheme. The proposed method was evaluated on synthetic datasets and real range data acquired with Microsoft’s Kinect. Our experimental evaluation demonstrates that our approach outperforms state-of-the-art range super-resolution algorithms while it also provides super-resolved photometric data.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    Supplementary material is available at http://www5.cs.fau.de/research/data/.

References

  1. Babacan, S.D., Molina, R., Katsaggelos, A.K.: Variational Bayesian super resolution. IEEE Trans. Image Process. 20(4), 984–999 (2011)

    Article  MathSciNet  Google Scholar 

  2. Bauer, S., Seitel, A., Hofmann, H., Blum, T., Wasza, J., Balda, M., Meinzer, H.-P., Navab, N., Hornegger, J., Maier-Hein, L.: Real-time range imaging in health care: a survey. In: Grzegorzek, M., Theobalt, C., Koch, R., Kolb, A. (eds.) Time-of-Flight and Depth Imaging. LNCS, vol. 8200, pp. 228–254. Springer, Heidelberg (2013)

    Google Scholar 

  3. Beder, C., Bartczak, B., Koch, R.: A comparison of PMD-cameras and stereo-vision for the task of surface reconstruction using patchlets. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007)

    Google Scholar 

  4. Bhavsar, A.V., Rajagopalan, A.N.: Range map superresolution-inpainting, and reconstruction from sparse data. Comput. Vis. Image Underst. 116(4), 572–591 (2012)

    Article  Google Scholar 

  5. Cui, Y., Schuon, S., Chan, D., Thrun, S., Theobalt, C.: 3D shape scanning with a time-of-flight camera. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1173–1180 (2010)

    Google Scholar 

  6. Elad, M., Feuer, A.: Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images. IEEE Trans. Image Process. 6(12), 1646–1658 (1997)

    Article  Google Scholar 

  7. Farsiu, S., Robinson, D., Elad, M., Milanfar, P.: Advances and challenges in super-resolution. Int. J. Imaging Syst. Technol. 14, 47–57 (2004)

    Article  Google Scholar 

  8. Farsiu, S., Robinson, D., Elad, M., Milanfar, P.: Fast and robust multiframe super resolution. IEEE Trans. Image Process. 13(10), 1327–1344 (2004)

    Article  Google Scholar 

  9. He, K., Sun, J., Tang, X.: Guided image filtering. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part I. LNCS, vol. 6311, pp. 1–14. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  10. Köhler, T., Haase, S., Bauer, S., Wasza, J., Kilgus, T., Maier-Hein, L., Feubner, H., Hornegger, J.: ToF meets RGB: novel multi-sensor super-resolution for hybrid 3-D endoscopy. Med. Image Comput. Comput. Assist. Interv. 16, 139–146 (2013)

    Google Scholar 

  11. Köhler, T., Haase, S., Bauer, S., Wasza, J., Kilgus, T., Maier-Hein, L., Feuner, H., Hornegger, J.: Outlier detection for multi-sensor super-resolution in hybrid 3D endoscopy. In: Deserno, T.M., Handels, H., Meinzer, H.-P., Tolxdorff, T. (eds.) Bildverarbeitung für die Medizin 2014. Informatik aktuell, pp. 84–89. Springer, Heidelberg (2014)

    Chapter  Google Scholar 

  12. Kurmankhojayev, D., Hasler, N., Theobalt, C.: Monocular pose capture with a depth camera using a sums-of-gaussians body model. In: Weickert, J., Hein, M., Schiele, B. (eds.) GCPR 2013. LNCS, vol. 8142, pp. 415–424. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  13. Liu, C.: Beyond pixels: exploring new representations and applications for motion analysis. Ph.D. thesis, Massachusetts Institute of Technology (2009)

    Google Scholar 

  14. Milanfar, P.: Super-Resolution Imaging. CRC Press, Boca Raton (2010)

    Google Scholar 

  15. Nabney, I.T.: NETLAB: Algorithms for Pattern Recognition. Advances in Pattern Recognition, 1st edn. Springer, Heidelberg (2002)

    Google Scholar 

  16. Park, J., Kim, H., Tai, Y., Brown, M., Kweon, I.: High quality depth map upsampling for 3D-TOF cameras. In: International Conference on Computer Vision, pp. 1623–1630 (2011)

    Google Scholar 

  17. Rajagopalan, A.N., Bhavsar, A., Wallhoff, F., Rigoll, G.: Resolution enhancement of PMD range maps. In: Rigoll, G. (ed.) DAGM 2008. LNCS, vol. 5096, pp. 304–313. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  18. Schultz, R.R., Stevenson, R.L.: Extraction of high-resolution frames from video sequences. IEEE Trans. Image Process. 5, 996–1011 (1996)

    Article  Google Scholar 

  19. Schuon, S., Theobalt, C., Davis, J., Thrun, S.: High-quality scanning using time-of-flight depth superresolution. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 1–7 (2008)

    Google Scholar 

  20. Schuon, S., Theobalt, C., Davis, J., Thrun, S.: LidarBoost: depth superresolution for ToF 3D shape scanning. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 343–350 (2009)

    Google Scholar 

  21. Schwarz, S., Sjostrom, M., Olsson, R.: A weighted optimization approach to time-of-flight sensor fusion. IEEE Trans. Image Process. 23(1), 214–225 (2014)

    Article  Google Scholar 

  22. Shotton, J., Girshick, R., Fitzgibbon, A., Sharp, T., Cook, M., Finocchio, M., Moore, R., Kohli, P., Criminisi, A., Kipman, A., Blake, A.: Efficient human pose estimation from single depth images. Pattern Anal. Mach. Intell. 35(12), 2821–2840 (2013)

    Article  Google Scholar 

  23. Wasza, J., Bauer, S., Haase, S., Schmid, M., Reichert, S., Hornegger, J.: RITK: the range imaging toolkit - a framework for 3-D range image stream processing. In: VMV, pp. 57–64. Eurographics Association (2011)

    Google Scholar 

Download references

Acknowledgments

The authors gratefully acknowledge funding of the Erlangen Graduate School in Advanced Optical Technologies (SAOT) by the German National Science Foundation (DFG) in the framework of the excellence initiative and the support by the DFG under Grant No. HO 1791/7-1.

Author information

Authors and Affiliations

  1. Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany

    Florin C. Ghesu, Thomas Köhler, Sven Haase & Joachim Hornegger

  2. Erlangen Graduate School in Advanced Optical Technologies (SAOT), Erlangen, Germany

    Thomas Köhler & Joachim Hornegger

Authors
  1. Florin C. Ghesu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thomas Köhler
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sven Haase
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Joachim Hornegger
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Thomas Köhler .

Editor information

Editors and Affiliations

  1. Department of Mathematics and Computer Science, University of Münster, Münster, Germany

    Xiaoyi Jiang

  2. Computer Science Department 5, University of Erlangen-Nürnberg, Erlangen, Germany

    Joachim Hornegger

  3. Department of Computer Science, University of Kiel, Kiel, Germany

    Reinhard Koch

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this paper

Cite this paper

Ghesu, F.C., Köhler, T., Haase, S., Hornegger, J. (2014). Guided Image Super-Resolution: A New Technique for Photogeometric Super-Resolution in Hybrid 3-D Range Imaging. In: Jiang, X., Hornegger, J., Koch, R. (eds) Pattern Recognition. GCPR 2014. Lecture Notes in Computer Science(), vol 8753. Springer, Cham. https://doi.org/10.1007/978-3-319-11752-2_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-11752-2_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-11751-5

  • Online ISBN: 978-3-319-11752-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation