Skip to main content
SpringerLink
Log in

Geometric primitive refinement for structured light cameras

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Three-dimensional camera systems are useful sensors for several higher level vision tasks like navigation, environment mapping or dimensioning. However, the raw 3-D data is for many algorithms not the best representation. Instead, many methods rely on a more abstract scene description, where the scene is represented as a collection of geometric primitives like planes, spheres, or even more complex models. These primitives are commonly estimated on individual point measurements, which are directly affected by the measurement errors of the sensor. This paper proposes a method for refining the parameters of geometric primitives for structured light cameras with spatially varying patterns. In contrast to fitting the model to a set of 3-D point measurements, we propose to use all information that belongs to a particular object simultaneously to directly fit the model to the image, without the detour of calculating disparities. To this end, we propose a novel calibration procedure which recovers the unknown internal parameters of the range sensors and reconstructs the unknown projected pattern. This is particularly necessary for consumer-structured light sensors whose internals are not available to the user. After calibration, a coarse model fit is considerably refined by comparing the observed structured light dot pattern with a predicted virtual view of the projected virtual pattern. The calibration and the refinement methods are evaluated on three geometric primitives: planes, spheres, and cuboids. The orientations of the plane normals are improved by more than 60%, and plane distances by more than 30% compared to the baseline. Furthermore, the initial parameters of spheres and cuboids are refined by more than 50 and 30%. The method also operates robustly on highly textured plane segments, and at ranges that have not been considered during calibration.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Notes

  1. https://orbbec3d.com.

References

  1. Bergamasco, F., Albarelli, A., Rodola, E., Torsello, A.: Rune-tag: a high accuracy fiducial marker with strong occlusion resilience. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 113–120 (2011)

  2. Bird, N., Papanikolopoulos, N.: Optimal image-based euclidean calibration of structured light systems in general scenes. IEEE Trans. Autom. Sci. Eng. 8(4), 815–823 (2011)

    Article  Google Scholar 

  3. Birk, A., Pathak, K., Vaskevicius, N., Pfingsthorn, M., Poppinga, J., Schwertfeger, S.: Surface representations for 3d mapping. Künstl. Intell. 24(3), 249–254 (2010)

    Article  Google Scholar 

  4. Biswas, J., Veloso, M.: Depth camera based indoor mobile robot localization and navigation. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 1697–1702 (2012)

  5. Borrmann, D., Elseberg, J., Lingemann, K., Nüchter, A.: The 3d hough transform for plane detection in point clouds: a review and a new accumulator design. 3D Res 2(2), 3 (2011)

    Article  Google Scholar 

  6. Feng, C., Taguchi, Y., Kamat, V.R.: Fast plane extraction in organized point clouds using agglomerative hierarchical clustering. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 6218–6225 (2014)

  7. Fernndez-Moral, E., Mayol-Cuevas, W., Arvalo, V., Gonzlez-Jimnez, J.: Fast place recognition with plane-based maps. In: 2013 IEEE International Conference on Robotics and Automation, pp. 2719–2724 (2013)

  8. Fuersattel, P., Placht, S., Balda, M., Schaller, C., Hofmann, H., Maier, A., Riess, C.: A comparative error analysis of current time-of-flight sensors. IEEE Trans. Comput. Imaging 2(1), 27–41 (2016)

    Article  MathSciNet  Google Scholar 

  9. Geiger, A., Moosmann, F., Car, Ö., Schuster, B.: Automatic camera and range sensor calibration using a single shot. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 3936–3943. IEEE (2012)

  10. Georgiev, K., Creed, R.T., Lakaemper, R.: Fast plane extraction in 3d range data based on line segments. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3808–3815 (2011)

  11. Herrera, D., Kannala, J., Heikkilä, J.: Accurate and practical calibration of a depth and color camera pair. In: International Conference on Computer Analysis of Images and Patterns, pp. 437–445. Springer (2011)

  12. Holz, D., Behnke, S.: Approximate triangulation and region growing for efficient segmentation and smoothing of range images. Robot. Auton. Syst. 62(9), 1282–1293 (2014)

    Article  Google Scholar 

  13. Horn, B.K.P.: Closed-form solution of absolute orientation using unit quaternions. J. Opt. Soc. Am. 4(4), 629 (1987)

    Article  Google Scholar 

  14. Khoshelham, K., Elberink, S.O.: Accuracy and resolution of kinect depth data for indoor mapping applications. Sensors 12(2), 1437–1454 (2012)

    Article  Google Scholar 

  15. McIlroy, P., Izadi, S., Fitzgibbon, A.: Kinectrack: 3d pose estimation using a projected dense dot pattern. IEEE Trans. Vis. Comput. Graph. 20(6), 839–851 (2014)

    Article  Google Scholar 

  16. Moreno, D., Taubin, G.: Simple, accurate, and robust projector-camera calibration. In: 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission (3DIMPVT), pp. 464–471 (2012)

  17. Mörwald, T., Richtsfeld, A., Prankl, J., Zillich, M., Vincze, M.: Geometric data abstraction using b-splines for range image segmentation. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 148–153 (2013)

  18. Nguyen, A., Le, B.: 3d point cloud segmentation: a survey. In: 6th International Conference on Robotics, Automation and Mechatronics (RAM), pp. 225–230 (2013)

  19. Placht, S., Fürsattel, P., Mengue, E.A., Hofmann, H., Schaller, C., Balda, M., Angelopoulou, E.: Rochade: robust checkerboard advanced detection for camera calibration. In: Computer Vision—ECCV 2014, Lecture Notes in Computer Science, vol. 8692, pp. 766–779. Springer (2014)

  20. Poppinga, J., Vaskevicius, N., Birk, A., Pathak, K.: Fast plane detection and polygonalization in noisy 3d range images. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3378–3383 (2008)

  21. Ryan Fanello, S., Rhemann, C., Tankovich, V., Kowdle, A., Orts Escolano, S., Kim, D., Izadi, S.: Hyperdepth: learning depth from structured light without matching. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)

  22. Salas-Moreno, R.F., Glocken, B., Kelly, P.H.J., Davison, A.J.: Dense planar slam. In: IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 157–164 (2014)

  23. Taguchi, Y., Jian, Y.D., Ramalingam, S., Feng, C.: Point-plane slam for hand-held 3d sensors. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 5182–5189 (2013)

  24. Trevor, A.J.B., Gedikli, S., Rusu, R.B., Christensen, H.I.: Efficient organized point cloud segmentation with connected components. In: Semantic Perception Mapping and Exploration (SPME) (2013)

  25. Weingarten, J., Siegwart, R.: 3d slam using planar segments. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3062–3067 (2006)

  26. Yamazaki, S., Mochimaru, M., Kanade, T.: Simultaneous self-calibration of a projector and a camera using structured light. In: 2011 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPR Workshops), pp. 60–67 (2011)

  27. Ye, Y., Song, Z.: A practical means for the optimization of structured light system calibration parameters. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 1190–1194 (2016)

  28. Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Friedrich-Alexander-University Erlangen-Nürnberg, Erlangen, Germany

    Peter Fuersattel, Andreas Maier & Christian Riess

  2. Metrilus GmbH, Erlangen, Germany

    Simon Placht

Authors
  1. Peter Fuersattel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Simon Placht
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Andreas Maier
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Christian Riess
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peter Fuersattel.

Additional information

This work was supported in part by the Research Training Group 1773 “Heterogeneous Image Systems,” funded by the German Research Foundation (DFG), and in part by the Erlangen Graduate School in Advanced Optical Technologies (SAOT) by the German Research Foundation (DFG) in the framework of the excellence initiative.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fuersattel, P., Placht, S., Maier, A. et al. Geometric primitive refinement for structured light cameras. Machine Vision and Applications 29, 313–327 (2018). https://doi.org/10.1007/s00138-017-0901-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-017-0901-z

Keywords

Search

Navigation