Skip to main content
SpringerLink
Log in

A novel method for reconstructing general 3D curves from stereo images

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Three-dimensional (3D) reconstruction of objects and scenes from camera images is of great interests due to its wide applications. Reconstruction based on feature point correspondence is an established approach. Existing research on curve-based reconstruction is limited to certain type of curves and constrained by case-dependent reconstruction accuracy. In view of that, this paper developed a new method to reconstruct general 3D curves from stereo images. Under proposal, a B-spline curve fitting is applied to sets of 2D edge points extracted from acquired stereo images. Derived approximating parametric curves are then used to construct conic surfaces. Further, robust iterative algorithms are developed to get intersection of corresponding conic surfaces to recover 3D curve. Due to the method design, proposed approach can reconstruct general 3D curves including both open and closed curves. The curve fitting technique and developed robust algorithms can meet accuracy requirement of many real applications. Validity of the proposed method is verified through experiments on a cylinder and teacup in laboratory and a real forging within a workshop.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Szeliski, R., Computer vision: algorithms and applications. Springer Science & Business Media (2010)

  2. Komodakis, N., Tziritas, G.: Real-time exploration and photorealistic reconstruction of large natural environments. Visual Comput. 25(2), 117–137 (2009)

    Google Scholar 

  3. Zhu, C., Leow, W.: Textured mesh surface reconstruction of large buildings with multi-view stereos. Visual Comput. 29(6–8), 609–615 (2013)

    Google Scholar 

  4. Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press,Cambridge. ISBN 978-0-521-54051-3 (2003)

  5. Kobbelt, L., Botsch, M.: A survey of point-based techniques in computer graphics. Comput. Graph. 28(6), 801–814 (2004)

    Google Scholar 

  6. Berger, M., Tagliasacchi, A., Seversky, L.M., Alliez, P., Guennebaud, G., Levine, J.A., Sharf, A., Silva, C.T.: A survey of surface reconstruction from point clouds. Comput. Graph. Forum 36(1), 301–329 (2016)

    Google Scholar 

  7. Zhao, C.S., Mohr, R.: Global three-dimensional surface reconstruction from occluding contours. Comput. Vis. Image Underst. 64(1), 62–96 (2010)

    Google Scholar 

  8. Habbecke, M., Kobbelt, L.: Iterative multi-view plane fitting. In: Vision, Modelling, and Visualization, pp. 73–80 (2006)

  9. Lieu, D.K., Sorby, S.A.: Visualization, Modeling, and Graphics for Engineering Design. Nelson Education, Scarborough (2015)

    Google Scholar 

  10. Fan, B., Wu, F., Hu, Z.: Robust line matching through line–point invariants. Pattern Recognit. 45(2), 794–8059 (2012)

    Google Scholar 

  11. Zhang, J., Zhao, Q., Xin, F., Luo, Z.: Line matching based on characteristic ratio invariants of collinear points. In: Fourth International Conference on Digital Home (2012)

  12. Tang, A.W.K., Ng, T.P., Hung, Y.S., Leung, C.H.: Projective reconstruction from line-correspondences in multiple uncalibrated images. Pattern Recognit. 39(5), 889–896 (2006)

    MATH  Google Scholar 

  13. Mai, F., Hung, Y.S., Chesi, G.: Projective reconstruction of ellipses from multiple images. Pattern Recognit. 43(3), 545–556 (2010)

    MATH  Google Scholar 

  14. Zhang, X.B., Tang, A.W.K., Hung, Y.S.: Projective reconstruction of general 3D planar curves from uncalibrated cameras. Int. Symp. Visual Comput. 2, 21–30 (2010)

    Google Scholar 

  15. Kaminski, J.Y., Shashua, A.: Multiple view geometry of general algebraic curves. Int. J. Comput. Vis. 56(3), 195–219 (2004)

    Google Scholar 

  16. Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: a factorization method. Int. J. Comput. Vis. 9(2), 137–154 (1992)

    Google Scholar 

  17. Sturm, P., Triggs, B.: A factorization based algorithm for multi-image projective structure and motion. Proc. Eccv96 Lond. 1065(3), 709–720 (1996)

    Google Scholar 

  18. Carr, J.C., Beatson. R.K., Cherrie, J.B., Mitchell, T.J., Fright, W.R., Mccallum, B.C., Evans, T.R.: Reconstruction and representation of 3D objects with radial basis functions, pp. 67–76 (2001)

  19. Schöps, T., Sattler, T., Häne, C., Pollefeys, M.: Large-scale outdoor 3D reconstruction on a mobile device. Comput. Vis. Image Underst. 157(C), 151–166 (2017)

    Google Scholar 

  20. Pollefeys, M., Gool, L.V., Vergauwen, M., Verbiest, F., Cornelis, K., Tops, J., Koch, R.: Visual modeling with a hand-held camera. Int. J. Comput. Vis. 59(3), 207–232 (2004)

    Google Scholar 

  21. Faugeras, O., Laveau, S., Robert, L., Csurka, G., Zeller, C.: 3-D reconstruction of urban scenes from sequences of images. Comput. Vis. Image Underst. 69(3), 145–168 (1995)

    Google Scholar 

  22. Heinly, J., Schonberger, J.L., Dunn, E., Frahm, J.M.: Reconstructing the world* in six days. In: Computer Vision & Pattern Recognition (2015)

  23. Agarwal, S., Snavely, N., Simon, I., Seitz, SM., Szeliski, R.: Building Rome in a day. In: IEEE International Conference on Computer Vision, pp. 72–79 (2009)

  24. Zhang, S.: High-speed 3D shape measurement with structured light methods: a review. Opt. Lasers Eng. 106, 119–131 (2018)

    Google Scholar 

  25. Gay, P., Rubino, C., Bansal, V., Del, Bue, A.: Probabilistic structure from motion with objects (psfmo). In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3075–3084 (2017)

  26. Baldacci, A., Bernabei, D., Corsini, M., et al.: 3D reconstruction for featureless scenes with curvature hints. Visual Comput. 32, 1605–1620 (2016)

    Google Scholar 

  27. Hartley, R.I.: Projective reconstruction from line correspondences. In: IEEE Computer Society Conference on Computer Vision & Pattern Recognition, pp. 903–907 (1994)

  28. Hartley, R.I.: Lines and points in three views and the trifocal tensor. Int. J. Comput. Vision 22(2), 125–140 (1997)

    MathSciNet  Google Scholar 

  29. Hofer, M., Maurer, M., Bischof, T.: Improving sparse 3D model for man-made environments using line-based 3D reconstruction. In: International Conference on 3d Vision (2014)

  30. Hofer, M., Maurer, M., Bischof, H.: Efficient 3D scene abstraction using line segments. Comput. Vis. Image Underst. 157, 167–178 (2016)

    Google Scholar 

  31. Zhang, G., Jin, H.L., Lim, J., Suh, I.H.: Building a 3-D line-based map using stereo SLAM. IEEE Trans. Robot. 31(6), 1364–1377 (2015)

    Google Scholar 

  32. Pumarola, A., Vakhitov, A., Agudo, A., Sanfeliu, A., oreno-Noguer, F.: PL-SLAM: real-time monocular visual SLAM with points and lines. In: IEEE International Conference on Robotics & Automation, pp. 4503–4508 (2015)

  33. Ma, S.D., Li, L.: Ellipsoid reconstruction from three perspective views. In: International Conference on Pattern Recognition, pp. 344–344 (1996)

  34. Quan, L.: Conic reconstruction and correspondence from two views. Pattern Analysis & Machine Intelligence IEEE Transactions on 18(2), 151–160 (1996)

    MathSciNet  Google Scholar 

  35. Shashua, A., Toelg, S.: The quadric reference surface: theory and applications. Int. J. Comput. Vis. 23(2), 185–198 (1997)

    Google Scholar 

  36. Cross, G., Zisserman, A.: Quadric reconstruction from dual-space geometry. In: IEEE International Conference on Computer Vision, pp. 25–31 (1998)

  37. Kaminski, J.Y., Fryers, M., Shashua, A., Teicher, M.: Multiple view geometry of non-planar algebraic curves. In: IEEE International Conference on Computer Vision, pp. 181–186 (2001)

  38. Sparr, G.: Simultaneous reconstruction of scene structure and camera locations from uncalibrated image sequences. In: Proceedings of International Conference on Pattern Recognition, vol. 1, p. 328 (2001)

  39. Heyden, A.: Projective structure and motion from image sequences using subspace methods. Proc. Scand. Conf. Image Anal. 2(7), 963–968 (1997)

    Google Scholar 

  40. Berthilsson, R., Heyden, A., Sparr, G.: Recursive structure and motion from image sequences using shape and depth spaces. In: Conference on Computer Vision & Pattern Recognition (1997)

  41. Heyden, A., Berthilsson, R., Sparr, G.: An iterative factorization method for projective structure and motion from image sequences. Image Vis. Comput. 17(13), 981–991 (1999)

    Google Scholar 

  42. Berthilsson, R., Astrom, K., Heyden, A.: Reconstruction of curves in R3, using factorization and bundle adjustment. In: Seventh IEEE International Conference on Computer Vision (2001)

  43. Berthilsson, R., Strm, K.: Reconstruction of 3D-curves from 2D-images using affine shape methods for curves. In: Conference on Computer Vision & Pattern Recognition, pp. 476–481 (1997)

  44. Berthilsson, R., Strm, K., Heyden, A.: Projective reconstruction of 3D-curves from its 2D-images using error models and bundle adjustments. In: Proceedings of the Scandinavian Conference on Image Analysis, pp. 963–968 (1997)

  45. Wu, H.: Yu Yizhou: photogrammetric reconstruction of free-form objects with curvilinear structures. Visual Comput. 21(4), 203–216 (2005)

    Google Scholar 

  46. Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22, 1330–1334 (2000)

    Google Scholar 

  47. Canny, J.: A Computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 6, 679–686 (1986)

    Google Scholar 

  48. Hu, M., Feng, J., Zheng, J.: An additional branch free algebraic B-spline curve fitting method. Visual Comput. 26(6–8), 801–811 (2010)

    Google Scholar 

  49. Yang, Z., Deng, J., Chen, F.: Fitting unorganized point clouds with active implicit B-spline curves. Visual Comput. 21(8–10), 831–839 (2005)

    Google Scholar 

  50. Abbena, E., Salamon, S., Gray, A.: Modern Differential Geometry of Curves and Surfaces with Mathematica. Chapman and Hall, London (2017)

    MATH  Google Scholar 

  51. Aigner, M., Juettler, B.: Robust fitting of implicitly defined surfaces using Gauss–Newton-type techniques. Visual Comput. 25(8), 731–741 (2009)

    Google Scholar 

  52. Wang, Z., Wu, F., Hu, Z.: MSLD: a robust descriptor for line matching. Pattern Recognit. 42(5), 941–953 (2009)

    Google Scholar 

  53. Fusiello, A., et al.: A compact algorithm for rectification of stereo pairs. Mach. Vis. Appl. Int. J. 12(1), 16–22 (2000)

    Google Scholar 

  54. Weyrich, T., et al.: Post-processing of scanned 3D surface data. SPBG 4, 85–94 (2004)

    Google Scholar 

  55. Umbaugh, S.E.: Digital Image Processing and Analysis: Human and Computer Vision Applications with CVIPtools. CRC Press, Boca Raton (2010)

    Google Scholar 

  56. Shrivakshan, G.T., Chandrasekar, C.: A comparison of various edge detection techniques used in image processing. Int. J. Comput. Sci. Issues (IJCSI) 9(5), 269 (2012)

    Google Scholar 

  57. Abdel-Malek, K., Yeh, H.J.: On the determination of starting points for parametric surface intersections. Comput. Aided Des. 29(1), 21–35 (1997)

    Google Scholar 

  58. Wang, W., Goldman, R., Tu, C.: Enhancing Levin’s method for computing quadric-surface intersections. Comput. Aided Geom. Des. 20(7), 401–422 (2003)

    MathSciNet  MATH  Google Scholar 

  59. Chen, Y., Shen, L.Y., Yuan, C.M.: Collision and intersection detection of two ruled surfaces using bracket method. Comput. Aided Geom. Des. 28(2), 114–126 (2011)

    MathSciNet  MATH  Google Scholar 

  60. Bajaj, C.L., Hoffmann, C.M., Lynch, R.E., Hopcroft, J.E.H.: Tracing surface intersections. Comput. Aided Geom. Des. 5(4), 285–307 (1988)

    MathSciNet  MATH  Google Scholar 

  61. Farin, G.: Curves and Surfaces for Computer-Aided Geometric Design: A Practical Guide. Elsevier, Amsterdam (2014)

    MATH  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China under Grant Nos. 51975119 and 51575107. This financial support is gratefully acknowledged. We also thank Shanghai Xinmin Heavy-duty Forging Limited for providing the workshop in Dongtai that enabled our third experiment.

Author information

Authors and Affiliations

  1. School of Mechanical Engineering, Southeast University, Nanjing, 211189, China

    Yijun Zhou, Jianan Zhao & Chen Luo

  2. Chengxian College, Southeast University, Nanjing, 210088, China

    Yijun Zhou

Authors
  1. Yijun Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jianan Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chen Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chen Luo.

Ethics declarations

Conflict of interest

This manuscript describes original work, and all authors approved the manuscript and declared they have no conflict of interests to this work.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, Y., Zhao, J. & Luo, C. A novel method for reconstructing general 3D curves from stereo images. Vis Comput 37, 2009–2021 (2021). https://doi.org/10.1007/s00371-020-01959-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-020-01959-6

Keywords

Search

Navigation