Elsevier

Pattern Recognition

Volume 36, Issue 6, June 2003, Pages 1255-1268
Pattern Recognition

3D reconstruction of free-formed line-like objects using NURBS representation

https://doi.org/10.1016/S0031-3203(02)00095-XGet rights and content

Abstract

3D reconstruction of arbitrary free-formed objects is an important and challenging problem in computer vision. In this paper, we first discuss the importance of primitive selection in 3D reconstruction. Subsequently, a theorem, which reveals the perspective invariance of NURBS, is shown, making it a good choice as primitive in 3D reconstruction. Consequently, based on this theorem a new paradigm of free-formed line-like object reconstruction using NURBS as primitives is proposed. Furthermore, an approach for determining weights for 3D NURBS is presented, and the width effect of curved line-like objects is analyzed. Finally, experiments with line-like objects and machine part demonstrate the feasibility of our approach and prove the superiority of our approach over the point- or segment-based approaches as well as the B-spline-based reconstruction approach in terms of robustness and accuracy.

Introduction

Humans, almost effortlessly, can understand a wide variety of shapes. However, finding a useful and general method for machine representation of shapes has been proven to be nontrivial. In fact, inferring 3D shape of objects in a scene from their perspective views is one of the most important and challenging problems in computer vision and has been at the core of computer vision from the early days. There are at least three reasons why solving this problem is a formidable endeavor. Firstly, by acquiring an image of 3D world using a camera, the information in depth dimension is lost during imaging process. Secondly, 2D images are related in a complex way to the structure of real world through the physics of image formation and its mapping geometry. Thirdly, since we are dealing with real sensors, we are confronted with the problem of processing noisy signals. This turns out to be a very complicated problem since this initial uncertainty must be tracked through all the representations that are built up by the system in order to achieve a final high accuracy 3D representation. Since the milestone work by Marr and Poggio two decades ago [1], a variety of approaches have been proposed worldwide. Among these approaches, the most difficult and time-consuming problem is to find corresponding points in different views. This raises the following question: Is it possible to reconstruct 3D object from its 2D images without this kind of pixel-to-pixel correspondence? The answer is yes if we use more global primitives in correspondence point matching. Therefore, the selection of matching primitives plays an important role in such an application of stereo vision. Up to now different kinds of primitives had been proposed, including edge points [2], set of points [3], segments [4], lines [5] as well as quadric or conic curves [6], [7]. The main reasons for using these primitives are the following:

  • (1)

    Availability: Edge points, vertices, or cusps are the most important features in scene; line segments are the components of polyhedral objects while many man-made objects have quadric curve edges.

  • (2)

    Detectability: Edge points, line segments and quadric curves can be extracted easily. There are a lot of edge detection and line segment extraction approaches available [8], [9] while quadric curve extraction can be found in [10], [11], [12].

  • (3)

    Perspective invariance: All structures of primitives mentioned above are preserved under perspective projection.


The implicit constraint in these primitives is a planar constraint. Unfortunately, in free-formed object reconstruction this constraint is not tenable. An intuitive deduction is to divide free-formed curves into piecewise line segments or conic arcs first. Then reconstruct them, respectively. Finally, join them together to form the reconstruction results like in Ref. [13]. But we argue that the merging of reconstructed segments itself is a very difficult problem and the errors in segment reconstruction will be accumulated sequentially if we simply put together one after another. Also, we conjecture because of the same reason, the reconstruction error in Ref. [13] is considerably large.

In this paper we will focus on the 3D reconstruction of free-formed line-like objects for the following reasons:

  • (1)

    The intersection of two free-formed 3D surface patches is usually a non-planar curve.

  • (2)

    If the object has some patterns on the curved surfaces, the boundaries of these patterns also will be non-planar curves.

  • (3)

    In the case of objects consisting of many small smooth surface patches, estimating non-planar curves of surface intersections may be more useful than estimating the surface patches themselves.

  • (4)

    Skeletons of pipe-like objects, like the Constant cross section Generalized Cylinders (CGCs) in Ref. [14], are often non-planar.


For the simplification of description, all these contours or skeletons are considered as line-like objects in the sequel. It is well known that NURBS is an efficient mathematical method for free-formed curve representation in computer graphics and in CAD/CAM/CAE applications [15], [16]. It has the following strengths:

  • (1)

    Unified representation of 2D and 3D curves. This implies that the same form of representation can be used both to extract 2D curve projections in images and to reconstruct 3D object in space.

  • (2)

    NURBS is completely determined by their control points, both in 2D and 3D cases. Thus, the reconstruction of free-formed line-like object is turned to the reconstruction of control points.

  • (3)

    Same form of representation in curves and surfaces: If we use 2D NURBS function and an array of control points to replace 1D BURBS function and control point vectors, a NURBS representation of free-formed surfaces can be obtained. Although we primarily address 3D reconstruction of free-formed line-like objects in this paper, due to this advantage our approach can be extended to 3D reconstruction of free-formed surfaces as well if a grid pattern is projected on the surface of the object. In this case, an array of control points will determine the 3D reconstructed surface, avoiding the errors and computational burden caused by interpolation of sparse 3D points.

  • (4)

    Standard geometry format: Because there are many CAD models now using NURBS representation, this kind of reconstruction can be applied easily in practical CIM, robot or virtual reality systems, etc.


All these desirable properties indicate the potential of NURBS as primitives of matching in 3D reconstruction. However, is it invariant with respect to perspective projection? How to reconstruct free-formed line-like objects with NURBS? These kinds of problems will be addressed in this paper in details. In the next section we review the relevant previous work and contrast it with our work. In Section 3, first we proved the perspective projection invariant of NURBS; then address the problem of how to determine the weights of 3D NURBS curves. In Section 4, the width effect of free-formed line-like objects on reconstruction is analyzed. In Section 5, the experiments of reconstruction using NURBS-based approach are presented. At the same time reconstruction accuracy analysis and comparison with point-based reconstruction, segment-based as well as the B-spline-based reconstruction approaches are given. All results illustrate that our reconstruction approach is superior in accuracy as well as the speed compared to the point-to-point reconstruction, the segment-based reconstruction, and the B-spline-based reconstruction. Finally, we conclude our work in Section 6.

Section snippets

Previous work

There are considerable previous works relevant to our work. They can be categorized into two classes. One class deals with the reconstruction of space curves while the other is the application of parametric curves in computer vision. Before reviewing the relevant previous works we state the problem of 3D reconstruction of free-formed objects first.

NURBS-based reconstruction

In [38], [39] the 3D reconstruction approaches based on B-spline representation were proposed. From their experimental results, it has been proved that the reconstruction accuracy is superior to the point-to-point vision and the segment-based vision techniques. In addition, we also noted that the fitting accuracy of object greatly effects the 3D reconstruction accuracy. It is a well-known fact that NUBRS approximation is much better than the standard B-spline approximation, particularly if

Width effect of line-like object on NURBS-based reconstruction

Till now we still assumed that the free-formed line-like object is widthless, i.e. its width can be ignored. In this section we will analyze the effect of the width of line-like object on our approach. For the sake of simplification, we consider line-like object as cylinder first. It is geometrically easy to understand that under affine projection the image of a cylinder is two parallel edge lines, which can assure the skeletons in different viewing images to be corresponding to the same

Experimental results

All images used in our experiments are captured in a trinocular vision system. Our program is running on a PC computer with Pentium II/233 processor using Matlab programming language. The typical executing time without curve fitting is about 260ms for the whole reconstruction.

Conclusion

In stereo vision, the accurate reconstruction of free-formed objects in 3D is confused by errors of edge extraction and correspondence matching. In this paper, we developed a new paradigm using NURBS as primitives to 3D reconstruction. Due to the more global nature of this kind of primitive as well as the perspective invariance, more accurate, robust and continuous reconstruction is obtained compared to the point-to-point-based, the segment-based approach, and even the B-spline-based approach.

About the Author—MINGYUE DING graduated from Beijing University of Aerospace and Aeronautics in 1982 and received his MS degree from The Chinese University of Electronic Science and Technology, Chengdu, China and his Ph.D. degree from Huazhong University of Science and Technology, Wuhan, China in 1985 and 1988. Since 1993, he is a professor at the Institute for Pattern Recognition and Artificial Intelligence, Huazhong University of Science and Technology. Currently he is on leave at the John P.

References (47)

  • S.D. Ma

    Conic-based stereo, motion estimation, and pose determination

    Int. J. Comput. Vision

    (1993)
  • R.O. Duda et al.

    Use of the Hough transform to extract lines and curves in pictures

    Commun. ACM

    (1972)
  • R.N. Czerwinski et al.

    Line and boundary detection in speckle images

    IEEE Trans. Image Process.

    (1998)
  • D.B. Cooper et al.

    On the computational cost of approximating and recognizing noise-perturbed straight lines and quadric arcs in the plane

    IEEE Trans. Comput.

    (1976)
  • M. Ding, C. Yan, Quadratic matching based reconstruction of space curve, International Symposium on Multi-Spectral...
  • F. Ulupinar et al.

    Shape from contourstraight homogenous generalized cylinders and constraint cross section generalized cylinders

    IEEE Trans. Pattern Anal. Mach. Intelligence

    (1995)
  • L. Piegl

    On NURBSa survey

    IEEE Trans. Comput. Graphic Appl.

    (1991)
  • G. Farin (Ed.), NURBS for curves and surface design,...
  • R. Szeliski et al.

    Matching 3D anatomicak surfaces with non-rigid deformations using Octree–Splines

    Int. J. Comput. Vision

    (1996)
  • Z. Zhang

    Iterative point matching for registration of free-formed curves and surfaces

    Int. J. Comput. Vision

    (1994)
  • M. Ding et al.

    Using space continuity and orientation constrain for range data acquisition

    Pattern Recognition

    (1994)
  • I.D. Faux et al.

    Computational Geometry for Design and Manufacture

    (1979)
  • F.S. Cohen et al.

    Invariant matching and identification of curves using B-spline representation

    IEEE Trans. Image Process.

    (1995)
  • Cited by (5)

    • NURBS-based geometric inverse reconstruction of free-form shapes

      2017, Journal of King Saud University - Computer and Information Sciences
      Citation Excerpt :

      To utilize the advantages of the above mentioned properties of the NURBS, very few efforts were made in the direction of the 3D reconstruction of the free-form curves and surfaces. Ding et al. (2003) proposed a method for reconstructing free-form space curves from their stereo views under the affine projection model. However, the obtained results demonstrate a possibility to use splines for the space curve reconstruction under the projective model.

    • Two-view curve reconstruction based on the snake model

      2011, Journal of Computational and Applied Mathematics
      Citation Excerpt :

      Taylor [2] adopts single-variable model and point correspondences to reconstruct combined objects in a single uncalibrated image. Ding et al. [3] discuss the importance of the chosen of primitives in reconstruction problems. Based on that, they propose a curve reconstruction method using NURBS as primitives and point out its advantages in perspective reconstruction comparing with using point, line and B-spline primitives.

    • Geometric Reconstruction of Cement Microstructure Image Based on Edge Response Scattering Points Extraction and MLS

      2020, 2020 7th International Conference on Information, Cybernetics, and Computational Social Systems, ICCSS 2020
    • Data modeling using NURBs curves and modified genetic algorithms

      2010, ASME International Mechanical Engineering Congress and Exposition, Proceedings (IMECE)
    • Scan line algorithm for NURBS surface trimming

      2007, Jisuanji Jicheng Zhizao Xitong/Computer Integrated Manufacturing Systems, CIMS

    About the Author—MINGYUE DING graduated from Beijing University of Aerospace and Aeronautics in 1982 and received his MS degree from The Chinese University of Electronic Science and Technology, Chengdu, China and his Ph.D. degree from Huazhong University of Science and Technology, Wuhan, China in 1985 and 1988. Since 1993, he is a professor at the Institute for Pattern Recognition and Artificial Intelligence, Huazhong University of Science and Technology. Currently he is on leave at the John P. Robarts Research Institute, London, Canada. Since 1987, he has published more than 120 academic papers in computer vision, image processing, pattern recognition as well as path planning.

    About the Author—YIJUN XIAO obtained his BA, MS, and Ph.D. degrees in 1993, 1996, 2000, respectively, from Huazhong University of Science and Technology, Wuhan, China. Currently he is a research assistant at the Department of Computing Science, University of Glasgow, United Kingdom.

    About the Author—JIAXIONG PENG is a professor, the first doctoral advisor and the founder for the doctoral program of Pattern Recognition and Intelligent System at Huazhong University of Science and Technology, Wuhan, China. He graduated from Northeastern University in 1957, has published over 200 journal papers and more than 150 conference papers in pattern recognition, information processing and automation. Ninety-six of them were collected by SCI, EI, ISTP and ISR. Also he got 81 different prizes.

    About the Author—FRIEDRICH WAHL studied electrical engineering at the Technical University of Munich, where he received his Diploma, his Ph.D. and the ‘venia legendi’ in digital signal and image processing in 1974, 1980 and 1984 respectively. From 1974 to 1981 Dr. Wahl conducted research at the Institute of Communication Engineering at the Technical University of Munich in the fields of pattern recognition, signal and image processing. From 1981 to 1986 Dr. Wahl worked at the IBM Research Labs in San Jose and Zurich in the areas of document analysis, industrial image analysis and machine vision. Since 1986 Dr. Wahl is Professor of Computer Science at the Technical University of Braunschweig, Germany, where he set up the new Institute for robotics, computer vision, pattern recognition and all aspects of computer science.

    The work is sponsored by the National Nature Science Foundation of China under the grant of 69975005 as well as the Alexander von Humboldt Foundation of Germany, partly supported by Trans-Century Outstanding Investigator Foundation of Education Ministry of China.

    View full text