Elsevier

Pattern Recognition Letters

Volume 22, Issue 2, February 2001, Pages 105-121
Pattern Recognition Letters

3-D localization and feature recovering through CAD-based stable pose calculation

https://doi.org/10.1016/S0167-8655(00)00087-8Get rights and content

Abstract

CAD-based computer vision methodologies have been used in practice for simplifying the process of geometric inspection. In general, it can be assumed that a three-dimensional (3-D) object lying on a planar surface has a finite number of stable poses. These poses precisely determine all feasible positions and orientations of an object, and can be utilized for localization and accurate shape matching. In this paper, procedures are proposed to efficiently determine all stable poses of piecewise smooth curved 3-D objects, and to perform 3-D localization and feature recovering. The stable pose determination process is based on a faceted approximation of the associated CAD model built by a CAD solid modeling package. Localization and feature recovering calculations are derived from Lowe's method. Several real 3-D objects are presented to illustrate the proposed methodology.

Introduction

An object's pose usually represents the rigid body transformation between the object frame (o,x,y,z) and the world frame (O,X,Y,Z). Throughout this paper, the world frame represents the frame of a vision system defined by the camera and the plane supporting the object. The pose of an object is specified by its position and orientation. Generally, it can be assumed that a three-dimensional (3-D) object has only a finite set of stable poses. When an object is in a stable position resting on a supporting plane (e.g., a table top or a belt conveyor), the potential energy of the object becomes a local minimum. Knowing all stable poses of an object is useful for many tasks in machine vision and robotics, including recognition and localization and grasp planning. Camps et al. (1991) used the geometric information of the stable poses of objects with planar surfaces along with the knowledge of the surface reflectance properties, light sources, sensors, and image processing operators to symbolically predict the features that will appear on the images of the objects in their object recognition/localization system. Wiegley et al. (1992) presented a method for estimating the distribution of stable poses in a stream of identical polyhedral parts that are randomly dropped under quasi-static conditions. Kriegman (1997) analyzed the problem of computing maximal capture regions for an object with a smooth convex hull assuming that its dynamics are dissipative. By knowing the capture region of the desired stable pose, it is only necessary to orient the object within that region since gravity will do the rest. The prediction methods proposed by Wiegley et al. and Kriegman have valuable applications in automatic robot planning and in the design of part feeders.

Consider the prototypical task of determining the identity and performing the geometric inspection of a machined part traveling along a conveyor belt into a visual inspection workcell. Typically, there are only a few part types under consideration, and geometric models that describe their shape are readily available from CAD systems. The location of the camera is also known, and a supporting plane constrains the object location vertically. If the scene is static, the object must be lying in a stable position. If it is known that the object is in a specific stable pose, the unknown spatial relationship between the part and the inspection system is reduced from six to three degrees of freedom (two for location and one for orientation).

The typical camera setting of a machine vision system in a manufacturing environment takes the optical axis of the camera as the direction of gravity, and the supporting plane is set horizontally. Obviously, the number of rotational degrees of freedom of an object is only one for a given pose, and the axis of rotation is perpendicular to the supporting plane. Clearly, a pose is stable only if the object is in contact with a supporting plane; otherwise, the object would fall. The vertical location of the supporting plane and the horizontal location of the object on the supporting plane, however, are unrelated to the geometric configuration of the object. Consequently, computing the stable pose of the object here is reduced to finding the longitudinal and latitudinal rotations between the object frame (o,x,y,z) and the world frame (O,X,Y,Z), which are around the X and Y axes, respectively, without considering the three translations and the rotation around the Z axis.

It is well known that the stable poses of an object can be found from its convex hull, and this is feasible for the case of polyhedral objects since several convex hull algorithms are available (Preparata and Shamos, 1985). However, such algorithms are no longer applicable when objects have curved surfaces, although there has been recent progress (Dobkin and Souvaine, 1990, Schäffer and Wyk, 1987). Murase and Nayar (1995) proposed a method of visual learning and recognition of 3-D objects from their appearance in 2-D images. A large set of images for an object's stable configurations is generated by automatically changing pose and illumination in small increments. This set can be compressed to attain a low dimensional subspace, called the eigenspace. Specifically, the eigenspace of the image set is obtained by computing the most prominent eigenvectors of the set. The proposed recognition system projects the image to the eigenspace to determine the object's pose in the image. This method only needs two appearance features, namely, object pose and illumination, rather than geometric features, such as edges, lines, and curves. This method unfortunately cannot handle substantial degrees of occlusion, and a large number of dimensions and huge memory requirements are required to perform robust recognition. This approach is not appropriate for geometric inspection because it cannot provide any dimensional information. Kriegman (1992) provided the conditions for the pose of a piecewise smooth curved 3-D object to be stable. Expressions in terms of polynomial equations were derived to find points of contact between the object and the supporting plane. The contact points can be determined by relating local curve geometry with the physical center of mass (c.o.m.) of the object. Nevertheless, the computational effort of solving those systems of polynomial equations is extremely heavy and this approach is very difficult to be automatically implemented for generic 3-D curved objects.

One of the recent trends in applying machine vision to manufacturing consists of expanding the role of CAD models (Newman and Jain, 1995). CAD models contain exact geometric specifications of the corresponding objects. In addition, the models are preexisting. Thus, they provide a ready and well-defined pattern for inspection. Most CAD models can provide a mathematical description of the shape of an object, including an explicit parameterization of the surface shape and a plain encoding of inter-surface relationships. Another advantage of utilizing CAD representations in machine vision is their flexibility. It is easy to add new object models to the inspection system. This paper presents an approach for calculating the stable poses of piecewise smooth curved 3-D objects based on the polyhedral approximation of their CAD models.

Once the identity of an object and/or its stable pose are unknown, the information about all possible stable poses of the candidate objects can be used to simplify the recognition, localization and shape matching processes. One of the most common 3-D model representation schemes is the viewer-centered representation that is particularly suitable for 3-D object recognition and localization from a single 2-D image. In this representation scheme, each 3-D object in the model database is represented as a set of 2-D projections taken from evenly distributed viewpoints in the off-line stage. A 3-D object recognition and localization problem using a single image is then converted to a 2-D pattern-matching problem to find the correct object and the correct viewpoint. This widely used scheme is inadequate for industrial inspection applications due to the following two reasons. First, the number of evenly distributed 2-D projections required to represent a 3-D object generally ranges from several tens to several hundreds depending on the complexity of the object. Such a large number of projections makes the on-line manipulation of the model database very inefficient and reduces the performance of the pattern matching procedure dramatically in terms of reliability and speed. Since the relative spatial relationship between the camera and the part being inspected is usually fixed in a manufacturing environment, the size of the model database can be significantly reduced by representing each 3-D object with a reduced set of 2-D projections corresponding to the stable poses of the object. Second, for accurate shape matching and geometric inspection, all six transform parameters (three parameters for location and three for rotation) must be determined between the object model and the captured scene. By determining the stable poses of the objects under inspection off-line, the on-line shape matching process is considerably simplified due to the reduction of the rotational degrees of freedom from three to one. Even if the stable poses are only approximately determined, they can still improve certain shape matching procedures, such as Lowe's iterative method (Lowe, 1987), with good initial values for two rotational transform parameters. An extension of Lowe's method that uses stable pose information is also proposed in this paper for 3-D localization and accurate shape matching.

The remainder of this paper is organized as follows. The stable pose calculation method is presented in Section 2, and the method is illustrated in Section 3 with three real 3-D objects. Section 4 shows how stable pose information can be used to simplify and speed up 3-D localization and feature recovering by using a modified Lowe's procedure. In Section 5, the solution found by the modified Lowe's method is stabilized by applying the singular value decomposition method. In Section 6, a real 3-D object is presented to demonstrate the proposed algorithm. Finally, concluding remarks are provided in Section 7.

Section snippets

Stable pose calculation using CAD models

A piecewise smooth 3-D object can be in contact with a supporting plane at surface points, along edges or at vertices. Combinations of these three types of contact lead to the consideration of several cases. A stable pose may have one contact point, two contact points or three contact points. A single contact point must be on a (smooth) surface. A straight contact edge can be thought of as two contact points. A planar contact surface can be thought of as three non-colinear contact points since

Stable pose calculation application examples

The proposed approach has been implemented and successfully employed to determine the stable poses of several real objects. The algorithm has been coded in C and implemented on a Gateway 2000 120 MHz Pentium machine. In this section, three real objects, a gampak, a blade adapter, and a cutting tool, are provided to illustrate the proposed approach.

Fig. 1 shows the CAD model of the gampak, an object used to support a game joystick. A faceted model with 173 facets is created in .stl file format.

Applying stable pose information to 3-D localization and flat feature recovering

3-D localization and shape analysis from a 2-D scene involves inverting the projection that occurs from the 3-D world into a 2-D image. Thus, the imaging process must be modeled to recover the camera transform parameters of an object with respect to its image. It is well-known that the shape of a 3-D object in the image plane is distorted by the perspective effect. For geometric inspection applications through shape matching, it is necessary to employ an approach based on the exact deviation

Stabilizing the solution for the iterative posing method using singular value decomposition

As long as there are more equations than unknowns, Lowe's iterative posing method, as described in Section 4, will usually converge in a steady manner from a wide range of starting positions. In industrial inspection applications, however, it is often the case that some of the matches are parallel or have other relationships that may lead to an ill-conditioned solution. To stabilize the solution, Lowe (1991) introduced prior constraints on the desired solution that specify the default to be

Application examples

The developed 3-D localization and feature recovering approach has been successfully applied to a number of real objects for determining the transformation parameters between object models and their corresponding images, and subsequently recovering the planar geometric features from the perspective projections. The algorithm has been coded in double precision C and implemented on a HP 9000 C100 workstation (CPU speed of 110 MHz). One of the objects employ for stable pose calculation, the

Conclusions

A procedure has been presented to calculate the stable poses of piecewise smooth curved 3-D objects from their relevant CAD models. Then, the obtained stable poses are used for 3-D object localization and accurate shape matching. The proposed approach can determine the stable poses much faster than the generic approach that solves a system of polynomial equations by relating local curve geometry with the c.o.m. of the object. The algorithm has been implemented in C on a Gateway 2000 120 MHz

Acknowledgements

The authors would like to thank Dr. Joseph O'Rourke for providing the code to determine the convex hull of a 3-D object. This research was partially supported by the National Science Foundation under Grant DDM 90-57066.

References (14)

There are more references available in the full text version of this article.
View full text