Recognizing 3-D objects by using a Hopfield-style optimization algorithm for matching patch-based descriptions

Abstract

A new method is proposed for recognizing 3-D objects by using a Hopfield-style optimization algorithm based on matching patch-based image and model descriptions. To obtain the image descriptions, range images are employed to extract reliable high-level patch features. In the optimization process, the objective function is a Liapunov function which encodes a set of geometric constraints on the descriptions. The optimization is implemented in a Hopfield network with its interconnections encoding the imposed unary, binary and bounding edge constraints. At first, the paper makes an explanation on a new pre-processing method for deriving the required image description. It then presents the structure of the used Hopfield network that is able to recognize multiple model objects all at the same time. Experimental results based on synthetic or real range images are also reported.

Introduction

Recognition of 3-D curved objects is one of serious problems to be solved for machines to understand complex scenes in real environments.[1] According to a common and dominating view in the machine vision community, we can usually divide an on-line recognition process into the following two consecutive stages:2, 3 (1) image description for extracting image features from sensor data; and (2) scene interpretation for fulfilling the recognition task on the basis of a model database (model-base). In such a model-based approach, the recognition is accomplished finally by matching the image descriptions with the object models to reason about the object occurrences and determine the poses (locations and orientations) of the objects in 3-D space.

A prominent characteristic of the model-based approach is its robustness in addressing diversity both of the objects to be recognized and the circumstances surrounding them. By exploiting the a prior model knowledge, we can reduce uncertainties due to object occlusion and errors built up during the pre-processing stage. At the same time, however, there still exist other difficulties we face in constructing systems useful and reliable in practical applications. Two typical ones are: (1) how to derive the image description if many complex objects appear in the image in any poses, overlapping and touching each other; and (2) how to establish the image-and-model correspondence efficiently when a large model base has to be taken into account.

In the paper, we propose a new object recognition method for solving the above two problems by using range images as input data. The central idea is to transform the image-and-model matching problem into a Hopfield-style optimization process.[4] The objective function to be optimized is defined as a Liapunov function that encodes a set of geometric constraints on the features employed in describing the model objects. It is minimized subsequently in an asynchronous Hopfield network with its synaptic interconnection weights encoding the imposed constraints and the neuron states corresponding to the mutual compatibility of image and model feature pairs.

To deal with their 3-D nature, we represent curved objects in terms of their spatial boundary patches. A patch-based description is derived from a range image by a pre-processing procedure which segments the image into isolated patches, determine their types and poses, and extract further more relational features. After that, a Hopfield network is formed with its synaptic weights encoding the unary, binary, and bounding edge similarity of all pairs of image and model patches. It then begins to evolve dynamically until reaching a stable state, which suggests a reasonable global interpretation of the image under consideration. We have made some experiments using synthetic or real images and obtained encouraging results on the practical applicability of the method.

There have been a number of papers that reported researches similar with ours each in a certain aspect. A brief review on the related work is given in the next section. Our method, however, has the following new and important contributions:

• (1) We developed a systematic approach to the object recognition problem, starting from pre-processing of raw image data. The proposed image description method is effective for range images and has bceen practically applied for processing and analyzing real complex images.

• (2) The Hopfield network used here for matching patch-based image and model descriptions has some new features that have not been mentioned thus far in the vision literature. In particular, it exploited successfully the negative evidence of patches between different models for the purpose of enforcing uniqueness constraints on the patch compatibility. For the reason, we not only improved the reliability of the recognition of single objects but also succeeded in recognizing multiple objects simultaneously from a complicated object pile.

Moreover, although it is designed for range images as the immediate input, the method can also be extended with minor modifications for other types of 3-D data, such as needle images.[5] It is also possible to extend it to scene descriptions including such 2-D features as vertices and edges.

The paper is organized as follows: Section 2gives a preliminary statement on the problem of object recognition using patch-based descriptions and then introduce some related work. Section 3presents the method of deriving the patch-based image description from dense range data. Section 4explains the detailed Hopfield-style optimization algorithm used for matching the image and model descriptions. In Section 5we show some experimental results, and in Section 6, we make several concluding remarks and point out open problems.