3D object recognition from static 2D views using multiple coarse data channels

Abstract

A 3D object recognition system is described that employs novel multiresolution representation and coarse encoding of feature information. Modifications are bought to classic feature extraction methods by proposing the use of wavelet transform maxima for directing the actions of feature extraction modules. The reasons behind the use of a multi-channel architecture are described, together with the description of the feature extraction and coarse modules. The targeted field of application being automatic categorisation of natural objects, the proposed system is designed to run on ordinary hardware platforms and to process an input in a short timeframe. The system has been evaluated on a variety of 2D views of a set of 5 synthetic objects designed to present various degrees of similarity, as being rated by a panel of human subjects. Parallels between these ratings and the system’s behaviour are drawn. Additionally a small set of photomicrographs of fish larvae has been used to assess the system’s performance when presented with very similar, non-rigid shapes. For comparison, the parameters extracted from each image were fed into two categorisers, discriminant analysis and multilayer feedforward neural network with backpropagation of error. Experimental evidence is presented which demonstrates the efficacy of the methods. The satisfactory categorisation performances of the system are reported, and conclusions are drawn about the system’s behaviour.

Introduction

Computational studies of vision in the past decades have highlighted the complexity of processes involved in performing visual tasks and the inherent difficulty of building computational systems that perform 3D object recognition and scene comprehension. A series of computational studies and theories reviewed by Hildreth and Ullman [1] describe vision as a chain of processes that, based on the retinal image, yield increasingly complex representations of the visible world. Among the theories on visual information representation in biological systems (the work of Marr [2], Biederman [3], Ullman [4], [5] and Edelman and Weinshall [6]), at the present the ones discussing viewer-centred representation seen to have more experimental evidence in their support (Tarr and Pinker [7], Edelman and Bulthoff [8]). The experimental results reviewed by Edelman [9] contradict also the theories centred around the idea of representation with reconstruction [2]. Hence there was a gradual shift in vision research from theories postulating the necessity of using extremely detailed, often complete representations of the world (the work of Freuder, Tenenbaum and Barrow cited in [2]), towards more relaxed frameworks based on viewpoint-dependent encoding of views and storage of multiple views (as suggested by Ullman and Basri [5]).

Arriving at this fundamental problem of representing the visible world, an important task in the design of a recognition system is the selection of descriptors that would constitute the building blocks of the internal representation of the analysed objects. Still, the virtual impossibility of obtaining a universally valid set of features and categorisation criteria based on these has been pointed out by studies in the field of taxonomy. Sokal [10] highlighted the existence of individual differences in taxonomic judgement, since a group of human classifiers can arrive at correct categorisation of objects based on quite different sets of features considered to be salient.

In analysing images for object recognition purposes, researchers usually have tended to focus on object and image properties that are also salient to human enquiry. Thus texture descriptions [11], edge positions [12] and statistical descriptions of pixel densities [13] have all been used to segment images into their component parts. Categorisation follows, providing object recognition. Most of these methods rely on extracting very precise measurements of, for example, symmetries or shape description (as illustrated by methods proposed by Brady [14], Khotanzad and Liou [15]). None have proved reliable analysis tools for understanding natural images or images with noise and clutter obscuring the objects of interest. As an alternative, a method was developed by Ellis et al. [16] that draws on the concept of Ullman’s multiple visual routines [17]. The principle of operation is the registration at low resolution of multiple parameters that describe the object scene in an image. If many of these `coarse channels’ are analysed in concert a solution to the particular analysis may be found – one which may not be apparent when using high resolution data. This is similar in concept to finding a global minimum in a multidimensional descriptor space so often described in artificial neural network research (a good example being Rumelhart and McClelland’s work [18]). The coarse channel principle has been applied successfully to the automatic categorisation of 23 species of field collected marine plankton, in a system developed by Culverhouse et al. [19]. It is also applied here to the task of three dimensional object recognition.

This approach of non-exact feature description and low-resolution encoding of features also constitutes the central concept of other recently developed systems that do not necessarily employ multiple data channels. Bradsky and Grossberg [20] describe a system that uses in its preprocessing stage an array of Gaussian receptive fields in order to decrease the dimensionality of the data. The system developed by Mel [21] employs a large array of filters placed on the input image, the outputs being coarse coded as histograms for achieving viewpoint-invariance. In a conceptually related way, Schiele and Crowley [22], [23] have used multidimensional receptive field histograms characterising 2D views of objects in classification and in determination of favourable viewpoints for recognition. Edelman’s Chorus scheme [24] utilises a receptive field array that provides low-dimensional description of the input data. The main difference between these approaches and the authors’ system is that the attention of the system is directed by a module employing Mallat’s [25] multiresolution analysis (MRA) towards areas of the image that contain potentially relevant features for categorisation. Therefore it does not analyse the entire surface of the input image (e.g. by placing a large array of receptive fields on the image).

The proposed system constitutes an engineering solution, since the processing algorithms were designed to run on largely available hardware platforms and to perform analysis in a sufficiently short timeframe that makes it usable in laboratory conditions.