Dynamic motion analysis using wavelet flow surface images

Abstract

We have developed a motion analysis method that combines the wavelet transform with the flow surface image technique. In dynamic navigation environments, whenever objects move, the projections of the objects onto the image plane also move. These projections build over time spatiotemporal surfaces of their movement and volumes created by the surfaces. This paper presents a new method for the interpretation of optical flow for moving objects from a sequence of images. Flow surface images of the moving objects are created within the wavelet-derived space, chosen from seven different directionally sensitive detail images using the 3D wavelet decomposition. The motion estimation algorithm concentrates on the integration of information from the flow surface images, followed by a quadratic patch parameterization and determination of flow paths of the end points of edges on the flow surface images. The results of two experimental studies with an object exhibiting out-of-plane translation and rotation are also reported.

Introduction

An important goal of computer vision is the analysis of the motion characteristics of the observer and the objects in dynamic scenes for the recovery of 3D surface information. There are three approaches that can be used to interpret the motion of objects: gradient approach, correspondence approach, and spatiotemporal method. Good reviews can be found in (Aggarwal and Nandhakumar, 1988; Barron et al., 1994; Beauchemin and Barron, 1995). An approach which uses a fusion of results from all three methods was developed by Singh (1991).

The gradient approach is based on the computation of image changes in space and time (e.g. Horn and Schunck, 1981; Snyder, 1991). The gradient method does not allow for acceleration in the optic flow calculation and is sensitive to noise in the gradients. In addition, the brightness constancy assumption is violated by light source motion and/or shadows. The method has recently been extended by Negahdaripour (1998) to include variations in lighting by decoupling geometric and radiometric sources of scene gradients. Similar studies along these lines can be found in (Cornelius and Kanade, 1983; Schunck, 1985; Nagel, 1989; Verri and Poggio, 1989). A recent experimental study by Brandt (1997) indicated that small-support low-pass prefiltering of the input sequence led to better accuracy in the optic flow computation.

In the correspondence method, points, lines, or regions of objects in the first image are put into correspondence with similar features later in the sequence (e.g. Huntsberger and Jayaramamurthy, 1987; Liu and Huang, 1986; Roach and Aggarwal, 1980). A study by Weng et al. (1993) indicated that even small errors of one or two pixels in image coordinates used for point correspondence cause an ambiguous interpretation of translation or rotation due to violation of the epipolar constraint. Their solution was to use a two-step approach that consisted of a linear solution of the closed-form correspondence equations, followed by a minimization of image errors using a minimum variance estimator technique. A recent analysis by Zhang (1995) demonstrated that 3D motion and structure parameters can be derived from corresponding line segments in just two perspective images. His study uses an epipolar (EPI) geometry constraint on the common part of the corresponding line segments.

Another approach for motion analysis and the method used in our study is to consider a 3D spatiotemporal image (e.g. Allmen and Dyer, 1991; Baker and Bolles, 1989; Rangachar et al., 1990), where a temporal cube is built by stacking together a sequence of images. Work also related to this approach is the spatiotemporal filters of Fleet and Jepson (1990) and Heeger (1988), where a frequency space analysis is performed using tuned filters. A recent study by Otsuka et al. (1997) used a 3D Hough transform for planar surface detection to derive spatiotemporal surfaces (trajectory surfaces). Although robust, the method suffers from a high computational cost from the Hough transform. We use a local region growing approach followed by a Monge patch parameterization to derive the spatiotemporal surfaces in the wavelet coefficient spaces.

In general, 3D motion analysis computes the motion and structure of objects from a sequence of images using two steps. The first step is to compute observables (features or optical flow) in the images; followed by the extraction of the motion and structure of objects using the observables. Closed form solutions for derivation of the structure and motion paramers have been developed by Subbarao (1989) and Waxman et al. (1987). This paper uses a combination of feature extraction and optical flow derived from the flow surface of wavelet-derived edge points in a sequence of images for the first motion analysis step. In a further study, Chang (1997) used the solution of a linear set of equations derived from the optical flow of the surfaces to derive the structure and motion parameters of the objects.

Huntsberger et al. (1994) focused on wavelet-derived epipolar-plane image (EPI) analysis, and alleviated some of the problems in the inter pretation of translation in depth and occlusion/disocclusion. Allmen and Dyer, 1990, Allmen and Dyer, 1991 have studied the spatiotemporal surface flow method for motion estimation. They assumed that the arc length of a segment is constant throughout the temporal slices. Although this assumption is applied to vertical and horizontal movement, out-of-plane translation would be problematic. Chang and Huntsberger (1996) in their study of a wavelet flow surface motion analysis method created spatiotemporal surfaces for pure translation and pure rotation of simple lines. The partial derivatives from the patch parameterization of the flow surface were then used to derive the optical flow. The wavelet method has an advantage over that of Allmen and Dyer (1990) in that directional information is automatically supplied by the wavelet channels, which leads to a reduction in the amount of computational processing.

In this paper, we present a new approach for the derivation of optical flow using wavelet flow surface images. This approach does not require texture for differentiation or correlation, but only uses spatiotemporal flow surfaces. The new method applies factors for noise reduction, feature grouping, and surface smoothness. Using the directional sensitivity of wavelet-derived coefficients, the computational speed is faster than the previous methods. The slope of the wavelet-derived flow surfaces are computed to determine optic flow for out-of-plane translation and rotation of a rigid object.

Section 2contains a brief review of wavelet decomposition and characteristics of 3D wavelet coefficients. This is followed by a discussion of the flow surface method for the extraction of the optical flow. The final section gives the results of some experimental studies of a object undergoing out-of-plane translation and rotation in an industrial environment.