A catadioptric sensor with multiple viewpoints

Abstract

Conventional cameras with a limited field of view often lose sight of objects when their bearings change suddenly due to a significant turn of the observer (robot), the object, or both. Catadioptric omnidirectional sensors, consisting of a camera and a mirror, can track objects and estimate their distances more robustly.

The shapes of mirrors used by such sensors have differing merits. This paper discusses several advantages of the conical mirror over other shapes of mirrors in current use. A perspective projection unwarping of the conical mirror images is developed and demonstrated. This has hitherto been considered impossible for mirrors with multiple viewpoints.

An estimation of distance (range) over a large surrounding area is crucial in mobile robotics. A solution is proposed here in the form of an omnidirectional stereo apparatus with two catadioptric sensors in a vertical coaxial arrangement. The coaxial stereo requires very simple matching since the epipolar lines are the radial lines of identical orientations in both omnidirectional images. The radial matching is supported by a novel polar edge finder which uses discrete cosine transform and returns image gradients expressed in polar coordinates.

Introduction

This paper offers some reasoned suggestions for the best modality for autonomous mobile robots sensors. The introduction follows this sequence of arguments: amongst the senses, vision is usually the best choice; amongst vision sensors, omnidirectional sensors are the best for navigation and mapping; amongst omnidirectional vision sensors, catadioptric sensors are the best in dynamic environments; amongst catadioptric omnidirectional sensors, the conical mirror with a perspective camera offers the best image quality and resolution.

Passive visual sensors have known advantages in comparison with active sensors currently popular in much of robotics; specifically no mutual interference or detection and relatively accurate localisation of objects even at large distances. These are the overriding reasons why vision is the distal sense of choice for most biological organisms in air and clear water. If a robot is to emulate some aspects of (biological) mobility, then it needs, most of all, to emulate the (biological) visual abilities.

Some of the most important tasks in vision for robotics include autonomous navigation, site modelling (mapping) and surveillance. They all benefit from using panoramic $360°$ images produced by omnidirectional visual sensors.

Early attempts at using omnidirectional sensors included camera clusters [1] and various arrangements of mechanically rotating cameras and planar mirrors [2], [3], [4]. These mostly had problems with registration, motion, or both. Fisheye lens cameras have also been used to increase the field of view [5] but they proved difficult because of their irreversible distortion of nearby objects and the lack of a single viewpoint.

Single viewpoint projection geometry exists when the light rays arriving from all directions intersect at a single point known as the (single) effective viewpoint. For example, by placing the centre of the perspective camera lens at the outer focus of a hyperbolic mirror, the inner focus then becomes the single effective viewpoint.

Catadioptric sensors [6] consist of a fixed dioptric camera, usually mounted vertically, plus a fixed rotationally symmetrical mirror suspended above or below the camera. The advantages of catadioptric sensors derive from the fact that, unlike the rotating cameras, their ‘scanning’ of the surroundings is almost instantaneous, the camera exposure time being usually shorter than the full circle mechanical rotation time. Shorter exposure time means fewer image capture problems caused by motion and vibration of the camera, or by moving objects.

Suitability for use in dynamic environments is clearly an important consideration, especially as one of the chief benefits of omnidirectional vision in general is the ability to retain objects in view even when their bearings have changed suddenly and significantly. Catadioptric omnidirectional sensors are therefore ideally suited to visual navigation [7], visual guidance applications [8], using stereopsis, motion analysis [9], and site mapping [10].

The main practical problem with catadioptric sensors is that the details of the image can have relatively poor resolution, as the image depicts a large area. The resolution problem is unfortunately compounded by mirrors whose shapes have curved cross-sections. Such radially curved mirrors include the three popular quadric surface mirrors (elliptic, hyperbolic and parabolic) which are known to possess a single viewpoint at their focal points.

A single viewpoint is generally thought to be necessary for an accurate unwarping of images and for an accurate perspective projection which is relied on by most current computer vision methods [11]. The single viewpoint projection has been endorsed and recommended by [12], [13], [14], [15], [16], [17], [18] and others.

There have been few attempts at analysing multi-viewpoint sensors [19], [20], [21], although various people [22] used them previously without analysis.

An omnidirectional sensor’s resolution can be improved by using several planar mirrors with a separate camera for each one. The mirrors are placed in some spatial arrangement, for instance in a six sided pyramid [23]. The mirrors are carefully adjusted so that all the reflected camera positions coincide and thus form a single effective viewpoint. However, such arrangements are awkward, expensive, and sensitive to alignment errors. The hexagonal pyramid apparatus would require no fewer than 12 precisely placed cameras for stereopsis! Also, the coverage of the surrounding area is not isotropic.

This paper proposes a solution to the above problems which combines the benefits of the planar mirrors (no radial distortion, no radial loss of resolution) with the advantages of the rotationally symmetric catadioptric sensor (short exposure, isotropic imaging). The only shape of mirror that satisfies these requirements is the cone.

Section 2 summarises the projection and the unwarping transformation for a single conical mirror.

Section 3 describes an omnidirectional stereo system using two coaxial conical mirrors and two cameras.