Multiple homographies with omnidirectional vision for robot homing

https://doi.org/10.1016/j.robot.2010.02.009Get rights and content

Abstract

Two relevant issues in vision-based navigation are the field-of-view constraints of conventional cameras and the model and structure dependency of standard approaches. A good solution of these problems is the use of the homography model with omnidirectional vision. However, a plane of the scene will cover only a small part of the omnidirectional image, missing relevant information across the wide range field of view, which is the main advantage of omnidirectional sensors. The interest of this paper is in a new approach for computing multiple homographies from virtual planes using omnidirectional images and its application in an omnidirectional vision-based homing control scheme. The multiple homographies are robustly computed, from a set of point matches across two omnidirectional views, using a method that relies on virtual planes independently of the structure of the scene. The method takes advantage of the planar motion constraint of the platform and computes virtual vertical planes from the scene. The family of homographies is also constrained to be embedded in a three-dimensional linear subspace to improve numerical consistency. Simulations and real experiments are provided to evaluate our approach.

Introduction

In order to perform navigation, mobile robots need to interact with the environment, and for this purpose many different types of sensor are available. From all of them, vision systems stand out because they provide very rich information and because of their versatility and low cost. In recent years the use of omnidirectional cameras in robotics is also growing because of their effectiveness due to the panoramic view from a single image. Camera motion between two views can be obtained from rigidity constraints across the views. In general, problems like guided matching, motion computation, or visual control are directly tackled from multi-view tensors like the fundamental matrix, which assume a general model with respect to the unknown scene structure. The problem of robust estimation of the epipolar geometry and autocalibration from point correspondences using omnidirectional vision has been studied in [1], [2], [3], [4]. However, there are common situations in which the fundamental matrix fails, reducing the applicability of the model [5], [6], [7], [8]. In particular, the fundamental matrix is ill conditioned with short baseline and undefined with pure rotations. Moreover, its estimation with general algorithms degenerates when all the points lie on a plane. However, this degeneracy with coplanar scene points can be overcome if the camera is calibrated by using the five-point algorithm [9]. Different approaches for vision-based robot navigation are given in [10], [11], [12]. The Fourier components of the omnidirectional images stored in a visual memory are used in [10]; a trifocal tensor-based approach is presented in [11]; and the homing task is solved in [12] using angular information of a minimal set of three landmark correspondences with isotropic distribution.

A good solution with planar scenes is the homography model. The homography has been used with omnidirectional vision in recent works for visual control [13], visual odometry [14], or navigation using a visual memory of the environment following a sequence of subgoals [15], [16]. However, if no plane is detected, a homography-based approach would fail. This problem can be solved using a virtual plane [17], but, in general, estimation based on a virtual plane with wide baseline is not robust to mismatches, noise or occlusions, and a dominant plane is preferred to get better accuracy. Additionally, a plane of the scene will cover only a small part of the omnidirectional image, missing relevant information across the wide range field of view, which is the main advantage of using omnidirectional sensors. A proposal for using several homographies defined by real planes is presented in [18] to improve reliability in the visual control task. Another issue is the precision with almost planar scene structures: a precise homography would reject good point matches close to the plane and, on the other hand, joining closer points to support the homography will reduce its precision.

In this paper, we present a new approach for computing multiple homographies from virtual planes and its application in an omnidirectional image-based control scheme for mobile robot homing. The multiple homographies are robustly computed from a set of matches using a method that relies on virtual planes independently of the scene structure. Thus, the approach is not compromised by the non-planarity of the scene. Additionally, the use of a set of virtual planes can be supported by any inlier across the image, taking advantage of the wide field of view of omnidirectional images. The method considers the planar motion constraint of the platform and, additionally, the virtual planes are constrained to be vertical with respect to the motion plane of the robot. These constraints allow us to parameterize the homographies to be computed from only two point correspondences. This minimal sample size improves the performance of the robust estimation with respect to outliers. An additional constraint used for improving the numerical consistency of the family of homographies obtained is that they are embedded in a three-dimensional linear subspace. The performance of the method is tested and compared with classical approaches [6] in terms of percentage of spurious matches, planarity of the scene and image noise showing the advantages of our proposal.

We also propose a new visual control approach taking advantage of the multiple homography estimation method. The system consists of a calibrated omnidirectional camera mounted on top of a differential drive robot. The visual control task uses the classical idea of homing, where the target location is defined by an image previously taken at that location. The current and target images are related by the set of homographies, which are used to design our controller. This visual homing takes advantage of the wide field of view of the omnidirectional camera being independent of the scene structure.

The paper is organized as follows. Section 2 describes the camera model and the homography matrix within our framework. Section 3 presents the computation of a homography considering motion and plane constraints, and the multiple homography estimation method is proposed in Section 4. Section 5 presents the control scheme for omnidirectional vision-based homing. Simulations and real experiments given in Section 6 show the performance of the proposed scheme.

Section snippets

Imaging geometry

In this section we define the geometry of the imaging system and describe the omnidirectional camera model using the unified sphere model [19], [20]. The homography matrix is also introduced within our framework.

Estimation of a vertical homography from two points

In this section we present the estimation method of a homography considering not only the motion constraints of the platform but also an additional constraint on the plane that defines the homography. Let us consider a set of coplanar points in the world belonging to a vertical plane, and then n=(nx,ny,0)T. Therefore, the plane is aligned with the z-axis of the omnidirectional camera. Thus, the homography matrix corresponding to a planar motion scheme and considering a vertical plane is H=[h11h

Estimation of multiple homographies

In this section we present the algorithm to compute multiple homographies from a set of point correspondences obtained from the omnidirectional images. A constraint on the rank is also used to enforce numerical consistency across the family of homographies.

In the framework considered here, the minimal set for vertical homography estimation is two point correspondences. Assuming the presence of mismatches, an estimation method robust to outliers is required. We present Algorithm 1, which is

Visual control scheme

In this section we show how the multiple homography estimation method previously described is used in a new visual control scheme. The goal is to design a control law for autonomous robot navigation based on omnidirectional visual information using the classical idea of homing, where the desired location is defined by an image taken previously at that location. Then, a current image together with a reference image is used to compute the set of multiple homographies. The family of homographies

Experiments

In this section we present experimental validation of the approach presented. First, we test the multiple homography estimation method with simulated and real data. And second we evaluate the omnidirectional visual control scheme with simulations and real experiments.

Conclusion

We have presented a new approach for estimation of multiple homographies defined from virtual vertical planes with omnidirectional image pairs. The minimal set of point correspondences required for homography estimation is reduced to two, considering constraints on the camera motion and on the virtual planes. As output, the method gives a family of homographies of rank 3 and the relative orientation of the cameras. The classical methods based on epipolar geometry fail with planar scenes, but

Acknowledgements

The authors would like to thank Jesús Bermúdez for his contribution in the experiments. This work was supported by projects DPI2006-07928, DPI2009-08126 and DPI2009-14664-C02-01.

Gonzalo López Nicolás received his M.Sc. degree in Industrial Engineering and his Ph.D. in Systems Engineering and Computer Science from the University of Zaragoza (Spain) in 2003 and 2008, respectively. He is a member of the Robotics, Perception and Real-Time Group. His current research interests are focused on visual control, autonomous robot navigation and application of computer vision to robotics.

References (32)

  • G. López-Nicolás et al.

    Switching visual control based on epipoles for mobile robots

    Robotics and Autonomous Systems

    (2008)
  • E. Menegatti et al.

    Image-based memory for robot navigation using properties of omnidirectional images

    Robotics and Autonomous Systems

    (2004)
  • G. López-Nicolás et al.

    Visual control through the trifocal tensor for nonholonomic robots

    Robotics and Autonomous Systems

    (2010)
  • Z. Zivkovic et al.

    From images to rooms

    Robotics and Autonomous Systems

    (2007)
  • T. Svoboda et al.

    Epipolar geometry for central catadioptric cameras

    International Journal on Computer Vision

    (2002)
  • B. Micusik, T. Pajdla, Estimation of omnidirectional camera model from epipolar geometry, in: IEEE Computer Society...
  • C. Geyer, K. Daniilidis, Mirrors in motion: epipolar geometry and motion estimation, in: IEEE International Conference...
  • B. Micusik et al.

    Structure from motion with wide circular field of view cameras

    IEEE Transactions on Pattern Analysis and Machine Intelligence

    (2006)
  • P.H.S. Torr et al.

    The problem of degeneracy in structure and motion recovery from uncalibrated image sequences

    International Journal of Computer Vision

    (1999)
  • R.I. Hartley et al.

    Multiple View Geometry in Computer Vision

    (2004)
  • G. López-Nicolás et al.

    Automatic matching and motion estimation from two views of a multiplane scene

  • D. Nister

    An efficient solution to the five-point relative pose problem

  • A.A. Argyros et al.

    Robot homing by exploiting panoramic vision

    Autonomous Robots

    (2005)
  • S. Benhimane, E. Malis, A new approach to vision-based robot control with omni-directional cameras, in: IEEE...
  • D. Scaramuzza et al.

    Appearance-guided monocular omnidirectional visual odometry for outdoor ground vehicles

    IEEE Transactions on Robotics

    (2008)
  • H. Hadj-Abdelkader, Y. Mezouar, P. Martinet, Path planning for image based control with omnidirectional cameras, in:...
  • Cited by (20)

    View all citing articles on Scopus

    Gonzalo López Nicolás received his M.Sc. degree in Industrial Engineering and his Ph.D. in Systems Engineering and Computer Science from the University of Zaragoza (Spain) in 2003 and 2008, respectively. He is a member of the Robotics, Perception and Real-Time Group. His current research interests are focused on visual control, autonomous robot navigation and application of computer vision to robotics.

    Josechu Guerrero graduated in Electrical Engineering from the University of Zaragoza in 1989. He obtained his Ph.D. in 1996 from the same institution. Currently he is Associate Professor and Deputy Director of the Department of Computer Science and Systems Engineering. His research interests are in the area of computer vision, particularly in 3D visual perception, photogrammetry, visual control, omnidirectional vision, robotics and vision-based navigation.

    Carlos Sagüés received his M.Sc. and Ph.D. degrees from the University of Zaragoza, Spain. During the course of his Ph.D. he worked on force and infrared sensors for robots. Since 1994 he has been Associate Professor and, since 2009, Full Professor with the Department of Computer Science and Systems Engineering, being also Head Teacher in this Department. His current research interests include control systems, computer vision, visual robot navigation and multi-vehicle cooperative control.

    View full text