Multiple homographies with omnidirectional vision for robot homing
Introduction
In order to perform navigation, mobile robots need to interact with the environment, and for this purpose many different types of sensor are available. From all of them, vision systems stand out because they provide very rich information and because of their versatility and low cost. In recent years the use of omnidirectional cameras in robotics is also growing because of their effectiveness due to the panoramic view from a single image. Camera motion between two views can be obtained from rigidity constraints across the views. In general, problems like guided matching, motion computation, or visual control are directly tackled from multi-view tensors like the fundamental matrix, which assume a general model with respect to the unknown scene structure. The problem of robust estimation of the epipolar geometry and autocalibration from point correspondences using omnidirectional vision has been studied in [1], [2], [3], [4]. However, there are common situations in which the fundamental matrix fails, reducing the applicability of the model [5], [6], [7], [8]. In particular, the fundamental matrix is ill conditioned with short baseline and undefined with pure rotations. Moreover, its estimation with general algorithms degenerates when all the points lie on a plane. However, this degeneracy with coplanar scene points can be overcome if the camera is calibrated by using the five-point algorithm [9]. Different approaches for vision-based robot navigation are given in [10], [11], [12]. The Fourier components of the omnidirectional images stored in a visual memory are used in [10]; a trifocal tensor-based approach is presented in [11]; and the homing task is solved in [12] using angular information of a minimal set of three landmark correspondences with isotropic distribution.
A good solution with planar scenes is the homography model. The homography has been used with omnidirectional vision in recent works for visual control [13], visual odometry [14], or navigation using a visual memory of the environment following a sequence of subgoals [15], [16]. However, if no plane is detected, a homography-based approach would fail. This problem can be solved using a virtual plane [17], but, in general, estimation based on a virtual plane with wide baseline is not robust to mismatches, noise or occlusions, and a dominant plane is preferred to get better accuracy. Additionally, a plane of the scene will cover only a small part of the omnidirectional image, missing relevant information across the wide range field of view, which is the main advantage of using omnidirectional sensors. A proposal for using several homographies defined by real planes is presented in [18] to improve reliability in the visual control task. Another issue is the precision with almost planar scene structures: a precise homography would reject good point matches close to the plane and, on the other hand, joining closer points to support the homography will reduce its precision.
In this paper, we present a new approach for computing multiple homographies from virtual planes and its application in an omnidirectional image-based control scheme for mobile robot homing. The multiple homographies are robustly computed from a set of matches using a method that relies on virtual planes independently of the scene structure. Thus, the approach is not compromised by the non-planarity of the scene. Additionally, the use of a set of virtual planes can be supported by any inlier across the image, taking advantage of the wide field of view of omnidirectional images. The method considers the planar motion constraint of the platform and, additionally, the virtual planes are constrained to be vertical with respect to the motion plane of the robot. These constraints allow us to parameterize the homographies to be computed from only two point correspondences. This minimal sample size improves the performance of the robust estimation with respect to outliers. An additional constraint used for improving the numerical consistency of the family of homographies obtained is that they are embedded in a three-dimensional linear subspace. The performance of the method is tested and compared with classical approaches [6] in terms of percentage of spurious matches, planarity of the scene and image noise showing the advantages of our proposal.
We also propose a new visual control approach taking advantage of the multiple homography estimation method. The system consists of a calibrated omnidirectional camera mounted on top of a differential drive robot. The visual control task uses the classical idea of homing, where the target location is defined by an image previously taken at that location. The current and target images are related by the set of homographies, which are used to design our controller. This visual homing takes advantage of the wide field of view of the omnidirectional camera being independent of the scene structure.
The paper is organized as follows. Section 2 describes the camera model and the homography matrix within our framework. Section 3 presents the computation of a homography considering motion and plane constraints, and the multiple homography estimation method is proposed in Section 4. Section 5 presents the control scheme for omnidirectional vision-based homing. Simulations and real experiments given in Section 6 show the performance of the proposed scheme.
Section snippets
Imaging geometry
In this section we define the geometry of the imaging system and describe the omnidirectional camera model using the unified sphere model [19], [20]. The homography matrix is also introduced within our framework.
Estimation of a vertical homography from two points
In this section we present the estimation method of a homography considering not only the motion constraints of the platform but also an additional constraint on the plane that defines the homography. Let us consider a set of coplanar points in the world belonging to a vertical plane, and then . Therefore, the plane is aligned with the -axis of the omnidirectional camera. Thus, the homography matrix corresponding to a planar motion scheme and considering a vertical plane is
Estimation of multiple homographies
In this section we present the algorithm to compute multiple homographies from a set of point correspondences obtained from the omnidirectional images. A constraint on the rank is also used to enforce numerical consistency across the family of homographies.
In the framework considered here, the minimal set for vertical homography estimation is two point correspondences. Assuming the presence of mismatches, an estimation method robust to outliers is required. We present Algorithm 1, which is
Visual control scheme
In this section we show how the multiple homography estimation method previously described is used in a new visual control scheme. The goal is to design a control law for autonomous robot navigation based on omnidirectional visual information using the classical idea of homing, where the desired location is defined by an image taken previously at that location. Then, a current image together with a reference image is used to compute the set of multiple homographies. The family of homographies
Experiments
In this section we present experimental validation of the approach presented. First, we test the multiple homography estimation method with simulated and real data. And second we evaluate the omnidirectional visual control scheme with simulations and real experiments.
Conclusion
We have presented a new approach for estimation of multiple homographies defined from virtual vertical planes with omnidirectional image pairs. The minimal set of point correspondences required for homography estimation is reduced to two, considering constraints on the camera motion and on the virtual planes. As output, the method gives a family of homographies of rank 3 and the relative orientation of the cameras. The classical methods based on epipolar geometry fail with planar scenes, but
Acknowledgements
The authors would like to thank Jesús Bermúdez for his contribution in the experiments. This work was supported by projects DPI2006-07928, DPI2009-08126 and DPI2009-14664-C02-01.
Gonzalo López Nicolás received his M.Sc. degree in Industrial Engineering and his Ph.D. in Systems Engineering and Computer Science from the University of Zaragoza (Spain) in 2003 and 2008, respectively. He is a member of the Robotics, Perception and Real-Time Group. His current research interests are focused on visual control, autonomous robot navigation and application of computer vision to robotics.
References (32)
- et al.
Switching visual control based on epipoles for mobile robots
Robotics and Autonomous Systems
(2008) - et al.
Image-based memory for robot navigation using properties of omnidirectional images
Robotics and Autonomous Systems
(2004) - et al.
Visual control through the trifocal tensor for nonholonomic robots
Robotics and Autonomous Systems
(2010) - et al.
From images to rooms
Robotics and Autonomous Systems
(2007) - et al.
Epipolar geometry for central catadioptric cameras
International Journal on Computer Vision
(2002) - B. Micusik, T. Pajdla, Estimation of omnidirectional camera model from epipolar geometry, in: IEEE Computer Society...
- C. Geyer, K. Daniilidis, Mirrors in motion: epipolar geometry and motion estimation, in: IEEE International Conference...
- et al.
Structure from motion with wide circular field of view cameras
IEEE Transactions on Pattern Analysis and Machine Intelligence
(2006) - et al.
The problem of degeneracy in structure and motion recovery from uncalibrated image sequences
International Journal of Computer Vision
(1999) - et al.
Multiple View Geometry in Computer Vision
(2004)
Automatic matching and motion estimation from two views of a multiplane scene
An efficient solution to the five-point relative pose problem
Robot homing by exploiting panoramic vision
Autonomous Robots
Appearance-guided monocular omnidirectional visual odometry for outdoor ground vehicles
IEEE Transactions on Robotics
Cited by (20)
Target tracking for mobile robot platforms via object matching and background anti-matching
2010, Robotics and Autonomous SystemsCitation Excerpt :Vision systems are the most important component for a mobile robot which can provide very rich information of the real-world [2,3]. In a monocular vision system, one camera is used to capture the visual information from the real-world which makes the vision system very low cost [4]. It is noted that object tracking is a key technique in various robot vision applications such as surveillance [5,6], video compression [7], autonomous vehicle navigation [8], and perceptual user interfaces [9], etc.
DeepFuseNet of Omnidirectional Far-Infrared and Visual Stream for Vegetation Detection
2021, IEEE Transactions on Geoscience and Remote SensingAn Image Depth Information Extraction Based on Parallel Random Evolutionary Algorithm
2021, Proceedings of the 5th International Conference on Electronics, Communication and Aerospace Technology, ICECA 2021Vegetation Segmentation for Sensor Fusion of Omnidirectional Far-Infrared and Visual Stream
2019, IEEE Journal of Selected Topics in Applied Earth Observations and Remote SensingReduction of the uncertainty in feature tracking
2018, Applied Intelligence
Gonzalo López Nicolás received his M.Sc. degree in Industrial Engineering and his Ph.D. in Systems Engineering and Computer Science from the University of Zaragoza (Spain) in 2003 and 2008, respectively. He is a member of the Robotics, Perception and Real-Time Group. His current research interests are focused on visual control, autonomous robot navigation and application of computer vision to robotics.
Josechu Guerrero graduated in Electrical Engineering from the University of Zaragoza in 1989. He obtained his Ph.D. in 1996 from the same institution. Currently he is Associate Professor and Deputy Director of the Department of Computer Science and Systems Engineering. His research interests are in the area of computer vision, particularly in 3D visual perception, photogrammetry, visual control, omnidirectional vision, robotics and vision-based navigation.
Carlos Sagüés received his M.Sc. and Ph.D. degrees from the University of Zaragoza, Spain. During the course of his Ph.D. he worked on force and infrared sensors for robots. Since 1994 he has been Associate Professor and, since 2009, Full Professor with the Department of Computer Science and Systems Engineering, being also Head Teacher in this Department. His current research interests include control systems, computer vision, visual robot navigation and multi-vehicle cooperative control.