A new image-based visual servoing method with velocity direction control

Abstract

In image-based visual servoing (IBVS), for more satisfying trajectories, the target is always expected to move in straight lines to the desired positions. In this paper, a new IBVS controller for trajectory adjustment is obtained by optimizing a multi-objective function about adjusting a direction in the image plane. By applying the control law based on the optimized parameters, less redundant motion can be achieved in contrast to the traditional methods. Meanwhile, it will be beneficial for reducing the risk of the target leaving the field of view (FOV) and the robustness to the uncertainties of depth and camera internal parameters remains. The experimental results of the 6 degrees of freedom (DOF) robot with eye-in-hand configuration demonstrate the effectiveness and practicability of the proposed method.

Introduction

Nowadays, the vision of robot is involved in all walks of our life. Visual servoing (VS) is a motion control process that uses imaging technology to move marked feature to desired positions [1], and can be integrated with many disciplines, such as optimization methods [2], sliding mode [3], fuzzy control [4], robust control [5], switching control [6], adaptive control [7], [8], neural networks [9], [10], [11], reinforcement learning [12] and so on. Generally, VS is divided into position-based visual servoing (PBVS), image-based visual servoing (IBVS) and hybrid visual servo. Each method has advantages and disadvantages [13], [14].

PBVS needs to get geometric shape of the target, intrinsic parameters of the camera and observed image plane feature [15]. Then, the estimated pose of the target relative to the camera is asked. The main task of PBVS is to reduce the difference between the current pose and the desired one, which can effectively execute the linear camera motion in the cartesian coordinate frame. Therefore, the calibration accuracy of geometric and camera model of the target seriously affects and restricts the performance of PBVS [16]. Moreover, PBVS is controlled in three-dimensional space and needs complex calculation, so it is difficult to obtain satisfactory image plane trajectories, which may cause the image feature to leave the FOV.

IBVS is fundamentally different from PBVS in that it does not estimate the relative pose of the target, which is implied in image feature. So IBVS is to control feature in the image plane [17]. It has been widely used in industrial fields because of its robustness, such as unmanned systems [18], aircrafts [19], fault diagnosis [20] and so on. However, IBVS needs to worry about feature leaving the FOV, and it can produce trajectories with redundant motion that exceed the robot’s motion limitation, especially if it requires a large angle of rotation around the camera’s optical axis. Due to this characteristic, in [21], a trajectory planning method is developed to eliminate the limitation caused by robot’s FOV. Another low complexity IBVS control scheme can finish tasks respecting the FOV constrains by rigorous derivations in [22]. IBVS needs only one 3D information, that is, the depth information of feature points, but it is robust to the estimation bias of depth and the calibration error of model parameters. Therefore, in recent decades, there have been many different control methods combined with IBVS. The proportional controller is the most basic method to decrease the states exponentially, and there are numerical elegant approaches, whose applications have been developed during the past decades. A partially decoupled control law is respectively designed for rotation and translation can be found in [23]. In [24], [25], an online IBVS controller based on the robust model predictive control (RMPC) is proposed, and there is another method using acceleration as command named augmented IBVS (AIBVS) in [26]. In general, the improvement of monocular VS control is often based on the choice between exploring new visual features and designing new controllers. Except for novel control law, many researchers find other proper image features for enhancing the tracking performance, including the luminance of pixel [27], points in a new virtual visual space [28] as well as image moments [29]. Nonetheless, a more flexible controller is still to be expected.

Hybrid visual servoing, called 2-1/2-D visual servoing, is the combination of PBVS and IBVS, in order to avoid the shortcomings of them. By decomposing the homography matrix, the translation and rotation motion of robot’s end-effector can be controlled respectively in [30], so as to simultaneously optimize the motion trajectories in the image plane and cartesian space. Another scheme in [31] is a modified controller based on optimization, which includes the constrains of FOV and actuator. However, it has poor robustness and is especially sensitive to image noise. Moreover, it needs to calculate and decompose the homography matrix at all times in the control process, which leads to calculation pressure and reduces real-time performance of the system.

In this paper, a new IBVS control method with velocity direction control is defined. A multi-objective optimization framework different from [1] is designed to solve the optimal parameters, in order to minimize the gaps between the velocity direction and its boundaries. Compared with the classical control methods, it can achieve the same excellent performance for the long-distance translational motion. For the rotation task with large angles, it can effectively avoid redundant motion and complete the task with trajectories closer to straight lines. The stability proof of the controller will be given in the paper. Finally, the results of experiments will demonstrate the effectiveness of the proposed method.

The layout of the paper is organized as follows. In Section II, the details of the IBVS and its system descriptions are shown. The core part is in Section III, in which the novel control law based on the velocity direction control for IBVS system is derived. In Section IV, several experimental results are presented to verify the effectiveness of the proposed control strategy. Finally, the conclusions are given in the last section.