Robotic Control with Partial Visual Information

Abstract

We consider a robotic setting and a class of control tasks that rely on partial visual information. These tasks are difficult in the sense that at every given moment, the available information is insufficient for the control task. This implies that the image Jacobian, which relates the image space and the control space, is no longer of full rank. However, the amount of information collected throughout the control process is still large and thus seems sufficient for carrying out the task. Such situations commonly arise when the object is frequently occluded from one of the cameras in a stereo pair or when only one moving camera is available. We propose a generic control rule for such tasks and characterize the conditions required for the success of the task. The analysis is based on the observation that mathematically the behavior of such systems is related to a class of row-action optimization algorithms which are special cases of POCS (Projection On Convex Sets) algorithms. In the second part of the paper we focus on one particular task from this class: position and orientation control with a single rotating camera. We show that this task can be carried out, in principle, for any camera rotation and suggest efficient control and camera moving strategies. We substantiate our claims by simulations and experiments. Interestingly, it seems that the advisable control law is not consistent with simple intuition.