A new approach for the estimation of non-cooperative satellites based on circular feature extraction

Highlights

• A pose estimation method is proposed based on circular feature extraction.

• Neither additional sensors nor knowledge information is required in our method.

• A closed-form solution is provided in quadratic form.

• The orientation-duality problem has been solved.

• The visual feedback may enhance the safety and stability of OOS operations.

Abstract

Pose estimation of non-cooperative satellites has been a hot topic in the study of astronautics as the visual feedback will highly enhance the safety of on-orbit services. A stereo vision system is proposed in this paper. It works as an eye-to-hand vision camera in the final approach phase Based on circular feature extraction, a closed-form solution is presented. The position and orientation of the adapter ring can be figured out in real-time as well as the unknown radius. Neither additional sensors nor prior knowledge is required, and the orientation-duality problem has been solved. It works well on the partial ellipses and is robust to outliers, noise and occlusions. Experimental results on both synthetic and real images have demonstrated the effectiveness and efficiency of the proposed method.

Introduction

The geostationary orbit (GEO), a unique satellite orbit of the human beings, is a very precious but limited orbit resource [1]. Satellites in such an orbit are at rest relative to the earth, monitoring weather, communications and surveillance [2], [3]. Nowadays, GEO has become the most crowded orbit around the earth as satellites differ in location by longitude only.

It is reported by UCS (Union of Concerned Scientists) that, at the end of 2016, 493 out of 1381 satellites executed their mission in GEO [4]. Frequent failures of GEO satellites result in high economic cost and great risk for development and operation. Once the satellites failed, they may be boosted into a disposal orbit, even though the communications relay payloads are still functional [5]. Therefore, on-orbit services (OOS) technologies have attracted much attention in the study of spacecraft life-extension and on-orbit debris removal [6], [7]. Such technologies take lower costs and prolong legacy satellites flying for up to fifteen years.

In early tasks, astronauts regularly travelled on-board of the shuttle to perform the OOS operations, such as the repairing of the Hubble Space Telescope (HST) and the assembling of the International Space Station (ISS). However, the space shuttle or other manned spacecraft cannot reach the geostationary orbit. Thus, the servicing satellite should have a high degree of autonomy to dock the unmanned targets.

In traditional rendezvous and docking missions, transponders or reflectors may be used as the navigation sensors [8]. They facilitate the pose estimation by indicating the position of fixed points on the target spacecraft. However, most of the existing satellites of GEO were not designed to be serviced. These satellites are non-cooperative targets, i.e., neither artificial features for pose estimation nor specially designed mechanisms for docking are mounted on these satellites.

Cameras are commonly used for relative pose estimation of non-cooperative satellites as they call for lower mass and energy. The visual feedback can enhance the safety and stability of OOS operations [9]. As the given target cannot provide effective cooperative information for pose estimation, we have to rely on the natural features.

Some researchers have theoretically investigated on the non-cooperative pose estimation methods based on geometric features such as solar panels, motor nozzle, and communication antennas. However, these components are not strong enough to be grasped [10]. In contrast, the adapter ring is a good candidate to be captured by the space robot manipulator. It is a high-strength torus with a radius of either 1194 mm or 1666 mm [11], [12]. Besides, the circular projection on the image is an ideal feature for the following reasons:

• Due to the symmetry, a circle can be defined with only three parameters.

• Its perspective projection is always an ellipse.

• It can be used for pose estimation without solving the disparity map.

• Described in quadratic form, a closed-form solution can be figured out.

Some organizations and researchers have carried out studies on the pose estimation of non-cooperative satellites. Cropp et al. [13] proposed a relative pose estimation method for the known target. But the consistency of the straight line cost much time. Similarly, Lichter and Dubowsky [14] presented a new estimation method for the dynamic state, geometric shape and model parameters of the on-orbit targets. However, it required a number of 3D sensors, leading to the high cost for OSS tasks. Inaba proposed et al. [15] a method for on-orbit identification, but the shape, size and mass of the target should be given first. Based on the 3D model matching, an iterative closest point algorithm was proposed for relative attitude and position estimation [16]. However, the image processing time was about five seconds and the attitude estimation step took a further eleven seconds at most. Stepanov et al. [17] realized position estimation method based on the 3D model of the International Space Station. Thus, the 3D shape should be given at first. Similarly, Opromolla et al. [18] made use of the template matching technique for pose determination. Palmerini et al. [19] proposed an advanced vision-based method for the dual task of identifying a target and evaluating the relative position. However, it could not get the depth information.

Instead of relying on the given model of the targets, some researchers investigated into the pose estimation methods based on the edge features such as circular adapter ring, triangular bracket or rectangular solar panel. Du et al. [20] made use of the vanishing points to solve the orientation-duality problem. Nevertheless, the result was related to a known diameter as prior information. Miao and Zhu [21] adopted the Euclidean distance invariance to deal with the pose ambiguity. However, it was hard to find a fixed point, especially in proximity operations. Li et al. [22] employed the line structured light to eliminate the ambiguity. The additional line laser projector increased the energy requirement at the same time. Xu et al. [23] proposed a estimation method based on a binocular stereo vision system. The radius of the ring could be estimated as well. Peng et al. [24] improved the calculation speed by pre-processing on the FPGAs. Meng et al. [25] combined the linear features with circular edges to eliminate the ambiguity of monocular estimation. Unfortunately, it was hard to find a fixed line, especially during proximity operations.

Apart from these traditional methods, some investigators have sought to determine the pose information using machine learning techniques [26], [27]. Pesce [28] proposed a novel pose estimation scheme that combined the Principal Component Analysis (PCA) and the RANSAC algorithms together. This scheme proved to be robust with respect to spacecraft symmetry. Sharma [29] introduced a novel method to threshold the weak gradient intensities from the background. This method could extract the main body features as well as particular structures with two parallel processing flows. Capuano et al. [30] realized two vision-based approaches for the pose determination of known and unknown satellites, respectively. Later, they [31] modified the filter-based SLAM architectures [32] to perform robust Simultaneous Estimation of Pose and Shape (SEPS) for targets completely unknown. Sharma et al. [33] discussed the position estimation problem based on the bounding box detection. Besides, soft classification [34] was then introduced to estimate the orientation of the target. Proença et al. [35] first investigated into photorealistic rendering for monocular pose estimation. Then, a deep learning framework was introduced that modelled the orientation ambiguity as a mixture of Gaussians.

In this paper, a stereo vision system is proposed to estimate the position and orientation of the non-cooperative satellites. The adapter ring, a commonly used component, is picked up as the typical feature. A closed-form solution is then provided in quadratic form. It is robust to outliers and occlusions and can be used as a visual feedback for the OSS operations. This vision system mainly works as an eye-to-hand camera during the final approach phase (20 m–1 m), where the adapter ring is generally visible. For rotating targets with unknown inertia, active detumbling technologies are suggested to slow down the tumbling rate first.

The rest of the paper is organized as follows: In Section 2, a length-based line segment detector is introduced that extracts piecewise-linear segments from the source image. Each segment is a collection of primary corner points which keep the basic geometric and photometric boundaries. Based on the edge connectivity, a robust ellipse detection method is presented in Section 3. It can deal with the partial ellipse by searching the edge fragments in the missing area. Section 4 gives a closed-form solution for pose estimation. The position of the target satellite is represented by the centre of the adapter ring. The orientation is expressed by the pitch and yaw angles. Section 5 comments on the experimental results and Section 6 concludes the paper.