Relative Pose and Inertia Determination of Unknown Satellite Using Monocular Vision

Ivanov, Danil; Ovchinnikov, Mikhail; Sakovich, Marianna

doi:https://doi.org/10.1155/2018/9731512

International Journal of Aerospace Engineering

On this page

Abstract Introduction Numerical Simulation Conclusion Data Availability Conflicts of Interest Acknowledgments References Copyright Related Articles

Research Article | Open Access

Volume 2018 | Article ID 9731512 | https://doi.org/10.1155/2018/9731512

Relative Pose and Inertia Determination of Unknown Satellite Using Monocular Vision

Danil Ivanov,¹Mikhail Ovchinnikov,¹and Marianna Sakovich¹

Academic Editor: Vaios Lappas

Received07 Mar 2018

Accepted05 Sept 2018

Published29 Oct 2018

Abstract

The paper proposes a two-stage algorithm for autonomous relative motion determination of noncooperative and unknown object flying in space. The algorithm is based on image processing and can be applied to motion determination of space debris with unknown geometry and dynamic characteristics. The first stage of the algorithm is aimed at forming a database of possible reference points of the object during continuous observation. Tensor of inertia, initial velocity, and angular velocity of the object are also estimated. Then, these parameters are used in the second stage of the algorithm to determine the relative motion in real time. The algorithm is studied numerically and tested using the video of the Chibis-M microsatellite separation.

1. Introduction

A large amount of space debris flying around the Earth prevents intensive exploration of near-Earth orbits. Space community aims to remove the most dangerous fragments and inactive satellites. Many missions were proposed in the last decade. One of the most perspective concepts is to launch the special “chaser” satellite that will collect debris and move them to the low Earth orbit, so they will be rapidly deorbited by atmospheric drag. The most crucial and complicated part of this algorithm is the debris collection [1], but for the task solving it is necessary to determine the relative motion of the debris of unknown shape and dynamic characteristics.

In recent years, relative position and attitude estimation of spacecraft have been extensively studied, and many methods have been proposed. Chaser satellite must be equipped with the onboard sensor for relative pose determination, i.e., relative attitude and relative center of mass position. The most reliable sensor type is an optical one that usually comprises a CCD camera. In the paper [2], an overview of the existing visual-based systems is presented, and in the paper [3], the state of the art of the cooperative and uncooperative satellite pose determination techniques are presented. A set of papers are devoted to the developing and flight testing of the algorithms for relative center of mass position determination of the unknown target using angles-only visual navigation [4–6]. The relative pose navigation system based on image processing was implemented in several projects; for example, PRISMA [7, 8], it is used during docking stage of the “Progress” to ISS [9] and also of the cargo spacecraft ATV [10–12]. In the paper [13], a set of the relative initial pose estimation techniques are compared. A number of papers are devoted to the developing of image-based navigation for satellites using laboratory facility [14–19]. In the papers mentioned above, the motion determination algorithms are usually based on recognition of specially designed satellites’ marks which positions are known in the target-related reference frame. Moreover, the dynamic characteristics such as tensor of inertia or position of the center of mass are used for target satellite motion prediction, i.e., the 3D model of the target is known. In the case of the unknown target before the pose determination, a prior monitoring stage should proceed and the model of the target is estimated. The well-known concept of the simultaneous localization and mapping (SLAM) can be adopted for these tasks [20]. A set of papers are devoted to the problem using images obtained by multiple cameras installed onboard of the chaser satellite [21–23]. For example, a SLAM approach based on the extended Kalman filter and inertia tensor determination using stereovision measurements is proposed in the paper [22]. A feature-based SLAM algorithm using the stereovision system is developed in [23]. There are a very few papers on the pose determination of unknown object using monocular vision measurements. In the paper [24], a purely monocular-based vision SLAM algorithm for pose tracking and shape reconstruction of an unknown target is presented; however, it is assumed that the target rotates with constant angular velocity and the target inertia tensor is not estimated. The fusion of the data obtained from the monocular camera and a range sensor is utilized in the algorithm for the pose, and shape reconstruction of a noncooperative spacecraft is considered in the paper [25]; the unscented Kalman filter is used for the estimation of variables, where is the number of the feature points. The current paper is quite similar to [25], but we divided the pose determination algorithm into two stages—the most computational power consumed determination of the reference point position and inertia tensor using least square method and the second, determination of the relative pose only based on the data obtained by the first stage. So, it is not necessary to estimate the reference points and the inertia tensor all the time since it is assumed that they are constant since the unknown satellite is considered as a rigid body.

In this paper, a noncooperative and unknown object relative motion determination algorithm based on the image processing of satellite illuminated by the Sun is considered. Initially, the target satellite has unknown geometry and mass distribution. It is assumed that the object has some features (surfaces, lines, or points) which can be recognized in the picture. The algorithm consists of two stages. First, the reference features’ positions in target satellite reference frame are estimated during continuous observation. Also, the tensor of inertia is determined using video information. During this stage, the database of observable reference features is formed. These features will be used later for relative motion estimation in real time. The pose vector of a relative state that consists of the center of mass position, linear velocity, attitude quaternion, and angular velocity vectors is estimated. The main idea of the first stage of the algorithm is to find such dynamical parameters and positions of reference points that minimize the difference between the points’ pixel coordinates on frames and the estimated by model ones.

The second stage of the algorithm is briefly described as follows. For a given initial relative state vector, the satellite reference point positions are projected to the video frame. Reference points’ projections are searched then in the neighborhood of these positions. Found reference points’ positions are inaccurate due to a noise and discrete sensing element. Least-square method (LSM) is used to minimize the sum of squared distances between the predicted and obtained reference points’ positions in the frame. In such a way, the relative position and attitude are roughly estimated. This estimation is used as the Kalman filter input to compute the state vector with higher precision. In the paper, the relative motion determination accuracy is investigated depending on the parameters of the objects. The algorithm is tested using the video of the Chibis-M microsatellite separation.

2. Equations of Motion

Consider two satellites, a chaser and a target, flying along the circular LEO. Let the relative translational motion be described by well-known Clohessy-Wiltshire equations [26]: where , , and are coordinates of the target satellite with respect to the orbital reference frame , origin is placed in the chaser satellite center of mass which moves along a circular orbit of radius and with orbital angular velocity , axis is directed along chaser satellite radius-vector from the center of the Earth, is a normal vector to the orbit plane, and is the vector product of and (see Figure 1).

Consider the attitude motion of both satellites. If there is no control torque, then after neglecting perturbations except gravitational torque, the attitude motion could be described by where is an absolute angular velocity of the target spacecraft, is its inertia tensor, is a direction cosine matrix that is used for the target spacecraft frame rotation relative to the orbital frame, is a unit vector directed from the Earth to the satellite center of mass written in the frame , and is a target satellite center of mass.

3. Measurement Model

Consider two satellites move so close to each other that details of the sunlit spacecraft structures can be recognized. The coordinate system is the chaser satellite body-fixed frame, and the system is the target body-fixed frame (Figure 1).

Let the CCD camera’s optical axis be parallel to the axis. If we shift the coordinate frame along the axis and take the center of the CCD matrix as the center of the coordinate frame, we get a coordinate frame of the camera (Figure 2). Suppose the distance is known. Consider a feature point of the target satellite which has a radius-vector in the frame . On the other hand, the point has coordinates in the frame . Figure 2 illustrates these two vector relations and equality .

This vector expression can be written in the frame as follows: where is a transformation matrix from the frame to the frame . Vector and matrix are relative position and relative attitude of the two satellites correspondingly.

Consider an image of the reference feature on the CCD matrix. According to projective geometry, the point has the coordinates in the picture defined by where is the camera focus distance with negative sign. The coordinates in image (in pixels) can be measured. The measurements of the reference features can be processed to estimate relative position and relative orientation.

As a reference, features of the target object can be considered special surfaces of unique shape and illumination (for example, solar panels), lines, or special points. Consider for concretization the tips of the antennas as reference features (Figure 3).

4. First Stage: Reference Features and Motion Parameter Determination Algorithm

Let the target satellite be uncooperative with unknown reference point positions ( in (3)). Assume that the chaser satellite observes the target during continuous time. In that time, the reference features or its candidates can be tracked on the CCD images. The new feature candidates can arise in the field of view or disappear in the shadow during the observation. As a result, a sequence of the measured coordinates of the features is available for processing. The problem is to estimate the positions of the reference features using these measurements. It can be done by LSM using the equations of motion if initial conditions and tensor of inertia of the target satellite are known. If the parameters are unknown, it is possible to estimate them jointly with the reference point positions.

First, consider how to track the reference feature position from image to image. When the first image of the target is obtained, it is reasonable to filter it for noise elimination. Then, the image can be converted to the binary one by using predetermined threshold. On the binary image, it is easy to find boundaries of the detectable reference feature positions. If the feature is a point, then the center of the mark can be calculated as a center of illumination or center of the boundary. Assume all positions of the reference points on the first image are measured. On the next image, we will search the corresponding feature points in the certain -neighborhood of the . If the feature point is found in -neighborhood of the previous point, then assume that it is the same features and collect it in special time column with coordinates of the th feature points on the sequence of frames. If there is no feature point in -neighborhood of the , assume that the point is shadowed or it is a point with not stable luminosity properties and it cannot be a reference point. Such an algorithm of feature point association fails if the target object moves or rotates too fast and it is applicable only in the close range relative motion when all the details of the target surface can be recognized in the image.

As a result of the feature point association algorithm, the database of the possible reference points is obtained. It is reasonable to process the feature points with the lengthiest records. It means that some of the point was observed for a rather long time and it is the points with stable luminosity.

Consider a vector of estimated parameters: where is a vector to the target satellite center of mass at the moment of the first image acquisition, is an initial vector of relative velocity, is a vector part of mutual attitude quaternion at the first moment, is an initial vector of angular velocity, is a tensor of inertia of the target object, and is the radius-vector of the th reference point in the body-fixed frame. If the vector is defined, the measured coordinates of the reference points can be calculated by integrating equations of motion (1) and (2) and using measurement model (3) and (4). So, the propagated measurement vectors of th reference point at time step are functions of the estimated vector .

Consider an error function: where are measured and collected in database reference points coordinates and are propagated values of corresponding measurements. The problem of minimization of the function can be solved only numerically.

When one solves the task of minimization of (6), the problem of the observability of the vector must be discussed. In the paper [27], the covariance matrix nonsingularity method was applied to the observability problem of pose estimation using line-of-sight measurements. The main idea of the method is to construct the Fisher information matrix and to define its rank. The Fisher matrix is defined as follows:

If the rank of the is equal to the length of vector , then is observable, i.e., it is possible to achieve vector from measurements. Numerically, it has been studied that if there is more than three observable reference points, it is possible to estimate vector .

The scheme of proposed algorithm is presented in Figure 4. A video with continuous target satellite observation is required for the algorithm. Then, the separate frames are extracted from the video. The frames are processed to have a binary contour image, where the centers of the feature points are calculated like described above. Its pixel coordinates are collected in the feature point database. When all the frames are processed, the feature point database is filtered to eliminate all the noise data like a shortly observed feature points. Finally, the resulting database is used for solving the minimization problem (6). So, the estimated feature point positions, tensor of inertia, and initial conditions are the result of the algorithm.

The proposed algorithm has a set of restrictions. The satellite on the image can be recognized only if the background is dark and direct sunlight does not saturate the image. The temporary occlusion of the reference points can occur during the motion. However, the first stage of the algorithm uses a continues sequence of the images for reconstructing the 3D model of the reference point position, so occlusion of the reference points might affect the provided accuracy but this will not lead to the confusion of the position of the object.

5. Numerical Simulation

Consider a numerical example of the algorithm. Let’s simulate the relative motion of the microsatellite Chibis-M during its separation from the cargo-vehicle Progress [28]. Let the tips of the antennas be the reference points.

As initial values of the estimated vector were chosen, next parameters are as follows:

The initial value of the reference point coordinates is random. The algorithm processed a sequence of the video frames. After applying the reference feature association algorithm, the database of the measurements is constructed. Table 1 demonstrates a part of the database, where coordinates (in pixels) are collected for each reference point. As one can see at frames 18 and 19, two points disappeared since it has been shadowed. The database measurements were used for estimation of the parameter vector . Figures 5–7 demonstrate the convergence process of the solution of the minimization problem. One can see that the errors of the components of the vector steadily decrease as the number of iterations increases.

(a)

(b)

(a)

(b)

6. The Second Stage: Relative Pose Determination Algorithm

When the reference point positions are determined, the problem of real time motion estimation can be solved [28]. Due to the estimation errors and noise in the CCD matrix, obtained reference point positions have error. So, a root-mean-square method should be applied. After determination, at least of three reference points on image, evaluate the simulated coordinate corrections using least squares procedure. Define an error functional as where and are the simulated coordinates of the reference marks and and are the centers of the objects in the picture in the -neighborhood of the reference marks. According to (3), point’s position in the frame depends on the position of the spacecraft center of mass. where are the coordinates of the spacecraft center of mass and are the vector part of the quaternion that describes its attitude. Let denote the vector of parameters. The problem of minimization of the function can be solved only numerically.

Because of the noise in the reference point measurements, the resulting vector of parameters should be filtered. At each step, one has propagated value obtained by integration of the relative motion equations. It is reasonable to use a constant gain Kalman filter to estimate the state vector . Constant gain Kalman filter does not require too much computational power and can be easily implemented on-board. So, the final estimated value can be calculated as where is a constant gain matrix.

Note that the developed approach of the relative position determination has some restrictions. For example, it is necessary to find, at least, four reference marks that are not uniplanar to determine spacecraft position explicitly. In addition, the algorithm cannot operate if the Sun lights the camera or the second spacecraft is in the eclipse. If the second spacecraft has a symmetric form, there is an ambiguity when solving (3). The shooting frequency should be higher than the rotational velocity of the spacecraft.

7. Algorithm Validation Using the “Chibis-M” Launch Video

The developed algorithm was applied for video processing of the “Chibis-M” separation from the cargo vehicle “Progress-13M”. Figure 8 illustrates the satellite “Chibis-M” in the docking hatch of cargo vehicle “Progress-13M” during undocking from ISS.

After undocking, the cargo vehicle was in a circular orbit at the altitude about 600 km, and the “Chibis-M” was separated on the 25th of January, 2012 at 2:18:00 UTC. Figure 9 shows the direction of the “Progress-13M” frame axes.

Figure 10 illustrates the frame from the video of the “Chibis-M” separation. The video was ruined by sunlight, but the long antennas are well visible. So, it is possible to recognize the frame, and Figure 11 shows a binary image of the frame.

Next, the binary image is used as input for the algorithm described above, and the current relative phase state is defined using the least squares procedure (Figure 12).

Unfortunately, due to the overexposure of the video, the reference point on the images could be recognized only during three seconds (it is about 100 frames). The obtained frames were processed by the first stage of the proposed algorithm for the reference point database creation, and for its position estimation, along with initial condition of the relative motion equations and inertia tensor determination. As a result, the reference point position is estimated with the 3 cm accuracy. An error in the estimated principal moments of the inertia tensor is about 0.3 kg·m². Figures 13 and 14 show the results of relative pose determination of the Chibis-M satellite at the second stage of the algorithm.

The root-mean-square error of determination is about 1 cm/s for the linear velocity and is about 0.5 deg/s for Y component of the angular velocity and 4 deg/s for its X component and Z component (not a high determination accuracy of the angular velocity components is specified by the noise on the reference mark images). The value of the angular velocity is equal to 4.24 deg/s. In 30 seconds after the separation, the navigation system was actuated; the magnitude of the angular velocity was measured by a rate-of-turn gyroscope and equaled 4.5 deg/s. Thus, the angular velocity evaluated by the algorithm proved to be close to the angular velocity measured by the rate-of-turn gyroscope. It indicates a satisfactory precision of the algorithm.

8. Conclusion

The proposed monocular vision-based two-stage relative pose determination algorithm is a convenient instrument for relative navigation in case of operations with noncooperative and unknown target satellite. It can be also useful for space removal missions when the space debris has an unknown shape. The algorithm has a number of limitation: it can be applied only for short relative distances, the target satellite must have some well-recognizing features, and the satellites’ relative motion could not be too fast. Nevertheless, the numerical simulation shows that the algorithm is stable, after the feature point position determination, it estimates the relative motion in real time.

Data Availability

The data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Acknowledgments

The work is supported by the Russian Science Foundation, grant no. 14-11-00621.

References

N. Van der Pas, J. Lousada, C. Terhes, M. Bernabeu, and W. Bauer, “Target selection and comparison of mission design for space debris removal by DLR’s advanced study group,” Acta Astronautica, vol. 102, pp. 241–248, 2014.
View at: Publisher Site | Google Scholar
P. Jasiobedzki, S. Se, T. Pan, M. Umasuthan, and M. Greenspan, “Autonomous satellite rendezvous and docking using lidar and model based vision,” in Proceedings Volume 5798, Spaceborne Sensors II, pp. 54–65, Orlando, FL, USA, 2005.
View at: Publisher Site | Google Scholar
R. Opromolla, G. Fasano, G. Rufino, and M. Grassi, “A review of cooperative and uncooperative spacecraft pose determination techniques for close-proximity operations,” Progress in Aerospace Science, vol. 93, pp. 53–72, 2017.
View at: Publisher Site | Google Scholar
G. Gaias, S. D’Amico, and J.-S. Ardaens, “Angles-only navigation to a noncooperative satellite using relative orbital elements,” Journal of Guidance, Control, and Dynamics, vol. 37, no. 2, pp. 439–451, 2014.
View at: Publisher Site | Google Scholar
J.-S. Ardaens and G. Gaias, “Flight demonstration of spaceborne real-time angles-only navigation to a noncooperative target in low earth orbit,” Acta Astronautica, 2018.
View at: Publisher Site | Google Scholar
S. D’Amico, J.-S. Ardaens, G. Gaias, H. Benninghoff, B. Schlepp, and J. L. Jørgensen, “Noncooperative rendezvous using angles-only optical navigation: system design and flight results,” Journal of Guidance, Control, and Dynamics, vol. 36, no. 6, pp. 1576–1595, 2013.
View at: Publisher Site | Google Scholar
R. Noteborn, P. Bodin, R. Larsson, and C. Chasset, “Flight results from the PRISMA optical line of sight based autonomous rendezvous experiment,” in Proceedings of the 4th International Conference on Spacecraft Formation Flying Missions & Technologies, pp. 18–20, Montreal, Canada, May 2011.
View at: Google Scholar
M. Delpech, J. C. Berges, S. Djalal, P. Y. Guidotti, and J. Christy, “Preliminary results of the vision based rendezvous and formation flying experiments performed during the PRISMA extended mission,” in 1st IAA Conference on Dynamics and Control of Space Systems, p. 20, Porto, Portugal, March 2012.
View at: Google Scholar
A. A. Boguslavsky, V. V. Sazonov, S. M. Sokolov, A. I. Smirnov, and K. S. Saigiraev, Automatic Vision-based Monitoring of a Spacecraft Docking to the Orbital Station, Prepr. KIAM RAS, 2004.
M. Ganet, I. Quinquis, J. Bourdon, and P. Delpy, “ATV GNC during rendezvous with ISS,” in International Conference on Dynamics Control Systems and Structures in Space, p. 10, Cambridge, UK, 2002.
View at: Google Scholar
L. Strippoli, P. Colmenarejo, and H. Strauch, “Mission analysis and guidance, navigation, and control design for rendezvous and docking phase of advanced reentry vehicle mission,” in Progress in Flight Dynamics, Guidance, Navigation, Control, Fault Detection, and Avionics, pp. 123–136, St. Petersburg, Russia, 2013.
View at: Publisher Site | Google Scholar
G. Casonato and G. B. Palmerini, “Visual techniques applied to the ATV/1SS rendezvous monitoring,” in 2004 IEEE Aerospace Conference Proceedings (IEEE Cat. No.04TH8720), p. 625, Big Sky, MT, USA, 2004.
View at: Publisher Site | Google Scholar
S. Sharma and S. D’Amico, “Comparative assessment of techniques for initial pose estimation using monocular vision,” Acta Astronautica, vol. 123, pp. 435–445, 2016.
View at: Publisher Site | Google Scholar
M. Sabatini, R. Monti, P. Gasbarri, and G. B. Palmerini, “Adaptive and robust algorithms and tests for visual-based navigation of a space robotic manipulator,” Acta Astronautica, vol. 83, pp. 65–84, 2013.
View at: Publisher Site | Google Scholar
M. Sabatini, G. B. Palmerini, R. Monti, and P. Gasbarri, “Image based control of the “PINOCCHIO” experimental free flying platform,” Acta Astronautica, vol. 94, no. 1, pp. 480–492, 2014.
View at: Publisher Site | Google Scholar
M. Sabatini, G. B. Palmerini, and P. Gasbarri, “A testbed for visual based navigation and control during space rendezvous operations,” Acta Astronautica, vol. 117, pp. 184–196, 2015.
View at: Publisher Site | Google Scholar
D. Ivanov, M. Koptev, Y. Mashtakov et al., “Determination of disturbances acting on small satellite mock-up on air bearing table,” Acta Astronautica, vol. 142, pp. 265–276, 2018.
View at: Publisher Site | Google Scholar
D. S. Ivanov, M. D. Koptev, Y. V. Mashtakov et al., “Laboratory facility for microsatellite mock-up motion simulation,” Journal of Computer and Systems Sciences International, vol. 57, no. 1, pp. 115–130, 2018.
View at: Publisher Site | Google Scholar
D. Ivanov, M. Koptev, M. Ovchinnikov, S. Tkachev, N. Proshunin, and M. Shachkov, “Flexible microsatellite mock-up docking with non-cooperative target on planar air bearing test bed,” Acta Astronautica, 2018.
View at: Publisher Site | Google Scholar
H. Durrant-Whyte and T. Bailey, “Simultaneous localization and mapping: part I,” IEEE Robotics & Automation Magazine, vol. 13, no. 2, pp. 99–110, 2006.
View at: Publisher Site | Google Scholar
V. Pesce, M. Lavagna, and R. Bevilacqua, “Stereovision-based pose and inertia estimation of unknown and uncooperative space objects,” Advances in Space Research, vol. 59, no. 1, pp. 236–251, 2017.
View at: Publisher Site | Google Scholar
S. Segal, A. Carmi, and P. Gurfil, “Stereovision-based estimation of relative dynamics between noncooperative satellites: theory and experiments,” IEEE Transactions on Control Systems Technology, vol. 22, no. 2, pp. 568–584, 2014.
View at: Publisher Site | Google Scholar
A. Sonnenburg, M. Tkocz, and K. Janschek, “EKF-SLAM based approach for spacecraft rendezvous navigation with unknown target spacecraft,” IFAC Proceedings Volumes, vol. 43, no. 15, pp. 339–344, 2010.
View at: Publisher Site | Google Scholar
S. Augenstein and S. M. Rock, “Improved frame-to-frame pose tracking during vision-only SLAM/SFM with a tumbling target,” in 2011 IEEE International Conference on Robotics and Automation, pp. 3131–3138, Shanghai, China, 2011.
View at: Publisher Site | Google Scholar
R. Volpe, M. Sabatini, and G. B. Palmerini, “Pose and shape reconstruction of a noncooperative spacecraft using camera and range measurements,” International Journal of Aerospace Engineering, vol. 2017, Article ID 4535316, 13 pages, 2017.
View at: Publisher Site | Google Scholar
W. H. Clohessy, “Terminal guidance system for satellite rendezvous,” Journal of the Astronautical Sciences, vol. 27, no. 9, pp. 653–658, 1960.
View at: Publisher Site | Google Scholar
D. Sun and J. L. Crassidis, “Observability analysis of six-degree-of-freedom configuration determination using vector observations,” Journal of Guidance, Control, and Dynamics, vol. 25, no. 6, pp. 1149–1157, 2002.
View at: Publisher Site | Google Scholar
D. Ivanov, S. Karpenko, M. Ovchinnikov, and M. Sakovich, “Satellite relative motion determination during separation using image processing,” International Journal of Space Science and Engineering, vol. 2, no. 4, p. 365, 2014.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2018 Danil Ivanov et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

1339

Downloads

1302

Citations