Abstract
This paper presents an efficient online technique used for finite-horizon, nonlinear, stochastic, regulator, and tracking problems. This can be accomplished by the integration of the differential SDRE filter algorithm and the finite-horizon state dependent Riccati equation (SDRE) technique. Unlike the previous methods which deal with the linearized system, this technique provides finite-horizon estimation and control of the nonlinear stochastic systems. Further, the proposed technique is effective for a wide range of operating points. Simulation results of a missile guidance system are presented to illustrate the effectiveness of the proposed technique.
1. Introduction
Strategic military dependence on missile technology has been growing rapidly since its first use at the beginning of the twentieth century. The critical requirement regarding missile usage is to lead the missile robustly and accurately from its launch point to its designated end point or target. The missile target could be a certain point on its required orbit in space or a moving hostile object either flying or rolling on terrain. To achieve this requirement, three operations have to be completed and they are described in the literature as the guidance, navigation, and control (GNC) process [1].
Guidance systems can be categorized into four main categories. These categories are command, beam rider, autonomous, and homing guidance [2]. A fifth possible category is a combination of two or more of the previous guidance methods such as some types of cruise missiles that switch from autonomous guidance to homing guidance in the terminal phase for more accurate hits. The difference between command guidance and homing guidance is the location of the guidance computer, which can be on the launching station or on missile board, respectively.
Both command and homing guidance methods can use radio frequency (RF), infrared (IR), or optical sensors and are mostly used for maneuvering targets. An autonomous guidance system is usually used when predefined way points are desired or a certain target with known position has to be reached. Different types of sensors can be used for precise navigation such as inertial sensors and digital scene matching area correlation (DSMAC) radar.
In order to orient the missile toward intended target points, it is vital to acquire the correct information about the states of the targets during the flight of the vehicles. One of the most widely used ways to achieve this task is the utilization of seekers which act as the guidance sensors mentioned earlier [3]. Physically, the measurement capability of seekers is restricted due to some physical, optical, and electronic limitations such as limited field of view (FOV), atmospheric transmittance, and noise effects. Regarding these characteristics, basically two types of seekers are employed in the relevant applications: strap-down or body-fixed seekers and gimbaled seekers. The strap-down seekers are directly mounted on the considered vehicle body. Therefore, their measurements become relative to the body-fixed reference frame of the missile. For relieving the FOV limitations of the strap-down seekers, the gimbaled seekers are preferred in some of the implementations. In this scheme, the seeker is mounted on a platform supported by two orthogonal gimbals usually powered by DC motors and stabilized by means of rate gyro feedback. This way, the FOV range of the seeker is increased considerably. Also, the line of sight (LOS) angle and the LOS angular rate can be measured directly independently of the missile motion.
The majority of homing guided missiles use gimbaled seekers; an example of gimbaled seekers is shown in Figure 1 [2]. This allows the sensor to be pointed at the target when the missile is not. This is important for two main reasons. One is that, before and during launch, the missile cannot always be pointed at the target. Rather, the missile points its seeker at the target using information from its launching station. After this, the seeker remains locked on the target, even if the launching platform moves. When the missile is launched, it may not be able to control the direction it points at until the rocket motor drives the missile to high enough speed for fins to control its direction of travel. Until then, the gimbaled seeker needs to be able to track the target independently.
Even while it is under positive control and on its way to intercept the target, it probably will not be pointing directly at it; unless the target is moving directly toward or away from the launching platform, the shortest path to intercept the target will not be the path taken while pointing straight at it, since it is moving laterally with respect to the missile’s view. Old missiles would simply point towards the target and chase it (pursuit guidance); this was inefficient. Modern missiles are smarter and use the gimbaled seeker head combined with what is known as proportional guidance in order to avoid oscillation and to fly an efficient intercept path.
The control technique used for the gimbal system on a tactical missile must provide fast and precise tracking of relative error signals created by the missile’s signal processing unit. Poor performance during engagement will result in large miss distances which may lead to a low probability of mission success. The equations describing the gimbal system under consideration are highly nonlinear. In order to accurately calculate the missile-target LOS angle and its rate, accurate nonlinear control of the motion of the gimbaled seeker through the attached DC motors is required. The linear control techniques become inadequate and it becomes necessary to use nonlinear control. The competitive era of rapid technological change has motivated the rapid development of nonlinear control theory for application to challenging complex dynamical real-world problems [4].
There exist many nonlinear control design techniques; each has benefits and weaknesses. Most of them are limited in their range of applicability, and use of a certain nonlinear control technique for a specific system usually demands choosing between different factors, for example, performance, robustness, optimality, and cost. One of the highly promising and rapidly emerging techniques for nonlinear for nonlinear control design is the state dependent Riccati equation (SDRE) technique [5, 6]. Although the SDRE has a great impact in the missile guidance and control area [7–9], none of these works have addressed the problem of finite-horizon control of nonlinear systems.
Based on the great potential of the SDRE for infinite-horizon control of nonlinear systems [10], this paper offers an online technique for finite-horizon regulation and tracking of nonlinear stochastic systems. This is accomplished by integrating of the differential SDRE filter with the finite-horizon SDRE technique. A change of variables is used to convert the nonlinear differential Riccati equation (DRE) to a linear differential Lyapunov equation (DLE) [11–13]. The tracking problem is solved in real time [14].
The structure of the paper is as follows. Section 2 presents a brief overview of the standard Kalman filter, which is the base of the differential SDRE filter. Section 3 presents the SDRE finite-horizon regulator problem. In Section 4, the finite-horizon tracking problem is discussed. Section 5 presents the nonlinear regulator with incomplete state information. Section 6 presents the tracking for nonlinear stochastic systems. In Section 7, the simulation results are presented, followed by the last section with the conclusions of this paper.
2. Standard Kalman Filter
The Kalman filter was developed by Kalman in 1960 [15]. The filter can be used to estimate the states of continuous-time or discrete-time linear systems. In this section, a brief overview of continuous-time Kalman filter for linear systems, which is needed in later sections, is given.
Consider the linear, continuous-time, stochastic system with dynamic model: where and are process and measurement (white, Gaussian) random noises with zero mean (i.e., ) and covariances and , respectively, and are assumed to be uncorrelated. The estimated state is given by which can be rewritten as where is the estimator gain and is the state estimate with initial value where stands for expected, average, or mean value and is considered intuitively equal to the estimate and is the initial corresponding state estimate error covariance.
Let us define the error between the true or actual state and the state estimate as Substituting from (1) and (4) which can be rewritten as where Use the results from [16] on the propagation of state vector and the corresponding state estimate error covariance which is obtained from Now, use result (12) for error dynamics (9) where is to be solved in forward direction with initial condition given in (6).
We have the condition on for minimum error variance as Using the Kalman gain (14) in the covariance relation (13), we get with initial condition .
This is called the continuous-time differential Riccati equation (CDRE) arising in state estimation.
Figure 2 shows a structure of the standard linear continuous-time Kalman filter.
3. Nonlinear Finite-Horizon Regulator via Differential SDRE
Finite-horizon control of nonlinear systems is a challenging problem in the control field due to the complexity of time dependency of the Hamilton-Jacobi-Bellman (HJB) differential equation. In these problems, the differential Riccati equation (DRE) cannot be solved in real time because the DRE arising in the control can only be solved backward in time from its known final value. To overcome this problem, an approximate analytical approach is used [11–13] to convert the original nonlinear Ricatti equation to a linear differential Lyapunov equation that can be solved in closed format at each time step.
3.1. Problem Formulation
The nonlinear system considered in this paper is in the form This nonlinear system can be expressed in a state dependent like linear form, as where , , .
The goal is to find a state feedback control law of the form , that minimizes a cost function given by [17]: where and are symmetric positive semidefinite matrices and is a symmetric positive-definite matrix. Moreover, is a measure of state accuracy and is a measure of control effort [18].
3.2. Solution for Finite-Horizon Differential SDRE Regulator
To minimize the above cost function (18), a state feedback control law is given as where is a symmetric, positive-definite solution of the differential SDRE; strictly speaking it could be called state dependent differential Riccati equation (SDDRE) of the form with the final condition The resulting trajectory becomes the solution of the state dependent closed-loop dynamics As the SDRE is a function of (), we do not know the value of the states ahead of present time step. Consequently, the state dependent coefficients cannot be calculated to solve (20) with the final condition (21) by backward integration from to . To overcome this problem, an approximate analytical approach is used, which converts the original nonlinear differential Ricatti equation to a linear differential Lyapunov equation, that can be solved in closed form at each time step [13].
4. Nonlinear, Finite-Horizon Tracking Using Differential SDRE
4.1. Problem Formulation
Consider the nonlinear system given by (16). Let be the desired output. The goal is to find a state feedback control law that minimizes a cost function given by where , and are symmetric positive semidefinite matrices, and is a symmetric positive-definite matrix.
4.2. Solution for Finite-Horizon Tracking Using Differential SDRE
To minimize the cost function (23), a feedback control law is given as where is a symmetric, positive-definite solution of the differential SDRE of the form with the final condition The resulting SDRE-controlled trajectory becomes the solution of the state dependent closed-loop dynamics where is a solution of the state dependent nonhomogeneous vector differential equation with the final condition
Similar to Section 3, we do not know the value of the states ahead of present time step. Consequently, the state dependent coefficients cannot be calculated to solve (28) with the final condition (29) by backward integration from to . To overcome this problem, an analytical approach is used, which converts the original nonlinear differential equation to a linear differential equation, that can be solved in closed form at each time step [14].
5. Finite-Horizon Regulation for Stochastic Systems via Differential SDRE
5.1. Finite-Horizon Estimation
Suppose that the entire state is not available but only the output is measurable. Let us reproduce the nonlinear system with noises in state dependent form: In order to find the best estimate and the corresponding covariance matrix , we use the results of Section 2. At each time step, the filter (estimate) equations are where , the differential SDRE estimator (filter) gain, is obtained as and is the solution of the matrix differential Riccati equation is to be solved in forward direction with initial condition for any real-time implementation, whereas the standard differential Riccati equation, arising in control problem, is to be solved in backward direction with a final condition.
Note that we used subscript for matrices and above to indicate that they refer to the estimation problem and we designate the matrices and with the subscript to indicate that they refer to the control problem in stochastic control.
The minimization of is equivalent to minimization of
5.2. Finite-Horizon Control
At each time step, similar to the results of nonlinear regulator obtained in Section 3, except that the state is now the estimated where is the controller gain, is the approximate solution of the SDRE, and (35) represents the separation principle.
The summary of the nonlinear regulator problem is shown in Figure 3, and the overview of the process of finite-horizon SDDRE regulation technique for stochastic systems is summarized in Figure 4.
The entire algorithm of combined estimation and control leading to the nonlinear regulator problem is shown in the following steps.
(1) Finite-Horizon Estimator. Consider the following.(i)At each time step, solve the differential matrix Riccati equation in the forward direction with the initial condition .(ii)Obtain the differential SDRE estimator (filter) gain as (iii)Solve for the estimated state from with the initial condition .
(2) Finite-Horizon Controller. Consider the following.(i)Solve algebraic Riccati equation (ARE) to calculate the steady state value (ii)Use a change-of-variables procedure and assume that such that .(iii)Calculate the value of the closed-loop matrix as .(iv)Calculate the value of by solving the algebraic Lyapunov equation [19] (v)Solve the differential Lyapunov equation The solution of (42) is given by [20] (vi)Calculate the value of from (vii)Finally, calculate the value of the control as
6. Finite-Horizon Tracking for Stochastic Systems via Differential SDRE
6.1. Finite-Horizon Estimation
Similar to Section 4, the minimization of is equivalent to minimization of where the estimated error .
6.2. Finite-Horizon Control
At each time step, similar to the results of nonlinear tracking obtained in Section 4, except that the state is now the estimated The summary of the nonlinear tracking problem is shown in Figure 5. Here, we see that the original plant is subjected to process noise and measurement noise . Figure 6 summarized the overview of the process of finite-horizon SDDRE tracking technique for stochastic systems.
The entire algorithm of combined estimation and control leading to nonlinear tracking problem is shown in the following steps.
(1) Finite-Horizon Estimator. Consider the following.(i)At each time step, solve the differential matrix Riccati equation in the forward direction with the initial condition .(ii)Obtain the differential SDRE estimator (filter) gain as (iii)Solve for the estimated state from with the initial condition .
(2) Finite-Horizon Controller. Consider the following.(i)Solve for similar to the differential SDRE regulator problem in Section 5.(ii)Calculate the steady state value from the equation (iii)Use the change-of-variables procedure and assume that .(iv)Solve the resulted differential equation to obtain (v)Calculate the value of from (vi)Calculate the value of the finite-horizon control as
7. Simulation Results
For numerical simulation and analysis, the developed nonlinear finite-horizon regulation and tracking technique with incomplete state information is applied in two illustrative examples; in the first example the developed technique is implemented for a DC motor attached to a realistic gimbaled seeker system, and in the second example the developed technique is applied to two powered vehicles, that are initially at different ranges and aspect angles from a stationary target, with the requirement to hit the target at the same time, and this technique is known as Time on Target (TOT).
7.1. Example 1 (Missile Gimbaled Seeker)
The missile examined in this paper is a semiactive missile. The semiactive missile system is one that selects and chases a target by following the energy from an external source, such as tracking radar, reflecting from the target. This illuminating radar may be ground-based, shipborne, or airborne. The semiactive missile gets the target data and locked on the target before firing from the missile launcher, and the main task of the missile seeker is to track the locked target and to minimize the miss distance. While solving the guidance equations, the final time was assumed to be fixed and the constraint of zero miss distance was imposed. Also, there is no constraint on the lateral acceleration capability of the missile. In other words, the semiactive missile was assumed to have finite time of flight and infinite lateral acceleration capability.
The developed nonlinear finite-horizon tracking technique with incomplete state information is implemented for a DC motor attached to a realistic gimbaled seeker system, A computer code written under MATLAB environment is employed to solve a missile simulation model [21]. The code is devoted to evaluate the structure of the 6-degree of freedom (6 DOF) model in conjunction with the calculation of the desired seeker angles via numerical implementation [22]. Proportional navigation is the guidance method used in these simulations. In this guidance method, the guidance commands are generated in proportion to the LOS angular rate [23].
Extensive simulation has been carried out in pitch plane only for better illustrations using two engagement scenarios: (1) a fixed target and (2) a maneuvering target.
The nonlinear equations of the system can be written in the form where is the armature resistance, is the armature inductance, is the voltage applied to the motor, is the current through the motor, is the back emf voltage, is the moment of inertia of the load, is the viscous friction coefficient, is the torque generated by the motor, is the angular position of the motor, is the angular velocity of the motor, is the back emf constant, is the torque constant of the motor, and is the motor static friction.
The signum function is defined as or it can be written in this form: The nonlinear state equations can be written in the form where , , , . Or alternatively in state dependent form The weight matrices are chosen to be The covariances of the noises have been taken as
Case 1 (fixed target). Consider a fixed target; in this case the desired seeker angle will be ; that is, the problem is now a regulator problem.
The simulations were performed for final time of 8 seconds, and the engagement scenario is shown in Figure 7. The resulting trajectories for the demanded and achieved seeker angles are presented in Figure 8, and the error is shown in Figure 9.
In Figure 8, the solid line denotes the estimated (with noise) angle trajectory of the finite-horizon tracking controller, and the dashed line denotes the desired seeker angle.
As shown in Figure 7, a successful hit is observed. Figure 8 shows that the finite-horizon differential SDRE nonlinear regulating algorithm gives excellent results and the developed technique is able to solve the differential SDRE finite-horizon nonlinear regulator problem with a zero average error and standard deviation.
Case 2 (maneuvering target). Consider a highly maneuvering target. The simulations were performed for final time of 10 seconds, and the engagement scenario is shown in Figure 10. The resulting trajectories for the demanded and achieved seeker angles are illustrated in Figure 11, and the error is shown in Figure 12.
In Figure 11, the solid line denotes the estimated (with noise) angle trajectory and the dashed line denotes the desired seeker angle trajectory.
Figure 10 shows that a successful hit is observed with acceptable miss distance. Comparing these trajectories in Figure 11, it is clear that the gimbaled seeker is achieving good tracking even when the target is executing a high-g maneuver. The gimbaled seeker controlled by the developed method is able to track maneuvering target with standard deviation error of , which is very acceptable considering the high maneuver. After the maneuver, the vehicle flies constant velocity so that unaugmented PRONAV can be used to achieve the successful hit [24].
7.2. Example 2 (Time on Target)
Time on Target (TOT) is the military coordination of artillery fire by many weapons so that all the munitions arrive at the target at precisely the same time. This is useful because more attacks can land on the target at the same time with no warning, and that will improve the overall attack accuracy [25].
The developed nonlinear finite-horizon tracking technique with incomplete state information is implemented for a motor attached to the tracker of two powered vehicles with the requirement to hit a stationary target at the same time, and these vehicles are initially at different ranges and aspect angles from the target, as illustrated in Figure 13.
As shown in Figure 13, the line of sight between the stationary target and vehicle A is larger than the line of sight between vehicle B and the same target. To guarantee that the munition from both vehicles will hit the target at the same time, it is required that the munition from the further vehicle, vehicle B, to make a certain maneuver to the target, and the munition from vehicle A to make a direct path to the target with no maneuver. This scenario can grantee that the munitions from both vehicles A and B hit the target at the same time, as shown in Figure 14.
The dynamic equations for the tracker system are as follows: where is the control voltage, the motor inductance, the current through the motor winding, the motor winding resistance, the motor’s back electromagnetic force constant, the error angle, the mass of tracker, the gravitational constant, and the damping (friction) constant.
The nonlinear state equations for the system are written in the state dependent form where , , , and .
Let the selected weighted matrices be The covariances of the noises have been taken as
(1) Vehicle A. Consider a direct path (shot) to the target; in this case the desired tracker angle will be ; that is, the problem is now a regulator problem.
The simulations were performed for final time of 6 seconds, and the engagement scenario is shown in Figure 15. The resulting trajectories for the demanded and achieved tracker angles are presented in Figure 16, and the error is shown in Figure 17.
In Figure 16, the solid line denotes the estimated (with noise) angle trajectory of the finite-horizon tracking controller, and the dashed line denotes the desired tracker angle.
As shown in Figure 17, the finite-horizon differential SDRE nonlinear regulating algorithm with incomplete state information gives excellent results and is able to solve the nonlinear regulator problem with a zero average angle error.
(2) Vehicle B. Consider a maneuvering path (shot) to the target, such that the final time is to be 6 seconds; that is, the problem is now a tracking problem.
The engagement scenario is shown in Figure 18. The resulting trajectories for the demanded and achieved tracker angles are presented in Figure 19, and the error is shown in Figure 20.
In Figure 19, the solid line denotes the estimated (with noise) angle trajectory of the finite-horizon tracking controller, and the dashed line denotes the desired tracker angle.
Figure 18 shows that a successful hit is observed. Comparing these trajectories in Figure 19, it is clear that the system is achieving very good tracking even when the vehicle shot is executing a maneuver. The tracker controlled by the developed technique is able to hit the target in exactly 6 seconds with standard deviation error of .
8. Conclusion
In this paper, we proposed a finite-horizon regulation and tracking techniques used for nonlinear stochastic systems. The main idea of the proposed technique is to integrate the differential SDRE filter algorithm and the finite-horizon SDRE technique. The finite-horizon SDRE is based on change of variables that converts the nonlinear differential Riccati equation (DRE) to a linear differential Lyapunov equation. The proposed technique is effective for wide range of operating points. Simulation results for a gimbaled seeker in a missile system and Time on Target engagement scenario are included to demonstrate the effectiveness of the presented technique.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.