Efficient Neural Network Modeling for Flight and Space Dynamics Simulation

Abstract

This paper represents an efficient technique for neural network modeling of flight and space dynamics simulation. The technique will free the neural network designer from guessing the size and structure for the required neural network model and will help to minimize the number of neurons. For linear flight/space dynamics systems, the technique can find the network weights and biases directly by solving a system of linear equations without the need for training. Nonlinear flight dynamic systems can be easily modeled by training its linearized models keeping the same network structure. The training is fast, as it uses the linear system knowledge to speed up the training process. The technique is tested on different flight/space dynamic models and showed promising results.

1. Introduction

An artificial neural network (ANN), usually abbreviated as neural network (NN), is a mathematical model that is inspired by the structure of the brain biological neural networks. A neural network consists of an interconnected group of elements called neurons, arranged in different layers, and it processes information by propagating data through these elements. Neural networks use “weights” to change the parameters of the connections to the neurons. These weights are calculated through a learning process of the available data. Simply speaking, neural networks are nonlinear statistical data modeling tools used to model complex relationships between inputs and outputs. Two challenges face neural network designer: the selection of neural network topology (i.e., number of layers and number of neurons) and the initialization of the weights. It is well recognized that a perfect way to decide an appropriate architecture and assign initial values to start neural network training has yet to be established. A common criticism of neural networks came from those two challenges where wrong selection of topology or initial values for weights leads to slow operation [1–5].

A lot of work has already been devoted to the application of neural networks (NN) in dynamics and control, both with respect to controller design and to system modeling and identification [6–10]. Modeling dynamic systems or controllers by neural networks consumes a lot of the network designer time in selecting the appropriate structure and in training the network and had to be done offline. The network size also affects the execution time and has high impact on network design and selection especially for real-time problems.

The network design technique given in this paper is systematic to relieve the network designer from the intensive task of finding an appropriate structure for the neural model. For linear systems, based on the work done by Kaiser [11] and Kassem and Sameh [12], the paper can find the network weights and biases analytically so that the neural network can work directly without the need for training.

Nonlinear systems can be modeled by training its linearized models about specific operating point without the need to change the size or the structure of the network. The training is fast and can be done online, as it uses the linear system knowledge to speed up the training process [12].

A modified version of time-delay neural network is presented in this work, and when it is combined with the analytical modeling, it can fast and accurately model linear and nonlinear dynamic systems. The technique is tested on different flight and space dynamics models and showed good match with the linear and nonlinear equations.

2. NN Analytical Model with One Hidden Neuron

This section represents the mathematical equations for linearizing a neural network model consisting of an input neuron, a hidden neuron, and an output neuron (Figure 1). The input and output neurons have linear activation functions, and the hidden neuron has a sigmoid function. The ability of this neural network model to approximate the straight-line equation 𝑦=𝑎𝑥+𝑏 is an essential feature to allow direct representation of difference equations. During the following calculations, 𝜎(𝑥) denotes a continuous nonlinear transfer function applied in the neurons such as the sigmoid function 𝜎(𝑥)=1/(1+𝑒^−𝑥).

Figure 1 

The model used for approximating the function 𝑦=𝑎𝑥+𝑏.

Assuming that 𝑦=𝑎𝑥+𝑏 is to be approximated for 𝑥∈[𝑥₀,𝑥₁], 𝑥₁>𝑥₀ within an error limit of 𝜀, 𝜀>0, select 𝑧₀,𝑧₁ on the sigmoid horizontal axis with 𝑧₁>𝑧₀ such that |𝜎̇(𝑧)−𝑞𝑎|<𝜀/|𝑥₁−𝑥₀| for 𝑧∈[𝑧₀,𝑧₁] with a constant 𝑞∈𝑅. In other words, this step selects the interval on the horizontal axis of the sigmoid, where the sigmoid function can be seen as linear.

Define a function 𝑓, 𝑓∶[𝑥₀,𝑥₁]→[𝑧₀,𝑧₁] as 𝑓(𝑥)=𝑧₀+𝑧₁−𝑧₀𝑥₁−𝑥₀𝑥−𝑥₀=𝑧₀−𝑧₁−𝑧₀𝑥₁−𝑥₀𝑥₀+𝑧₁−𝑧₀𝑥₁−𝑥₀𝑥=𝑝₁+𝑝₂𝑥. (1) Finally, further modifications (see Kaiser 1994 [11] for details) yield𝑎𝑥+𝑏=𝑤𝑝₂𝑥+𝑝₁+𝑤₀, (2) where 1𝑤=𝑝₂𝑞,𝑤₀=−𝑎𝑝₁𝑝₂−𝜎(0)𝑝₂𝑞+𝑏. (3)

Taking 𝑝₂ as the weight to the corresponding input unit, 𝑝₁ as the bias of the hidden neuron, 𝑤 as the weight of the link between the hidden neuron and an output unit, and 𝑤₀ as the bias of the output unit results in the desired approximation. This analytic approximation forms the base for the algorithm that will be used to model the dynamics of linear systems. Figure 1 shows the model used in this paper for approximating the linear function.

3. Modified Time-Delay Neural Networks

Time-delay neural networks (TDNNs), introduced by Waibel et al. [13], form a class of neural networks with a special topology. They are used for position-independent recognition of features within a larger pattern. In order to be able to recognize patterns place or time invariant, older activation and connection values of the feature units have to be stored. This is performed by making a copy of the feature units with all their outgoing connections in each time step before updating the original units. The total number of time steps saved by this procedure is called delay.

In this work, the TDNN is modified to fit exactly the general difference equation for a linear 𝑛th order dynamic system with constant coefficients shown as follows [14]: 𝑦(𝑛+𝑘)+𝑎_𝑛−1𝑦(𝑛+𝑘−1)+⋯+𝑎₁𝑦(𝑘+1)+𝑎₀𝑦(𝑘)=𝑓(𝑘), (4) where 𝑦(𝑖), 𝑖=𝑘,𝑘+1,…,𝑘+2 denotes the discrete variable 𝑦 at the 𝑖th instant and 𝑓(𝑘) denotes the input value which could be another difference equation.

The output 𝑦(𝑛+𝑘) is separated to one side of the equation, while its delayed values and the input are moved to the other side of the equation as follows:𝑦(𝑛+𝑘)=−𝑎_𝑛−1𝑦(𝑛+𝑘−1)−⋯−𝑎₁𝑦(𝑘+1)−𝑎₀𝑦(𝑘)+𝑓(𝑘). (5) Now, the output value is just a summation of linear relations in the form 𝑦=𝑎𝑥 which can be modeled analytically as an NN with one hidden neuron. This way, the one hidden neuron NN is used as a building block, and the complete network is built systematically as shown in Figure 2.

Figure 2 

NN for modeling difference equation.

4. The Algorithm

First, the nonlinear system is linearized about an operating point (trim condition) using any linearization technique such as Taylor series or small disturbance theory. Second, an analytic NN model is build using this linear model after discretization. Finally, the model is trained with the nonlinear model data points and with initial values for weights and biases taken from the linear analytic NN model. The detailed modeling algorithm is shown in Algorithm 1. For linear systems steps 2, 3, and 4 only apply, as there is no linearization, and there is no need for training.

(1) Starting with the nonlinear model in the form of 𝑥̇(𝑡)=𝑓(𝑥,𝑢,𝑡), where x are states vector and u is input vector. (2) Linearize the nonlinear model around an operating point and represent it in state space form as follows: 𝑥̇(𝑡)=𝐴𝑥+𝐵𝑢, where A, B are the system and input matrices, respectively. (3) Discretize the continuous linearized system using finite difference, the equations will be 𝑥̇𝑖+1=(𝐴+𝐼Δ𝑡)𝑥𝑖+𝐵Δ𝑡𝑢𝑖, where I is the identity matrix and Δt is the time step. (4) Build an analytic NN to replicate the linear model using the proposed linear system given in section two and Figure 2. (5) Train the linear analytic NN with the nonlinear model data and with initial weights and biases taken from the linear analytic NN. (6) If the final NN model is not satisfactory, then increment model size by adding another linear model at different operating point, repeat steps 2 to 5, and concatenate the two models to get a better approximation. The number of final concatenated models depends on the required accuracy and the acceptable execution time. (7) Stop

Algorithm 1 

NN algorithm for modeling nonlinear systems.

5. Test Cases

Three test cases will be given in this section illustrating the application of the proposed modeling approach using NN. The first two test cases in Sections 5.1 and 5.2 show the result of modeling of unstable linear dynamics without details of the proposed algorithm. The third nonlinear test case presented in Section 5.3 will be accompanied with more details.

5.1. Helicopter Longitudinal Dynamics

The state space linear model of longitudinal motion of a helicopter [15] is given by 𝑞̇𝜃̇𝑢̇=𝑞𝜃𝑢+0−0.40−0.01100−1.49.8−0.026.39.8𝛿, (6) where 𝑞, 𝜃, and 𝑣 are pitch rate, pitch angle of the fuselage, and horizontal forward velocity, respectively, and the control input 𝛿 is the rotor tilt angle. The system is unstable with both zeros and poles in the right hand plane, for example, system poles 𝑝_1,2=0.3776±0.3445𝑗;𝑝₃=−0.375. (7) Using the linear algorithm, the NN can fit the system’s three states very accurately for unit step 𝛿 input as shown in Figure 3.

(a)

(b)

(c)

Figure 3 

NN versus state-space models for helicopter dynamics described by (6).

5.2. F-16 Longitudinal Dynamics

The equations for longitudinal motion of the F-16 fighter are considered for a trimmed flight condition with 𝑈=502 ft/s, 𝛼=0.0393 rad, 𝜃=0.0393 rad, and 𝑞=0 rad/s. The state vector is 𝑥=[Δ𝑢,Δ𝛼,Δ𝜃,Δ𝑞], where Δ𝑣, Δ𝛼, Δ𝜃, and Δ𝑞 are the longitudinal velocity, angle of attack, pitch angle, and pitch rate, respectively. The single control input is the elevator deflection 𝛿_𝐸 in degrees. The state equation in matrix form becomes [16]𝑥̇=𝐴𝑥+𝐵𝛿_𝐸, (8) where,0.𝐴=−1.9311𝑒−28.8157−32.17−0.575−2.5389𝑒−4−1.018900.905100012.9465𝑒−120.82230−1.0774𝐵=0.1737−2.1499−0.1756 (9) Using the linear algorithm, the NN can fit the system’s four states very accurately for unit step input 𝛿_𝐸, as shown in Figure 4.

(a)

(b)

(c)

(d)

Figure 4 

NN versus state-space models for F-16 longitudinal dynamics.

5.3. Generic Fighter Nonlinear Flight Dynamics

Consider the flight dynamics model of a high-performance aircraft for the short-period mode [17–19], where the motion involves rapid changes to the angle of attack and pitch attitude at roughly constant airspeed. The model covers both linear and nonlinear flight behavior through an extensive range in angle of attack. The model is mathematically described as [18]𝛼̇=𝑞+9.168𝐶_𝑧−1.834𝛿_𝑒̇+7+7.362,𝑞=5.73−𝛼−1.5𝛿_𝐸+2.865, (10) where𝐶_𝑧=−0.07378𝛼𝛼≤14.36,0.09722𝛼²−2.865𝛼+20.0414.36<𝛼≤15.6,−0.01971𝛼²+0.7439𝛼−7.80815.6<𝛼<19.6,−0.4733−0.01667𝛼𝛼≤19.6, (11)

and 𝛼 is the angle of attack in degrees, 𝑞 is the pitch rate in degrees per second, 𝛿_𝑒 is the elevator control surface deflection in degrees, and 𝐶_𝑍(𝛼) is the normal force coefficient and the sole nonlinearity in the model. The system dynamics can be summarized as(a) with angle of attack <14.36, the aircraft has stable dynamics, (b) with angle of attack between 14.36 and 19.6 degrees, the aircraft has oscillatory instability, (c) onset of limit cycle oscillations (LCO) at high angles of attack.

It is obvious that a single linear model cannot represent this nonlinear behavior. So, this example will show the strength of the proposed algorithm. It will start with a single linear NN model and then train it to capture the nonlinear behavior, then adding a second model at different operating point to improve the accuracy.

The linearized model for 𝛼<14.36 deg is given by 𝛼̇𝑞̇=𝛼𝑞+𝛿−0.7671−5.730−1.8348.595^𝐸+−5.4762.865. (12)

The discretized system (12) using Δ𝑡=0.004 s is given by𝛼_𝑖+1𝑞_𝑖+1=9.973𝑒−014.0𝑒−03𝛼−2.292𝑒−021.00𝑞^𝑖_𝑖+𝛿−7.336𝑒−03−3.438𝑒−02^𝐸𝑖+.−2.1904𝑒−021.1460𝑒−02 (13) Using the linear algorithm, the NN can fit the system’s two states very accurately for the unit step input 𝛿_𝐸, as shown in Figure 5. This assumes that the angle of attack does not exceed value 14.6 deg.

(a)

(b)

Figure 5 

Linear NN model versus state-space model for fighter linearized dynamics.

We can see the difference when using the linear NN model for predicting behavior of the nonlinear system when angle of attack exceeds value of 14.6 deg., as shown in Figure 6. Therefore, the NN needs some training. A step input of 𝛿_𝐸=−11 for five seconds is used to train the system, and a fifteen seconds step input with same value (𝛿_𝐸=−11) is used for testing, and the results are shown in Figure 7. Initial weights and biases were taken from the linear NN to speed up the convergence.

(a)

(b)

Figure 6 

NN trained model versus nonlinear models for fighter dynamics.

(a)

(b)

Figure 7 

NN trained model 1 versus nonlinear model for fighter dynamics.

When initiating the NN learning with zero or random weights and biases, it has been noticed that it does not converge to the correct values or in best case converges after 5 times the required number of iterations compared with the case of using linear NN weights and biases.

It is clear that the NN works relatively good compared with its size (4 sigmoid elements). The model can be improved by adding another model (at different operating points).

Linearizing the system for 𝛼>19.6, we get the following state space model:𝛼̇𝑞̇=𝛼𝑞+𝛿−0.1531−5.730−1.8348.595^𝐸+8.7152.865. (14) Equation (14) is discretized using Δ𝑡=0.004s to give 𝛼_𝑖+1𝑞_𝑖+1=9.994𝑒−014.0𝑒−03𝛼−2.292𝑒−021.00𝑞^𝑖_𝑖+𝛿−7.336𝑒−03−3.438𝑒−02^𝐸𝑖+.3.486𝑒−021.146𝑒−02 (15) Same procedure is used to produce the NN model for this linear system. Adding the two NN models, we get double the size NN (8 sigmoid elements). After training the concatenating model, we can get an improved model which produces a much better results as shown in Figure 8. It is clear that this improved model can catch the limit cycle very well, so there is no need to add another NN model in the region between 14.36 and 19.6 degrees.

(a)

(b)

Figure 8 

NN trained model 2 versus nonlinear model for generic fighter dynamics.

5.4. Satellite Attitude Dynamics

This final case study shows a space-related problem and also illustrates the effectiveness of the proposed NN for systems with sparse dynamic matrix (𝐴). In this case study, the linearized attitude dynamics for a symmetric satellite for small angles and gravity gradient torques approximation is given by [20]𝑥̇=𝐴𝑥+𝐵𝑢, (16) where𝜃𝜙𝜓𝜃̇𝜙̇𝜓̇𝛿𝑥=,𝐴=000100000010000001₁000000𝛿₂000𝛿₃00𝛿₄0𝛿₅0,0𝐵=1000000000𝐼_𝑦01𝐼_𝑥10000𝐼_𝑧𝑇,𝑢=𝑇^𝑥_𝑦𝑇_𝑧,𝛿₁=−3𝜔²₀𝐼_𝑦𝐼_𝑥−𝐼_𝑧,𝛿₂𝜔=−4²⁰𝐼_𝑥𝐼_𝑦−𝐼_𝑧,𝛿₃𝜔=−⁰𝐼_𝑥𝐼_𝑦−𝐼_𝑧−𝐼_𝑥,𝛿₄𝜔=−²⁰𝐼_𝑧𝐼_𝑦−𝐼_𝑥,𝛿₅𝜔=−⁰𝐼_𝑦𝐼_𝑧+𝐼_𝑥−𝐼_𝑦. (17) The satellite altitude = 700 km, its moments of inertia 𝐼_𝑥=80, 𝐼_𝑦=82, and 𝐼_𝑧=9 kg-m², and an initial attitude is represented by 𝜃(0) = 5°, 𝜙(0) = 5°, and 𝜓(0) = 5°. Figure 9 shows initial condition response of the satellite using the state-space and NN models. The NN model is built by direct substitution without the need to train the network, and the results show good match. Also, the NN model is very efficient, as it uses only eight (one hidden neuron NN) for matrix A representing only the nonzero elements.

(a)

(b)

(c)

Figure 9 

NN versus state-space models for satellite dynamics.

6. Conclusions

The procedure and the test cases described in this paper show that the linear neural network based on the analytic one hidden neuron NN is best suited as a starting point for fast learning nonlinear networks. It also shows that the modified Time-delay neural network combined with the analytical network form a very efficient algorithm for flight dynamics equations approximations. The presented algorithm relieves the network designer from the work intensive task of finding a suitable structure for the neural network, given that the designer has at least some qualitative knowledge about the modeling task. It also shows a systematic way of increasing the size of the neural network to improve the accuracy without the need of guessing. It is clear that the algorithm gives some guidance to the learning process and helps the network escaping local minima by using initial guess from the linear model. This illustrates the importance and usefulness of this algorithm in the field of neural network modeling of nonlinear flight dynamics.