Complexity

Volume 2018, Article ID 3671428, 12 pages

https://doi.org/10.1155/2018/3671428

## About Extracting Dynamic Information of Unknown Complex Systems by Neural Networks

^{1}UPV/EHU, Alda, Urquijo, s/n, Bizkaia, Spain^{2}UHU, ETSI, Campus de El Carmen, Huelva, Spain

Correspondence should be addressed to Eloy Irigoyen; sue.uhe@neyogiri.yole

Received 18 October 2017; Revised 25 April 2018; Accepted 16 May 2018; Published 8 July 2018

Academic Editor: Marek Reformat

Copyright © 2018 Eloy Irigoyen et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

This work presents a straightforward methodology based on neural networks (NN) which allows to obtain relevant dynamic information of unknown nonlinear systems. It provides an approach for cases in which the complex task of analyzing the dynamic behaviour of nonlinear systems makes it excessively challenging to obtain an accurate mathematical model. After reviewing the suitability of multilayer perceptrons (MLPs) as universal approximators to replace a mathematical model, the first part of this work presents a system representation using a model formulated with state variables which can be exported to a NN structure. Considering the linearization of the NN model in a mesh of operating points, the second part of this work presents the study of equilibrium states in such points by calculating the Jacobian matrix of the system through the NN model. The results analyzed in three case studies provide representative examples of the strengths of the proposed method. Conclusively, it is feasible to study the system behaviour based on MLPs, which enables the analysis of the local stability of the equilibrium points, as well as the system dynamics in its environment, therefore obtaining valuable information of the system dynamic behaviour.

#### 1. Introduction

The innumerable strategies and new proposals in the control system area are generally based on the knowledge of the system to be controlled. In some cases, its model is achieved by mathematical and analytical procedures, because such model is a simplified representation of the system, or the problem to study is highly restricted to a specific operating point. Although the modelling is often approached by the means of mathematical and analytical methods, with the growing complexity involved in the systems and the requirements of accurate representation, such approaches turn into an increasingly challenging solution.

Consequently, considering the maturity of computational intelligence (CI) techniques and new enabler technologies, CI methods represent an attractive alternative to develop accurate modelling and control solutions for highly complex models [1–4].

Taking into account the need to obtain precise models of complex systems which are applicable in a wide operating range, the CI techniques are an ideal method to reproduce the behaviour of a complex system [2, 5, 6] or, as it will be presented in this work, to analyze the dynamics of systems.

When studying a process in engineering, in biomedical field, in natural sciences, and even in social systems, approaching the analysis from a dynamic point of view can be very attractive and convenient, depending on the focus of such study. A dynamic analysis of the system can provide wide and very rich information related to how the system will respond under certain inputs. Moreover, it can allow to study its dynamic behaviour through the analysis of the stability in open-loop, both locally and globally. In addition, it will be possible to study whether certain nonlinear phenomena affect the system, for example bifurcations, saddle, and limit cycles [7, 8].

The methods traditionally applied in control engineering are based on linear approximations around several operating points of the system. This is suitable when problems are studied and solved in a local domain. However, there is a trend to approach bigger problems with a more abstracted and global perspective, leading to the use of nonlinear methods [9–12]. At some point of complexity, and certainly if the system involves unknown parts and other uncertainties, an entirely formal analysis of the system becomes unsustainable [13–15].

One of the main reasons for the use of nonlinear models is based on the dynamics of linear systems, since conventional mathematical formulations are not rich enough to reproduce a series of phenomena that usually appear in the real life [2, 16, 17]. The dynamic behaviour of a linear system, without considering its order, is basically governed by the eigenvalues of the corresponding state matrix [8]. On the contrary, nonlinear systems show a much richer behaviour, with self-excited oscillations (i.e., limit cycles), aperiodic behaviours and critically sensitive to the initial conditions [7], and chaos [18], as well as other dynamic phenomena exclusive to nonlinear systems, such as the existence of multiple states of equilibrium and bifurcations [19], among others.

The typically appropriate initial approach to analyze nonlinear systems is to use a representation of the system by means of a mathematical model, generally represented in state variables. This is possible assuming that sufficient information and knowledge of the system is available to generate its state equations, provided that the system dynamic is not extremely complex. In many applications, current research deals with the study of unknown complex systems, whether due to a complex dynamic, high dimensionality, or lack of information about the physical relationships that govern the behaviour of the system. In such situations, the techniques from the field of intelligent control can help to improve these studies, as Barragán et al. present in [20, 21] using Fuzzy logic to define a formal methodology for analyzing the dynamic behaviour of nonlinear systems, or Grande et al. in [11] to extract qualitative models of spatial evolution from a chemical system. In the same sense, neural networks (NN) become a powerful technique, since they are able to model highly complex nonlinear systems from input-output data. Proper selection and training of a basic structure such as a multilayer perceptron (MLP) can accurately reproduce the behaviour of a nonlinear system. This modelling technique can be used, both qualitatively and analytically [6, 22–25], taking into account that MLPs are universal approximators, either for a function [26–29] or its derivative [30, 31]. Thus, although the system might be unknown, it is possible to obtain a NN model of its behaviour, representing its dynamics in the workspace studied. In a formal sense, a NN model is a mathematical model. Hence, from this NN model, it is possible to study several aspects of the real dynamics of the system, conditioned only by the high precision of the model. This can be achieved with an exhaustive experimental stage where the topology of a NN is selected, capable of faithfully reproducing the behaviour of the system with the expected precision. This approach in solving this problem allows dealing with nonlinear systems, where modeling by traditional mathematical techniques can be challenging.

During the analysis and design of control solutions, knowing the equilibrium states of a system, as well as the stability of such states, is an aspect of great interest. When the model of the system is completely unknown, this information could help to clarify how the system works, even to ease the design of an appropriate control. It should be noted that despite the existence of recent works that present formal analysis methodologies based on Fuzzy logic [17, 20, 21, 32] the authors have not currently found any work focused from the NN point of view, under a general approach as presented in this proposal.

This work presents a straightforward and easy to use methodology for extracting information from unknown systems using NNs. The main objective of this proposal is to develop a method that allows obtaining information on the dynamics of nonlinear systems, when there is no mathematical model, neither accurate nor approximate, to analyze them. In these situations, any additional information reached by new methods is significant, especially when this information is related to the analysis of the presence of equilibrium states and their local stability, as presented below. In this work, an MLP neural network is trained with a set of measured values of inputs and outputs of supposedly unknown systems, in order to reproduce the behaviour of these systems. For this purpose, taking into account that the dynamic study aims at analyzing the behaviour of each nonlinear system in their corresponding equilibrium states, the dataset of examples to train the NN will be obtained from its entire operating range. More specifically, the equilibrium states of three nonlinear systems will be studied through their NN models, which reproduce their corresponding state variable models. The equilibrium states are reached by a precise linearization in a grid of operating points extracted from the NN models, and subsequently performing a study of local stability. Using this information, the local stability of equilibrium states is obtained, as well as the system dynamics in the vicinity of the studied points, achieving valuable information about the dynamic behaviour of the nonlinear system.

This paper is organized as follows: Section 2 presents the problem and the formulation associated with it, explaining how it will be dealt with throughout the document. In Section 3, the procedures to obtain the linearization of a system and its extension to a NN model are explained. Section 4 presents three case studies to demonstrate the proposed approach, based on solid results. Finally, this work finalizes with the corresponding conclusions.

#### 2. Problem Formulation

A generic continuous dynamic system will be considered, represented by state variables , where and depict a input system of order with representing the inputs and the state variables. is a static nonlinear map defined as [33]. An equivalent NN model, based on a MLP structure, which can estimate both the state variables of the continuous system and the system output , is represented by the equations in (1) [34–36].
where is the regression vector and the vector of parameters of the NN, the inner weights, and the biases. is the function realized by the MLP, defined as . To model the evolution of each *i*th state variable, a MLP structure has to be trained adapting the parameters mentioned above. In the training process, the needed information is provided for both the states , with , and the system inputs , with .

By selecting a simple MLP structure that consists of one hidden layer of neurons, with sigmoidal activation functions, and a linear output layer, the NN output can be calculated by the following general expression [35]: where is the number of hidden neurons, represents the output neuron (for the case of several NN outputs), are the output layer weights and biases, is the activation function of each hidden neuron, and is the sum of weighted inputs to each hidden neuron, as shown in where are the weights and biases of the hidden layer and is the input vector to the NN, being the vector dimension.

Taking into account that a NN will reproduce the evolution of each state variable , the neural model can be related to the state model [35] as

From the above representation, in order to simplify the methodology in studying the obtained NN models, each state variable will be modeled by a different NN.

#### 3. Information Obtained from the Neural Network Model

After obtaining an accurate model of a system, it is a fact that this model can be used to obtain system information through well-known techniques. In this section, a very important technique is presented to study nonlinear systems in two phases, as required by the methodology of this work. Firstly, the linearization of a neural state model will be exposed in detail. Secondly, the study of the equilibrium states of an unknown nonlinear system will be presented. This study is carried out through a NN model that reproduces the behaviour of the aforementioned nonlinear system. The study of the equilibrium states from the NN model, together with the study of their local stability from the linearization, allows to analyze the operational behaviour of a system from a qualitative point of view.

##### 3.1. Linearization of a Neural Model

Linearization is one of the most commonly used techniques in solving design problems in the field of nonlinear control systems, even though it is necessary to point out that this is a technique not ideal in many situations where the effects of nonlinearities are not negligible. It is a very convenient technique for the control of not excessively complex systems or in situations when the dynamics of the system is approximately known in regions where the system behaviour is close to a linear one, basically around equilibrium states.

Thereby, apart from being a method that aims at the control of systems, linearization could be a powerful resource to obtain information from a nonlinear system. It could be considered that, except in some situations, the behaviour of a nonlinear system around an equilibrium state is analogous to the one observed after linearization of the system in such state [19, 37, 38]. So, the study and calculation of equivalent linear systems from a nonlinear NN model can be a powerful technique to obtain information concerning the real nonlinear system analyzed.

The generic state model, obtained from a nonlinear system, is represented by

The first-order simplification of the Taylor series of the nonlinear system, in the domain of the state , can be determined an approximation as where , , and are (), (), and matrices, respectively.

Being and , the matrices of the linearized system are obtained as

For the rest of the presented work, the time dependence of state variables and system inputs will be suppressed in order to abbreviate the expressions.

If is an equilibrium state of the system, the matrix will be zero, since by definition, an equilibrium state leads the state equation to zero.

When the system (7) is represented by a NN model, the equivalent mathematical model is shown as (6). Linearizing the (7) around the state , the new equivalent mathematical model of a linearized neural model is represented by (10), being and the *q*th and *v*th vector components of and , respectively. These components are also the set of inputs of the NN.

By extending the previous (10) with (6), and consequently, (12) is obtained as in a reduced form to work with

Subsequently, based on the works of Pirabakaran and Becerra [23] and Larrea [39], where these derivatives are calculated through the internal connections of the NN, (13) can be calculated separately for each of its terms. For this purpose, it is necessary to define the activation function of the hidden layer neurons, . In this work, for the neurons of the hidden layer of the selected multilayer perceptron structure, the activation function hyperbolic tangent is chosen. The selection of a smooth activation function is performed to enable the calculation of the partial derivatives shown in (16). Applying the chain rule, we decompose the first part in three partial derivatives.

The first partial derivative of (14) is straightforward.

As the activation function is , we obtain that the second partial derivative of (14) results in
where is precisely the value of the *h*th neuron output for the MLP neural network, , whilst the third partial derivative in (14) results in

Then, the overall solution for is being an integer value into the interval .

For the second derivative , the procedure is similar to the previous one, obtaining the expression where has integer values into the interval .

Substituting (18) and (19) in (12), the equivalent mathematical model of a nonlinear system based on NN and linearization around a state is depicted by

##### 3.2. Equilibrium States and Local Stability

In order to perform an exhaustive analysis of the nonlinear system, it is first necessary to obtain an appropriate neural state model of that system, as presented in (1). This could be done through some of the existing modelling techniques [35], either online [40, 41] or offline [1, 6]. Subsequently, it is important to locate the different equilibrium states of the system. The search and location of such equilibrium states in a control system are one of the first problems that have to be solved in order to develop a study of the behaviour of the system. Before designing the control system, the identification and analysis of the equilibrium states provide valuable information about the behaviour of the system, especially in the case of a nonlinear system, since these states are the most relevant cases to study such systems through linearization techniques. In order to locate the equilibrium states of the system, (21) must be solved.

For nonlinear dynamics, the equilibrium states could be very difficult to solve analytically, so it is necessary to use numerical methods [26–28, 30]. Given the mathematical model of a NN system characterized by a MLP, see (3), the set of nonlinear equations to solve is the following:

These equations represent the *n* MLPs that model the nonlinear system under study. When analyzing the dynamics of nonlinear systems, the calculation of their equilibrium states can be a notable problem. In contrast to linear systems, where one or infinite equilibrium states exist, a nonlinear system can contain one, none, a finite number, or infinite states of equilibrium. For the resolution of the set of equations in (22), both numerical or more complex methods can be utilized. Complex methods, as bioinspired algorithms (i.e., evolutionary computation techniques), can locate a large number of solutions, but its slower convergence is a clear disadvantage in comparison with numerical methods. Then, in order to solve the set of nonlinear equations in (22), the use of numerical methods will be proposed, since they can offer a rapid convergence and precision in the obtained results [42, 43]. In this sense, the Levenberg-Marquardt (L-M) method [44] with the extension proposed by Moré [45] will be performed. This algorithm needs initial conditions to initialize the search. Thus, taking into account that it will be necessary to maximize the probability of finding every existing equilibrium state, a thin grid of points in the ranges of inputs and state variables will be used. Although this is the methodology that has been used in this article, the important thing in this case is to obtain as many solutions as possible, regardless of the method used to find them. Therefore, any other algorithm to solve (22) would be perfectly valid.

The L-M algorithm requires the Jacobian matrix of the system, in order to accelerate its convergence. This matrix can be obtained, either with explicit calculation or with some technique to approximate it. In the previous Section 3.1, the calculation of the Jacobian matrix of a NN model has been solved under a general approach. Therefore, this matrix can be included into the numerical algorithm to enhance its precision and velocity of convergence.

Furthermore, the Jacobian matrix can be used, both for solving and finding the equilibrium states and for the linearization of the system in each of the solutions obtained. In this way, it is also possible to study the characteristics of the located equilibrium states, from the eigenvalues of the dynamic matrix of the linearized system. This analysis could improve the interpretation of system dynamics; it could help to study the local stability, even to observe more complex behaviours, such as bifurcations, saddle points, or limit cycles.

#### 4. Case Studies

In this section, three different examples are presented. These examples come from different areas, being nonlinear electrical, mechanical, and biological systems, which initially will be considered as unknown. The algorithms have been implemented with the tools of MATLAB R, both for the MLP neural network training and for the calculation of the NN model linearization.

##### 4.1. Equilibrium States of a Tunnel-Diode Circuit

Let the tunnel-diode circuit shown in Figure 1, where , 5 k*Ω*, pF, and *μ*H, with and as variables of the system, and the nonlinear relation between both. This is a case of study broadly used [20, 21], with the state model expressed by (23).