Abstract
In the article, we apply complex-valued neural networks (CVNNs) to differential-algebraic neural networks (DANNs) and establish a new type of differential-algebraic complex-valued neural network (DACVNN) with delay (DDACVNN). First of all, the focus of existence and uniqueness of the solution to DDACVNN is addressed. Additionally, a theorem of global exponential stability (GES) of DDACVNN is investigated. In particular, in the discussion of this article, there is no restriction on whether the activation function requires that the real and imaginary parts can be dissociated. Finally, we will give two examples, namely, the activation function can separate the real and imaginary parts, and the activation function cannot separate the real and imaginary parts, both of which can confirm the truth of the effectiveness of theoretical results.
1. Introduction
Due to the various applications of artificial neural networks (ANNs) in graphics processing, combinatorial optimization, signal processing, and intelligent control, especially, in recently years, as artificial intelligence develops, the research on neural networks has received growing concern. When designing neural networks to solve application problems, it is crucial to guarantee the stability of the model under consideration. In the hardware implementation of neural networks, since the transmission of signals is delayed over time, delays inevitably occur, which may lead to network instability. Hence, the research on time-delay systems is meaningful and necessary. Also many research results can be seen in [1, 2] and its references.
During the past period, due to the diverse applications in multitudinous engineering and technical territories, such as pattern recognition, associative memory, and hole filling, complex-valued neural networks (CVNNs) are becoming increasingly needed in applications, as in [3]. Hence, it can be found initially from [4–6] that numerous CVNNs have been put forward and discussed.
As we all know, the state variables, activation functions, and connection weights of complex-valued systems are all defined in the complex number domain, and the analysis method is very different from that of real-valued systems. Actually, CVNNs have more superior characteristics than real-valued neural networks in complex signal processing and can solve more complex problems. For example, a single real-valued neuron cannot settle the XOR issue and the unearthing symmetry issue, and the application of complex-valued neurons can solve these problems (see [7]). For CVNNs, the staple difficulty is to look for an opposite activation function. Judging by Liouville’s theorem, CVNNs cannot select a smooth and bounded function as the activation function, so how to choose a suitable activation function for research also needs to be paid attention to the problem. With the wide application of the neural network, the development of neural network models that can handle complex-valued problems is needed in sundry fields, so the research on CVNNs is meaningful. At present, many researchers have studied the stability of CVNNs, and a large number of related results have been reported (see [8]).
Differential-algebraic systems depend on a combination of differential equations and algebraic equations. Compared with the differential equation system, the differential-algebraic system, as a mathematical model, is more perfect in practical application, such as economic system, chemical process system, and robot system, all of which belong to differential-algebraic systems. Over the past two decades, it has been evidenced that the differential-algebraic (different from the accepted differential geometry) methods could lead to a clearer comprehension of some control concepts and their interrelationships, such as observability, reversibility, decoupling and controller specification form, and observability, existence, and uniqueness of minimal implementations. In [9], the control method problem of a nonlinear differential-algebraic system is considered. In [10], the stability analysis of nonlinear differential-algebraic systems is solved using tools from classical control theory. In [11], a new type of differential-algebraic neural network with delay (DDANN) is proposed. And, one kind of novel mathematical expression combining differential equation and algebraic equation is designed in [12]. In addition to a large number of differential-algebraic systems directly proposed in various disciplines and engineering practices, the theoretical study of differential-algebraic systems also provides new ideas and methods for the study of certain problems in normal systems. For example, optimal control of normal systems, control of specified outputs, and singular perturbations can all be reduced to problems in the theory of differential-algebraic systems. Therefore, the theoretical study of differential-algebraic systems has a wide range of practical engineering background and profound theoretical value. It is important to note that, however, the analysis of differential-algebraic neural networks (DANNs) is still in its early stages.
Based on the above-mentioned argumentation, the target of this article is to investigate a new type of differential-algebraic complex-valued neural network (DACVNN) with delay (DDACVNN). An essential consequent on the global existence and uniqueness of the solution is first proposed. It also shows that under some simple assumptions, we can obtain the unique solution of the system under the compatible initial-value problem. We then derive the global exponential stability of this system. Studying a complex differential-algebraic system has always been a more challenging subject. Approximately stated, the novelties of this article include as follows:(1)Apply CVNNs to DANN and establish a new type of DDACVNN. In particular, in the discussion in this article, there is no restriction on whether the real and imaginary parts of the activation function are required to be clearly separated.(2)The sufficient conditions to ensure the existence and uniqueness of DDACVNN solutions are expressed, in terms of Picard existence and uniqueness theorem.(3)An innovative approach is offered to discuss the stability of DDACVNN; that is, under the present system, global exponential stability (GES) is equivalent to global exponential self-synchronization (GESS), which we also rigorously prove in Lemma 1. Thus, a theorem of GES of DDACVNN is investigated.
The structure of the article is as follows. Section 2 provides the preliminaries, the model description about a DACVNN model, a necessary assumption, and a useful lemma. The main results are given in Section 3. Two illustrative examples are discussed in Section 4 to show the feasibility of the results. At last, some conclusions are dedicated in Section 5.
2. Preliminaries, Model Description, and Hypothesis
2.1. Notation
In the whole article, signifies the imaginary unit, which is . For complex number , the notation means the module of . and stand, respectively, the set of real numbers and all -dimensional real-valued vectors. , , and represent, respectively, the set of all complex numbers, the set of all -dimensional complex-valued vectors, and the set of all complex-valued matrices. For , let and . For or , , we define . The matrix is called a nonnegative matrix if for all and . We use to denote th row sum of and let to be infinity norm of a matrix with . We use to denote the maximum eigenvalue of a . For , denotes a matrix norm defined by , where shows the conjugate transpose of complex-valued matrix .
2.2. Model Description
This section presents a DACVNN model, introduces some functional conceptions, and concludes with a basic hypothesis and two lemmas.
In the first place, a complex-valued singular neural network with time delay can be considered aswhere matrix may be singular ; is the time delay; stands for the complex-valued neuron state vector; signifies state feedback coefficient matrix, which ; and correspond to, respectively, the complex-valued connection weight matrix and the complex-valued delayed connection weight matrix; and denote complex-valued neuron activation functions; infers an external complex-valued input signal.
More generally, let . Next, system (1) can compile a DDACVNN indicated as follows:where is the time delay; and ; , , are the positive constants; , , , and ; , , , and ; , and ; , and ; and .
We define a Banach space , which consists of all continuous complex-valued functions . denotes with regard to . Towards , we donate , which signifies in . Let for , where .
Furthermore, we proceed to consider the initial conditions of DDACVNN (2) listed as follows:
The initial conditions (3) of DDACVNN (2) are known as compatible; the necessary and sufficient condition iswhere is known as compatible initial value (CIV).
About CVNNs, the foremost difficulty is to look for an opposite activation function. Judging by Liouville’s theorem, CVNNs cannot choose a smooth and bounded function as the activation function, so how to choose a suitable activation function for research also needs to pay attention to the problem. Therefore, in order for the following to proceed smoothly, we need to make the following hypothesis:
Hypothesis 1. , , , and ; ) satisfy the Lipschitz continuity condition in the complex domain, and there exist positive diagonal matrices , , , and such thatandfor any and .
Remark 1. In the discussion of the article, we merely require that the complex-valued activation functions globally Lipschitz.
It is worth noting that, in [13, 14], the activation function is required to be manifested by severing the real and imaginary parts aswhere , , (or ) and (or ) demonstrate, respectively, the real and imaginary parts of . Hypothesis 1 indicates that the model discussed in this article is suitable for both the case where the activation function can separate the real and imaginary parts, and the case where it cannot be clearly separated. For instance, , .
Finally, an important lemma is offered, which will be applied in the following sections.
Lemma 1. Let matrix have nonpositive off-diagonal elements (i.e., ); then, each of the following conditions is equivalent to the statement that is a nonsingular matrix.(1)All the leading principal minors of are positive(2)There are n positive constants such that(3)There are n positive constants such that(4)The diagonal elements of are all positive and there exists a positive vector such that or .
3. Main Results
3.1. The Issue of Existence and Uniqueness of Solutions
The purpose of this subsection is to discuss the existence and uniqueness of the solution with the CIV problem (2) and (3).
First, we discuss the following DACVNN:where the functions and are the continuous and differentiable functions.
We consider the initial conditions of DACVNN (10) as follows:
The initial conditions (11) are considered to be compatible; the necessary and sufficient condition is
Theorem 1. We assume that Hypothesis 1 is satisfied, and is a matrix. Afterward, for every CIV condition (11) and (12), there is a unique solution on interval of DACVNN (10), where .
Proof. First, we define a function such thatfor any and .
In order to achieve the above effect, without loss of generality, we only need to show that, for every and , there exists a , of (13) such thatLet . On the basis of Lemma 1, there exist such thatas is a matrix. Thus, we can getwhere .
We define an operatorwhere .
According to the definition of the operator , we can getregarding whichever . By calculating the infinity norm in (18), we can getwhereThrough (19), we can readily find that is a contraction operator on . Based on the contraction mapping principle, there exists a unique so that , which isLet , thus (14) satisfies. Therefore, there is a function such that (14) satisfies.
According to (14) and Hypothesis 1, one haswhich shows that the function is continuous in .
Furthermore, from (14) and DACVNN (10), we see thatPicard existence and uniqueness theorem ensures existence and uniqueness of the solution of system (23) considered on interval for any initial value . Furthermore, let , and so there exists a unique solution on interval of the DACVNN (10), with the CIV .
Furthermore, we consider the extension of from to .
Theorem 2. We presume that Hypothesis 1 is satisfied, if is a matrix; afterward, for every CIV , DDACVNN (2) has a unique solution on .
Proof. For , DDACVNN (2) with CIV can be restated as DACVNN (10), in whichThrough Theorem 1, we can know that DDACVNN (2), which is DACVNN (10), has a unique solution on . Progressively, we are able to prove that there exists a unique solution on for any of DDACVNN (2).
The existence and uniqueness of the solutions of complex-valued differential-algebraic systems is one of the most basic problems, and it is the premise and basis for discussing the system. Therefore, it is very important and necessary to study the existence and uniqueness of the solution of this system. For these reasons, this article considers the fundamental concepts of the solution of complex-valued differential-algebraic systems via Picard’s theorem.
3.2. Global Exponential Stability
We will consider the globally exponential stability (GES) and globally exponential self-synchronization (GESS) of DDACVNN (2) in this section.
The is an equilibrium point of DDACVNN (2) if and only if
Definition 1. If regarding any , and regarding any solution of DDACVNN (2) corresponding to the CIV , there exists an equilibrium point satisfyingwhere and are constants, then DDACVNN (2) is called GES.
Definition 2. If for any , and for any solutions and of the DDACVNN (2) corresponding to the two CIVs and , there are two constants and satisfyingthen DDACVNN (2) is called GESS.
Remark 2. Generally speaking, exponential stability and exponential self-synchronization are two completely different concepts. Evidently, exponential stability implies exponential self-synchronization but not vice versa. However, under DDACVNN (2), we have the following claim: GESS can deduce GES.
Lemma 2. GESS is equivalent to GES in DDACVNN (2).
Proof. We only need to show that GES can be deduced by GESS. Firstly, let , signify a solution of DDACVNN (2) with CIV .
Suppose DDACVNN (2) is GESS, namely,for any and , where and are constants.
We define an operator with , where so that . By (14), one hasso we can get that is a contraction. Then, through the Banach contraction principle, it can be obtained that has a unique fixed point , which is the equilibrium point of DDACVNN (2).
After giving the above lemma, we next introduce the following two lemmas. The introduction of these two lemmas plays a key role in the testimony of the next theorem.
Lemma 3 (see [15]). Let and satisfyingfor , where , thenfor , where is a constant sufficing and .
Lemma 4 (see [11]). Let . and are, respectively, continuously differentiable and continuous functions, sufficing
For , where there are positive real numbers , and . And and as if , wherewhere is a constant satisfying , , and . In addition, if there exists satisfying
then
for , where , and .
Theorem 3. We assume that hypothesis 1 holds and is a matrix. Letandwhere is a constant sufficing . Then, if , DDACVNN (2) is GES.
Proof. On the basis of Lemma 1, we only need to certify that DDACVNN (2) is able to achieve GESS.
For any two different solutions and of DDACVNN (2), let and . In line with DDACVNN (2), we can obtain thatandLet and , thenandLet and . Thus, one hasandBy (42) and (43), it follows thatAccording to Lemma 1 for the reason that is a matrix since . According to Lemma 2, we can deduce that there exists so thatAccording to Lemma 2, and by (44) and (45), one hasfor , where , and . From (46), one haswhere , and , which signifies that DDACVNN (2) is GESS. In accordance with Lemma 1, DDACVNN (2) is GES.
Remark 3. In this article, we innovatively combine complex values with DANN and do not study complex-valued in real-valued methods (i.e., divide complex values into real and imaginary parts). And the requirements for the activation function are also reduced a lot (only the condition of global Lipschitz needs to be satisfied), so many restrictions on the activation function are eliminated, making the results of this article more effective.
Remark 4. Above, we propose a new paradigm and skill to discuss the stability of DDACVNN (2). It is strictly proved that, in Lemma 1, the DDACVNN (2) is GES if it can achieve GESS, which is able to reduce the argument of exponential stability in DDACVNN (2). Using a differential-algebraic inequality introduced, the GESS problem is studied, and the GES theorem of DDACVNN is given. Finally, in Theorem 1, it is successfully studied whether DDACVNN (2) can achieve GESS and thus achieve GES.
Remark 5. On the basis of this article, we can also discuss the case of time-varying delays. In fact, in the case where the time delays is time-varying delays, the results of this article still hold. We will continue to consider this issue in future research.
Remark 6. In addition, we can also consider nonlinear differential-algebraic systems. Objectively speaking, there are many essential differences between nonlinear differential-algebraic systems and ordinary differential equations and linear differential-algebraic systems. The most fundamental difference is that of nonzero exponents. In general, in an exponential sense, ordinary differential equations can be thought of as zero-exponential systems, a special case of differential-algebraic systems. The differential-algebraic system we mentioned, generally, refers to the nonzero exponential system, and the transition from the zero-exponential system to the nonzero exponential system is a leap, just like the transition from finite-dimensional space to infinite-dimensional space in mathematical analysis. Therefore, it will inevitably lead to qualitative changes in many issues. All the above have led to the very slow development of the research on nonlinear differential-algebraic systems.
4. Examples
At last, two numerical examples are proposed to demonstrate the above achievements in this section. Respectively, the activation function can separate the real and imaginary parts, and the other is that they cannot be clearly separated.
Example 1. We consider DDACVNN (2), taking and . We allow and ; where , for and , and .
We obviously see that Hypothesis 1 holds and and . By calculation, we haveAndwhich is a matrix. On the basis of Theorem 1, we can conclude that for any CIV , DDACVNN (2) has a unique solution.
Moreover, we can continue to calculate the following:andwhere such that ; then, .
In the example, all the conditions of Theorem 2 are fully satisfied, so in the light of Theorem 2, DDACVNN (2) is GES.
Numerical simulations of Example 1 are exhibited in Figure 1.
Example 2. We consider DDACVNN (2), taking and . We allow and ; , for and , and .
We obviously see that hypothesis 1 holds and and . By calculation, we haveAndwhich is a matrix. On the basis of Theorem 1, we can conclude that for any CIV , DDACVNN (2) has a unique solution.
Moreover, we can continue to calculate the following:andwhere such that ; then, , .
In the example, all the conditions of Theorem 2 are fully satisfied, so on the basis of Theorem 2, DDACVNN (2) is GES.
Numerical simulations of Example 2 are exhibited in Figure 2.
Remark 7. In the above, we have obtained that the activation function of the system considered in the article is feasible regardless of whether the real and imaginary parts can be departed, and the above two examples also prove the correctness of the derived conclusions.
5. Conclusion
Looked through the article, we establish a new type of DDACVNN. First, the existence and uniqueness of the DDACVNN solution is addressed. In addition, the GES theorem of DDACVNN is also studied. In particular, we verify that our activation function does not have the restriction that the real and imaginary parts must be separated. Finally, we also give two examples to confirm the validity of the theoretical results.
For the sake of simplicity, we only consider a simpler form of the differential-algebraic system. And studying a more complex differential-algebraic system has always been a more challenging subject. For example, we can conduct a series of studies similar to traditional neural networks on differential-algebraic systems, such as multiple stability, synchronicity, and periodic oscillations. At the same time, we can also study a differential-algebraic system with a high degree of exponential, or a differential-algebraic system with variable delays. In addition, we can also consider nonlinear differential-algebraic systems. And these issues will be considered in the future.
Data Availability
No data were used to support this study.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this article.
Acknowledgments
This work was supported by the Natural Science Foundation of China under grant 61976084, the Natural Science Foundation of Hubei Province of China under grant 2021CFA080, and the Young Top-Notch Talent Cultivation Program of Hubei Province of China.