Research Article | Open Access
Xinsong Yang, Jinde Cao, "Synchronization of Discontinuous Neural Networks with Delays via Adaptive Control", Discrete Dynamics in Nature and Society, vol. 2013, Article ID 147164, 9 pages, 2013. https://doi.org/10.1155/2013/147164
Synchronization of Discontinuous Neural Networks with Delays via Adaptive Control
The drive-response synchronization of delayed neural networks with discontinuous activation functions is investigated via adaptive control. The synchronization of this paper means that the synchronization error approaches to zero for almost all time as time goes to infinity. The discontinuous activation functions are assumed to be monotone increasing which can be unbounded. Due to the mild condition on the discontinuous activations, adaptive control technique is utilized to control the response system. Under the framework of Filippov solution, by using Lyapunov function and chain rule of differential inclusion, rigorous proofs are given to show that adaptive control can realize complete synchronization of the considered model. The results of this paper are also applicable to continuous neural networks, since continuous function is a special case of discontinuous function. Numerical simulations verify the effectiveness of the theoretical results. Moreover, when there are parameter mismatches between drive and response neural networks with discontinuous activations, numerical example is also presented to demonstrate the complete synchronization by using discontinuous adaptive control.
In the last decades, synchronization of coupled chaotic systems (including fractional-order chaotic systems and integer-order chaotic systems [1–3]) has received increasing research attention from different branches of science and application fields due to its potential applications such as secure communication, biological systems, and information science [4, 5]. Along with the presentation of different kinds of synchronization, such as complete synchronization , lag synchronization [7, 8], quasi-synchronization [9, 10], projective synchronization [11–13], and generalized synchronization [14, 15], many control methods have been developed, for instance, state feedback control [9, 16], and adaptive control [7, 15, 17]. The adaptive control technique derives special attention since its control gains need not to be known in advance and can self-adjust according to the designed adaptive law.
Delayed neural networks, as a class of important functional differential equations, have witnessed many applications in different areas such as signal processing, associative memories, classification of patterns, and optimization. Therefore, investigating dynamical behaviors of neural networks with various parameters has long been an intensive research topic, such as stability of equilibrium point  and chaos synchronization . However, the activation functions in most of known models including those in [7, 18] are accompanied by the assumption of continuity or even Lipschitz continuity. Actually, neural networks with discontinuous neuron activations are ideal models for the case where the gain of the neuron amplifiers is very high and is frequently arising in the applications [19, 20]. Therefore, in the literature, there were some results on dynamical behaviors of neural networks with discontinuous activation functions. For instance, Forti et al. investigated the stability and global convergence of delayed and nondelayed neural networks with discontinuous activations in [19, 20]; authors in [21–23] studied the global robust stability of delayed neural networks with discontinuous neuron activations; in [24, 25], authors considered existence and global convergence of periodic and almost periodic solutions of neural networks with discontinuous activations.
On the other hand, although there were many results concerning synchronization of chaotic neural networks, few published papers considered the same issue for neural networks with discontinuous activations, and we only found [10, 26]. The difficulty comes from the discontinuity of activations. The methods utilized to analyze the stability of neural networks with discontinuous activations cannot be extended to chaos synchronization case directly. In , authors investigated quasi-synchronization of discontinuous neural networks with and without parameter mismatches, that is, synchronization error can only be controlled to a small region around zero, but cannot approach to zero. Results of  revealed that complete synchronization is difficult to be realized due to the discontinuity of activation function. It is known that one of the most important applications of chaos synchronization is in secure communication. When chaos synchronization is applied to secure communication, only when the drive and response systems achieve complete synchronization can the transmitted signal be fingered out. Therefore, it is necessary to study complete synchronization of neural networks with discontinuous activations. In , complete synchronization of discontinuous neural networks was investigated via approximation and linear matrix inequality (LMI) approach. But we find that, through the approximation approach used in , the control gain is uncertain and may be very large, which leads to inapplicablility in practice. On the other hand, some results on synchronization and control of discontinuous dynamical systems are complex, which are difficult to be verified. For instance, synchronization criteria obtained in  were in terms of integral inequality, and the restrict condition on the discontinuity of discontinuous function was weakened, that is, as time goes to infinity, the discontinuous function approaches to a continuous function. From the above analysis, investigating the synchronization of neural networks with discontinuous activations is really a tough task.
Being motivated by the above analysis, this paper investigates asymptotic complete synchronization of neural networks with discontinuous activation functions via adaptive control technique. Because of the discontinuity of the activation functions, the solution is in the sense of differential inclusion by the Filippov theory , and the complete synchronization of this paper means that the state error between the derive (or master) and response systems approaches to zero for almost all (a.a.) time as time goes to infinity. We do not impose the restriction conditions of growth condition on activation function. The discontinuous activations are only assumed to be monotone increasing and can be unbounded. Due to the mild condition on the discontinuous activations, the precise control gain is difficult to be determined, and the state feedback control is not so good as the adaptive control technique. Under the framework of Filippov solution, by using Lyapunov function and chain rule of differential inclusion, rigorous proofs are given for the asymptotic stability of the error system of the coupled systems. Numerical simulations show the effectiveness of the theoretical results. Moreover, when there are parameter mismatches between driver and response neural networks with discontinuous activations, numerical example is presented to demonstrate the complete synchronization by using discontinuous adaptive control.
Notations. In the sequel, if not explicitly stated, matrices are assumed to have compatible dimensions. stands for the identity matrix of -dimension. is the space of real number. The Euclidean norm in is denoted as , accordingly, for vector , , where denotes transposition. represents each component of is zero. denotes a matrix of -dimension, .
The rest of this paper is organized as follows. In Section 2, a model of delayed neural networks with discontinuous activation functions is described. Some necessary assumptions, definitions, and lemmas are also given in this section. Our main results and their rigorous proofs are described in Section 3. In Section 4, two examples with their numerical simulations are offered to show the effectiveness of our results. In Section 5, conclusions are given, and at last, acknowledgments.
In this paper, we consider the delayed neural network which is described as follows: where is the state vector; , in which , are the neuron self-inhibitions; is the transmission delay; and are the connection weight matrix and the delayed connection weight matrix, respectively; the activation function represents the output of the network; is the external input vector.
As for neural networks (1), we give the following assumption condition.
Assumption 1. For every , is monotone nondecreasing and has at most a finite number of jump discontinuities in every compact interval.
Remark 2. Assumption 1 was used in . Any function satisfying this assumption condition does not need to be continuously differentiable in compact interval. However, the continuous differentiability is necessary in [10, 26]. On the other hand, the “monotone nondecreasing" can be replaced by “monotonic function.” For instance, if we replace the matrices and in Example 13 (see Section 4 of this paper) with is the same function as that in example and Obviously, the is monotone increasing, and the is monotone decreasing, but the obtained system and the original system are identical. Furthermore, the results of this paper are also applicable to neural networks with continuous monotone activation functions, since continuous function is a special case of discontinuous function.
Since is discontinuous at isolate jumping points, one cannot define a solution in the conventional sense. Therefore, we resort to the notion of Filippov solution and stability results on differential inclusion . Filippov solution is one of notions to deal with the discontinuity that determine the solution on the discontinuous surface with a set-valued mapping.
The Filippov set-valued map of at is defined as follows : where is the closure of the convex hull of the set , , and is the Lebesgue measure of set .
Definition 3 (see ). A function , , is a solution (in the sense of Filippov) of the discontinuous system (1) on if (i)is continuous on and absolutely continuous on ; (ii)there exists a measurable function , such that for almost all (a.a.) and
Note that the measurable function is a single-value function which is called the measurable selection of . Any function satisfying (6) is called an output associated to . In this paper, we assume that the trajectory of the solution of neural network (1) is chaotic.
The next definition is the initial value problem (IVP) associated to system (1).
Definition 4 ((IVP), see ). For any continuous function and measurable selection such that for a.a. by an initial value problem associated to (1) with initial condition , one means the following problem: find a couple of functions , such that is a solution of (1) on for some , is an output associated to , and
Lemma 5 (see ). Suppose that Assumption 1 is satisfied. Then, any IVP for (1) has at least a local solution defined on for some .
Since chaotic system has strange attractors, there exists a bounded region containing all attractors of it such that every orbit of the system never leaves them. Hence, in view of Lemma 5, the solution of (1) is defined on .
Lemma 6 ((Chain rule), see ). If is C-regular, and is absolutely continuous on any compact subinterval of . Then, and are differentiable for a.a. and where is the Clark generalized gradient of at .
Lemma 7 (see [30, page 174]). Let , . Assume , is a measurable function on . If is monotone on , then is differentiable for a.a. and (i), for ;(ii), where , .
Consider the neural network model (1) as the driver system, the controlled response system is where is the state of the response system, is the controller to be designed, and the other parameters are the same as those defined in system (1).
3. Main Results
In this section, rigorous mathematical proofs about complete synchronization between systems (9) and (1) under adaptive control are given. Remarks are given to specify that, under Assumption 1, state feedback control is not applicable.
Obviously, is the equilibrium point of the error system (13) when . If system (13) realizes global asymptotical stability at the origin for any given initial condition, then the global asymptotical synchronization between (11) and (12) (or (1) and (9)) is achieved.
Theorem 9. Suppose that Assumption 1 is satisfied. Then the neural networks (1) and (9) can achieve global asymptotical synchronization under the following adaptive controller: where is an arbitrary positive constant.
Proof. Since satisfies Assumption 1, is a single-valued measurable function satisfying . Therefore are monotone increasing and measurable functions on . In view of Lemma 7, is differentiable for a.a and there exist positive constants such that , . Consequently, for a.a. , there holds
Define the following Lyapunov functional candidate: where and are positive constants to be determined.
Then, for a.a. , computing the derivative of along trajectories of error system (13), we get from Lemma 6 and the calculus for differential inclusion in  that where , .
It follows from (15) and (17) that Take . Then one derives from (18) that
Take . Then Therefore, for a.a. we have According to Definition 8, the neural networks (1) and (9) achieve global asymptotical synchronization. Moreover, from (16), , , approach to some constants as . This completes the proof.
Remark 10. Although for a.a. there exist positive constants such that , , these are usually unknown because the function is uncertain. Hence, in this paper, for neural networks with discontinuous activations, using state feedback control to synchronize (1) and (8) is not good, since the maximum value of control gain cannot be ascertained. However, adaptive control technique can synchronize this class of neural networks as the control gains increase according to the adaptive laws. This is the main reason why we choose adaptive control method to study the synchronization issue of the considered model.
Remark 11. Under Assumption 1, complete synchronization of neural networks with discontinuous activation functions can be achieved in this paper. However, based on the growth condition used in [32, 33], authors in  only got the quasi-synchronization criteria of systems (1) and (9) by state feedback control. Therefore, results of this paper improve corresponding parts of those in .
Remark 12. The synchronization criteria in this paper are simple and can be easily verified in practice. In , new conditions on synchronization of linearly coupled dynamical networks with non-Lipschitz right-hand sides were derived, but the discontinuous functions were weakened to be weak-QUAD and semi-QUAD, which means that the discontinuous function approaches to a continuous function, and the criteria were expressed in integral inequalities. Such synchronization criteria may be not easily verified in practice, especially in the case that there are countable discontinuities for the discontinuous functions. Hence, results of this paper improve those in .
4. Numerical Examples
In this section, we provide two examples to show that our theoretical results obtained above are effective. Example also show that, when the discontinuous neural networks have parameters mismatches, synchronization is still realized under the discontinuous adaptive control developed in our previous works.
Example 13. Consider the delayed neural network model (1) with the following parameters: , , , is identity matrix of 2-dimension, and
the activation function is with
Figure 1 shows chaotic-like trajectory of (1) with initial condition , .
Obviously, in this example is monotone increasing and is discontinuous at , so the activation function satisfies Assumption 1. It follows from Theorem 9 that system (9) can synchronize the driver system (1) under the adaptive controller (14).
In the numerical simulations, we use the forward Euler method, which was used in  to obtain numerical solution of differential inclusions. The parameters in the simulations are taken as step-length is 0.01, , , for all , . we get the simulation results shown in Figures 2 and 3. Figure 2 describes the trajectories of the error states. Figure 3 represents the time response of . Figures 2 and 3 show that synchronization error approaches to zero quickly as time goes and the control gain turns out to be some constants when the synchronization has been realized. Numerical simulations verify the theoretical results perfectly.
Remark 14. In Example 13, the activation function is not generalized differentiable at , nor is global Lipschitz. However, these two conditions are necessary in . Hence, this example demonstrates that results of this paper improve corresponding results in .
When there are parameter mismatches between drive and response systems, only quasi-synchronization is realized if state feedback control technique or continuous adaptive control technique is utilized. In our previous research, a simple but all-powerful discontinuous adaptive control was designed to synchronize chaotic systems with uncertain perturbations. The following example is given to demonstrate that the discontinuous adaptive control is also applicable to synchronize neural networks with discontinuous activations. For more details of the discontinuous adaptive controller, see [6, 7, 15]. The models used in the following example are taken from example in .
Example 15. Consider the delayed neural network model (1) with the activation function as
the other parameters are the same as those in Example 13. We label system (1) with such activation as . The chaotic-like trajectory of system can be seen in Figure 4.
The response system with parameter mismatches is assumed to be the same as that in , which is described as follows: where
The chaotic-like trajectory of system (25) is shown in Figure 5, which is different from that in Figure 4.
According to the analysis in [7, 15], the systems and (25) can realize synchronization under the following discontinuous adaptive controller: where , and are arbitrary positive constants.
In the numerical simulations, we still use the Euler method. The parameters in the simulations are taken as step-length is 0.001, , , , for all , , , . We get the simulation results shown in Figures 6–8. Figure 6 describes the trajectories of the error states as time involves. Figures 7 and 8 represent the time response of and . Simulations demonstrate that the neural networks with discontinuous activations and parameters mismatches achieve synchronization by utilizing the discontinuous adaptive control technique.
Remark 16. It can be seen from Figures 7 and 8 that the final control gains are and , which are much smaller than . In , was utilized to control system (25), and only quasi-synchronization was achieved. This example demonstrates that the designed discontinuous adaptive controller is really useful. Since in our previous works such adaptive controller has been discussed in details, we use it here without any proof.
In this paper, new definition of synchronization for discontinuous dynamical systems is proposed. Under this definition, synchronization of delayed neural networks with discontinuous activation functions via adaptive control is studied. The discontinuous activations in the neural networks are assumed to be monotone increasing and can be unbounded. By utilizing the framework of Filippov solution, Lyapunov function and chain rule of differential inclusion, sufficient conditions guaranteeing the realization of asymptotic complete synchronization of the considered model are derived. Numerical simulations verify the effectiveness of the theoretical results. When there are parameter mismatches between the drive and response neural networks with discontinuous activations, a useful discontinuous adaptive controller can achieve the same goal. Results of this paper are also applicable to neural networks with continuous monotone activation functions.
This work was jointly supported by the National Natural Science Foundation of China (NSFC) under Grants nos. 61263020, 61272530, 11072059, the Natural Science Foundation of Jiangsu Province of China under Grants no. BK2012741, and the Scientific Research Fund of Yunnan Province under Grant no. 2010ZC150, the Scientific Research Fund of Chongqing Normal University under Grants no. 12XLB031 and no. 940115.
- P. Zhou and Y. X. Cao, “Function projective synchronization between fractional-order chaotic systems and integer-order chaotic systems,” Chinese Physics B, vol. 19, no. 10, Article ID 100507, 2010.
- L. Pan and J. Cao, “Exponential synchronization for impulsive dynamical networks,” Discrete Dynamics in Nature and Society, vol. 2012, Article ID 232794, 20 pages, 2012.
- P. Zhou, R. Ding, and Y. Cao, “Multi drive-one response synchronization for fractional-order chaotic systems,” Nonlinear Dynamics, vol. 70, no. 2, pp. 1263–1271, 2012.
- S. Sundar and A. A. Minai, “Synchronization of randomly multiplexed chaotic systems with application to communication,” Physical Review Letters, vol. 85, no. 25, pp. 5456–5459, 2000.
- S. Bowong, F. M. Moukam Kakmeni, and H. Fotsin, “A new adaptive observer-based synchronization scheme for private communication,” Physics Letters, Section A, vol. 355, no. 3, pp. 193–201, 2006.
- X. Yang, J. Cao, and J. Lu, “Stochastic synchronization of complex networks with nonidentical nodes via hybrid adaptive and impulsive control,” IEEE Transactions on Circuits and Systems I, vol. 59, no. 2, pp. 371–384, 2012.
- X. Yang, J. Cao, Y. Long, and W. Rui, “Adaptive lag synchronization for competitive neural networks with mixed delays and uncertain hybrid perturbations,” IEEE Transactions on Neural Networks, vol. 21, no. 10, pp. 1656–1667, 2010.
- E. M. Shahverdiev, S. Sivaprakasam, and K. A. Shore, “Lag synchronization in time-delayed systems,” Physics Letters, Section A, vol. 292, no. 6, pp. 320–324, 2002.
- T. Huang, C. Li, W. Yu, and G. Chen, “Synchronization of delayed chaotic systems with parameter mismatches by using intermittent linear state feedback,” Nonlinearity, vol. 22, no. 3, pp. 569–584, 2009.
- X. Liu, T. Chen, J. Cao, and W. Lu, “Dissipativity and quasi-synchronization for neural networks with discontinuous activations and parameters,” Neural Networks, vol. 24, pp. 1013–1021, 2011.
- P. Zhou and W. Zhu, “Function projective synchronization for fractional-order chaotic systems,” Nonlinear Analysis: Real World Applications, vol. 12, no. 2, pp. 811–816, 2011.
- X. Wu, C. Xu, J. Feng, Y. Zhao, and X. Zhou, “Generalized projective synchronization between two different neural networks with mixed time delays,” Discrete Dynamics in Nature and Society, vol. 2012, Article ID 153542, 19 pages, 2012.
- P. Zhou, R. Ding, and Y. Cao, “Hybrid projective synchronization for two identical fractional-order chaotic systems,” Discrete Dynamics in Nature and Society, vol. 2012, Article ID 768587, 11 pages, 2012.
- N. F. Rulkov, M. M. Sushchik, L. S. Tsimring, and H. D. I. Abarbanel, “Generalized synchronization of chaos in directionally coupled chaotic systems,” Physical Review E, vol. 51, no. 2, pp. 980–994, 1995.
- X. Yang, Q. Zhu, and C. Huang, “Generalized lag-synchronization of chaotic mix-delayed systems with uncertain parameters and unknown perturbations,” Nonlinear Analysis: Real World Applications, vol. 12, no. 1, pp. 93–105, 2011.
- B. Xin and T. Chen, “Projective synchronization of -dimensional chaotic fractional-order systems via linear state error feedback control,” Discrete Dynamics in Nature and Society, Article ID 191063, 10 pages, 2012.
- X. Yang and J. Cao, “Adaptive pinning synchronization of complex networks with stochastic perturbations,” Discrete Dynamics in Nature and Society, vol. 2010, Article ID 416182, 21 pages, 2010.
- H. Hu, “On stability of nonlinear continuous-time neural networks with delay,” IEEE Transactions on Neural Networks, vol. 13, pp. 1135–1143, 2000.
- M. Forti and P. Nistri, “Global convergence of neural networks with discontinuous neuron activations,” IEEE Transactions on Circuits and Systems I, vol. 50, no. 11, pp. 1421–1435, 2003.
- M. Forti, P. Nistri, and D. Papini, “Global exponential stability and global convergence in finite time of delayed neural networks with infinite gain,” IEEE Transactions on Neural Networks, vol. 16, no. 6, pp. 1449–1463, 2005.
- Z. Guo and L. Huang, “LMI conditions for global robust stability of delayed neural networks with discontinuous neuron activations,” Applied Mathematics and Computation, vol. 215, no. 3, pp. 889–900, 2009.
- Y. Wang, Y. Zuo, L. Huang, and C. Li, “Global robust stability of delayed neural networks with discontinuous activation functions,” IET Control Theory & Applications, vol. 2, no. 7, pp. 543–553, 2008.
- Y. Zuo, Y. Wang, L. Huang, Z. Wang, X. Liu, and X. Wu, “Robust stability criterion for delayed neural networks with discontinuous activation functions,” Neural Processing Letters, vol. 29, no. 1, pp. 29–44, 2009.
- X. Liu and J. Cao, “On periodic solutions of neural networks via differential inclusions,” Neural Networks, vol. 22, no. 4, pp. 329–334, 2009.
- W. Lu and T. Chen, “Almost periodic dynamics of a class of delayed neural networks with discontinuous activations,” Neural Computation, vol. 20, no. 4, pp. 1065–1090, 2008.
- X. Liu and J. Cao, “Synchronization control of discontinuous neural networks via approximation,” in Proceedings of the Chinese Control and Decision Conference (CDC '10), pp. 782–787, May 2010.
- B. Liu, W. Lu, and T. Chen, “New conditions on synchronization of networks of linearly coupled dynamical systems with non-Lipschitz right-hand sides,” Neural Networks, vol. 25, pp. 5–13, 2012.
- A. F. Filippov, “Differential equations with discontinuous right-hand sides,” in Mathematics and Its Applications, Soviet Series, Kluwer Academic Publishers, Boston, Mass, USA.
- F. H. Clarke, Optimization and Nonsmooth Analysis, John Wiley & Sons, New York, NY, USA, 1983.
- Z. Jiang and Z. Wu, Real Analysis, Higher Education Publisher, Beijing, China, 2nd edition, 2005.
- B. E. Paden and S. S. Sastry, “A calculus for computing Filippov's differential inclusion with application to the variable structure control of robot manipulators,” IEEE Transactions on Circuits and Systems, vol. 34, no. 1, pp. 73–82, 1987.
- M.-F. Danca, “Synchronization of switch dynamical systems,” International Journal of Bifurcation and Chaos in Applied Sciences and Engineering, vol. 12, no. 8, pp. 1813–1826, 2002.
- L. Huang, J. Wang, and X. Zhou, “Existence and global asymptotic stability of periodic solutions for Hopfield neural networks with discontinuous activations,” Nonlinear Analysis. Real World Applications, vol. 10, no. 3, pp. 1651–1661, 2009.
- M. F. Danca, “Controlling chaos in discontinuous dynamical systems,” Chaos, Solitons and Fractals, vol. 22, no. 3, pp. 605–612, 2004.
Copyright © 2013 Xinsong Yang and Jinde Cao. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.