About this Journal Submit a Manuscript Table of Contents
Abstract and Applied Analysis
Volume 2013 (2013), Article ID 761237, 11 pages
http://dx.doi.org/10.1155/2013/761237
Research Article

Exponential Stability and Numerical Methods of Stochastic Recurrent Neural Networks with Delays

1School of Automation Science and Engineering, South China University of Technology, Guangzhou 510640, China
2Guangdong Electric Power Design Institute, Guangzhou 510663, China

Received 18 April 2013; Accepted 12 July 2013

Academic Editor: Alexander I. Domoshnitsky

Copyright © 2013 Shifang Kuang et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Exponential stability in mean square of stochastic delay recurrent neural networks is investigated in detail. By using Itô’s formula and inequality techniques, the sufficient conditions to guarantee the exponential stability in mean square of an equilibrium are given. Under the conditions which guarantee the stability of the analytical solution, the Euler-Maruyama scheme and the split-step backward Euler scheme are proved to be mean-square stable. At last, an example is given to demonstrate our results.

1. Introduction

It is well known that neural networks have wide range of applications in many fields, such as signal processing, pattern recognition, associative memory, and optimization problems. Stability is one of the main properties of neural networks, which is preconditions in the designs and applications of neural networks. Time delays are unavoidable in neural networks systems, which is frequently the important source of poor performance or instability. Thus, stability analysis of neural networks with various delays has been extensively investigated; see [110].

In real nervous systems, the synaptic transmission is a noisy process brought on by random fluctuations from the release of neurotransmitters and other probabilistic causes [11]. Hence, noise should be taken into consideration in modeling. Recently, some sufficient conditions for exponential stability of stochastic delay neural networks have been presented in [1218]. Similar to stochastic delay differential equations, most of stochastic delay neural networks do not have explicit solutions. Most of existing researches related to the stability analysis of equilibrium point were focused on the appropriate Lyapunov function or functional. However, there is no very effective method to find such Lyapunov function or functional. Thus it is very useful to establish numerical methods for studying the properties of stochastic delay neural networks. There are many papers concerned with the stability of numerical solutions for stochastic delay differential equations ([1929] and references therein). But there has been a few literatures about the exponential stability of numerical methods for stochastic delay neural networks. To the best of the authors knowledge, only [3032] studied the exponential stability of numerical methods for stochastic delay Hopfield neural networks. The stability of numerical methods for stochastic delay recurrent neural networks remains open, which motivates this paper. The main aim of the paper is to investigate the mean-square stability (MS stability) of the Euler-Maruyama (EM) method and the split-step backward Euler (SSBE) method for stochastic delay recurrent neural networks.

The remainder of the paper is comprised of four sections. Some notations and the conditions of stability to the analytical solution are given in Section 2. The MS stability of the EM method and the SSBE method is proved in Sections 3 and 4, respectively. In Section 5, an example is provided to illustrate the effectiveness of our theory.

2. Model Description and Analysis of Analytical Solution

Throughout the paper, unless otherwise specified, we will employ the following notations. Let be a complete probability space with a filtration satisfying the usual conditions (i.e., it is increasing and is right continuous, while contains all -null set) and the expectation operator with respect to the probability measure. Let denote the Euclidean norm of a vector or the spectral norm of a matrix. Let and denote the family of continuous functions from to with the norm . Denote by the family of all bounded -measurable valued random variables. We assume to be a standard Brownian motion defined on the probability space.

Consider the stochastic delay recurrent neural networks of the form

Model (1) can be rewritten in the following matrix-vector form: where is the state vector associated with the neurons; with represents the rate with which neuron will reset its potential to the resting state in isolation when disconnected from the network and the external stochastic perturbation; and denote the connection weight matrix and the delayed connection weight matrix, respectively; and are activation functions, , , , , where is the transmission delay; , . Moreover, is an -dimensional Brown motion defined on the complete probability space , and , , is the diffusion coefficient matrix.

To obtain our results, we impose the following standing hypotheses.(H1), and .(H2) Both and satisfy the Lipschitz condition. That is, for each , there exist constants , such that (H3) satisfies the Lipschitz condition, and there are nonnegative constants such that

It follows from [33] that under the assumptions (H1)–(H3), system (1) or (2) has a unique strong solution , and is a measurable, sample continuous and -adapted process. Clear, (2) admits the trivial solution .

Definition 1. The trivial solution of system (1) or system (2) is said to be exponentially stable in mean square if there exists a pair of positive constants and such that holds for any . In this case

Using Itô’s formula and nonnegative semimartingale convergence theorem, [12, 14] discussed the exponential stability of stochastic delayed neural network. Employing the method of variation parameter and inequality techniques, several sufficient conditions ensuring th moment exponential stability of stochastic delayed recurrent neural networks are derived in [17]. With the help of the Lyapunov function and Halanay-type inequality, a set of novel sufficient conditions on mean-square exponential stability of stochastic recurrent neural networks with time-varying delays was established in [18]. In this paper, we will give a new sufficient condition to guarantee exponential stability in mean square of stochastic delayed recurrent neural networks (1) by using Itô’s formula and inequality techniques.

Theorem 2. If (1) satisfies (H1)–(H3), and the following holds.(H4) For ,
Then (1) is exponentially stable in mean square.

Proof. By (H4), there exists a sufficiently small positive constant such that
Set ; applying Itô’s formula to along (2), we obtain where
Notice that
Therefore, we have
Notice that , so we can obtain from the previous inequality which implies

Corollary 3. If (1) satisfies (H1)–(H3), the following holds.(H5) For ,
Then (1) is exponentially stable in mean square.

Proof. The are nonnegative constants, so we have condition (H4) from (H5). Therefore we can directly derive Corollary 3 by Theorem 2.

3. Stability of EM Numerical Solution

Let and denote the increments of the time and Brownian motion, respectively. For system (1), the discrete EM approximate solution is defined by where is a stepsize which satisfies for a positive integer , and is an approximation to ; if , we have . We assume that is -measurable at the mesh points .

Suppose that the following condition is satisfied:(H6).

Definition 4. A numerical method is said to be mean-square stable (MS stable), if there exists an , such that any application of the method to (1) generates numerical approximations , which satisfy for all with .

Now we analyze the stability of EM numerical solution.

Theorem 5. Under conditions (H1)–(H3) and (H5)-(H6), the Euler method applied to (1) is MS stable with and , where

Proof. From (16), we have
Squaring both sides of the previous equality, we obtain
Noting that , ,, and , and , where , are -measurable; hence
Let . Applying the inequalities and conditions (H2) and (H3), we obtain from (21) and (20)
Thus where
Then
By the recursion we conclude that if which is equivalent to
If , (27) reduces to
By conditions (H5) and (H6), we know that . Thus, (28) holds for . Let ; then . That is, the EM method of (1) is MS stable. This proof is completed.

Theorem 6. For ,  and , there exists a positive constant such that where depends on , and so on but not upon . is defined in (16).

The proof is similar to Theorem  7 in [20].

4. Stability of SSBE Numerical Solution

In this section, we will construct the SSBE scheme to (1) and analyze the stability of the numerical solution. The adaptation of SSBE method to (1) leads to a numerical process of the following type:

The notations are same to the definition in (16). Now we present another main results of this paper.

Theorem 7. Assume that (H1)–(H3) and (H5)-(H6) hold. Define
Then the SSBE method applied to (1) is MS stable with and , where .

Proof. From (30), we have
Squaring both sides of (32), we obtain
It follows from inequality and (H2) that
Letting , we have
On the other hand, from (30), we obtain
Noting that , , . From (36) and (H3), we have
Substituting (35) into (37), we obtain where
Then
By the recursion we conclude that if which is equivalent to , where
Since , by (H3), (H5), and (H6), we have . This implies that
Thus, (41) holds for . Let ; then . That is, the SSBE method of (1) is MS stable. The proof of the theorem is completed.

Theorem 8. For ,  and , there exists a positive constant such that where depends on , and so on but not upon . is defined in (30).

The proof is similar to Theorem  3.2 in [26].

5. Example

In this section, we will discuss an example to illustrate our theory and compare the restrictions on stepsize of the stable SSBE method with that of the EM method.

Example 1. Let be a two-dimensional Brown motion. Consider the following stochastic delay recurrent neural networks:
Let , ,
It is obvious that , , , and . So (H1)–(H3) are satisfied. By computation, Therefore conditions (H4)–(H6) also hold. By Theorem 2, system (45) is exponentially stable in mean square. The EM scheme and the SSBE scheme to (45) are also MS stable by Theorems 5 and 7.
Now, we can conclude that the EM method and the SSBE method to (45) are MS stable with from Figure 1. It verifies the validity of Theorems 5 and 7. The EM method is not stable, and the SSBE method is MS stable with from Figure 2, which shows that the stability of the SSBE method is more superior to EM. Figure 3 illustrates that the SSBE method is unstable with .

fig1
Figure 1: MS stability of the numerical solutions to (45) with ; (a) EM, (b) SSBE.
fig2
Figure 2: Instability of EM numerical solutions and MS stability of SSBE numerical solutions to (45) with ; (a) EM, (b) SSBE.
761237.fig.003
Figure 3: Instability of SSBE numerical solutions of system (45) with .

6. Conclusions

The model of stochastic neural network can be viewed as a special kind of stochastic differential equation; the solution is hard to be explicitly expressed. It not only has the characteristics of the general stochastic differential equations but also has its own features; its stability is connected with the activation functions and the connection weight matrixes. So it is necessary to discuss the stability of stochastic neural network. Different from the previous works on exponential stability of stochastic neural networks, both Lyapunov function method and two numerical methods are used to study the stability of stochastic delay recurrent neural networks. Under the conditions which guarantee the stability of the analytical solution, the EM method and the SSBE method are proved to be MS stable if the step size meets a certain limit. We can analyze other numerical methods for different types of stochastic delay neural networks in future.

Acknowledgments

This work was supported by the National Natural Science Foundation of China (nos. 60904032, 61273126), the Natural Science Foundation of Guangdong Province (no. 10251064101000008), and the Fundamental Research Funds for the Central Universities (no. 2012ZM0059).

References

  1. C. M. Marcus and R. M. Westervelt, “Stability of analog neural networks with delay,” Physical Review A, vol. 39, no. 1, pp. 347–359, 1989. View at Publisher · View at Google Scholar · View at MathSciNet
  2. J. Wu, Introduction to Neural Dynamics and Signal Transmission Delay, vol. 6, Walter de Gruyter, Berlin, Germany, 2001. View at Publisher · View at Google Scholar · View at MathSciNet
  3. X. Li, L. Huang, and J. Wu, “Further results on the stability of delayed cellular neural networks,” IEEE Transactions on Circuits and Systems. I. Fundamental Theory and Applications, vol. 50, no. 9, pp. 1239–1242, 2003. View at Publisher · View at Google Scholar · View at MathSciNet
  4. X. M. Li, L. H. Huang, and H. Zhu, “Global stability of cellular neural networks with constant and variable delays,” Nonlinear Analysis. Theory, Methods & Applications, vol. 53, no. 3-4, pp. 319–333, 2003. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet
  5. S. Arik, “Stability analysis of delayed neural networks,” IEEE Transactions on Circuits and Systems. I. Fundamental Theory and Applications, vol. 47, no. 7, pp. 1089–1092, 2000. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet
  6. S. Arik, “Global asymptotic stability analysis of bidirectional associative memory neural networks with time delays,” IEEE Transactions on Neural Networks, vol. 16, no. 3, pp. 580–586, 2005.
  7. T. Liao and F. Wang, “Global stability for cellular neural networks with time delay,” IEEE Transactions on Neural Networks, vol. 11, no. 6, pp. 1481–1484, 2000.
  8. X. Liao, Q. Liu, and W. Zhang, “Delay-dependent asymptotic stability for neural networks with distributed delays,” Nonlinear Analysis. Real World Applications, vol. 7, no. 5, pp. 1178–1192, 2006. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet
  9. S. Kuang, F. Deng, and X. Li, “Stability and hopf bifurcation of a BAM neural network with delayed self-feedback,” in Proceedings of the 7th International Symposium on Neural Networks, vol. 6063 of Lecture Notes in Computer Science, pp. 493–503, 2010.
  10. O. Faydasicok and S. Arik, “Further analysis of global robust stability of neural networks with multiple time delays,” Journal of the Franklin Institute. Engineering and Applied Mathematics, vol. 349, no. 3, pp. 813–825, 2012. View at Publisher · View at Google Scholar · View at MathSciNet
  11. S. Haykin, Neural Networks, Prentice-Hall, NJ, USA, 1994.
  12. S. Blythe, X. Mao, and X. Liao, “Stability of stochastic delay neural networks,” Journal of the Franklin Institute, vol. 338, no. 4, pp. 481–495, 2001. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet
  13. L. Wan and J. Sun, “Mean square exponential stability of stochastic delayed Hopfield neural networks,” Physics Letters A, vol. 343, no. 4, pp. 306–318, 2005.
  14. Q. Zhou and L. Wan, “Exponential stability of stochastic delayed Hopfield neural networks,” Applied Mathematics and Computation, vol. 199, no. 1, pp. 84–89, 2008. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet
  15. Y. Zhang, D. Yue, and E. Tian, “Robust delay-distribution-dependent stability of discrete-time stochastic neural neural networks with time-varying delay,” Neurocomputing, vol. 72, no. 4, pp. 1265–1273, 2009.
  16. Z. Wang, Y. Liu, M. Li, and X. Liu, “Stability analysis for stochastic Cohen-Grossberg neural networks with mixed time delay,” IEEE Transactions on Neural Networks, vol. 17, no. 3, pp. 814–820, 2006.
  17. Y. Sun and J. Cao, “pth moment exponential stability of stochastic recurrent neural networks with time-varying delays,” Nonlinear Analysis. Real World Applications, vol. 8, no. 4, pp. 1171–1185, 2007. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet
  18. C. Huang, Y. He, and H. Wang, “Mean square exponential stability of stochastic recurrent neural networks with time-varying delays,” Computers & Mathematics with Applications, vol. 56, no. 7, pp. 1773–1778, 2008. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet
  19. U. Küchler and E. Platen, “Strong discrete time approximation of stochastic differential equations with time delay,” Mathematics and Computers in Simulation, vol. 54, no. 1–3, pp. 189–250, 2000. View at Publisher · View at Google Scholar · View at MathSciNet
  20. E. Buckwar, “Introduction to the numerical analysis of stochastic delay differential equations,” Journal of Computational and Applied Mathematics, vol. 125, no. 1-2, pp. 297–307, 2000. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet
  21. D. J. Higham and P. E. Kloeden, “Convergence and stability of implicit methods for jump-diffusion systems,” International Journal of Numerical Analysis and Modeling, vol. 3, no. 2, pp. 125–140, 2006. View at Zentralblatt MATH · View at MathSciNet
  22. X. Mao and S. Sabanis, “Numerical solutions of stochastic differential delay equations under local Lipschitz condition,” Journal of Computational and Applied Mathematics, vol. 151, no. 1, pp. 215–227, 2003. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet
  23. M. Liu, W. Cao, and Z. Fan, “Convergence and stability of the semi-implicit Euler method for a linear stochastic differential delay equation,” Journal of Computational and Applied Mathematics, vol. 170, no. 2, pp. 255–268, 2004. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet
  24. P. Hu and C. Huang, “Stability of stochastic θ-methods for stochastic delay integro-differential equations,” International Journal of Computer Mathematics, vol. 88, no. 7, pp. 1417–1429, 2011. View at Publisher · View at Google Scholar · View at MathSciNet
  25. A. Rathinasamy and K. Balachandran, “T-stability of the split-step θ-methods for linear stochastic delay integro-differential equations,” Nonlinear Analysis. Hybrid Systems, vol. 5, no. 4, pp. 639–646, 2011. View at Publisher · View at Google Scholar · View at MathSciNet
  26. H. Zhang, S. Gan, and L. Hu, “The split-step backward Euler method for linear stochastic delay differential equations,” Journal of Computational and Applied Mathematics, vol. 225, no. 2, pp. 558–568, 2009. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet
  27. M. Song and H. Yu, “Numerical solutions of stochastic differential delay equations with Poisson random measure under the generalized Khasminskii-type conditions,” Abstract and Applied Analysis, vol. 2012, Article ID 127397, 24 pages, 2012. View at Zentralblatt MATH · View at MathSciNet
  28. Q. Li and S. Gan, “Stability of analytical and numerical solutions for nonlinear stochastic delay differential equations with jumps,” Abstract and Applied Analysis, vol. 2012, Article ID 831082, 13 pages, 2012. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet
  29. X. Ding, K. Wu, and M. Liu, “Convergence and stability of the semi-implicit Euler method for linear stochastic delay integro-differential equations,” International Journal of Computer Mathematics, vol. 83, no. 10, pp. 753–761, 2006. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet
  30. R. Li, W. Pang, and P. Leung, “Exponential stability of numerical solutions to stochastic delay Hopfield neural networks,” Neurocomputing, vol. 73, no. 4–6, pp. 920–926, 2010.
  31. F. Jiang and Y. Shen, “Stability in the numerical simulation of stochastic delayed Hopfield neural networks,” Neural Coputing and Applications, vol. 22, no. 7-8, pp. 1493–1498, 2013.
  32. A. Rathinasamy, “The split-step θ-methods for stochastic delay Hopfield neural networks,” Applied Mathematical Modelling, vol. 36, no. 8, pp. 3477–3485, 2012. View at Publisher · View at Google Scholar · View at MathSciNet
  33. X. Mao, Stochastic Differential Equations and Applications, Horwood, Chichester, UK, 2nd edition, 2008. View at MathSciNet