Abstract

The study of recurrent neural networks with piecewise constant transition or control functions has attracted much attention recently because they can be used to simulate many physical phenomena. A recurrent and discontinuous two-state dynamical system involving a nonnegative bifurcation parameter is studied. By elementary but novel arguments, we are able to give a complete analysis on its asymptotic behavior when the parameter varies from 0 to . It is hoped that our analysis will provide motivation for further results on large-scale recurrent McCulloch-Pitts-type neural networks and piecewise continuous discrete-time dynamical systems.

1. Introduction

It is generally accepted that the McCulloch-Pitts model of a neural network can be used as the components of computer-like systems. That is, where neural networks are commonly used to learn something, a McCulloch-Pitts neuron is constructed to do a particular job. Although the same job can be done by means of traditional Boolean components, it is interesting to see how it all works using components which are closer to “biological” components. One particular important component can be described by means of the step (activation) function defined by with a nonnegative (threshold) real parameter . Roughly, a neuron may receive an inhibitory (indicated by a sign) value if the input signal has strength exceeding the biological threshold and otherwise it remains intact with an excitory (indicated by a sign) value.

Therefore, if we let be the state value of a neural unit during the time period , then the following recurrence relation: may be used to describe a one-neuron McCulloch-Pitts system where the state value is updated from the two most recent state values. There are now many studies that are concerned with such neural networks.

For a neural network system that contains two or more neural units, things are much more complicated, and a variety of models can be designed. In usual practice, we can build a mathematical model and use it to generate simulation results, and from these results, we may understand the general properties of our models. What is now more difficult but important is to make mathematical conclusions that may provide a full understanding of all the properties observed.

In this paper, we consider a two-neuron dynamical neural network system: for , where , and is the function in. Note that if we make the change of variables where is an arbitrary positive number, then (3) becomes the “more general” system: and hence all properties of (5) can be inferred from those of (3).

In this paper, we are concerned with the asymptotic behavior of (3) when the nonnegative parameter is treated as a bifurcation variable and allowed to vary from to (cf. [1, 2]). What is interesting is that a complete asymptotic and bifurcation analysis can be obtained. With such success, it is expected that more general discontinuous recurrent McCulloch-Pitts-type neural networks [3] can be dealt with to some extent in a similar manner.

To simplify matters, note that if we let and then the above system (3) can be written as where we write and for the sake of convenience. The above vector equation is a three-term recurrence relation. Hence for given and in the plane, a unique sequence can be calculated from it. Such a sequence is called a solution of (7) determined by and .

Depending on the locations of and , it is clear that the behavior of the corresponding solution may differ. For this reason, it is convenient to distinguish various parts of . Before doing so, it is convenient to adopt several simplifying conventions. First, we set Next, if and are real intervals, their cross-product will be denoted by instead, and we will assume that this product receives the priority attention in a mathematical expression. By means of these convections, Clearly, is a partition of . Other subsets of the plane will be introduced in the subsequent sections.

For solutions originated from and in the above subsets, we will show that they are all “asymptotically periodic.” More precisely, we say that a positive integer is a period of a scalar or vector sequence if for all and that is the least or prime period of if is the least among all periods of . The sequence is said to be -periodic if is the least period of . The sequence is said to be asymptotically periodic if there exist real numbers , where is a positive integer such that In case is also an -periodic sequence, we say that is an asymptotically -periodic sequence tending to the limit -cycle (this term is introduced since the underlying concept is similar to that of the limit cycle in the theory of ordinary differential equations) . For the sake of convenience, for such an asymptotically -periodic sequence , we write Note that in case is asymptotically -periodic, then it is a sequence convergent to for some . The converse is also true. In such a case, we simply write instead of .

Having these terminologies at hand, our main issue is to show that each solution of (3) originated from is either asymptotically - or -periodic. Note, however, that since is a discontinuous function, the standard theories that employ continuous arguments cannot be applied to yield asymptotic criteria. Fortunately, we may resort to elementary arguments as to be seen below.

Before doing so, let us make a few remarks. First note that our system (7) is autonomous (time invariant) and also symmetric in the sense that under two sets of “symmetric initial conditions,” the behaviors of the corresponding solutions are also “symmetric.” This statement can be made more precise in mathematical terms. However, a simple example is sufficient to illustrate this: if is a solution of (3) with , then as will be seen below, and . If we now replace the condition with the symmetric initial condition , then we will end up with the conclusion that and . Such two conclusions will be referred to as dual results, and the principle of proof for either one can be applied to that of the other.

In the sequel, we will first distinguish three different cases (i) , (ii) , and (iii) and then consider different , in , or and the (asymptotic) behaviors of the corresponding solutions determined by them.

We will need the following simple but useful results. First, let be real scalar (or vector) sequences that satisfy where , and is a real number (resp., a real vector).(i)If is a sequence which satisfies (12), then (ii)If is a sequence which satisfies (13), then

Second, the function in (6) satisfies where and .

Third, we need to consider various ordering arrangements for three nonnegative integers , , and or four nonnegative integers , , , and . First, the ordering arrangements of three integers , , and can be classified into cases:

Indeed, let . Then, since either , or , we see that These are equivalent to by comparing the two sets of statements (18) and (19).

By similar reasoning, there are 12 ordering arrangements for four nonnegative integers , , , and :

In the sequel, if and are real intervals, then we adopt the convention that On the other hand, since , we have If we let then

2. The Case

In this section, we assume that .

Let be a solution of (3). Let us consider first the case where . Then by (7) and (16), we see that Hence, we may see further that . Similarly, since we see that if , then . By similar considerations, we may build a “directed graph” represented by Table 1.

In this table, we record the fact that as the entry and so forth.

Next, we let be a solution of (3) again. If , then from Table 1 we see that . Now that , we may use Table 1 to infer that again. By induction, we see that for all . By (7), we then see that and hence by (14) and (15), we see that as . If , then from the entry of Table 1 we see that . Now that , by Table 1 again, we see that By induction, we see that and for . By (7), we may see further that for , and hence by (14) and (15), and as . By considering and in different parts of the plane, we may apply the same principle to obtain Table 2.

In this table, there are seven indeterminate cases. Let us go through two cases. First, let . Then in view of (25), for some . There are then two subcases: (i) or (ii) . In the former case, by Table 1, and (hence) and by induction, In the latter case, we may similarly show that . By considering in different parts of the plane, we may then construct the following three self-explanatory Tables 3, 4, and 5.

As a consequence, if and , then the solution of (3) originated from and will satisfy . Then by Table 2, .

Next, let . Then by (25), for some As mentioned before, there are twelve different ordering arrangements B1–B12 for these integers. In case B1 holds, we may make use of Table 1 repeatedly to show that (which is recorded in self-explanatory Table 6), and then by Table 2, we finally see that .

In conclusion, we have shown the following result.

Theorem 1. Suppose that . Then, a solution of (3) must either be asymptotically -periodic tending to the limit cycles or asymptotically -periodic tending to the limit cycles

We remark that, as can be seen from the above exhaustive arguments, the “region of attraction” of each limit cycle can be given, here if is the set of all solutions of (3) that tend to the limit cycle , then is called the region of attraction of .

Example 2. Consider the region of attraction of the limit cycle . By Table 2, we first see that solutions with initial values in or tend to the limit cycle (and hence ); and solutions with initial values in the seven indeterminate cases may or may not tend to the limit cycle .
Therefore, we next consider Table 3. If where , then and hence . If where , then and hence . Furthermore, no other cases in Tables 3, 4, and 5 can lead to , or .
Next, we consider Table 6. By direct examination, there is no case which leads to or . There is no case, except possibly B10, that leads to with , or with . However, if B10 holds and , then implies , which in turn implies (by Table 3) that . Hence, .
In summary, the region of attraction of is the union of the following sets:

3. The Case

In this section, we assume that .

Lemma 3. Suppose that . Let be a solution of (3) originated from . Then, there exists such that and .

Proof. Let be a solution of (3) originated from . Assume to the contrary that for all . Then, By (14) and (15), we see that as . But implies is in the interior of which is disjointed from . A contradiction is, thus, obtained.

In view of Lemma 3, we only need to investigate the asymptotic behaviors of solutions of (3) such that and are not simultaneously in . To this end, we first note that Hence, if we let then As in the previous section, we may build a table (consisting of two separate parts Tables 7 and 8) which plays the same role as Table 1.

Then, we may construct Tables 914 which play the same role as Tables 36.

Then by Lemma 3 and these tables, we may show the following result.

Lemma 4. Suppose that . Let be a solution of (3). Then, there exists such that .

Proof. In view of Lemma 3, we may assume that and , or, and .
Consider first the case where and . Then, and for some . By Table 11, if , then . Since , there exists some such that . By the last entry in Table 9, we see that there is some such that . If , then . Since , there exists some such that . By the last entry in Table 9, we see that there is some such that .
Consider next the case where and . Then, for some . By Table 12 if , then . By the last entry in Table 9, we see that there is some such that . If , then . By the last entry in Table 9, we see that there is some such that .
Consider next the case where and . Then, and for some . By Table 13, if A1 holds, that is, , then . Then, there exists such that . By Table 12 again, if , then . By the last entry in Table 9, we see that there is some such that ; if , then . By the last entry in Table 9, we see that there is some such that . If A5 holds, that is, , then . Then, there exists such that . By Table 10 again, if , then . By the last entry in Table 9, we see that there is some such that ; if , then . By the last entry in Table 9, we see that there is some such that . The other cases are similarly proved.
Consider next the case where and . Then, and for some . Next, we need to consider six cases: (i) , (ii) , (iii) , (iv) , (v) , and (vi) . Suppose that holds, by Table 14, then . Furthermore, we need to consider six cases: A1–A6. Suppose that A3 holds, that is, , then by Table 13, . Then there exists such that . By the third entry in Table 9, we see that there is some such that . Suppose that A6 holds, that is, , then by Table 13, . If , then by Table 11, . Then, there exists such that . By the last entry in Table 9, we see that there is some such that ; if , then by Table 11, . Then there exists some such that . By the last entry in Table 9, we see that there is some such that . The other cases are similarly proved.
The proof is complete.

Theorem 5. Suppose that . Then, every solution of (3) tends to .
Indeed, let be a solution of (3). In view of Lemma 4, one may assume that . Then by the entry in Table 7, one may sees that and by induction, for all . By (14) and (15), then sees that as required.

4. The Case

Suppose that . Then, If we let then Let be a solution of (3). Let us consider first the case where . Then by (7) and (16), we see that Note that (in view of our assumption that ) Hence, we may see further that . Similarly, since we see that if , then and . By similar considerations, we may build a table (consisting of two separate parts Tables 15 and 16) which plays the same role as Table 1.

Next, we let be a solution of (3) again. If , then by the entry of Table 15, we see that . Now that , we may use the entry of Table 16 to infer again. By induction, we see that for all . By (7), we then see that and hence by (14) and (15), we see that as . By considering and in different parts of the plane, we may apply the same principle to obtain Table 17.

In this table, there are twelve indeterminate cases. Let us go through one case. Let . Then, for some . There are then two subcases: (i) or (ii) . In the former case, by Table 15, and (hence) and so forth, and finally, In the latter case, we may similarly show that . By considering in different parts, we may then construct four self-explanatory Tables 18, 19, 20, and 21.

As a consequence, if and , then the solution of (3) originated from and will satisfy . Then by Table 17, .

Next, let . Then, for some As mentioned before, there are twelve different ordering arrangements B1–B12 for the nonnegative integers . In case B2 holds, we may make use of Table 22 repeatedly to show that (which is recorded in self-explanatory Tables 22 and 23). If , then by Table 19, . By Table 17 again, .

In conclusion, we have shown the following result.

Theorem 6. Suppose that . Then, a solution of (3) must be either asymptotically -periodic tending to the limit cycles or or asymptotically 2-periodic tending to the limit cycle .

We remark that, as can be seen from the above exhaustive arguments and Example 2, the region of attraction of each limit cycle can be given precisely (by implementing a simple computer program, if necessary).

5. Conclusions

In neural network terminologies, we have discussed a simple two-neuron recurrent McCulloch-Pitts-type neural network with a nonnegative threshold . For , all solutions tend to the limiting state ; for , all solutions must either be tending to the limiting states or , or asymptotically -periodic tending to the limit cycles , or ; and for , all solutions must either be tending to the steady states , or or asymptotically -periodic tending to the limit cycle . In all cases, the corresponding region of attraction of each limit cycle can be given.

Our proofs show that more general multiple-neuron recurrent McCulloch-Pitts-type neural networks possess similar behaviors. However, the derivations may involve more delicate graph theoretic arguments and are better left for other studies.