Abstract
This paper presents a novel approach for clustering, which is based on quasi-consensus of dynamical linear high-order multiagent systems. The graph topology is associated with a selected multiagent system, with each agent corresponding to one vertex. In order to reveal the cluster structure, the agents belonging to a similar cluster are expected to aggregate together. To establish the theoretical foundation, a necessary and sufficient condition is given to check the achievement of group consensus. Two numerical instances are furnished to illustrate the results of our approach.
1. Introduction
With a rapid increase in the scale of massive data and information, the probing of potential knowledge in big data, such as data structures and certain unknown correlations, has been attracting more and more scholars. As an essential technique of data mining, clustering has been widely applied in various practical fields, such as linguistics [1], bone microarchitecture analysis [2], community detection [3], wind power prediction [4], image segmentation [5, 6], and even genome expression study [7].
Clustering is about assigning certain set of data points into different clusters, for the purpose of highlighting the similitude of data points being organized in the same cluster, while simultaneously reflecting the distinctions between different clusters [8]. In the past decades, diverse clustering algorithms and criteria have been developed adequately, for example, normalized cut [6] and min-max cut [9] algorithms, the graph partitioning algorithms [10], the algorithms based on optimization of modularity functions [11], and the spectral clustering algorithms [12].
In addition to the abovementioned literature, some clustering algorithms have also been presented which are built on the study of certain dynamical behaviors of networks, for example, the random walk algorithms and the network synchronization algorithms. Random walk implies shifting along a random route, with each step to a new nearest vertex. By random walk algorithms [13], the clustering structure of a given network topology could be probed with the aid of generated random walk flows. Conversely, the network synchronization algorithm stands from another perspective [14], which considers coordinating different signals among vertices. Typically certain Kuramoto vibrator is employed, such that synchronization of vibrations could be achieved within the same group.
In this research article, a novel method for clustering is presented, which is rooted upon observation to the motions of dynamical multiagent systems. Concretely speaking, for the purpose of discerning the different affiliations of vertices in a given network, the selected dynamical multiagent system should be linked with the concerned network topology, and there should be a one-to-one correspondence between each pair of dynamical agent and vertex. The agents keep on moving inside a common space, either physical or abstract. Those corresponding to the vertices that are affiliated with any specific cluster should aggregate as time elapses. Finally, the clustering structure would be uncovered automatically, according to the formation of agent positions.
Analogical topics as discussed above are usually referred to as group consensus in the field of control theory. Yu and Wang earlier were concerned with this phenomenon in [15]. Hu et al. discussed group consensus for two classes of dynamical multiagent systems, respectively, which are systems with hybrid protocols (there is discontinuous transfer of information between nearest groups [16]) and systems with distinct groups comprising different types of agents [17]. To achieve group synchronization of the coupled vibrators, Su et al. considered both the adaptive pinning control strategy [18] and the case with multiple leaders [19]. Xie et al. [20] addressed the group consensus problem of first-order systems. Other recent relevant studies include [21–23].
The novelty of the current work compared with the aforementioned results in the literature lies in several perspectives. As far as we know, in the existing researches, the expected clustering formation is prescribed in advance, paying primary attention to synthesis of specific information transfer protocol for each group, so that the agents assigned to the group could ultimately aggregate. In contrast, in the framework of the current paper, the cluster distribution should completely be determined by the topology of network itself, without the need of any prerequisite knowledge aforehand. In this regard, the advantage of the current work is explicit, especially in its potential practicability.
Besides introducing a novel method for clustering, a major contribution of this paper is proposing a criterion for checking whether or not a high-order LTI (Linear Time-Invariant) multiagent system can reach group consensus, which generalizes the existing well-known necessary and sufficient condition for consensus achievement [24–28]. In fact, the existing condition on consensus is a particular instance of the condition for group consensus presented here.
In addition to the above, the current paper provides an exemplification of the usage of unstable dynamical systems, whereas, in contrast, unstable systems are conventionally regarded as being insignificant in control theory.
Theoretical studies on dynamical multiagent systems have already been extensive in the area of control theory, especially on the consensus problem. However, the application instances corresponding to these theories that can well support them are still scarce. Our exploration attempts to introduce a practical scenario from the field of data analysis, under a motivation to facilitate applying, verifying, or enriching certain relevant researches.
The remaining part of the current paper holds the following organization. The preliminaries and model formulation are introduced in Section 2. In Section 3, the clustering method is expounded in detail, based on theoretical analysis on condition for group consensus. Section 4 exemplifies the technique by two simulation cases. Finally, this paper is concluded in Section 5.
2. Model Formulation and Preliminaries
The topology of the network for clustering can be expressed by , with being the vertex set and the weighted edge set. The is assumed to be a graph of th order, being undirected and connected. The ultimate objective of clustering is to assign the elements of into divergent affiliations, in accordance with the topology of .
For such a purpose, a procedure rooted in observing the motions of dynamical multiagent systems is put forward, to figure out the clustering formation automatically. The procedure is composed of the following two fundamental steps.
Step 1. Define a dynamical multiagent system as follows and attach with it the network topology concerned, with each agent associated with one particular vertex.where expresses the state of agent which is moving in a normed state space , is the weight indicating the strength of communication link between vertices and , and the functions and express the self-dynamics and the dynamical interagent information transfer protocol, respectively.
Step 2. Determine appropriate initial values and then let the dynamical system operate autonomously.
As time elapses, group consensus or quasi-consensus would manifest, as long as the overall setup of both the structure and parameters of dynamical multiagent system (1) is feasibly configured. Finally, a clustering result can be concluded via observing the setup formed by the agent states within the state space.
The definition below formulates the concept of group consensus discussed in this paper.
Definition 1. For the dynamical multiagent system (1), ifthen agents and reach an agreement. For a vertex set , if , the agents and reach an agreement, and then system (1) reaches a consensus in . If consensus is reached in each , respectively, withthen the entire dynamical system reaches a group consensus.
Remark 2. The assumption that the graph is both undirected and connected is merely due to practical requirements of clustering and is not due to any technical limitations. This can be understood through later discussions.
3. Group Consensus of Dynamical Linear Time-Invariant Systems
In this section, the detail of the clustering process is elaborated by employing high-order LTI multiagent systems, which are described aswhere expresses the state vector of agent i and matrices express the LTI self-dynamics of the agents and the information transfer protocol between nearest agents, respectively. The technique for successfully acquiring a rational clustering result depends on appropriately selecting values of the pair of matrices and .
To this end, some previous results on consensus are reviewed first.
Lemma 3 (see [28]). For the dynamical LTI high-order system (4), if it reaches consensus, then, as , the motion of any agent is regulated by the equation .
Lemma 4 (see [29]). The Laplacian matrix of a directed graph has exactly one zero eigenvalue iff includes a spanning tree, with the corresponding eigenvector . Besides, all the remaining eigenvalues have positive real parts.
Corollary 5. The Laplacian matrix of an undirected graph has exactly a single zero eigenvalue iff is connected, with the associated eigenvector . Besides, all the remaining eigenvalues .
Lemma 6 (see [28]). For the dynamical multiagent system (4) with as the eigenvalues of the Laplacian matrix of the directed graph , if is not Hurwitz then the system reaches consensus iff (1)the graph topology includes a spanning tree;(2)all the matrices are Hurwitz.
Lemma 3 implies that if a certain subset of agents reaches a consensus, then the overall motion of this subset will ultimately be dominated by the matrix A. As long as A is Hurwitz, the motions of distinct consentaneous subsets would independently converge to the origin of state space, even without any exchange of information. Consequently, in order to differentiate agents between distinct affiliations, matrix A should not be Hurwitz. Actually, it is preferable for A to be unstable to avoid the possibility of critical stability, such that distinct clusters would mutually diverge.
Theorem 7. Consider the dynamical system (4). Suppose that the spectrum of Laplacian matrix of the directed graph with spanning tree iswith the series of matricesbeing not Hurwitz andbeing Hurwitz. The pair of agents and (or and , if ) reaches agreement iff the th row of the product possesses the configuration:where represents any feasible solution of the matrix equation represents the nonsingular matrix that transforms the Laplacian matrix into the similar Jordan canonical formand “” denotes any arbitrary value.
Proof. If the stack vector of the states of agents is defined asthen the system dynamics can be described byLet ; then (12) can be transformed intowhich is equivalent towhere “” denotes an indefinite matrix that may be either or zero. Define the auxiliary vectors and the stacked form asIt is evident that andSubstituting (9) into the above equation yieldsIt follows thatAccording to (14), ; thus because is Hurwitz. Also, holds, leading to , because is Hurwitz and . A similar analysis can be recursively conducted till is derived. As a result, it can be concluded thatDue to the structure of and the fact that , the limits of the first n entries of vector are zero and the limits of the entries indexed by to are indefinite, while the limits of the remaining entries are all zero. In other words,Equation (19) can be rewritten asAccording to the structure of , the fact that agents and reach an agreement implies that the entries in the left side of (22) indexed from to tend to approach zero. That is to say, If , then (23) is true because of the structure of (21). Conversely, if , by noticing the fact that agreements are independent of the initial states and concerning the structure of (21), one can conclude that
Remark 8. Theorem 7 can verify the agreement between any two agents, since the index assignment is trivial for a network. With this theorem, an overall portrait of the group consensus can be easily derived.
Remark 9. For the sake of the clustering application, only the undirected graph should be concerned. However, Theorem 7 is more generalized since it addresses the systems with graphs that could have directed arcs.
Remark 10. Matrix can be figured out by solving the matrix equation (9) aswith expressing a Moore-Penrose inverse [30] of the singular Laplacian matrix . Besides, matrix is constituted by the generalized eigenvectors of . Therefore, it is simple to compute the matrix product .
Remark 11. Both the matrices and are derived from the Laplacian matrix. Thus, if only the value of is definite, the clustering result merely depends on the topology of graph.
Remark 12. Theorem 7 is a bridge connecting the concept of consensus with the diverse nonconsensus cases. The smaller the value of , the higher the degree of freedom that matrix would possess, for the sake of ensuring agreement. Here are two extremes: if , then the overall system reaches a consensus, whatever the value of is; oppositely, if all the matrices are not Hurwitz, then there should be no agreement at all. Actually, the existing well-known criterion for checking the consensus [24–28] (summarized as Lemma 6 here) can be regarded as a particular case or corollary of Theorem 7 when .
4. Simulations on Clustering
This section will exemplify the presented technique of clustering by two simple and typical simulation instances.
Example 1. Consider a graph in Figure 1.
The weighted adjacency matrix iswith the associated Laplacian matrixand spectrum . Set the corresponding multiagent system as In this instance, and are unstable and are Hurwitz. Therefore, . It yields thatConsequently, Based on Theorem 7, it is explicitly known that agents 1~3 reach a consensus; meanwhile agents 4~6 reach another consensus. As a result, a clustering formation is clearly exhibited. The motions of agents in the 2-dimensional state space are illustrated in Figure 2.
Example 2. In the second example, consider the graph illustrated in Figure 3.
The adjacency matrix isand the Laplacian matrix iswith the spectrum . Let the associated agents be LTI second-order systems withIn this case, , are unstable, whereas are Hurwitz. Still, . It can be easily derived thatAs a resultAccording to Theorem 7, only two pairs of agents can achieve agreement, respectively, which are agents 1 and 2 and agents 5 and 6. The state trajectories obtained from the experiment are shown in Figure 4.
Although not all agents could achieve a precise group consensus in this case, from Figure 4, the six agents can still be differentiated as two distinct clusters, practically, because the speed of divergence between the two clusters is much faster than the speed of divergence of the agents inside any cluster, in a certain time span. The variation of difference between the states of agents 2 and 3 is illustrated in Figure 5, which is .
From Figure 5, one can clearly sense that the vicissitude of the difference between agent 3 and the other agents in the same cluster is sophisticated, including two phases: before , there is a tendency of agreement; however, afterwards, the tendency turns into a departure. This can be intuitively explained via a comparison with Example 1.
Let us inspect the dynamical equation of agent 3; that is,The vector field is composed of three components as shown above: the autonomous force, the interactive force from the nearest neighbors of the same cluster, and the interactive force from the other cluster. During the first phase of Example 2, the effect of the third component is relatively weak, because of the small value of ; thus the states of agents belonging to the same cluster, that is, 1, 2, and 3, tend to achieve a consensus; however, as time elapses and the distance between the two clusters increases, the effect of the third component also keeps on increasing and ultimately it overwhelms the diminished second component and tears agent 3 away from its nearest neighbors 1 and 2. By contrast, in Example 1, a precise group consensus can be reached because the interactive force from the third component is balanced upon every agent.
5. Conclusion
A novel method for clustering is introduced in the current paper, based on group consensus or quasi-consensus of the motions of dynamical systems. For the sake of categorizing the vertex set into disparate clusters, a specific dynamical multiagent system is attached to the graph addressed, with each vertex associated with an agent. If the dynamical system is appropriately configured, then the agents affiliated with the similar cluster should gradually assemble together in their common state space; meanwhile the different clusters diverge away mutually. In addition to the method of clustering, a primary contribution is the presentation of a criterion for checking whether or not a group consensus could be reached by high-order linear multiagent systems, which is a necessary and sufficient condition generalizing the previous well-known condition of consensus. In fact, the scope of possible utility of the condition for group consensus is broader, beyond the clustering scenario being dealt with here. Two typical simulation instances are exhibited to exemplify the process of clustering method. As to the potential future extensions, there are several explicit major directions: () the method can be utilized to handle various real-world applications such as image segmentation, electrocardiosignal analysis, and classification of natural languages; () a further exploration toward the mechanism of group consensus or quasi-consensus could be conducted, and a more straightforward correlation between group consensus and the topological characteristics of networks might be obtained; () the patterns of diverse nonconsensus motions can be studied in depth.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Acknowledgments
This work is supported by National Natural Science Foundation (NNSF) of China (Grants 61374054 and 61263002), by Fundamental Research Funds for the Central Universities (Grants 31920160003 and 31920170141), and by Program for Young Talents of State Ethnic Affairs Commission (SEAC) of China (Grant 2013-3-21).