Sign Inference for Dynamic Signed Networks via Dictionary Learning

Abstract

Mobile online social network (mOSN) is a burgeoning research area. However, most existing works referring to mOSNs deal with static network structures and simply encode whether relationships among entities exist or not. In contrast, relationships in signed mOSNs can be positive or negative and may be changed with time and locations. Applying certain global characteristics of social balance, in this paper, we aim to infer the unknown relationships in dynamic signed mOSNs and formulate this sign inference problem as a low-rank matrix estimation problem. Specifically, motivated by the Singular Value Thresholding (SVT) algorithm, a compact dictionary is selected from the observed dataset. Based on this compact dictionary, the relationships in the dynamic signed mOSNs are estimated via solving the formulated problem. Furthermore, the estimation accuracy is improved by employing a dictionary self-updating mechanism.

1. Introduction

Over the past few years, a number of mobile applications that allow users to enjoy networking have emerged. Correspondingly, there has been a proliferation in mobile online social networks (mOSNs). With the ubiquitous use of mobile devices and a rapid shift of technology, it is worthy to investigate the mOSNs from a privacy or security standpoint [1, 2]. The related applications are also extensive such as authentication and recommendation online. In this context, researches about mobile online networks where two opposite kinds of relationships can occur have become common; people not only form links to indicate friendship, support, or approval but signify disapproval or distrust of the opinions of others. It is natural to model such networks as signed networks, where the sign of a link weight can be either positive or negative, representing the status of a relationship. Analogous to traditional social networks analysis, the relationships in signed mOSNs can be represented as a graph, where nodes denote the objects (e.g., people or mobile terminals) and signed edges denote the relationships or links (e.g., a communication made between two people). The link structure of the resulting graph can be exploited to detect underlying groups of objects, predict missing links, and handle many other tasks [3–17].

One of the most fundamental theories that are applicable to signed social networks is social structural balance [5, 6, 16]. Structural balance corresponds to the possibility of exactly dividing the signed graph into two adversary subcommunities such that all edges within each subcommunity have positive weights while all edges joining agents of different communities have negative weights. Obviously, graphs of nonnegative weights are a special case of structural balance, in which one of the two subcommunities is empty. Since the assumption that structural balance exists in a real signed network might be too extreme, a concept called weak structural balance further generalizes structural balance by discussing the multiadversary-subcommunities partition of the signed graph [7].

Structural balance and weak structural balance have been shown to be valid to analyze signed networks. For instance, the sign inference problem, which aims to infer the unknown relationship between two objects, can be solved by mining balance information of signed networks from local and global perspectives [8–10, 12–17]. With the help of the result inferred, it is possible to predict the relationships so that legitimate participants can eliminate networking security vulnerabilities. Nevertheless, most of these state-of-the-art methods for sign inference problem are mainly considered from a static point of view, and dynamic scenarios are rarely taken into account. Therefore, it is necessary to establish a rational dynamic network model to infer the sign of relationships.

Actually, there exist several inherent qualities of mOSNs that are challenging to reliably sense the global states of relationships for the large networks in practice [2]. First, in contrast to traditional social networks, the observations of relationships in mOSNs are closely associated with the geographical environment, as well as the relative locations and signal coverage of mobile terminals/network access points. Due to these spatial constraints, such observations, which seem linearly related to the global data of relationships (i.e., linearly sampled from the global data), are bound to miss a significant number of values. Consequently, they are not sufficient to unambiguously infer the true status by the traditional solutions of linear-inverse problem in general. Second, in mOSNs different relations between entities may appear at different times. Accordingly, observations of the networks vary during a time period long enough. These dynamic interactions over time essentially introduce time dimension to the problem of mining, the potential relationship structures. Third, despite maintaining the dynamic performance, the underlying relationships in reality always display some “redundancy” attributed to the gradual/periodic variation [3], the relative stability, and so forth. Owing to the aforementioned characteristics of mOSNs, the mass redundant data generated in variant scenarios will result in resource challenge. Hence, although many observers collect features for at least part of the networks, there are still serious impediments to reliable large-scale or network-wide data processing. After these aspects of mOSNs are learned, it is reasonable to organize the entire relationship dataset in the form of tensor coincident with its spatiotemporal structure. Meanwhile, efficient relationship inference approaches associated with the tensor model are required to overcome the obstacles of this data processing.

The aim of this paper is to develop algorithms for the sign inference in signed mOSNs in global and spatiotemporal evolvement perspectives. In particular, we assume that the signed mOSN possesses an underlying dynamic weakly balanced complete network structure. Suppose that we are given an incomplete networking observation tensor (or 3-dimensional array), which consists of the adjacency matrices corresponding to the snapshots of the underlying dynamic weakly balanced complete network at times 𝑇₀,𝑇₀+1,…,𝑇₀+𝑇−1,𝑇₀+𝑇. Then the sign inference task is to estimate the sign patterns of all possible links in the dynamic complete network at time 𝑇₀+𝑇. Utilizing the low-rank property of the weak structural balance and the features extracted from the observation tensor, we consider the inference via the incomplete relationship data as an underdetermined linear-inverse problem and develop an approach via a low-rank matrix reconstruction to solve this problem. Moreover, we regard the observation tensor as the training data set and choose a dictionary from it to improve the validity and efficiency of our inference approach. The dictionary selection method is designed by reducing the size of an overcomplete feature set extracted from the training dataset. Also, a dictionary self-updating mechanism is introduced to improve accuracy of the inference.

Here are the key contributions we make in this paper.(i)A dictionary selection approach based on group sparsity has been designed to generate a set containing minimal sizes of features to increase computational efficiency. Specifically, the observation tensor is considered to be the raw materials for feature extraction.(ii)The sign inference problem referring to the weakly balanced mOSNs is formulated as a low-rank matrix reconstruction from the selected dictionary. Under certain mild conditions, a low-rank matrix reconstruction algorithm is applied to solve the sign inference problem, and it turns out to be much more accurate and efficient than other inference methods in the literature. A dictionary self-updating mechanism is also introduced to adjust the dynamic characteristics of the network and improve the sensing accuracy.

The rest of this paper is organized as follows. In Section 2, we build the model of the dynamic signed network. Some basics of balance theory are also reviewed for the sake of integrality. In Section 3, we first extract the initial candidate feature pool from the observation tensor and propose a dictionary selection approach. Then we propose our low-rank matrix reconstruction method to solve the sign inference problem. The implementation details of the dictionary self-updating procedure are also proposed. In Section 4, we conduct numerical experiments which demonstrate the validity of our network model for sign inference and justify the performance of our methods as well. Finally, we present our conclusions in Section 5.

2. Background and Preliminaries

2.1. Dynamic Signed Network Structure

Formally, a dynamic undirected signed network is represented as a dynamic graph 𝒢=(𝒱,ℰ), where 𝒱 is the vertex set of size 𝑛 and ℰ is the edge set varying over time. A network snapshot denoted by 𝒮_𝑡=(𝒱,ℰ_𝑡,𝐀^(𝑡)) presents the connections of 𝒢 observed at time 𝑡. Here, ℰ_𝑡 is the subset of ℰ and 𝐀^(𝑡)∈{−1,0,1}^𝑛×𝑛 is the adjacency matrix of 𝒮_𝑡 with the signed weights 𝑎^(𝑡)_𝑖𝑗=1if𝑖and𝑗havepositiverelationship,−1if𝑖and𝑗havenegattiverelationship,0ifrelationshipbetween𝑖and𝑗isunknown. (1)

Particularly, for each time 𝑡, a zero entry in 𝐀^(𝑡) is treated as an unknown relationship based on the acknowledgement that some potential attitudes exist between any two entities, even if the relationship itself is not observed. From this viewpoint, we can assume that there exists an underlying dynamics complete signed network 𝒢, in which only some partial relationships are observed at times 𝑇₀,𝑇₀+1,…,𝑇₀+𝑇−1, respectively. Correspondingly, we let 𝒜∈{−1,1}^{𝑛×𝑛×𝑇}denote the three-dimensional tensor that contains relationship information between all pairs of entities in 𝒢. Thus, the observation tensor 𝒜 consisting of a series of network snapshots can be represented as 𝒜_{𝑖,𝑗,𝑡}=𝑎^(𝑡)_𝑖𝑗=𝒜_{𝑖,𝑗,𝑡},𝑖,𝑗,𝑡∈Ω,0,otherwise, (2) where Ω is the index set of the observed entries. Let 𝒫_Ω be the orthogonal projection operator onto the span of tensors vanishing outside Ω so that the (𝑖,𝑗,𝑡)th component of 𝒫_Ω(𝒳) is equal to 𝒳_{𝑖,𝑗,𝑡} when (𝑖,𝑗,𝑡)∈Ω and zero otherwise. Then we have 𝒫_Ω(𝒜)=𝒜 (shown in Figure 1) and 𝒫_Ω𝑡(𝐀^(𝑡))=𝐀^(𝑡) for each time slice 𝑡, where ⋂_𝑡Ω_𝑡=∅ and ⋃_𝑡Ω_𝑡=Ω.

(a)

(b)

Figure 1 

Illustration of the adjacency tensor, the cube units symbolize the data of relationships: (a) the adjacency tensor of the observed network and (b) the adjacency tensor of the underlying complete network.

While the above kind of signed networks is called homogeneous, that is, relationships of the networks are between the same kinds of entities, a signed network can also be heterogeneous. In a heterogeneous signed network, there can be more than one kind of entities, and relationships between same or different entities can be positive and negative, such as YouTube with two kinds of entities—users and videos. Moreover, this three-dimensional network adjacency tensor can increase dimensions (e.g., spatial dimension, etc.) to adapt to a wider range of scenarios. In this paper, we mainly focus our attention on three-dimensional homogeneous signed networks.

2.2. Weak Structural Balance

Structural balance theory was first formulated by Heider [18] in order to understand the structure in a network of individuals whose mutual relationships are characterized in terms of friendship and hostility. Formally, a triad is considered balanced if the product of the signs in the triad is positive; that is, it contains an even number of negative edges. This is in agreement with principles such as “a friend of my friend is more likely to be my friend” and “an enemy of my friend is more likely to be my enemy” [6]. The configurations of balanced and unbalanced triads are shown in Figure 2. One possible weakness of this theory is that the defined balance relationships might be too strict. In this perspective, by extending the fundamental beliefs in real networks, weak structural balance is proposed as a way of eliminating the assumption that “the enemy of my enemy is my friend” [7]. Equivalently, the case that “the enemy of my enemy is my enemy” is permitted. Therefore, the local structure of weak balance posits that only triads with exactly two positive edges are implausible and that all other kinds of triads should be permissible (also illustrated in Figure 2).

(a)

(b)

(c)

(d)

Figure 2 

Signed undirected connectivity configurations mentioned in Section 2.2: (i) (a) and (b) are balanced triads, (c) and (d) are unbalanced triads, and (ii) (a), (b), and (d) are weakly balanced triads.

The formal definition of weakly balanced networks is as follows.

Definition 1 (weakly balanced networks [7]). A (possibly incomplete) network is weakly balanced if and only if it is possible to obtain a weakly balanced complete network by filling the missing edges in its adjacency matrix. Furthermore, in terms of patterns of global structure, a complete network is weakly balanced if and only if the vertex set can be divided into 𝑟 clusters, 𝑟≥1, such that all the edges within clusters are positive and all the edges between clusters are negative.
There exists the literature discussing the approaches of clustering and sign prediction with respect to signed networks. Ideas derived from local balance of signed networks can be successfully used to yield algorithms for sign inference [9, 10]. Meanwhile, several works analyze the social interrelations from global perspective of structural balance [8, 13–15, 17]. In particular, it is shown in [8] that the adjacency matrix of weakly balanced networks has a “low-rank” structure, and the sign prediction methods based on low-rank modeling were proposed as well.

Theorem 2 (low-rank structure of signed networks [8]). The adjacency matrix 𝐀∈{1,−1}^𝑛×𝑛 of a complete 𝑟-weakly balanced network has rank 1, if 𝑟≤2, and has rank 𝑟 for all 𝑟>2.

Actually, since the global viewpoint of weak balance stated in Definition 1 obeys clustering characteristics presented in Theorem 2, for 𝐀, there exists an invertible matrix 𝐏 such that 𝐏𝐀𝐏^𝑇=𝟏_𝑛1𝟏_𝑛2𝟏−1_𝑛3𝟏−1⋱_𝑛𝑟, (3) where 𝟏_𝑛𝑖 on the primary diagonal is an 𝑛_𝑖-order square matrix whose entries are all 1 (∑^𝑟_𝑖=1𝑛_𝑖=𝑛) and the other entries of 𝐏𝐀𝐏^𝑇 are all −1. The 𝑛_𝑖-order square matrix indicates the 𝑖th cluster.

Notation. For 𝐗∈𝑅^𝑚×𝑛, let the mixed norm ‖𝐗‖_2,1=∑^𝑚_𝑖=1‖𝐗_𝑖⋅‖₂; the soft-thresholding operator 𝒟_𝜏(⋅)∶𝐗∈𝑅^𝑚×𝑛↦𝐘∈𝑅^𝑚×𝑛 is also defined obeying 𝐘_𝑖⋅=0if𝐗_𝑖⋅2𝜏≤𝜏,1−𝐗_𝑖⋅2𝐗_𝑖⋅otherwise, (4) where 𝐗_𝑖⋅ and 𝐘_𝑖⋅ denote the 𝑖th row of 𝐗 and 𝐘, respectively [15]. The invertible vectorization is denoted by vec(⋅)∶𝑅^𝑚×𝑛↦𝑅^𝑚𝑛.

Let 𝒮^1,1_𝜇,𝐿(𝑅^𝑚×𝑛) be the class of convex functions with Lipschitz gradient [19]. A continuous differentiable function 𝑓(𝐘) belongs to 𝒮^1,1_𝜇,𝐿(𝑅^𝑚×𝑛) for some 0≤𝜇≤𝐿 if for any 𝐗,𝐘∈𝑅^𝑚×𝑛 we have both of the following: ∇𝑓(𝐗)−∇𝑓(𝐘)_𝐹≤𝐿‖𝐗−𝐘‖_𝐹,∇𝑓(𝐗)−∇𝑓(𝐘),𝐗−𝐘≥𝜇‖𝐗−𝐘‖²_𝐹. (5)

3. Sign Inference via Dictionary Learning

In this section, we focus on a solution of the sign inference to estimate connection statuses via dictionary learning. As the preparation, we propose a large-scale dictionary selection method to generate the dictionary for inferring. Assume that we are given a (usually incomplete) network observation tensor 𝒜 sampled from an underlying dynamic weakly balanced complete network 𝒢 with the adjacency tensor 𝒜. As the description in Section 1, it is reasonable to suppose that most relationships between entities have their own stability in a long period of time in practice and subsequently the change in the scale of each subcommunity is limited. Apparently, this implies the strong dependence retained among the observed data. Combining these assumptions with the low-rank characteristic of weakly balanced complete networks, we extract an initial feature pool from the observation tensor 𝒜 and propose a dictionary selection method to compress the scale of the feature pool in Section 3.1. The corresponding algorithm is presented, respectively, in Section 3.2. With the trained dictionary, we propose our sign inference approach and dictionary updating mechanism in Section 3.3, which are also inspired by the low-rank characteristic of weakly balanced complete networks.

The method we propose to handle the dictionary selection is motivated by the Singular Value Thresholding (SVT) algorithm, which is a simple and efficient algorithm for nuclear norm minimization problems proposed by Cai et al. [20]. Our basic idea is to obtain the optimal solution of the trace norm minimization problem by solving its dual problem whose objective function can be shown to be continuously differentiable with Lipschitz continuous gradient. Specifically, we prove that the optimal solution of the primary problem can be readily obtained from the optimal solution of the dual problem. We first provide a brief review of the standard SVT algorithm.

Considering the problem min_𝐗𝜏‖𝐗‖_∗+12‖𝐗‖²_𝐹subjectto𝒫_Ω(𝐗)=𝒫_Ω(𝐌), (6) Cai et al. [20] give a theoretical analysis that, when 𝜏→∞, the optimal solution of problem (6) converges to that of the standard problem: min_𝐗‖𝐗‖_∗subjectto𝒫_Ω(𝐗)=𝒫_Ω(𝐌). (7) Given that 𝜏>0, the SVT algorithm operates as a linear Bregman iteration scheme. Furthermore, by defining the Lagrangian function of problem (6) as ℒ(𝐗,𝐘)=𝜏‖𝐗‖_∗+12‖𝐗‖²_𝐹+𝐘,𝒫_Ω(𝐌−𝐗), (8) where 𝐘 is the Lagrangian dual variable, we can derive its dual function as 𝑓(𝐘)=inf_𝐗ℒ(𝐗,𝐘). (9) Cai et al. show that SVT indeed optimizes the dual function 𝑓(𝐘) via the gradient ascent method.

3.1. Large-Scale Dictionary Selection

We address how to select the dictionary given an initial candidate feature pool in this subsection. To this end, we first extract an initial candidate feature pool from 𝒜, which is sampled from 𝒜. Since 𝒜 consists of the adjacency matrices 𝐀^(𝑡)  (𝑡=𝑇₀,𝑇₀+1,…,𝑇₀+𝑇−1), the matrix 𝐀^(𝑡) in 𝒜 can retain the information of 𝐀^(𝑡) more or less. Thus, we reserve the group of 𝐀^(𝑡) with relatively higher sample rate to extract features. We use singular value decomposition (SVD) to express each 𝐀^(𝑡) as a series of orthogonal bases in Hilbert space; that is, 𝐀^(𝑡)=^𝑟𝑡_𝑟=1𝜎_𝑟^(𝑡)𝐮_𝑟^(𝑡)𝐯_𝑟^(𝑡)𝑇=^𝑟𝑡_𝑟=1𝜎_𝑟^(𝑡)𝐔_𝑟^(𝑡), (10) where 𝐮_𝑟^(𝑡) and 𝐯_𝑟^(𝑡) are singular vectors of 𝐀^(𝑡) with eigenvalue 𝜎_𝑟^(𝑡), 1≤𝑟≤𝑟_𝑡. Without loss of generality, we sort 𝜎₁^(𝑡)≥𝜎₂^(𝑡)≥⋯≥𝜎_𝑟^(𝑡)_𝑡 the singular values of 𝐀^(𝑡) in descending order, and set 𝐿_𝑡=𝑟_𝑡−argmax_{𝑟=1,…,𝑟𝑡−1}𝜎_𝑟^(𝑡)≥𝜎^(𝑡)_𝑟+1. (11) Then, due to the low-rank property of the weakly balanced complete adjacency matrix, we keep the group of 𝐔_𝑟^(𝑡) corresponding to the 𝐿_𝑡 largest 𝜎_𝑟^(𝑡) as the features. By this procedure, we extract an initial candidate feature pool as {𝐔_𝑟^(𝑡)∶𝑇₀≤𝑡≤𝑇₀+𝑇−1,1≤𝑟≤𝐿_𝑡}, where each matrix 𝐔_𝑟^(𝑡)∈𝑅^𝑛×𝑛 denotes a feature. Equivalently, we can discuss 𝒬={vec(𝐔_𝑟^(𝑡))∶𝑇₀≤𝑡≤𝑇₀+𝑇−1,1≤𝑟≤𝐿_𝑡} and form the matrix Φ=[vec(𝐔₁),vec(𝐔₂),…,vec(𝐔_𝑆)] for convenience, where vec(𝐔_𝑠)=vec(𝐔_𝑟^(𝑡)), 1≤𝑟≤𝐿_𝑡, 1≤𝑠≤𝑆=∑_𝑡𝐿_𝑡, and 𝑇₀≤𝑡≤𝑇₀+𝑇−1.

Due to massive data of the initial feature pool Φ, we hope to find an optimal subset to form the dictionary Ψ=[vec(𝐔₁),vec(𝐔₂),…,vec(𝐔_𝐾)] such that the set Φ can be well reconstructed by Ψ and the size of Ψ is as small as possible. To achieve this goal, we select Ψ such that the rest of the features in Φ can be well reconstructed using it. Analogous to the optimization problem in [21], the basic problem is formulated as follows: min_𝐗‖𝐗‖_2,1subjecttoΦ𝐗=Φ, (12) where Φ∈𝑅^𝑁×𝑆  (𝑁=𝑛²), 𝐗∈𝑅^𝑆×𝑆, and ‖𝐗‖_2,1=∑^𝑆_𝑖=1‖𝐗_𝑖⋅‖₂. Apparently, ‖𝐗‖_2,1 enforces the group sparsity on the variable 𝐗 and the optimal solution usually contains zero rows. This means that not all features in Φ are necessary to be selected to reconstruct any data sample.

Motivated by SVT, we have the equivalent problem of (12) as follows: min_𝐗‖𝐗‖_2,1+12‖𝐗‖²_𝐹subjecttoΦ𝐗=Φ. (13) The Lagrangian function of problem (13) is defined as ℒ(𝐗,𝐘)=𝜏‖𝐗‖_2,1+12‖𝐗‖²_𝐹+⟨𝐘,Φ−Φ𝐗⟩, (14) and its dual function is 𝑓(𝐘)=inf_𝐗ℒ(𝐗,𝐘). (15) We first examine the properties of the dual function 𝑓(𝐘) and then show how to achieve the optimal solution of the problem (13) from its dual optimum directly. As the mixed norm ‖𝐗‖_2,1 is not differentiable, it is difficult to optimize the dual function 𝑓(𝐘) directly. However, we can obtain a useful property of the dual function𝑓(𝐘) as follows.

Theorem 3. For all 𝜏≥0, the dual function 𝑓(𝐘) is continuously differentiable with Lipschitz continuous gradient at most 𝑀. Furthermore, the primal optimal 𝐗 of the problem (13) is given by 𝐗=𝒟_𝜏Φ𝐘, (16) when the dual optimal 𝐘 of the problem (13) is obtained.

The proof of Theorem 3 is based on the following results.

Lemma 4. For each 𝜏≥0 and 𝐘∈𝑅^𝑚×𝑛, one has 𝒟_𝜏(𝐘)=argmin_𝐗𝜏‖𝐗‖_2,1+12‖𝐗−𝐘‖²_𝐹. (17)

As a matter of fact, considering the following optimization problem: min_𝑥∈𝑅𝜏|𝑥|+12𝑥−𝑦², (18) it is easy to show that the unique solution admits a closed form called the soft-thresholding operator, following a terminology introduced by Donoho and Johnstone [22]; it can be written that |𝑥|𝑦=𝜏0if≤𝜏,1−|𝑥|𝑥otherwise. (19) Thus, from a generalized view, one has Lemma 4.

Also, the following result can be deduced based on the properties of Moreau-Yosida regularization [23].

Lemma 5. For any 𝐗,𝐘∈𝑅^𝑚×𝑛, one has 𝒟_𝜏(𝐗)−𝒟_𝜏(𝐘)²_𝐹≤𝒟_𝜏(𝐗)−𝒟_𝜏(𝐘),𝐗−𝐘. (20) It follows that 𝒟_𝜏(𝐘) is globally Lipschitz continuous with modulus 1.

Proof of Theorem 3. Since 𝑓(𝐘)=inf_𝐗ℒ(𝐗,𝐘)=inf_𝐗𝜏‖𝐗‖_2,1+12‖𝐗‖²_𝐹+⟨𝐘,Φ−Φ𝐗⟩=inf_𝐗𝜏‖𝐗‖_2,1+12‖𝐗‖²_𝐹+⟨𝐘,Φ⟩−Φ^𝑇𝐘,𝐗=inf_𝐗𝜏‖𝐗‖_2,1+12𝐗−Φ^𝑇𝐘²_𝐹+⟨⟩−1𝐘,Φ2Φ^𝑇𝐘²_𝐹(𝐘)+⟨⟩−1=𝑔𝐘,Φ2Φ^𝑇𝐘²_𝐹 (21) and 𝑔(𝐘) is the Moreau-Yosida regularization of the mixed norm ‖⋅‖_2,1, using the well-known properties of Moreau-Yosida regularization [23], we get the results that 𝑔(𝐘) is a globally continuously differentiable convex function. Moreover, ∇𝑔(𝐘)=Φ(Φ^𝑇𝐘−𝒟_𝜏(Φ^𝑇𝐘)) and ∇𝑔(𝐘) is continuously differentiable with Lipschitz continuous gradient 𝜌; that is, for any 𝐘₁,𝐘₂∈𝑅^𝑁×𝑆, ∇𝑔𝐘₁−∇𝑔𝐘_2𝐹≤Φ^𝑇𝐘₁−𝐘_2𝐹≤𝜌𝐘₁−𝐘_2𝐹, (22) where 𝜌=sup_{‖𝐙‖𝐹=1,𝐙∈𝑅𝑁×𝑆}‖Φ^𝑇𝐙‖_𝐹. Then the gradient of 𝑓(𝐘) can be obtained as follows: ∇𝑓(𝐘)=∇𝑔(𝐘)+Φ−ΦΦ^𝑇𝐘=ΦΦ^𝑇𝐘−𝒟_𝜏Φ^𝑇𝐘+Φ−ΦΦ^𝑇𝐘=Φ−Φ𝒟_𝜏Φ^𝑇𝐘. (23) It follows that, for any 𝐘₁,𝐘₂∈𝑅^𝑁×𝑆, ∇𝑓₂𝐘₁−∇𝑓₂𝐘_2𝐹=Φ−Φ𝒟_𝜏Φ^𝑇𝐘₁−Φ+Φ𝒟_𝜏Φ^𝑇𝐘_2𝐹=Φ𝒟_𝜏Φ^𝑇𝐘₁−𝒟_𝜏Φ^𝑇𝐘_2𝐹≤ΦΦ^𝑇𝐘₁−𝐘_2𝐹≤𝑀𝐘₁−𝐘_2𝐹, (24) where the first inequality follows from (20) and 𝑀=sup_{‖𝐙‖𝐹=1,𝐙∈𝑅𝑁×𝑆}‖ΦΦ^𝑇𝐙‖_𝐹. When the dual optimal 𝐘 is obtained, by using the result of (21), we can get 𝐗=argmin_𝐗ℒ(𝐗,𝐘)=argmin_𝐗𝜏‖𝐗‖_2,1+12𝐗−Φ^𝑇𝐘²_𝐹=𝒟_𝜏Φ𝐘. (25) This concludes the proof.

Since 𝑓(𝐘) is the dual function of the objective function (13), 𝑓(𝐘) is concave. Let 𝑞(𝐘)(𝐘)𝜏𝒟=−𝑓=−_𝜏(Φ𝐘)_2,1+12𝒟_𝜏(Φ𝐘)²_𝐹+𝐘,Φ−Φ𝒟_𝜏(Φ𝐘), (26) which is convex. Thus, the following holds for any 𝐘₁,𝐘₂∈𝑅^𝑁×𝑆: 𝑞𝐘₁−𝑞𝐘₂,𝐘₁−𝐘₂≥0. (27) It is also easy to show that 𝑞(𝐘) belongs to the class 𝒮^1,1_0,𝑀(𝑅^𝑁×𝑆) and ∇𝑞(𝐘)=−Φ𝐈−𝒟_𝜏Φ^𝑇𝐘, (28) where 𝐈∈𝑅^𝑆×𝑆 is the identity matrix. Therefore, we can solve problem (13) by minimizing the objective function 𝑞(𝐘); that is, min_𝐘𝑞(𝐘). (29) Therefore, the dictionary Ψ is selected by the optimal solution 𝐘; that is, the 𝑖th column of Φ is chosen to be the atom of Ψ if ‖𝐘_𝑖⋅‖₂≠0. The optimization algorithm is presented in the next subsection.

3.2. Optimization Methods

In this subsection, we develop an efficient optimization algorithm to solve the dual problem (29). Because the objective function 𝑞(𝐘) is continuously differentiable with Lipschitz continuous gradient, it is feasible to utilize gradient-based optimization methods to achieve the optimal solution for their simplicity and low complexity within each iteration. However, classical gradient-based methods for functions with Lipschitz continuous gradient converge at a rate of 𝑂(1/𝑁), where 𝑁 is the number of iterations during optimization [19]. In fact, this is too slow especially when dealing with large-scale datasets. Note that Nesterov showed in his work [24] that an accelerated gradient algorithm can be constructed such that 𝑂(1/𝑁²), the lower bound on the convergence rate for gradient-based methods [25], is achieved when minimizing unconstrained smooth functions. With this consideration, in the following we propose an accelerated thresholding algorithm to solve these smooth convex optimization problems using Nesterov’s method with an adaptive line search scheme [19, 26].

We recall Nesterov’s method with an adaptive line search scheme as follows. Take the unconstrained smooth convex minimization problem min_𝐲∈𝑅^𝑛𝑞(𝐲), for instance, where 𝑞(𝐲) belongs to 𝒮^1,1_𝜇,𝐿(𝑅^𝑛), 𝜇≥0, and 𝐿<+∞. Nesterov’s method for this problem utilizes two sequences: {𝐲_𝑙} and {𝐬_𝑙}, 𝐲_𝑙,𝐬_𝑙∈𝑅^𝑛. The searching point 𝐬_𝑙 satisfies 𝐬_𝑙=𝐲_𝑙+𝛽_𝑙𝐲_𝑙−𝐲_𝑙−1, (30) where 𝛽_𝑙 is a tuning parameter. The approximate solution 𝐲_𝑙+1 can be computed as a gradient step of 𝐬_𝑙 as 𝐲_𝑙+1=𝐬_𝑙−1𝐿_𝑙∇𝑞𝐬_𝑙, (31) where 1/𝐿_𝑙 is the step size. Starting from an initial point 𝐲₀, 𝐬_𝑙 and 𝐲_𝑙+1 can be computed recursively according to (30) and (31) and can arrive at the optimal solution 𝐲. Although it has been shown that Nesterov’s method is a very powerful optimization technique for class 𝒮^1,1_𝜇,𝐿(𝑅^𝑛) [19], how to choose 𝛽_𝑙 and 1/𝐿_𝑙 in each iteration is a critical issue in Nesterov’s method. When they are set properly, the sequence {𝐲_𝑙} can converge to the optimal 𝐲 at a certain convergence rate. As a well-known scheme for setting 𝛽_𝑙 and 𝐿_𝑙, Nesterov’s constant scheme assumes 𝛽_𝑙 and 𝐿_𝑙 to be constant [19], while Nemirovski’s line search scheme requires 𝐿_𝑙 to monotonically increase, and 𝛽_𝑙 is independent of 𝐿_𝑙 [27]. Both of the settings result in slow convergence.

To overcome this drawback, an adaptive line search scheme for Nesterov’s method is proposed in [26]. Under the assumption that 𝜇, the low bound of 𝜇, is known in advance, this scheme is built upon the estimate sequence [19] defined as follows.

Definition 6 (estimate sequence [19]). A pair of sequences {𝜙_𝑙(𝐲)} and {𝜆_𝑙≥0} is called an estimate sequence of function 𝑞(𝐲) if lim_𝑘→∞𝜆_𝑘=0 and 𝜙_𝑙(𝐲)≤(1−𝜆_𝑙)𝑞(𝐲)+𝜆_𝑙𝜙_𝑜(𝐲), for all 𝐲∈𝑅^𝑛.

The estimate sequence defined in Definition 6 has the following important property.

Theorem 7 (see [19]). Let {𝜙_𝑙(𝐲)} and {𝜆_𝑘≥0} be an estimate sequence. For any sequence {𝐲_𝑙},   𝑞(𝐲_𝑙)−𝑞≤𝜆_𝑘(𝜙₀(𝐲)−𝑞)→0 if 𝑞(𝐲_𝑙)≤𝜙_𝑘≡min_𝐲∈𝑅^𝑛𝜙_𝑘(𝐲), where 𝑞 is the optimal objective function value.

We further specify the estimation sequence in [19]: 𝜙_𝑙(𝐲)=𝜙_𝑙+𝛾_𝑙2𝐲−𝐯_𝑙², (32) where the sequences {𝛾_𝑙}, {𝐯_𝑙}, and {𝜙_𝑙} satisfy 𝐯_𝑙+1=1𝛾_𝑙+11−𝛼_𝑙𝛾_𝑙𝐯_𝑙+𝜇𝛼_𝑙𝐬_𝑙−𝛼_𝑙∇𝑞𝐬_𝑙,𝛾_𝑙+1=1−𝛼_𝑙𝛾_𝑙+𝜇𝛼_𝑙,𝜙_𝑙+1=1−𝛼_𝑙𝜙_𝑙+𝛼_𝑙𝑞𝐬_𝑙−𝛼²_𝑙2𝛾_𝑙+1∇𝑞𝐬_𝑙²+𝛼_𝑙1−𝛼_𝑙𝛾_𝑙𝛾_𝑙+1×𝜇2𝐯_𝑙−𝐬_𝑙²+∇𝑞𝐬_𝑙,𝐯_𝑙−𝐬_𝑙. (33)

Then Algorithm  2 in [26] is proposed by modifying Nemirovski’s line search scheme with the adaptive parameters of this sequence, which satisfy Theorem 7.

Note that Theorem 3 indicates that the objective function 𝑞(𝐘) satisfies the conditions of using Nesterov’s method with an adaptive line search scheme. Therefore we directly extend Algorithm  2 in [26] to the high-dimensional scenarios to solve (29). The complete procedures are summarized in Algorithm 1.

(1) Input: 𝜇,𝛼−1=0.5, 𝐘−1=𝐘0, 𝐿−1=𝐿0, 𝛾0≥𝜇,𝜆0=1 (2) Output: 𝐘𝑁 (3) for  𝑙=1,2,…,𝑁  do (4) while true do (5)  compute 𝛼𝑘∈(0,1) as the root of 𝐿𝑙𝛼2𝑙=1−𝛼𝑙𝛾𝑙+𝛼𝑙𝜇,    𝛾𝑙+1=1−𝛼𝑙𝛾𝑙+𝛼𝑙𝜇,  𝛽𝑙=1−𝛼𝑙−1𝛾𝑙𝛾𝑙+𝐿𝑙𝛼𝑙𝛼𝑙−1; (6)  compute 𝐒𝑙=𝐘𝑙+𝛽𝑙𝐘𝑙−𝐘𝑙−1 (7)  compute 𝐘𝑙+1=𝐒𝑙−1/𝐿𝑙∇𝑞𝐒𝑙 (8)  if  𝑞𝐘𝑙+1≤𝑞𝐒𝑙−1/2𝐿𝑙∇𝑞𝐒𝑙2𝐹  then (9)   goto Step 14 (10)  else (11)   𝐿𝑙=2𝐿𝑙 (12)  end if (13)   end while (14)   set 𝜃=2𝐿𝑙𝑞𝐒𝑙−𝑞𝐘𝑙+1∇𝑞𝐒𝑙2𝐹,  𝐿𝑙+1=ℎ(𝜃)𝐿𝑙 (15)   set 𝜆𝑙+1=(1−𝛼𝑙)𝜆𝑙 (16) end for

Algorithm 1 

Adaptive line search scheme for dictionary selection.

In Algorithm 1, the while loop from Step 4 to Step 13 is designed to choose a proper step size to satisfy Step 8. As the Lipschitz gradient of 𝑞(𝐘) is 𝑀, 𝐿_𝑙 is upper bounded by 2𝑀 since Step 8 always holds when 𝐿_𝑙≥𝑀 [27]. In Step 14, we initialize 𝐿_𝑙+1=ℎ(𝜃)𝐿_𝑙, where ℎ(𝜃)=1,1≤𝜃≤5,0.8,𝜃>5, (34) and 𝜃>1 due to the condition in Step 8 [26]. Apparently, when 𝜃 is large, 𝐿_𝑙+1 can be adjusted to avoid the step size 1/𝐿_𝑙 becoming too small, which may slow down the convergence rate.

3.3. Sign Inference and Dictionary Update Mechanism

This subsection details how to use the dictionary to solve the sign inference problem. Actually, this problem bears similarity to the sign prediction problem in the static signed networks or the unsigned networks varying periodically [3, 8, 11, 12]. In this paper, we intend to infer the unknown relationship between a pair of entities 𝑖 and 𝑗 based on partial relationship observations of the entire dynamic network at time 𝑇₀+𝑇. We expect to accomplish this task with the help of the dictionary constructed by the relationship data for times 𝑇₀ through 𝑇₀+𝑇−1. As aforementioned, there exists strong dependence between the connection status at time 𝑇₀+𝑇 and the history relationship dataset in the dynamic network. We formulate the sign inference problem as follows: 𝐱=argmin_𝐱12‖𝐲−Ψ𝐱‖²₂+‖𝐱‖₁, (35) where Ψ is the dictionary and 𝐲 is the invertible vectorization of the matrix 𝐀^(𝑇0+𝑇) observed at time 𝑇₀+𝑇; that is, 𝐲=vec(𝐀^(𝑇0+𝑇)). Because 𝐀^(𝑇0+𝑇)=∑_𝑟𝜎^(𝑇0_𝑟^+𝑇)𝐔^(𝑇0_𝑟^+𝑇) by using SVD and subsequently vec(𝐀^(𝑇0+𝑇))=∑_𝑟𝜎^(𝑇0_𝑟^+𝑇)vec(𝐔^(𝑇0_𝑟^+𝑇)), we will estimate 𝐀^(𝑇0+𝑇) in the form of vector and transform the low-rank matrix reconstruction problem into a traditional 𝑙₁-norm minimization problem in compressive sensing. We solve (35) by applying backtracking-based adaptive orthogonal matching pursuit (BAOMP) method, which incorporates a simple backtracking technique to detect the previously chosen atoms’ reliability and then deletes the unreliable atoms at each iteration [28]. Then we force that 𝑎^(𝑇0+𝑇)_𝑖𝑗, the element of the resulting matrix, is equal to 1 if 𝑎^(𝑇0+𝑇)_𝑖𝑗>1 or equal to −1 if 𝑎^(𝑇0+𝑇)_𝑖𝑗≤0, to ensure the elements coinciding with the value setting of relationships.

Furthermore, assume that we are given a sequence input samples 𝐘=[𝐲^(𝑇0+𝑇),𝐲^{(𝑇0+𝑇+1)},…,𝐲⁽𝑇₎], where 𝐲^(𝑡)=vec(𝐀^(𝑡)), 𝑇₀+𝑇≤𝑡≤𝑇, the task of the sign inference becomes to reconstruct the complete adjacency matrices 𝐀^(𝑡) one by one. Since the 𝐀^(𝑡) may contain some features which are not included in dictionary, it is necessary to add these features into the dictionary to increase the accuracy of the inference. However, the inferred matrix is not the original matrix exactly and consequently the unobserved relationships are not really known. In contrast, the observed adjacency matrix 𝐀^(𝑡) retains all existing relationships. For this reason, we only use 𝐀^(𝑡) to extract the features rather than the optimal solution of (35). We apply the extracting approach in Section 3.2 and add the complementary features into the dictionary. Note that this operation will continuously increase the scale of the dictionary while the samples keep inputting for inference; the dictionary selection approach proposed in Section 3.2 will be applied to compact the dictionary once the size of the dictionary exceeds a predetermined bound.

4. Numerical Experiment

In this section, we perform experiments on synthetic networks and show that our low-rank model and dictionary learning method outperform other methods on the task of the sign inference for dynamic signed networks. To ensure that our results are reliable, we conduct all experiments 20 times and average out the results from all of the trials.

To construct synthetic networks, we first consider a weakly balanced complete network 𝒢 whose adjacency tensor is 𝒜. The slide of 𝒜 at time 𝑡 is an adjacency matrix 𝐀^(𝑡) in the form of (3). In addition, only a few patterns of 𝐀^(𝑡) exist in 𝒜. The observation tensor 𝒜 is formed by sampling some entries from 𝒜. Concretely, we let the adjacency tensor 𝒜 of 𝒢 consist of 50 250×250 matrices of complete 4-weakly balanced structure. For the network 𝒢, four clusters are generated randomly. The size of each cluster is larger than 20 and the sum of the sizes is 250. We further assume that only a part of network relationships is observed by uniform sampling with probability 𝑝∈(0,1). It results in 𝑛²𝑝 entries being randomly sampled from 𝐀^(𝑡), where 𝑝 is the fraction of observed entries. We choose a set of matrices whose lost rates are from 0.05 to 0.55 and apply the approach proposed in Section 3.2 to select the dictionary Ψ.

With the dictionary Ψ and the given observed matrix 𝐀^(𝑡) at time 𝑡≥𝑇₀+𝑇, the task of the sign inference is achieved by solving (35). We use BAOMP to estimate the complete matrix 𝐀^(𝑡) and compare the performance of our approach to two state-of-the-art methods, alternating least square (ALS) [29] and singular value projection (SVP) [30], for the sign inference problem. Different from accuracy defined by the relative error on the observed set in [8], we utilize the similarity between the inferred matrix and the original one to indicate the accuracy of estimation. The definition of the similarity is |⟨𝐀^(𝑡),𝐀^(𝑡)⟩|/‖𝐀^(𝑡)‖_𝐹‖𝐀^(𝑡)‖_𝐹. We vary the lost-rate of the original matrix 𝐀^(𝑡) from 0.5 to 0.999 and plot the inference accuracy in Figure 3 (lost-rate: 0.5, 0.6, 0.7, 0.8, 0.9, 0.95, 0.96, 0.97, 0.98, 0.99, 0.995, and 0.999). Apparently, dictionary learning outperforms ALS and SVP. To present our result more clearly, we also use a visual expression in which the white pixels represent 1 and the black pixels represent −1. Figure 4 shows one example of the sign inference and we find that relationships and the clusters can almost be accurately estimated by our inference approach.

Figure 3 

Accuracy of sign inference algorithms on synthetic datasets. In general, we can see that dictionary learning outperforms ALS and SVP.

(a)

(b)

Figure 4 

An example of the sign inference. (a) illustrates the original matrix. Given the matrix with 98% lost-rate, (b) is the result inferred by dictionary learning method. The similarity of inferred matrix is 0.9347.

5. Conclusion

In this paper, we establish a low-rank tensor model for the dynamic weakly balanced signed networks. With this model, we first extract the feature pool and propose an approach to extract the compact dictionary from pool. To improve the performance of the selection approach, we derive the corresponding dual problem and introduce an accelerated thresholding algorithm to solve the dual problem. Consequently, the optimal solution of the primary problem can be readily obtained from optimizing the dual problem. In addition, combined with the compact dictionary generation method, the sign inference approach is provided for estimating missing relationships of the dynamic weakly balanced signed networks at a certain time slice. Also, the approach is endowed with the function of the dictionary updating if relationship statuses change.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Acknowledgments

This research was jointly supported by National 973 Program of China (no. 2013CB329204), National 863 Program of China (no. 2011AA01A104), National Natural Science Foundation of China (no. 61100206), Research Fund for Doctoral Program of Higher Education of China (no. 20120005130001), China, and Fund of State Key Laboratory of Information Photonics and Optical Communications (Beijing University of Posts and Telecommunications), China.