Nonnegative Signal Decomposition with Supervision

Abstract

This paper presents a novel algorithm to numerically decompose mixed signals in a collaborative way, given supervision of the labels that each signal contains. The decomposition is formulated as an optimization problem incorporating nonnegative constraint. A nonnegative data factorization solution is presented to yield the decomposed results. It is shown that the optimization is efficient and decreases the objective function monotonically. Such a decomposition algorithm can be applied on multilabel training samples for pattern classification. The real-data experimental results show that the proposed algorithm can significantly facilitate the multilabel image classification performance with weak supervision.

1. Introduction

Signal decomposition, which separates a mixed source signal into its constitutive pure components, is an important step in many practical engineering problems. For example, in image classification or speech recognition, a single training or test sample we collected usually contains multiple additive signals from several classes. The decomposition of the mixed signals can yield pure feature representations for better classes model training and recognition. Figure 1 illustrates the decomposition for an image containing three object classes. From the given mixed signal on the top, we aim to derive a specific representation for each of its associated labels, which reflects a corresponding pure region in the image. The decomposed representation can be more accurate and informative for describing object categories.

Figure 1 

Illustration of the proposed decomposition with an example of an image: given the top image with multiple labels, from its holistic feature representation shown right to it, for each of its associated labels, we aim to derive a label-specific representation, which reflects its actual region in the image. It is more informative to model individual labels compared to the holistic representation.

Unsupervised signal decomposition, for example, blind source separation in signal processing, is a fundamental problem which has been well explored in the past decades [1, 2]. Various methods have been proposed for different tasks, including Independent Component Analysis (ICA) for speech signal [3] and Null Space Pursuit (NSP) for electricity consumption and global surface temperature [4]. In image processing, a straightforward way to decompose the image representation into label-pure representations is manual labeling. It is tedious and impractical in most cases. Most conventional approaches to automatic image decomposition focused on clustering based on low-level visual cues, such as the bottom-up image segmentation in which pixels are locally grouped on the basis of their appearances [5] and the top-down image parsing in which primitives (e.g., rectangles, sketches, and edges) are correlated based on a few grammar rules [6]. However, all these methods have not taken the supervision information into consideration. In a practical case, online images from internet are usually associated with the textual labels on the webpage, which can be utilized to guide the decomposition.

In this paper, we focus on the signal decomposition problem with label supervision; that is, given a set of mixed signals each with class labels it contains, we decompose each signal to the specific labels. To this aim; the fundamental principle in pattern classification is utilized that signals of the same label should be close to each other while those of different labels should be relatively separated in feature space. Based on this assumption, a novel algorithm to numerically decompose mixed signals with labels in a collaborative way is proposed. The decomposition is formulated as an optimization problem incorporating nonnegative constraint for practical consideration, due to the nonnegative property of most real-life signals. Furthermore, we present a nonnegative data factorization solution, which is shown to be efficient and can decrease the objective function monotonically. Such a decomposition can be applied on multi-label training samples for pattern classification. The real-data experimental results show that the proposed algorithm can significantly facilitate the multi-label image classification with weak supervision.

2. Notation

In the following, we will use these rules to facilitate presentation: for any matrix 𝐀, 𝐀_𝑖 means the 𝑖th row vector of 𝐀, its corresponding lowercase version 𝐚_𝑖 means the 𝑖th column vector of 𝐀, and 𝐀_𝑖𝑗 denotes the element of 𝐀 at the 𝑖th row and 𝑗th column. Throughout this paper we will also use the following notation: 𝑁: number of total training samples, 𝑁: number of valid tags in the training set, 𝐥_𝑖: label vector of the 𝑖th training sample,𝐯_𝑖: mixed signal representation vector of the 𝑖th training sample, 𝐕: mixed signal representations of all training samples (𝐕=[𝐯₁,𝐯₂,…,𝐯_𝑁]),𝐫^𝑐_𝑖: decomposed signal representation of the 𝑐th label in 𝑖th training sample,𝐫_𝑖: decomposed representation of the 𝑖th valid tag in the full training set, 𝐑: decomposed signal representations from all training samples (𝐑=[𝐫₁,𝐫₂𝑁,…,𝐫]),𝛼_𝑖𝑗: coefficient of normalized 𝐫_𝑗 vector in reconstructing 𝐯_𝑖, 𝐇: reconstruction coefficients matrix (𝐇_𝑖𝑗=𝛼_𝑗𝑖), 𝐘: membership matrix denoting relation between tags and samples, defined in (5).

Here a label represents a class, and a tag is a specific label instance. For a specific sample 𝑖∈{1,…,𝑁}, its signal representation can be calculated as a 𝐾-dimensional normalized vector 𝐯_𝑖 with ‖𝐯_𝑖‖₁=1, where ‖⋅‖₁ represents the ℓ¹-norm operator. The labels of this sample are denoted in a vector 𝐥_𝑖∈ℛ^𝐶 with only 0 or 1 elements: 𝑙^𝑐_𝑖=1iflabel𝑐iscontainedinsample𝑖,0ifthereisnolabel𝑐insample𝑖, (1) where 𝐶 denotes the number of labels. Let a 𝐾-dimensional vector 𝐫^𝑐_𝑖 denote the representation of the corresponding part for the 𝑐th label in the 𝑖th sample. For a set of training samples, we collect the representations for all nonzero label instances, that is, “tag”s, and reindex as vectors {𝐫_𝑖}, 𝑖∈{1,…,𝑁}, where 𝑁 is the number of valid tags in the set obtained by 𝑁=∑^𝑁_𝑖=1∑^𝐶_𝑐=1𝑙^𝑐_𝑖. Note that {𝐫_𝑖} are unknown as the objective to be computed.

3. Problem Formulation

In this section, the above decomposition problem is solved by nonnegative data factorization. The presented solution appears to be efficient. Furthermore, the monotonic decrease of the objective function can be proved regarding the convergence properties.

With 𝑁 training samples, to formulate the decomposition problem, we use such representations as follows: a matrix 𝐕=[𝐯₁,𝐯₂,…,𝐯_𝑁] to enclose the input holistic signal representations in columns; a 𝐾×𝑁 matrix 𝐑=[𝐫₁,𝐫₂𝑁,…,𝐫] to enclose the decomposed representations for all the tags, which are also arranged in columns, where 𝑁 is the total number of tags in the training set. The decomposition problem formulation consists of the following two parts.

(1) The first part is minimizing the error of reconstruction from the decomposed representations to the additive signal representations; that is, for all 𝑖∈{1,…,𝑁}, min𝐯_𝑖−_𝑗𝐫_𝑗²,ifthe𝑗thtagisassociatedwiththe𝑖thsample. (2)

Considering that, in the above formulation, {𝐫_𝑗} are not normalized, we introduce the coefficients and yield: min𝐯_𝑖−_𝑗𝛼_𝑖𝑗𝐫_𝑗²,ifthe𝑗thtagisassociatedwiththe𝑖thsample, (3)

where 𝛼_𝑖𝑗 represents the coefficient of 𝐫_𝑗 in the reconstruction of 𝐯_𝑖. Using matrix form and an 𝑁×𝑁 matrix 𝐇 with elements 𝐇_𝑖𝑗=𝛼_𝑗𝑖 to enclose the information of coefficients, the reconstruction error minimization can be formulated as argmin_𝐑,𝐇‖𝐕−𝐑𝐇‖².

Note that a mixed holistic representation should be reconstructed only from representations of the associated labels. Therefore, some elements of 𝐇 matrix must be 0; that is, 𝐇_𝑖𝑗=0,ifthe𝑖thtagisnotassociatedwiththe𝑗thsample. (4)

This information can be derived from the training labels. Here we release the 0-elements constraint for 𝐇 matrix and introduce a membership matrix 𝐘: 𝐘_𝑖𝑗=1if𝐫_𝑖isassociatedto𝐯_𝑗,0otherwise. (5)

Then the minimization problem becomes argmin_𝐑,𝐇‖𝐕−𝐑(𝐇•𝐘)‖². (6)

Note that 𝐘 matrix will be abandoned later in the proposed solution when the 0-elements constraint for 𝐇 matrix is already considered.

(2) The second part is minimizing intralabel distance between the decomposed signal representations of the same label. Here we use 𝐒 matrix to denote the intralabel information with 𝐒_𝑖𝑗=1if𝑖≠𝑗,𝐫_𝑖and𝐫_𝑗correspondtothesamelabel;0otherwise. (7)

To force the decomposed representations belonging to the same label to be similar to each other, we will minimize ∑_𝑖𝑗‖𝐫_𝑖−𝐫_𝑗‖²𝐒_𝑖𝑗 (here for convenience {𝐫_𝑖} are assumed as normalized, which will be described later), and we can derive in the matrix form _𝑖𝑗𝐫_𝑖−𝐫_𝑗²𝐒_{𝑖𝑗𝑖𝑗}𝐫^𝑇_𝑖𝐫_𝑖+𝐫^𝑇_𝑗𝐫_𝑗−2𝐫^𝑇_𝑖𝐫_𝑗𝐒_𝑖𝑗=2_𝑖𝑗𝐫^𝑇_𝑖𝐫_𝑖−2_𝑖𝑗𝐫^𝑇_𝑖𝐒_𝑖𝑗𝐫_𝑗=2𝑇𝑟𝐑𝐃𝐑^𝑇−2𝑇𝑟𝐑𝐒𝐑^𝑇=2𝑇𝑟𝐑𝐋𝐑^𝑇, (8) where 𝐃𝐋=𝐃−𝐒,_𝑖𝑗=_𝑘𝐒_𝑖𝑘if𝑖=𝑗,0otherwise. (9)

Representing the context among the decomposed local representations in a graph as shown in Figure 2, 𝐋 matrix is therefore the Laplacian matrix for the graph, based on which graph preserving energy [7] (similar to (8)) can be derived.

Figure 2 

An illustration showing the graph based representation which encodes the context among decomposed signals of multiple labels: each node represents the decomposed representation to be obtained for a specific tag; tags belonging to the same label are linked with edge weight 1, while edge weights between those of different labels are set to 0.

Note that 𝐑 may not be normalized. When measuring distance between different {𝐫_𝑖} in the graph based representation, we introduce 𝐐 for normalizing 𝐑. Since 𝐕 is known to be normalized in column, by minimizing the reconstruction error with (6), 𝐑(𝐇•𝐘) can be considered to be approximately normalized. Thus 𝐐 is designed as a diagonal matrix: 𝐐=diag𝐘𝐇^𝑇𝐞, (10) where 𝐞=[1,…,1,…,1] is a 𝑁 dimensional vector; that is, 𝐐_𝑖𝑗=_𝑘𝐘_𝑖𝑘𝐇_𝑖𝑘if𝑖=𝑗;0otherwise. (11)

Multiplying {𝐫_𝑖} with {𝐐_𝑖𝑖}, respectively, we can see that 𝐑𝐐 can be approximately columnwise normalized. If we take into account the normalization factor 𝐐, the second factor of the objective function minimizing intralabel distance is then yielded as min𝑇𝑟𝐑𝐐𝐋𝐐^𝑇𝐑^𝑇. (12)

Based on the above two parts, the presented decomposition problem can be formulated as a regularized nonnegative data factorization problem as argmin_𝐑,𝐇‖𝐕−𝐑(𝐇•𝐘)‖²+𝜆𝑇𝑟𝐑𝐐𝐋𝐐^𝑇𝐑^𝑇s.t.𝐑,𝐇≥0, (13) where 𝜆 is a weight to be empirically set in the experiments. The constraints show that 𝐑 and 𝐇 should be nonnegative to satisfy the reconstruction assumption, since in many real applications only when the signal representations and the reconstruction coefficients are nonnegative, can the proposed decomposition be physically meaningful. The decomposed label representations 𝐑 can be obtained from the fission of 𝐕 by solving (13). The solution will be given in the following. In the formulation, the given 𝐕 is normalized, but there is no constraint that 𝐑 or 𝐇 matrix should be normalized. In the following optimization process we will normalize 𝐇.

4. Optimizing Solution

The problem of (13) falls into the framework of nonnegative matrix factorization (NMF) [8] for its nonnegative constraints. NMF has shown its effectiveness in practical signal processing for image classification [9]. Here similar to the optimization strategy adopted in the previous work by Lee and Seung [8], we optimize the objective function of (13) in an iterative way with a multiplicative updating rule, which guarantees the non-negativity. For initialization, to get an original guess of the target decomposed representations, we initialize 𝐫^𝑐_𝑖=𝐯_𝑖•(𝐯^𝑐/∑^𝐶_{𝑐′=1,𝑙𝑐′𝑖=1}𝐯^𝑐′). Note that “•” stands for dot product in this paper, and 𝐯^𝑐 is the average representation of 𝑐th class from training samples associated with label 𝑐.

Most iterative procedures for solving high-order optimization problems transform the original intractable problem into a set of tractable subproblems and finally obtain the convergence to a local optimum. The proposed iterative procedure also follows this philosophy and optimizes 𝐇 and 𝐑 alternately. The proposed iteration rules can be proved to monotonically decrease the objective function value, and the theoretical proof is given in Section 5. In fact, although stationary is the necessary condition for achieving local minima, it is difficult to justify that any limit points are the stationary point. Lin [10] has slightly modified the original iterative algorithm of NMF to achieve the convergence safely. Their modification, however, will increase the computational complexity while achieving his similar performance, as reported in [10]. Therefore, in this work, we do not adopt that the modification and experiment results show that the proposed solution could usually converge to local minima. We give the iterative procedure as follows.

(1)  Optimize 𝐇 for Given 𝐑. For a fixed 𝐑, the objective function in (13) with respect to 𝐇 can be written as 𝐹(𝐇)=‖𝐕−𝐑𝐇‖²+𝜆𝑇𝑟𝐑diag𝐘𝐇^𝑇𝐞𝐋diag𝐘𝐇^𝑇𝐞𝐑^𝑇s.t.𝐇≥0. (14) Let Φ_𝑖𝑗 be the Lagrange multiplier for constraint 𝐇_𝑖𝑗≥0; the Lagrange factor is ∑_𝑖𝑗Φ_𝑖𝑗𝐇_𝑖𝑗=𝑇𝑟(Φ𝐇) and the Lagrange function ℒ is ℒ=‖𝐕−𝐑𝐇‖²+𝜆𝑇𝑟𝐑diag𝐘𝐇^𝑇𝐞𝐋diag𝐘𝐇^𝑇𝐞𝐑^𝑇+𝑇𝑟Φ𝐇^𝑇=𝑇𝑟𝐕𝐕^𝑇−2𝑇𝑟𝐕𝐇^𝑇𝐑^𝑇+𝑇𝑟𝐑𝐇𝐇^𝑇𝐑^𝑇+𝜆𝑇𝑟𝐑diag𝐘𝐇^𝑇𝐞𝐋diag𝐘𝐇^𝑇𝐞𝐑^𝑇+𝑇𝑟Φ𝐇^𝑇. (15)

The partial derivation of ℒ with respect to 𝐇 is 𝜕ℒ𝜕𝐇=−2𝐑^𝑇𝐕+2𝐑^𝑇𝐑𝐇+2𝜆𝐋𝐑^𝑇𝐑𝐞𝐞^𝑇𝐇𝐘^𝑇𝐘+Φ, (16) where we have used the following deductions: 𝑂=𝐑diag𝐘𝐇^𝑇𝐞𝐋diag𝐘𝐇^𝑇𝐞𝐑^𝑇=𝐑𝐋𝐘𝐇^𝑇𝐞𝐞^𝑇𝐇𝐘^𝑇𝐑^𝑇=_𝑖𝐑_𝑖𝐋𝐘𝐇^𝑇𝐞𝐞^𝑇𝐇𝐘^𝑇𝐑^𝑇_𝑖,𝜕𝑂𝜕𝐇=_𝑖𝐋^𝑇𝐑^𝑇_𝑖𝐑_𝑖2𝐞𝐞^𝑇𝐇𝐘^𝑇𝐘=2𝐋𝐑^𝑇𝐑𝐞𝐞^𝑇𝐇𝐘^𝑇𝐘. (17)(18)

Using the KarushKuhnTucker (KKT) condition [11] Φ_𝑖𝑗𝐇_𝑖𝑗=0, from (16), the following equations can be obtained for 𝐇_𝑖𝑗: −𝐑^𝑇𝐕_𝑖𝑗𝐇_𝑖𝑗+𝐑^𝑇𝐑𝐇_𝑖𝑗𝐇_𝑖𝑗+𝜆𝐋⋅𝐑^𝑇𝐑𝐞𝐞^𝑇𝐇𝐘^𝑇𝐘_𝑖𝑗𝐇_𝑖𝑗=−𝐑^𝑇𝐕_𝑖𝑗𝐇_𝑖𝑗+𝐑^𝑇𝐑𝐇_𝑖𝑗𝐇_𝑖𝑗+𝜆𝐃𝐑^𝑇𝐑𝐞𝐞^𝑇𝐇𝐘^𝑇𝐘_𝑖𝑗𝐇_𝑖𝑗−𝜆𝐒𝐑^𝑇𝐑𝐞𝐞^𝑇𝐇𝐘^𝑇𝐘_𝑖𝑗𝐇_𝑖𝑗=0, (19) which leads to the update rule 𝐇_𝑖𝑗⟵𝐇_𝑖𝑗𝐑^𝑇𝐕+𝜆𝐒𝐑^𝑇𝐑𝐞𝐞^𝑇𝐇𝐘^𝑇𝐘_𝑖𝑗𝐑^𝑇𝐑𝐇+𝜆𝐃𝐑^𝑇𝐑𝐞𝐞^𝑇𝐇𝐘^𝑇𝐘_𝑖𝑗. (20)

Remember that 𝐇^(𝑡)_𝑖𝑗=0 if 𝐘_𝑖𝑗=0, where 𝑡 means the 𝑡th iteration. Note that it could be satisfied by initializing 𝐇⁽⁰⁾_𝑖𝑗=0 if 𝐘_𝑖𝑗=0; therefore 𝐘 actually can be neglected without any influence. It is obvious that the updated 𝐇^(𝑡+1) is still nonnegative if the matrices 𝐑 and 𝐇^(𝑡) are nonnegative. After getting 𝐇^(𝑡+1), we normalize the column vectors of 𝐇^(𝑡+1) and consequently convey the norms to the basis matrix; namely, 𝐑⟵𝐑diag𝐘𝐇^𝑇𝐞,1𝐇⟵𝐇diag𝐇^𝑇𝐞. (21)(22) The above updating of 𝐑 and 𝐇 does not change the value of the objective function in (13).

(2)  Optimize 𝐑 for Given Normalized 𝐇. Then based on the normalized 𝐇 in (22), the objective function in (13) with respect to 𝐑 for given 𝐇 can be written as 𝐹(𝐑)=‖𝐕−𝐑(𝐇•𝐘)‖²+𝜆𝑇𝑟𝐑𝐋𝐑^𝑇=‖𝐕−𝐑𝐇‖²+𝜆𝑇𝑟𝐑𝐋𝐑^𝑇. (23)

Let Ψ_𝑖𝑗 be the Lagrange multiplier for constraint 𝐑_𝑖𝑗≥0, the Lagrange function ℒ is ℒ=‖𝐕−𝐑𝐇‖²+𝜆𝑇𝑟𝐑𝐋𝐑^𝑇+𝑇𝑟Ψ𝐑^𝑇=𝑇𝑟𝐕𝐕^𝑇−2𝑇𝑟𝐕𝐇^𝑇𝐑^𝑇+𝑇𝑟𝐑𝐇𝐇^𝑇𝐑^𝑇+𝜆𝑇𝑟𝐑𝐋𝐑^𝑇+𝑇𝑟Ψ𝐑^𝑇. (24)

The partial derivation of ℒ with respect to 𝐑 is 𝜕ℒ𝜕𝐑=−2𝐕𝐇^𝑇+2𝐑𝐇𝐇^𝑇+2𝜆𝐑𝐋+Ψ. (25)

Using the KKT condition Ψ_𝑖𝑗𝐑_𝑖𝑗=0, from (25), we obtain the following equations for 𝐑_𝑖𝑗: −𝐕𝐇^𝑇_𝑖𝑗𝐑_𝑖𝑗+𝐑𝐇𝐇^𝑇_𝑖𝑗𝐑_𝑖𝑗+𝜆(𝐑𝐋)_𝑖𝑗𝐑_𝑖𝑗=0, (26) which leads to the following update rule: 𝐑_𝑖𝑗⟵𝐑_𝑖𝑗𝐕𝐇^𝑇+𝜆𝐑𝐒_𝑖𝑗𝐑𝐇𝐇^𝑇+𝜆𝐑𝐃_𝑖𝑗. (27)

5. Convergence Proof of the Proposed Solution

Definition 1. Function 𝐺(𝐀,𝐀^′) is an auxiliary function for function 𝐹(𝐀) if the conditions 𝐺𝐀,𝐀^′≥𝐹(𝐀),𝐺(𝐀,𝐀)=𝐹(𝐀) (28) are satisfied.

Theorem 2. If 𝐺 is an auxiliary function, then 𝐹 is nonincreasing under the update 𝐀^(𝑡+1)=argmin_𝐀𝐺𝐀,𝐀^(𝑡), (29) where 𝑡 means the 𝑡th iteration.

5.1. Convergence of the Update Rule for H

Firstly let us go back to (14) and consider one element 𝐇_𝑎𝑏 of matrix 𝐇. Using 𝐹_𝑎𝑏 to denote the part of 𝐹(𝐇) relevant to 𝐇_𝑎𝑏, we can get 𝐹^′_𝑎𝑏=−2𝐑^𝑇𝐕+2𝐑^𝑇𝐑𝐇+2𝜆𝐋𝐑^𝑇𝐑𝐞𝐞^𝑇𝐇𝐘^𝑇𝐘_𝑎𝑏;𝐹^′′_𝑎𝑏=2𝐑^𝑇𝐑_𝑎𝑎+2𝜆[𝐋𝐑^𝑇𝐑𝐞𝐞^𝑇_𝑎𝑎𝐘^𝑇𝐘_𝑏𝑏. (30)

The auxiliary function of 𝐹_𝑎𝑏 is designed as 𝐺𝐇_𝑎𝑏,𝐇^(𝑡)_𝑎𝑏=𝐹_𝑎𝑏𝐇^(𝑡)_𝑎𝑏+𝐹^′_𝑎𝑏𝐇^(𝑡)_𝑎𝑏𝐇_𝑎𝑏−𝐇^(𝑡)_𝑎𝑏+𝐑^𝑇𝐑𝐇_𝑎𝑏+𝜆𝐃𝐑^𝑇𝐑𝐞𝐞^𝑇𝐇𝐘^𝑇𝐘_𝑎𝑏𝐇^𝑇_𝑎𝑏×𝐇_𝑎𝑏−𝐇^(𝑡)_𝑎𝑏². (31)

Lemma 3. Equation (32) is an auxiliary function for 𝐹_𝑎𝑏, the part of 𝐹(𝐇) only relevant to 𝐇_𝑎𝑏.

Proof. Since 𝐺(𝐇_𝑎𝑏,𝐇_𝑎𝑏)=𝐹_𝑎𝑏(𝐇_𝑎𝑏) is obvious, we need only show that 𝐺(𝐇_𝑎𝑏,𝐇^(𝑡)_𝑎𝑏)≥𝐹_𝑎𝑏(𝐇_𝑎𝑏).
To this aim, we compare the Taylor series expansion of 𝐹_𝑎𝑏(𝐇_𝑎𝑏)𝐹_𝑎𝑏𝐇_𝑎𝑏=𝐹_𝑎𝑏𝐇^(𝑡)_𝑎𝑏+𝐹^′_𝑎𝑏𝐇^(𝑡)_𝑎𝑏𝐇_𝑎𝑏−𝐇^(𝑡)_𝑎𝑏+12𝐹^′′_𝑎𝑏𝐇_𝑎𝑏−𝐇^(𝑡)_𝑎𝑏² (32) with (32) to find that 𝐺(𝐇_𝑎𝑏,𝐇^(𝑡)_𝑎𝑏)≥𝐹_𝑎𝑏(𝐇_𝑎𝑏) is equivalent to 𝐑^𝑇𝐑𝐇_𝑎𝑏+𝜆𝐃𝐑^𝑇𝐑𝐞𝐞^𝑇𝐇𝐘^𝑇𝐘_𝑎𝑏𝐇^𝑇_𝑎𝑏≥𝐑^𝑇𝐑_𝑎𝑎+𝜆𝐋𝐑^𝑇𝐑𝐞𝐞^𝑇_𝑎𝑎𝐘^𝑇𝐘_𝑏𝑏, (33)
where 𝐑^𝑇𝐑𝐇_𝑎𝑏=_𝑘𝐑^𝑇𝐑_𝑎𝑘𝐇^(𝑡)_𝑘𝑏≥𝐑^𝑇𝐑𝐇_𝑎𝑎𝐇^(𝑡)_𝑎𝑏,𝜆𝐃𝐑^𝑇𝐑𝐞𝐞^𝑇𝐇𝐘^𝑇𝐘_𝑎𝑏=𝜆_𝑘𝐃𝐑^𝑇𝐑𝐞𝐞^𝑇_𝑎𝑘𝐇_𝑘𝑏𝐘^𝑇𝐘_𝑏𝑏≥𝜆𝐃𝐑^𝑇𝐑𝐞𝐞^𝑇_𝑎𝑎𝐇_𝑎𝑏𝐘^𝑇𝐘_𝑏𝑏≥𝜆(𝐃−𝐒)𝐑^𝑇𝐑𝐞𝐞^𝑇_𝑎𝑎𝐇_𝑎𝑏𝐘^𝑇𝐘_𝑏𝑏=𝜆𝐋𝐑^𝑇𝐑𝐞𝐞^𝑇_𝑎𝑎𝐇_𝑎𝑏𝐘^𝑇𝐘_𝑏𝑏. (34)
Thus, (33) holds and 𝐺(𝐇_𝑎𝑏,𝐇^(𝑡)_𝑎𝑏)≥𝐹_𝑎𝑏(𝐇_𝑎𝑏).

Lemma 4. Equation (20) could be obtained by minimizing the auxiliary function 𝐺(𝐇_𝑎𝑏,𝐇^(𝑡)_𝑎𝑏).

Proof. By setting 𝜕𝐺(𝐇_𝑎𝑏,𝐇^(𝑡)_𝑎𝑏)/𝜕𝐇_𝑎𝑏=0, we have 𝜕𝐺𝐇_𝑎𝑏,𝐇^(𝑡)_𝑎𝑏𝜕𝐇_𝑎𝑏=𝐹^′_𝑎𝑏𝐇^(𝑡)_𝑎𝑏+2𝐑^𝑇𝐑𝐇+𝜆𝐃𝐑^𝑇𝐑𝐞𝐞^𝑇𝐘^𝑇𝐘_𝑎𝑏𝐇^(𝑡)_𝑎𝑏×𝐇_𝑎𝑏−𝐇^(𝑡)_𝑎𝑏=0. (35) Then the update rule for 𝐇 in (20) can be obtained.

5.2. Convergence of the Update Rule for R

Let us go back to (23) and consider any element 𝐑_𝑎𝑏 of matrix 𝐑. Now 𝐹_𝑎𝑏 is used to denote the part of 𝐹(𝐑) relevant to 𝐑_𝑎𝑏. It is easy to check that 𝐹^′_𝑎𝑏=−2𝐕𝐇^𝑇+2𝐑𝐇𝐇^𝑇+2𝜆𝐑𝐋_𝑎𝑏,𝐹^′′_𝑎𝑏=2𝐇𝐇^𝑇_𝑏𝑏+2𝜆𝐋_𝑏𝑏. (36)

The auxiliary function of 𝐹_𝑎𝑏 is designed as 𝐺𝐑_𝑎𝑏,𝐑^(𝑡)_𝑎𝑏=𝐹_𝑎𝑏𝐑^(𝑡)_𝑎𝑏+𝐹^′_𝑎𝑏𝐑^(𝑡)_𝑎𝑏𝐑_𝑎𝑏−𝐑^(𝑡)_𝑎𝑏+𝐑𝐇𝐇^𝑇_𝑎𝑏+𝜆(𝐑𝐃)_𝑎𝑏𝐑^𝑇_𝑎𝑏𝐑_𝑎𝑏−𝐑^(𝑡)_𝑎𝑏². (37)

Lemma 5. Equation (37) is an auxiliary function for 𝐹_𝑎𝑏, the part of 𝐹(𝐑) only relevant to 𝐑_𝑎𝑏.

Proof. Since 𝐺(𝐑_𝑎𝑏,𝐑_𝑎𝑏)=𝐹_𝑎𝑏(𝐑_𝑎𝑏) is obvious, we need only show that 𝐺(𝐑_𝑎𝑏,𝐑^(𝑡)_𝑎𝑏)≥𝐹_𝑎𝑏(𝐑_𝑎𝑏). To do this, we compare the Taylor series expansion of 𝐹_𝑎𝑏(𝐑_𝑎𝑏)𝐹_𝑎𝑏𝐑_𝑎𝑏=𝐹_𝑎𝑏𝐑^(𝑡)_𝑎𝑏+𝐹^′_𝑎𝑏𝐑^(𝑡)_𝑎𝑏𝐑_𝑎𝑏−𝐑^(𝑡)_𝑎𝑏+12𝐹^′′_𝑎𝑏𝐑_𝑎𝑏−𝐑^(𝑡)_𝑎𝑏² (38) with (37), and 𝐺(𝐑_𝑎𝑏,𝐑^(𝑡)_𝑎𝑏)≥𝐹_𝑎𝑏(𝐑_𝑎𝑏) is equivalent to 𝐑𝐇𝐇^𝑇_𝑎𝑏+𝜆(𝐑𝐃)_𝑎𝑏𝐑^𝑇_𝑎𝑏≥𝐇𝐇^𝑇_𝑏𝑏+𝜆𝐋_𝑏𝑏, (39) where 𝐑𝐇𝐇^𝑇_𝑎𝑏=_𝑘𝐑^(𝑡)_𝑎𝑘𝐇𝐇^𝑇_𝑘𝑏≥𝐑^(𝑡)_𝑎𝑏𝐇𝐇^𝑇_𝑏𝑏,𝜆(𝐑𝐃)_𝑎𝑏=𝜆_𝑘𝐑^(𝑡)_𝑎𝑘𝐃_𝑘𝑏≥𝜆𝐑^(𝑡)_𝑎𝑏𝐃_𝑏𝑏≥𝜆𝐑^(𝑡)_𝑎𝑏(𝐃−𝐒)_𝑏𝑏=𝜆𝐑^(𝑡)_𝑎𝑏𝐋_𝑏𝑏. (40)
Thus, (39) holds and 𝐺(𝐑_𝑎𝑏,𝐑^(𝑡)_𝑎𝑏)≥𝐹_𝑎𝑏(𝐑_𝑎𝑏).

Lemma 6. Equation (27) could be obtained by minimizing the auxiliary function 𝐺(𝐑_𝑎𝑏,𝐑^(𝑡)_𝑎𝑏).

Proof. By setting 𝜕𝐺(𝐑_𝑎𝑏,𝐑^(𝑡)_𝑎𝑏)/𝜕𝐑_𝑎𝑏=0, we can get 𝜕𝐺𝐑_𝑎𝑏,𝐑^(𝑡)_𝑎𝑏𝜕𝐑_𝑎𝑏=𝐹^′_𝑎𝑏𝐑^(𝑡)_𝑎𝑏+2𝐑𝐇𝐇^𝑇+𝜆𝐑𝐃_𝑎𝑏𝐑^(𝑡)_𝑎𝑏𝐑_𝑎𝑏−𝐑^(𝑡)_𝑎𝑏=0. (41) Then the update rule for 𝐑 in (27) can be obtained.

6. Experiment

A typical application of the proposed decomposition in real life is as follows: for a set of images related to a scene such as campus and each associated with multiple labels, we can decompose the image feature representations to pure label representations, to provide better label samples for modeling object categories, which therefore can facilitate object recognition, detection, and so on. It leads to a useful tool for automatic labeling of online images or personal photo album. There are two main reasons that the proposed decomposition is necessary here. First, with limited training samples, it is difficult to learn good label models using the original image presentations directly without decomposition. Since each sample is a mixture of several labels and the background for each label is various, it is better to decompose the image representation to labels optimally. Second, instances of the same label related to a scene usually have short intrasubspace distance, which supports the assumption of the proposed decomposition and guarantees the effectiveness.

To demonstrate a practical application of the proposed decomposition in automatic image annotation, we manually crawled images of “KAIST campus” from the Flickr photo sharing website [12], excluded the irrelevant images, and obtained 183 images (we share the data at http://rcv.kaist.ac.kr/~tengli/Teng_resource.html). They are labeled with nine concepts: “sky,” “plant (grass/tree/flower),” “water,” “ground,” “man-made (building/sign/road),” “car/bus,” “bicycle,” “person,” and “animal.” 150 images are used for training samples, which are used for class models training and can be applied to the proposed decomposition, and the rest are used for testing. Labels are highly mixed in the images.

In the implementation, the images and regions are represented using the Bag-of-Words (BoW) feature [13], where the number of visual words is set as 500 in the experiments for the tradeoff of classification accuracy and computational cost, and the feature dimension is therefore 500. With the decomposed label representations from the 𝑁 training images, that is, normalized 𝐑 matrix, a typical classifier, support vector machine (SVM) [14] is applied to learn label models and classify the testing samples. In the experiment of multilabel image classification, we adopted the area under ROC curve (AUC) measurement [15].

Table 1 gives the performance comparison between the image classification adopting the proposed nonnegative signal decomposition and the classic BoW method [13]. To further examine the performance, except for the well-known SVM, we also apply a recent popular Sparse Representation Classification (SRC) [16] as the classifier. To train using the original BoW image representations without applying decomposition yield AUC as low as around 0.5, whereas, to train label models using the decomposed results, the AUCs on the test samples are 0.77 and 0.76, in combination with SVM and SRC, respectively. Figure 3 gives the multi-label classification results for some sample images of KAIST campus Flickr dataset. The results appear to be promising.

Table 1 

Comparison of classification performance (AUC) on the KAIST campus images.

Figure 3 

Exemplar results of multilabel image classification on the KAIST campus images using the proposed decomposed training samples: corresponding regions of the correctly detected labels are marked in the images.

7. Conclusion

A new technique for the nonnegative signals decomposition considering the supervised additive label information has been developed in this paper, with the assumption that intralabel signals should be relatively closer to each other than interlabel signals. The nonnegative constraint makes it applicable for many practical cases. We formulated it as an optimization problem and proposed an efficient iterative solution based on nonnegative data factorization. Convergence analysis showed that the solution guarantees monotonic decrease of the objective function.