Abstract
The removal of mixed Gaussian-impulse noise plays an important role in many areas, such as remote sensing. However, traditional methods may be unaware of promoting the degree of the sparsity adaptively after decomposing into low rank component and sparse component. In this paper, a new problem formulation with regular spectral -support norm and regular -support norm is proposed. A unified framework is developed to capture the intrinsic sparsity structure of all two components. To address the resulting problem, an efficient minimization scheme within the framework of accelerated proximal gradient is proposed. This scheme is achieved by alternating regular -shrinkage thresholding operator. Experimental comparison with the other state-of-the-art methods demonstrates the efficacy of the proposed method.
1. Introduction
Image restoration [1–4] attempts to recover a clear image from the observations of real scenes. As a fundamental procedure, it has been applied to various application areas, such as image fusion [5] and action recognition [6]. However, typically, the noise characteristics of imaging camera is completely or partially unknown. Among these, the removal of mixed noise has not been investigated because the noise model is not easy to establish accurately.
Recently, a patch based method [7] for video restoration has attracted much attention [8–10]. This method also is extended to video in-painting for archived films. However, the mechanisms of modeling the sparsity level of the grouping patches remain unclear.
To deal with the lack of adaptivity in sparsity level [7], a robust video restoration algorithm is proposed. The main idea of the proposed method is to model the sparsity levels of the low rank component by regular spectral -support norm and sparse component by regular -support norm. Specially, a new problem formulation is presented, where the objective function is minimized under an upper bound constraint on the regularization term. However, it is not easy to solve the resulting problem. Some recent progress [11] in the theory of optimization on iterative shrinkage thresholding method is considered. And, an efficient alternating minimization scheme is proposed to solve the new objective.
1.1. Related Works
Recently, the problem of denoising image corrupted by mixed Gaussian-impulse noise has been studied in many different contexts [8–10, 12, 13]. These methods fall into three categories: variational methods [9, 12], sparse representation [8, 10], and patch based method [7].
Variational methods are a new class of the solutions to promote edge-preservation, such as total variation [14]. These methods first utilized some spatial filters to detect and remove the corrupted pixels, for example, adaptive center-weighted median filter [15] (ACWMF) or rank order absolute differences [16] (ROAD) detector. In [12], Cai et al. employed Mumford-Shah regularization term to encourage sparsity in gradient domain. In [9], Rodríguez et al. presented a novel optimization method for the generalized total variation regularization method. It can be seen that the denoised performance of these methods relies on the detection for the damaged candidates. The adaptivity of sparsity level of the regularization terms has not been investigated carefully.
Sparse representation based methods have been extended to this problem. In the main idea of this scheme, it is assumed that the signal can be described by linear combination of a spare number of elements or atoms of an overcomplete dictionary. In [8], an efficient image reconstruction method by posing norm on the error, and norm on image patches in learned dictionary, was proposed. In [10], Filipovic and Jukic reformulated a new problem formulation by enforcing - sparsity constraints. The resulting problem is solved by a mixed soft-hard thresholding method. However, it should be noted that these methods are time-consuming.
Patch based method is proven to be a state-of-the-art denoising scheme. In [7], Ji et al. approximated the patch stack by reformulating the problem as a low rank matrix completion problem. Despite its efficacy, one of the limitations of patch based method in [7] is that the degree of sparsity has not been considered carefully. When the underlying sparsity level is unknown, we may obtain a bias estimate, considerably. To alleviate these issues in a unified formulation, a new problem formulation is proposed.
1.2. Contributions
The main idea of this paper is to deal with the weakness of the approach in [7]. Existing methods, such as norm and trace norm, can not promote the sparsity level of all two components adaptively. The details or local fine content can not be represented and described well. To deal with these issues, a new problem formulation incorporating correlated and adaptive sparsity is proposed.
Our contributions can be summarized as follows.(1)A new problem formulation to model the sparsity level of the patches is proposed. A new norm extended from -support norm and ordered norm is presented.(2)An efficient minimization scheme with regular -shrinkage thresholding operator is proposed, which is based on the optimization framework of accelerated proximal gradient (APG) method.(3)Numerical experiments, compared to other state-of-the-art methods, demonstrate that the proposed method outperforms the related restoration methods.
1.3. Organization
The remainder of this paper is presented as follows. In Section 2, some basic notations are provided. In Section 3, a detailed description about the proposed objective function is given. In Section 4, an efficient minimization scheme within the framework of APG is proposed. Then, some experiments are conducted to validate the effectiveness of the proposed method in Section 5. Finally, we conclude the paper in Section 6.
2. Preliminaries
There are some notations presented for the simplicity of discussions. Frobenius norm and norm of a matrix are defined by and , respectively. For a scalar , the shrinkage operator [17] for norm minimization problem is defined as follows: where is a signum function; calculates the absolute value.
Assuming that is of rank , the singular value decomposition (SVD) of with nonnegative singular values is defined by , where denotes a diagonal matrix with the singular values. Based on the SVD computation, the nuclear norm is defined in the following way:where is the th largest singular value of . A solution with shrinkage operator to the nuclear norm is singular value shrinkage operator [18] , which can be expressed as follows:where is defined as follows:However, it should be noted that the shrinkage operator for norm is different from singular value shrinkage operator for nuclear norm. There operators play an important role in joint sparse and low rank matrix approximation.
In this paper, ordered -norm [19] is provided as follows:where is sorted in decreasing order. denotes the th largest element of the magnitude vector . is a trade-off vector in nonincreasing order. When is a constant vector, (5) reduces to norm. When and , then (5) reduces to -norm.
-support norm [20] is defined as follows:where denotes the bound of the sparsity level of , which is a positive integer. The details of -support norm are in traduced in [20, 21]. It has been extended to the case of matrix, which named by regular spectral -support norm. Although this norm provides the number of elements of the sparsity level, it lacks of efficient mechanism to promote the sparsity adaptively.
After taking advantage of both ordered norm and -support norm, regular -support norm is defined as follows:where is a positive regularization vector in nonincreasing order. And , where is the size of vector .
For the case of matrix, regular spectral -support norm is proposed, which can be expressed as follows:where . It can be noted that the singular values are arranged in nonincreasing order.
3. Problem Setup
This section introduces the objective function in detail. For each reference patch , similar patches in the spatial- and temporal-domain are obtained by utilizing the patch matching algorithm. The matched patches are denoted as , where stands for the size. It can be noted that each patch is rearranged as a vector with size through concatenating all columns into a column vector. At last, a matrix is generated after considering all the patches, which can be represented as follows:In this paper, we assumed that the observed patch matrix can be decomposed into three components:where stands for low rank component, is sparse component, and is additive noise. There are some regularization methods for (10), such as with nuclear norm (also known as the trace norm) and with norm. The problem formulation in [7] can be expressed as follows:where is nuclear norm, and for norm.
However, these norms may lead to a large estimation bias [19] but can not promote the sparsity level adaptively. For example, the limitations of norm have been investigated in [22, 23]. Similarly, some alternative cases, such as the quasinorm, also have been discussed in [24–26]. Thus, a suitable solution is required to recover these components.
To alleviate these limitations, a new problem formulation is proposed to model the sparsity levels both on and . Moreover, a unified formulation to describe the correlated variables is considered. To estimate the underlying structures of and , we focus on the following minimization function:where and are two positive regularization vectors in a nondecreasing order. stands for regular spectral -support norm on . denotes regular -support norm on . And is the standard deviation of noise .
The above formulation amounts to the constraint , which is considered more natural than usual formulation because it stands for the tolerance on the error.
After choosing a suitable , (12) can be reformulated as follows:where is a suitable positive value. It can be seen that these are some challenges to solve (13). First, regular -support -norm is posed on sparse component . Second, the low rank component is penalized by regular spectral -support norm.
There are several properties of our problem formulation in (13). First, the proposed regular spectral -support norm on and regular -support norm on aim to reconstruct the local structures clearly. It should be noted that these modeling strategies can adaptively promote the sparsity level with an upper bound. Second, to the best of our knowledge, this is the first time of combining the advantages of both ordered norm and -support norm to yield a robust subspaces estimation against noise. Third, although the optimization method in [27] is very similar to the proposed method, the proposed method can deal with more complex situations. Moreover, the proposed method can adopt the more challenging situations, such as the removal of mixed Gaussian, salt-and-pepper noise, and random value impulse noise. It should be noted that this noisy situation has not been explored in [7].
4. Proposed Method
4.1. Proposed Framework
In this section, an optimization framework using regular -shrinkage thresholding operator is presented. First, accelerated proximal gradient method (APG) is applied to the resulting problem because of its simplicity and popularity in imaging applications [28, 29]. Second, the proposed regular -shrinkage thresholding operator is applied to the two resulting subproblems. As showed in [22, 23], nonconvex regularization functions have been shown both theoretically and experimentally to provide better results than norm. Then, some explicit proximal mappings are developed.
APG based scheme aims to solve an unconstrained minimization problem bywhere is assumed to be a nonsmooth function and for a smooth function. Here, denotes the Lipschitz constant of the gradient of .
Applying the framework of APG to problem (13), we have the following expressions:Setting , then we have the following objective function:where the proximal points and in the framework of APG are defined in Algorithm 1. It can be noted that both the variables and are separable. Thus, there are two subproblems, which can be represented as follows:It can be seen that (17) is a minimization problem with regular spectral -support norm and (18) with regular -support -norm. To deal with these problems, regular -shrinkage thresholding operator (RK) is defined as follows:where denotes an operation of direct minus in nonincreasing order. denotes the sparsity level of the input vector. also is a vector in a nonincreasing order.
| ||||||||||||||||||||||||
Remark 1. There are some differences between regular -shrinkage thresholding operator and shrinkage operator [17]. First, the proposed operator models the sparsity level by the procedure of regular shrinkage adaptively. Second, the introduction of -support constraint can bound the degree of the sparsity. Third, the combination of regular shrinkage and -support leads to the modeling of the correlated variables robustly.
When applying the proposed operator to and , we havewhere denotes the eigenvalues of . It should be noted that the solution to can be viewed as a generalization of singular value shrinkage operator. Based on the framework of APG, an optimization framework for the objective function (16) is presented in Algorithm 1. The detailed procedures for the two subproblems are provided in Algorithm 1.
4.2. Some Implementation Details
In our implementation, the sampled image patches with overlapping regions are considered. Then, each frame of the restored video may be replaced by the recovered patches. For the synthesis process, the outcome of each selected pixel is accomplished by calculating the average of multiple estimates from the related patches. This procedure could deal with the artifacts along the boundaries of patches and restore fine details locally.
5. Experiments and Discussion
5.1. Experimental Settings
To demonstrate the effectiveness and efficacy of the proposed method, some experiments are conducted. We focus on the removal of mixed Gaussian-impulse noise. Two types of noisy situations, including mixed Gaussian and random value impulse noise (GRV) denoted by and mixed Gaussian, salt-and-pepper noise, and random value impulse noise (GSPRV) by , are tested. Some samples of three videos (http://trace.eas.asu.edu/yuv/) are displayed in Figure 1. The sizes of coastguard, flower, and news in our experiments are , , and , respectively. The parameter of the proposed method for the coastguard is set to 1000. For the other videos, that is, flower and news, is set to 2000. All experiments are performed in MATLAB R2014 running on a desktop with Intel Core i7 at 3.2 GHz.
(a)
(b)
Three related methods are compared with the proposed method, including VBM3D [30], RPCA based method [7], and - based method [8]. VBM3D based method is not originally designed for the removal of mixed Gaussian-impulse noise. To remedy this problem, adaptive center-weighted median filter [15] (ACWMF) is used to detect and remove the impulse noise firstly. Two indexes are taken for assessing the denoised performance of all competing methods, that is, peak-signal-to-noise ratio (PSNR) and feature-similarity (FSIM) index [31].
5.2. Mixed Gaussian and Random Value Impulse Noise
In this subsection, the denoising results for three different scenarios are presented, including , and . Numerical results on three videos are presented in Table 1. It can be observed that the proposed method outperforms all the competing methods with respect to FSIM and PSNR. Visual outcomes are demonstrated in Figure 2. The recovered result of proposed method is presented in Figure 2(f). To examine the recovered details, the selected parts in the visual results are enlarged. It can be noted that the proposed method can reconstruct more local details.
(a)
(b)
(c)
(d)
(e)
(f)
5.3. Mixed Gaussian, Salt-and-Pepper Noise, and Random Value Impulse Noise
In this subsection, the experimental results by the removal of Gaussian, salt-and-pepper, and random value impulse noise are demonstrated. Two noisy levels are assessed. The numerical results are presented in Table 2. It can be noted that the proposed method outperforms other methods. A visual assessment of the reconstruction performance of both algorithms is shown in Figure 3. As shown in the enlarged parts, the proposed method presented in Figure 3(f) recovers more local details than other methods.
(a)
(b)
(c)
(d)
(e)
(f)
5.4. Discussion
In this paper, an efficient image restoration scheme for hybrid Gaussian-impulse noise is proposed. The denoising performance of our method is examined in various noisy scenarios. When the strength of noisy levels increased, our method performed more efficiently than other methods. The outcomes of all experiments verified the effectiveness of the proposed method. The difference may be related to the modeling method and optimization strategy we taken. Moreover, the intrinsic sparsity structure of each decomposition component is explored. It should be noted that some limitations may be observed, such as being oversmooth on the local region.
In this paper, an alternating minimization method with regular -shrinkage thresholding operator is proposed. Specially, a universal modeling strategy by exploiting the adaptivity of sparsity structure leads to higher quality reconstructions. The proposed method may provide a new class of denoising methods to deal with mixed Gaussian-impulse noise. The numerical results from various experiments validated the effectiveness of the proposed method again.
6. Conclusion
In this paper, an efficient video restoration scheme is proposed for the removal of mixed Gaussian-impulse noise. Unlike traditional norm based methods, which treat all the values equally, the proposed method tries to explore the additional structure by regular spectral -support norm on low rank component and regular -support norm on sparse component. Then, the special structure can be promoted on the sparsity level of the decomposition matrices adaptively. To overcome the nonconvex problem, a solution with alternating regular -shrinkage thresholding operator is proposed. The proposed method has good practical performance with appropriate structures. The numerical results, compared to some state-of-the-art methods, demonstrate the advantages of the proposed method.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This work is jointly supported by National Natural Science Foundation of China (Grants nos. 61603249 and 61673262) and key project of Science and Technology Commission of Shanghai Municipality (Grant no. 16JC1401100).