Mathematical Problems in Engineering

Volume 2017, Article ID 3012910, 13 pages

https://doi.org/10.1155/2017/3012910

## A Coordinate Descent Method for Total Variation Minimization

^{1}Northeast Agricultural University, Harbin 150001, China^{2}Harbin Institute of Technology, Research Group for Computational Photography and Statistical Learning, School of Computer Science and Technology, Harbin 150001, China^{3}No. 211 Hospital of PLA, Harbin 150001, China^{4}Department of Computing, The Hong Kong Polytechnic University, Kowloon, Hong Kong

Correspondence should be addressed to Wangmeng Zuo; moc.liamg@ouzmwsc

Received 5 May 2017; Accepted 10 August 2017; Published 18 September 2017

Academic Editor: Gerardo Severino

Copyright © 2017 Hong Deng et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

Total variation (TV) is a well-known image model with extensive applications in various images and vision tasks, for example, denoising, deblurring, superresolution, inpainting, and compressed sensing. In this paper, we systematically study the coordinate descent (CoD) method for solving general total variation (TV) minimization problems. Based on multidirectional gradients representation, the proposed CoD method provides a unified solution for both anisotropic and isotropic TV-based denoising (CoDenoise). With sequential sweeping and small random perturbations, CoDenoise is efficient in denoising and empirically converges to optimal solution. Moreover, CoDenoise also delivers new perspective on understanding recursive weighted median filtering. By incorporating with the Augmented Lagrangian Method (ALM), CoD was further extended to TV-based image deblurring (ALMCD). The results on denoising and deblurring validate the efficiency and effectiveness of the CoD-based methods.

#### 1. Introduction

Total variation (TV), also known as the ROF model [1], was introduced by Rudin et al. The TV model is effective in preserving sharp and salient edges while suppressing noise and has been extensively adopted as a regularizer in various image restoration applications, for example, deblurring [2, 3], superresolution [4, 5], inpainting [6, 7], and compressed sensing [8, 9].

Recently, other image models, such as dictionary-based sparse coding [10–12] and nonlocal similarity [13–17], have been developed. Compared with these models, TV is much more efficient to be solved, making TV-based methods remain active in image and vision studies [17–24]. Moreover, TV may be complementary with the other models, and thus proper combination of them can lead to better performance [25, 26]. Besides, extensions of TV regularizer were also studied. For color images, TV can be extended to a class of vectorial TV (VTV) [27, 28], where interchannel correlation is taken into account to reduce the uneven color effects. While TV only considers first-order gradients, Total Generalized Variation (TGV) [29] was proposed to involve higher-order derivatives. For structure extraction, relative TV [30] was employed to distinguish structure from textures. Considering that the gradient distribution of each pixel is actually spatially variant, nonlocal extension of TV model [17, 31] was presented to leverage the similar patches for adaptive distribution estimation.

A basic TV minimization problem is TV-based image denoising formulated aswhere is the TV regularizer, is the trade-off parameter, and and are the latent clear image and the noisy observation, respectively. Various methods to solve TV denoising problem had been proposed and can be roughly categorized from three directions, that is, gradient based, Markov Random Fields (MRF) based, and CoD-based methods. First, gradient descent-based algorithms have been widely adopted in image processing tasks [18, 32–37]. As to TV minimization, gradient projection based PDE methods [1] originally were adopted to solve the associated nonlinear Euler-Lagrange equation. Following this line, a number of methods tried to directly solve primal variables [38–42]. To avoid nonsmoothness trap, the dual formulation of TV minimization was proposed and several variants came forward [43–45]. Recently, a hybrid primal dual scheme that alternatively solves primal and dual variables had been developed [46–48]. Most specially, Chambolle’s fixed point algorithm [43] solving dual variable is the most successful, which has been widely adopted in general image restoration methods, for example, TwIST [49], FISTA [50], and SALSA [51]. Second, TV minimization can be mapped to a class of binary MRFs [52–54], such that it can be solved by graph-cut techniques. Third, another entirely different direction is to employ CoD method, decomposing optimization problem with respect to each pixel and updating coordinate variables via some appropriate patterns. For the high efficiency of decomposed scalar optimizations, the CoD-based methods are usually efficient. However, the sole attempt based on CoD to solve TV minimization [55] only considers the anisotropic TV minimization, while isotropic TV minimization is unreachable for CoD-based methods, since it cannot be decomposed with respect to each pixel.

In this paper, we systematically study the CoD-based methods for TV minimization problem. First, we provide a unified formulation of anisotropic and isotropic TV minimization problem based on multidirectional gradients representation, via which the isotropic TV regularizer can also be decomposed into a sequence of scalar convex problems with respect to each pixel. The scalar convex problem can be efficiently solved, and by sequentially updating each pixel, the CoD-based denoising (CoDenoise) algorithm converges fast. Due to the nondifferentiability of TV regularizer, CoDenoise may get stuck at nonstationary points [55–57]; however fortunately it is experimentally verified that CoDenoise can bypass nonstationary points and converge to optimal solution by adding small random perturbations. The CoDenoise algorithm only requires updating the pixels poisoned by noises, due to which the CoDenoise algorithm is more efficient than other methods, especially for low noise levels. Interestingly, the CoDenoise algorithm can be interpreted as the recursive weighted median operations on noisy images. Based on the more recent progress in weighted median filter [58, 59], the CoDenoise algorithm should be much more improved in terms of efficiency. Then, by combining variable splitting strategy and Augmented Lagrangian Method (ALM), we further embed CoDenoise algorithm to solve general image restoration problem, for example, image deblurring, resulting in the ALMCD algorithm. In deblurring problems, the blurry images are usually poisoned by relatively low level noises, and thus the incorporated CoDenoise algorithm for denoising subproblem contributes significantly to efficiency improvement of the ALMCD algorithm. Compared with TwIST, FISTA, and SALSA, ALMCD can obtain satisfactory results but is more efficient.

Our contribution can be summarized from two aspects:(i)We systematically study the CoD-based methods for TV minimization and develop an extremely simple unified CoD-based solution for both anisotropic and isotropic TV minimization. The resulting CoDenoise algorithm is more efficient than gradient based and MRF based methods and achieves satisfactory denoising results.(ii)By incorporating with ALM, CoDenoise is extended to image deblurring problem. In the deblurring problems, the blurry images usually suffer from severe blur and relatively low level noises, and thus the proposed ALMCD algorithm with CoDenoise embedded for denoising subproblem is much more efficient and can concurrently provide satisfactory deblurring quality compared with several state-of-the-art methods.

This paper is organized as follows: Section 2 presents some preliminaries, including definition of TV regularizers and multidirectional gradient approximation of TV regularizers. The CoDenoise algorithm together with its convergence proof and computational complexity is proposed in Section 3. In Section 4, we embed CoDenoise to image deblurring. Section 5 demonstrates experimental results, and Section 6 ends this paper with some concluding remarks.

#### 2. Preliminaries

In this section, we first present the definitions of the discrete anisotropic and isotropic TV operators. In previous studies, CoD-based solution is only available for anisotropic TV minimization problem. To address this, we then introduce the multidirectional gradient representation to establish the connection between the anisotropic and isotropic TV models, making it possible to use the unified CoD method for TV minimization.

##### 2.1. The Discrete TV Operators

For an image with pixels, the discrete gradient operators including both horizontal gradient operator and vertical gradient operator are defined aswhere and . The anisotropic TV regularizer [50, 60] is defined asWith this definition, it is easy to obtain the anisotropic TV regularization with respect to coordinate asThus, the CoD method can be directly used to solve the anisotropic TV minimization problem. Similarly, the isotropic TV regularizer [50, 60] is defined asApparently the isotropic TV cannot be decomposed with respect to coordinate since the quadratic interactions with horizontal and vertical gradients, making the CoD method unfeasible to solve isotropic TV minimization problem. Therefore, to extend the results of CoD to isotropic TV minimization problem, we tempt to find a connection between and .

##### 2.2. Multidirectional Gradients Approximation

The isotropic TV regularizer can be approximated by multidirectional gradients representation, and thus the anisotropic and isotropic TV models can be connected in a unified formulation [61]. For any pair of real numbers and , the identityalways holds, which can be discretized by Riemannian approximation. Now, let be a set of points uniformly distributed in . Equation (6) can then be discretized as

Thus we can approximate TV regularizer aswhere and and . Equation (8) provides a unified formulation of anisotropic and isotropic TV models,In later context, we will use to represent the TV regularizers.

#### 3. The Unified Coordinate Descent Method for TV-Based Denoising

With regularizer, anisotropic and isotropic TV denoising models are reformulated in the unified formwhich is exactly anisotropic TV-based denoising when and infinitely approximates isotropic TV-based denoising when increases. We thus can decompose the objective function into a sequence of one-dimensional subproblems, which can be solved efficiently via simple convex optimization. With simple sequential updating pattern, we then obtain the unified CoD denoising algorithm for both anisotropic and isotropic TV minimization.

##### 3.1. The Coordinate Subproblem

Let first present equivalent decomposition of the image denoising objective function with respect to each pixel ,where and . Vectors and are both of length , which are the coefficients of and the combinations of its 4 neighbourhoods, respectively.

##### 3.2. Solving Subproblem

For simplicity, we unify the formulation of subproblems as

The scalar optimization problem is convex but nonsmooth. We assume (the case can be easily generalized). Let be the permutation of according to the ascending order of . Let be an ascending sequence with , , and . Let . Thus, (12) is transformed to

The solution to (13) can be obtained by making its first-order derivative be 0,

We then discuss the solution with different cases of :(1)When , Then is the optimal solution to (13), if .(2)When , Then is the optimal solution to (13), if .(3)When , Then is the optimal solution to (13), if .(4)When ,

Since , for , we thus haveand . Then is the optimal solution to (13), if .

As a summary, we notate procedures as an operator,

Interestingly, solution (20) can be interpreted as finding the median value of vector (28), which is discussed in Section 3.4.3.

##### 3.3. CoDenoise

Therefore, the subproblem with respect to (11) can be solved by

The following question is how to choose coordinate updating pattern. Li and Osher adopted the checkerboard pattern [55], in which the pixels are divided into black and white blocks. The pixels in the same group are not neighbors, and then the pixels in two blocks can be alternatively updated. Another greedy strategy is also popular [62], in which the selected coordinate makes the biggest contribution to the decrease of the energy function. And by the divide and conquer strategy, the corresponding coordinate can be searched with complexity [63].

The proposed CoDenoise algorithm adopted the simple cyclic updating pattern, sequentially sweeping each pixel. If the computed solution at new selected coordinate makes a big progress than that in last iteration (evaluated by a tolerance ), then it will be updated. In our implementation, we use a binary mask matrix to indicate whether a pixel will be updated or not. If any four neighbor of pixel is updated, is marked as , and the pixel will be updated in the next iteration, otherwise 0. For the nondifferentiability of TV norm, the solution generated by CoDenoise may get stuck at nonstationary points, which can be easily bypassed by adding small random perturbations. The perturbations decrease along with the increasing iteration number.

To stop the CoDenoise algorithm, we check whether the relative difference between two iterations is below tolerance ; that is,

The CoDenoise algorithm is summarized as Algorithm 1.