International Journal of Biomedical Imaging

Volume 2018, Article ID 9262847, 17 pages

https://doi.org/10.1155/2018/9262847

## Super-Resolution of Magnetic Resonance Images via Convex Optimization with Local and Global Prior Regularization and Spectrum Fitting

Nagoya Institute of Technology, Gokiso-cho, Showa-ku, Nagoya-shi, Aichi, Japan

Correspondence should be addressed to Naoki Kawamura; pj.ca.hcetin.ui@arumawak

Received 24 May 2018; Revised 27 July 2018; Accepted 7 August 2018; Published 2 September 2018

Academic Editor: Ahmed Soliman

Copyright © 2018 Naoki Kawamura et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

Given a low-resolution image, there are many challenges to obtain a super-resolved, high-resolution image. Many of those approaches try to simultaneously upsample and deblur an image in signal domain. However, the nature of the super-resolution is to restore high-frequency components in frequency domain rather than upsampling in signal domain. In that sense, there is a close relationship between super-resolution of an image and extrapolation of the spectrum. In this study, we propose a novel framework for super-resolution, where the high-frequency components are theoretically restored with respect to the frequency fidelities. This framework helps to introduce multiple simultaneous regularizers in both signal and frequency domains. Furthermore, we propose a new super-resolution model where frequency fidelity, low-rank (LR) prior, low total variation (TV) prior, and boundary prior are considered at once. The proposed method is formulated as a convex optimization problem which can be solved by the alternating direction method of multipliers. The proposed method is the generalized form of the multiple super-resolution methods such as TV super-resolution, LR and TV super-resolution, and the Gerchberg method. Experimental results show the utility of the proposed method comparing with some existing methods using both simulational and practical images.

#### 1. Introduction

Magnetic resonance (MR) imaging is one of the most important methods for observing three-dimensional (3D) soft tissues with high contrast (e.g., [1–3]). However in order to assure sufficiently high signal-to-noise ratio (SNR), MR images often have anisotropic spatial resolution: The spatial resolution along the through-slice direction is lower than the resolution along the in-plane direction. The spatial resolution along the through-slice direction is mainly determined by the slice thickness, and there is a trade-off between the slice thickness and the SNR of MR images. Increase of the slice thickness would degrade the spatial resolution along the through-slice direction, though the SNR of each slice image would be improved by the increase of the slice thickness because the quantity of hydrogen nuclei included in the measured slice increases and the magnitude of the signals emitted by the hydrogen nuclei also increases. This is a reason why slice thickness is often set as thick as several times the pixel size and the spatial resolution along the through-slice direction is lower than that of slice images.

Super-resolution techniques (e.g., [1, 2, 4–6]) can restore detailed patterns unobservable in given images and can be used for improving spatial resolutions of MR images. In this article we at first focus on a conventional super-resolution algorithm, which was proposed by Gerchberg [7]. The algorithm improves the spatial resolution of a given image by using the prior knowledge of an outer boundary of the target and of the measurable frequency range of a target spatial pattern of the given image. The physical meaning of the prior knowledge used in the algorithm is well understandable and the algorithm can be applied to MR images straightforwardly. The method iteratively improves the spatial resolution of a given image and it is proved that the restored image converges toward the true solution when the prior knowledge and the reality are consistent [8]. In practice, the results obtained by the Gerchberg algorithm may be affected by ringing artifacts [9–11] and are degraded by measurement noises. Reference [12] formulated the algorithm in a signal-extrapolation framework and the method is now called the Papoulis-Gerchberg algorithm [9, 10, 12].

The Gerchberg algorithm is essentially the same with the projection onto convex sets (POCS) method, which was later defined by Youla [8, 13]. Many POCS methods have been proposed where the super-resolution problem is solved by iteratively projecting a given image onto two or more convex sets, each of which represents each of the introduced constraints on the reconstructed image (e.g., [14–18]). The constraints to be introduced vary depending on the available prior knowledge of images: The knowledge that can be employed by the POCS method includes the range of pixel values [15, 19], the fidelity of the data [15, 17, 20], and nonnegativity [16, 17]. In our study, we assume that both the measured frequency range and the outer boundary of a target in a given MR image are known, which means the POCS method can be applied for improving the spatial resolution of the given MR image by representing the knowledge with two convex sets in an image space.

Super-resolution of images is an ill-posed problem, and regularized optimization techniques are widely used for solving the super-resolution problem. One of the most popular regularizers is total variation (TV) of images (e.g., [21–23]), which helps to reduce ringing artifacts and noises in images while preserving the edges [21, 24]. In addition to TV, rank regularization has also attracted much attention for solving ill-posed problems such as an image completion problem (e.g., [25–27]). It is reported that one can improve the performance of image super-resolution by combining the TV regularization with the low-rank one [23]: high-resolution images are obtained by minimizing a cost function, in which both a TV regularization term and a low-rank regularization term are included.

Recently, there are more and more deep learning-based super-resolution proposed. Learning-based approaches exploit internal or external database to super-resolve an image [28–31]. For example, SRGAN [32] trains and generates high-frequency patterns of input images. LAPGAN [33] exploits the Laplacian pyramid of images, where the high-resolution images can be well represented as straightforward hierarchy summations of generated high-frequency patterns and a low-resolution image. The resultant images of them are extremely realistic.

Learning-based methods, however, can largely improve spatial resolution of input images only when sufficient number and variation of training data are available and target images can be regarded as drawn from the probability distribution the training data represent. For example, given a set of sufficiently large number of training CT images of healthy subjects, one can improve the spatial resolution of a CT image of a new healthy subject well but it would be difficult to improve the resolution if it is a CT image of a subject with tumors. It should be noted that, in medical image processing, collecting sufficient number and variety of medical images for the training is challenging [34]. Compared with learning-based approaches, mathematical model-based approaches, which include POCS methods, can be applied to images that are consistent with the employed mathematical models and are not affected by the bias of the collected training data.

In this study, we propose a framework for incorporating the Gerchberg algorithm into a regularized optimization based method of super-resolution. In this framework, we can use the knowledge of the outer contour of a target and of the measured frequency range with the conventional regularizers simultaneously for computing higher spatial resolution images. Combining TV regularization with the Gerchberg term, one can suppress ringing artifacts often generated by the Gerchberg method. Here, it should be noted that the incorporation of the Gerchberg method into regularized optimization based methods is not so straightforward because the Gerchberg method obtains high-resolution images not by explicitly minimizing some cost function but by iteratively projecting an image onto convex sets. The main contributions of the present study are as follows: we reformulate the projections in the Gerchberg super-resolution algorithm using linear matrix equations, we formulate a convex optimization problem, in which the reformulated projections and the low-TV/low-rank regularization are represented in a cost function and constraints, we explicitly describe the algorithm for solving the convex optimization problem with the alternating direction method of multipliers (ADMM), and we present extensive experimental evaluations conducted using the proposed method.

The proposed method has the following theoretical limitations on the input MR images in order to solve an inverse problem: (i) the boundary of an image object can be labeled in a reasonable time and the backgrounds are composed of its blur and noise, (ii) the blur kernel or PSF of the observation is known in advance, and (iii) the image noise obeys the normal distribution.

The remainder of this paper is organized as follows. In Section 2.1, we explain the notations used in this study. Next, we provide a problem statement regarding the present study in Section 2.2. In Section 2.3, we review the Gerchberg algorithm and recent regularization-based approaches. The proposed method and the description of its explicit solvers are explained in Section 2.4. Variational experimental results are presented in Sections 3–3.5. In Section 3.6, we discuss the behavior and various aspects of the proposed method. Finally, we give our conclusions in Section 4.

#### 2. Materials and Methods

##### 2.1. Notations

In this study, a vector is denoted by a bold small letter and a matrix is denoted by a bold capital letter . A 3D tensor is denoted by a bold calligraphic letter . The -th entry of a matrix is denoted by and the -th entry of a 3D tensor is denoted by .

Given a vector , the tensor folding operator is denoted by fold, and its adjoint operator is vec. Given a vector , its matricization is denoted by . Given a tensor , the -th mode unfolding operator is denoted by , and its adjoint operator is .

Given that is the singular value decomposition for a matrix , a singular value soft-thresholding operator [25, 35] is defined aswhere and is the -th singular value of . The operator is the Hadamard (element-wise) product.

##### 2.2. Problem Statement

Without loss of generality, we can assume that a field of view (FOV) of an MR image is a cubic space. Let the side length of the cubic FOV be denoted by and let the three mutually orthogonal directions corresponding to the sides of the cubic FOV be denoted by a -axis, a -axis, and a -axis.

For simply describing the method, we assume that the slice thickness and the slice spacing are equal and that an MR image consists of slice images, each of which has voxels. It follows that the voxel size along the through-slice direction is given by and that the voxel size in each slice image is given by , where . holds in many MR images in order to assure high SNR. (Increase of the slice thickness would degrade the spatial resolution along the through-slice direction, though the SNR of each slice image would be improved by the increase of the slice thickness because the quantity of hydrogen nuclei included in the measured slice increases and the magnitude of the signals emitted by the hydrogen nuclei also increases.) Let the scaling factor be denoted by , where . The spatial resolution along the through-slice direction is times lower than the resolution along the in-plane directions in an MR image.

In the experiment here, we assume that two MR images are given. When multiple MR images are given, it is assumed that the MR images are obtained with mutually orthogonal directions of slice-selective gradient. Let 3D tensors, , , denote MR images of the same FOV obtained with the slice-selective gradient parallel to the -axis and the -axis, respectively. Let a tensor denote an isotropic noise-free MR image of the FOV obtained by an ideal MR scanner. It is assumed that any measured MR image of the FOV, , can be generated from by appropriately eliminating higher frequency components in the corresponding direction of the slice-selective gradient followed by downsampling by .

Let the Fourier transform of be denoted by and let denote a frequency region only in which the Fourier components of are measured: Outside of the region, , in the frequency space, the frequency components are zero. As shown in Figure 1, it should be noted that does not cover the whole spectrum space and that diagonal high-frequency regions are not observed in any of the images. The objective here is to estimate/complete the unknown frequency components and reconstruct a high-resolution MR image.