Abstract
Magnetic resonance imaging has significant applications for disease diagnosis. Due to the particularity of its imaging mechanism, hardware imaging suffers from resolution and reaches its limit, and higher radiation intensity and longer radiation time will cause damage to the human body. The problem is expected to be solved by a superresolution algorithm, especially the image superresolution based on sparse reconstruction has good performance. Dictionary generation is a key issue that affects the performance of superresolution algorithms, and dictionary performance is affected by dictionary construction parameters: balance parameters, dictionary size, overlapping block size, and a number of training sample blocks. In response to this problem, we propose an optimal dictionary construction parameter search method through the experiment to find the optimal dictionary construction parameters on the MR image and compare them with the dictionary obtained by multiple sets of random dictionary construction parameters. The dictionary we searched for the optimal parameters of the dictionary construction training has more powerful feature expressions, which can improve the superresolution effect of MR images.
1. Introduction
Magnetic resonance imaging (MRI) becomes more and more widely used in medical clinical applications and plays an increasingly important role in the diagnosis of various diseases [1–5]. The mechanism of MR imaging is different from that of natural images. The hydrogen protons of human organs are magnetized under the action of external magnetic fields and generate a magnetic resonance phenomenon under the action of a magnetic field. The changing magnetic signals are converted into electrical signals by induction coils and fill the K space. Finally, an MR image is generated through the Fourier transform. The method of improving resolution relies heavily on increasing the magnetization of more free water in human tissues and organs [6], which will cause the increase of the radiation time and radiation intensity of the main magnetic field of the magnetic resonance imager and the loaded electromagnetic waves [7]; excessive radiation can lead to serious consequences, such as overheating of the human body and protein inactivation [8], causing harm to the human body and not suitable for clinical application. From the perspective of current imaging methods and technologies, the hardware imaging resolution reaches the limit value. To increase the resolution, software superresolution technology must be used to increase the image resolution.
Image superresolution methods are mainly based on interpolation, reconstruction, and learning. Li and Orchard [9] proposed a new edge-directed image interpolation (NEDI) method; Wang and Ling [10] proposed an Edge-Adaptive Interpolation Algorithm (EAIA), combined with bilinear and NEDI methods; Giachetti and Asuni [11] proposed an interpolation based on iterative curvature method based on the NEDI method. But the interpolation-based method does not essentially increase the image information. Irani and Peleg [12] proposed an iterative backprojection method (IBP), Schultz and Stevenson [13] proposed a superresolution method based on maximum posterior probability (MAP), Patti et al. [14] proposed the convex set projection method, and in [15], the projection onto convex sets superresolution reconstruction method was used for the superresolution of cardiac valve MR images. POCS algorithm is not good at maintaining the image edge and can not restore the high-frequency information on the image. The superresolution method based on reconstruction regards low-resolution observation images as a constraint condition of the original high-resolution image, and a series of solution spaces satisfying the constraint condition can be obtained through the alternating iteration method. Most of the above algorithms use prior knowledge such as the edge characteristics of the image, the nonnegativity of pixels and local smoothing characteristics to construct constraints and then solve the optimization problem through an iterative algorithm. The reconstruction algorithms are computationally expensive, and the resulting image is reconstructed too smoothly. Learning-based superresolution methods mainly include dictionary learning and deep learning. Yang et al. [16] proposed a superresolution reconstruction algorithm based on sparse representation. This method effectively overcomes the problem of inaccurate representation caused by using a fixed number of neighbors. On the basis of the methods [17], adaptive sparse field selection and adaptive regularization are applied to superresolution. Yang et al. [18] proposed a double-geometric neighborhood embedding method (DGNE), which uses multiview features and local spatial neighbors of image blocks to find image features-spatial manifold embedding. Zhang et al. [19] combined subspace division and local regressor learning through the mixture of experts method to further improve the quality of image reconstruction. With the development of deep learning. Shi et al. [20] propose a novel image SR method that integrates both locals and global information for effective image recovery. This is achieved by, in addition to TV, low-rank regularization that enables utilization of information throughout the image. The reconstructed image will produce stripe distortion, and the texture and other details will be blurred to some extent. Dong et al. [21] used convolutional neural networks for image superresolution reconstruction for the first time. This method first uses bicubic interpolation to enlarge it to the target size and then passes through a three-layer convolution network doing the nonlinear mapping. The results obtained are output as high-resolution images, and experiments show that it has achieved good results. Since then, Residual Dense Network [22], SRGAN [23], and many deep networks [24, 25] are proposed and used. Deep learning methods are based on data driving, and Network performance is affected by the amount of data. However, due to the particularity and privacy of MR images, it is difficult to obtain large amounts of data. MR image superresolution task is more suitable for methods that weak dependence on data volume.
Our contributions are threefold. First, we propose a superresolution architecture based on joint dictionary learning suitable for a small number of MR images. Second, we analyze the effect of dictionary parameters on dictionary performance and find the optimal dictionary parameters through parameters learning. Third, experiments prove that our proposed method can achieve state-of-the-art performance, even if there are only a few image data.
2. Algorithm and Analysis
2.1. Algorithm
We propose a superresolution architecture based on joint dictionary learning suitable for a small number of MR images. The algorithm framework is shown in Figure 1. Through corresponding high and low-resolution image block training, learn high and low-resolution dictionary, low-resolution image blocks are sparsely represented by a low-resolution dictionary, and the sparse coefficients can be used for high-resolution image reconstruction. The performance of high and low-resolution dictionaries directly affects the image reconstruction effect.
Training a joint dictionary requires the use of high-resolution image block sets and low-resolution image block sets. Training image pair is represented by . and represents features extracted from image blocks. The dictionary training of two image feature spaces is expressed as follows [16]:
According to the idea of the joint training method, the image blocks corresponding to the two image feature block spaces are concatenated to form a new image feature block space, so formula (3) [16] can be obtained:M is the dimensions of the low-dimensional image feature block, and N is the dimensions of the high-dimensional image feature block. It can be seen from formula (3) that balance parameters, dictionary block size, overlapping block size, and the number of dictionary blocks have an important impact on the performance of the dictionary. We obtain the optimal parameters of dictionary construction through experimental analysis so as to achieve the improvement of dictionary performance and image reconstruction effect.
2.2. Parameter Set and Analysis
The parameter set is written as parameter = [, overlap, n, spn], where is balance parameter, the overlap is the size of the overlap block, n is the size of the dictionary block, and spn is the number of the exemplar patch.
From the mechanism of reconstruction perspective, changes of the parameter can cause changes in the structure and quantity of the data calculated by the dictionary, which has a great influence on the effect of reconstruction. The specific analysis is as follows.
According to the formula in equation (3), the balance parameter is used to balance sparsity and low-resolution dictionary sparse, which represents image block errors. It can be analyzed from the equation that the sparsity is inversely proportional to the error. In order to find the minimum sparsity coefficient, the error will increase, and vice versa, so there must be an optimal value to minimize the value of equation (3).
The overlapped block overlap is the size of the overlapped portion among the image blocks, and is divided into the selection of sample blocks in dictionary training and overlap of test image blocks in reconstruction. In order to ensure that the detailed features of the sample block can be extracted in the dictionary training, the maximum overlapped block method is used to select samples, which makes 1 pixel gradually change between the training sample blocks of a localized image. In order to eliminate the boundary blur caused by the feature extraction of the test image block and the reliability of the connection between the reconstructed blocks, the overlap between the blocks is required. The larger the overlap block, the larger the constraint between the reconstructed blocks, and the better the reconstruction effect.
MR images are expected to show various tissue structures clearly, with high tissue resolution. However, the outline of the diseased tissue can not be seen clearly and separated from the surrounding structure, so it is important to clearly display the texture details, especially the boundaries between different organizational structures, and to some extent, reduce the chances of misjudgment of the diagnosis due to the blurring of the picture. Then, for the dictionary and the reconstructed image, the size of the dictionary block used to represent the feature will affect the number of effective features. The smaller the dictionary block, the fewer features are generated, which makes the reconstructed image have limitations and larger errors, such as the most extreme 1 × 1 and 2 × 2 blocks. The features they can describe are limited. However, the oversized block is also problematic. The image is too large, and the features described by the image block can be combined and described by several smaller feature blocks, then they lose the properties of the feature block, such as extreme cases; the test image itself is a large feature block; of course, this is unreasonable. Therefore, there must be an optimal value for image segmentation, which makes image reconstruction better.
We adopt the method of sparse representation and reconstruction of image blocks. When training the dictionary, a large number of sampling image blocks are required. The number of image sampling blocks has a certain influence on the reconstructed quality. If the image block sample is too little, it is not enough to complete the training of the dictionary. If there are too many image sample blocks, especially the features of some image sample blocks that are not obvious or typical, the characterization of the dictionary cannot be improved even if there is much training. Is there an optimal number of partitions? Since the selected image sampling blocks are randomly extracted, it is difficult to extract the required sample blocks in an accurate number of blocks, so there is no optimal number of sample blocks. Therefore, the selection of the sample block is as long as a certain amount. Too little or too much can both not change the image reconstruction effect.
3. Experimental Results and Analysis
3.1. Dataset
The experiments are all set as follows: The method adopts the experimental framework of Section 1 of Chapter 2. 81; representative pictures of different categories in the image library were selected as the training samples of the high-resolution dictionary. These MR images were obtained from Siemens 3T platforms using a 32-channel head coil. Low-resolution images are generated by the degradation of high-resolution images. LR images are generated following the steps: (1) the high-resolution images are transformed from image space to K space by FFT, (2) in the K space, the outer high frequency is truncated, (3) through the inverse Fourier transform, the truncated k space data are transformed into the image space to generate the corresponding low-resolution images. This mimics the actual acquisition of LR and HR images by MRI scanners. In the experiment, as shown in Figure 2, five images corresponding to different types of MR images are selected as test samples.
(a)
(b)
(c)
(d)
(e)
3.2. Joint Optimization of Parameter
3.2.1. Balance Parameter λ
The optimal value of the balance parameter is verified by the experiments below. The initial parameter configures are as follows: the dictionary size is 512, the balance parameter = 0.1, and the block size is 5 × 5, the overlap block is 4, the number of sample blocks is 100000, and the test samples are, respectively, selected from the head, ankle, carotid artery, knee, and neck, as shown in Figure 2.
It can be seen from Figure 4 that the value of PSNR decreases significantly with the increasing λ when λ > 0.1. On the contrary, when λ < 0.1, the value of PSNR decreases slowly with λ decreasing. As the balance parameter of the sparsity, λ exists optimal value, which makes PSNR maximum. For further verification, let λ = 0.1 as the optimal balance parameter. The super resolution ratio is 1 : 4.
(a)
(b)
The experiment used a superresolution ratio of 1 : 4, and other experimental parameters are the same as those in experiment ratio 1 : 2. The experimental results are shown in Table 2.
Figure 3 is obtained from Table 2. It can be seen from the figure that the extreme point is near , and the experiment with a superresolution ratio of 1 : 4 has the same conclusion as the experiment with a superresolution ratio of 1 : 2.
3.2.2. Overlap Block
The relationship between the image reconstruction effect and overlapped blocks is verified by the following experiment. The initial parameters in the experiment are the same as those in Experiment Balance parameter, and the changed parameters are the size of the overlapped region. The overlapped blocks 1–4 are used to represent the superposed pixels. The experimental results can be seen in Table 3.
According to the experimental results, the images with superresolution ratios of 1 : 2 and 1 : 4 are demonstrated, respectively, in Figure 5, where the abscissa represents the number of overlapped blocks, and the ordinate is the corresponding PSNR. It can be seen from the figure that as the overlay area of the overlapped blocks decreases, the value of PSNR decreases. This is because the larger the overlay area where the image blocks selected for the reconstructed block, the larger the constraint between the reconstructed blocks, and this is easy to find the closest image block to be connected. The more the pixel points at the edge of the image block overlap, the easier it is to eliminate the truncation error caused by the feature extraction. It also has a certain inhibitory effect on noise.
(a)
(b)
3.2.3. Dictionary Blocking Size
The following experiments show the quality of the reconstructed image when having the different blocking conditions for the same test image, where the set of dictionaries is generated with the altering size of the blocked image.
Other experimental parameters do not change, and the changed parameters are the block size of the image blocking. The image blocking of the dictionary has the same requirements as the image blocking of the test image.
The experimental results are as follows: when the superresolution ratio is 1 : 2, 8 high-definition dictionaries with the image block from 3 × 3 to 10 × 10 are generated. Three image blocks are shown in Figure 6. It can be seen that as the image block size increases, the dictionary block becomes more and more complicated. The resulting PSNR values are shown in Table 4.
(a)
(b)
(c)
Table 4 shows the value of the superresolution reconstruction PSNR corresponding to the different block training dictionaries of images. Since the reconstructed overlapped block is 4, which exceeds the block size of the sample block itself, the reconstructed samples are not correct when the dictionaries are, respectively, corresponding to block 3 × 3 and block 4 × 4. The two data sets are not analyzed. The other data corresponding to dictionary block and PSNR are shown in Figure 7.
The abscissa in Figure 7 only represents the block of the dictionary and the image. For example, the abscissa 5 indicates that the dictionary block is 5 × 5 and so on. Increasing the block size will reduce the value of PSNR when the overlap block size is unchanged. That is to say, the block is not bigger always better. When the block is large, the number of dictionary blocks that represent the image feature block will increase, and the reconstruction error will become larger. It can be seen that the preferred block value is 5 × 5 or 6 × 6 blocks, and the calculation efficiency 5 × 5 blocks is optimal.
When the superresolution ratio is 1 : 4, high-resolution dictionaries from 5 × 5 to 13 × 13 are generated through experiment. Three high-resolution dictionaries are shown in Figure 8, where the dictionary block becomes more and more complicated with the number of the blocks increasing. But the too-large block causes too many singular matrices when calculating the dictionary block, which causes the dictionary block information to be lost. The larger the block, the fewer the valid dictionary blocks. This will lead to a decrease in PSNR values, as shown in Figure 8(c). The parameters in the experiment only change is the superresolution ratio of 1 : 4, and the experimental results are shown in Table 5.
(a)
(b)
(c)
The data in Table 5 are the corresponding PSNR values generated by the superresolution reconstruction of the test sample with different block training dictionaries for the corresponding image. In order to intuitively distinguish the influence of the block on the reconstruction, the horizontal coordinate is the image block and the ordinate is the PSNR, as shown in Figure 9.
The abscissa in Figure 9 only shows the image blocking situation. The preferred PSNR corresponds to a 10 × 10 or 11 × 11 image blocking. Taking into account the calculation amount, 10 × 10 image blocking is the best. If the image blocking is too small, it cannot represent features fully, and if the image blocking is too large, the algorithm itself has limitations.
Comparing results corresponding to the superresolution ratios of 1 : 4 and 1 : 2, they have their own best partitions. The image blocks with a superresolution ratio of 1 : 4 are approximately double that of 1 : 2. This is because the image local information required for 4 times superresolution becomes larger, and naturally, the image block needs to be correspondingly larger.
The above two experiments compared the results where the overlapped blocks are fixed as 4. But in the overlapped block experiment, the larger the overlapped blocks, the better the results. The experiment did not consider the best case of overlapped blocks. Next, we will consider that if the best block changes when blocking the different overlapped blocks corresponding maximum.
The experiment verified the effect of the maximum overlap block experiment on the superresolution performance. The parameters are the same as those before the experiments. The changed parameters are only the block size and the overlapped block. For example, the block size is n × n, and the overlapped block value is n − 1. When the superresolution ratio is 1 : 2, the experimental results can be seen in Table 6.
The data show the superresolution reconstruction, where each test sample corresponds to different blocks and overlap blocks. For comparison, the data are plotted as shown in Figure 10.
The horizontal coordinate in Figure 11 only indicates the difference of the block. It can be clearly seen that the reconstructed effect is better from Figure 11 when the dictionary block is 5 × 5 or 6 × 6 blocks. The smaller blocks make the features of the blocks insufficient, and the too-large blocks need to increase the number of calculated pixels. The increase in the size of the dictionary representation block caused by the increase of the feature block makes the error larger and impacts the PSNR effect. In this experiment, the largest overlap block is used to make each component block reach the best reconstruction. It can be seen that the better block value is still 5 × 5 or 6 × 6, and it is best to select a 5 × 5 block for the calculation efficiency.
The above experiment verified the effect of the maximum overlap block experiment on the superresolution performance. The parameters are the same as those in other experiments. The changed parameters are only the block size and the overlapped block. When the superresolution ratio is 1 : 4, the experimental results are shown in Table 7. The obtained data is still plotted with the block size as the abscissa and PSNR values as the ordinate, as shown in Figure 11.
It can be seen from Figure 11 that the 10 × 10 training dictionary has the best superresolution reconstruction when the superresolution ratio is 1 : 4. The above experiment shows that the block size has the highest value and is related to the superresolution ratio. The larger the superresolution ratio is, the larger the block is needed. The change of the overlap block does not influence the result of the optimal block. For the MR image, the optimal block with a superresolution ratio of 1 : 2 is 5 × 5, and the optimal block with a superresolution ratio of 1 : 4 is 10 × 10.
3.2.4. Number of Sampling Blocks
The experiment uses the same parameters as other experiments. The changed parameters are the sample amount of sample image blocks, and the data can be overlap extraction. The superresolution ratio is 1 : 2, and the experimental results are shown in Table 8. The data is taken as an abscissa in the image block sampling with different numbers of training dictionaries, and the image is plotted with PSNR values as the ordinate, as shown in Figure 12.
It can be seen from Figure 12 that the number of sample blocks below 10000 blocks is too small. Since the sample blocks that do not meet the requirements are removed in the algorithm, the MR images have many black or dark areas, and these gray scales are not changed much. Samples with little change in gray, all zeros, or near all zeros are rejected, which greatly reduces the number of blocks actually involved in the calculation. Therefore, as the training sample block, a training sample block with insufficient features reduces the value of PSNR when reconstructed. On the contrary, the large increase in the number of blocks does not cause a significant change in the PSNR, nor does it have a maximum value, showing a fluctuating change. All the training sample blocks participate in the training of the dictionary. Too many blocks will increase the training time, and there is no positive significance for the generation of the HD dictionary. Therefore, it is better to select 150000 sampling blocks. The following experiment with a superresolution 1 : 4 is verified.
The data in Table 9 are taken as an abscissa in the image block sampling with different numbers of training dictionaries, and the image is plotted with PSNR values as the ordinate, as shown in Figure 13. As can be seen from Figure 13, the conclusion with the superresolution ratio of 1 : 4 and is the same as that with the superresolution ratio of 1 : 2 in block selection, while the dictionary cannot be trained with a superresolution ratio of 1 : 4 when the number of blocks is 1000. There are more requirements on the number of dictionaries. Considering the reduction of dictionary training time, it is better to select 150,000 blocks.
3.3. Experiment Simulation of Comprehensive Parameters
The previous section analyzes several parameters that affect the superresolution effect. The values of the optimal parameters of the superresolution MR image are shown in Table 10.
The validity of the optimal parameters is verified by the experiments below. The parameters select several sets of random parameters to form a random group training dictionary, which is compared with the dictionary of optimal parameter training, as shown in Table 11. The PSNR results obtained by experiments are shown in Table 12 below.
Comparing the data in Tables 12 and 13, superresolution PSNR data in the optimal group are higher than that in the random group, no matter the superresolution ratio is 1 : 2 or 1 : 4. This shows that the parameters of the optimal group are the best parameter values.
From the experimental results can be seen, the five human body parts of the superresolution effect have obvious differences. The head and carotid artery superresolution effect is best, and ankle superresolution effect is the worst. This is mainly because each part contains different water components. More water components can produce more hydrogen protons. Under the action of magnetic field and radio frequency pulse, high-frequency information will be generated, which can better generate image edge, texture, and other details.
4. Conclusion
We propose a joint dictionary learning framework for superresolution of MR images, in which changes in dictionary construction parameters will cause changes in the training dictionary and thus affect the performance of superresolution reconstructed images. We have learned the optimal dictionary construction parameters through a large number of experiments and verified that the automatically learned dictionary construction parameters could effectively improve the performance of the dictionary and enhance the expression ability of the image blocks, thereby achieving better MR image superresolution effects.
Data Availability
We have not used specific data from other sources for the simulation of the results. The two popular MRI datasets in this paper, fast MRI Dataset and IXI Dataset, can be freely downloaded from the website https://fastmri.org/and http://www.brain-development.org/.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
The authors would like to thank Dr. Xie Haozhe and Dr. Wang Tingting for their assistance in the writing of this paper. This work was supported by National Science Foundation of China under Grant Nos. 61671170 and 61872085, Science and Technology Foundation of National Defense Key Laboratory of Science and Technology on Parallel and Distributed Processing Laboratory (PDL) under Grant no. 6142110180406, Science and Technology Foundation of ATR National Defense Key Laboratory under Grant no. 6142503180402, China Academy of Space Technology (CAST) Innovation Fund under Grant No. 2018CAST33, Joint Fund of China Electronics Technology Group Corporation, and Equipment Pre-Research under Grant No. 6141B08231109.