Abstract
To accelerate the data acquisition speed of magnetic resonance imaging (MRI) and improve the reconstructed MR images’ quality, we propose a parallel MRI reconstruction model (SPIRiTNet), which combines the iterative selfconsistent parallel imaging reconstruction model (SPIRiT) with the cascaded complex convolutional neural networks (CCNNs). More specifically, this model adopts the SPIRiT model for reconstruction in the kspace domain and the cascaded CCNNs with dense connection for reconstruction in the image domain. Meanwhile, this model introduces the data consistency layers for better reconstruction in both the image domain and the kspace domain. The experimental results on two clinical knee datasets as well as the fastMRI brain dataset under different undersampling patterns show that the SPIRiTNet model achieves better reconstruction performance in terms of visual effect, peak signaltonoise ratio, and structural similarity over SPIRiT, Deepcomplex, and DONet. It will be beneficial to the diagnosis of clinical medicine.
1. Introduction
Magnetic resonance (MR) imaging (MRI) is a modern clinical medical tool with multiparameter, multisequence, multidirectional, and high soft tissue resolution. But acquiring fully sampled kspace data tends to take a long time. Reducing the number of kspace data can accelerate the acquisition speed, but undersampled kspace data will cause artifacts in the reconstructed image. Therefore, how to reconstruct MR images with high quality from undersampled kspace data has become a significant problem.
Parallel imaging is an effective way to accelerate the acquisition speed, which uses a set of multichannel coils to acquire data simultaneously and encode spatial information using the sensitivity differences of the coils to reduce the number of necessary gradient encoding steps and the time required for imaging [1]. Reconstruction methods for parallel MRI mainly include the sensitivity encoding (SENSE) [2], the generalized autocalibrating partially parallel acquisitions (GRAPPA) [3], and the iterative selfconsistent parallel imaging reconstruction (SPIRiT) [4]. Among them, SPIRiT is a widely used coilbycoil autocalibrating model for parallel MRI, which avoids the sensitivity information estimation in the SENSE reconstruction model due to the implicit use of sensitivity information. To improve the reconstructed images’ quality, the SPIRiT model was combined with the joint total variation regularization term [5, 6], simultaneous 2D low rankness in kspace [7], nonlocal lowrank constraints [8], and joint sparsity and sparsifying transform learning [9].
Recently, deep learningbased methods have been proposed to reconstruct parallel MRI from undersampled kspace data. Deep learning models for MR image reconstruction can be categorized into two main groups: one of them is the endtoend nonexpanding iterative methods that directly use a standard network structure to learn the mapping from input to output. Specifically, Lee et al. [10] proposed a deep residual learning network composed of magnitude and phase networks separately trained for reconstructing parallel MRI. Huang et al. [11] proposed a variant of Unet with channelwise attention (CWA) as reconstruction block together with long skip connection technique. Sriram et al. [12] proposed an endtoend variational network for parallel MRI reconstruction. Yiasemis et al. [13] proposed a recurrent variational network (RecurrentVarNet) for reconstructing parallel MR images via multiple recurrent blocks. In recent years, other networks such as transformer [14] and diffusion model [15] have also been used in MRI reconstruction tasks to further improve the quality of reconstruction and achieve good results.
The other is a physical modelbased method that starts from a hypothetical optimization problem and unfolds the iterative optimization algorithm to obtain a deep network. Specifically, Hammernik et al. [16] proposed a variational network was proposed to learn critical parameters for parallel MRI reconstruction from fully sampled multicoil kspace data. Aggarwal et al. [17] proposed a modelbased reconstruction framework with a deep learned prior (MoDL), which involved dataconsistency terms and learned convolution neural networks (CNNs) to capture redundant information of images. Sriram et al. [18] proposed the GRAPPANet model by combining the GRAPPA model with deep neural networks to achieve improved reconstructions. Pramanik et al. [19] proposed a modelbased deep learning framework (DeepSLR) to accelerate the structured lowrank (SLR) matrixcompletion algorithm significantly, which trained a CNNbased filterbank to estimate the annihilation relations.
To improve the reconstruction performance, we propose the SPIRiTNet model that simultaneously utilizes the correlations of multicoil kspace data and complex convolutional neural networks (CCNNs) to capture redundant information of parallel MR images. Specifically, the SPIRiT calibration block and the kspace data consistency layer are used to reconstruct the undersampled kspace data, and then the cascaded CCNNs with dense connections and image domain data consistency layers are used to further reconstruct the calibrated kspace data.
Our main contributions are as follows: First, we use SPIRiT for initialization of undersampled kspace data. Second, we use a CCNN with dense connection for further capturing the redundant information of the MR images reconstructed by SPIRiT. Third, we conduct experiments on two clinical knee datasets as well as the fastMRI brain dataset. The experimental results show that our model achieves better results in terms of visual effects and quantitative metrics compared to the SPIRiT, Deepcomplex, and DONet models.
The outline of this article is given as follows. We propose a parallel MRI reconstruction model (SPIRiTNet) by combining the cascaded CCNNs with the SPIRiT model in Section 2. Simulated experimental results and analysis are given in Section 3. Finally, Section 4 summarizes this paper.
2. Methods
2.1. Notation and Preliminaries
In this article, the domain of real numbers and the domain of complex numbers are denoted as and , respectively. The relevant mathematical manipulation operators are denoted by Eulerian letters, the matrices are denoted as boldface capital letters, and some basic constants are denoted by unbolded capital letters. We provide the symbols used in Algorithm 1 proposed in this paper along with their corresponding meanings in Table 1.

2.2. Problem Formulation
The undersampled kspace data of a multicoil image are given by the following equation:where denotes the multicoil image to be reconstructed, and denote the real and imaginary parts of , denotes the multicoil undersampled kspace data, and denote the real and imaginary parts of , denotes the total number of received coils, denotes the number of single coil undersampled data points, denotes the total number of pixel points of the image to be reconstructed, and denote the number of rows and columns of the image. denotes the system encoding matrix, and denote the undersampling operator and Fourier transform operator for multicoil images. is an identity matrix of size , denotes the Kronecker product, denotes the undersampling matrix sampled from kspace, and denotes the 2D Fourier transform operator [8] for singlecoil images. The parallel MRI reconstruction based on deep learning can be represented as follows:where denotes the forward mapping function of the deep learning network parameterized by , which includes millions of adjustable network weights. is the data fidelity term.
2.3. Algorithmic Framework
In contrast to realvalued natural images, MR images are complex, and each pixel point contains magnitude and phase information. Some of the networks use the magnitude image for training [20]. Some of the networks consider the real and imaginary parts separately [19]. The former methods neglect the phase information during reconstruction, and the latter methods lose the correlation between the real and imaginary parts. The CCNN can take both magnitude and phase information into account [21]. Therefore, we choose the CCNN as the benchmark network for feature learning.
The SPIRiTNet mainly consists of a kspacedomain reconstruction block and an imagedomain reconstruction block. The kspacedomain reconstruction block consists of a SPIRiT calibration block (SCB) and a kspace data consistency (KDC) layer. The imagedomain reconstruction block consists of a complexvalued convolution block (CCB) and an imagedomain data consistency (IDC) layer. Figure 1 illustrates the architecture of the proposed SPIRiTNet.
2.3.1. SPIRiT Calibration Block
In the SCB, the multigroup convolution kernels are first estimated from the autocalibration signals (ACS), and then used to reconstruct the missing kspace data (linear interpolation), as shown in Figure 2.
Let denote the coil of the kspace data , and denote the kspace value of the coil at position , denote the kspace points in the neighborhood of position , denote a set of convolution kernels estimated from the ACS, and is the conjugate transpose of . The reconstruction of the point of the coil at position is given by the following equation:
can be obtained by solving the following optimization problem:
Equation (3) can also be expressed in matrix form as follows:where is a matrix containing a series of convolution kernel operators at appropriate positions.
2.3.2. kSpace Data Consistency Layer
After the undersampled kspace data are calibrated, kspace data consistency operations are applied, as shown in Figure 3. Kspace data consistency operations can be expressed as follows:where and are the operators that select the acquired and nonacquired points from the whole kspace respectively, and are the operators that put them back in the right positions in the kspace, respectively. Therefore, denotes the zerofilled kspace data.
2.3.3. ComplexValued Convolution Block
kspace data reconstructed by SCB and KDC are fed into the imagedomain reconstruction block after inverse Fourier transformation. The input of the imagedomain reconstruction block is as follows:
The imagedomain reconstruction block consists of cascaded CCBs and IDC layers. Each CCB contains convolution units , where , as shown in Figure 4. Each convolution unit includes a complex convolution unit and a rectified linear unit (ReLU), except the last convolution unit only contains a complex convolution unit. The convolution kernel of each complex convolution unit is of size , and and denotes the real and imaginary parts of . The output feature map for the convolution unit of the () CCB is as follows:where , , and are the input feature maps for the convolution unit of the CCB.
In addition, the output feature maps of each previous layer are concatenated to ensure that the output features can be reused. Let denote the dense concatenation operation, then the final output feature map for the convolution unit of the CCB is formulated as follows:
The final output of the CCB is as follows:
2.3.4. ImageDomain Data Consistency Layer
Each CCB is followed by an imagedomain data consistency (IDC) layer to ensure acquired kspace data unchanged. The IDC layer is shown in Figure 5. The output of the IDC layer is formulated as follows:
It should be noted that the output of the IDC layer is directly used as the input of the CCB:
Then, we obtain a parallel MRI reconstruction algorithm SPIRiTNet based on the SPIRiT model and CCNN, which is summarized in Algorithm 1.
After the final multicoil image is reconstructed, the square root of sum of squares (SOS) method is adopted to obtain a magnitude image. SOS is defined as follows:
3. Experimental Results
3.1. Experimental Setup
The knee datasets and the fastMRI brain datasets are used to evaluate the performance of all the reconstruction models [16]. The knee datasets were obtained using a clinical 3T MRI scanner with two different sequences called “Coronal Proton Density (CoronalPD)” and “Sagittal Proton Density (SagittalPD),” respectively. The scan parameters of the CoronalPD weighted sequence are TR = 2,750 ms, TF = 4 ms, TE = 27 ms, matrix size = 320 × 288 × 15, and voxel size = 0.49 × 0.44 × 3 . The scan parameters of the SagittalPD weighted sequence are TR = 2,800 ms, TF = 4 ms, TE = 27 ms, matrix size = 384 × 307 × 15, and voxel size = 0.46 × 0.36 × 3 . Each sequence consists of 20 subjects. Twenty slices from No. 11−30 for each subject are selected. All the slices are padded or cropped to 320 × 320 pixels to ensure that all images have the same size. Fourteen patients are randomly selected for training, three for validation, and three for testing. The fastMRI brain datasets were obtained using a clinical 3T MRI scanner. The T1 postweighted sequences are selected for experiment, 50 patients’ data for training, eight patients for validation, and eight patients for test, eight slices for each patient are selected. All the slices are also padded or cropped to 320 × 320 pixels to ensure that all images have the same size.
All compared models are trained and tested on a desktop computer with an Intel (R) Core (TM) [email protected] GHz processor, 64 GB of RAM memory, an NVIDIA GeForce RTX 3090 GPU (24 GB of memory) and an Ubuntu 20.04 operating system (64bit). We compare the proposed model SPIRiTNet with one traditional model SPIRiT [4] and two deep learning models (Deepcomplex [21] and DONet [22]). Except for SPIRiT implemented in Python, all other compared models are implemented using TensorFlow 2.4. The model parameters of SPIRiTNet are shown in Table 2. Besides, the proposed SPIRiTNet is trained using the Adam optimization algorithm and Kaiming initialization method [23]. ACS size of 24 × 24 is used for all the undersampling patterns for the SPIRiTbased algorithms. It should be noted that the small amount of unacquired data of ACS does not affect the calibration of SPIRiT.
We employ the peak signaltonoise ratio (PSNR) and structural similarity (SSIM) for the evaluation of reconstructed images’ quality. PSNR is defined as follows:where denotes the mean square error between the ground truth image and the reconstructed image , and denotes the maximum value of .
SSIM is defined as follows:where are constants, and denote the mean values of and , respectively, and denote the variance of and , respectively, and denotes the covariance of and . Larger PSNR and SSIM values indicate better reconstruction quality.
3.2. Evaluation of Quantitative Metrics
Tables 3 and 4 tabulate the quantitative metric results (average PSNR and SSIM values with standard deviation) for the reconstructed MR images via the compared models under the 3x and 5x accelerated 1D random undersampling (1DRU) pattern, 2D random undersampling (2DRU) pattern, 2D Poissondisc undersampling (2DPU) pattern, and radial undersampling pattern (RADU) for the CoronalPD and SagittalPD datasets, respectively. As shown in Tables 3 and 4, SPIRiTNet consistently outperforms SPIRiT, Deepcomplex, and DONet in terms of average PSNR and SSIM values in all cases. We also provide box plots of the average difference in PSNR between SPIRiTNet and the compared models (Deepcomplex, DONet, and SPIRiT) on the CoronalPD and SagittalPD datasets from the 3x and 5x accelerated 1DRU, 2DRU, 2DPU, and RADU patterns respectively, as shown in Figure 6.
(a)
(b)
(c)
(d)
Table 5 tabulates the quantitative metric results (average PSNR and SSIM values with standard deviation) for the reconstructed MR images via the compared models under the 4x and 8x accelerated 1D equispaced undersampling (1DEU) pattern for the real kspace raw data in the fastMRI brain dataset. It can be seen from Table 5 that the SPIRiTNet model achieves some improvement in the metrics results compared to SPIRiT, Deepcomplex, and DONet models.
3.3. Evaluation of the Visual Effect
We visually compare the reconstructed MR images via all the models. The reconstructed image and its corresponding fully sampled image are utilized to plot an error map, while 2x magnification is performed at the same location on both the reconstructed image and the error map to observe the reconstruction details. We choose different undersampling patterns, acceleration factors, and datasets to observe the reconstruction results.
First, it can be seen from Figure 7 that under the 1D undersampling pattern, there are many artifacts in the reconstructed image of SPIRiT model, Deepcomplex, and DONet greatly reduce those artifacts, and our proposed SPIRiTNet further removes the artifacts. Second, as can be seen from Figures 8−10, under the 2DRU, 2DPU, and RADU patterns, the four models achieve relatively clear reconstructed images. The Deepcomplex and DONet models achieve good results in the overall reconstruction quality. However, we can see from the zoomedin error maps that the SPIRiTNet model achieves the best reconstruction quality in terms of details. Third, Figure 11 shows the reconstruction results of the real kspace raw data from the fastMRI brain dataset. We can see that there are still obvious artifacts in the reconstructed image of SPIRiT. Deepcomplex and DONet models remove most of the artifacts, and the SPIRiTNet model further removes the artifacts. The above experimental results prove that the SPIRiTNet model can effectively remove the artifacts and improve the reconstruction MRI quality.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
(j)
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
(j)
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
(j)
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
(j)
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
(j)
3.4. Ablation Studies
To prove the effectiveness of the SCB and dense concatenation, we compare the baseline models SPIRiT and Deepcomplex, Deepcomplex with SCB, and Deepcomplex with SCB and dense concatenation (namely our proposed SPIRiTNet) on the CoronalPD dataset with the 2DPU pattern. It can be observed from Table 6 that introducing SCB to the baseline model Deepcomplex effectively improves the PSNR and SSIM values of the reconstructed images, while the feature reusability of the dense concatenation leads to further PSNR and SSIM improvement. This indicates that both SCB and dense concatenation are indispensable and important parts of our proposed SPIRiTNet.
We also incorporate the SCB and dense concatenation with DONet. As shown in Table 7, DONet with SCB and dense concatenation achieves higher PSNR than DONet with SCB and the baseline models SPIRiT and DONet.
3.5. Significance Test
We perform the Wilcoxon signedrank test in PSNR between SPIRiTNet and the competing methods (SPIRiT, Deepcomplex, and DONet). Table 8 shows the values of the Wilcoxon signedrank test, and it can be seen that the values between the proposed SPIRiTNet and the competing methods are less than a significance level of 0.05 under all undersampling patterns and with different acceleration factors, which proves that the difference between our method and the other three competing methods is significant at a significance level of 0.05.
4. Conclusions
In this paper, we propose a parallel MRI reconstruction model (SPIRiTNet), which combines the cascaded CCNNs with the SPIRiT model. This model simultaneously utilizes the correlations of multicoil kspace data and a CCNN to capture redundant information of parallel MRI. Simulation experiments on two clinical knee datasets as well as the fastMRI brain dataset show that SPRiTNet outperforms SPIRiT, Deepcomplex, and DONet in terms of visual effects and quantitative metrics. Additional ablation experiments demonstrate that introducing the SPIRiT model to the deep learning network can effectively improve the reconstruction quality.
Data Availability
The knee data used to support the findings of this study have been deposited in the Zenodo repository at https://doi.org/10.5281/zenodo.10668971. The dataset contains 11−30th slices of 20 patients. Three patients (No. 7, 9, and 6) are randomly selected for validation, three patients (No. 1, 17, and 14) for testing, and other 14 patients for training. We also provide permanent identifiers (at https://doi.org/10.5281/zenodo.10556450) for our reconstruction results of the CoronalPD knee dataset and the fastMRI brain dataset. We provide theses .h5 files of our test results for one to verify the accuracy of our metrics. We also provide the Python code for reading .h5 files and calculating PSNR metrics.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Acknowledgments
The authors would like to acknowledge Kerstin Hammernik (CoronalPD dataset and SagittalPD dataset), Michael Lustig (SPIRiT code), Shanshan Wang (Deepcomplex code), and ChunMei Feng (DONet code) for making their in vivo datasets and codes publicly available. This work was supported in part by the Yunnan Fundamental Research Project (202301AT070452) and the National Natural Science Foundation of China (61861023).