#### Abstract

Existing channel estimation schemes for wideband systems generally estimate the channel matrix by pre-estimating the common support, which has a limited validity owing to the effect of beam squint. It is worth noting that the learned approximate message passing (LAMP) network need not pre-estimate the support and has obtained a relatively reliable estimation quality in the narrowband systems. In order to degrade the performance penalty caused by inaccurate estimation of the support in the wideband millimeter-wave MIMO systems, a prior-aided Gaussian mixture LAMP (GM-LAMP) network combining with residual learning is presented. Specifically, the multicarrier Gaussian mixture threshold shrinkage function is constructed for the GM-LAMP network, which can directly estimate the wideband beamspace channel while avoiding pre-estimating the support; then, considering the impact of channel noise and the coarse estimation error by the GM-LAMP, a residual network (ResNet) is designed to improve the estimation performance. Simulation results validate the efficiency of the proposed network (referred to as GLAMP-ResNet) with the lower computational complexity compared with the existing schemes.

#### 1. Introduction

Beamspace channel estimation is essential for wideband millimeter-wave (mmWave) Multiple-Input Multiple-Output (MIMO) [1, 2] systems with lens antenna array to select a small number of dominant beams with larger path gains [3] and reduce the number of radio-frequency (RF) chains [4] for the lower hardware cost and power consumption. However, the effect of beam squint (Beam squint is used to imply that the support (i.e., the index set of non-zero elements in a sparse vector) of wideband beamspace channel tends to be frequency-dependent) [5] brings new challenges for the beamspace channel estimation, especially under the wider wideband and the larger number of antennas [6].

By exploiting the sparsity of beamspace channel, the beamspace channel estimation can be formulated as a sparse signal recovery problem [7]. Existing channel estimation schemes for wideband systems generally estimate the channel matrix under the common support assumption in frequency domain [8–10], which has a limited validity owing to the effect of beam squint and inaccurate estimation of frequency-dependent support. Specifically, a simultaneous orthogonal matching pursuit (SOMP)-based method is presented in [8]. Therein, the channel estimation is expressed as a multiple measurement vector problem with a common support (i.e., the channel support at different frequencies is assumed to be the same), which can be solved by the SOMP algorithm. In 2017, the orthogonal matching pursuit (OMP)-based algorithm [9] was proposed. It first estimates the support of the wideband beamspace channel at some frequencies independently by the OMP algorithm. Then, it combines them into the common support at all frequencies. Unfortunately, the common support assumption in [8, 9] has limited validity in the practical wideband mmWave MIMO systems. To circumvent this problem, the successive support detection (SSD) algorithm [10] is proposed, where each support is uniquely determined by its spatial direction at the carrier frequency. Yet, it assumes a fixed sparsity for different frequencies, which will discard more nonzero elements of the components with greater sparsity.

Notedly, the studies in [8–10] adopt the scheme of pre-estimating support to estimate the wideband beamspace channel. However, therein, different frequencies share a common support and identical sparsity, which is not completely consistent with the actual channel. It is worth noting that the learned approximate message passing (LAMP) network need not pre-estimate the support and has obtained a relatively reliable estimation quality in the narrowband systems [11, 12]. If we can construct the threshold shrinkage function (TSF) of the LAMP network for wideband systems, the possibility of achieving satisfactory estimation accuracy will increase.

In this paper, a prior-aided Gaussian mixture LAMP (GM-LAMP)-based network combining with the residual network (ResNet) is presented, referred to as the GLAMP-ResNet. The contributions of this paper can be summarized as follows:(1)The GM-LAMP network is proposed to obtain a preliminary estimation of the wideband beamspace channel. In particular, the multicarrier Gaussian mixture TSF (mcGM-TSF) is constructed in the GM-LAMP network. Different from the channel estimation methods in [8–10], the mcGM-TSF need not pre-estimate the support, which can degrade the performance penalty caused by inaccurate estimation of the support in the wideband millimeter-wave MIMO systems. Furthermore, the mcGM-TSF exploits the prior information that beamspace channel elements follow the Gaussian mixture distribution, which can improve the estimation performance compared with the traditional TSF of the LAMP network.(2)The ResNet is designed to refine the preliminary estimation of the GM-LAMP network. Specifically, the ResNet contains several residual blocks and each consists of three convolutional layers, and the basic idea of ResNet is to learn the mapping from the output of GM-LAMP to the noise. After training, the ResNet can further reduce the impact of channel noise and refine the coarse estimation from the GM-LAMP network.(3)Simulation results verify that the proposed GLAMP-ResNet can achieve better channel estimation performance by normalized mean squared error (NMSE), which has the comparable computational complexity, compared with the traditional LAMP network.

#### 2. System Model

This paper considers an uplink wideband mmWave MIMO system with subcarriers, as shown in Figure 1, where the base station (BS) employs a lens antenna array with antennas and RF chains to serve single-antenna users. To characterize the dispersive wideband mmWave MIMO channel, the Saleh−Valenzuela multipath channel model [13, 14] is adopted. The spatial channel vector of a certain user at subcarrier can be presented aswhere is the number of paths, and are the complex gain and the time delay of the -th path, respectively, and is the spatial direction at subcarrier that can be defined as , where is the frequency of subcarrier with and presenting the carrier frequency and bandwidth, respectively, is the speed of light, is the physical direction, is the antenna spacing, and denotes the array steering vector, where [15].

The wideband mmWave MIMO system as Figure 1 employs a lens antenna array to convert spatial channel into beamspace channel, thereby this mmWave system is also often referred to as the beamspace MIMO system. The role of lens antenna array can be simulated by a spatial discrete Fourier transform matrix , whose elements denote the array steering vectors of predefined spatial directions. Specifically, the can be expressed as , where , are the spatial directions. Then, the beamspace channel vector of a certain user at subcarrier can be expressed as .

In the uplink channel estimation, the orthogonal pilot transmission strategy is used and a certain user is considered without loss of generality. After instants of pilot transmission, the received pilot vector at BS after combining, CP removal, and point FFT can be presented aswhere is the transmitted pilot symbol, is the noise vector with representing the noise power, is the effective noise vector and of size () is the combining matrix, whose elements can be randomly selected from the set with equal probability if the adaptive selection network is realized by low-cost 1 bit phase shifters [16]. Combining all frequencies with assuming [10], the overall measurement matrix can be obtained aswhere is the measurement matrix, is the beamspace channel of a certain user, and is the effective noise.

#### 3. Wideband Beamspace Channel Estimation

##### 3.1. Pre-Estimation Based on GM-LAMP

In the proposed GM-LAMP network as shown in Figure 2, the overall measurement matrix will be inputted to the AMP network for training layer parameters under the Gaussian mixture channel prior assumption (Considering the previous contribution of modeling beamspace channel with the Gaussian mixture distribution [17]). For the -layers GM-LAMP network, the -th layer channel pre-estimation is as follows:where is one of the layer parameters to be trained, for with first-layer inputs , , and . The and are called Onsager correction [18], which can be obtained by calculating the derivative of with respect to . It should be noted that contains the channel data of users and each user has one carrier in narrowband mmWave systems [11]. However, a single user has multicarriers in wideband mmWave systems, so the dimension of in (4) is (i.e., has the channel characteristics of subcarriers). In this paper, the mcGM-TSF is constructed to make it suitable for wideband mmWave systems. The in (4) can be presented bywhere is the noise variance in the -th layer, is the number of Gaussian variables following to the Gaussian mixture distribution, , ,and , respectively, represent the probability, mean, and variance of the -th Gaussian variable in the -th layer; the set of previous distribution parameters is a trainable matrix, generally called the shrinkage parameters, and can be expressed aswhere denotes the probability density function of the -th Gaussian component in the -th layer.

By constructing the mcGM-TSF as (5) in the GM-LAMP network, the pre-estimation of wideband beamspace channel can be obtained by (4). After iterations, the final output of the GM-LAMP network can be written as .

##### 3.2. Residual Optimization Based on ResNet

Considering the impact of channel noise and the coarse estimation error by the GM-LAMP network, the ResNet is introduced to narrow the gap between the pre-estimated channel and the label value . Compared with some traditional convolutional neural networks, e.g., AlexNet, VGG, and GoogleNet, the ResNet has achieved promising performance by introducing fast links that directly pass the data flow to later layers, which can ease the training of the neural network. As a result, signal attenuation caused by multiple stacked nonlinear transformations can be effectively avoided and faster training speed can be achieved. Therefore, this paper introduces the ResNet as an important module to further improve the estimation performance. The proposed ResNet is composed of multilevel residual blocks, where each block consists of convolutional layers and activation function layers as shown in Figure 3.

In the offline training phase, and are the input and label of the ResNet, respectively, and the corresponding residual optimization problem can be modelled aswhere is the optimization variables, also the trainable parameters, and is the output of the ResNet.

##### 3.3. Structure of the Proposed GLAMP-ResNet

In the proposed GLAMP-ResNet scheme as shown in Figure 2, the number of Gaussian variables in the shrinkage function is set as , and the mean and variance of all Gaussian variables are both set as considering the sparsity of the wideband beamspace channel. The convolution kernels of , , and the features mapping of , , are adapted, respectively, to the first, second, and third convolution layer of each residual block (The values of the hyper-parameters, such as the size of the convolution kernels and the number of feature maps are mainly chosen according to the scale and dimension of the dataset; furthermore, the trade-off between estimation accuracy and computational complexity also needs to be considered). In addition, the number of residual blocks is equal to (considering an attractive trade-off between the computational complexity and system performance). Then, the output of the GLAMP-ResNet can be written aswhere and are the mapping function in the GM-LAMP network and ResNet, respectively.

#### 4. Simulation Results

In our simulations, a lens antenna array with and is used. The number of users and instants of pilot transmission is set as and , respectively. The channel parameters are set as follows [19, 20]: (1) ; (2) for ; (3) for .

In order to better determine the number of GM-LAMP network layers (*T*) and the number of residual blocks (*F*), the NMSE performance comparison experiments have been conducted. The comparison results are shown in Figures 4 and 5, respectively. Relying on the classic AMP network, the number of *T* can be determined through the NMSE performance of the AMP network under different layers. It can be seen from Figure 4 that the NMSE performance can hardly be improved with increasing layers (*T*) when *T* is equal to 8, thereby the optimal number of *T* is determined as 8. Similarly, the number of residual blocks (*F*) can be determined by testing the NMSE performance under different number of residual blocks. It can be seen from Figure 5 that the improvement of NMSE performance reaches one limitation when the number of F is increased to 3, thereby the number of F is set to 3.

Figure 6 shows the NMSE performance of the proposed GLAMP-ResNet compared with the existing wideband beamspace channel estimation algorithms. It can be seen that the performance of the SOMP algorithm [8] and the OMP algorithm [9] are inferior, because they both assume that the support of sparse channel is the same in frequency domain. In the SSD algorithm [10] the support is differentiated by the spatial direction, thereby the corresponding performance is significantly improved. From another perspective, the AMP algorithm introduces the threshold shrinkage function to pass the approximate channel estimation among different layers of the AMP network; however, the parameters of different layers are fixed, which induces the limited performance; in contrast, the Learned AMP (LAMP) can optimize the layer parameters by deep learning, and the NMSE performance can be improved by about 4 dB, which has been verified in this paper; then, the channel prior information of beamspace channel elements is exploited as the GM-LAMP algorithm, or the LAMP channel pre-estimation is combined with residual learning as the LAMP-ResNet, which can bring 1 dB and 3 dB performance improvement compared with the LAMP network, respectively; inspired by them, the GLAMP-ResNet is proposed with the NMSE up to 7 dB compared with the SSD algorithm.

To further verify the validity and universality of the GLAMP-ResNet, this paper tests the NMSE performance of the trained GLAMP-ResNet under different number of subcarriers (*M*) as Figure 7. It can be seen from Figure 7 that the NMSE performance is relatively better under , because the number of subcarriers in the training set is . When the number of subcarriers is changed during online estimation, it is found that the NMSE performance does not change significantly, which indicates that the NMSE performance of the GLAMP-ResNet is hardly influenced by the number of subcarriers. Besides, Figure 8 also verifies the stability of the proposed method under different bandwidth.

For the computational complexity, a single convolutional layer in the ResNet needs the computation as [21, 22], where and are the spatial lengths of the filter and the output feature map, respectively, and are the number of input and output channels, respectively, and the calculation time of the AMP and GM-LAMP is comparable to ; here, , , and are the number of AMP network layers, subcarriers, and antennas, respectively. Since the product is far larger than in mmWave MIMO systems, the computational complexity of the GLAMP-ResNet can be approximated as . By contrast, the complexity of both the OMP and SOMP algorithm can be presented by , and the complexity of the SSD algorithm is . Here, (e.g., ) denotes the sparsity level of the beamspace channel. Thus, the proposed scheme can improve the system performance without significantly increasing the computational complexity.

#### 5. Conclusion

This paper introduces one GLAMP-ResNet scheme for wideband beamspace channel estimation in mmWave MIMO systems, which mainly involves the following works: (1) constructing the multicarriers Gaussian mixture threshold shrinkage function for the GM-LAMP network to pre-estimate the wideband beamspace channel, not the support, thus the key challenge of channel estimation error from support estimation bias is avoided; (2) the strategy of pre-estimation combined with the second denoising can improve the final channel estimation accuracy without significantly increasing computation. Besides, the proposed channel estimation scheme can be applied in the other communication systems.

#### Data Availability

The simulation code and data used to support the findings of this study are available from the corresponding authors upon request.

#### Conflicts of Interest

The authors declare that they have no conflicts of interest regarding the publication of this paper.

#### Acknowledgments

This study is partly supported by the National Science Foundation of China: 61871176 and 61901159; the Applied Research Plan of Key Scientific Research Project for Henan Colleges and Universities: 22A510013.