#### Abstract

An iterative intercell interference cancellation algorithm is introduced to improve the receiver performance of uplink transmission in multicell networks. At first, the uplink signal detection is performed independently in each cell according to minimum mean squared error (MMSE) criterion. Subsequently, the detection results are applied to reconstruct the transmit signals of different users and cancel their interference to neighboring cells. With the help of reconstruction results, the MMSE detection matrix of each cell is updated. The channel responses of both efficient and interference links are estimated with the help of pilots. The pilot allocation parameter is introduced to indicate the quality of channel estimation. The simulation results indicate that intercell interference can be greatly mitigated by the proposed algorithm with a moderate number of receiver antennas at the base station.

#### 1. Introduction

Multicell cooperative receiver processing [1, 2] has attracted much attention from academia in recent years. The efficient cooperation algorithm can efficiently cancel the intercell interference and improve the spectral efficiency. In [3, 4], the belief propagation method in multicell detection is proposed and the soft detection results are iteratively transferred in the cell-user factor graph. Furthermore, the two-dimensional (2D) intercell model is introduced in [5] and the Gaussian belief propagation algorithm is proposed to mitigate the intercell interference. Based on the 2D model proposed in [5], the algorithm in [6] proposes a kind of two-dimensional belief propagation equalizer to improve the uplink receiver performance. Recently, a novel measure call leakage capacity is introduced in [7] to optimize the multicell cooperative processing, which has distributive property. However, the detection performance at the receiver is not fully analyzed in [7].

In the existing research [3–6], the multicell detection is mainly based on the idea of belief propagation, and its complexity increases exponentially with the number of transmit antennas. When the number of transmit antennas becomes large, the complexity of belief propagation algorithm is very high. Therefore, this paper introduces an iterative multicell interference cancellation algorithm based on minimum mean square error (MMSE) criterion under multiantenna configuration conditions. It is noted that the idea of MMSE-based iterative processing has been widely used. For instance, in [8], the MMSE-based iterative algorithm is only applied for channel estimation, while signal detection is not included in the iterative processing. In addition, in [9], the iterative intercell interference is performed on the downlink transmission of MC-CDMA (Multicarrier Code Domain Multiple Access), while the multiantenna receiver is not applied at the receiver and the effect of channel estimation error is not fully considered.

In this paper, the iterative intercell interference cancellation in multicell uplink transmission is proposed. At first, channel estimation is performed with the help of uplink pilots. In order to suppress the pilot contamination [10, 11] in neighboring cells, the pilot sequences in different cells are orthogonal with each other. This scheme is called nonreused pilot configuration [12]. Based on the channel estimation results, the iterative intercell interference cancellation algorithm is proposed. With channel estimation results, the base station (BS) in each cell performs uplink signal detection individually and reconstructs its interference to other cells. Subsequently, the reconstruction uplink signals are exchanged between neighboring cells for further interference cancellation. In our algorithm, the interference cancellation can be iteratively executed to increase the accuracy of signal reconstruction in each cell. From the simulation results, it is shown that the proposed scheme achieves performance close to that of interference-free case with a moderate number of receiver antennas at the base station.

According to the above descriptions, our contributions mainly include the following two points. The first contribution is that the iterative detection algorithm based on minimum mean squared error (MMSE) criterion is introduced. The detection results are exploited to reconstruct the intercell interference, which is cancelled at the receiver in the uplink transmission. During iterative processing, the MMSE receiver matrix is updated based on the signal detection result in the previous iteration. According to the proposed iterative algorithm, the detection performance of the uplink transmission is improved. The second contribution is that the detection at the receiver is implemented based on the estimated channel responses. The channel estimation error is fully considered and its effect is verified in the simulations.

In the following, lower and upper bold face letter denote the vector and matrix, respectively. For the matrix , , , and denote its transpose, Hermitian, and pseudoinverse, respectively. In addition, denotes a vector whose element is taken column-wise from matrix . The identity matrix is denoted by . The operator () denotes the statistical expectation. For a complex scalar , denotes its complex conjugate. The operator denotes the Kronecker product.

#### 2. System Model

In this paper, our focus is on uplink signal transmission in multicell scenario. We assume that there is only one base station per cell and the base station has multiple antennas. Since each cell has only one base station, the cell index also denotes its corresponding base station. Furthermore, we assume that base stations of multicells can exchange detection results through backhaul link during iterative signal detection. In the following, we assume that the base station of each cell has the same number of antennas and each cell has the same number of users. Consequently, the base station antenna number of each cell is denoted by and the user number in each cell is denoted by . Moreover, the overall cell number in the uplink transmission is denoted by . The above scenario description is reflected in Figure 1 with .

In the multicell scenario shown in Figure 1, the uplink transmission in the th cell is affected by the users in the neighboring cells. For clearness, when describing the uplink signal model in the th cell, the neighboring cell index is denoted by , . In addition, there are multiple users in each cell. Consequently, the intercell channel response from user in cell to BS is given by , where denotes receiver correlation corresponding to , and is a Rayleigh fading channel coefficient vector. With users in the th cell, is introduced to denote the channel coefficient matrix from users in the th cell to BS .

In the following analysis, the block fading assumption is applied, in which the channel response maintains constant during one coherent interval. In addition, the pilot sequence is inserted into the beginning of each coherent interval and the duration of the pilot transmission is called the training phase.

In order to describe the pilot configuration, the pilot allocation parameter , is introduced to indicate pilot reuse in the -cell scenario shown in Figure 1. For example, with , this means that the pilot sequences of different cells are not the same. When , this means that at least two cells share the same pilot sequence set.

During the training phase, denotes the pilot sequence of user in cell , in which denotes length of pilot sequence. In the following, the pilot sequence length is set equal to . At the same time, the overall possible pilot sequence number is set equal to . With Chu sequences [13] applied in this paper, the above pilot length and number setting ensures orthogonal property between different pilot sequences. During pilot sequence allocation, we assume that users in the cell can choose different pilot sequences. Therefore, the channel estimations of users in the same cell have no interference with each other. The pilot contamination only comes from intercell pilot reuse, which depends on the pilot allocation parameter . For example, with and , the above setting means that users in cell 1 have different pilot sequences. At the same time, users in cell 2 share the same pilot sequences with cell 1. Consequently, the pilot contamination between cell 1 and cell 2 exists.

In this paper, the transmit power of each pilot sequence is proportional to its length , so we have . During the training phase, the received signals corresponding to the pilots at BS are packed into the following matrix formulation.where denotes the received signal matrix at BS during training phase, denotes the channel response from user of cell to BS , and is the additive white Gaussian noise matrix at BS whose elements have zero mean and variance.

According to the above analysis, when the pilot allocation parameter , the second term in expression (1) introduces pilot contamination. As , the pilot sequences of users in different cells are orthogonal with each other, and the effect of pilot contamination is eliminated.

After pilot transmission, the data symbols of users are transmitted. In the following, denotes the data symbol of user in the th cell. With intercell interference, the received signal at the base station of the th cell is given bywhere the second equation transforms the signal model into the matrix formulation, , and is an additive white Gaussian noise (AWGN) vector.

Based on the above two expressions, the following discussion is mainly focused on the iterative intercell interference cancellation algorithm with pilot-aided channel estimation.

#### 3. Channel Estimation and Intercell Interference Cancellation

At first, the channel estimation is realized based on the pilot sequences. The estimation is realized according to minimizing mean squared error (MMSE) criterion. Afterwards, the iterative intercell interference cancellation algorithm is introduced.

##### 3.1. Channel Estimation

With the pilot allocation parameter , more than one cell has the same pilot sequence set. In addition, we assume that the user number in each cell is equal to . Consequently, the set is introduced to denote user index set which has the same pilot sequence to user in cell .

In channel estimation, the channel response is obtained based on the received signal and the pilot sequence. In order to better represent the process of channel estimation, the received signal matrix in (1) needs to be rewritten in the vector formulation , whose dimensionality is equal to . In order to accommodate the received signal , the pilot sequence is extended to the following matrix expression , which has rows and columns. Therefore, the received pilot signal model shown in (1) can be rewritten as follows: where .

The pilot setting described in Section 2 makes the extended pilot matrix has the following property:

According to the above property, by left multiplying , we havewhere denotes the intercell pilot contamination and , which obeys complex Gaussian distribution .

Based on MMSE criterion [14], the estimation result of is given by In addition, the intercell channel response , can be also obtained by the process similar to expression (5) and (6).

In the following, we define as channel estimation error of . Consequently, the covariance matrix of the above error can be given by Moreover, for the matrix , denotes its channel estimation error. And the covariance of is given by Similarly, the intercell channel estimation error and its covariance can be also achieved.

In the next subsection, the iterative intercell interference cancellation algorithm is implemented based on the above channel estimation results.

##### 3.2. Iterative Intercell Interference Cancellation

For simplicity, the proposed iterative algorithm is firstly given in two-cell scenario. In each cell, there is only one user. The two-cell signal model is given byThe iterative structure with two cells is given in Figure 2. When iteration number is equal to , we define and as the estimation result of the interferences and , respectively. In addition, the residual interference at iteration is given as follows:and their covariance matrices are given by When no cancellation happens with , we have andwhere and denote the channel estimation error covariance based on the analysis shown in the above section.

For , the following expectations is required to calculate the linear MMSE (LMMSE) estimators of and of . They are given byand

Based on (12)-(14) and LMMSE estimation in [14], the signal detection and intercell interference construction can be denoted by the following expressions:and the covariance matrices of estimation errors are given by

After and are exchanged between the two BSs, they are subtracted from the received signal. So we have

When the iterative processing converges, and are used to generate the final detection result of and . Otherwise, they are used to calculate the estimator and for based on expression (13)-(15).

Furthermore, for the general multicell networks, we define to be the interference estimation result from cell to BS in the th iteration. At first, we define . The estimation error at iteration is denoted by . Similar to (10), we define the covariance matrix of , Similar to that in (12), we haveBefore estimating the interference to other cells at one iteration, each BS subtracts from its received signal the interference estimated by other cells. Similar to (17), the signal after subtraction at BS of iteration is defined as , which is denoted byIt is noted that interference from cell to BS is independent of estimation error . To obtain the LMMSE estimator of using at iteration , we need the following expressions:and which can be written asApplying (19), (20), and (22), we obtain the LMMSE estimator of at iteration asand the covariance matrix of the estimation errorIn addition, the LMMSE estimator of at iteration can be expressed as

##### 3.3. Distributed Implementation

In practice, interferences from nonneighboring cells are much smaller than those from neighboring cells. For the sake of complexity, we assume that each BS only interferes its neighboring BSs. Generally speaking, for any BS , the set contains its neighboring BSs. Consequently, at BS , the covariance matrices and shown in (22) and (24) are generated from and with . Because of the block fading assumption, needs to be built only once for one coherent interval in the th iteration.

##### 3.4. Complexity Analysis

According to the above analysis, for interference cancellation of cells, the main complexity comes from expression (22)-(25). With known , is firstly calculated according to expression (25) and its complexity is . Afterwards, with and , expressions (23) and (24) are achieved, whose complexity is equal to and , respectively. Finally, according to expression (22), the updating of in the th iteration has a complexity of .

With cells, the overall complexity in one iteration is given by

#### 4. Numerical Results

In this section, the performance of the proposed iterative interference cancellation algorithm is given. Users in each cell are randomly distributed. The intracell receiver correlation matrix is given by . At the same time, the intercell receiver correlation matrix is given by . The parameter is defined as similar to that in [15], where is the distance between the user of cell to BS and is the distance between the user with the same index of cell to BS . According to the above analysis, the parameter reflects the strength of intercell interference. In addition, the above setting makes the received signals of different antennas be independent with each other. The transmit modulation symbols of the users are 4QAM symbols. The length of pilot sequence is given by , in which the pilot allocation parameter is configurable. Unless specifically declared, the MMSE channel estimation results are applied in the proposed iterative algorithm. The distributive implementation indicates that intercell interference cancellation only performs between neighboring BSs.

In Figure 3, the bit error rate (BER) performance of the proposed algorithm with is given. The simulation parameters in Figure 3 are given in Table 1.

From Figure 3, it can be seen that the pilot allocation parameter is equal to and respectively, when . In this case, there is no intercell interference. When , the length of pilot sequence is longer than that with . Consequently, the channel estimation performance with is better. The simulation result shown in Figure 3 indicates the BER performance advantage of over the case with when .

In addition, the performance curve with and can be seen as the lower bound of the case with . From Figure 3, it can be seen that after 1 iteration, the performance curve with and can approach the lower bound with and . This means that the proposed algorithm can efficiently cancel the intercell interference when . In addition, when and , the intercell interference becomes large. After 2 iterations, the performance loss of and is about 1 dB and 1.3 dB compared with the lower bound with and when BER approaches .

In order to facilitate the effect of channel estimation, the simulation results with different pilot allocation parameters in scenario are given in Figure 4. The pilot allocation parameter is equal to , , and , respectively. When becomes larger, there exist more severe channel estimation errors. With , the proposed algorithm can achieve better performance that with and . In addition, the results in Figure 4 show that, after 2 iterations, the BER performance convergence can be achieved when is equal to , , and .

In Figure 5, the scenario with and is considered. The corresponding parameters are given in Table 2. For comparison, the performance curve of the intercell interference-free case with and is given. It can be seen that, with , the BER performance is better than that with because lower parameter indicates longer pilot sequences and more reliable channel estimation results. In addition, it can be seen that, compared with the case with perfect channel estimation, the performance loss of is smaller than 1 dB when BER reaches . In the following, the curve with and can be seen as the lower bound of the proposed iterative algorithm with . Figure 5 shows that, after 2 iterations, the performance curve with and has about 1.4dB loss compared with the above lower bound.

In addition, it can be seen from Figure 1 that the distribution of cells presents a hexagonal structure. This means that there are at most 6 neighbors for each cell. In addition, we assume that the intercell interference mostly comes from neighboring cells. Therefore, the pilot allocation parameter is set equal to with . Under this setting, the pilot contamination only comes from nonneighboring cells. Because of large path loss, the effect of nonneighboring cells can be ignored. The simulation results with and support the above analysis. It can be seen that, with 2 iterations, the performance curve with and approaches that with and . This indicates that the interference of nonneighboring cells has very little effect on the proposed iterative algorithm.

Furthermore, the convergence property of the proposed algorithm is given in Figure 6. When , the BER performance can achieve convergence after 2 iterations, with equal to , , and , respectively. The performance gain is very marginal with more than 2 iterations. When is equal to , the channel estimation performance is the same for the cases with and . The simulation results shows that the performance with and is better than that with and since fewer neighboring cells introduces less intercell interference. At the same time, it is noted that the case with and can also converge after 2 iterations.

#### 5. Conclusions

In this paper, an iterative intercell interference cancellation algorithm is proposed. The principle of this algorithm is based on minimizing mean squared error (MMSE) criterion. In addition, the concrete expressions of the iterative cancellation are derived. From the numerical results, it can be seen that intercell interference can be mitigated efficiently with a moderate number of antennas.

#### Data Availability

The data used to support the findings of this study are available from the corresponding author upon request.

#### Conflicts of Interest

The authors declare that they have no conflicts of interest.

#### Acknowledgments

This work was supported by National Natural Science Foundation of China (61601047, 61671080, and 61871050) and Huawei HIRP Project HIRPO20161103.