#### Abstract

In this letter, we propose a virtual channel (VC) optimization approach with a closed-loop and adaptive scheme for overloaded MIMO systems. With this approach, each input data stream goes through a VC which is generated at the transmitter; then it is transmitted to a receiver through the actual wireless channels. The VCs are concatenated with the actual wireless channels. Through VC optimization, the values of which can be adjusted to reduce the channel correlation, leading to a much improved system performance. Compared to the conventional overloaded MIMO systems, the overloaded MIMO systems with this approach can achieve significantly better performances in terms of the system capacity and symbol error rate (SER). The method that uses genetic algorithm (GA) for finding the optimal VC vector is described. Simulation results illustrate the effectiveness of the proposed approach.

#### 1. Introduction

Massive multiple-input multiple-output (MIMO), also known as large MIMO, has been considered as one of the most promising candidate technologies for future 5G communications due to its ability to achieve high throughput and spectral efficiency and enhance the energy efficiency of sensor networks [1–5]. In a massive MIMO system, the base station (BS) is equipped with an antenna array of hundreds or even thousands of antennas to simultaneously serve tens or hundreds of mobile terminals in the same time-frequency resources [5–7]. Since the number of antennas is limited in a mobile terminal due to its physical size, the overloaded MIMO system in which the number of transmit antennas is larger than the number of receive antennas is considered [8, 9]. Because the rank of the channel matrix equals the number of receive antennas, but less than the number of transmitted data streams in this case, the channel matrix becomes rank-deficient [10–12], and then the performance of the minimum mean square error (MMSE) precoding leads to the floor effect. In an overloaded MIMO system, the channel correlation practically exists, which may lead to a serious system performance degradation, especially in the massive MIMO systems. In [13], an open-loop and nonadaptive precoding scheme with predetermined coefficients is proposed for the overloaded MIMO-OFDM systems to reduce the correlation effect. A WL-MMSE precoder employing real-valued transmit symbols for the downlink large-scale MIMO systems presented in [14] has shown that it achieves a substantially higher sum rate than the systems employing the conventional MMSE precoding. In [15], an iterative coordinate Tomlinson-Harashima precoding (THP) algorithm is proposed for the overloaded multiuser MIMO systems to achieve better BER and sum rate performances. But the improvement of these methods is still very limited. Thus, more effective methods that mitigate the channel correlation effects for the overloaded MIMO systems are highly desirable.

In this letter, we propose a virtual channel (VC) approach with the closed-loop and adaptive scheme for a downlink overloaded MIMO system, namely, the VC overloaded MIMO system. With this approach, each input data stream goes through a VC which is generated at the transmitter before it is transmitted to a receiver through the actual wireless channels. Since the VCs are concatenated with the actual wireless channels, the system performance not only depends on the actual wireless channels but also depends on the VCs. Therefore, by optimizing the VCs according to feedback information the channel correlation can be largely reduced. Thus, compared to the conventional overloaded MIMO systems, that is, the nonprecoded ones, an overloaded MIMO system with VCs can provide a much improved performance.

Thus, the main contributions of this letter are summarized as follows.

Propose a closed-loop and adaptive VC optimization approach that can significantly mitigate the channel correlation effect and improve the system performance for an overloaded MIMO system in terms of the system capacity and the symbol error rate (SER).

Present a genetic algorithm (GA) to find the optimal VC vector for a VC overloaded MIMO system and show that it outperforms the conventional overloaded MIMO systems through simulation results.

*Notations*. We use upper and lower case boldface to denote matrices and vectors, respectively. , , , , , , , and denote the transpose, conjugate, conjugate transpose, expectation, absolute value square, determinant, identity matrix, and trace of a matrix, respectively. denotes a circularly symmetric complex Gaussian random variable with mean zero and variance .

#### 2. System Model

Consider a downlink VC overloaded MIMO system equipped with transmit antennas and receive antennas (), as shown in Figure 1. At the transmitter, each input data stream goes through a VC and is designated to the th () transmit antenna, where and are the amplitude and phase of , respectively. For convenience, it is assumed that all input data streams are statistically independent; that is, (; ; ) and the power of is normalized; that is, . All the input data streams form the input signal vector, which can be expressed as . Also, let denote the VC vector of the system corresponding to the input signal vector . Therefore, the signal transmitted by the th transmit antenna is

The received signal vector can be described aswhere is the additive white Gaussian noise vector, and the components of are statistically independent with each component distributed according to . is the total transmit signal power. denotes the actual wireless channel matrix; is the wireless channel vector corresponding to the th transmit antenna. is transmit signal vector.

When , the VC overloaded MIMO system becomes a conventional one, that is, the nonprecoded one.

From Figure 1, it can be seen that the VCs can be optimized according to the feedback information like SNR by using proper algorithms. In this letter, GA is applied as the optimization algorithm to find the optimal VC vector to achieve better performance in terms of capacity and SER for the VC overloaded MIMO system.

#### 3. Performance Optimization for the VC Overloaded MIMO System

##### 3.1. Capacity

The capacity of a VC overloaded MIMO system is given as [16, 17]wherewhere is a correlation matrix of the transmit signal vector . Then from (3) and (4), the capacity can be rewritten as

##### 3.2. SER

For arbitrary* J*-QAM signals, the SER of the VC overloaded MIMO system can be expressed as [18] where is a function defined as and SNR denotes the total received signal-to-noise ratio of the VC overloaded MIMO system, which is given aswhere is the noise power at the receiver.

Then from (6) and (8), the SER can be rewritten as

##### 3.3. VC Vector Optimization Using GA

From (5) and (9), it can be seen that in order to obtain the maximal capacity and the minimum SER, we should maximize the received SNR, since both capacity and SER appear to be monotone functions of this quantity. Therefore, the VCs should be properly adjusted to achieve the maximal SNR under a transmit signal power constraint. To attain an optimal VC vector of the system, we consider maximizing the received signal power since the noise power can be viewed as a constant. Therefore, the optimization criterion is stated as

The optimization criterion (10) can be realized by a GA algorithm which is based on the natural genetics and can find the global optimal solution with comparatively low complexity [19, 20]. The performance of a GA algorithm is largely affected by the relevant parameters such as the population size, the number of generation, the crossover probability, and the mutation probability. Hence, we consider applying GA to adaptively search for the optimal VC vector .

Suppose the total number of generations is , and the size of population is , which denotes the total number of individuals in GA. The VC vector is designated as an individual in the population. At the generation, the individual is defined as . According to (10), we define as the fitness function of GA. Then the steps to optimize the VC vector using GA are as follows.

*Step 1 (initialization). *Set the generation counter and set a reasonable number for . Randomly generate all the individuals of the initial population represented by chromosomes having the form of binary strings with length .

*Step 2 (fitness calculation). *Calculate the fitness function value of each chromosome in the initial population.

*Step 3. *(i)Update the generation counter .(ii)Selection: rank all chromosomes from the best to the worst according to the fitness function values , and select the parents to create offspring for the next generation based on the fitness proportionate method. Compute the probability of the chromosome at the generation to be selected as /.(iii)Crossover: apply a single point crossover operator to do the crossover operation. Randomly select a crossover point (an integer between 1 and within the chromosome). Then a pair of the selected parent chromosomes swap the information with a crossover probability after the crossover point to produce two child chromosomes.(iv)Mutation: make a small change in the child chromosomes to lead a broader searching space, with a mutation probability of being toggled (e.g., or ) for every bit in the child chromosomes.(v)Calculate the fitness function values of the chromosomes in the new population.

*Step 4 (stopping criterion). *If = , the algorithm stops and choose the individual with the largest fitness function value from the new population as the optimal VC vector . If , go back to Step 3.

From the above steps, it can be seen that there is no computation unmanageable in any step and the proposed algorithm is practically implementable. Furthermore, the computational complexity of GA is approximately , where arises from the selection probability computation, and accounts for the computational complexity of the fitness function values.

#### 4. Simulation Results

In this section, the simulation results are presented to compare the capacity and SER performances of the VC overloaded MIMO systems with those of the conventional overloaded MIMO systems. Here, we assume that is an actual wireless channel matrix for static channels and is perfectly known at the receiver. The GA is applied to find the optimal VC vector. The population size of GA is chosen to be . We set other initial parameters as , , and .

In Figure 2, the capacities of the VC overloaded MIMO systems are compared against that of the conventional overloaded MIMO systems for different antenna configurations. It is observed that the VC overloaded MIMO system shows better performance than the conventional overloaded MIMO system in terms of the system capacity with any antenna configurations. Furthermore, with a larger number of transmit antennas and receive antennas, the effectiveness and superiorities of the VC overloaded MIMO system can be better exhibited.

Figure 3 shows the SER performances of the VC overloaded MIMO systems, in comparison with those of the conventional overloaded MIMO systems and the overloaded MIMO systems with open-loop precoding for different antenna configurations. It can be seen that the overloaded MIMO system with open-loop precoding achieves much lower SER than the conventional overloaded MIMO system at high SNR but performs worse than the VC overloaded MIMO system. The VC overloaded MIMO system attains a gain of about 4.5 dB in SNR over the conventional overloaded MIMO system at . Moreover, the VC overloaded MIMO system performs better than the conventional overloaded MIMO system, at any SNRs. Furthermore, as shown in Figure 3, in all cases the VC overloaded MIMO systems outperform the overloaded MIMO systems with open-loop precoding, exhibiting a great advantage in improving transmission reliability.

Figure 4 shows the convergence property of GA for a VC overloaded MIMO system under different conditions. It can be seen that GA with a larger population size has a better chance to find the optimal solution (Figure 4(a)). However, the crossover probability and the mutation probability which lead to the best convergence property are moderate values as shown in Figures 4(b) and 4(c). Furthermore, Figure 4(d) shows a good convergence property of GA at different SNRs. Therefore, in order to obtain the best convergence property, a reasonable combination of the parameters should be set up for GA.

**(a)**

**(b)**

**(c)**

**(d)**

#### 5. Conclusion

In this paper, we propose a closed-loop and adaptive VC optimization approach that can significantly improve the performances of the downlink overloaded MIMO systems according to feedback information. Specifically, by using GA to find the optimal VC vector, the performance optimization can be achieved. Simulation results demonstrate that the VC overloaded MIMO system outperforms the conventional overloaded MIMO system as well as the overloaded MIMO systems with open-loop precoding in the system capacity and SER.

#### Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

#### Acknowledgments

This work was supported by the University Doctoral Research Foundation of China (Grant 20130181110006).