Abstract

A prototype filter design for FBMC/OQAM systems is proposed in this study. The influence of both the channel estimation and the stop-band energy is taken into account in this method. An efficient preamble structure is proposed to improve the performance of channel estimation and save the frequency spectral efficiency. The reciprocal of the signal-to-interference plus noise ratio (RSINR) is derived to measure the influence of the prototype filter on channel estimation. After that, the process of prototype filter design is formulated as an optimization problem with constraint on the RSINR. To accelerate the convergence and obtain global optimal solution, an improved genetic algorithm is proposed. Especially, the History Network and pruning operator are adopted in this improved genetic algorithm. Simulation results demonstrate the validity and efficiency of the prototype filter designed in this study.

1. Introduction

As an alternative modulation scheme for the fifth-generation (5G) communication, offset quadrature amplitude modulation-based orthogonal frequency division multiplexing (FBMC/OQAM) outperforms orthogonal frequency division multiplexing (OFDM) with respect to spectral efficiency, side lobes, and sensitivity to asynchronous transmission [1, 2]. As one of the important components of FBMC/OQAM systems, the prototype filter determines the performance of the whole system.

In recent years, many effective methods about prototype filter design have been brought up. All these methods can be divided into two classes [3]: direct approaches and indirect approaches. Indirect approaches mainly include the windowing-based method [4, 5] and the frequency-sampling method [68]. Direct approaches optimize all coefficients of the prototype filter. Due to many more degrees of design freedom, direct approaches have the potential to achieve better performance than indirect approaches [9].

However, the complexity grows rapidly along with the number of coefficients in the direct approach. In order to reduce the computational complexity, the -based branch and bound algorithm is used to design the prototype filter in [10]. In [11], the box-based branch and bound (Box-BB) method based on algorithm is proposed to decrease the number of iterations. Spectral factorization is employed to design the final prototype filter in [12], but this method is unstable because the factor is chosen randomly. The existence of unknowns in the aforementioned methods makes it difficult to obtain the global optimum.

The genetic algorithm (GA) is a powerful evolutionary algorithm which is able to solve this difficulty [13, 14]. GA is adopted to handle the thinning optimization problem in [15]. The Baldwin effect is added into GA via a History Network to avoid the inappropriate answer in [16], but there are so many random operators in this method. In [17], the random assignments of value during initialization and mutation operator are deleted by the pruning operator to reach the answer more quickly. In this study, an improved GA based on the History Network and pruning operator is proposed to deal with these random operators.

Furthermore, accurate channel estimation (CE) at the receiver is indispensable for reliable signal detection. There have already been some studies on the CE for the FBMC/OQAM system.

In [18], two modified CE methods were proposed by placing guard symbols at different sides of reference preamble symbol, the existence of guard symbols reduces spectrum efficiency. In order to increase spectrum efficiency, coded auxiliary pilot symbols that also carry information are designed in [19]. However, at least two to four pilot symbols are needed to achieve acceptable performance in practice. In [20], an efficient preamble structure is proposed to achieve a better CE with less preamble consumption. Because there is no pilot in the compressed sensing method, CE methods based on compressed sensing are proposed further to increase the spectral efficiency. In [21], when prior sparse knowledge is unknown, an adaptive, regularized, compressive sampling matching pursuit chooses the support set to achieve better CE accuracy. Inspired by the same theory as in [21], two improved CE methods are derived, respectively, based on the interference approximation method (IAM) and pair of real pilots in [22]. The key to these methods is that how to prove that the FBMC/OQAM system is sparse. However, whether the system, like the scattering channel, is sparse is debatable.

Few studies have taken the CE into account in prototype filter design. In [23], the prototype filter is designed by utilizing the mean square error (MSE) of the CE. However, zero symbols inserted in the preambles will reduce the system’s spectral efficiency. Inspired by the preamble structure described in [20], preamble without guard symbols is considered in these studies to acquire higher transmission efficiency. The proposed structure will improve the accuracy of CE by increasing pseudo pilot power.

The influence of noise may be also ignored in prototype filter design. For example, in [24], the weight signal-to-interference ratio is treated as the utility function to design the prototype filter under highly doubly dispersive channel without noise. However, the noise power is generally equal to or even larger than the interference power in practical systems. Therefore, the noise influence should be given sufficient attention in prototype filter design. Usually, the signal-to-interference plus noise ratio (SINR) is adopted to measure the noise influence. A bigger SINR means better system performance. Therefore, it will be difficult to set the threshold when the SINR is added to the constraints. Under this condition, in this study, the reciprocal of the signal-to-interference plus noise ratio (RSINR) is adopted to measure the influence of noise.

Simulation results indicate that the proposed preamble structure achieves outperformance compared to the IAM, the improved GA shows less convergence time and average generations than the proposed GAs in [16, 17]; the designed prototype filter in this study is better than in [23, 24] when it comes to the performance of the stop-band energy and the magnitude response.

The rest of the study is organized as follows: In Section 2, the FBMC/OQAM model is described, the preamble is proposed, and the CE is obtained. In Section 3, the process of prototype filter design is formulated by minimizing the stop-band energy with constraints on the RSINR, and the GA with the History Network and pruning operator is utilized to solve the optimization problem. The results of the simulations are discussed in Section 4, and finally, a conclusion is given in Section 5.

2. System Model

As one of the contributions of this paper, the preamble structure with larger pseudo pilot power and spectrum efficiency is introduced in this section.

At first, the FBMC/OQAM system model is built. Instead of transmitting complex symbols , the FBMC/OQAM system transmits the real-valued symbols which are from the constellation mapping of the real components and imaginary components . According to [25], the continuous-time baseband equivalent of the transmitting signal in FBMC/OQAM can be written aswhere denotes the number of subcarriers, denotes the subcarrier spacing, denotes the constellation interval, and denotes the offset between the real part and imaginary part of the OQAM symbol. denotes the prototype filter function. is an additional phase term:where can be chosen arbitrarily. For simplification, in this study, .

Considering the multipath channel with an additive white Gaussian noise, which is denoted as , when the transmitted FBMC/OQAM signal is passed through, the baseband version of the received signal is described aswhere denotes the channel impulse response (CIR) and presents the maximum delay spread of the channel. Especially, we assume that in the time interval for the flat fading channel. In that case, rewrite (3) aswhere represents the channel impulse frequency response.

The demodulation signal at the position can be written aswhere represents the inner product of and and .

The orthogonal condition in FBMC/OQAM iswhere denotes real part of the argument and denotes the Dirac’s delta function.

Thus, the FBMC/OQAM system model has been built. After that, in order to design the new preamble structure, the relationship between preamble and CE is necessary to be analyzed.

2.1. Relationship between Preamble and CE

We define , , where , and then, the inner product of the pulse shaping filter in equation (5) can be rewritten aswhere denotes conjugate operation. Since , , and , equation (7) can be rewritten aswhere denotes the ambiguity function of the filter, which can be expressed as

When is an even function, its instantaneous autocorrelation function is also an even conjugate. is the inverse Fourier transformation of , and therefore, is a real-valued function [26].

We can get equation (10) from equation (5):

According to the conventional OFDM CE method, the channel frequency response can be estimated by when the pilot symbol is transmitted at time-frequency position .

We define , and and are, respectively, the neighborhoods of and . and , where denotes the neighborhood of position . With the increase in and , becomes very close to zero. For example, for an isotropic orthogonal transform algorithm (IOTA) function, when , we have [27]

From equation (11), we can conclude that one column of zero symbols is enough to prevent the preamble from being interfered by adjacent unknown data symbols. However, the zero symbols affect the CE performance. The structure of the IAM is shown in Figure 1. It can be seen from Figure 1 that there are two columns of zero symbols for IAM. According to [27], when only one column of zero symbols is inserted in the preamble, its CE performance is worse than the IAM structure.

Since and are of inverse signs, equation (10) can be rewritten as

We assume . The CE based on the IAM can be obtained bywhere . can be treated as the pseudo pilot. Equation (13) implies that a larger pseudo pilot power leads to a better CE performance. The power of pseudo pilot for the IAM is shown in the following equation:where denotes the variance of the transmitted FBMC/OQAM symbols .

Obviously, CE based on the IAM has two drawbacks: (1) the preamble duration is , which leads to the low spectrum efficiency, and (2) the second item in equation (13) cannot be ignored when both the signal-to-noise ratio and constellation mapping order are high.

The fact is that the power of pseudo pilot is mainly affected by the pilot symbols that surround for the IAM. Therefore, it can be increased by changing the structure. Additionally, the zero symbols in the IAM will be redundant if the intersymbol interference is alleviated by the preamble.

In order to overcome two drawbacks of the IAM, a new preamble structure is proposed in this paper. It is shown in Figure 2. For sufficient accuracy and concise description, only three-tap interference of neighborhood symbols is considered.

The pseudo pilot power of the proposed structure is

From (15), we can conclude that the proposed preamble structure is able to improve CE performance for its larger pseudo pilot power. Obviously, it also increases the frequency spectrum efficiency.

2.2. RSINR

In order to take the influence of prototype filter on CE into account while designing the prototype filter, the RSINR is chosen to measure these influences. The expression of RSINR based on the CIR is derived in this section.

First, we rewrite equation (10) aswhereand then, the channel frequency impulse response is estimated by

Because in equation (18) belongs to a random signal, its value cannot be determined during transmission. As a result, the term is unknown. Equation (19) will be invalid if is unknown. To solve this problem, the predecision method is adopted. The process of estimation based on the predecision method is shown as follows.

First, the initial channel frequency response can be obtained from (16):

Next, in the term can be rebuilt based on zero-forcing equalization. Estimate based on the predecision method, which is shown as follows:

Here, denotes the predecision operator.

Then, can be estimated by

Finally, since has been obtained, the CE is obtained by substituting into equation (19). Equation (23) shows the final channel frequency response expression:

By inverse Fourier transform, the CIR of the FBMC/OQAM system is obtained as follows:

We then discretize the CIR into form as follows:where denotes the smallest integer larger than or equal to and denotes the sampling period.

The desired power of the system is closely related to the transmitted symbol. The transmitted symbol at the position can be estimated as follows:

The QAM symbols are reconstructed by combining

We then definewhere is the number of channel taps at the sampling rate and .

The desired power of the system iswhere denotes the variance of the complex symbol . According to the definition of the variance, is the real number.

Similarly, the interference power and the noise power are expressed as [28]where denotes the variance of the noise .

Thus, the expression of RSINR is obtained by equations (29)–(31):

3. Prototype Filter Design

3.1. Problem Formulation

The prototype filter in the FBMC/OQAM system is assumed to be real-valued, symmetric, and of unit energy. According to the previous discussion, when there is intrinsic interference and noise in the system, the prototype filter will influence the CE. Therefore, the RSINR will be chosen as an important constraint in problem formulation. At the same time, the prototype filter design should minimize the stop-band energy. Thus, the prototype filter design problem can be formulated aswhere is the magnitude response of the prototype filter, represents a constant threshold, and denotes the length of the prototype filter. According to [29], .

The prototype filter used to be designed without considering the interaction between prototype filter and CE. Therefore, utilizing RSINR to measure the influence of prototype filter on CE and regarding RSINR as the constraint of prototype filter design problem are also one of the contributions of this paper.

There are many variables in P1. The objective function and constraint functions all have quadratic forms. Therefore, it is difficult to solve the optimization problem P1 directly. As everyone knows, orthogonal transformation does not change the nature of the original function. However, it can be employed to change the form of (33).

First of all, the first constraint function in (33) implies that only half of the filter coefficients are independent. In order to reduce the number of variables, we substitute with a new variable :

The objective function can be rewritten with a vector form as , where is a real symmetric positive-definite matrix. , respectively, denote positive eigenvalues in ascending order, and , respectively, denote corresponding unit orthogonal eigenvectors.

Then, the orthogonal transformation is applied to the variable , where denotes the new orthogonal variable.

Finally, the optimization problem after the variable substitution and orthogonal transformation is expressed as

3.2. Problem Solution

In this part, an improved GA with History Network and pruning operator is proposed to solve the optimization problem expressed in (35).

All the potential global answers of GA exist in a search space; Gorges in GA will expand this search space. Further, the inappropriate space may be searched many times during iterations. These increase the convergence time of GA.

In [16], the knowledge of search space learned by the previous generation is passed on to the next generation via an History Network to narrow the search space, and the Baldwin effect is applied to avoid searching the same space in different iterations. Figure 3 shows the flowchart of the proposed GA in [16].

However, the influence of random operators ignored in [16] will slow down the convergence speed of GA. The random assignment operators may result in invalid iterations, early termination, and jumping out of the loop with local optimum. Inspired by [17], the pruning operator is applied in this paper to control these random operators. Pruning operator is used to take away the inappropriate amounts resulting in increment convergence time from population and reach the global optimal answer more quickly. Flowchart of improved GA in this study is shown in Figure 4.

It can be seen from Figure 4 that the improvement of the improved GA proposed in this paper mainly exists in four parts: initialization, DRS, History Network, and mutation.

The pruning operator of the initialization part directly deletes the invalid subset of solution to narrow the domain at the beginning.

The time order of the pruned initialization is shown aswhere presents the number of genomes in each chromosome, presents the number of chromosomes of each generation, and is determined by the pruning function. It is shown aswhere is the reduced ratio of search space.

In DRS part, the range of direction change is limited by pruning operator. As a result, those individuals are discarded who are so different to avoid being too far from the last reasonable direction.

There are many exemplars stored in the History Network. It is required to limit the variation of the exemplars to a certain range. Pruning operator is applied to avoid either duplication with existing exemplars or drastic changes during the updating process.

Mutation is critical for reaching optimal solution. In the process of mutation, the value of genome is replaced randomly to create new generation for the next iteration. However, the random substitution likely results in inappropriate value. It requires a lot of iterations to eliminate the impact of inappropriate values on the search. As a result, iterations in the whole algorithm are increased. Pruning operator is able to help appoint appropriate value as much as possible. In all circumstances, the better the appointment of value during mutation, the faster it is to reach the best answer.

The time order of the pruned search loop is shown aswhere presents the solution space.

Since the redundant random operators have been pruned, the computational complexity of the whole algorithm is certainly reduced.

4. Simulation Result

4.1. Problem Formulation

The CE performance based on the proposed preamble is simulated in this simulation. The truncation of an IOTA filter with length [30] is chosen as the prototype filter. The fundamental parameters are listed in Table 1.

Figure 5 illustrates the CE performances of the proposed structure (A) and the IAM (B) for two kinds of constellation mapping. For 16-QAM modulation, when the signal-to-noise ratio (SNR) is low (SNR < 15 dB), the CE performances of A are slightly better than those of B. Along with the SNR increasing, the CE performance of A becomes better and better and the advantages become more and more obvious. When SNR > 15 dB, the IAM performance curve tends to flatten due to the performance platform. Similarly, for 4-QAM modulation, the IAM performance curve tends to flatten when SNR > 20 dB.

The CE performances of two methods with different number of subcarriers for 4-QAM modulation and 16-QAM modulation are also, respectively, compared in Figures 6 and 7. It is obvious that the curve of A with 512 subcarriers is lower than the one with 256 subcarriers in Figure 6, so is B. It implies that a larger number of subcarriers mean a better CE performance. These two curves of A are both lower than those of B for the same number of subcarriers. Curves in Figure 6 show that A is better than B about 0.4 dB for 256 subcarriers and 1 dB for 512 subcarriers at . Modulation of curve 16-QAM in Figure 7 also illustrates the same conclusion. Therefore, the proposed preamble structure has advantages with respect to CE.

In order to lower the impact of predecision errors, CE with different number of iterations is simulated in Figure 8. Perfect bound represents the upper bound of performance. The curves with iterations show better performance than the curve without iterations, which indicates iteration can reduce the predecision errors and improve the CE performance. The difference between curves with different number of iterations is obvious when the number is less than four. But the curve with four iterations is almost overlapped with the curve with five iterations. The reason why curves are overlapped when the number of iterations are more than four is that the value of is close to its true value after four iterations. Therefore, there is no need to carry out more than four iterations in order to save time and reduce the complexity.

4.2. Improved GA performance

The performance of the improved GA is analyzed in this section. According to [16], values of parameters in History Network are assigned as follows: the Good Threshold, the Bad Threshold, and the Update Threshold are set to 10%, 25%, and 10% of the mutation radius, respectively. The mutation radius is defined as the maximum Euclidean distance from the old search space position before mutation to the new position after mutation. The rest of the parameters of the improved GA are exhibited in Table 2.

The average generations and the convergence time of different algorithms are, respectively, shown in Figure 9 and Table 3. Figure 9 implies that the generations of other three algorithms are much more than the improved GA proposed in this paper. It illustrates that the pruning operation in the History Network deletes the invalid random value assignment and avoids redundant individuals. As can been seen from Table 3, the improved GA proposed in this paper reduces the convergence time varying from 8% to 22.1% when compared to the standard GA, the accelerated GA in [16], and the pruned GA in [17]. It reveals that the improved GA proposed in this paper can reduce the time to reach the best answer and accelerate the convergence speed. In conclusion, the combination of pruning operation and History Network is more effective than just utilizing any one of them.

4.3. Performance of the Designed Prototype Filter

In this section, the validity of the designed prototype filter is verified through a series of simulations.

The average BER performances of FBMC/OQAM system with different number of subcarriers for three prototype filters are, respectively, shown in Figures 10 and 11. The prototype filters designed by method A and method B are, respectively, described in [23, 24], and method C is the one in this paper. The simulation results in Figure 10 reveal that the average BER performance of the system with the prototype filter designed by method C is superior to the other two methods for M = 256. It can be seen from Figure 10 that method A and method B show similar BER performance when SNR is larger than 14 dB. Meanwhile, method B and method C show similar performance when SNR is smaller than 8 dB. Although there are some intersections between these three lines in Figure 10, the outperformance of method C is obvious when SNR is larger than 10 dB. Similar conclusion can be drawn from Figure 11.

Since the proposed filter is designed with a constraint on the RSINR, not only the influence of noise is considered to guarantee the effectiveness of the method but also the desired power is increased as the interference power decreases. However, the influence of noise which should not be ignored in practice was not considered in [23]. The existence of the noise destroys the efficiency of the method proposed in [23], especially when SNR of the system is at the low level. The desired symbol power was also not considered in [23], which may have obtained the solution for the optimization problem not being the minimum one.

In [24], the objective of the prototype filter design is to maximize the signal-to-weighted interference radio. Large stop-band energy at high SNR level destroys the orthogonal condition, resulting in a much poorer BER performance than the method proposed in this paper.

Figure 12 shows the magnitude responses of IOTA, the filter designed in [23] and in [24], and that designed in this study with M = 256. The red curve in Figure 12 shows that the first side lobe of the prototype filter designed in this study is about −40 dB. It is better than the filter designed in [23] and that designed in [24] of about 8 dB and 9 dB. It is also superior to IOTA of about 1.5 dB. A lower side lobe means a better time-frequency localization property. In order to further reveal the outperformance of the prototype filter proposed in this study, Table 4 represents the stop-band energy of different filters. It shows that the stop-band energies of the IOTA filter, the filter designed in [23], and that designed in [24] are −32 dB, −43 dB, and −41 dB, respectively. The proposed prototype filter achieves the lowest stop-band energy (−51 dB). It means the proposed prototype filter outperforms three other filters in filtering clutter. Because the proposed prototype filter is designed with constraints on RSINR and the noise influence has been taken into account, the proposed prototype filter in this study shows better performance in filtering clutter.

5. Conclusion

In this study, a new prototype filter was designed based on the CE for the FBMC/OQAM system in frequency-selective channels. The proposed method is able to minimize the stop-band energy and increase the CE performance. RSINR, which is utilized to measure the influence of the prototype filter on the CE, is an important constraint for prototype filter design. A new preamble structure was proposed to save the frequency spectrum before the RSINR calculation. Then, an optimization problem of prototype filter design was formulated to minimize the stop-band energy with constraint on the RSINR. Finally, an improved GA is employed to solve this optimization problem by utilizing a History Network and a pruning operation in order to obtain low convergence time and the global optimal solution. Simulation results verified that the preamble structure proposed in this paper increased the CE accuracy and spectral efficiency, the improved GA accelerated the convergence speed, and the designed prototype filter outperforms other filters.

There are still many works that are worth doing in the future. On the one hand, if the preamble symbol is encoded, the spectrum efficiency can be further increased; on the other hand, the History Network is the key part of the proposed GA; how to set up the History Network and choose its exemplar more efficiently relate to the performance of the whole algorithm.

Data Availability

No data were used to support this study.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Acknowledgments

This work was supported by the National Natural Science Foundation of China under Grant no. 61671468.