Abstract

This paper presents a novel approach, based on the whale optimization algorithm (WOA), for channel estimation in wireless communication systems. The proposed method provides a means to accurately estimate the wireless channel, while not requiring the statistical characteristics of the channel. This method uses the WOA to search for the best channel statistical characteristics toward the ultimate goal of having the smallest bit error rate (BER). The proposed approach is aimed at enhancing the efficiency of pilot-based OFDM systems under frequency-selective fading channels used in the performance testing of 5G New Radio gNodeB. In terms of BER and mean square error (MSE), the performance of the proposed WOA-based channel estimation algorithm is evaluated and compared with the conventional least square (LS) and minimum mean square error (MMSE) algorithms. The simulation results demonstrate that the proposed algorithm provides highly competitive performance over the MMSE algorithm and significantly outperforms the LS algorithm in a variety of system configurations. Since the requirement on prior channel statistics information makes the MMSE algorithm impractical or extremely complex, the proposed WOA-based channel estimation algorithm should be a suitable and promising candidate for dealing with channel estimation problems. The simulation framework is implemented in MATLAB and available upon request.

1. Introduction

In wireless communication systems, orthogonal frequency division multiplexing (OFDM) has been the most suitable choice of physical layer waveform to mitigate multipath channel distortion. Although many studies propose a number of promising multicarrier waveform variants (for instance, FBMC, GFDM, and UFMC [1]) with advanced features, the OFDM technology remains a competitive candidate for future telecommunication systems (5G and beyond) due to its distinctive performance and computational efficiency. In these wireless systems, channel estimation is a crucial process to improve overall system performance. Consequently, numerous channel estimation techniques have been considered to accurately estimate the channel response in frequency and time domains.

The least square algorithm (LS) is widely used for pilot-based channel estimation due to its simplicity and independence from channel statistics knowledge [2]. However, its performance is significantly degraded in deep-null fading channels. The minimum mean square error (MMSE) algorithm, in contrast, offers superior performance. Unfortunately, the main drawback of MMSE is that it requires channel statistics and noise variance as prior knowledge, which makes the algorithm inapplicable in most practical deployment scenarios. In addition to these conventional algorithms, deep learning-based channel estimation approach has drawn more attention in recent works since deep learning models are expected to capture more complex statistical properties of practical wireless communication systems [35]. However, these models need to be trained before use, and a mismatch between training and application scenarios could negatively impact the system’s performance.

Recently, the whale optimization algorithm (WOA) has been proven to exhibit excellent performance in solving challenging optimization problems in a variety of engineering domains [610]. Specifically, the survey in [7] examined and categorized 82 publications that highlight the possibility of WOA addressing real-world issues. Motivated by this, our ideal is to reveal the potential of WOA to improve the channel estimation process in wireless telecommunication systems. In this paper, we first suggest a novel WOA-based channel estimation technique, then implement it, and prove its efficiency in specific 5G channels.

1.1. Related Work

For channel estimation, the most simple algorithm is the LS approach, but it poorly performs in highly dynamic environments. The most effective algorithm appears to be the MMSE which is usually used as a benchmark. The study [11] proposed a robust MMSE estimator that fully exploits the channel correlation properties in time and frequency domains. Some modified MMSE algorithms [12, 13] were suggested to reduce the computational complexity, while its performances barely deteriorate. It is worth mentioning that a mismatch between the estimated and true channel and noise statistics can significantly decrease the system performance. Therefore, it makes the MMSE impractical in such scenarios. A thorough overview of the MMSE channel estimation for OFDM and other waveforms is presented in [14]. The study in [15] proposed a channel estimation scheme in the time domain over fast fading channels. The key factor of this scheme is to add zero pad before the OFDM symbol. Another study [16] considered various pilot configurations to provide a means for reducing the pilot-to-data ratio. For mmWave and massive MIMO systems, the study [17] presented a grouping-based channel estimation approach, which exploits the sparsity of mmWave MIMO channels to improve the estimation accuracy and reduce the overhead. Recently, deep learning-based channel estimation and equalization have become popular topics in literature. For example, ChannelNet [4] and ReEsNet [5], DL models yield close performance to the MMSE algorithm. The majority of these articles evaluated the accuracy of the channel estimation using the BER and MSE metrics. Additionally, the performance of MMSE algorithm is frequently used as a benchmark for comparison.

The whale optimization algorithm has many benefits that make it a viable optimizer for wireless communications. In [18], the WOA algorithm efficiently resolves three issues related to resource allocation in wireless networks. Additionally, this study describes several potential WOA-based applications for multicarrier NOMA-enabled MEC systems, interference management in UDNs, and UAV trajectory optimization. In [19], a discrete version of WOA is used for topology control in wireless sensor networks. As a result, the number of active nodes is significantly reduced while maintaining network coverage and connectivity characteristics. In order to secure the data traffic in wireless mesh networks, the study in [20] employed a modified WOA to improve the attach detection ratio in intrusion detection systems. Similarly, an improved binary WOA was proposed in [21] to increase the classification accuracy and dimensional reduction in the feature selection of intrusion detection. Especially for antenna design, the work in [22] presented an antenna optimized based on the WOA, which is suitable for the WiFi 5 GHz frequency band.

As aforementioned, the WOA has found significant applications across a wide range of fields. Indeed, the WOA has demonstrated higher performance than recent metaheuristic techniques [6, 7]. For example, in comparison with other swarm intelligence techniques, it is robust and simple to implement. Moreover, fewer control parameters are needed for the method; practically, only one parameter (the time interval) needs to be adjusted. Despite the WOA’s superior performance, we are not aware of any comprehensive research dedicated to revealing the potential of WOA to improve the channel estimation process in wireless telecommunication systems.

1.2. Our Contribution

Our approach of utilizing the whale optimization algorithm for channel estimation has two significant advantages. Firstly, unlike the MMSE algorithm, it does not require prior information on channel statistics. Additionally, compared to deep learning-based channel estimation approach with a training procedure, the WOA-based channel estimation is more flexible in terms of deployment possibilities.

The main contributions of our work are summarized as follows: (1)We present a thorough solution to the pilot-based channel estimation problem based on the whale optimization algorithm (namely, WOA-CE), which can enhance the performance of OFDM systems under frequency-selective fading conditions(2)We conducted extensive experiments on the WOA-based channel estimation approach to analyze and compare its performance with the conventional LS and MMSE channel estimation algorithms in a variety of scenarios. Specifically, we evaluated overall system performance and channel estimation accuracy in terms of bit error rate (BER) and the mean square error (MSE), respectively, in the 5G NR TDLC-300 fading channel [23]. In addition, our simulation framework considered various modulation constellations (QPSK, 16QAM, and 256QAM) and three pilot patterns. The MATLAB simulation framework is available upon request from https://link.uit.edu.vn/WOA-CE

According to the statistical results from our simulation framework, the proposed WOA-based channel estimation algorithm outperforms the conventional LS algorithm and is comparable to the MMSE algorithm in terms of BER and MSE for considered scenarios.

The remainder of the paper is organized as follows: Section 2 provides a brief survey of the channel estimation background and conventional algorithms for pilot-based OFDM systems. Section 3 presents the proposed channel estimation approach based on the whale optimization algorithm. In Section 4, simulation results and discussion are presented. Finally, Section 5 concludes the paper.

2. Channel Estimation Background

In this section, channel estimation methods in the OFDM system are presented. OFDM has a frame structure in which a resource grid is conveyed on frequency-domain subcarriers and time domain OFDM symbols. complex-valued data and pilot symbols are included in the resource grid. The resource grid is transformed into a baseband signal at the transmitter using the inverse discrete Fourier transform (IDFT). A multipath channel with additive white Gaussian noise damages the transmitted signal. The discrete Fourier transform (DFT) is utilized at the receiver to create a received resource grid . The element of is stated as follows for the th subcarrier and th OFDM symbol:

Channel estimation is an important process where the frequency channel response is estimated to use in the next stages for equalizing the distorted resource grid . To estimate the channel, a part of the resource grid is used for inserting pilot symbols. Figure 1 illustrates a comb-type pilot pattern with pilot density. This pilot pattern has been introduced for fast fading estimation when the channel varies even inside one OFDM block. Firstly, channel response values at pilot positions, denoted as , are estimated. Then, the whole channel response matrix is obtained from by different algorithms. To increase the accuracy of the channel estimate process while keeping a respectable ratio of pilot symbols to the total, the number of pilot symbols and their placements should be carefully determined depending on the deployment situations.

Because of its simplicity, the LS estimator is frequently employed in practice. The LS method may be used to determine the frequency channel response at pilot points in the manner described below: where and are the elements of received resource grid and transmitted resource grid , respectively, at pilot positions.

The more advanced algorithm for channel estimation is MMSE which exploits knowledge of channel statistics. In this algorithm, the LS estimated values are multiplied with a weight matrix to provide the MMSE estimated values as follows: where is calculated as in [24] using the channel autocorrelation matrix at pilot positions and the cross-correlation matrix between channel LS estimated values and the true channel values. It is shown in [13] that the MMSE algorithm provides better performance than LS algorithm. Since the MMSE technique is computationally intensive and necessitates more channel statistics data, it is challenging to execute in practice.

3. Proposed WOA-Based Channel Estimation

In this section, we describe a novel method for applying the WOA to improve the accuracy of pilot-based channel estimation in wireless systems. This method is a promising candidate, since it provides superior channel estimation accuracy while not requiring the statistical characteristics of the channel.

3.1. Whale Optimization Algorithm

The solution to optimization problems has been studied for a long time and appears to be a crucial issue not only in mathematics but also in other scientific and technical disciplines. Many algorithms can give global optimal results for all nonlinear problems such as dynamic programming, branch-and-bound (BnB), and so on, but the calculation is very complicated. Metaheuristic optimization algorithms are becoming more and more popular and efficient in technical applications; they are based on fairly simple concepts inspired by nature by imitating biological or physical phenomena. Swarm techniques that mimic the social behavior of groups of animals have proven to be very dominant with physics and evolution-based algorithms. Therefore, swarm optimization algorithms work very well on communication applications requiring high speed.

Whale optimization algorithm (WOA) is a novel nature-inspired metaheuristic optimization algorithm that mimics the social behavior of humpback whales inspired by a spiraling bubble-net strategy [6]. WOA is tested by the author with 29 mathematical optimization problems and 6 structural design problems for optimization results and proved that WOA is very competitive compared with modern extreme search algorithms.

First, we define the optimization problem with the objective function as the mapping ( is also called fitness), in which is the number of dimensions of the variable and is the position vector of the object under consideration at time . The problem needs to determine the optimal position, through 3 methods of updating weights to simulate the behavior of humpback whales:

Encircling prey (exploitation phase): this is the stage when the whales have identified a target and are surrounded to attack the prey. Since any position in space, the WOA algorithm assumes that the current best solution is the target prey (or close to the optimal position). Once the best-looking member is identified, other members will try to update their location in the direction of the best search. This behavior is represented by the following equation: where is the next position vector and is the current best position vector. Vector (same dimension as ) indicates the direction to move from the current position: where is the random value in the interval and is the absolute value of each element. The value in (6) represents the degree of proximity to the prey and is calculated as follows: where is a random value in the interval and decreases linearly from 2 to 0 during the iteration (in both the exploration and exploitation phases), which can be determined through the following equations:

Therefore, the simulation surrounds the prey as it gets closer to the end of the iteration (the value of decreases), the amplitude of oscillation will decrease, and the information about the prey will be detailed, finding out the location more exactly.

Bubble-net attacking method (exploitation phase): the updated position will follow a spiral instead of a straight line as is the case with siege attacks. First, calculate the distance between the whale located at and the prey located at .

A spiral equation is then generated between the whale’s position and the prey to mimic the humpback’s twisting motion as follows: where is the constant to determine the shape of the logarithmic spiral and is the random number in . Humpback whales that swim around their prey in a constricting circle and along a spiral are better able to encircle them instead of heading directly to the current prey location. To model the concurrent behavior between the two mechanisms, it is assumed that there is a 50% probability to choose between a siege mechanism or a spiral model for updating the position of the whale during optimization ( is a random value in ).

Search for prey (exploration phase): the exploration phase approach is similar to the prey encirclement phase based on the variation of the value. If the absolute value is greater than 1, then force the foraging whale to move away from a certain member. The direction of movement of the whale during the search phase is where is a random position from a member. Whales randomly search each other’s positions according to the following equation:

3.2. Apply WOA to Channel Estimation

An optimization algorithm is used to search the statistical characteristics of the channel. Schools of whales move in -dimensional space, where is the number of channel features to know about the channel and noise. Each agent position represented a certain channel, and noise is updated over time by the WOA to reach the best position expected to be close to the true channel. The pseudocode of the original WOA algorithm is described in [6]. The definition of the best position is presented in the following section.

MMSE requires knowing noise variance and the cross-correlation matrix between the true channel and temporary channel estimated in the frequency domain. Therefore, the number of properties to know in advance is too large for the whales to perform well if applied WOA directly. Not to mention, this also increases computational complexity and time. We propose to use WOA to search for some statistical characteristics, thereby deducing the autocorrelation matrix of the channel.

The elements of and in Equation (4) can be obtained as follows [2]:

In an exponentially decreasing multipath power delay profile (PDP), the frequency-domain correlation is given as where is the subcarrier spacing and is the root mean squared (RMS) delay spread. The time domain correlation is calculated by the first kind of 0th-order Bessel function: where is the maximum Doppler frequency and is the OFDM symbol duration. From the requirement to know a lot of information about the channel, we now only need to know a few characteristics such as delay spread, maximum Doppler shift frequency, and noise power.

3.3. Define Objective Function

In this section, the optimal position is determined for the whales to look for. Ideally, the best position is the actual channel and noise power. Unfortunately, at the receiver, the whales are completely blind to the channel, so they cannot tell if the current position is close to the real propagation. We try to define a cost function that moves the swarm toward the ultimate goal of having the smallest BER.

While moving, each position of the whales can correspond to a constellation (from channel equalization). A position is considered better if the constellation corresponding to that position has a smaller dispersion (Figure 2). To determine the variance of the symbols after equalization, we cluster the signal into clusters ( depends on the modulation type, e.g., 4 for QPSK) and use the distance of that signal to the reference signal to represent dispersion as follows:

where and are the estimated signal and the reference signal, respectively.

The optimization problem now can be written as

The pseudocode of WOA-based channel estimation algorithm is presented in Algorithm 1. The parameters of the WOA-CE model are listed in Table 1.

Initialize the feasible propagation characteristics
for each iteration
for each search agent
  Update and
  if
   if
    Update the position of the current search agent by (6)
   else if
    Select a random search agent
    Update the position of the current search agent by (13)
   end
  else if
   Update the position of the current search by (11)
  end
end
 Check if any search agent goes beyond the search space and amend it
 Estimate channel using (14) and propagation characteristics
 Equalize channel
 Clustering based on modulation type
 Calculate the fitness of each search agent as cluster variance
 Update if there is a smaller variance
end
return
3.4. Complexity and Convergence Analysis

The complexity of the WOA-based estimator is greater than that of the ideal LS and MMSE estimators. As previously stated, the LS estimator is very simple to perform, leading to relatively poor results. The ideal MMSE, although very efficient, cannot be performed due to the assumption of channel correlation and known noise variance. In practical applications, these quantities are fixed or estimated by some method, possibly in an adaptive response manner. This has increased the complexity of the MMSE estimator and reduced the performance compared to the ideal MMSE many times over. Opposed to the ideal MMSE estimator, our proposed method ensures feasibility when applied because it does not need a preestimation step of noise power and a large correlation matrix. Specifically, the complexity of the algorithm is , proportional to the number of swarms and the number of iterations . A useful point of this algorithm is that the migration of swarm members is independent so it can be done in parallel when implemented on hardware, so the algorithm can greatly reduce the complexity to if parallel computation is applied.

We conduct an experiment to investigate the convergence of the whale optimization algorithm, as well as compare it with other optimization algorithms. When compared with general nonlinear programming algorithms, the method we choose can give faster convergence results, in other words, faster processing. This is suitable for the problem in wireless communications, although it does come with a slight trade-off in accuracy. When compared within the same group of smart search algorithms, it is challenging to claim that one search is better than another due to the randomness and convexity factors of a particular function. There are two important features of convergence: the number of iterations to converge and the value of the objective function upon convergence. However, due to the computational time limitation when estimating the channel, we investigated the optimal value after a specific number of iterations. The convergence graph below is the average of the experiments that we have statistics. From Figure 3, we see that the algorithms start to converge from about the 6th iteration. This is a relatively small number that can meet the speed requirements. Also at the time of convergence it is easy to see that WOA gives the most optimal value even though the previous iterations, the algorithms are still competing for the best method. We want to emphasize that the comparison of swarm optimization algorithms is being placed on a specific situation, not on a general function.

4. Results and Discussion

4.1. Simulation Environment

To demonstrate our proposal, we consider a physical layer simulation model by MATLAB with one transmitting and one receiving antenna (SISO) configuration. The simulation data and pilots are randomly generated following a uniform distribution. Modulation types including QPSK, 16QAM, and 256QAM were used for both the data and the pilot signal in this experiment. The pilots are arranged in comb fashion with (Figure 2) of interest being 3, 6, and 12. In other words, the pilots are mapped equally spaced on the resource grid with a pilot density of 1/3, 1/6, and 1/12.

Regarding the OFDM transmission system, the orthogonal signals are implemented by 4096-point discrete Fourier transform (DFT) and the guard interval is inserted by 1024 sample length cyclic prefix (based on the 5G NR standard). Subcarrier spacing is 30 kHz.

For channel modeling, multipath fading propagation conditions are implemented by a tapped delay line model. We use the TDL-C300 channel model, suggested for signal reception at the 5G NR gNodeB base station, with 12 channel paths and a delay spread of 300 ns. The tapped delay line model of 5G NR TDL-C300 channel is described in Table 2.

To provide the system performance in various deployment scenarios, we consider a wide range of SNR values of 0 dB to 30 dB with a 1 dB step. Simulation parameters of the OFDM system are listed in Table 3.

To demonstrate the efficiency of our proposed approach WOA-CE, we have compared its performance with existing competitors (LS and MMSE estimators) in terms of BER and MSE which are defined as follows: where BER indicates the overall system performance and MSE is a measurement for channel estimation accuracy.

4.2. Results and Analysis

Figure 4 presents the BER performance of considered channel estimators including LS, MMSE, and WOA-CE in a configuration of 16QAM and 1/6 pilot density. In addition, the BER performance for ideal channel estimation (i.e., full channel state information is obtained) is also presented and serves as a benchmark of the best but impractical case. The results demonstrate that, at a typical value of , the proposed WOA-CE has comparable performance with the MMSE and significantly outperforms the LS algorithm. Moreover, compared to the ideal channel estimation case, the WOA-CE performance is worse but the difference is negligible.

In Figure 5, the MSE for the LS, MMSE, and WOA-CE estimators is depicted. In general, the MMSE and WOA-CE curves continue to be close to one another and apart from the LS curve. More interestingly, we can observe that the performance of MMSE and WOA-CE estimators exhibits a slight difference depending on the SNR values. Specifically, in the high SNR area (above 20 dB), the WOA-CE performs better than the MMSE, whereas in the low SNR area, the situation is the opposite. The reason is that the objective function is not defined to reduce MSE but tries to lower BER by making the received signal after channel equalization “most clearly” clustered. At low SNR conditions, the signal is heavily affected by noise, so it is almost impossible to cluster. This means that the objective function does not work effectively when the noise is too large. On the contrary, when the SNR increases, the signal is relatively distributed in clusters, so that the whales can find a good enough position for the signal after equalization to be closest to the ideal constellation. This is demonstrated by the results in Figure 5.

To examine the influence of different modulation constellations on the BER performance of the proposed WOA-CE algorithm, we conducted more experiments on the QPSK and 256QAM constellations (in addition to the 16QAM case in Figure 4). The results are presented in Figure 6. It is demonstrated that our proposed WOA-CE algorithm continues to provide competitive performance over the MMSE algorithm within a wide range of modulation orders.

Another aspect that needs to be considered is the trade-off between the pilot density and the BER performance. The BER performance for various pilot densities is presented in Figure 7. It is shown that as the pilot density decreases (from 1/3 to 1/12), the BER performance of WOA-CE and MMSE algorithms degrades as well, but not dramatically (about 3 dB at ). It offers a possibility of reducing pilot overhead for better bandwidth efficiency without noticeably worsening BER performance. In contrast, the LS algorithm is heavily affected by the pilot density value. It is interesting that the LS estimator with 1/3 pilot density works effectively, and it might be a rival with MMSE and WOA-CE algorithms. However, with 1/12 pilot density, the LS estimator is seriously inaccurate since its curve is unable to reach the level of for any SNR values.

5. Conclusion

This paper presented a novel approach, based on the whale optimization algorithm, for channel estimation in wireless communication systems (WOA-CE). For pilot-based OFDM systems, the performance of the proposed WOA-CE algorithm has been evaluated and compared with the most popular channel estimation algorithms (LS and MMSE) in the 5G NR TDLC-300 frequency-selective channel. The BER and MSE have been used as performance metrics. Moreover, we conducted experiments in various system configurations including three constellations (from low to high modulation orders: QPSK, 16QAM, and 256QAM) and three pilot density values (1/3, 1/6, and 1/12).

The simulation results show that the WOA-CE algorithm provides highly competitive performance over the MMSE algorithm and significantly outperforms the LS algorithm. Since the requirement for prior channel statistics information makes the MMSE algorithm impractical or extremely complex, the proposed WOA-CE algorithm should be a suitable candidate for dealing with channel estimation problems in OFDM-based wireless systems. For future directions, the performance of the proposed WOA-CE can be improved and evaluated in the presence of channel coding and MIMO configurations.

Data Availability

The simulation framework is implemented in MATLAB and available upon request from https://link.uit.edu.vn/WOA-CE.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Acknowledgments

This research is funded by the University of Information Technology - Vietnam National University Ho Chi Minh City under grant number D1-2023-05.