Abstract
Accurate and high-resolution weather radar data reflecting detailed structure information of radar echo plays an important role in analysis and forecast of extreme weather. Typically, this is done using interpolation schemes, which only use several neighboring data values for computational approximation to get the estimated, resulting the loss of intense echo information. Focus on this limitation, a superresolution reconstruction algorithm of weather radar data based on adaptive sparse domain selection (ASDS) is proposed in this article. First, the ASDS algorithm gets a compact dictionary by learning the precollected data of model weather radar echo patches. Second, the most relevant subdictionaries are adaptively select for each low-resolution echo patches during the spare coding. Third, two adaptive regularization terms are introduced to further improve the reconstruction effect of the edge and intense echo information of the radar echo. Experimental results show that the ASDS algorithm substantially outperforms interpolation methods for ×2 and ×4 reconstruction in terms of both visual quality and quantitative evaluation metrics.
1. Introduction
China Next Generation Weather Radar (CINRAD) have been widely applied in operational research and forecast on medium-scale and short-duration strong weather phenomena. Many of the advantages of the CINRAD are associated with its high temporal (approx. 6 minutes) and spatial (approx. 1° × 1 km) resolution. However, the problem related to the weak detection capability at long distances also deserve attention [1, 2]. As shown in Figure 1, the radar beam width increases with the increase of detection distance, resulting in the accuracy of radar products susceptible to beam broadening and ground clutter.
Focusing on IQ data, more high-resolution information can be obtained by oversampling the IQ signal in range and azimuth; next, combine whitening filtering and distance oversampling techniques to obtain better parameter estimates [3, 4]. In article [5], authors proposed an operative adaptive pseudo whitening technique for single polarization radar, which sustains the performance of pure whitening under a high signal-to-noise ratio and equals or exceeds the performance of digital matched filter under a low signal-to-noise ratio. Experiments on the WSR-88D (KOUN) radar show that under the same condition, applying adaptive pseudo whitening to the range oversampled signal can reduce the standard deviation of the differential reflectivity estimate to 0.25 dB. Multiscale statistical characteristics of radar reflectivity data in small-scale intense precipitation condition are determined.
Due to the light computation, interpolation methods are widely applied in increasing the resolution of radar data to obtain detailed echo structure information. Conventional interpolation methods based on low-pass filtering such as bilinear, bicubic, and Cressman interpolation [6, 7] are featured by easy implementation and high speed of operation. However, one of the major limitations is that these methods mainly use several neighboring data values for computational approximation to get the estimation and fail to exploit the large-scale context information, which leads to the deficiency in recovering edge and high-frequency information of weather radar echo.
Accurate and fine-grained estimation of high-resolution information benefits from regularization constraints based on the characteristic of weather radar data. In this article, we propose an adaptive sparse domain selection (ASDS) algorithm for superresolution reconstruction of weather radar data, which learn a compact dictionary from a precollected dataset of example weather radar echo patches by using the principal component analysis (PCA) algorithm, then the most pertinent subdictionaries are adaptively selected for each low-resolution echo patches during the sparse coding. Besides the sparse regularization, autoregressive (AR) and nonlocal (NL) adaptive regularization are also incorporated based on the local and nonlocal spatial characteristic of weather radar data.
Key contributions of the proposed research are as follows:(1)An efficient superresolution reconstruction algorithm of weather radar data based on adaptive sparse domain selection (ASDS) is proposed in this article(2)Two adaptive regularization terms are introduced to improve the reconstruction effect of the edge and intense echo information of the radar echo(3)The proposed ASDS algorithm substantially outperforms interpolation methods for ×2 and ×4 reconstruction in terms of both visual quality and quantitative evaluation metrics
The remaining part of the article proceeds as follows. Section 2 describes the existing work in the subject area, and Section 3 discusses the methodology, starting from the brief introduction of the preliminaries of the article. The proposed ASDS algorithm and its details are provided in Section 4. The implementation details of ASDS are described in Section 5. Several experimental results and discussions are presented to validate the effectiveness of ASDS in Section 6. Section 7 concludes the article.
2. Literature Review
This section describes the research work done so far in the subject area.
Torres and Curtis [5] proposed an iterated functions systems (IFS) interpolation method on the basis of bilinear interpolation, which effectively utilize the fractal law of precipitation to downscale satellite remote sensing precipitation products and improved the resolution of input variables of hydrological model. Fast Fourier transform is also applied to temporal interpolation of weather radar data, which achieve accurate and efficient performance by windowing the input data [8]. The research article [9] proposed a spline interpolation method to generate satellite precipitation estimates with more refined resolution, which slightly outperformed the methods based on linear regression and artificial neural network. The research article [10] proposed an interpolation method to improve the resolution of radar reflectivity data, which effectively uses the hidden Markov tree (HMT) model as a priori information to well capture the multiscale statistical characteristics of radar reflectivity data in small-scale strong precipitation condition.
The regularization-based downscaling methods transform the downscaling problem into a constrained optimization problem by setting up a priori model of the radar echo data. Based on the non-Gaussian distribution of precipitation reflectivity data [11, 12], recast downscaling problem into ill-posed inverse problem to generate high-resolution information by adding regularization constraint based on Gaussian-scale mixtures model. The research article [13] is also selected to solve the inverse problem by exploiting the statistical properties of spatial precipitation data [8]. Based on the sparsity of precipitation fields in wavelet domain, sparse-regularization method [14] is applied to obtain high-resolution estimation in an optimal sense. Inspired by the sparsity and rich amount of nonlocal redundancies in weather radar data, [15] further incorporate the nonlocal regularization on the basis of sparse representation, which achieve promising performance in recovering detail echo information. The research article [16] proposed a dictionary based for downscaling short-duration precipitation events. Similarly, in the research article [17], adaptive regularized sparse representation for weather radar echo superresolution reconstruction is discussed.
3. Methodology
This section describes the research methodology proposed for the subject title.
3.1. Weather Radar Data: Sparse Representation
In this section, we present the characteristic of weather radar data and give a brief introduction to the sparse representation related to this article.
3.2. Characteristic of Weather Radar Data
The level-II data products used in this article come from S-band China Next Generation Weather Radar (CINRAD-SA). For reflectivity data, each elevation cut has 360 radials and each radial has 460 range bins. For radial velocity data, each elevation cut has 360 radials and each radial has 920 range bins. The range resolution of reflectivity and radial velocity data are 1 km and 0.25 km, respectively. The weather radar data include a large number of missing-data values that exceed the coding threshold or the corresponding range bin is in a region of the atmosphere where weather phenomena do not exist [15, 18, 19]. The missing-data statistics of the first elevation cut under large- and small-scale weather system and cloudless condition is shown in Figure 2, from which we can see that missing-data has a high percentage of weather radar data, especially in cloudless condition. Due to the presence of large amounts of invalid data, weather radar data are typically featured by high sparsity. Inspired by the sparsity of weather radar data, it is feasible to apply sparse representation for resolution enhancement.
3.3. Sparse Representation
The basic idea of sparse representation is that a signal (N is the size of weather radar echo) can be expressed as a linear combination of basis vectors (dictionary elements); and are over complete dictionaries, and the coefficient satisfies a certain sparse constraint, which can be expressed as and denotes the norm. Using sparse prior, to reconstruct from to the sparse coefficient of in dictionary can be obtained by where parameter is the trade-off between sparse representation and reconstruction error, and norm is used to calculate the number of nonzero elements in coefficient vector . However, norm is nonconvex, and its optimization is a NP hard problem, which is usually replaced by norm or weighted norm. The minimization of the aforementioned equation can be transformed into a convex optimization problem [20, 21]. Equation (1) can be expressed as
In equation (2), the norm is the local sparsity constraint to ensure that the sparse coding vector is sufficiently sparse. A number of algorithms, such as iterative threshold algorithm [22] and Bregman splitting algorithm [23, 24], have been proposed to solve equation (2); the reconstructed high-resolution echo .
In addition, it is assumed that is a radar echo patch with size of extracted from high-resolution echo , i.e., , where represents the window function extracted from the upper left to the lower right patch with overlap for . Using over complete dictionary the sparse coding vector is obtained by minimizing the following equation:
Each weather radar echo patch can be sparsely expressed as follows: so a series of sparse coding vectors and reconstructed high-resolution radar echo patches can be obtained. By averaging all the reconstructed radar echo patches, the whole reconstructed high-resolution radar echo can be obtained by where N is the number of radar echo patches and equation (2) can be formulated as follows:
Rest of the details of the methodology are explained in a separate section 3, discussing the Adaptive Sparse Domain Selection Model in its subsequent sections.
4. Adaptive Sparse Domain Selection Model
On the basis of sparse representation, the adaptive sparse domain selection (ASDS) model proposed in this includes the following: subdictionaries learning and adaptive selection of the subdictionary and adaptive regularization. An adaptive sparse domain selection (ASDS) scheme is a model which learns a series of compact subdictionaries and allocates adaptively each local patch a subdictionary as the sparse domain.
4.1. Subdictionaries Learning
Dictionary learning is a key problem in the sparse representation model. is used to represent the set of radar echo patches of the dictionary used for training. is obtained through a high-pass filter. is divided into K clusters by K-Means algorithm, which belongs to hard clustering. The Euclidean distance and the sum of squares of errors are taken as the similarity and minimum optimization objective, respectively. The remaining problem is to learn a dictionary for each cluster , and the objective function of the design dictionary can be formulated by where denotes the coefficient representation matrix of relative to dictionary . Equation (6) is a joint optimization problem of and , which can be solved by the K-means singular value decomposition (K-SVD) algorithm with alternating optimization [24]. However, the norm joint minimization problem in equation (6) consumes a lot of computing resources. Since the number of elements in subdataset obtained through K-means is limited and the elements often support similar modes, it is unnecessary to learn over-complete dictionary from subdataset. In addition, a compact dictionary will greatly reduce the computational cost of sparse coding for a given radar echo patch. Based on the aforementioned considerations, we use principal component analysis (PCA) algorithm to learn a compact dictionary.
By calculating the covariance matrix of . The orthogonal transformation matrix is obtained by the PCA method. According to the theory of PCA, the following equation can be deduced:where denotes the representation coefficient. If the first r main eigenvectors in are selected to construct the dictionary , the representation coefficient is . To get a better balance between the reconstruction error and the sparsity of the coefficients, the optimal solution of can be obtained by
Finally, the subdictionary of subdictionaries for sparse coding. While obtaining the dictionary, calculate each the centroid of the cluster , so pairs can be obtained.
4.2. Adaptive Selection of Subdictionary
Taking wavelet base as the dictionary, the initial estimation of can be calculated by solving equation (5) using the iterative shrinkage algorithm [25]. Denote by the estimation of and denote by a local patch of . For each , a subdictionary will be selected from dictionary set for reconstruction, which can be completed in the subspace of . Apply SVD to the covariance matrix of including all the centroids to get the PCA transformation matrix of . Let be the projection matrix composed by the first several most significant eigenvectors, the distance between and can be calculated in the subspace of .
By solving equation (9), subdictionary is adaptively selected and assigned to echo patch , so we can update the estimation () of by minimizing equation (5).
4.3. Adaptive Autoregressive Regularization
To ensure the local stability of radar echo during the reconstruction process, we use spatial autoregressive (AR) regularization to constrain the optimal solution of sparse coefficients and realizes the parameter adjustment of superresolution local reconstruction. For each cluster , an AR model can be obtained by training the sample echo patches in the cluster. In the construction of the training model, too high an order may lead to data over fitting. Therefore, this article uses a window of 3 3, and the training order is 8. By solving the least square problem in the following formula, the AR model parameter vector of the cluster can be calculated bywhere is the center value of the weather radar echo patch and is the column vector composed of eight surrounding data values. A series of AR model parameter vectors can be obtained by equation (10).
For each radar echo patch, an AR model is adaptively selected from for reconstruction. The specific selection scheme is the same as that of the subdictionary; let , then the AR model will be assigned to patch . The autoregressive regularization constraint is to solve the minimum error between the actual value and the estimated value of the center of the radar echo patch , which is equivalent to minimizing and denotes the column vector composed of 8 data values around . By adding the autoregressive regularization term as a local constraint to the solution process of the sparsity coefficient, the sparse vector is obtained by solving the following minimization problem:where is a constant that balances the constraint of AR regularization term. For convenience of expression, we rewrite the third term as , where is an identity matrix and
Therefore, equation (11) can be rewritten as follows:
4.4. Adaptive Nonlocal Regularization
Statistics show that the weather radar echo contains many repetitive structures and shapes. As shown in Figure 3, the red box in the left plane position indicator (PPI) and the black box in the right PPI indicate the given radar echo patch and the radar echo patches, that is, nonlocally similar to it, respectively; many similar and repetitive structures can be observed between two echo patches (for example, the reflectivity data of the first layer elevation angle of CINRAD-SA radar (10 : 36 (BJT) on June 23, 2016). This has 360 radials, and each radial has 460 range bins; this nonlocal redundant information has the effect of enhancing the sparse decomposition stability of weather radar echoes, improving the quality of weather radar echo reconstruction. Therefore, this paper introduces the regularization constraint based on nonlocal (NL) similarity to further improve the reconstruction effect.
For each weather radar echo patch , search for similar patches in the whole echo . The criteria for screening are as follows:where is the preset threshold and the linear representation coefficient is obtained by
In equation (15), is the normalization factor, is the control term of weight, and the expected error term is calculated by where denotes the central value of , denotes the central value of , and is the identity matrix, and
Therefore, by incorporating the expected error term, equation (5) can be improved as follows:where is a constant that balances the constraint of the NL regularization term.
4.5. Algorithm of ASDS
By combining local autoregressive (AR) regularization constraint and nonlocal (NL) self-similar regularization constraint into equation (5), we obtain a final sparse representation algorithm based on ASDS to reconstruct weather radar echo, which can be described as where the second norm is an adaptive regularization term based on the local AR model, which ensures that the estimated weather radar echo is locally stationary, and the third norm is a nonlocal similar regularization term, which uses nonlocal redundancy to enhance each local echo patch. The algorithm flow of ASDS is shown in Algorithm 1.
|
5. Implementation Details
(1)Data: the dictionary learning and performance evaluation of the proposed ASDS algorithm is carried out using the reflectivity data from S-band China's new generation weather radar (CINRAD-SA), which include the precipitation experimental data of South China in Guangdong on May to June 2016, the tornado and hail data of Yancheng in Jiangsu on May 27, 2008, the tornado data of Yancheng in Jiangsu on June 23, 2016, Nantong in Jiangsu on July 06, 2016, and the typhoon data of Xuzhou in Jiangsu in August 2018.(2)Degradation method: the weather radar echo degradation process includes three processes: blurring, down-sampling, and system noise. Degradation process can be formulated by where represents the degradation operation (e.g., blurring kernel, down-sampling operation) and represents the weather radar receiver noise, which obeys the zero-mean Gaussian distribution.(3)Parameter Settings: in the experiment, the original echo data are used as the high-resolution (HR) radar echo; the low-resolution radar echo can be obtained through blurring, subsampling, and noise processing, where fuzzy kernel is a Gaussian kernel with size of and standard deviation of 1.5, the subsampling factors are 2 and 4, and the additive noise is white noise. In addition, the initialization of the parameters in the ASDS algorithm is as follows: the number of subdictionaries obtained through K-PCA is K = 200, and the regularization parameters of the sparse coefficient term, AR term, and NL term are , respectively. In dictionary learning processes, the total amount of echo patches with a size of is 16000. To find the optimal number of iterations, we calculated the PSNR values of the reflectivity data for different weather conditions and down-sampling scales. As shown in Figure 4, the optimal number of iterations is , and the iteration error is and P = 150.(4)Evaluation metrics: to evaluate the effectiveness of the proposed ASDS algorithm, we compare ASDS with bicubic and IFS interpolation in terms of visual quality and quantitative results. The peak-signal-to-noise ratio (PSNR) (dB) and structural similarity (SSIM) [26] are used to quantitatively evaluate the reconstruction effect.6. Experimental Results and Analysis
In this section, we compare ASDS with bicubic and IFS interpolation in terms of visual quality and quantitative results.
6.1. Visual Quality Comparison
The reflectivity data of the first elevation cut under precipitation (Heyuan, Guangdong, China, 12 : 06, June 15, 2016) and typhoon (Xuzhou, Jiangsu, China, 11 : 00, August 18, 2018) condition are selected to test the performance of ASDS under large-scale weather system and to compare it with bicubic and IFS interpolation.
Intense precipitation convective cells are often embedded in a lower intensity region, which shows high aggregation and sparse correlation. Therefore, capturing as much detailed information as possible about the intense echo can help in operational research and forecast on intense precipitation. As shown in the black box of Figure 5, it can be seen that under ×2 reconstruction, the bicubic reconstructed radar echo lose most of the intense echo information, resulting in overly smooth radar echo, which indicate that the simple interpolation method using several neighboring data values for computational approximation to get the estimated value is not capable of being applied in intense precipitation condition. Although IFS interpolation performs better than bicubic in recovering intense echo information, it has lack in rebuilding echo with more fine-grained structure. Compared with bicubic and IFS interpolation, ASDS algorithm has better ability in recovering more of the fine-grained structure and is notably better at preserving sharp edges associated with the large-scale features. Under ×4 reconstruction, compared with the terrible performance of bicubic and IFS interpolation in recovering the detail information of radar echo, ASDS algorithm is capable of recovering the edge and highlighting the location of intense echo. As shown in the black box of Figure 6, the ASDS reconstructed echo is most similar to the original echo and achieve the best subjective quality under typhoon condition.
Tornado is an intense unstable weather condition of small-scale convective vorticity, with a central wind speed of up to 100–200 m/s. A tornado cycle is short, usually only for a few minutes, with the longest not being more than a few hours, so tornado detection and early warning forecast are very difficult. Most of the tornadoes, especially, those above EF-2 (Enhanced fujita) mainly occur in supercell storms [27, 28]. Hook echo is the area where tornadoes may occur in supercell thunderstorms. Therefore, recovering as much detailed information as possible about the hook echo from degraded weather radar echo can help in tornado detection and forecast. As shown in the black box of Figure 7, although both the IFS interpolation and the ASDS algorithm can recover most of the hook echo detail information under ×2 reconstruction, the hook echo reconstructed by the ASDS algorithm is closer to the ground truth. Under ×4 reconstruction, although all methods fail to recover the fine-grained structure of hook echo, the ASDS algorithm has the better performance in highlighting the location of intense echo.
6.2. Quantitative Results Comparison
Figure 8 shows the direct comparison between the estimated values and the ground truth data in the cases corresponding to the ones showing in Figures 5–7. In Figure 8, the bicubic and IFS scatters are heavily distributed above the 1 : 1 diagonal in the weak echo region (10–30/dBZ), that is, underestimation, and heavily distributed above the 1 : 1 diagonal in the intense echo (40–60/dBZ) region, that is, overestimation. The scatter of ASDS are mostly distributed near the 1 : 1 diagonal, which indicates that the estimation of ASDS is better consistent with the ground truth, especially in the intense echo region. Statistically speaking, ASDS has the better performance for coefficient of determination (R2) than bicubic and IFS. To further validate the effectiveness of ASDS under different weather condition (e.g., large- and small-scale weather system, cloudless), the statistical comparison is made in terms of quantitative evaluation metrics (PSNR, SSIM). All the results on ×2 and ×4 reconstruction are shown in Table 1, from which we can see that ASDS shows the best PSNR and SSIM in all weather conditions, with PSNR is even more than 2 dB higher than bicubic interpolation. We can also see that ASDS with NL regularization produces subjectively superior results than with AR regularization, indicating promising performance of exploiting nonlocal correlation information of weather radar echo.
(a)
(b)
(c)
7. Conclusion
Different from the interpolation methods, this method only utilizes the neighboring information. In this article, we propose the ASDS algorithm for superresolution reconstruction of weather radar data, which effectively exploit the sparsity and local and nonlocal characteristics of weather radar data by introducing sparse, local, and nonlocal regularization. Experimental results show that the ASDS algorithm substantially outperforms interpolation methods for and rebuilding in terms of both visual quality and quantitative evaluation metrics. Although we use the K-PCA algorithm to obtain the compact dictionary from the training set in advance instead of updating the dictionary during each iteration, which can greatly reduce the time required for superresolution reconstruction, the ASDS algorithm still has a large amount of computation. With the increasing demand for extreme weather warning and forecast, the ASDS algorithm still has a lot of room for optimization.
Data Availability
The datasets used during the current study are available from the corresponding author on reasonable request.
Conflicts of Interest
The authors declare that there are no conflicts of interest.
Acknowledgments
This work was supported by the National Key R&D Program of China (2018YFC01506100), the http://dxNational Natural Science Foundation of China (U20B2061), and the http://dxDepartment of Science and Technology of Sichuan Province (2020ZYD051).