Abstract
This study adopted an improved fuzzy clustering and optical flow method for the multiscale identification and forecasting of a cloud system based on the cloud images from a 10.8-micron infrared channel of the Advanced Geosynchronous Radiation Imager. First, we used the locally constrained fuzzy c-means (FCM) clustering method to identify typhoon-dominant cloud systems. Second, we coupled the background field-constrained optical flow method with the semi-Lagrangian scheme to forecast typhoon-dominant cloud systems. The experimental results for Typhoon Maria showed that the improved FCM method was able to effectively identify changes in the cloud system while retaining its edge information through the effective removal of the offset field. The identified dominant cloud system was consistent with the precipitation field of the Global Precipitation Measurement mission. We optimized the semi-Lagrangian nonlinear extrapolation of the optical flow field by introducing background field information, thus improving the forecast accuracy of the optical flow field. Based on the assessment indicators of structural similarity, normalized mutual information, peak signal-to-noise ratio, relative standard deviation, and root mean square error, the forecast results demonstrated that the forecast effect of the background field-constrained optical flow method was better than that of the standard optical flow method.
1. Introduction
Meteorological satellite cloud images not only play an important role in the field of real-time objective analysis, monitoring, and identification, but also provide a key source of information to assist forecasters. These images can also be used to monitor severe weather, such as rainstorms and typhoons, which occur, develop, and move rapidly, causing significant destruction. Geostationary weather satellites can provide a wide range of all-weather satellite cloud image information and can capture the changes and movement trends of typhoon cloud systems that can be used to realize real-time tracking and warning of dominant cloud systems [1]. Using image processing, pattern recognition, and other methods, satellite cloud image segmentation can detect a storm system, a process that can be utilized to avoid the uncertainty inherent in the determination of the location of a typhoon’s center using traditional visual interpretation. This technique can also provide the basis for decision making that can prevent, or at least mitigate, typhoon disasters.
Domestic and foreign researchers have conducted many fruitful studies on identification and tracking technology based on satellite cloud images. The identification methods for typhoon cloud systems primarily include the following: cloud image pixel, region, and context association; texture analysis; and the threshold method [2]. Among these approaches, the threshold method is widely used in image segmentation. For example, Wang et al. [3] synthesized the threshold method, mathematical morphology, and mathematical statistics method in order to examine the segmentation of a typhoon cloud system using infrared cloud images. Bai and Wei [4] concluded that this method could not effectively segment dense clouds in the central region of a typhoon. Zhang et al. [5] proposed a single-threshold algorithm that enhances contrast using discrete contour transformation. The threshold method also has limitations, however, mainly because the pixel-based approach cannot use the local information in the image.
Some researchers have proposed the use of machine learning to segment or classify cloud systems based on the pixel or texture features of satellite cloud images. For example, Collet et al. [6] carried out cloud system segmentation based on the Markov random field model. Xu et al. [7] applied support vector machines to the segmentation of typhoon spiral cloud bands, but due to irrelevant segmentation based on pixel classification, the probability of incorrect segmentation between classes increased [8]. Multispectral feature classification is another commonly used method based on pixel classification. For example, González et al. [9] proposed multispectral image cloud segmentation based on nonparametric mean shift clustering. The multispectral method, however, cannot identify objects with similar spectral characteristics [10]. Ameur et al. [10] proposed the texture structure method, which utilizes sum and difference histograms to obtain the texture parameters in all directions of the satellite image. Berendes et al. [11] proposed the use of the standard deviation finite adaptive clustering method to identify and classify convective clouds in daytime satellite images.
Some scientists have proposed a cloud system segmentation method based on satellite cloud image regions. Mukherjee and Acton [8] used the fuzzy c-means (FCM) method to construct scale images based on boundary-preserving morphological operators and segment cloud systems in scale image space. In light of the sensitivity of the standard FCM method to “outliers,” Ahmed et al. [12] proposed an FCM method that introduced neighborhood information and coupled the de-offset field in order to achieve magnetic resonance imaging image segmentation. Wang et al. [13] constructed the gradient vector flow (GVF) model based on regional information, integrated FCM clustering information, and image structure tensor information into the GVF model, reduced the influence of weak boundaries and noise, and obtained a better segmentation result. To extract the cloud structure of tropical cyclones (TCs), Wei et al. [14] promoted the Chan–Vese model of a polyphase state and segmented the multichannel cloud image, achieving good results. Wang et al. [15] realized the identification and dynamic tracking of convective clouds based on FCM, the cross-correlation method, and cubic spline interpolation. Johnson et al. [16] proposed the collaborative method of an objective measurement and numerical weather forecast model to solve problems such as satellite image noise, partial information loss, and low image resolution and realized the central target identification and tracking of TCs. Johnson et al. [16] filtered images and flow fields at different stages to remove the noise generated by seasonal changes and improper registration to solve the phenomenon of a “nonzero offset field in the satellite image histogram of a tropical cyclone.”
At present, the main nowcasting methods include the cross-correlation technique [17], centroid tracking [18], and the optical flow method and its variants [19]. Carvalho and Jones [20] studied the automatic identification and tracking of mesoscale convective systems using the maximum spatial correlation tracking method. Mueller et al. [18] established the automatic nowcasting system. Dawe and Austin [21] used complex microphysics and large-scale eddy simulation technology to forecast the evolution of cloud clusters, although complex cloud evolution calculations were not suitable for fast-tracking and forecasting with satellite cloud images [15]. The standard optical flow method is the globally constrained solution method proposed by Horn and Schunck (the HS method) [22]. Lucas and Kanade [23] proposed a locally constrained solution method (the LK method) based on the HS method. For fast-changing precipitation processes, the advantages of the optical flow method are obvious. In the extrapolation process of the standard optical flow method, the optical flow field generally is calculated using only the cloud system information at two adjacent moments. Once the optical flow field has been determined, this flow field remains unchanged during the entire forecast period, whereas the actual typhoon cloud system includes generation, evolution, and extinction.
Given the complexity and variability of the typhoon cloud system, this study improved the standard optical flow method based on multitime and multispace background information. First, we used local constraints to de-offset the field FCM method [12, 13] in order to identify typhoon-dominant cloud systems. Second, we forecasted the typhoon cloud system based on the background field-constrained optical flow method coupled with the semi-Lagrangian scheme. Finally, we applied this method to the cloud image of a 10.8-micron infrared channel of the Advanced Geosynchronous Radiation Imager (AGRI) from the Feng-Yun-4A (FY4A) satellite [24]. We used Typhoon Maria as an example to conduct identification and experimental forecasts.
2. Cloud System Identification Model: Local Constraints to De-Offset the FCM Method
The primary disadvantage of standard fuzzy c-means (FCM) clustering is that the objective function does not consider any spatial relationship, making it sensitive to noise and outliers, which leads to the incorrect classification of noise pixels due to their abnormal features [13]. In this study, we adopted the method of Ahmed et al. [12] to take the neighborhood information of each pixel into consideration when estimating the offset field, as follows:where and are the true and observed values of the jth pixel, respectively, and is the offset field of the jth pixel.
The improved local constraint-based de-offset field FCM is defined as follows:
The constraint conditions arewhere is the ith cluster center; is the set of neighbors that exist in the window around and the elements in this set; is the offset field of the corresponding element; is the cardinality of ; is the number of categories; is the number of feature vectors; is the fuzzy membership degree of the jth element belonging to the ith category; is the fuzzy index, with in this study; and parameter controls the contribution rate of neighborhood pixels to the center pixels.
According to the Lagrange extremum method, via derivation, the fuzzy membership degree , cluster center , and offset field are obtained as follows:
3. Forecast Models: Background-Constrained Optical Flow Method Coupled with Semi-Lagrangian Extrapolation
Based on the standard optical flow method, we introduced pyramid-layering technology in this study to reduce errors caused by fast-moving clouds. At the same time, we introduced background information to optimize the semi-Lagrangian nonlinear extrapolation of the optical flow field, and we considered the change in cloud morphology in order to improve the forecast accuracy of the optical flow method.
3.1. Optimization Model of the Standard Optical Flow Method: The Background Field-Constrained Optical Flow Method
The basic idea of the optical flow method is to differentiate the time and space of the pixel values (gray value or brightness temperature) of two continuous images in order to obtain the optical flow field [25]. The mathematical constraint equation of the optical flow method is defined as follows:where is the brightness temperature of the field of view (FOV) at time ; and are the west-east (x-axis) and north-south (y-axis) flow field components, respectively; , , and represent the partial derivatives of brightness temperature with respect to , , and , respectively; and is the variation amplitude of the observed brightness temperature of the FOV.
To solve the ill-posed problem of two unknowns ( and ) and one constraint in equation (5), Horn and Schunck defined the optical flow constraint as a global variational problem (i.e., the HS method) and solved the problem by minimizing the cost function in equation (6) [22].
The solution formula of the standard HS optical flow method is as follows:where is a tunable parameter and optimizes the weight of the smoothing terms .
Considering changes in a satellite infrared cloud, this study optimized the standard HS optical flow field based on cloud background information. First, we used the weighted sum of the first time optical flow fields in order to obtain the new optical flow field. Second, we used semi-Lagrangian extrapolation for the forecast based on the new optical flow field, where the weighted coefficient followed a Gaussian distribution.
3.2. Semi-Lagrangian Forecast
The semi-Lagrangian forecast for the same time step length is always located on the grid point to endpoint along the fluid particle track [26]. The forecast is based on the vector trajectory of a satellite cloud, which is applicable to rotating clouds in large-scale environmental fields. As the key to the computational fluid trajectory, the semi-Lagrangian trajectory equation is defined as follows [27]:where and are the observed and forecast brightness temperatures of the FOV, respectively; is the forecast lead time; is the initial time; is the forecast value at time and position ; and is the displacement vector of the cloud cluster for the entire forecast process.
During the process of the semi-Lagrangian forecast, the overall forecast is divided into time steps and the interval of each time step is ; thus, . We obtained each time step through the iterative solution of equation (8), where the final displacement vector is the sum of vectors of the time step, as follows:
Since this investigation only carried out the preliminary application of the algorithm, we set the time step to be the temporal resolution of the satellite observations.
4. Typhoon Cloud System Identification and Experimental Forecasts
4.1. Data Description
China’s new geostationary FY4A satellite was successfully launched on December 11, 2016. The FY4A/AGRI has 14 channels, including three visible, three near-infrared, and eight infrared. In this study, we only considered the brightness temperature of the channel in the atmospheric window for which the central wavelength of the AGRI was 10.8 microns. The time interval between two consecutive images of the AGRI brightness temperature was 15 minutes, and the spatial resolution of the channel used was 4 km. The observation data were in HDF format.
The domain of the Global Precipitation Measurement (GPM) IMERG precipitation data used in this investigation covered 89.95°S-89.95°N latitude and 179.95°W-179.95°E longitude, with a spatial resolution of 0.1° × 0.1° and a temporal resolution of 30 minutes. The unit of the GPM IMERG precipitation was mm/30 minutes [28].
4.2. Background of Typhoon Maria
Super Typhoon Maria (international code: 1808) was the eighth storm of the 2018 typhoon season in the Pacific Ocean. At 05:00, July 8, 2018, Coordinated Universal Time (UTC), Maria had developed into a super typhoon in the northwest Pacific Ocean and was located approximately 1930 km to the southeast of Yilan County, Taiwan. At 09:10 UTC, July 11, 2018, Maria made landfall on the Huangqi Peninsula of Lian Jiang, Fujian Province. The maximum wind force near the center was level 14 (42 m/s), and the lowest pressure was 960 hPa.
4.3. Analysis of Experimental Typhoon Cloud System Identification Based on a Locally Constrained De-Offset Field Using FCM
4.3.1. Preliminary Selection of a Cloud Image De-Offset Field and Brightness Temperature Threshold
The FY4A satellite is located in geostationary orbit and may cause interference in the observation or signal transmission process. Thus, the satellite cloud image contains noise. Prior to applying the standard FCM to typhoon cloud system segmentation, the Gaussian kernel filter was employed to denoise the cloud image, in which the variance was set as 0.5. Since the improved FCM features a strong robustness to cloud image noise outliers, however, cloud image denoising was not necessary.
Because the grayscale of a typhoon’s dense cloud area obeys a particular variation rule, this study preliminarily used the threshold method to screen the dense cloud area of the cloud image in order to eliminate the surface, sea surface, and other information unrelated to the cloud clusters. Based on prior knowledge and experimental optimization, we selected 285 K as the threshold of the brightness temperature and the dense cloud region in the FY4A/AGRI atmospheric window area of the 10.8-micron channel. That is to say, in order to select only cloud pixels, the brightness temperature threshold was set to 285 K, and the standard and improved FCM clustering methods were applied only to those pixels (or FOVs) with brightness temperatures lower than 285 K.
The initial clustering center of the FCM with local constraints to the de-offset field was provided by the k-means [13]. Given that clouds are generally classified as high clouds, mid-level clouds, low clouds, or other mixed clouds, this investigation referred to the study of Berendes et al. [11]. The cloud pixels (or FOVs) were divided into six categories, with the classification algorithm only a function of the brightness temperature. Each category corresponded to one particular cloud type. We took the first category to be the identification result of the dominant cloud system.
In this study, the dominant cloud system was defined as the group of pixels (or FOVs) with the smallest cluster center obtained by the standard and improved FCM methods. Taking Typhoon Maria at 00:00 UTC, July 10, 2018, as an example, the cluster centers obtained by the standard FCM were 212.28 K, 227.29 K, 240.50 K, 255.03 K, 268.65 K, and 281.31 K. The first category of the brightness temperature subset, which had a cluster center temperature of 212.28 K, was the “dominant cloud system.” The brightness temperature subset within which the minimum value of the cluster center was situated was the “edge” of the dominant cloud system. The “minimum value of the cluster center” was obtained using the improved FCM. Based on the characteristics of the standard and improved FCM algorithms, if the cloud images were different, the cluster center values of the dominant cloud system were different, although the dominant cloud system belonged to the minimum value of the cluster center class.
Figure 1 shows the original AGRI cloud images of Typhoon Maria on July 10, 2018, the identification result from the improved FCM, the dominant cloud system “edge” structure, and the offset field estimation.
(a)
(b)
(c)
(d)
Unlike Johnson et al. [16], who used filtering images at different stages to solve the problem of a “nonzero offset field in the satellite image histogram of a tropical cyclone,” we coupled the offset field information with the identification model in this study. We then estimated the offset field while identifying the dominant cloud system.
In this investigation, only the cloud pixels (or FOVs) were classified into six categories. Due to the characteristics of the algorithm, if the classification was divided into four or eight categories, it would have a certain impact on the classification results, but little impact on the dominant cloud system (figure omitted). This categorization scheme may require further examination in the future, but in this study, the “dominant cloud system” obtained by the standard and improved FCM methods was consistent with the precipitation field distribution of the GPM, as shown in Figure 2. The low brightness temperature area of the AGRI corresponds to the strong precipitation area of the GPM [28].
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
(j)
4.3.2. Experimental Identification of the Cloud System and Comparative Analysis with the Global Precipitation Measurement Field
Figure 2 shows the identification results based on the standard FCM (denoted as the FCM algorithm) and the local constraints de-offset field FCM (denoted as FCM + offset field algorithm) for different development stages of Typhoon Maria [12]. In addition, the distributions of the dominant cloud system derived from the different methods, as well as the Global Precipitation Measurement (GPM) field [28] at the corresponding times, are included.
Figure 2 reveals that the distribution of brightness temperature in the typhoon-dominant cloud systems identified by the standard and improved FCM methods is consistent with the precipitation fields of the GPM. In addition, the low brightness temperature area of the AGRI corresponds to the strong precipitation area of the GPM [28]. Moreover, it can be observed that, compared with the clustering results of the standard FCM, the dominant cloud system obtained by the improved FCM is closer to the shape of the GPM precipitation.
A great deal of information concerning a typhoon’s motion, wind field, and heavy rainfall is contained in its spiral cloud bands [4, 7]. Compared with the standard FCM, the improved FCM can identify parts of the spiral cloud band structure from a typhoon in the AGRI brightness temperature image. The spiral rainband of a typhoon is also reflected in the GPM precipitation field.
The experimental results depend on the characteristics of the algorithm, with the dominant cloud coverage areas varying between the different FCM methods.
4.4. Analysis of the Experimental Forecast of Typhoon-Dominant Cloud Systems
4.4.1. Experimental Forecasts Based on the Background Field-Constrained Optical Flow Method Coupled with the Semi-Lagrangian Scheme
We carried out the dominant cloud system forecast, which was based on the brightness temperature of the dominant cloud system result identified by the standard and improved FCM methods. Rather than forecasting the entire cloud image, we obtained a background-constrained optical flow method (denoted as background constraints) according to the weighted summation results of the first four temporal background fields based on the standard optical flow method (denoted as standard optical flow).
Figure 3 shows the distribution of the typhoon eye (denoted as typhoon eye) in the optical flow field of the cloud system obtained at 02:45 and 03:00 UTC on July 10, 2018, based on the standard and improved optical flow methods, respectively.
(a)
(b)
(c)
(d)
(e)
(f)
Different from the standard optical flow method, which only considers the information between two continuous images, the background field-constrained optical flow method introduces the information among the previous four (or more) images. It can be seen from Figure 3 that the flow field information of the background field constraint is richer than that of the standard method.
Due to the limited observation data of typhoons over the ocean (with the exception of satellite data), verification can only be conducted by comparing forecast results with actual results. The “actual” results in this study, however, rely on the classification results of the standard and improved FCM methods. Therefore, in future research, we plan to determine whether the “track” of the flow field derived from the thermodynamic and dynamic structures of typhoons is reasonable, in order to verify the feasibility of our method.
Figure 4 shows the actual characteristics of the dominant cloud system identified by the standard and improved FCM methods, and the forecast results of the standard optical flow method as well as the background field-constrained optical flow method. Among them, the standard optical flow method used twice: 02:45 on July 10, 2018 (denoted as 20180710024500), and 03:00 on July 10, 2018 (denoted as 20180710030000). The background field constraint was based on the data of the first two hours, in addition to the data of the first four hours before 02:45 on July 10, 2018, for a total of six hours of data. We forecast the distribution of the dominant cloud system (denoted as FCM + standard optical flow and FCM + background constraints) 15 min later.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
Note that all of times in this manuscript are expressed as Coordinated Universal Time (UTC), and all of the selected data include the first four background field times.
Figure 4 reveals that the forecast results of the standard and background field-constrained optical flow methods were consistent with the actual conditions. Further, it can be observed that, compared with the clustering results of the standard FCM, the dominant cloud system obtained by the improved FCM was closer to the GPM precipitation pattern. The forecasted brightness temperature of the spiral cloud band corresponded well with the spiral rainband of the GPM [4, 7]. The dominant cloud system forecast by the improved FCM coupled with the background-constrained optical flow method was also closer to the form of the GPM precipitation. These findings indicate that the identification results of the dominant cloud system in this study exerted an influence on the later forecast.
4.4.2. Verification of the Forecast Effect of the Dominant Cloud System
For the experiment presented in Figure 4, we used quantitative evaluation indices to measure and compare the forecast accuracy of the different methods. We adopted the following quantitative methods: structural similarity (SSIM), normalized mutual information (NMI), peak signal-to-noise ratio (PSNR), relative standard deviation (RSD), and root mean square error (RMSE) [29]. For comparison with the results of the standard and background field-constrained optical flow methods, the forecast time was 15 min.
As shown in Tables 1 and 2, the dominant cloud system of Typhoon Maria forecast by the standard and background-constrained optical flow methods exhibited a high similarity with the cloud structure identified by the FCM, and the error was small. Since the background-constrained optical flow method introduced optical flow information for the first four background field times, its forecast effect was better than that of the standard method. In addition to the NMI, the overall scoring result decreased after adding the de-offset field, which was related to the previous identification result of the “dominant cloud system.”
It should be noted that (1) compared with the standard FCM, the AGRI dominant cloud system obtained by the improved FCM is closer to the GPM precipitation field. In the strong precipitation area of the GPM, the AGRI brightness temperature is low, which indicates spiral cloud and is consistent with the spiral rainband of the GPM (Figure 2). As demonstrated in our concurrent research, the spiral rainband of a typhoon can also be retrieved based on the brightness temperature of different AGRI channels (manuscript to be submitted). (2) Compared with the standard optical flow method, the improved optical flow method has a better effect on forecasting the dominant cloud system. Finally, (3) the forecast quantitative evaluation indices of the combination of the improved FCM and the optical flow methods are worse than those of the standard FCM. This is due to the fact that the “actual” (or comparison object, i.e., “truth value”) in this study relied on the identification result of the standard and improved FCM methods, i.e., the comparison object is different. The combination of the improved FCM and the improved optical flow method has a poor forecast effect, which may be due to the fact that the offset field information is not used in the calculation of the optical flow field.
5. Summary and Future Work
This study examined the identification and forecast algorithm of a typhoon-dominant cloud system. During the specific implementation process, based on the cloud image from a 10.8-micron infrared channel of the FY4A/AGRI satellite, we adopted local constraints and the de-offset field of an FCM method as well as the background field-constrained optical flow method. We used Typhoon Maria as an example to conduct identification and forecast experiments.
Our main conclusions are as follows:(1)Based on the standard FCM, this study introduced the offset field estimation, which solved the problem of the “nonzero offset field in the satellite image histogram of a tropical cyclone” found by Johnson et al. [16]. This estimation exhibited the ability to forecast the cloud system with specific time and space scales when identifying the movement of storm clouds with different scales. Taking Typhoon Maria as an example, we identified the dominant cloud system. The result of the identification of the typhoon-dominant cloud system corresponded well with the GPM distribution. The low brightness temperature area of the AGRI corresponded to the strong precipitation area of the GPM. The spiral cloud band of the AGRI brightness temperature corresponded well with the spiral rainband of the GPM.(2)Based on the background-constrained optical flow coupled with the semi-Lagrangian method, we effectively forecast changes in typhoon-dominant cloud systems. After introducing the background field constraint, the optical flow information was more abundant. We introduced the semi-Lagrangian nonlinear extrapolation in order to improve the accuracy of the optical flow field forecasts by considering cloud rotation.(3)Based on the SSIM, NMI, PSNR, RSD, and RMSE quantitative evaluation indices, we discovered that the forecast effect of the background field-constrained optical flow method was better than that of the standard optical flow method.
The conclusions of this study provide a theoretical basis and application method for the quantitative process of cloud system identification, tracking, and forecasting based on meteorological satellite cloud images. The extensive content of an image segmentation study and the diversity of the segmentation methods resulted in some deficiencies, including few case studies and insufficient representativeness. Going forward, we will introduce a multispectral cloud image gradient, texture, and regional information in order to further optimize the segmentation results of the dominant cloud system. This optimization should improve the ability of the optical flow method to solve the generation, evolution, and extinction of cloud clusters. A large number of practical case studies have been conducted, and the results are expected to be applied to typhoon monitoring, forecasting, and early warning in order to enhance public meteorological services.
Data Availability
The data used to support the findings of this study are available from the corresponding website. FY4A/AGRI data are available at http://satellite.nsmc.org.cn/portalsite/default.aspx?currentculture=en-US. GPM IMERG data are available at https://pmm.nasa.gov/data-access/downloads/gpm.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Authors’ Contributions
Gen Wang performed the experiments, analyzed the data, built the model, and drafted the manuscript. Dong-Yong Wang and Wei Han provided technical support for using the FCM method and contributed to the analysis, discussion, and editing of the manuscript. Jian Yin provided important suggestions regarding the optical flow method as well as manuscript editing.
Acknowledgments
This work was supported by the National Natural Science Foundation of China (grants no. 41675108 and 41805080) and the Natural Science Foundation of Anhui Province (grant no. 1708085QD89), as well as the Anhui Public Welfare Research Linkage Project (grant no. 1604f0804002) and the Key Research and Development Program Projects of Anhui Province (grant no. 201904a07020099). The authors would like to thank NASA and the National Satellite Meteorological Center for freely providing the GPM IMERG and FY4A/AGRI data online. The authors thank LetPub (http://www.letpub.com) for its linguistic assistance during the preparation of this manuscript.