Abstract
This study conducted a comprehensive evaluation of three satellite precipitation products (TRMM (Tropical Rainfall Measuring Mission) 3B42, CMORPH (the Climate Prediction Center (CPC) Morphing algorithm), and PERSIANN (Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks)) using data from 52 rain gauge stations over the Meichuan watershed, which is a representative watershed of the Poyang Lake Basin in China. All the three products were compared and evaluated during a 9-year period at different spatial (grid and watershed) and temporal (daily, monthly, and annual) scales. The results showed that at daily scale, CMORPH had the best performance with coefficients of determination () of 0.61 at grid scale and 0.74 at watershed scale. For precipitation intensities larger than or equal to 25 mm, RMSE% of CMORPH and TRMM 3B42 were less than 50%, indicating CMORPH and TRMM 3B42 might be useful for hydrological applications at daily scale. At monthly and annual temporal scales, TRMM 3B42 had the best performances, with high ranging from 0.93 to 0.99, and thus was deemed to be reliable and had good potential for hydrological applications at monthly and annual scales. PERSIANN had the worst performance among the three products at all cases.
1. Introduction
Precipitation plays an important role in hydrological cycling and is indispensable forcing data for hydrological modelling. Because precipitation has high spatial heterogeneity and temporal variability, conventional precipitation measurements at point-based gauge stations usually cannot provide enough information for hydrological applications (e.g., distributed hydrological modelling) especially in areas with sparse stations [1, 2]. In contrast, satellite remote sensing can provide the spatial precipitation data over large areas in a temporally continuous way. In recent years, satellite precipitation products have been developing rapidly and become a new and promising precipitation data source for various hydrological studies.
Currently there are several quasi-global high-resolution satellite precipitation products including TRMM (tropical rainfall measuring mission) multisatellite precipitation analysis (TMPA) [3], CMORPH (the climate prediction center (CPC) morphing algorithm) [4, 5], and PERSIANN (precipitation estimation from remotely sensed information using artificial neural networks) [6, 7]. Because such products have global (or quasi-global) orientation, the performances of satellite precipitation products are expected to vary from place to place. It is thus necessary to evaluate the performances of satellite precipitation products with local rain gauge data before these products can be used with high confidence in a specific study area. Such evaluation and intercomparison can also help to identify the most accurate and appropriate satellite precipitation product among various alternatives.
A few studies have been done to evaluate the performances of satellite precipitation products in different regions. For example, Xue et al. [8] evaluated two versions of TRMM 3B42 (V6 and V7) products in the mountainous Wangchu Basin of Bhutan using rain gauge data. The results showed that TRMM 3B42 V7 products have a significant upgrade from the 3B42 V6 products in precipitation accuracy and can serve as inputs to distributed hydrological modelling in that study area. Stampoulis et al. [9] analysed the errors of the CMORPH and PERSIANN precipitation products using rainfall data derived from weather radar rainfall estimates over the Mediterranean during heavy precipitation events and found that CMORPH exhibited better performance than PERSIANN.
In this study, we mainly focus on the performances of satellite precipitation data in the Poyang Lake Basin of China, which is an important tributary of the Yangtze River. Li et al. [10] evaluated the TRMM 3B42 V6 product in Xinjiang catchment of Poyang Lake Basin. However, they used only five rain gauge stations for the 15,500 km2 catchment, which is too sparse to generate a comprehensive evaluation of satellite precipitation product in such a large area. Hu et al. [11] compared the performances of six satellite rainfall products, including TRMM 3B43 V6, TRMM 3B42RT V6, CMORPH, GSMaP MWR+, GSMaP MVK+, and PERSIANN, with ground rain gauges located in the Ganjiang watershed of the Poyang Lake Basin, but their evaluation was only performed at the monthly scale. Liu et al. [12] evaluated both V6 and V7 of TRMM 3B42 precipitation products using rain gauge data over the Meichuan watershed of the Poyang Lake Basin at multitemporal scales (daily, monthly, and annual). Nevertheless, multitemporal scale evaluations of other commonly used satellite precipitation products such as CMORPH and PERSIANN have not been conducted yet. This work is necessary to provide a comprehensive evaluation of various commonly used satellite precipitation products in the Poyang Lake Basin.
In addition, the grid-based satellite precipitation products were usually directly compared with the point-based rain gauge data in most existing studies [10, 11]. However, there exists a significant discrepancy of spatial scales between pointed-based rain gauges and satellite grid pixels (e.g., 0.25° spatial resolution), and precipitation could vary across a single satellite pixel. The scale discrepancy between grid-based and pointed-based data might lead to errors in evaluation [13]. Therefore, scale transformation should be conducted to make the scales of rain gauge data and satellite precipitation data consistent.
This paper aims to evaluate the performances of three commonly used satellite precipitation products (TRMM 3B42, CMORPH, and PERSIANN) in a representative watershed of the Poyang Lake Basin using dense rain gauge data at consistent temporal and spatial scales. Section 2 introduces the study area and the datasets. Section 3 presents the methods used in this study. Section 4 describes and discusses the evaluation results. Section 5 concludes and discusses the future research directions.
2. Study Area and Datasets
2.1. Study Area
The Meichuan watershed, a representative watershed of Poyang lake basin, was selected as the study area due to the availability of dense rain gauge network. It is located within 26°0′–27°8′N and 115°36′–116°38′E (Figure 1). The total drainage area is 6366 km2 and the elevation ranges from 151 to 1425 m. The average slope over the watershed is 9%. This watershed is characterized by subtropical wet climate with an annual mean air temperature of 17°C and annual mean precipitation of 1706 mm.
2.2. Datasets
2.2.1. Rain Gauge Data
There are 52 rain gauge stations around the Meichuan watershed. The measured daily precipitation data from these 52 gauges were obtained from the Hydrologic Yearbooks published by the Hydrographic Office of Jiangxi Province in China. The available time period for daily precipitation data is 9 years covering 2001–2005 and 2007–2010 due to the data missing in 2006. These rain gauge data were considered as ground truth for evaluation of three satellite precipitation products in this study.
2.2.2. TRMM 3B42 Precipitation Products
The TMPA products provide precipitation for the spatial coverage of 50°N-S at the latitude-longitude resolution. The TRMM 3B42 product is one type of the TMPA products, and it is calibrated and merged with monthly rain gauge data. More detailed information regarding the processing and generation of on TRMM 3B42 can be found in [14]. The temporal resolution of TRMM 3B42 is 3-hourly, thus allowing us to obtain daily precipitation for evaluation. The latest Version 7 TRMM 3B42 products can be freely downloaded from Goddard Earth Sciences Data and Information Services Center (http://mirador.gsfc.nasa.gov). There are two kinds of TRMM 3B42 data available, 3-hourly precipitation (corresponding to the eight time period per day, i.e., UTC 00, 03, 06, 09, 12, 15, 18, and 21) and daily aggregated precipitation. The daily aggregated precipitation is obtained by summing all 8 sets of 3-hourly precipitation totals for a given day. Fortunately, the daily rain gauge stations measured precipitation during the same period as daily aggregated TRMM products (from UTC 00 to UTC 24). Therefore, the daily aggregated TRMM 3B42 products were directly used in this study.
2.2.3. CMORPH Precipitation Products
The CMORPH products provide precipitation for the spatial coverage of 60°N-S. The previous CMORPH is a pure satellite precipitation product using only satellite observation data [4]. In the latest CMORPH Version 1.0, bias correction was conducted by adjusting the satellite estimates against a daily rain gauge analysis [5]. The Version 1.0 CMORPH products can be accessed from the following website ftp://ftp.cpc.ncep.noaa.gov/precip/global_CMORPH. Three spatial and temporal resolutions can be selected: 8 km-30 min, 0.25°-3 hourly, and 0.25°-daily. In this study, the 0.25°-daily bias-corrected Version 1.0 CMORPH data were used.
2.2.4. PERSIANN Precipitation Products
The PERSIANN products use the artificial neural network technique to estimate rainfall rate from satellite observations and have spatial quasi-global coverage of 60°N-S [6]. Both raw PERSIANN product and bias-corrected PERSIANN product can be obtained from its product website. The bias-corrected PERSIANN precipitation maintains total monthly precipitation estimates to be consistent with GPCP (global precipitation climatology project) product. In this study, the bias-corrected PERSIANN data at the spatial resolution of 0.25° and temporal resolution of 3-hourly were download from the following website http://fire.eng.uci.edu/PERSIANN/adj_persiann_3hr.html. The 3-hour data were then aggregated into daily values.
3. Methodology
For comprehensive evaluation, the comparison between the satellite precipitation data and the rain gauge data was conducted at two spatial scales: the grid scale and the watershed scale. For each spatial scale, three temporal scales (i.e., daily, monthly, and annual) were further performed for evaluation. For the evaluation at grid scale, the upscaling procedure was adopted to solve the scale discrepancy between the point-based rain gauge data and the grid-based satellite precipitation data. The rain gauge data were transformed into the areal precipitation at the same scale as satellite precipitation products (i.e., 0.25° in this study). To achieve such transformation, many interpolation methods can be used, such as Thiessen polygon, IDW (inverse distance weighting), and Kriging. Since the Thiessen polygon method has been reported to be simple and robust [15], it was thus adopted in this study. In order to estimate the precipitation of each satellite grid from rain gauge data, the Thiessen polygons were then intersected with the satellite grid polygons. Figure 2 depicts the intersected polygons computed from Thiessen polygons and satellite precipitation grids. The grid-scale precipitation values were computed using the area weighted sum method from the rain gauge data, as shown in (1):where is the aggregated grid-scale precipitation, is the number of intersected Thiessen polygons within a grid, is the percentage of area for intersected Thiessen polygon in the grid, and is the precipitation value of the intersected Thiessen polygon .
After the scale transformation, grid-scale areal precipitation from rain gauge data was obtained and then used as ground truth for comparison with satellite precipitation data at a consistent scale. For the watershed-based evaluation, we first calculated the areal average precipitation of the whole watershed (i.e., average values of all the grids in the watershed) from satellite precipitation data and the rain gauge data, respectively. Then the comparison between these datasets was conducted.
Four statistical indictors were computed for the evaluation, that is, the coefficient of determination (), the bias, the root mean square error (RMSE), and the relative RMSE (labeled as RMSE%). The represents the proportion of variability in one variable that is accounted for by another variable. For a linear regression model, is simply the square of the correlation coefficient between two variables. The bias reflects the degree to which the measured value is over- or underestimated [16]. The RMSE is a frequently used measure of differences between two variables. The RMSE% is computed as RMSE divided by the mean precipitation of rain gauge data, and it can be used to evaluate the reliability of satellite precipitation product. When RMSE% is less than 50%, the satellite precipitation data are considered to be reliable, while they are unreliable when RMSE% is equal to or is greater than 50% [17]. The formulas of the four indictors are described as follows:where is grid scale or aggregated watershed scale precipitation from the satellite precipitation data, is the aggregated grid scale or watershed scale precipitation from rain gauge data, is the total number of data; is the index of data, is the average value of , and is the average value of .
4. Results and Discussion
4.1. Evaluation Results at Daily Scale
4.1.1. Overall Performance
Three satellite precipitation products (TRMM 3B42, CMORPH, and PERSIANN) were compared with the data of rain gauges at both the grid scale and the watershed scale. Figure 3 shows the scatter plots of data from rain gauge stations versus three satellite precipitation products at two spatial scales. There is no missing value in the TRMM 3B42 and CMORPH products, with 59166 data points for the grid-based evaluation and 3287 points for the watershed-based evaluation during the nine-year period. However, some missing values exist in the PERSIANN product, and there were no daily data available over the entire watershed during 17 days, leading to slightly fewer data points (Figure 3). The statistical indicators are also included in Figure 3.
(a) TRMM 3B42 grid-scale
(b) CMORPH grid-scale
(c) PERSIANN grid-scale
(d) TRMM 3B42 watershed-scale
(e) CMORPH watershed-scale
(f) PERSIANN watershed-scale
TRMM 3B42 overestimated precipitation on the whole with a bias value of 0.04, while CMORPH and PERSIANN underestimated precipitation with a bias value of −0.07 and −0.12, respectively. At the grid scale, the CMORPH product had the best overall performance with of 0.61 and RMSE of 6.67 mm/day, and TRMM 3B42 ( of 0.52 and RMSE of 9.16 mm/day) had better performance than PERSIANN ( of 0.39 and RMSE of 9.91 mm/day). This is perhaps because that the CMORPH product was calibrated using daily rain gauge analysis, while both TRMM 3B42 and PERSIANN products were calibrated using monthly precipitation data. At the watershed scale, as expected, these three types of satellite precipitation data showed a better agreement with the rain gauge data, with of 0.74 for CMORPH, 0.69 for TRMM 3B42, and 0.49 for PERSIANN, respectively.
Since the performance of satellite estimates for relatively short time period (e.g., daily) often appears as a function of precipitation intensity [18], RMSE% for different precipitation intensities (divided according to rain gauge data) was plotted in Figure 4. Generally, RMSE% decreased with the increase of precipitation intensity for all the three precipitation products at both grid and watershed scales. At the grid scale, for precipitation less than 10 mm, RMSE% of all the three products was all high (larger than 350), while for precipitation larger than 10 mm, RMSE% all decreased dramatically (less than 100). RMSE% at the watershed scale had similar trends to those at the grid scale and the RMSE% values were smaller. CMORPH had the best performance among these three products, and TRMM 3B42 had better performance than PERSIANN. Although satellite precipitation products had relatively large errors for small precipitation, when precipitation intensities were larger than 50 mm, the RMSE% values of CMORPH and TRMM 3B42 at grid scale were close to or less than 50% (47.64 and 50.21 for CMORPH, 53.60 and 50.26 for TRMM 3B42 when precipitation intensities were 50–100 mm and >100 mm, resp.). When precipitation intensities were larger than 25 mm, the RMSE% values of CMORPH and TRMM 3B42 at watershed scale were also less than 50% (45.85 and 36.5 for CMORPH, 45.86 and 40.2 for TRMM 3B42 when precipitation intensities were 25–50 mm and >50 mm, resp.). These results indicated that CMORPH and TRMM 3B42 might have potential for daily hydrological applications.
(a) Grid scale
(b) Watershed scale
4.1.2. The Performance of Capturing Storms
There were frequent severe floods in the recent decades over the Poyang Lake Basin, including the study area. Accurate estimation of extreme storms is of great importance for flood control and watershed management in this region. Therefore, it is necessary to evaluate the performance of the three satellite precipitation products in capturing storms. The Heidke skill score was adopted to evaluate the performance of different satellite precipitation products in capturing storms higher than a threshold [19]. Satellite precipitation estimations were divided into four cases according to whether they captured storms correctly: true positives (both satellite and rain gauge precipitation were higher than the threshold), false positives (satellite precipitation was higher than the threshold, while rain gauge precipitation was lower than the threshold), false negatives (satellite precipitation was lower than the threshold, while rain gauge precipitation was higher than the threshold), and true negatives (both satellite and rain gauge precipitation were lower than the threshold). The Heidke skill score (HSS) can be computed aswhere , , and , , , , means the numbers of occurrences of the above four cases (true positives, false positives, false negatives, and true negatives). The range of the HSS is to 1. A perfect set of predictions would be scored as 1.0, a set of random predictions would have an excepted score of zero, and sets of predictions having fewer hits than what would be excepted by chance would have negative scores. The HSS of three satellite precipitation products for storm thresholds ranging from 10 mm to 100 mm was plotted in Figure 5. Generally, the HSS decreased with the increase of storm threshold, and CMORPH had the best performance. At the grid scale, the HSS ranged from 0.2 to 0.6, indicating that all satellite precipitation estimations at grid scale are better than chance performance. When the storm threshold was less than or equal to 60 mm, the HSS of CMORPH were larger than 0.4, indicating that CMORPH captured moderate storms effectively. At the watershed scale, when the storm threshold was less than or equal to 80 mm, the HSS of both CMORPH and TRMM was also larger than 0.4. But for the storm threshold of 100 m, all the three products have HSS of around zero. This suggested that all the three satellite precipitation products could not capture extreme storms effectively, especially at watershed scale.
(a) Grid scale
(b) Watershed scale
In order to further analyze the performance of satellite precipitation products in capturing extreme storms, the annual maximum daily precipitation and 5-day areal average precipitation at the watershed scale were computed from the rain gauge data and the three satellite precipitation products. As shown in Figure 6, there are obvious differences between the results obtained from the rain gauge data and the three satellite precipitation products. The statistics of the errors of maximum daily precipitation and 5-day satellite precipitation are shown in Table 1. TRMM 3B42 had the smallest average errors in estimating maximum daily precipitation, while CMORPH has the smallest average errors in estimating maximum 5-day precipitation. In terms of the maximum errors of both maximum daily precipitation and 5-day precipitation, TRMM 3B42 showed the best performance and CMORH had better performance than PERSIANN. However, even in the best case, the relative maximum errors reached 32.92% and 34.34% for the maximum daily precipitation and 5-day satellite precipitation, respectively. Therefore, all the three satellite precipitation products were deemed to have limited capabilities in capturing extreme storms because of their large relative errors.
(a)
(b)
The results shown in Table 1 suggested that CMORPH and TRMM 3B42 might be useful for hydrological applications at daily scale. However, since all the three products had poor performances in the estimation of small precipitation values and extreme storms, local calibration with rain gauge (or ground radar) data using data assimilation methods (e.g., optimum interpolation) [18, 20, 21] should be carried out to further improve the daily precipitation estimations before they are used in real-world hydrological applications at daily scale.
4.1.3. Seasonal Variation of Daily Accuracy
Figure 7 plots the seasonal variation of accuracy for these three satellite precipitation products (TRMM 3B42, CMORPH, and PERSIANN) at daily and grid scales. It is found that both RMSE% and absolute value of bias showed seasonal fluctuations. In spring and summer, when it rains relatively more, the RMSE% values were lower and the absolute values of bias were smaller. CMORPH had the smallest RMSE%, and TRMM 3B42 had smaller RMSE% than PERSIANN. The fluctuations of the accuracy of the PERSIANN product were the largest among the three satellite precipitation products, with very high RMSE% values and absolute values of bias in the winter.
(a) Monthly variation of RMSE%
(b) Monthly variation of abs(bias)
4.2. Evaluation Results at Monthly Scale
4.2.1. Overall Performance
The daily precipitation data were accumulated to monthly total precipitation for the rain gauge data and three satellite precipitation products (TRMM 3B42, CMORPH, and PERSIANN). The monthly total precipitation data from four datasets were compared at both the grid scale and the watershed scale. The results with statistical indicators are shown in Figure 6. There are totally 1944 comparison points at the grid scale and 108 points at the watershed scale during the nine-year period. Good agreements with the rain gauge data were observed for all the three satellite precipitation products at the grid scale, with of 0.93 for TRMM 3B42, 0.89 for CMORPH, and 0.83 for PERSIANN, respectively. As expected, such agreements are even better at the watershed scale with of 0.98 for TRMM 3B42, 0.94 for CMORPH, and 0.88 for PERSIANN. The higher accuracy at monthly scale than at daily scale is due to the fact that the errors at daily scale were nearly symmetrical (see Figure 3) and thus could cancel each other out after the aggregation. In terms of all four statistical indicators (Figure 8), TRMM 3B42 had the best performance at monthly scale, and PERSIANN had the largest errors.
(a) TRMM 3B42 grid scale
(b) CMORPH grid scale
(c) PERCIANN grid scale
(d) TRMM 3B42 watershed scale
(e) CMORPH watershed scale
(f) PERCIANN watershed scale
4.2.2. Seasonal Variation of Monthly Accuracy
Figure 9 plots the seasonal variation of RMSE% of three satellite precipitation products (TRMM 3B42, CMORPH, and PERSIANN) at monthly and grid scales. The results show that the RMSE% values of all the three satellite precipitation products show seasonal fluctuations, with smaller RMSE% values in spring and summer when the precipitation is relatively high. TRMM 3B42 had the smallest RMSE% at monthly scale, and the RMSE% values of TRMM 3B42 in April, May, and June were all below 20%. The RMSE% values of TRMM 3B42 and CMORPH in all months were below 50%, indicating that these two satellite precipitation products have stable and acceptable accuracy and thus can be used in real-world hydrological applications at monthly scale. In all 12 months, PERSIANN had the largest errors among the three satellite precipitation products.
4.3. Evaluation Results at Annual Scale
The accumulated monthly precipitation data were further accumulated to annual total precipitation for the rain gauge data and the three types of satellite precipitation data. The annual precipitation data from the three satellite precipitation products (TRMM 3B42, CMORPH, and PERSIANN) are plotted versus those from the rain gauge data at two spatial scales in Figure 8. There are totally 162 data points at the grid scale and only 9 data points at the watershed scale during the nine-year period. The degree of reliability was improved as the aggregation in the temporal scale. All the three satellite precipitation products had high (0.84–0.99) and small relative RMSE (4%–14%). TRMM 3B42 had the best performance, and CMORPH had better performance than PERSIANN. It can be concluded that the annual accumulated precipitations from all three satellite precipitation products, especially TRMM 3B42, are reliable at both the grid scale and the watershed scale. However, Figure 10 clearly shows the tendency for overestimation by TRMM 3B42 but underestimation by both CMORPH and PERSIANN.
(a) TRMM 3B42 grid scale
(b) CMORPH grid scale
(c) PERCIANN grid scale
(d) TRMM 3B42 watershed scale
(e) CMORPH watershed scale
(f) PERCIANN watershed scale
5. Conclusions
In this study, three satellite precipitation products (TRMM 3B42, CMORPH, and PERSIANN) were evaluated against rain gauge data during a nine-year period over the Meichuan watershed in China during a nine-year period. The evaluation was conducted at grid and watershed spatial scales and at daily, monthly, and annual temporal scales. For the evaluation at the grid scale, the point-based rain gauge data were first transformed to the consistent grid scale using the Thiessen polygon method for the grid versus grid comparison. At the watershed scale, average areal precipitation of the watershed was calculated by averaging values of all the grids in the watershed.
Comparisons with rain gauge data showed that, at daily scale, three satellite precipitation products had ranging from 0.39 to 0.61 at grid scale and ranging from 0.49 to 0.74 at watershed scale. For the precipitation intensities less than 25 mm/d, RMSE% of all the three precipitation products exceeded 50% at watershed scale, while for precipitation intensities larger than or equal to 25 mm, RMSE% of CMORPH and TRMM 3B42 was less than 50%. As far as the capability in capturing storms is concerned, all three products performed poorly in capturing extreme storms larger than 100 mm/d. However, the moderate to large storms (<80 mm) can be well captured by CMORPH and TRMM 3B42. These results suggested that CMORPH and TRMM 3B42 might be useful for hydrological applications at daily scale. However, because they had relatively poor performances in estimating small precipitation and extreme storms, local calibration with rain gauge or ground radar data should be carried out to further improve the daily precipitation estimates before they are used in real-world hydrological applications at daily scale. As the temporal scales increase, the performances of all the three satellite precipitation products were improved. At monthly and annual temporal scales, TRMM 3B42 had the best performances with high values ranging from 0.93 to 0.99 and low relative RMSE% values ranging from 4% to 23%. CMORPH and PERSIANN also had good performances at monthly and annual scales, all with values larger than 0.83 and RMSE% values smaller than 38%. Therefore, it can be concluded that satellite precipitation products, especially the TRMM 3B42 product, are reliable and have good potential for hydrological applications when they are used at monthly and annual scales. In addition, there were obvious seasonal fluctuations in the accuracies of all three precipitation products, with higher accuracies in wet seasons than in dry seasons. These seasonal fluctuations of accuracies should be considered when these satellite precipitation products are used in real-world applications.
In the future, hydrological simulations using satellite precipitation data as inputs should also be conducted in the Poyang Lake Basin to investigate whether the errors in satellite precipitation products can be tolerated by hydrological models.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
This study was funded by the University Science Research Project of Jiangsu Province (no. 14KJB170009), the National Natural Science Foundation of China (no. 41023010), the Thousand Talents Program of China, the Scientific Research Starting Foundation for High-level Scholars, Nanjing Normal University, and the Priority Academic Program Development of Jiangsu Higher Education Institutions.