Abstract

The reconstruction of periodic signals that are embedded in noise is a very important task in many applications. This already difficult task is even more complex when some observations are missed or some are presented irregularly in time. Kolmogorov-Zurbenko (KZ) filtration, a well-developed method, offers a solution to this problem. One section of this paper provides examples of very precise reconstructions of multiple periodic signals covered with high level noise, noise levels that make those signals invisible within the original data. The ability to reconstruct signals from noisy data is applied to the numerical reconstruction of tidal waves in atmospheric pressure. The existence of such waves was proved by well-known naturalist Chapman, but due to the high synoptic fluctuation in atmospheric pressure he was unable to numerically reproduce the waves. Reconstruction of the atmospheric tidal waves reveals a potential intensification on wind speed during hurricanes, which could increase the danger imposed by hurricanes. Due to the periodic structure of the atmospheric tidal wave, it is predictable in time and space, which is important information for the prediction of excess force in developing hurricanes.

1. Introduction

The spectral analysis of longitudinal data is the method of analyzing natural periodicities of data that are observed over long periods of time. However, the problem with longitudinal studies is that data is not collected at regular (or equally spaced) time intervals. Studies have been conducted as to how to handle the problem presented by having missing values. Examples of such studies include separating the data into the missing part and the nonmissing part called the EM (Expectation Maximization) algorithm by Shumway and Stoffer [1] and the CLEAN method formulated by Baish and Bokelmann [2], which they tested on simulated noisy data and due to their results applied this method to seismological data. Other researchers have also considered estimating the missing data [35].

The Kolmogorov-Zurbenko (KZ) filter is an efficient means of analyzing data even when datasets are considered to have missing values ([68], Kolmogorov-Zurbenko filters). Several computer algorithms are available that will investigate a time series with irregularly spaced data, including the nonequispaced Fast Fourier Transform (NFFT) developed by Keiner et al. and the Mathematics Faculty at Chemnitz University of Technology (offered in their free software) and the KZA algorithm provided in R-software by Close and Zurbenko [9].

This paper highlights the strength of a method of periodic signal reconstruction that can be used for longitudinal data or more specifically a time series with missing values, namely, the Kolmogorov-Zurbenko Fourier Transform (KZFT) filter. The strength of the filter in determining spectra of datasets with irregularly spaced (or missing) values and its ability to reconstruct the periodic signal after determining such dominant frequencies are presented. This method is then used to reconstruct the atmospheric tidal wave at locations during times of major meteorological events. The goal of this paper is to discuss the effects of the atmospheric tidal waves on the intensification of major hurricanes/storms such as Super Storm Sandy and Hurricane Katrina.

2. The KZFT Algorithm and Irregularly Spaced Data

KZ algorithms were developed for use in a highly noisy environment and are able to provide a numerical result that is very close to the actual hidden signals. With high accuracy the KZ algorithms are able to detect the frequency characteristics of the hidden signals as well as provide signal reconstructions. More details can be found in Zurbenko [6], Zurbenko and Porter [7], and Yang and Zurbenko [10]. All KZ algorithms are currently available in R-software.

The spectra are important diagnostic tools for the reconstruction of hidden signals. The spectrum displays the dominant frequencies of the data. Dominant frequencies can be detected and then used to reconstruct the periodic signal inherent to the noisy dataset. A highly efficient example of such reconstruction for equally spaced data is provided in Potrzeba and Zurbenko [11]. The authors will investigate the efficiency of such reconstructions by simulations of complicated hidden signals in a very noisy environment. Through simulation it will be demonstrated that this filter is extremely capable of determining spectra and reconstructing periodic signals from noisy datasets that have irregularly-spaced data (or data that is a time series with missing values).

A signal that contains a couple of periodic sine waves is complex; therefore it can be difficult to recognize its periodic components. The total amplitude of the summation may oscillate with low difference frequency between basic components. When such a signal is buried in noise the outcome appears absolutely chaotic. Unequally spaced data or data with missing observations can also provide extra difficulties for the recovery of the original signal. The authors will estimate the accuracy of the reconstruction of such an original signal through simulations. For the simulations, the authors formed a wave of 17,000 consecutive time observations. In order to consider a longitudinal dataset with irregularly-spaced data a percentage of observations were randomly “removed” from the wave formed from 17,000 consecutive time observations. The parameter for the percentage of missing observations is denoted by . Close and Zurbenko’s [9] KZFT algorithm in the KZA package of R-software was used. This algorithm has three main parameters. When using this algorithm one must specify a vector of observations, a vector of the observations corresponding time, = window size of the filter, = number of iterations of the filter, and = frequency. When using this tool to estimate spectra the frequency parameter, , should be set to NULL. An adaptive algorithm, DZ [12] for spectral estimation, is also available in the same package. The DZ parameter in the KZFT algorithm defaults to 5%; it is used to address the energy levels of interest and may be adjusted by users.

The aforementioned simulated signal was used to test the KZFT algorithm’s ability to accurately determine spectra of datasets with missing values. Several simulations were performed, first with low percentage of missing values, , with KZFT parameters and . The dominant frequencies were accurately displayed in the spectrum even with heavier noise added to the signal (noise ~ and again with noise ~ ). The percentage of missing information, , was increased following these successful results. Values of hidden frequencies were reconstructed with the help of the DZ algorithm (the smoothing parameter that is at 5%) and used for further reconstruction. The reconstructed signal was compared with the original signal over the entire timeframe and randomly chosen small intervals of time to observe quality of reconstruction.

For practical considerations, the percentage of missing values was increased to . The KZFT algorithm was applied with parameters and to determine the spectrum. The result of this simulation was that the spectrum again accurately depicted the dominant known frequencies. To further test the use of the KZFT algorithm, the signal with was embedded within a noisier dataset by increasing noise to and then repeated again with noise ~ added to the signal. For each simulation, the filter was able to precisely capture the hidden frequencies detected by the DZ algorithm in the noisy signal.

Due to the abovementioned successful results, the missing rate of observations was further increased to to determine if the spectrum could continue to capture the dominant frequencies. Using a wider window length of and iterations, the KZFT algorithm was used to determine the spectrum for the simulated longitudinal dataset. It is apparent in Figure 1 that the dominant frequencies of 0.08 and 0.10 cycles per unit time are identifiable as the signal’s inherent frequencies.

The results of these aforementioned simulations demonstrate the effectiveness of the KZFT filter to estimate spectra for longitudinal data where time is randomly, uniformly, i.i.d. over an interval. The parameters ( and ) were first selected to keep the amount of available time points in a window equal and then increased to improve the results. A shorter window length (smaller ) is possible, but it will yield a much wider statistical confidence. The current window sizes for the meteorological data, in Section 3, were chosen from the consideration that within ~1 lunar cycle its astronomic influences will remain the same. This assumption is realistic for practical purposes where observations are considered to be made at arbitrary time points. Computer readings always provide a limited accuracy (days, hours, etc.), therefore a discrete scale is used in practice, with the exception that those time points can be missed. After using the spectra to determine any periodic signal inherent to data, researchers can reconstruct those periodic signals.

Following the determination of spectra for the simulated longitudinal datasets where time points were missed with probabilities 20%, 60%, and 70%, the KZFT algorithm was then applied to reconstruct each signal. With the dominant frequencies determined from each spectrum, the KZFT filter was then applied by setting the frequency parameter ( ) to each of the determined frequencies; varying window sizes and number of iterations have been used. For multiple peaks in a spectrum, the KZFT filter was applied multiple times, adjusting for each desired frequency parameter. The resulting periodic signal is the summation of these filters. The accuracy is approximately , where is the number of available observations within the window length; for more information regarding the variance of the KZFT filter and the resolution of wavelets see Yang and Zurbenko [10] and Zurbenko and Porter [7]. Accuracy may be increased by two ways; by increasing the window size parameter, or through repeated simulations. The former may have some computer limitations.

The authors demonstrate the ability of the KZFT algorithm’s reconstructive properties for longitudinal data that was the sum of two sine waves about periods 10 and 12.5 with noise ~ with . Given the corresponding frequencies found from the spectrum, the KZFT filter was applied to the data to reconstruct the signal. The KZFT filter in the KZA package of R-software has a parameter = frequency. By defining this parameter for each of the known dominant frequencies found in the spectrum, the authors applied this filter with parameters and to reconstruct the signal about each frequency (0.08 and 0.10 cycles per unit time). The reconstructed signal was determined by applying the KZFT filter twice (once about each dominant frequency) and then by summing the results of each filter. The correlation between the true signal and the reconstructed signal was 96.4% displayed in Figure 2. The original observations provide no guess of the complex, hidden periodicity, which was perfectly reconstructed by the algorithm. If necessary those already high correlations can be improved essentially with wider time windows in use. The high quality of reconstruction does not affect the major study of effects of hidden periodicities.

Since the correlation between the reconstructed signal and the true signal was high with 60% missing observations, the percentage of missing values was increased to . The KZFT filter was applied to the simulated signal that was the sum of two sine waves about frequencies 0.10 and 0.08 cycles per unit time with noise ~ , with . The KZFT parameters used were and (a higher window size was used for the higher rate of missing observations) to reconstruct the signal about each frequency determined from the spectrum. The correlation between the true signal and the reconstructed signal was 92.1%.

The ability to determine spectra and to reconstruct periodic signals allows researchers to perform important diagnostics on any longitudinal dataset or time series with missing data. The figures in this section clearly indicate the strong ability of the algorithm to accurately restore small hidden periodic signals from a very noisy original dataset that is absolutely, chaotically observed in time. This method has a wide range of practical uses.

3. Application of Tidal Wave Reconstruction in Determination of Most Hazardous Moments of Hurricanes

Tidal waves in the five oceans greatly influence all aspects of life in the oceans and on their shores. Schedules of those tides are available and their predictions are widely distributed as a highly important factor affecting surrounding life. Tides in the oceans can be measured directly and it is a continual procedure to do so in the network over the globe. Existence of similar tides in atmospheres around the globe has been discovered by distinguished scientist Chapman [13, 14].

Atmospheric tides can be noticeable for us only by changing atmospheric pressure. Measurements of atmospheric pressure are available in the network over the Globe, but these measurements are strongly affected by much larger synoptic fluctuations; thus making the possibility of the direct observation of tides in the atmosphere almost impossible. Chapman proved the existence of those tides, but could not reconstruct those waves in a numerical way. The development of Kolmogorov-Zurbenko (KZ) filtration ([6, 7, 10], Kolmogorov-Zurbenko filters) made that solution possible [15, 16]. Tidal waves in the atmosphere are generated by lunar gravitational forces and solar thermal influences, having strict astronomic frequencies , , , and cycles per hour that correspond to 1 and 1/2 sun and moon days on Earth. Existence of those strict lines in the spectrum of atmospheric pressure and the method of the reconstruction has been developed by [15, 16].

One goal of this paper is to examine the effects of atmospheric tidal waves on the intensification of hurricanes, such as Super Storm Sandy of 2012, Hurricane Katrina of 2005, and Hurricane Rita of 2005. Any developed hurricane has strictly synoptic reasons for its development and trajectory in time and space. The strength of hurricanes is usually described by a level of low pressure in the center that is characteristic of the hurricane itself. Outside, high pressure is a feeding force of the hurricane, thus creating circular descending winds proportionally to the difference in pressure. Periodic fluctuations in outside pressure due to the tidal wave will create periodic fluctuations in the strength of the wind. The difference in pressure is the original force yielding acceleration, which is the derivative of speed. The 24-25 hour periodic wave in pressure will be approximately a one-quarter period ahead of the resulting periodic wind speed, by connection between the periodic sine wave and its derivative. Thus, one may expect a 4–7 hour delay in wind speed compared with the periodic wave in the outside pressure due to the atmospheric tidal wave. These general notions are very well supported by data analysis of wind speed and pressure in the aforementioned hurricanes. The fluctuations in strength of hurricanes are organized in time when the hurricane itself is moving due to the synoptic conditions in the area. Spatial scale of tidal waves in the atmosphere is about 6000 miles and those spatial scales are approximately periodic with a period of approximately 25 hours. Hence, for each hurricane, every 25 hours there are potentially more dangerous moments. During hurricane trajectory in space there exists more dangerous times of expected force. In this paper the authors are not addressing the details of dynamic forces that are responsible for hurricane constructions; the scope of this paper is statistical evidence of periodic reinforcements of developed hurricanes due to the effect of the atmospheric tidal wave.

Data collection during hurricanes is frequently interrupted by their destructive forces; therefore the authors will use the methods described in the second section of this paper to process data with many of the missing values. As it was shown in the previous section, the power of our methods of reconstruction of a periodic wave is not diminished with those missing observations. The data used in this paper is from the ISD-Lite (Integrated Surface Database) from the National Climatic Data Center (NCDC) (http://www.ncdc.noaa.gov/oa/climate/isd/index.php?name=isd-lite).

Figure 3 displays raw atmospheric pressure data along with the reconstructed tidal wave during Super Storm Sandy at Atlantic City International, New Jersey, during October 2012. With the approaching storm the atmospheric pressure rapidly declined; measurements were interrupted during the center of the storm’s arrival, but it did not affect the ability of the reconstruction of the tidal wave. It is shown that the raw pressure and reconstructed periodic wave were in-phase. Figure 4 displays the tidal wave shifted forward 5 hours and sustained wind speed at that area; the periodic influence of the tidal wave on wind speed was visually and statistically evident. It was determined that the highest correlation between wind speed and the tidal wave was achieved with a shift of 5 hours; therefore, the reconstructed tidal wave was shifted 5 hours ahead in time. The recorded landing time was a few hours later than the highest effect of the tidal wave, so if an actual hurricane arrived at a point a few hours earlier its effect could have been substantially stronger.

Figure 5 displays the tidal wave reconstruction shifted 5 hours ahead and wind speed during Hurricane Katrina in New Orleans, Louisiana, in August 2005. It displays undeniable evidence of periodic force of hurricane intensified by the atmospheric tidal wave. The wind speed practically repeats periodic fluctuations in tidal wave in the approaching hurricane. There was no record of wind speed at the landing time due to the destruction of the hurricane. Nevertheless, the authors were able to reconstruct the tidal wave during the entire event. Shown in the figure are high wind speeds with high tide at the recorded area before landing time. Sustained wind speed as a 3-hour moving average of wind speed and the reconstructed tidal wave are observed to be in-phase. There was a short relief of wind speed due to the low tide hours before landing time. The landing time coincides with a secondary peak in the tidal wave; hours later the peak in the tide came to the area together with enormous destruction. In many cases, data collections, unfortunately, are interrupted in and around the landing time of hurricanes; therefore it is difficult to analyze correlation during these events. It is the authors’ consideration that pattern influences with the tidal wave that are clearly displayed before the event will remain, regardless of the fact that formal correlation analysis is impossible within the rapidly changing wind speeds during the event.

Figure 6 displays a reconstructed tidal wave shifted forward 7 hours and sustained wind speed during Hurricane Rita at Houston, Texas, during September 2005. Again there is an obvious correlation of wind speed with the amplitude of the atmospheric tidal wave before the arrival of the hurricane. Hurricane Rita was expected to be very strong and cities were evacuated prior to its arrival. It is seen in Figure 6 that the landing time of the hurricane coincides with lowest effect of tidal wave in the area. Destruction caused by that hurricane was sufficiently low. It might have been a very different story if the hurricane had arrived to the area 12 hours earlier or 12 hours later.

4. Conclusions and Future Developments

Restorations of periodic signals in a very noisy environment, with the possibility of substantially missed observations, recently became possible and provided the opportunity to look at the same events with new technological opportunities of time and space data analysis. This opportunity enabled the authors to better understand time and space descriptions for extreme weather effects and climate fluctuations [1618]. It permits one to see fluctuations and effects in time and space that are very well hidden in high level noise or events of different natural lows.

This paper provides a few examples of evidence of periodic reinforcements of extreme weather phenomena. Weather fluctuations are regularly described by synoptic forces, which permit predictions up to a few weeks in advance. Tidal waves in the atmosphere are smaller than synoptic fluctuations of pressure, but they are organized by astronomic forces of gravity of the moon and sun daily thermal effect that are periodic by nature. The periodic nature of the atmospheric tidal waves makes them predictable for a long time into the future. Thus, for weather prediction analysis, atmospheric tidal waves are creating an opportunity to improve total prediction by using highly, well predictable values of tidal waves. Hurricane trajectories and landing times are synoptically, very well predicted about 1-2 days in advance. Tidal waves in the atmosphere can be reproduced up to one month in advance. Extra security precautions should be made in areas where a hurricane is scheduled to arrive, together with the high tide effect. Schedules of the tides in atmosphere should be reconstructed everywhere just as these schedules exist with tides in the oceans. The atmosphere has no shore bounds so atmospheric tides are more continuously constructed over the globe and can be easily reproduced over it with a prediction opportunity of at least one month ahead. The amplitude of tidal waves in the atmosphere has slow fluctuations due to differences in astronomic forces, which may also be examined to make predictions of tides for a long time into the future.

Tables of tidal waves in the atmosphere should be created in time and space over the planet and these new schedules will be just as valuable as the schedules of tides on the shorelines of the oceans. This information provides awareness for extreme weather events, such as hurricanes, that will arrive at a vulnerable time at specific locations. It will be very interesting to examine the time record of the pressure in the center of developed hurricanes as long as such data will be available. It will also be interesting to detect the periodic component of tidal waves in the center and compare it with tidal wave outside the hurricane. There is a possibility of phase shift between periodic components, which may contribute to further reinforcement of hurricane. In proper synoptic conditions it is likely that atmospheric tides are affecting the strength and probability of appearance of tornados. The appearance of extreme events is controlled by synoptic conditions, but in the examples examined in this paper, tidal waves are making the difference in their strength in time and space due to their periodic structure.

Gravity forces are applicable to any substance, including the plasma of the sun; so gravity of the planets must create tidal waves on the sun. Tidal forces are coherent for big distances and can easily create currents. Coriolis forces will cause cyclones in those currents. Cyclonic turbulences in such currents can be strongly reinforced by the electromagnetic structure to create strong eruptions. The frequency of such eruptions is dictated by periodicities of tidal waves amplitudes, which for example, have close to an 11-year cycle due to the periodicities in planets’ movements. Examination of those so-called difference frequencies caused by planets periodicities around the Sun was provided in Zurbenko and Potrzeba [16]. The difference frequency of 11 years came from the examination of a couple of planets with the most influential gravity on the sun. All of the planets together may cause a range of several difference frequencies, including some very long periods. Restoration of those periodicities can be run as a big scale computer project, which may reproduce entire difference frequencies for a very long period of time. The restoration of those periodicities may help to explain and predict long term changes of the climate on Earth. The climate on Earth changed dramatically within a substantial timeframe. Those changes were partially due to the changing Sun activities. The reproduction of periods in those activities may provide partial explanation to the long scale climate changes on Earth. Sun activity is affecting all aspects of human life. It might cause magnetic storms and it might affect rate of some diseases; see Valachovic and Zurbenko [19]. Any periodic events can be predicted for an extended time. For signals that are embedded in high levels of noise, it is truly a treasure that the periodic signals can be detected and forecasted for extended time.

Acknowledgment

The authors express their gratitude to Dr. Robert Henry from New York State Department of Environmental Conservation for providing data that was used in this paper and for the many discussions regarding the results.