Compressing Phase Space Detects State Changes in Nonlinear Dynamical Systems
Equations governing the nonlinear dynamics of complex systems are usually unknown, and indirect methods are used to reconstruct their manifolds. In turn, they depend on embedding parameters requiring other methods and long temporal sequences to be accurate. In this paper, we show that an optimal reconstruction can be achieved by lossless compression of system’s time course, providing a self-consistent analysis of its dynamics and a measure of its complexity, even for short sequences. Our measure of complexity detects system’s state changes such as weak synchronization phenomena, characterizing many systems, in one step, integrating results from Lyapunov and fractal analysis.
Dynamics of natural systems is often described by nonlinear equations. When those equations are unknown, we can reproduce the system dynamics through the reconstruction of the manifold from the time course of one of its variables [1–3]. Phase space reconstruction has been widely applied in modeling and predictions of several nonlinear systems, such as ecological, climate, and neural ones [4–7]. The embedding theory proposed by Takens  allows one to reconstruct a one-to-one map of the attractor of a dynamical process using time-lagged values of a single system variable. The delay-coordinate map is built from the time series by vectors in of the form , where is the time delay. To correctly build the embedding of -dimensional manifold , it is crucial to choose adequate values for and , i.e., the embedding parameters.
According to the Whitney theorem, the diffeomorphism on is ensured by choosing an embedding dimension  and the result may be generalized also to noninteger (fractal) dimension . Whitney theorem has been relaxed, for example, in [11, 12], but still those studies provide an upper bound for the estimation of . Several methods were developed to estimate the minimum possible embedding dimension , and usually those methods are based on the fact that when evaluating some quantities on a delay-coordinate map, they do not vary for higher than the proper embedding dimension. Those diffeomorphism invariants could be, for example, the largest Lyapunov exponent or the percentage of false nearest neighbours [14, 15], where the latter option, in its implementation introduced by Cao , is currently the most used method to estimate the minimum .
To estimate the embedding dimension, methods that involve the fact that entropies are diffeomorphism invariants have been proposed and include, for example, differential entropy  and permutation entropy , where the latter has the advantage to take into account the temporal information contained in the time series . Kolmogorov complexity, also known as algorithmic entropy, was proposed in 1968 as a measure of the amount of information of the trajectory of a dynamical process  and is defined as the length of the shortest description that produces the trajectory as output. Even if Kolmogorov complexity cannot be computed, for the trajectories of a dynamical system, it is usually approximated using lossless compression algorithms, following the theorem of Brudno who, in 1978, wrote the equality between Kolmogorov complexity and entropy rate . Nevertheless, to date, estimating embedding dimension is still far from being an easy task, although this parameter is critical to gain insights about the physics of the underlying dynamical system.
In this paper, we show that optimal embedding dimension can be estimated through a measure of the Kolmogorov complexity, which is here evaluated using the compression algorithm introduced by Lempel and Ziv . Our dimension estimate could represent a more robust measure than other information estimators because it is independent on the system representation , so it may be estimated without prior knowledge of the value of optimal time delay . The main advantage of our approach is that we explore the geometry of the manifold of the dynamical system with complexity measures that capture the rich information about the underlying dynamics and reveal change in the system state that is otherwise difficult to detect [25–27]. In particular, here we show that exploring how the system approaches its proper embedding dimension can reveal the emergence of chaotic synchronization phenomena in a coupled drive-response system.
2. Low-Dimensional Chaotic Systems
To estimate the optimal embedding dimension , we built , an ensemble of delay-coordinate maps from as a function of time delay and . Then, at fixed and , we discretize values of delay coordinate map by using a grid with bin size : . We computed the entropy rate of the resulting sequence of symbols through a Lempel–Ziv data compression algorithm :where is the shortest subsequence starting at index that does not appear in the window of length . We evaluated entropy rate for the entire ensemble of delay-coordinate maps and estimated as the optimal embedding dimension the one such that
This means that the optimum embedding dimension is the one at which entropy rate has at least a component that behaves as a nonlinear function of , that is, . That choice was suggested by the fact  that the system with causal interactions among their elements has entropy that grows as a nonextensive function of their size , where the nonextensive component is described by a power law function .
To estimate the optimal dimension for the embedding, avoiding the evaluation of the optimal time delay , we tested our algorithm with a specific set of values and found robust results with respect to the choice of this parameter. Figure 1(a) shows an example for a single realization of a Lorenz signal, where estimation of optimal does not change for different values. Figure 1(b) shows the results of our algorithm for a set of chaotic systems. Specifically, we consider Logistic, Hénon and Ikeda maps, Rossler, Lorenz, and Mackey–Glass systems with three different time delays, widely used to model the dynamics of several natural phenomena, from chemical reactions to climate. For each system, we computed our measures across 50 different realizations and compared our estimates with correlation dimension () measures [30–33]. We found that for most of the tested systems, our dimension estimate is close to Whitney’s upper bound , while for Mackey–Glass systems that we tested at three different time delays, we found that our measures are close to the lower bound delimited by .
3. Unidirectionally Coupled Systems
In coupled chaotic systems with a drive-response configuration, generalized synchronization (GS) may occur if the state of response system does not depend on its initial condition but depend only on the state of the driver , that is, if there is a functional relation between trajectories in the phase space, . When is the identity, there is identical synchronization, which is easy to detect because the synchronized motion becomes simply a sharp line in vs plane . Otherwise, when differs from the identity, weak GS may emerge, and this phenomenon is difficult to detect. Different methods to detect GS have been proposed .
For instance, it has been proven that synchronization occurs when all of the conditional Lyapunov exponents are negative , while it is possible to gain insight into the kind of synchronization that is acting by considering the dimension of the global synchronization manifold with respect to the dimension of the driver system : if , then the response system does not have an effect on the global dimension and there is identical synchronization. Otherwise, if , the global manifold has a fractal structure and the synchronization is weak . To reveal weak GS in a coupled system, two different classes of measures are needed, namely, conditional Lyapunov exponents and dimension(s) of the global manifold.
Here, we show that the analysis of the dimension of the response system through lossless complexity measures can easily detect the emergence of GS. To this aim, we studied synchronization phenomena between two unidirectional chaotic systems, where GS takes place as a function of coupling factor . We studied the optimal dimension of the systems assuming, as we did for noncoupled systems, that entropy is well described by a nonextensive function of number of elements . That assumption is especially well posed when the system is weakly sensitive to initial conditions, where it was proven [38, 39] that the usual Shannon entropy measures are not appropriate and a new measure of entropy has to be introduced that depends on sensitivity to initial conditions and the multifractal spectrum.
3.1. Heterogeneous Systems
As a first example, we considered a unidirectionally coupled system in which the autonomous driver is a Rossler oscillator:and the driven one, , is a Lorenz oscillator:
This type of system was investigated in previous works [40–42]. In Figure 2(a), we show that, similarly to previous studies, GS arises for a threshold coupling strength , where the conditional Lyapunov exponent becomes negative. We computed Lyapunov exponents using the pull-back method [43, 44] which relies on the Gram–Schmidt orthonormalization of Lyapunov vectors while integrating the dynamical system with a fourth-order Runge–Kutta algorithm (integration time step ). We computed exponents with 5000 time points, after discarding the first 10000 iterations. The correlation dimension is estimated by using 25000 time points and looking for the plateau in the function , indicating a suitable scaling relationship. As shown in Figure 2(b), of the global manifold is higher than of the driver Rossler system, indicating that at the threshold , the whole system undergoes a regime of weak synchronization.
For each coupling value, we estimated the optimal embedding dimension as the average across 50 realizations of the system dynamics. Time series with 1000 time points were used for the estimation. When approaching the synchronization threshold , increases abruptly and assumes values between the two extremes of two independent Lorenz and Rossler systems. Furthermore, it is worth noting that the trend of estimates is opposite to the trends of both and conditional Lyapunov exponents, suggesting that those measures are referring to different but complementary properties of the dynamical system. Previous studies investigated how measures of entropy and complexity are both needed to describe natural systems, since they capture different properties of the dynamics [46, 47]. In particular, Lyapunov exponents and fractal dimension measures were usually related to the degree of randomness and disorder of the dynamics, while our hypothesis is that , which is the dimension at which the entropy rate is described by a nonlinear function, is related to the length of the patterns, i.e., to regularities in the dynamics that allow for its compression.
3.2. Identical Systems
A second example we considered is the unidirectionally coupled system formed by two identical Hénon maps , where the driver is described by the system:and the driven one is described by
We computed Lyapunov exponents using the pull-back method with 5000 time points and we found that the conditional exponent takes negative values in two different intervals of couplings: in a window and then for (see Figure 3(a)). As shown in Figure 3(b), in the first window , the correlation dimension of the global manifold is higher than the correlation dimension of an independent Hénon system: , indicating that the synchronization is weak in this interval. Furthermore, for coupling values higher than , we have that , showing that for high couplings, identical synchronization takes place. Both the coupling strength intervals and the two different regimes for GS are revealed with a single embedding measure. Here, we computed for each coupling value the optimal embedding dimension as the average across 50 realizations, 1000 time points each, using lossless compression of the dynamics. We found that in interval, the complexity of the coupled system increases, giving rise to an increased estimated of the global manifold. For , the optimal has a drop, showing that there is a change in the system state; in particular, estimates take values typical of an independent Hénon map, revealing an identical synchronization.
In conclusion, we have shown that complexity measures used to reconstruct the geometry of the manifold of a dynamical system can be used to gain many insights about the system itself, even when the underlying governing equations are not known. We observed how the irregularity of the dynamics, expressed by entropy rate estimates, reaches a plateau and remains constant by increasing the dimension of the manifold, providing a robust and parameter-free estimate of the intrinsic optimal dimension. Our measure is quite stable for different values of time delay , providing a desirable method for the reconstruction of the manifold that relies only on a single estimate.
We choose to relate complexity of the system to the way at which entropy rate measures depart from extensive functions and become nonlinear functions of the number of system dimensions. How to properly evaluate complexity has been a debated topic in last years. One of the most debated issue is the fact that information theoretic estimates like Shannon entropy measure the degree of randomness of the system and do not take into account system’s dynamical organization, whereas ideal complexity measures should treat both random and lower distributions as minimally complex . In our approach, we focused on the entropy component that deviates from extensivity, arguing that it contains the information that has to be related to effective system’s complexity.
To detect synchronization, usually quantities related to the randomness of the dynamics [49, 50], such as Lyapunov exponents and fractal dimension, are investigated. However, to be estimated in a reliable way, those quantities require long time series, in particular to compute correlation dimension, which is also potentially biased by user’s choices about proper scale and dimensions. Our method, on the contrary, gives robust results for shorter time series and has the advantage to capture and distinguish, with a single measure, different synchronization regimes. Furthermore, the dimension at which the time series reaches its maximum disorder is informative and gives us insights about the intrinsic structure of the system. The way in which the optimal embedding dimension varies as a function of the parameters ruling the system dynamics highlights state changes, as long as they affect regularities in dynamical patterns. In this paper, we focused more specifically on the detection of generalized synchronization in coupled chaotic systems, a phenomenon that appears in many biological and physiological processes [51–53], as well as in geophysical fluid dynamics , but it is notoriously difficult to unravel. Additionally, the detection of synchronization phenomena permits the identification of causal drivers and leads to a better description and prediction of system dynamics. The key role that causal influence among observables has for the forecasting of their time course has been addressed in many studies related, for example, to ecological , financial , and multiscale human mobility systems [57, 58]. Our method paves the way for applications to more complex dynamics exhibiting phenomena that usually require multiple complexity measures to be detected, showing that lossless compression of system’s dynamics in the phase space can be suitably used for this purpose.
The data that support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
E. Ott, T. D. Sauer, and J. A. Yorke, Coping with Chaos: Analysis of Chaotic Data and the Exploitation of Chaotic Systems, John Wiley & Sons, New York, NY, USA, 1994.
G. Sugihara and H. Ye, Science, vol. 353, p. 922, 2016.
F. Takens, Detecting Strange Attractors in Turbulence, Springer, Berlin, Germany, 1981.View at: Publisher Site
O. Melchert and A. K. Hartmann, “Analysis of the phase transition in the two-dimensional Ising ferromagnet using a Lempel‐Ziv string‐parsing scheme and black-box data-compression utilities,” Physical Review E - Statistical, Nonlinear, and Soft Matter Physics, vol. 91, p. 1, 2015.View at: Google Scholar
N. Rulkov and D. I. Abarbanel, Physical Review, vol. 51, 1995.View at: Publisher Site
S. Boccaletti, A. N. Pisarchik, C. I. del Genio, and A. Amann, Synchronization: From Coupled Systems to Complex Networks, Cambridge University Press, Cambridge, UK, 2018.
D. P. Feldman, C. S. Mctague, J. P. Crutchfield, D. P. Feldman, C. S. Mctague, and J. P. Crutchfield, “The organization of intrinsic computation: Complexity-entropy diagrams and the diversity of natural information processing,” Chaos: An Interdisciplinary Journal of Nonlinear Science, vol. 18, no. 4, Article ID 043106, 2008.View at: Publisher Site | Google Scholar