Research Article | Open Access
Valeria d’Andrea, Manlio De Domenico, "Compressing Phase Space Detects State Changes in Nonlinear Dynamical Systems", Complexity, vol. 2020, Article ID 8650742, 6 pages, 2020. https://doi.org/10.1155/2020/8650742
Compressing Phase Space Detects State Changes in Nonlinear Dynamical Systems
Equations governing the nonlinear dynamics of complex systems are usually unknown, and indirect methods are used to reconstruct their manifolds. In turn, they depend on embedding parameters requiring other methods and long temporal sequences to be accurate. In this paper, we show that an optimal reconstruction can be achieved by lossless compression of system’s time course, providing a self-consistent analysis of its dynamics and a measure of its complexity, even for short sequences. Our measure of complexity detects system’s state changes such as weak synchronization phenomena, characterizing many systems, in one step, integrating results from Lyapunov and fractal analysis.
Dynamics of natural systems is often described by nonlinear equations. When those equations are unknown, we can reproduce the system dynamics through the reconstruction of the manifold from the time course of one of its variables [1–3]. Phase space reconstruction has been widely applied in modeling and predictions of several nonlinear systems, such as ecological, climate, and neural ones [4–7]. The embedding theory proposed by Takens  allows one to reconstruct a one-to-one map of the attractor of a dynamical process using time-lagged values of a single system variable. The delay-coordinate map is built from the time series by vectors in of the form , where is the time delay. To correctly build the embedding of -dimensional manifold , it is crucial to choose adequate values for and , i.e., the embedding parameters.
According to the Whitney theorem, the diffeomorphism on is ensured by choosing an embedding dimension  and the result may be generalized also to noninteger (fractal) dimension . Whitney theorem has been relaxed, for example, in [11, 12], but still those studies provide an upper bound for the estimation of . Several methods were developed to estimate the minimum possible embedding dimension , and usually those methods are based on the fact that when evaluating some quantities on a delay-coordinate map, they do not vary for higher than the proper embedding dimension. Those diffeomorphism invariants could be, for example, the largest Lyapunov exponent or the percentage of false nearest neighbours [14, 15], where the latter option, in its implementation introduced by Cao , is currently the most used method to estimate the minimum .
To estimate the embedding dimension, methods that involve the fact that entropies are diffeomorphism invariants have been proposed and include, for example, differential entropy  and permutation entropy , where the latter has the advantage to take into account the temporal information contained in the time series . Kolmogorov complexity, also known as algorithmic entropy, was proposed in 1968 as a measure of the amount of information of the trajectory of a dynamical process  and is defined as the length of the shortest description that produces the trajectory as output. Even if Kolmogorov complexity cannot be computed, for the trajectories of a dynamical system, it is usually approximated using lossless compression algorithms, following the theorem of Brudno who, in 1978, wrote the equality between Kolmogorov complexity and entropy rate . Nevertheless, to date, estimating embedding dimension is still far from being an easy task, although this parameter is critical to gain insights about the physics of the underlying dynamical system.
In this paper, we show that optimal embedding dimension can be estimated through a measure of the Kolmogorov complexity, which is here evaluated using the compression algorithm introduced by Lempel and Ziv . Our dimension estimate could represent a more robust measure than other information estimators because it is independent on the system representation , so it may be estimated without prior knowledge of the value of optimal time delay . The main advantage of our approach is that we explore the geometry of the manifold of the dynamical system with complexity measures that capture the rich information about the underlying dynamics and reveal change in the system state that is otherwise difficult to detect [25–27]. In particular, here we show that exploring how the system approaches its proper embedding dimension can reveal the emergence of chaotic synchronization phenomena in a coupled drive-response system.
2. Low-Dimensional Chaotic Systems
To estimate the optimal embedding dimension , we built , an ensemble of delay-coordinate maps from as a function of time delay and . Then, at fixed and , we discretize values of delay coordinate map by using a grid with bin size : . We computed the entropy rate of the resulting sequence of symbols through a Lempel–Ziv data compression algorithm :where is the shortest subsequence starting at index that does not appear in the window of length . We evaluated entropy rate for the entire ensemble of delay-coordinate maps and estimated as the optimal embedding dimension the one such that
This means that the optimum embedding dimension is the one at which entropy rate has at least a component that behaves as a nonlinear function of , that is, . That choice was suggested by the fact  that the system with causal interactions among their elements has entropy that grows as a nonextensive function of their size , where the nonextensive component is described by a power law function .
To estimate the optimal dimension for the embedding, avoiding the evaluation of the optimal time delay , we tested our algorithm with a specific set of values and found robust results with respect to the choice of this parameter. Figure 1(a) shows an example for a single realization of a Lorenz signal, where estimation of optimal does not change for different values. Figure 1(b) shows the results of our algorithm for a set of chaotic systems. Specifically, we consider Logistic, Hénon and Ikeda maps, Rossler, Lorenz, and Mackey–Glass systems with three different time delays, widely used to model the dynamics of several natural phenomena, from chemical reactions to climate. For each system, we computed our measures across 50 different realizations and compared our estimates with correlation dimension () measures [30–33]. We found that for most of the tested systems, our dimension estimate is close to Whitney’s upper bound , while for Mackey–Glass systems that we tested at three different time delays, we found that our measures are close to the lower bound delimited by .
3. Unidirectionally Coupled Systems
In coupled chaotic systems with a drive-response configuration, generalized synchronization (GS) may occur if the state of response system does not depend on its initial condition but depend only on the state of the driver , that is, if there is a functional relation between trajectories in the phase space, . When is the identity, there is identical synchronization, which is easy to detect because the synchronized motion becomes simply a sharp line in vs plane . Otherwise, when differs from the identity, weak GS may emerge, and this phenomenon is difficult to detect. Different methods to detect GS have been proposed .
For instance, it has been proven that synchronization occurs when all of the conditional Lyapunov exponents are negative , while it is possible to gain insight into the kind of synchronization that is acting by considering the dimension of the global synchronization manifold with respect to the dimension of the driver system : if , then the response system does not have an effect on the global dimension and there is identical synchronization. Otherwise, if , the global manifold has a fractal structure and the synchronization is weak . To reveal weak GS in a coupled system, two different classes of measures are needed, namely, conditional Lyapunov exponents and dimension(s) of the global manifold.
Here, we show that the analysis of the dimension of the response system through lossless complexity measures can easily detect the emergence of GS. To this aim, we studied synchronization phenomena between two unidirectional chaotic systems, where GS takes place as a function of coupling factor . We studied the optimal dimension of the systems assuming, as we did for noncoupled systems, that entropy is well described by a nonextensive function of number of elements . That assumption is especially well posed when the system is weakly sensitive to initial conditions, where it was proven [38, 39] that the usual Shannon entropy measures are not appropriate and a new measure of entropy has to be introduced that depends on sensitivity to initial conditions and the multifractal spectrum.
3.1. Heterogeneous Systems
As a first example, we considered a unidirectionally coupled system in which the autonomous driver is a Rossler oscillator:and the driven one, , is a Lorenz oscillator:
This type of system was investigated in previous works [40–42]. In Figure 2(a), we show that, similarly to previous studies, GS arises for a threshold coupling strength , where the conditional Lyapunov exponent becomes negative. We computed Lyapunov exponents using the pull-back method [43, 44] which relies on the Gram–Schmidt orthonormalization of Lyapunov vectors while integrating the dynamical system with a fourth-order Runge–Kutta algorithm (integration time step ). We computed exponents with 5000 time points, after discarding the first 10000 iterations. The correlation dimension is estimated by using 25000 time points and looking for the plateau in the function , indicating a suitable scaling relationship. As shown in Figure 2(b), of the global manifold is higher than of the driver Rossler system, indicating that at the threshold , the whole system undergoes a regime of weak synchronization.
For each coupling value, we estimated the optimal embedding dimension as the average across 50 realizations of the system dynamics. Time series with 1000 time points were used for the estimation. When approaching the synchronization threshold , increases abruptly and assumes values between the two extremes of two independent Lorenz and Rossler systems. Furthermore, it is worth noting that the trend of estimates is opposite to the trends of both and conditional Lyapunov exponents, suggesting that those measures are referring to different but complementary properties of the dynamical system. Previous studies investigated how measures of entropy and complexity are both needed to describe natural systems, since they capture different properties of the dynamics [46, 47]. In particular, Lyapunov exponents and fractal dimension measures were usually related to the degree of randomness and disorder of the dynamics, while our hypothesis is that , which is the dimension at which the entropy rate is described by a nonlinear function, is related to the length of the patterns, i.e., to regularities in the dynamics that allow for its compression.
3.2. Identical Systems
A second example we considered is the unidirectionally coupled system formed by two identical Hénon maps , where the driver is described by the system:and the driven one is described by
We computed Lyapunov exponents using the pull-back method with 5000 time points and we found that the conditional exponent takes negative values in two different intervals of couplings: in a window and then for (see Figure 3(a)). As shown in Figure 3(b), in the first window , the correlation dimension of the global manifold is higher than the correlation dimension of an independent Hénon system: , indicating that the synchronization is weak in this interval. Furthermore, for coupling values higher than , we have that , showing that for high couplings, identical synchronization takes place. Both the coupling strength intervals and the two different regimes for GS are revealed with a single embedding measure. Here, we computed for each coupling value the optimal embedding dimension as the average across 50 realizations, 1000 time points each, using lossless compression of the dynamics. We found that in interval, the complexity of the coupled system increases, giving rise to an increased estimated of the global manifold. For , the optimal has a drop, showing that there is a change in the system state; in particular, estimates take values typical of an independent Hénon map, revealing an identical synchronization.
In conclusion, we have shown that complexity measures used to reconstruct the geometry of the manifold of a dynamical system can be used to gain many insights about the system itself, even when the underlying governing equations are not known. We observed how the irregularity of the dynamics, expressed by entropy rate estimates, reaches a plateau and remains constant by increasing the dimension of the manifold, providing a robust and parameter-free estimate of the intrinsic optimal dimension. Our measure is quite stable for different values of time delay , providing a desirable method for the reconstruction of the manifold that relies only on a single estimate.
We choose to relate complexity of the system to the way at which entropy rate measures depart from extensive functions and become nonlinear functions of the number of system dimensions. How to properly evaluate complexity has been a debated topic in last years. One of the most debated issue is the fact that information theoretic estimates like Shannon entropy measure the degree of randomness of the system and do not take into account system’s dynamical organization, whereas ideal complexity measures should treat both random and lower distributions as minimally complex . In our approach, we focused on the entropy component that deviates from extensivity, arguing that it contains the information that has to be related to effective system’s complexity.
To detect synchronization, usually quantities related to the randomness of the dynamics [49, 50], such as Lyapunov exponents and fractal dimension, are investigated. However, to be estimated in a reliable way, those quantities require long time series, in particular to compute correlation dimension, which is also potentially biased by user’s choices about proper scale and dimensions. Our method, on the contrary, gives robust results for shorter time series and has the advantage to capture and distinguish, with a single measure, different synchronization regimes. Furthermore, the dimension at which the time series reaches its maximum disorder is informative and gives us insights about the intrinsic structure of the system. The way in which the optimal embedding dimension varies as a function of the parameters ruling the system dynamics highlights state changes, as long as they affect regularities in dynamical patterns. In this paper, we focused more specifically on the detection of generalized synchronization in coupled chaotic systems, a phenomenon that appears in many biological and physiological processes [51–53], as well as in geophysical fluid dynamics , but it is notoriously difficult to unravel. Additionally, the detection of synchronization phenomena permits the identification of causal drivers and leads to a better description and prediction of system dynamics. The key role that causal influence among observables has for the forecasting of their time course has been addressed in many studies related, for example, to ecological , financial , and multiscale human mobility systems [57, 58]. Our method paves the way for applications to more complex dynamics exhibiting phenomena that usually require multiple complexity measures to be detected, showing that lossless compression of system’s dynamics in the phase space can be suitably used for this purpose.
The data that support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
- E. Ott, T. D. Sauer, and J. A. Yorke, Coping with Chaos: Analysis of Chaotic Data and the Exploitation of Chaotic Systems, John Wiley & Sons, New York, NY, USA, 1994.
- N. H. Packard, J. P. Crutchfield, J. D. Farmer, and R. S. Shaw, “Geometry from a time series,” Physical Review Letters, vol. 45, no. 9, pp. 712–716, 1980.
- J. D. Farmer and J. J. Sidorowich, “Predicting chaotic time series,” Physical Review Letters, vol. 59, no. 8, pp. 845–848, 1987.
- M. Casdagli, “Nonlinear prediction of chaotic time series,” Physica D: Nonlinear Phenomena, vol. 35, no. 3, pp. 335–356, 1989.
- M. Chavez, M. Valencia, V. Navarro, V. Latora, and J. Martinerie, “Functional modularity of background activities in normal and epileptic brain networks,” Physical Review Letters, vol. 104, p. 1, 2010.
- G. Sugihara and H. Ye, Science, vol. 353, p. 922, 2016.
- S. L. Brunton, B. W. Brunton, J. L. Proctor, E. Kaiser, and J. Nathan Kutz, “Chaos as an intermittently forced linear system,” Nature Communications, vol. 8, p. 1, 2017.
- F. Takens, Detecting Strange Attractors in Turbulence, Springer, Berlin, Germany, 1981.
- H. Whitney, “Differentiable manifolds,” The Annals of Mathematics, vol. 37, no. 3, p. 645, 1936.
- T. Sauer, J. A. Yorke, and M. Casdagli, “Embedology,” Journal of Statistical Physics, vol. 65, no. 3-4, pp. 579–616, 1991.
- A. Haefliger and M. W. Hirsch, “On the existence and classification of differentiable embeddings,” Topology, vol. 2, no. 1-2, pp. 129–135, 1963.
- F. Fuquan, “Embedding four manifolds in R7,” Topology, vol. 33, no. 3, pp. 447–454, 1994.
- C. J. Cellucci, A. M. Albano, and P. E. Rapp, “Comparative study of embedding methods,” Physical Review E - Statistical Physics, Plasmas, Fluids, and Related Interdisciplinary Topics, vol. 67, p. 13, 2003.
- M. B. Kennel, R. Brown, and D. I. Abarbanel, “Determining embedding dimension for phase-space reconstruction using a geometrical construction,” Physical Review A, vol. 45, 1992.
- H. D. I. Abarbanel and M. B. Kennel, “Local false nearest neighbors and dynamical dimensions from observed chaotic data,” Physical Review E, vol. 47, no. 5, pp. 3057–3068, 1993.
- L. Cao, “Practical method for determining the minimum embedding dimension of a scalar time series,” Physica D: Nonlinear Phenomena, vol. 110, no. 1-2, pp. 43–50, 1997.
- T. Gautama, D. P. Mandic, and M. M. Van Hulle, “On nonlinear modular neural filters,” in Proceedings of the ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing-Proceedings, vol. 6, p. 29, Philadelphia, PA, USA, March 2005.
- M. Riedl, A. Müller, and N. Wessel, “Practical considerations of permutation entropy,” The European Physical Journal Special Topics, vol. 222, no. 2, pp. 249–262, 2013.
- C. Bandt and B. Pompe, “Permutation entropy: a natural complexity measure for time series,” Physical Review Letters, vol. 88, no. 17, 2002.
- A. N. Kolmogorov, “Three approaches to the quantitative definition of information∗,” International Journal of Computer Mathematics, vol. 2, no. 1–4, pp. 157–168, 1968.
- A. A. Brudno, “The complexity of the trajectories of a dynamical system,” Russian Mathematical Surveys, vol. 33, no. 1, pp. 197-198, 1978.
- J. Ziv and A. Lempel, “Compression of individual sequences via variable-rate coding,” IEEE Transactions on Information Theory, vol. 24, no. 5, pp. 530–536, 1978.
- F. Argenti, V. Benci, P. Cerrai, A. Cordelli, S. Galatolo, and G. Menconi, “Information and dynamical systems: a concrete measurement on sporadic dynamics,” Chaos, Solitons & Fractals, vol. 13, no. 3, pp. 461–469, 2002.
- A. M. Fraser and H. L. Swinney, “Independent coordinates for strange attractors from mutual information,” Physical Review A, vol. 33, no. 2, pp. 1134–1140, 1986.
- O. Melchert and A. K. Hartmann, “Analysis of the phase transition in the two-dimensional Ising ferromagnet using a Lempel‐Ziv string‐parsing scheme and black-box data-compression utilities,” Physical Review E - Statistical, Nonlinear, and Soft Matter Physics, vol. 91, p. 1, 2015.
- R. Avinery, M. Kornreich, and R. Beck, “Universal and accessible entropy estimation using a compression algorithm,” Physical Review Letters, vol. 123, no. 17, 2017.
- S. Martiniani, P. M. Chaikin, and D. Levine, “Quantifying hidden order out of equilibrium,” Physical Review X, vol. 9, no. 1, p. 11031, 2019.
- M. Lin, W.-J. Hsu, and Z. Q. Lee, “Predictability of individuals’ mobility with high-resolution positioning data,” in Proceedings of the 2012 ACM Conference on Ubiquitous Computing-UbiComp, p. 381, New York, NY, USA, September 2012.
- W. Bialek, I. Nemenman, and N. Tishby, “Predictability, complexity, and learning,” Neural Computation, vol. 13, no. 11, pp. 2409–2463, 2001.
- P. Grassberger and I. Procaccia, “Characterization of strange attractors,” Physical Review Letters, vol. 50, no. 5, 1983.
- P. Grassberger and I. Procaccia, “Measuring the strangeness of strange attractors,” Physica D: Nonlinear Phenomena, vol. 9, no. 1-2, pp. 189–208, 1983.
- P. Grassberger, “On the fractal dimension of the Henon attractor,” Physics Letters A, vol. 97, no. 6, pp. 224–226, 1983.
- P. Grassberger and I. Procaccia, “Estimation of the Kolmogorov entropy from a chaotic signal,” Physical Review A, vol. 28, no. 4, pp. 2591–2593, 1983.
- N. Rulkov and D. I. Abarbanel, Physical Review, vol. 51, 1995.
- S. Boccaletti, A. N. Pisarchik, C. I. del Genio, and A. Amann, Synchronization: From Coupled Systems to Complex Networks, Cambridge University Press, Cambridge, UK, 2018.
- L. M. Pecora and T. L. Carroll, “Synchronization in chaotic systems,” Physical Review Letters, vol. 64, 1990.
- K. Pyragas, “Conditional Lyapunov exponents from time series,” Physical Review E, vol. 56, no. 5, pp. 5183–5188, 1997.
- V. Latora, M. Baranger, A. Rapisarda, and C. Tsallis, “The rate of entropy increase at the edge of chaos,” Physics Letters A, vol. 273, no. 1-2, pp. 97–103, 2000.
- M. Ponce-Flores, J. Frausto-Solís, G. Santamaría-Bonfil, J. Pérez Ortega, and J. J. González-Barbosa, “Time series complexities and their relationship to forecasting performance,” Entropy, vol. 22, p. 1, 2020.
- K. Pyragas, “Weak and strong synchronization of chaos,” Physical Review E, vol. 54, no. 5, p. 63, 1996.
- R. Q. Quiroga, J. Arnhold, and P. Grassberger, “Learning driver-response relationships from synchronization patterns,” Physical Review E, vol. 61, no. 5, pp. 5142–5148, 2000.
- R. Quian Quiroga, A. Kraskov, T. Kreuz, and P. Grassberger, “Performance of different synchronization measures in real data: A case study on electroencephalographic signals,” Physical Review E, vol. 65, p. 14, 2002.
- G. Benettin, L. Galgani, A. Giorgilli, and J.-M. Strelcyn, “Lyapunov Characteristic Exponents for smooth dynamical systems and for Hamiltonian systems; a method for computing all of them. Part 1: Theory,” Meccanica, vol. 15, no. 1, pp. 9–20, 1980.
- A. Wolf, J. B. Swift, H. L. Swinney, and J. A. Vastano, “Determining Lyapunov exponents from a time series,” Physica D: Nonlinear Phenomena, vol. 16, no. 3, pp. 285–317, 1985.
- R. Hegger, H. Kantz, and T. Schreiber, “Practical implementation of nonlinear time series methods: the TISEAN package,” Chaos: An Interdisciplinary Journal of Nonlinear Science, vol. 9, no. 2, pp. 413–435, 1999.
- J. P. Crutchfield and D. P. Feldman, “Regularities unseen, randomness observed: levels of entropy convergence,” Chaos: An Interdisciplinary Journal of Nonlinear Science, vol. 13, no. 1, pp. 25–54, 2003.
- D. P. Feldman, C. S. Mctague, J. P. Crutchfield, D. P. Feldman, C. S. Mctague, and J. P. Crutchfield, “The organization of intrinsic computation: Complexity-entropy diagrams and the diversity of natural information processing,” Chaos: An Interdisciplinary Journal of Nonlinear Science, vol. 18, no. 4, Article ID 043106, 2008.
- T. Deacon and S. Koutroufinis, “Complexity and dynamical depth,” Information, vol. 5, no. 3, pp. 404–423, 2014.
- H. D. I. Abarbanel, N. F. Rulkov, and M. M. Sushchik, “Generalized synchronization of chaos: the auxiliary system approach,” Physical Review E, vol. 53, no. 5, pp. 4528–4535, 1996.
- S. Boccaletti, J. Kurths, G. Osipov, D. L. Valladares, and C. S. Zhou, “The synchronization of chaotic systems,” Physics Reports, vol. 366, no. 1-2, pp. 1–101, 2002.
- L. Glass, “Synchronization and rhythmic processes in physiology,” Nature, vol. 410, no. 6825, pp. 277–284, 2001.
- C. Chen, S. Liu, X.-q. Shi, H. Chaté, and Y. Wu, “Weak synchronization and large-scale collective oscillation in dense bacterial suspensions,” Nature, vol. 542, no. 7640, pp. 210–214, 2017.
- M. Frasca, A. Buscarino, A. Rizzo, L. Fortuna, and S. Boccaletti, “Synchronization of moving chaotic agents,” Physical Review Letters, vol. 100, no. 4, 2008.
- G. S. Duane and J. J. Tribbia, “Synchronized chaos in geophysical fluid dynamics,” Physical Review Letters, vol. 86, no. 19, pp. 4298–4301, 2001.
- H. Ye, R. J. Beamish, S. M. Glaser et al., “Equation-free mechanistic ecosystem forecasting using empirical dynamic modeling,” Proceedings of the National Academy of Sciences, vol. 112, no. 13, pp. E1569–E1576, 2015.
- R. M. May, S. A. Levin, and G. Sugihara, “Ecology for bankers,” Nature, vol. 451, no. 7181, pp. 893-894, 2008.
- H. Barbosa, M. Barthélemy, G. Ghoshal et al., “Human mobility: models and applications,” Physics Reports, vol. 734, pp. 1–74, 2018.
- M. De Domenico, A. Lima, and M. Musolesi, “Interdependence and predictability of human mobility and social interactions,” Pervasive and Mobile Computing, vol. 9, no. 6, pp. 798–807, 2013.
Copyright © 2020 Valeria d’Andrea and Manlio De Domenico. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.