An algorithm based on the convergent cross mapping method for the detection of causality in uni-directionally connected chaotic systems

In this paper, we present some improvements to the convergent cross mapping (CCM) algorithm for detecting causality in uni-directionally connected chaotic systems. The basic concept of the CCM algorithm is that the causal influence of system X on system Y appears as mapping of the neighbouring states in the reconstructed d -dimensional manifold, M , to the neighbouring states in the reconstructed d-dimensional manifold, M , and this effect is evaluated using the correlation coefficient between the estimated and observed values of M . We proposed a composite indicator of causality as the ratio between the correlation coefficient and the Shannon entropy of the distribution of the residuals between the estimated and observed values of M . Application of the proposed approach to four master-slave Rössler and Lorenz systems and real-world data showed that the new algorithm allowed a slight increase in capability to reveal the presence and direction of couplings.


Introduction
Quantification of the causal effects between simultaneously observed systems from the analysis of time series recordings is essential in many scientific fields, including economics, climatology, ecosystems, electrical activity of the brain, or cardiorespiratory relations.Estimating the interdependence between the observed variables provides valuable knowledge regarding the processes that generate time series [1].The Granger causality method [2] is the most well-known and principal method for identifying directional interactions between variables from their time series, and many modifications and extensions of the Granger causality test has been developed.The Granger test focuses on determining whether one time series is useful in forecasting another.If useful, the first system causally affects the second system.The conventional Granger's causality test is based on autoregressive models and is particularly useful in stochastic linearly interconnected systems [3].However, its appropriateness of direct application to nonlinear systems depends on the specific problem to be analyzed [4][5][6].Therefore, new approaches have been proposed, including nonlinear extensions of Granger's causality [4], transfer entropy [7], conditional mutual information [8] and measures evaluating distances of conditioned neighbors in reconstructed state spaces [9][10][11][12][13], to name a few.In 2012, Sugihara et al. [5] introduced another method based on state space reconstruction.The method was called convergent cross-mapping (CCM) and involves evaluating distances between conditioned neighbours in reconstructed state spaces and tests for causation between the driver ("master") system  and the driven ("slave") system Y, by measuring the extent to which the historical states of reconstructed state space,  , can reliably estimate states of reconstructed state space,  .Similarity between points,  , and estimates,  , is evaluated by assessing their correlation.To infer the direction of coupling, the opposite direction must also be assessed.Then, the direction of coupling is inferred based on asymmetries emerging from the calculations of the two possible causal directions.
In this paper, we propose a simple improvement to the CCM method by introducing a composite parameter for inferring the direction of causation composed of two components: (1) the correlation coefficient between the cross-mapped and observed values of the reconstructed state space, and (2) Shannon entropy of the distribution of the residuals between the cross-mapped and observed values of the reconstructed state space.

Convergent cross mapping
CCM [5,14] is a method for detecting causality between two systems represented by time series.CCM is based on Takens' embedding theorem, exploiting the geometry of attractors of coupled dynamical systems and constructs a map between mutual neighborhoods in state spaces of the coupled dynamical systems under study.
Let  =  , and  =  be two time series of finite length  ∈ ℕ.To establish cross mapping from  to , first we reconstruct the attractor manifold  as a set of -dimensional vectors [15] , where  is the embedding dimension,  is the time delay, and • denotes the transpose of a real matrix.We find the  + 1 nearest neighbors of  in  and denote their time indices (from the closest to the farthest) by  , ⋯ ,  .These indices will be used in the construction of the cross mapping of  for  ,  = 1 +  − 1 , ⋯ ,  by: where  =  ∕ ∑  ,  = exp −  ,  ∕   ,  and  ,  is the Euclidean distance between vectors  and .The cross mapping from Y to X is defined analogously.The skill of cross-map estimates is indicated by the correlation coefficient  between  and  | (or between  and  | , respectively).Sugihara et al. [5] declare in the supplement: "If  and  are dynamically coupled, the nearest neighbors of  should identify the time indices of corresponding nearest neighbors on  .As  increases, the attractor manifold fills in and the distances among the  + 1 nearest neighbors shrinks.Consequently,  | should converge to  and  | should converge to  .In this way, we use convergence of the nearest neighbors to test whether there is a correspondence between states on  and states on  ".Cross-mapping that converges in only one direction is the criterion for unidirectional causality.One of the fundamental concepts of CCM is that when causation is unilateral,  →  ( drives ), then it is possible to estimate  from , but not  from .It follows that a high value of the correlation coefficient between  and  | comparing to the value of the correlation coefficient between  and  | indicates that system  drives system .This runs counter to Granger's intuitive scheme.For more details on the CCM method see [5,14,16,17].

Description of the algorithm
The simulation results obtained from the CCM method applied to uni-directionally coupled non-identical oscillators ( drives ) shows that probability distribution of the residuals taken as the Euclidean distance   ,  | between manifold  =  and manifold  | =  obtained by cross mapping using a manifold  , and the Euclidean distance   ,  | between manifold  =  and manifold  | =  obtained by cross mapping using a manifold  , differs.More specifically, the probability distribution of   ,  | has a taller, narrower shape (Fig. 1).It is therefore reasonable to combine the Shannon entropy of the probability distributions of   ,  | or   ,  | , and the corresponding correlation coefficient to enhance the inference of the direction of instantaneous causality.Furthermore, we compute the relationship between vectors of the manifold  and its cross-mapped estimate  | and between vectors of the manifold  and its cross-mapped estimate  | , by determining the 2D correlation coefficient, and introduce interdependence measures  | and  | as: where  is the 2D correlation coefficient between vectors of the manifold  and its cross-mapped estimate  | ,  is the 2D correlation coefficient between vectors of the manifold  and its cross-mapped estimate  | ,  is the Shannon entropy of the probability distribution of   ,  | , and  is the Shannon entropy of the probability distribution of   ,  | .

Results
To demonstrate the effectiveness of the cross-mapping evaluated by the proposed interdependence measures (), we compared it with the cross-mapping evaluated using the correlation coefficient () on four test examples of uni-directionally connected chaotic Rössler and Lorenz-type systems that were coupled with variable coupling strengths.In the examples used in this paper, the possibility of correlation instead of causation was excluded, and therefore the aspect of convergence was not used.The first and second datasets originated from the coupling of two Rössler systems [17], which were coupled via a one-way driving relationship between variables  of the driving system and variable  of the responsive system: The causal relationship between the two systems was calculated using 7000 time-delayed vectors of  and  with a time delay equal to 3 and embedding dimension of 7. Eight nearest neighbours were used.All data processing and analyses were performed using Matlab software (MathWorks, Natick, MA) by adopting the ideas from reference [18], among others.Consequently, we denoted the direction from  to  as | and the direction from  to  as |.Taking, for example, the measure CM: if  drives , the measure  | is expected to be higher than  | .As we can see from Fig. 2, both CME and CM measures show that  drives  until the onset of synchronization.The effect of the entropy contribution in the | direction is little, but is considerable in the | direction, especially at moderate and high coupling strengths.As a result, the gap between  | and  | becomes larger, compared to the gap between  | and  | , i.e. the direction of causation can be determined more reliable.Furthermore, in the case of the coupled Rössler systems with  = 1.075,  = 1.0, the independent-samples t-test showed that this difference between gaps across all coupling strengths is near to statistical significance ( = 0.0866).The third and fourth datasets originated from the coupling of two Lorenz systems [17], which are coupled via one-way driving relationship between variables  of the driving system and variable  of the responsive system: where  1 = 28,5,  1 = 27.5 and  2 = 39,  2 = 35.The coupling strength  was chosen from 0 to 11.6 with a step size 0:4.The data were generated using the Matlab solver of ordinary differential equations ode45.The first 2000 data points were discarded.The total number of obtained data was 14000 and the causal relationship between the two systems was calculated using 7000 time-delayed vectors of  and  , with a time delay equal to 3 and embedding dimension of 7. As we can see from Fig. 3, the results are similar to the coupled Rössler systems, although the effect of entropy is greater.To demonstrate the performance of the proposed algorithm on real-world data, we applied the algorithm for the quantification of coupling between respiration and heart rate variability (HRV) because is well known that HRV is strongly related to respiration.For this purpose, we used data from the Fantasia database (https://physionet.org/physiobank/database/fantasia/).The data that are free of artifacts were acquired from 12 young subjects.The R-peaks of the electrocardiogram (ECG) signal were detected by means of a threshold method and the RR time series were obtained.In this study, we removed the mean of every RR time series either as of every respiration signal so that they oscillate around zero.The independent-samples t-test showed a statistically significant ( = 0.0097) difference between  | and  | and much more statistically significant ( = 0.000454) difference between  | and  | .

Conclusions
A simulation involving examples of uni-directionally connected chaotic of Rössler and Lorenz-type systems and real-world time series shows that the proposed substitutions in the CCM method reinforce the asymmetries between the calculations of the two possible causal directions and thus improve determination of the presence and direction of coupling, and also detection of the onset of full synchronization in uni-directionally connected chaotic systems.