Detecting the direction of causal time series

ABSTRACT

We propose a method that detects the true direction of time series, by fitting an autoregressive moving average model to the data. Whenever the noise is independent of the previous samples for one ordering of the observations, but dependent for the opposite ordering, we infer the former direction to be the true one. We prove that our method works in the population case as long as the noise of the process is not normally distributed (for the latter case, the direction is not identifiable). A new and important implication of our result is that it confirms a fundamental conjecture in causal reasoning --- if after regression the noise is independent of signal for one direction and dependent for the other, then the former represents the true causal direction --- in the case of time series. We test our approach on two types of data: simulated data sets conforming to our modeling assumptions, and real world EEG time series. Our method makes a decision for a significant fraction of both data sets, and these decisions are mostly correct. For real world data, our approach outperforms alternative solutions to the problem of time direction recovery.

References

1. Balian, R. (1992). From microphysics to macrophysics. Springer.
2. Breidt, F. J., & Davis, R. A. (1991). Time-reversibility, identifiability and independence of innovations for stationary time series. Journal of Time Series Analysis, 13(5), 379--390.
3. Brockwell, P. J., & Davis, R. A. (1991). Time Series: Theory and Methods. Springer, 2 edn.
4. Darmois, G. (1953). Analyse générale des liaisons stochastiques. Rev. Inst. Internationale Statist., 21, 2--8.
5. Diks, C., van Houwelingen, J., Takens, F., & DeGoede, J. (1995). Reversibility as a criterion for discriminating time series. Physics Letters A, 15, 221--228.
6. EEGdata (2008). This data set (experiment 4a, subject 3) can be downloaded after registration. Website, 3.6.2008, 3:51pm. http://ida.first.fraunhofer.de/projects/bci/competition_iii/#datasets.
7. Gretton, A., Bousquet, O., Smola, A., & Schöölkopf, B. (2005). Measuring statistical dependence with Hilbert-Schmidt norms. In ALT, 63--78. Springer-Verlag.
8. Gretton, A., Fukumizu, K., Teo, C.-H., Song, L., Schölkopf, B., & Smola, A. (2008). A kernel statistical test of independence. In Advances in Neural Information Processing Systems 20, 585--592. Cambridge, MA: MIT Press.
9. Hawking, S. (1995). A brief history of time. Bantam.
10. Hoyer, P. O., Janzing, D., Mooij, J., Peters, J., & Schöölkopf, B. (2009). Nonlinear causal discovery with additive noise models. Proceedings of the Twenty-Second Annual Conference on Neural Information Processing Systems (NIPS).
11. Janzing, D. (2007). On causally asymmetric versions of Occam's Razor and their relation to thermodynamics. http://arxiv.org/abs/0708.3411.
12. Jarque, C. M., & Bera, A. K. (1987). A test for normality of observations and regression residuals. International Statistical Review, 55 (2), 163--172.
13. Kankainen, A. (1995). Consistent testing of total independence based on the empirical characteristic function. PhD Thesis, University of Jyväskylä.
14. Mamai, L. V. (1963). On the theory of characteristic functions. Select. Transl. Math. Stat. Probab., 4, 153--170.
15. Mandelbrot, B. (1967). On the distribution of stock price differences. Operations Research, 15 (6), 1057--1062.
16. Pearlmutter, B. A., & Parra, L. C. (1997). Maximum likelihood blind source separation: A context-sensitive generalization of ica. In Advances in Neural Information Processing Systems 9, 613--619. MIT Press.
17. Peters, J., Janzing, D., Gretton, A., & Schöölkopf, B. (2009). Kernel methods for detecting the direction of time series. In: Proccedings of the 32nd Annual Conference of the German Classification Society (GfKl 2008), 1--10.
18. Reichenbach, H. (1999). The direction of time. Dover.
19. RProject (2009). The r project for statistical computing. Website, 15.1.2009, 1:07pm. http://www.r-project.org/.
20. Shimizu, S., Hoyer, P. O., Hyvärinen, A., & Kerminen, A. (2006). A linear non-gaussian acyclic model for causal discovery. Journal of Machine Learning Research, 7, 2003--2030.
21. Skitovic, V. P. (1962). Linear combinations of independent random variables and the normal distribution law. Select. Transl. Math. Stat. Probab., 2, 211--228.
22. Smola, A. J., Gretton, A., Song, L., & Schöölkopf, B. (2007). A Hilbert space embedding for distributions. In Algorithmic Learning Theory: 18th International Conference (ALT), 13--31. Springer-Verlag.
23. Weiss, G. (1975). Time-reversibility of linear stochastic processes. J. Appl. Prob., 12, 831--836.

Index Terms

• Computing methodologies

  • Machine learning

    • Machine learning approaches

      • Factorization methods

        • Canonical correlation analysis

  • Modeling and simulation

    • Model development and analysis

      • Model verification and validation

      • Modeling methodologies

    • Simulation evaluation

• Mathematics of computing

  • Probability and statistics

    • Statistical paradigms

      • Regression analysis

      • Time series analysis