Skip to main content
SpringerLink
Log in

River runoff causal discovery with deep reinforcement learning

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Causal discovery from river runoff data aids flood prevention and mitigation strategies, garnering attention in climate and earth science. However, most climate causal discovery methods rely on conditional independence approaches, overlooking the non-stationary characteristics of river runoff data and leading to poor performance. In this paper, we propose a river runoff causal discovery method based on deep reinforcement learning, called RCD-DRL, to effectively learn causal relationships from non-stationary river runoff time series data. The proposed method utilizes an actor-critic framework, which consists of three main modules: an actor module, a critic module, and a reward module. In detail, RCD-DRL first employs the actor module within the encoder-decoder architecture to learn latent features from raw river runoff data, enabling the model to quickly adapt to non-stationary data distributions and generating a causality matrix at different stations. Subsequently, the critic network with two fully connected layers is designed to estimate the value of the current encoded features. Finally, the reward module, based on the Bayesian information criterion (BIC), is used to calculate the reward corresponding to the currently generated causal matrix. Experimental results obtained on both synthetic and real datasets demonstrate the superior performance of the proposed method over the state-of-the-art methods.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Algorithm 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Data Availability

The real river runoff data can download from the Bavarian Environmental Agency (Bayerisches Landesamt für Umwelt, https://www.lfu.bayern.de) at https://www.gkd.bayern.de/en/rivers/discharge.

Notes

  1. http://www.gkd.bayern.de

References

  1. Wang C, Liu L, Sun S, Wang W (2022) Rethinking the framework constructed by counterfactual functional model. Appl Intell 52(11):12957–12974

    Article  Google Scholar 

  2. Cai R, Huang L, Chen W, Qiao J, Hao Z (2023) Learning dynamic causal mechanisms from non-stationary data. Appl Intell 53(5):5437–5448

    Google Scholar 

  3. McGraw MC, Barnes EA (2018) Memory matters: A case for granger causality in climate variability studies. J Clim 31(8):3289–3300

    Article  Google Scholar 

  4. Runge J, Tibau XA, Bruhns M, Muñoz-Marí J, Camps-Valls G (2020) The causality for climate competition. In: NeurIPS 2019 Competition and Demonstration Track, pp. 110–120. PMLR

  5. Nowack P, Runge J, Eyring V, Haigh JD (2020) Causal networks for climate model evaluation and constrained projections. Nat Commun 11(1):1415

    Article  Google Scholar 

  6. Peng Z, Liu W, An S (2020) Haze pollution causality mining and prediction based on multi-dimensional time series with ps-fcm. Inf Sci 523:307–317

    Article  MathSciNet  Google Scholar 

  7. Zhou W, Yu S, Chen B (2022) Causality detection with matrix-based transfer entropy. Inf Sci 613:357–375

    Article  Google Scholar 

  8. Böhnisch AF, Felsche E, Ludwig R (2023) European heatwave tracks: Using causal discovery to detect recurring pathways in a single-regional climate model large ensemble. Environmental Research Letters

  9. Qu Y, Montzka C, Vereecken H (2021) Causation discovery of weather and vegetation condition on global wildfire using the pcmci approach. In: 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS, pp. 8644–8647. IEEE

  10. Shi H, Zhao Y, Liu S, Cai H, Zhou Z (2022) A new perspective on drought propagation: Causality. Geophys Res Lett 49(2):2021–096758

    Article  Google Scholar 

  11. Song C, Simmons BI, Fortin MJ, Gonzalez A (2022) Generalism drives abundance: A computational causal discovery approach. PLoS Comput Biol 18(9):1010302

    Article  Google Scholar 

  12. Runge J (2023) Modern causal inference approaches to investigate biodiversity ecosystem functioning relationships. nature communications 14(1):1917

  13. Lehmann J, Kretschmer M, Schauberger B, Wechsung F (2020) Potential for early forecast of moroccan wheat yields based on climatic drivers. Geophys Res Lett 47(12):2020–087516

    Article  Google Scholar 

  14. Chopra R, Magazzino C, Shah MI, Sharma GD, Rao A, Shahzad U (2022) The role of renewable energy and natural resources for sustainable agriculture in asean countries: do carbon emissions and deforestation affect agriculture productivity? Resour Policy 76

  15. Mhalla L, Chavez-Demoulin V, Dupuis DJ (2020) Causal mechanism of extreme river discharges in the upper danube basin network. J R Stat Soc: Ser C: Appl Stat 69(4):741–764

    Article  MathSciNet  Google Scholar 

  16. Gnecco N, Meinshausen N, Peters J, Engelke S (2021) Causal discovery in heavy tailed models. Ann Stat 49(3):1755–1778

    Article  MathSciNet  Google Scholar 

  17. Sheng Z, Wen S, Feng Zk, Gong J, Shi K, Guo Z, Yang Y, Huang T (2023) A survey on data-driven runoff forecasting models based on neural networks. IEEE Transactions on Emerging Topics in Computational Intelligence 7(4):1083–1097. https://doi.org/10.1109/TETCI.2023.3259434

    Article  Google Scholar 

  18. Liu N, Qian L, Yan D, Hu W, Hong M (2023) A nonlinear dynamical model for monthly runoff forecasting in situations of small samples. Mathematical Geosciences 1–21

  19. Liu Q, Li S, Zhou G, Feng Z, Wang G, Qiu Q (2022) Attribution of nonstationary changes in the annual runoff of the Weihe River using the de-nonstationarity method. Hydrol Res 53(3):407–418. https://doi.org/10.2166/nh.2022.106, https://iwaponline.com/hr/article pdf/53/3/407/1029010/nh0530407.pdf

  20. Ebert-Uphoff I, Deng Y (2012) Causal discovery for climate research using graphical models. J Clim 25(17):5648–5665

    Article  Google Scholar 

  21. Zhou X, Huang W, Zhang N, Hu W, Du S, Song G, Xie K (2015) Probabilistic dynamic causal model for temporal data. In: 2015 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE

  22. Zhao W, Zhang R, Cao H, Tan W (2019) Factor contribution to soil organic and inorganic carbon accumulation in the loess plateau: Structural equation modeling. Geoderma 352:116–125

    Article  Google Scholar 

  23. Wang Y, Liu X, Wang T, Zhang X, Feng Y, Yang G, Zhen W (2021) Relating land-use/land-cover patterns to water quality in watersheds based on the structural equation modeling. CATENA 206

  24. Garland G, Edlinger A, Banerjee S, Degrune F, García-Palacios P, Pescador DS, Herzog C, Romdhane S, Saghai A, Spor A et al (2021) Crop cover is more important than rotational diversity for soil multifunctionality and cereal yields in european cropping systems. Nature Food 2(1):28–37

    Article  Google Scholar 

  25. Runge J, Bathiany S, Bollt E, Camps-Valls G, Coumou D, Deyle E, Glymour C, Kretschmer M, Mahecha MD, Muñoz-Marí J et al (2019) Inferring causation from time series in earth system sciences. Nat Commun 10(1):2553

    Article  Google Scholar 

  26. Runge J, Nowack P, Kretschmer M, Flaxman S, Sejdinovic D (2019) Detecting and quantifying causal associations in large nonlinear time series datasets. Sci Adv 5(11):4996

    Article  Google Scholar 

  27. Harries D, O’Kane TJ (2021) Dynamic bayesian networks for evaluation of granger causal relationships in climate reanalyses. Journal of Advances in Modeling Earth Systems 13(5):2020–002442

    Article  Google Scholar 

  28. Silva FN, Vega-Oliveros DA, Yan X, Flammini A, Menczer F, Radicchi F, Kravitz B, Fortunato S (2021) Detecting climate teleconnections with granger causality. Geophys Res Lett 48(18):2021–094707

    Article  Google Scholar 

  29. Kumar P, Kuttippurath J, Mitra A (2022) Causal discovery of drivers of surface ozone variability in antarctica using a deep learning algorithm. Environmental science: Processes & impacts 24(3):447–459

    Google Scholar 

  30. Ye Z, Tozuka T (2022) Causal relationship between sea surface temperature and precipitation revealed by information flow. Frontiers in Climate 4:1024384

  31. Runge J (2020) Discovering contemporaneous and lagged causal relations in autocor-related nonlinear time series datasets. In: Conference on Uncertainty in Artificial Intelligence, pp. 1388–1397. PMLR

  32. Gerhardus A, Runge J (2020) High-recall causal discovery for autocorrelated time series with latent confounders. Adv Neural Inf Process Syst 33:12615–12625

    Google Scholar 

  33. Lu NY, Zhang K, Yuan C (2021) Improving causal discovery by optimal bayesian network learning. Proceedings of the AAAI Conference on Artificial Intelligence. 35:8741–8748

    Article  Google Scholar 

  34. Zhu S, Ng I, Chen Z (2020) Causal discovery with reinforcement learning. In: International Conference on Learning Representations (2020). https://openreview.net/forum?id=S1g2skStPB

  35. Wang X, Du Y, Zhu S, Ke L, Chen Z, Hao J, Wang J (2021) Ordering-based causal discovery with reinforcement learning. In: IJCAI, pp. 3566–3573. https://doi.org/10.24963/ijcai.2021/491

  36. Yang D, Yu G, Wang J, Wu Z, Guo M (2023) Reinforcement causal structure learning on order graph. Proceedings of the AAAI Conference on Artificial Intelligence. 37:10737–10744

    Article  Google Scholar 

  37. Liu Y, Wu H, Wang J, Long M (2022) Non-stationary transformers: Exploring the stationarity in time series forecasting. In: Advances in Neural Information Processing Systems

  38. Zheng X, Aragam B, Ravikumar PK, Xing EP (2018) Dags with no tears: Continuous optimization for structure learning. Advances in neural information processing systems 31

  39. Asadi P, Davison AC, Engelke S (2015) Extremes on river networks. The Annals of Applied Statistics, 2023–2050

  40. Engelke S, Hitz AS (2020) Graphical models for extremes. J R Stat Soc Ser B Stat Methodol 82(4):871–932

    Article  MathSciNet  Google Scholar 

  41. Papagiannopoulou C, Miralles DG, Decubber S, Demuzere M, Verhoest NE, Dorigo WA, Waegeman W (2017) A non-linear granger-causality framework to investigate climate-vegetation dynamics. Geoscientific Model Development 10(5):1945–1960

    Article  Google Scholar 

  42. Weichwald S, Jakobsen ME, Mogensen PB, Petersen L, Thams N, Varando G (2020) Causal structure learning from time series: Large regression coefficients may predict causal links better in practice than small p-values. In: NeurIPS 2019 Competition and Demonstration Track, pp. 27–36. PML

  43. Bussmann B, Nys J, Latré S (2021) Neural additive vector autoregression models for causal discovery in time series. In: Discovery Science: 24th International Conference, DS 2021, Halifax, NS, Canada, October 11–13, 2021, Proceedings 24:446–460. Springer

  44. Sanchez P, Liu X, O’Neil, AQ, Tsaftaris SA (2023) Diffusion models for causal discovery via topological ordering. In: The Eleventh International Conference on Learning Representations. https://openreview.net/forum?id=Idusfje4-Wq

Download references

Acknowledgements

This work was partly supported by National Natural Science Foundation of China Research Program (62276010, 62106009), in part by R &D Program of Beijing Municipal Education Commission (KZ202210005009, KM202210005030).

Author information

Authors and Affiliations

  1. The Beijing Municipal Key Laboratory of Multimedia and Intelligent Software Technology, Beijing Institute of Artificial Intelligence, Faculty of information Technology, Beijing University of Technology, Beijing, China

    Junzhong Ji, Ting Wang & Jinduo Liu

  2. Public Meteorological Service Center of China Meteorological Administration, Beijing, China

    Muhua Wang & Wei Tang

Authors
  1. Junzhong Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ting Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jinduo Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Muhua Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wei Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Junzhong Ji led and designed the study. Ting Wang drafted and revised the article with guidance from Jinduo Liu. Muhua Wang and Wei Tang provided guidance for this study. All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Jinduo Liu.

Ethics declarations

Conflicts of interest

The authors declare that they have no conflict of interest.

Ethical and informed consent for data used

This study did not involve human or animal subjects. The data used in this paper were obtained from publicly available sources and comply with relevant laws and regulations. The references are clearly indicated.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ji, J., Wang, T., Liu, J. et al. River runoff causal discovery with deep reinforcement learning. Appl Intell 54, 3547–3565 (2024). https://doi.org/10.1007/s10489-024-05348-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-024-05348-7

Keywords

Search

Navigation