Skip to main content
SpringerLink
Log in

Climate modeling with neural advection–diffusion equation

  • Regular Paper
  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

Owing to the remarkable development of deep learning technology, there have been a series of efforts to build deep learning-based climate models. Whereas most of them utilize recurrent neural networks and/or graph neural networks, we design a novel climate model based on two concepts, the neural ordinary differential equation (NODE) and the advection–diffusion equation. The advection–diffusion equation is widely used for climate modeling because it describes many physical processes involving Brownian and bulk motions in climate systems. On the other hand, NODEs are to learn a latent governing equation of ODE from data. In our presented method, we combine them into a single framework and propose a concept, called neural advection–diffusion equation (NADE). Our NADE, equipped with the advection–diffusion equation and one more additional neural network to model inherent uncertainty, can learn an appropriate latent governing equation that best describes a given climate dataset. In our experiments with three real-world and two synthetic datasets and fourteen baselines, our method consistently outperforms existing baselines by non-trivial margins.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. Each research domain has its own preference on graph models. For instance, social networks typically assume scale-free networks.

  2. A well-posed problem means (i) its solution uniquely exists, and (ii) its solution continuously changes as input data changes.

  3. https://www.ncdc.noaa.gov/cdo-web/.

References

  1. Zaytar MA, El Amrani C (2016) Sequence to sequence weather forecasting with long short-term memory recurrent neural networks. Int J Comput Appl 143:7–11

    Google Scholar 

  2. Shi X, Chen Z, Wang H, Yeung D-Y, Wong W-K, Woo W-C (2015) Convolutional lstm network: a machine learning approach for precipitation nowcasting. NeurIPS 28

  3. Shi X, Gao Z, Lausen L, Wang H, Yeung D-Y, Wong W-K, Woo W-C (2017) Deep learning for precipitation nowcasting: a benchmark and a new model. Adv Neural Inf Process Syst 30

  4. Liu Y, Racah E, Correa J, Khosrowshahi A, Lavers D, Kunkel K, Wehner M, Collins W et al (2016) Application of deep convolutional neural networks for detecting extreme weather in climate datasets. arXiv preprint arXiv:1605.01156

  5. Racah E, Beckham C, Maharaj T, Ebrahimi Kahou S, Prabhat M, Pal C (2017)Extremeweather: a large-scale climate dataset for semi-supervised detection, localization, and understanding of extreme weather events. Adv Neural Inf Process Syst 30

  6. Kurth T, Treichler S, Romero J, Mudigonda M, Luehr N, Phillips E, Mahesh A, Matheson M, Deslippe J, Fatica M (2018) Exascale deep learning for climate analytics. In: International conference for high performance computing, networking, storage and analysis. IEEE

  7. Cheng L, Zang H, Ding T, Sun R, Wang M, Wei Z, Sun G (2018) Ensemble recurrent neural network based probabilistic wind speed forecasting approach. Energies 11(8):1958

    Article  Google Scholar 

  8. Cheng W, Shen Y, Zhu Y, Huang L (2018) A neural attention model for urban air quality inference: learning the weights of monitoring stations. In: Proceedings of the AAAI conference on artificial intelligence, vol 32(1)

  9. Hossain M, Rekabdar B, Louis SJ, Dascalu S (2015) Forecasting the weather of nevada: a deep learning approach. In: IJCNN. IEEE

  10. Ren X, Li X, Ren K, Song J, Xu Z, Deng K, Wang X (2021) Deep learning-based weather prediction: a survey. Big Data Res 23:100178

    Article  Google Scholar 

  11. Tekin SF, Karaahmetoglu O, Ilhan F, Balaban I, Kozat SS (2021) Spatio-temporal weather forecasting and attention mechanism on convolutional lstms. arXiv preprint arXiv:2102.00696

  12. Rasp S, Lerch S (2018) Neural networks for postprocessing ensemble weather forecasts. Mon Weather Rev

  13. Scher S (2018) Toward data-driven weather and climate forecasting: approximating a simple general circulation model with deep learning. Geophys Res Lett 45:22

    Article  Google Scholar 

  14. Seo S, Liu Y (2019) Differentiable physics-informed graph networks. arXiv preprint arXiv:1902.02950

  15. Seo S, Meng C, Liu Y (2019) Physics-aware difference graph networks for sparsely-observed dynamics. In: ICLR

  16. Lin Y, Mago N, Gao Y, Li Y, Chiang Y-Y, Shahabi C, Ambite JL (2018) Exploiting spatiotemporal patterns for accurate air quality forecasting using deep learning. In: ACM SIGSPATIAL

  17. Zhang P, Jia Y, Gao J, Song W, Leung H (2018) Short-term rainfall forecasting using multi-layer perceptron. IEEE Trans Big Data 6:93–106

    Article  Google Scholar 

  18. Liu H, Mi X, Li Y (2018) Smart deep learning based wind speed prediction model using wavelet packet decomposition, convolutional neural network and convolutional long short term memory network. Energy Convers Manag 166:120–131

    Article  Google Scholar 

  19. Zhu Q, Chen J, Zhu L, Duan X, Liu Y (2018) Wind speed prediction with spatio-temporal correlation: a deep learning approach. Energies 11:705

    Article  Google Scholar 

  20. Chattopadhyay A, Hassanzadeh P, Pasha S (2020) Predicting clustered weather patterns: a test case for applications of convolutional neural networks to spatio-temporal climate data. Sci Rep 10:1–13

    Article  Google Scholar 

  21. De Bézenac E, Pajot A, Gallinari P (2019) Deep learning for physical processes: incorporating prior scientific knowledge. J Stat Mech Theory Exp 2019:124009

    Article  MathSciNet  MATH  Google Scholar 

  22. Han J, Liu H, Xiong H, Yang J (2022) Semi-supervised air quality forecasting via self-supervised hierarchical graph neural network. IEEE Trans Knowl Data Eng

  23. Wang Y, Song G, Du L, Lu Z (2019) Real-time estimation of the urban air quality with mobile sensor system. ACM Trans Knowl Discov Data (TKDD) 13(5):11–19

    Article  Google Scholar 

  24. Xu Y, Zhu Y, Shen Y, Yu J (2019) Fine-grained air quality inference with remote sensing data and ubiquitous urban data. ACM Trans Knowl Discov Data (TKDD) 13(5):1–27

    Article  Google Scholar 

  25. Tey FJ, Wu T-Y, Chen J-L (2022) Machine learning-based short-term rainfall prediction from sky data. ACM Trans Knowl Discov Data (TKDD) 16(6):1–18

    Article  Google Scholar 

  26. Qiu M, Zhao P, Zhang K, Huang J, Shi X, Wang X, Chu W (2017) A short-term rainfall prediction model using multi-task convolutional neural networks. In: 2017 IEEE international conference on data mining (ICDM). IEEE, pp 395–404

  27. Liu X, Tan P-N, Abraham Z, Luo L, Hatami P (2018) Distribution preserving multi-task regression for spatio-temporal data. In: 2018 IEEE international conference on data mining (ICDM). IEEE, pp 1134–1139

  28. Wilson T, Tan P-N, Luo L (2018) A low rank weighted graph convolutional approach to weather prediction. In: 2018 IEEE international conference on data mining (ICDM). IEEE, pp 627–636

  29. Zhang W, Han L, Sun J, Guo H, Dai J (2019) Application of multi-channel 3d-cube successive convolution network for convective storm nowcasting. In: 2019 IEEE international conference on big data (big data). IEEE, pp 1705–1710

  30. Rolnick D, Donti PL, Kaack LH, Kochanski K, Lacoste A, Sankaran K, Ross AS, Milojevic-Dupont N, Jaques N, Waldman-Brown A (2022) Tackling climate change with machine learning. ACM Comput Surv (CSUR) 55(2):1–96

    Article  Google Scholar 

  31. Banerjee A (2011) Probabilistic graphical models for climate data analysis. In: Proceedings of the 2011 workshop on climate knowledge discovery, pp 3–3

  32. Geng Y-A, Li Q, Lin T, Zhang J, Xu L, Yao W, Zheng D, Lyu W, Huang H (2020) A heterogeneous spatiotemporal network for lightning prediction. In: 2020 IEEE international conference on data mining (ICDM), pp 1034–1039

  33. Chen RTQ, Rubanova Y, Bettencourt J, Duvenaud DK (2018) Neural ordinary differential equations. In: NeurIPS

  34. Greydanus S, Dzamba M, Yosinski J (2019) Hamiltonian neural networks. Adv Neural Inf Process Syst 32

  35. Finzi M, Wang KA, Wilson AG (2020) Simplifying Hamiltonian and Lagrangian neural networks via explicit constraints. Adv Neural Inf Process Syst 33:13880–13889

    Google Scholar 

  36. Cranmer M, Greydanus S, Hoyer S, Battaglia P, Spergel D, Ho S (2020) Lagrangian neural networks. arXiv preprint arXiv:2003.04630

  37. Lutter M, Ritter C, Peters J (2019) Deep Lagrangian networks: using physics as model prior for deep learning. arXiv preprint arXiv:1907.04490

  38. Stocker T (2011) Introduction to climate modelling. Springer, Berlin

    Book  Google Scholar 

  39. Larwa B (2019) Heat transfer model to predict temperature distribution in the ground. Energies 12(1):25

    Article  Google Scholar 

  40. Dormand JR, Prince PJ (1980) A family of embedded Runge–Kutta formulae. J Comput Appl Math 6(1):19–26

    Article  MathSciNet  MATH  Google Scholar 

  41. Zhuang J, Dvornek N, Li X, Tatikonda S, Papademetris X, Duncan J (2020) Adaptive checkpoint adjoint method for gradient estimation in neural ode. In: ICML

  42. Eagleson GK (1973) Brownian motion and diffusion. J R Stat Soc Ser A (Gener) 136(1):105–106

    Article  Google Scholar 

  43. Shikano Y, Wada T, Horikawa J (2014) Discrete-time quantum walk with feed-forward quantum coin. Sci Rep 4:1–7

    Article  Google Scholar 

  44. dos SantosMendes R, Lenzi EK, Malacarne LC, Picoli S, Jauregui M (2017) Random walks associated with nonlinear Fokker–Planck equations. Entropy 19(4):155

    Article  MathSciNet  Google Scholar 

  45. Plastino A, Curado E, Nobre F, Tsallis C (2018) From the nonlinear Fokker–Planck equation to the Vlasov description and back: confined interacting particles with drag. Phys Rev E 97:022120

    Article  MathSciNet  Google Scholar 

  46. Mendes G, Ribeiro M, Mendes R, Lenzi E, Nobre F (2015) Nonlinear Kramers equation associated with nonextensive statistical mechanics. Phys Rev E 91:052106

    Article  MathSciNet  Google Scholar 

  47. Wu F, Souza A, Zhang T, Fifty C, Yu T, Weinberger K (2019) Simplifying graph convolutional networks. In: ICML. PMLR

  48. Wang Y, Wang Y, Yang J, Lin Z (2021) Dissecting the diffusion process in linear graph convolutional networks. Adv Neural Inf Process Syst 34:5758–5769

    Google Scholar 

  49. Choi J, Jeon J, Park N (2021) Lt-ocf: learnable-time ode-based collaborative filtering. In: Proceedings of the 30th ACM international conference on information and knowledge management, pp 251–260

  50. Chamberlain BP, Rowbottom J, Goronova M, Webb S, Rossi E, Bronstein MM (2021) Grand: graph neural diffusion. In: ICML

  51. Choi J, Hong S, Park N, Cho S-B (2022) Gread: graph neural reaction–diffusion equations. arXiv preprint arXiv:2211.14208

  52. DeWan A, Dubois N, Theoharides K, Boshoven J (2010) Understanding the impacts of climate change on fish and wildlife in North Carolina. Defenders of Wildlife, Washington

    Google Scholar 

  53. Hanna SR, Briggs GA, Hosker RP Jr (1982) Handbook on atmospheric diffusion. Technical report, National Oceanic and Atmospheric Administration

  54. Wu Z, Pan S, Long G, Jiang J, Chang X, Zhang C (2020) Connecting the dots: multivariate time series forecasting with graph neural networks. In: Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery and data mining, pp 753–763

  55. Brouwer ED, Simm J, Arany A, Moreau Y (2019) Gru-ode-bayes: continuous modeling of sporadically-observed time series. In: NeurIPS

  56. Kidger P, Morrill J, Foster J, Lyons T (2020) Neural controlled differential equations for irregular time series. Adv Neural Inf Process Syst 33:6696–6707

    Google Scholar 

  57. Lyons T, Caruana M, Lévy T (2004) Differential equations driven by rough paths (2004) École D’Eté de Probabilités de Saint-Flour XXXIV

  58. Gozolchiani A, Havlin S, Yamasaki K (2011) Emergence of El Niño as an autonomous component in the climate network. Phys Rev Lett 107(14):148501

    Article  Google Scholar 

  59. Yang X-S (2001) Small-world networks in geophysics. Geophys Res Lett 28(13):2549–2552

    Article  Google Scholar 

  60. Humphries MD, Gurney K, Prescott TJ (2006) The brainstem reticular formation is a small-world, not scale-free, network. Proc R Soc B Biol Sci 273:503–511

    Article  Google Scholar 

  61. Humphries MD, Gurney K (2008) Network ‘small-world-ness’: a quantitative method for determining canonical network equivalence. PLoS ONE 3:e0002051

    Article  Google Scholar 

  62. Watts DJ, Strogatz SH (1998) Collective dynamics of ‘small-world’networks. Nature 393(6684):440–442

    Article  MATH  Google Scholar 

  63. Dormand JR (1996) Numerical methods for differential equations: a computational approach. CRC Press, Cambridge

    MATH  Google Scholar 

  64. Zang C, Wang F (2020) Neural dynamics on complex networks. In: KDD

  65. Raissi M, Perdikaris P, Karniadakis GE (2019) Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J Comput Phys 378:686–707

    Article  MathSciNet  MATH  Google Scholar 

  66. Sanchez-Gonzalez A, Heess N, Springenberg JT, Merel J, Riedmiller M, Hadsell R, Battaglia P (2018) Graph networks as learnable physics engines for inference and control. In: ICML. PMLR

  67. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: CVPR

  68. Hwang J, Choi J, Choi H, Lee K, Lee D, Park N (2021) Climate modeling with neural diffusion equations. In: 2021 IEEE international conference on data mining (ICDM). IEEE, pp 230–239. https://doi.org/10.1109/ICDM51629.2021.00033

  69. Bai L, Yao L, Li C, Wang X, Wang C (2020) Adaptive graph convolutional recurrent network for traffic forecasting. Adv Neural Inf Process Syst 33:17804–17815

    Google Scholar 

  70. Choi J, Choi H, Hwang J, Park N (2022) Graph neural controlled differential equations for traffic forecasting. Proc AAAI Conf Artif Intell 36(6):6367–6374

    Google Scholar 

  71. Chen RTQ, Duvenaud DK (2019) Neural networks with cheap differential operators. In: NeurIPS

  72. Kong L, Sun J, Zhang C (2020) SDE-net: equipping deep neural networks with uncertainty estimates. In: ICML

Download references

Acknowledgements

Noseong Park is the corresponding author. This work was supported by the Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korean government (MSIT) (No. 2020-0-01361, Artificial Intelligence Graduate School Program at Yonsei University, 10%), and (No. 2022-0-00857, Development of AI/data-based financial/economic digital twin platform, 45%) and (No. 2022-0-00113, Developing a Sustainable Collaborative Multi-modal Lifelong Learning Framework, 45%).

Author information

Authors and Affiliations

  1. Yonsei University, Seoul, South Korea

    Hwangyong Choi, Jeongwhan Choi & Noseong Park

  2. University of California, Los Angeles, CA, USA

    Jeehyun Hwang

  3. Arizona State University, Tempe, AZ, USA

    Kookjin Lee

  4. Texas A &M University–Commerce, Commerce, TX, USA

    Dongeun Lee

Authors
  1. Hwangyong Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jeongwhan Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jeehyun Hwang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kookjin Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dongeun Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Noseong Park
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hwangyong Choi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Choi, H., Choi, J., Hwang, J. et al. Climate modeling with neural advection–diffusion equation. Knowl Inf Syst 65, 2403–2427 (2023). https://doi.org/10.1007/s10115-023-01829-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-023-01829-2

Keywords

Search

Navigation