Abstract
In recent years, there has been an increased interest in understanding and predicting the weather using weather station data with Spatial-Temporal Graph Neural Networks (STGNN). However, it has large prediction errors as a result of the inherent non-linearities and the influence of dynamic spatio-temporal auto-correlation. Using a continuously-varying graph topology chronologically, while embedding domain knowledge to enforce validity, can effectively resolve the issue, but the implementation of such concept constitutes an interdisciplinary challenge for researchers. A Dynamic Graph Former (DGFormer) model is proposed to address this challenge. It combines a topology learner through a deep generative layer with domain knowledge enhancement inserted into the STGNN structure, where the derived physics-guided method allows for an efficient integration with the earth system. For capture of the optimal topology, we merge a node-embedding-based similarity metric learning and the superposition principle as physical assistants into the dynamic graph module. We evaluate our model with a real-world weather dataset on short-term (12 hours) and medium-range (360 hours) prediction tasks. DGFormer achieves outstanding performance with obvious improvements by up to 34.84% at short-term prediction and by up to 23.25% at medium-range prediction compared with the state-of-the-art methods. We also conducted detailed analyses for cities in three regions and visualized the dynamic graph, revealing the characteristics, advantages, and graph visualization of our model.
Similar content being viewed by others
Data Availability
We shared our code and data. Readers can find our model at https://github.com/xzwbsz/DGFormer. The dataset was partly sampled from WeatherBench [60] and partly provided by National Meteorological Center of China.
References
Bauer P, Thorpe A, Brunet G (2015) The quiet revolution of numerical weather prediction. Nature 525(7567):47–55
Lin H, Gao Z, Xu Y, Wu L, Li L, Li SZ (2022) Conditional local convolution for spatio-temporal meteorological forecasting. Proceedings of the AAAI conference on artificial intelligence 36:7470–7478
Pathak J, Subramanian S, Harrington P, Raja S, Chattopadhyay A, Mardani M, Kurth T, Hall D, Li Z, Azizzadenesheli K, et al (2022) Fourcastnet: A global data-driven high-resolution weather model using adaptive fourier neural operators. arXiv:2202.11214
Worley PH, Mirin AA, Craig AP, Taylor MA, Dennis JM, Vertenstein M (2011) Performance of the community earth system model. In: Proceedings of 2011 international conference for high performance computing, networking, storage and analysis, pp 1–11
Weyn JA, Durran DR, Caruana R (2019) Can machines learn to predict weather? using deep learning to predict gridded 500-hpa geopotential height from historical weather data. Journal of Advances in Modeling Earth Systems 11
Hersbach H, Bell B, Berrisford P, Hirahara S, Horányi A, Muñoz-Sabater J, Nicolas J, Peubey C, Radu R, Schepers D et al (2020) The era5 global reanalysis. Q J R Meteorol Soc 146(730):1999–2049
Lam R, Sanchez-Gonzalez A, Willson M, Wirnsberger P, Fortunato M, Pritzel A, Ravuri S, Ewalds T, Alet F, Eaton-Rosen Z et al (2022) Graphcast: Learning skillful medium-range global weather forecasting. arXiv:2212.12794
Kashinath K, Mustafa M, Albert A, Wu J, Jiang C, Esmaeilzadeh S, Azizzadenesheli K, Wang R, Chattopadhyay A, Singh A et al (2021) Physics-informed machine learning: case studies for weather and climate modelling. Phil Trans R Soc A 379(2194):20200093
Zhou Z, Lin G, Yang K, BAI L, Wang Y et al (2022) Greto: Remedying dynamic graph topology-task discordance via target homophily. In: The eleventh international conference on learning representations
Keisler R (2022) Forecasting global weather with graph neural networks. arXiv:2202.07575
Wang R, Kashinath K, Mustafa M, Albert A, Yu R (2020) Towards physics-informed deep learning for turbulent flow prediction. In: Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pp 1457–1466
Cai W, Ng B, Geng T, Wu L, Santoso A, McPhaden MJ (2020) Butterfly effect and a self-modulating el niño response to global warming. Nature 585(7823):68–73
Kitaev N, Kaiser Ł, Levskaya A (2020) Reformer: The efficient transformer. arXiv:2001.04451
Molteni F, Buizza R, Palmer TN, Petroliagis T (1996) The ecmwf ensemble prediction system: Methodology and validation. Q J R Meteorol Soc 122(529):73–119
Lamarque JF, Emmons LK, Hess PG, Kinnison DE, Tilmes S, Vitt F, Heald CL, Holland EA, Lauritzen PH, Neu J (2012) Cam-chem: description and evaluation of interactive atmospheric chemistry in the community earth system model. Geosci Model Dev 5:369–411
Benjamin SG, Brown JM, Brunet G, Lynch P, Saito K, Schlatter TW (2019) 100 years of progress in forecasting and nwp applications. Meteorol Monogr 59:13–1
Hewage P, Trovati M, Pereira E, Behera A (2021) Deep learning-based effective fine-grained weather forecasting model. Pattern Anal Applic 24(1):343–366
Ebert-Uphoff I, Hilburn K (2023) The outlook for AI weather prediction. Nature Publishing Group UK London
Shi X, Chen Z, Wang H, Yeung D-Y, Wong W-K, Woo W-C (2015) Convolutional lstm network: A machine learning approach for precipitation nowcasting. Advances in Neural Information Processing Systems 28
Bi K, Xie L, Zhang H, Chen X, Gu X, Tian Q (2022) Pangu-weather: A 3d high-resolution model for fast and accurate global weather forecast. arXiv:2211.02556
Gao Z, Shi X, Wang H, Zhu Y, Wang YB, Li M, Yeung D-Y (2022) Earthformer: Exploring space-time transformers for earth system forecasting. Adv Neural Inf Process Syst 35:25390–25403
Wu H, Zhou H, Long M, Wang J (2023) Interpretable weather forecasting for worldwide stations with a unified deep model. Nature Machine Intelligence, pp 1–10
Kurth T, Subramanian S, Harrington P, Pathak J, Mardani M, Hall D, Miele A, Kashinath K, Anandkumar A (2022) Fourcastnet: Accelerating global high-resolution weather forecasting using adaptive fourier neural operators. arXiv:2208.05419
Qi Y, Li Q, Karimian H, Liu D (2019) A hybrid model for spatiotemporal forecasting of pm2. 5 based on graph convolutional neural network and long short-term memory. Sci Total Environ 664:1–10
Cachay SR, Erickson E, Bucker AFC, Pokropek E, Potosnak W, Osei S, Lütjens B (2020) Graph neural networks for improved el ni\(\backslash ^{\sim }\) no forecasting. arXiv:2012.01598
Zhou H, Zhang S, Peng J, Zhang S, Li J, Xiong H, Zhang W (2021) Informer: Beyond efficient transformer for long sequence time-series forecasting. In: Proceedings of the AAAI conference on artificial intelligence vol 35, pp 11106–11115
Yu B, Yin H, Zhu Z (2018) Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. In: Proceedings of the 27th international joint conference on artificial intelligence (IJCAI)
Li Y, Yu R, Shahabi C, Liu Y (2018) Diffusion convolutional recurrent neural network: Data-driven traffic forecasting. In: International conference on learning representations (ICLR ’18)
Agarwal A, Caesar L, Marwan N, Maheswaran R, Merz B, Kurths J (2019) Network-based identification and characterization of teleconnections on different scales. Sci Rep 9(1):8808
Li P, Yu Y, Huang D, Wang Z-H, Sharma A (2023) Regional heatwave prediction using graph neural network and weather station data. Geophys Res Lett 50(7):2023–103405
Wilson T, Tan P-N, Luo L (2018) A low rank weighted graph convolutional approach to weather prediction. In: 2018 IEEE international conference on data mining (ICDM). IEEE
Wang R, Maddix D, Faloutsos C, Wang Y, Yu R (2021) Bridging physics-based and data-driven modeling for learning dynamical systems. In: Learning for dynamics and control, pp 385–398. PMLR
Skarding J, Gabrys B, Musial K (2021) Foundations and modeling of dynamic networks using dynamic graph neural networks: A survey. IEEE Access 9:79143–79168
Sankar A, Wu Y, Gou L, Zhang W, Yang H (2020) Dysat: Deep neural representation learning on dynamic graphs via self-attention networks. In: Proceedings of the 13th international conference on web search and data mining, pp 519–527
Kazemi SM, Goel R, Jain K, Kobyzev I, Sethi A, Forsyth P, Poupart P (2020) Representation learning for dynamic graphs: A survey. J Mach Learn Res 21(1):2648–2720
BELYTSCHKO T (1989) The finite element method: linear static and dynamic finite element analysis: Thomas jr hughes. Comput Aided Civ Infrastruct Eng 4(3):245–246
Wang R, Yu R (2021) Physics-guided deep learning for dynamical systems: A survey. arXiv:2107.01272
Khandelwal A, Xu S, Li X, Jia X, Kumar V (2020) Physics guided machine learning methods for hydrology
Yuan K, Zhu Q, Li F, Riley WJ, Torn M, Chu H, McNicol G, Chen M, Knox S, Delwiche K et al (2022) Causality guided machine learning model on wetland ch4 emissions across global wetlands. Agric For Meteorol 324:109115
Reichstein M, Camps-Valls G, Stevens B, Jung M, Denzler J, Carvalhais N (2019) Deep learning and process understanding for data-driven earth system science. Nature 566(7743):195–204
Willard J, Jia X, Xu S, Steinbach M, Kumar V (2020) Integrating physics-based modeling with machine learning: A survey. 1(1):1–34. arXiv:2003.04919
Greenleaf A, Kurylev Y, Lassas M, et al. Cloaked electromagnetic, acoustic, and quantum interference, based on the description in 'Gudder SP (1970) A superposition principle in physics. J Math Phys 11(3):1037–1040
Oroian M, Amariei S, Escriche I, Gutt G (2013) A viscoelastic model for honeys using the time-temperature superposition principle (ttsp). Food Bioproc Tech 6:2251–2260
Helias M, Dahmen D (2020) Statistical field theory for neural networks. Lecture Notes in Physics
Halmos PR, Savage LJ (1949) Application of the radon-nikodym theorem to the theory of sufficient statistics. Ann Math Stat 20(2):225–241
Sutskever I, Vinyals O, Le QV (2014) Sequence to sequence learning with neural networks. Advances in Neural Information Processing Systems 27
Chen Y, Wu L, Zaki MJ (2020) Iterative deep graph learning for graph neural networks: Better and robust node embeddings
Tian FR (1994) The whitham-type equations and linear overdetermined systems of euler-poisson-darboux type. Duke Math J 74(1):203–221
Cattaneo C, Fontana L (2003) D’alembert formula on finite one-dimensional networks. J Math Anal Appl 284(2):403–424
Wazwaz, A-M (2010) Partial Differential Equations and Solitary Waves Theory
Distribution, M (2005) Maxwell-boltzmann distribution. Curve
Cubasch U, Voss R, Hegerl GC, Waszkewitz J, Crowley TJ (1997) Simula935 tion of the influence of solar radiation variations on the global climate with an 936 ocean-atmosphere circulation model. Climate Dynamics 13(11):757–767
Black CA, Macdonald TH (1965) Long-wave radiation
Carpenter I, Archibald R, Evans KJ, Larkin J, Micikevicius P, Norman M, Rosinski J, Schwarzmeier J, Taylor MA (2013) Progress towards accelerating homme on hybrid multi-core systems. Int J High Perform Comput Appl 27(3):335–347
Kopera MA, Giraldo FX (2014) Analysis of adaptive mesh refinement for imex discontinuous galerkin solutions of the compressible euler equations with application to atmospheric simulations. J Comput Phys 275:92–117
Qin R, Duan C (2017) The principle and applications of bernoulli equation. In: Journal of physics: conference series, vol 916, pp 012038. IOP Publishing
Meniko R, Plohr BJ (1989) The riemann problem for fluid flow of real materials. Rev Mod Phys 61(1):75–130
Belkin M, Niyogi P (2001) Laplacian eigenmaps and spectral techniques for embedding and clustering. Adv Neural Inf Process Syst 14(6):585–591
Wang S, Li Y, Zhang J, Meng Q, Meng L, Gao F (2020) Pm2. 5-gnn: A domain knowledge enhanced graph neural network for pm2. 5 forecasting. In: Proceedings of the 28th international conference on advances in geographic information systems, pp 163–166
Rasp S, Dueben PD, Scher S, Weyn JA, Thuerey N (2020) Weatherbench: A benchmark data set for data-driven weather forecasting. Journal of Advances in Modeling Earth Systems 12(11)
Bai L, Yao L, Li C, Wang X, Wang C (2020) Adaptive graph convolutional recurrent network for traffic forecasting. Adv Neural Inf Process Syst 33:17804–17815
Xu D, Ruan C, Korpeoglu E, Kumar S, Achan K (2020) Inductive representation learning on temporal graphs. arXiv:2002.07962
Wu Z, Pan S, Long G, Jiang J, Zhang C (2019) Graph wavenet for deep spatial-temporal graph modeling. CoRR arXiv:1906.00121
Vitart F, Ardilouze C, Bonet A, Brookshaw A, Chen M, Codorean C, Déqué M, Ferranti L, Fucile E, Fuentes M et al (2017) The subseasonal to seasonal (s2s) prediction project database. Bull Am Meteorol Soc 98(1):163–173
Medium-Range Weather Forecasts EC (2023) S2S, ECMWF, Reforecasts, Instantaneous and Accumulated. https://apps.ecmwf.int/datasets/data/s2s-reforecasts-instantaneous-accum-ecmf/
Liang J, Chen K, Xian Z (2021) Assessment of fy-2g atmospheric motion vector data and assimilating impacts on typhoon forecasts. Earth Space Sci 8(6):2020–001628
Runge J, Petoukhov V, Donges JF, Hlinka J, Jajcay N, Vejmelka M, Hartman D, Marwan N, Paluš M, Kurths J (2015) Identifying causal gateways and mediators in complex spatio-temporal systems. Nat Commun 6(1):8502
Li F, Zhu Q, Riley WJ, Yuan K, Wu H, Gui Z (2022) Wetter california projected by cmip6 models with observational constraints under a high ghg emission scenario. Earth’s Future 10(4):2022–002694
Silva FN, Vega-Oliveros DA, Yan X, Flammini A, Menczer F, Radicchi F, Kravitz B, Fortunato S (2021) Detecting climate teleconnections with granger causality. Geophys Res Lett 48(18):2021–094707
Funding
This work was supported by the Natural Science Foundation of Jilin Province (Grant 20230101062JC), the National Key Research and Development Plan of China (Grant 2017YFC1502306), and the National Natural Science Foundation of China (No. 42175052, No. 61902143, No. 62272190, and No. U19A2061).
Author information
Authors and Affiliations
Contributions
Z. Xu. (First Author) : Conceptualization, Methodology, Software, Investigation, Formal Analysis, Writing - Original Draft; X. Wei: Resources, Supervision; J. Hao: Data Curation, Software, Validation; J. Han: Data Curation, Software; H. Li (Corresponding Author): Resources, Supervisio, Visualization, Writing - Review and Editing; C. Liu: Resources, Supervision; Z. Li: Visualization, Investigation; D. Tian: Conceptualization, Investigation; N. Zhang: Software, Validation; All authors have reviewed the manuscript. We confirm that the manuscript has been read and approved by all named authors and that there are no other persons who satisfied the criteria for authorship but are not listed. We further confirm that the order of authors listed in the manuscript has been approved by all of us. We understand that the Corresponding Author is the sole contact for the Editorial process. He/she is responsible for communicating with the other authors about progress, submissions of revisions and final approval of proofs.
Corresponding author
Ethics declarations
Competing of interest
I declare that the authors have no competing interests as defined by Springer, or other interests that might be perceived to influence the results and/or discussion reported in this paper.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Xu, Z., Wei, X., Hao, J. et al. DGFormer: a physics-guided station level weather forecasting model with dynamic spatial-temporal graph neural network. Geoinformatica 28, 499–533 (2024). https://doi.org/10.1007/s10707-024-00511-1
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10707-024-00511-1