Abstract
Due to the considerable computational demands of physics-based numerical weather prediction, especially when modeling fine-grained spatio-temporal atmospheric phenomena, deep learning methods offer an advantageous approach by leveraging specialized computing devices to accelerate training and significantly reduce computational costs. Consequently, the application of deep learning methods has presented a novel solution in the field of weather forecasting. In this context, we introduce a groundbreaking deep learning-based weather prediction architecture known as Hierarchical U-Net (HU-Net) with re-parameterization techniques. The HU-Net comprises two essential components: a feature extraction module and a U-Net module with re-parameterization techniques. The feature extraction module consists of two branches. First, the global pattern extraction employs adaptive Fourier neural operators and self-attention, well-known for capturing long-term dependencies in the data. Second, the local pattern extraction utilizes convolution operations as fundamental building blocks, highly proficient in modeling local correlations. Moreover, a feature fusion block dynamically combines dual-scale information. The U-Net module adopts RepBlock with re-parameterization techniques as the fundamental building block, enabling efficient and rapid inference. In extensive experiments carried out on the large-scale weather benchmark dataset WeatherBench at a resolution of 1.40625\(^\circ \), the results demonstrate that our proposed HU-Net outperforms other baseline models in both prediction accuracy and inference time.
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10994-023-06445-3/MediaObjects/10994_2023_6445_Fig1_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10994-023-06445-3/MediaObjects/10994_2023_6445_Fig2_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10994-023-06445-3/MediaObjects/10994_2023_6445_Fig3_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10994-023-06445-3/MediaObjects/10994_2023_6445_Fig4_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10994-023-06445-3/MediaObjects/10994_2023_6445_Fig5_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10994-023-06445-3/MediaObjects/10994_2023_6445_Fig6_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10994-023-06445-3/MediaObjects/10994_2023_6445_Figa_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10994-023-06445-3/MediaObjects/10994_2023_6445_Figb_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10994-023-06445-3/MediaObjects/10994_2023_6445_Fig7_HTML.png)
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Availability of data and materials
This article uses the weather prediction public dataset WeatherBeach, a widely used and publicly available dataset.
Code availability
References
Alley, R. B., Emanuel, K. A., & Zhang, F. (2019). Advances in weather prediction. Science, 363(6425), 342–344.
Arakawa, A. (2004). The cumulus parameterization problem: Past, present, and future. Journal of Climate, 17(13), 2493–2525.
Bauer, P., Quintino, T., Wedi, N., Bonanni, A., Chrust, M., Deconinck, W., Diamantakis, M., Düben, P., English, S., & Flemming, J. et al. (2020). The ECMWF scalability programme: Progress and plans. European Centre for Medium Range Weather Forecasts.
Bauer, P., Thorpe, A., & Brunet, G. (2015). The quiet revolution of numerical weather prediction. Nature, 525(7567), 47–55.
Betts, A. (1973). Non-precipitating cumulus convection and its parameterization. Quarterly Journal of the Royal Meteorological Society, 99(419), 178–196.
Bi, K., Xie, L., Zhang, H., Chen, X., Gu, X., & Tian, Q. ( 2022) Pangu-weather: A 3d high-resolution model for fast and accurate global weather forecast. arXiv preprint arXiv:2211.02556..
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., G. Heigold, G., & Gelly S. et al., (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929.
Ganaie, M. A., Hu, M., Malik, A., Tanveer, M., & Suganthan, P. (2022). Ensemble deep learning: A review. Engineering Applications of Artificial Intelligence, 115, 105151.
Hatamizadeh, A., Nath, V., Tang, Y., Yang, D., Roth, H. R., & Xu, D. (2021). Swin unetr: Swin transformers for semantic segmentation of brain tumors in mri images. In: Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 7th International Workshop, BrainLes. (2021). Held in Conjunction with MICCAI 2021, Virtual Event, September 27, 2021. Revised Selected Papers, Part I.,2022, 272–284.
He, K., Zhang, X., Ren, S., & Sun, J. (2015). Deep residual learning. Image Recognition, vol. 7.
Hu, Y., Chen, L., Wang, Z., & Li, H. (2023). Swinvrnn: A data-driven ensemble forecasting model via learned distribution perturbation. Journal of Advances in Modeling Earth Systems, 15(2), e2022MS003211.
Keisler, R. (2022). Forecasting global weather with graph neural networks. arXiv preprint arXiv:2202.07575.
Lin, H., Gao, Z., Xu, Y., Wu, L., Li, L., & Li, S. Z. (2022). Conditional local convolution for spatio-temporal meteorological forecasting. Proceedings of the AAAI Conference on Artificial Intelligence, 36(7), 7470–7478.
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021). Swin transformer: Hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF international conference on computer vision, pp. 10012–10022.
Lynch, P. (2008). The origins of computer weather prediction and climate modeling. Journal of Computational Physics, 227(7), 3431–3444.
Molteni, F., Buizza, R., Palmer, T. N., & Petroliagis, T. (1996). The ecmwf ensemble prediction system: Methodology and validation. Quarterly Journal of the Royal Meteorological Society, 122(529), 73–119.
Nguyen, T., Brandstetter, J., Kapoor, A., Gupta, J. K., & Grover, A. (2023). Climax: A foundation model for weather and climate. arXiv preprint arXiv:2301.10343.
Pathak, J., Subra manian, S., Harrington, P., Raja, S., Chattopadhyay, A., Mardani, M., Kurth, T., Hall, D., Li, Z., & Azizzadenesheli K. et al.,(2022) Fourcastnet: A global data-driven high-resolution weather model using adaptive fourier neural operators. arXiv preprint arXiv:2202.11214.
Pincus, R., Barker, H. W., & Morcrette, J.-J.(2003). A fast, flexible, approximate technique for computing radiative transfer in inhomogeneous cloud fields. Journal of Geophysical Research: Atmospheres, vol. 108, no. D13, .
Randall, D., Khairoutdinov, M., Arakawa, A., & Grabowski, W. (2003). Breaking the cloud parameterization deadlock. Bulletin of the American Meteorological Society, 84(11), 1547–1564.
Rasp, S., Thuerey, N. (2021) Data-driven medium-range weather prediction with a resnet pretrained on climate simulations: A new model for weatherbench. Journal of Advances in Modeling Earth Systems, vol. 13, no. 2, p. e2020MS002405.
Rasp, S., Dueben, P. . D., Scher, S., Weyn, J. . A., Mouatadid, S., & Thuerey, N. (2020). Weatherbench: A benchmark data set for data-driven weather forecasting. Journal of Advances in Modeling Earth Systems, 12(11), e2020MS002203.
Ritchie, H., Temperton, C., Simmons, A., Hortal, M., Davies, T., Dent, D., & Hamrud, M. (1995). Implementation of the semi-lagrangian method in a high-resolution version of the ecmwf forecast model. Monthly Weather Review, 123(2), 489–514.
Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation,in Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015: 18th International Conference, Munich, Germany, October 5–9, 2015. Proceedings, Part,III(18), 234–241.
Ronneberger, O, Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., W. M. W. III, & Frangi, A. F., (Eds.), Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015—18th International Conference Munich, Germany, October 5 - 9, 2015, Proceedings, Part III, ser. Lecture Notes in Computer Science, vol. 9351. Springer, , pp. 234–241.
Schultz, M. G., Betancourt, C., Gong, B., Kleinert, F., Langguth, M., Leufen, L. H., Mozaffari, A., & Stadtler, S. (2021). Can deep learning beat numerical weather prediction? Philosophical Transactions of the Royal Society A, 379(2194), 20200097.
Shi, X., Chen, Z., Wang, H., Yeung, D.-Y., Wong, W.-K., & Woo, W.-c. (2015) Convolutional lstm network: A machine learning approach for precipitation nowcasting. Advances in Neural Information Processing Systems, vol. 28, .
Weyn, J. A., Durran, D. R., Caruana, R., & Cresswell-Clay, N. (2021) Sub-seasonal forecasting with a large ensemble of deep-learning weather prediction models. Journal of Advances in Modeling Earth Systems, vol. 13, no. 7, p. e2021MS002502.
Funding
This work is supported by National Key Research and Development Program of China [Grant 2022YFB3305401] and the National Nature Science Foundation of China [Grant 62003344]
Author information
Authors and Affiliations
Contributions
BX: Methodology, Validation, Formal analysis, Writing original draft, Visualization; XW, SL, JL and CL: Resources, Writing—review and editing, Supervision.
Corresponding author
Ethics declarations
Conflict of interest
Not applicable.
Ethics approval
Not applicable.
Consent to participate
Not applicable.
Consent for publication
Not applicable.
Additional information
Editors: Vu Nguyen, Dani Yogatama.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Xu, B., Wang, X., Li, J. et al. Hierarchical U-net with re-parameterization technique for spatio-temporal weather forecasting. Mach Learn 113, 3399–3417 (2024). https://doi.org/10.1007/s10994-023-06445-3
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10994-023-06445-3