Skip to main content
SpringerLink
Log in

Challenges and opportunities for a hybrid modelling approach to earth system science

  • Review Paper
  • Published:
CCF Transactions on High Performance Computing Aims and scope Submit manuscript

Abstract

Artificial Intelligence (AI), particularly in the fields of Machine learning (ML) and Deep learning (DL) have become important tools for the scientific community in general. By coupling traditional modelling and simulation with the new AI approaches in a hybrid fashion we can advance our current science in leaps and bounds. In the field of Earth System Science (ESS) this is particularly so and, although the technology is still somewhat nascent, the opportunities and potential benefits are enormous. Furthermore, this mixed approach offers a pathway to solving the extremely demanding future science goals in this space that cannot be solved through computing capability alone. This paper examines the state of the art in the application of HPC + AI to the domain of ESS, identifying several important application areas and techniques for hybrid modelling. We also look at the challenges that currently limit widespread adoption of hybrid modelling and delve into potential solutions to those limitations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Availability of data and material

Not Applicable.

Code availability

Not Applicable.

Notes

  1. https://developer.download.nvidia.com/video/gputechconf/gtc/2019/presentation/s9217-deep-learning-for-improved-utilization-of-satellite-data-in-weather-forecasting.pdf.

  2. https://github.com/Dobiasd/frugally-deep.

  3. https://clima.caltech.edu/.

References

  • Ackmann, J., Peter, D., Tim, P., and Piotr, S. 2020. Machine-learned Preconditioners for Linear Solvers in Geophysical Fluid Flows. arXiv. https://arxiv.org/pdf/2010.02866.pdf.

  • Ammar, A., S. Labroue, E. Obligis, M. Crepon, and S. Thiria. 2016. Building a Learning Database for the Neural Network retrieval of Sea Surface Salinity from SMOS Brightness Temperatures. arXiv e-prints. https://ui.adsabs.harvard.edu/abs/2016arXiv160104296A.

  • Bauer, P., Peter Dueben, T., Hoefler, T.Q., Schulthess, T., Wedi, N.: The digital revolution of earth-system science. Nat. Comput. Sci. (2021). https://doi.org/10.1038/s43588-021-00023-0

    Article  Google Scholar 

  • Beucler, Tom, Michael S. Pritchard, Stephan Rasp, Pierre Gentine, Jordan Ott, and Pierre Baldi. 2019. Enforcing Analytic Constraints in Neural-Networks Emulating Physical Systems. arXiv: Comput. Phys. Accessed 3 28, 2021. https://arxiv.org/abs/1909.00912.

  • Bharaskaran, P., R. Kumar, R. Barman, and R. Muthalagu. 2013. A new approach for deriving temperature and salinity fields in the Indian Ocean using artificial neural networks. J. Mar. Sci. Technol. 160–175.

  • Bolton, T., Zanna, L.: Applications of deep learning to ocean data inference and subgrid parameterization. J. Adv. Model. Earth Syst. (JAMES) (2019). https://doi.org/10.1029/2018MS001472

    Article  Google Scholar 

  • Bonavita, M., Laloyaux, P.: Machine learning for model error inference and correction. J Adv Model Earth Syst (2020). https://doi.org/10.1029/2020MS002232

    Article  Google Scholar 

  • Brenowitz, N., and C. Bretherton. 2019. "Peering into the black box of machine learning parameterization. AGU Fall Meeting. https://ui.adsabs.harvard.edu/abs/2019AGUFMIN44A..03B.

  • Carmen, J., Clune T., Francis G., Govett M., Brian G.ss, Anke K., Tsendgar L., et al. 2017. Position Paper on High-Performance Computing for Earth System Models. https://doi.org/10.7289/V5862DH3.

  • Chantry, M., Sam, H., Pete,r D., Inn,a P., and Tim, P. 2021. Machine learning emulation of gravity wave drag in numerical weather forecasting. arXiv. https://arxiv.org/pdf/2101.08195.pdf.

  • Chapman, W., Subramanian, A., Dell Monache, L., Xie, S., Ralph, F.: Improving stmospheric river forecasts with machine learning. Geophys. Res. Lett. (2019). https://doi.org/10.1029/2019GL083662

    Article  Google Scholar 

  • Curcic, M. 2019. A parallel Fortran framework for neural networks and deep learning. arXiv. https://arxiv.org/pdf/1902.06714.pdf.

  • Ducournau, A., and R. Fablet. 2016. Deep learning for ocean remote sensing: an application of convolutional neural networks for super-resolution on satellite-derived SST data. 9th IAPR Workshop on Pattern Recogniton in Remote Sensing (PRRS). Cancun, Mexico. https://doi.org/10.1109/PRRS.2016.7867019.

  • Dueben, P., Bauer, P.: Challenges and design choices for global weather and climate models based on machine learning.Geosci Model (2018). https://doi.org/10.5194/gmd-11-3999-2018

  • Fuhrer, O., Tarun, C., Torsten, H., Grzegorz, K., Xavier, L., David, L. Daniel L., et al. 2017. Near-global climate simulation at 1 km resolution: establishing a performance baseline on 4888 GPUs with COSMO 5.0. Geosci. Model Dev. 11 (4): 1665–1681. Accessed 3 27, 2021. https://geosci-model-dev.net/11/1665/2018/gmd-11-1665-2018.pdf.

  • Galea, D., Lawrence, B., Kunkel, J.: detecting tropical cyclone using deep learning techniques.EGU General Assembly (2020). https://doi.org/10.5194/egusphere-egu2020-9870

  • Goodfellow, I., Jean P.-A., Mehdi M., Bing X, David W.-F., Sherjil O., Aaron C., Yoshua B.. 2014. Generative Adversarial Networks. Proceedings of the International Conference on Neural Information Processing Systems (NIPS 2014). 2672–2680.

  • Gronquist, P., Chengyuan, Y., Tal Ben-Nun, Peter Dueben, Shigang Li, and Torsten Hoefler. 2020. "Deep Learning for Post-Processing Ensemble Weather Forecasts." arXiv. https://arxiv.org/abs/2005.08748.

  • Hall, D., J. Stewart, C Bonfanti, M. Govett, S. Maksimovic, and L. Trailovic. 2019. "Deep learning for improved use of satellite observations." Ninth Symposium on Advances in Modeling and Analysis using Python. Phoenix. https://ams.confex.com/ams/2019Annual/meetingapp.cgi/Paper/353938.

  • Hennigh, O., Susheela, N., Mohammad, N., Akshay, S., Kaustubh, T., Max, R., Jose del, A. F., Wonmin, B., Zhiwei, F., Sanjay, C. 2020. NVIDIA SimNet: an AI-accelerated multi-physics simulation framework. arXiv. https://arxiv.org/abs/2012.07938.

  • Higashiyama, K., Fujimoto, Yu., Hayashi, Y.: Feature extraction of NWP data for Wind Power Forecasting using 3D-Convolutional Neural Networks. Energy Procedia 155, 350–358 (2018)

    Article  Google Scholar 

  • Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. (1997). https://doi.org/10.1162/neco.1997.9.8.1735

    Article  Google Scholar 

  • Hogan, R., Bozzo, A.: A flexible and efficient radiation scheme for the ECMWF model. J Adv Moldel Earth Syst (2018). https://doi.org/10.1029/2018MS001364

    Article  Google Scholar 

  • James, S., Yushan, Z., and Ferghal, O. 2017. A Machine Learning Framework to Forecast Wave Conditions. arXiv. https://arxiv.org/pdf/1709.08725.pdf.

  • Kadow, C., David, M.H., and Uwe, U. 2020 Artificial intelligence reconstructs missing climate information. Nat Geosci. https://doi.org/10.1038/s41561-020-0582-5.

  • Kim, S., Seungkyun, H., Minsu, J., Sa-Kwang, S. 2017a. Deeprain: ConvLSTM network for precipitation prediction using mulitchannel radar data. Clim Inform 89–92.

  • Kim, S.K., Sasha, A., Jiwoo, L., Chengzhu, Z., Aaron, W., Dean, W. 2017b. Massive Scale Deep Learning for detecting Extreme Climate events. Clim Inform. Boulder.

  • Kim, S., Ji-Sun, K., Minho, L., Sa-Kwang, S. 2018. DeepTC: ConvLSTM Network for Trajectory Prediction of Tropical Cyclone using Spatiotemporal Atmospheric Simulation. https://openreview.net/forum?id=HJlCVoPAF7.

  • Krasnopolsky, V., M. Fox-Rabinovitz, Y. Hou, S. Lord, and A. Belochitski. 2010. Accurate and Fast Neural Network Emulations of Model Radiation for the NCEP Coupled Climate Forecast System: Climate Simulations and Seasonal Predictions. Mon Weather Rev. Boston. https://doi.org/10.1175/2009MWR3149.1.

  • Krizhevsky, A., Sutskever, I., Hinton, G.: ImageNet classification with deep convolutional networks. Neural Imform Process Syst (2012). https://doi.org/10.1145/3065386

    Article  Google Scholar 

  • Kumler-Bonfanti, C., Stewart, J., David, H., Mark, G. 2020. Tropical and Extratropical Cyclone Detection Using Deep Learning.

  • Kurth, T, Sean, T., Joshua, R., Mayur, M, Nathan, L, Everett, P, Ankur, M., et al. 2018. "Exascale deep learning for climate analytics. Proceedings of the Intenational Conference for High Performance Computing, Networking, Storage, and Analysis. Dallas, Texas: IEEE Press.

  • LeCun, Y., Yoshua,. B. 1995. Convolutional networks for images, speech and time series. The handbook of brain theory.

  • LeCun, Y., Bernhard, B., John, D., Donnie, H., Richard, H., Wayne, H., Lawrence, J. 1989. Backpropogation applied to handwritten zip code recognition. Neural Computation. MIT Press. 541–551.

  • Lee, Y.-J., Christina, B., Lidia, T., Brian, E., Mark, G., and Jebb, S. 2017. Using Deep Learning for Targeted Data Selection, Improving Satellite Observation Utilization for Model Initialization. AGU Fall Meeting.

  • Lee, C.S., Sohn, E., Park, J.D., Jang, J.-D.: Estimation of soil moisture using deep learning based on satellite data: a case study of South Korea. Gisci Remote Sens (2019). https://doi.org/10.1080/15481603.2018.1489943

    Article  Google Scholar 

  • Matsuoka, D., Nakano, M., Sugiyama, D., Uchida, S.: Deep learning approach for detecting tropical cyclones and their precursors in the simulation by a cloud-resolving global nonhydrostatic atmospheric model. Prog Earth Planet Sci (2018). https://doi.org/10.1186/s40645-018-0245-y

    Article  Google Scholar 

  • Mestre, O., Michael, Z., Maxime, T., Marion, R., and Harold, P. 2018. Machine learning techniques applied to NWP post-processing. Météo-France.

  • Mommert, M, Mario, S., Marcel, N., Linus, S., and Damian, B.. 2020. Characterization of Industrial Smoke Plumes from Remote Sensing Data. https://arxiv.org/abs/2011.11344v1.

  • Montavon, G, Binder, A., Lapuschkin, S., Samek, W., and Muller, K. 2019. Layer-Wise Relevance Proogation: An Overview. Exaplainable AI: Interpreting, Explaining and Visualizing Deep Learning. Lecture Notes in Computer Science (Springer). https://doi.org/10.1007/978-3-030-28954-6_10.

  • Mouatadid, S., Steve, E., and Andre, E. 2017. Non-uniforn spatial downscaling of climate variables. Clim Inform. 65–68.

  • Olah, C., A. Mordvintsev, and L. Schubert. 2017. Feature visualization: How neural networks build up their understanding of images. doi:https://doi.org/10.23915/distill.00007

  • Ouala, S., Cédric, H., and Ronan, . 2018. Sea surface temperature prediction and reconstruction using patch-level neural network representations. arXiv: Machine Learning. Accessed 3 29, 2021. https://arxiv.org/abs/1806.00144.

  • Pal, A., and Norman, M. 2017. Emulating RRTMG Radiation with Deep Neural Networks for the Accelerated Model for Climate and Energy. AGU Fall Meeting. https://ui.adsabs.harvard.edu/abs/2017AGUFMNG33A0188P.

  • Pan, S., Pan, N., Tian, H., Friedlingstein, P., Sitch, S., Shi, S., Arora, V., et al.: Evaluation of global terrestrial evapotranspiration using state-of-the-art approaches in remote sensing, machine learning and land surface modeling.Hydrol. Earth Syst. Sci (2020). https://doi.org/10.5194/hess-24-1485-2020

  • Playout, C., and Asanobu, K.. 2019. Latent Space Representation and RNN for Image-based Typhoon Intensity Analysis and Prediction. Clim Inform. Paris. 47–52.

  • Prabhat, K., Kashinath, M., Mudigonda, S., Kim, L., Kapp-Schwoerer, A., Graubner, E.K., et al.: ClimateNet: an expert-labeled open dataset and deep learning architecture for enabling high-precision analyses of extreme weather.Geosci Model Dev (2021). https://doi.org/10.5194/gmd-14-107-2021

  • Priya, R., and K. Vani. 2019. Deep Learning Based Forest Fire Classification and Detection in Satellite Images. 11th International Conference on Advanced Computing (ICoAC). Chennai. 61–65. https://doi.org/10.1109/ICoAC48765.2019.246817.

  • Rahman, S. M., O. San, and A. Rasheed. 2018. A Hybrid Approach for Model Order Reduction of Barotropic Quasi-Geostrophic Turbulence. Fluids.

  • Raissi, M., Paris, P., George, K. 2019. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Computational Physics 686–707.

  • Rasp, S., Stephan, R., Michael, S.P., and Pierre, G. 2018. Deep learning to represent subgrid processes in climate models. Proceedings of the National Academy of Sciences of the United States of America 115 (39): 9684–9689. Accessed 3 28, 2021. https://pnas.org/content/115/39/9684.

  • Reichstein, M., Camps-Valls, G., Stevens, B., Jung, M., Denzler, J., Caravalhais, N., Prabhat: Deep learning and process understanding for data-driven Earth system science. Nature (2019). https://doi.org/10.1038/s41586-019-0912-1

    Article  Google Scholar 

  • Ronneberger, O., Fischer, P., and Brox, T.. 2015. U-net: convolutional networks for biomedical image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). Springer.

  • Rudy, S. H., Alessandro, A., Steven, L. B., and Nathan Kutz, J.. 2018. Data-driven identification of parametric partial differential equations. arXiv: Numerical Analysis. Accessed 3 30, 2021. https://arxiv.org/abs/1806.00732.

  • Rupe, A., James Crutchfield, Karthik Kashinath, and Prabhat. 2017. A physics-based approach to unsupervised discovery of coherent structures in spatiotemporal systems. Climate Informatics 101–104.

  • Rupe, A., Karthik, K., Nalini, K., Victor, L., Prabhat, and James, C. 2019. "Towards Unsupervised Segmmentation of Extreme Weather events. Clim. Inform. Paris. 213–218.

  • Seo, S., Arash, M., George, B.-W., and Yan, L. 2017. Graph convolutional autoencoder with recurrent neural networks for spatiotemporal forecasting. Clim Inform. 85–88.

  • Sever, G., Jeff, A., Stan, P., and Charlie, C.. 2019. Performance Evaluation of the Weather Research and Forecasting (WRF) Model on the DOE Summit Supercomputer. 9th International Symposium on Atmospheric Sciences (ATMOS 2019). Istanbul, Turkey. 743–749.

  • Singh, A., Adrian, A., and Brian, W. 2019. Downscaling numerical weather Models with GANs. Clim. Inform. Paris. 275–278.

  • Sonderby, C., Lasse, E., Jonathon, H., Mostafa, D., Avital, O., Tim, S., Shreya, A., Jason, H., and Nal, K. 2020. MetNet: A Neural Weather Model for Precipitation Forecasting. arXiv. https://arxiv.org/abs/2003.12140.

  • Ukkonen, P., Pincus, R., Hogan, R., Nielsen, K., Kaas, E.: Accelerating radiation computations for dynamical models with targeted machine learning and code optimization. J. Adv. Model. Earth Syst. (2020). https://doi.org/10.1029/2020MS002226

    Article  Google Scholar 

  • Veerman, M., Pincus, R., Stoffer, R., van Leeuwen, C., Podereanu, D., varHeerwaarden, C.: Predicting atmospheric optical properties for radiative transfer computations using neural networks. Philos. Trans. r. Soc. (2020). https://doi.org/10.1098/rsta.2020.009

    Article  Google Scholar 

  • Wang, L., Scott, K., Clausi, D.: Sea ice concentration estimation during freeze-up from SAR imagery using a convolutional neural network. Remote Sens (2017). https://doi.org/10.3390/rs9050408

    Article  Google Scholar 

  • Wang, J., Balaprakash, P., Kotamarthi, R.: Fast domain-aware neural network emulation of a planetary boundary layer parameterization in a numerical weather forecast model.Geosci. Model Dev. (2019). https://doi.org/10.5194/gmd-12-4261-2019

  • Wimmers, A., Velden, C., Cossuth, J.: Using deep learning to estimate tropical cyclone intensity from satellite passive microwave imagery. Mon. Weather Rev. (2019). https://doi.org/10.1175/MWR-D-18-0391.1

    Article  Google Scholar 

  • Yin, Y., Arthur, P., Emmanuel, D.B., and Patrick, G. 2019. Unsupervised Inpainting for Occluded Sea Surface Temperature Sequences. Clim. Inform.

  • Zhang, Z.J., Karthikeyan, D. 2015. Machine learning methods for Data-Driven Turbulence Modeling. 22nd AIAA Computational Fluid Dynamics Conference. https://doi.org/10.2514/6.2015-2460.

  • Zhao, W. L., Wen, L. Z., Pierre, G., Markus, R., Yao, Z., Sha, Z, Yeqiang, W, et al. 2019. "Physics‐constrained machine learning of evapotranspiration. Geophys. Res. Lett.. Accessed 3 29, 2021. https://agupubs.onlinelibrary.wiley.com/doi/pdf/https://doi.org/10.1029/2019gl085291.

Download references

Funding

No funds, grants, or other support was received.

Author information

Authors and Affiliations

  1. NVIDIA AI Technology Centre, Singapore, Singapore

    Simon See & Jeff Adie

Authors
  1. Simon See
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jeff Adie
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design, material preparation, data collection and analysis. The first draft of the manuscript was written by JA and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Jeff Adie.

Ethics declarations

Conflict of interest

The authors have no conflicts of interest to declare that are relevant to the content of this article.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

See, S., Adie, J. Challenges and opportunities for a hybrid modelling approach to earth system science. CCF Trans. HPC 3, 320–329 (2021). https://doi.org/10.1007/s42514-021-00071-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s42514-021-00071-y

Keywords

Search

Navigation