Skip to main content
SpringerLink
Log in

Spatiotemporal Model with Attention Mechanism for ENSO Predictions

  • Conference paper
  • First Online:
Artificial Neural Networks and Machine Learning – ICANN 2023 (ICANN 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14262))

Included in the following conference series:

  • 675 Accesses

Abstract

Climate disasters such as floods and droughts often cause significant losses to human life, national economy, and public safety. The El Niño Southern Oscillation (ENSO) is one of the most important interannual climate signals in tropical regions, and has a global impact on atmospheric circulation and precipitation. Accurate ENSO predictions can help prevent related climate disasters. Recently, convolutional neural networks (CNNs) have shown the best techniques for ENSO prediction. However, it is difficult for convolutional kernels to capture the long-distance features of ENSO due to the locality of convolution itself. We regard ENSO prediction as a spatiotemporal series prediction problem, and propose an ENSO non-stationary spatiotemporal prediction deep learning model based on a new attention mechanism and a recurrent neural network, called ENSOMIM. The model expands the Receptive field of the network to achieve the learning space characteristics of local and global interaction, and uses high-order nonlinear spatiotemporal neural networks to encode long-term time series features. In order to adequate training the model, we also add historical simulation data to the training set and conduct transfer learning. The experimental results indicate that ENSOMIM is more suitable for large-scale and long-term prediction. During the testing period from 2015 to 2023, ENSOMIM's Niño3.4 index’s all-season correlation skill improved by 11% compared to classical CNNs, and the root mean square error decreased by 29%. It can provide effective predictions for a lead time of up to 20 months. Therefore, ENSOMIM can serve as a powerful tool for predicting ENSO events.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. McPhaden, M.J., Zebiak, S.E.: ENSO as an integrating concept in earth science. Science 314(5806), 1740–1745 (2006)

    Article  Google Scholar 

  2. Bjerknes, J.: Atmospheric teleconnections from the equatorial Pacific. Mon. Weather Rev. 97(3), 163–172 (1969)

    Article  Google Scholar 

  3. Lin, J., Qian, T.: Switch between el nino and la nina is caused by subsurface ocean waves likely driven by lunar tidal forcing. Sci. Rep. 9(1), 1–10 (2019)

    Google Scholar 

  4. Siegert, F., Ruecker, G.: Increased damage from fires in logged forests during droughts caused by El Nino. Nature 414(6862), 437–440 (2001)

    Article  Google Scholar 

  5. Ward, P.J., Jongman, B.: Strong influence of El Niño Southern Oscillation on flood risk around the world. Proc. Natl. Acad. Sci. 111(44), 15659–15664 (2014)

    Article  Google Scholar 

  6. Tang, Y., Zhang, R.H.: Progress in ENSO prediction and predictability study. Natl. Sci. Rev. 5(6), 826–839 (2018)

    Article  Google Scholar 

  7. Masson, S., Terray, P.: Impact of intra-daily SST variability on ENSO characteristics in a coupled model. Clim. Dyn. 39(3), 681–707 (2012)

    Article  Google Scholar 

  8. Alexander, M.A., Matrosova, L.: Forecasting pacific SSTs: linear inverse model predictions of the PDO. J. Clim. 21(2), 385–402 (2008). https://doi.org/10.1175/2007JCLI1849.1

    Article  Google Scholar 

  9. Barnston, A.G., Van, den, Dool, H.M., Zebiak, S.E.: Long-lead seasonal forecasts—where do we stand. Bull. Am. Meteorol. Soc. 75(11), 2097–2114 (1994)

    Google Scholar 

  10. Xue, Y., Leetmaa, A.: Forecasts of tropical Pacific SST and sea level using a Markov model. Geophys. 27(2), 2701–2704 (2000)

    Google Scholar 

  11. Hirst, A.C.: Unstable and damped equatorial modes in simple coupled ocean-atmosphere models. J. Atmos. Sci. 43(6), 606–632 (1986)

    Article  Google Scholar 

  12. Zebiak, S.E., Cane, M.A.: A model el niño–southern oscillation. Mon. Weather Rev. 115(10), 2262–2278 (1987)

    Article  Google Scholar 

  13. Barnett, T.P., Graham, N.: ENSO and ENSO-related predictability. Part I: prediction of equatorial Pacific sea surface temperature with a hybrid coupled ocean–atmosphere model. J. Clim. 6(8), 1545–1566 (1993)

    Google Scholar 

  14. Luo, J.J., Yuan, C.: Current status of intraseasonal–seasonal-to-interannual prediction of the Indo-Pacific climate. Indo-Pacific Climate variability and predictability. pp. 63–107 (2016)

    Google Scholar 

  15. Jin, E.K., Kinter, J.L.: Current status of ENSO prediction skill in coupled ocean–atmosphere models. Clim. Dyn. 31(6), 647–664 (2008)

    Article  Google Scholar 

  16. Ren, F.M., Yuan, Y.: Review of progress of ENSO studies in the past three decades. Adv. Meteorol. Sci. Technol. 2(3), 17–24 (2012)

    Google Scholar 

  17. Clarke, A.J.: El Niño physics and El Niño predictability. Annu. 6(1), 79–99 (2014)

    Google Scholar 

  18. Fang, X., Xie, R.: A brief review of ENSO theories and prediction. Sci. China Earth Sci. 63, 476–491 (2020)

    Article  Google Scholar 

  19. Petersik, P.J., Dijkstra, H.A.: Probabilistic forecasting of El Niño using neural network models. Geophys. Res. Lett. 47(6), e2019GL086423 (2020)

    Google Scholar 

  20. Mahesh, A., Evans, M., Jain, G.: Forecasting El Niño with convolutional and recurrent neural networks. In: 33rd Conference on Neural Information Processing Systems (NIPS 2019), pp. 8–14. Vancouver, Canada (2019)

    Google Scholar 

  21. Broni-Bedaiko: El Niño-southern oscillation forecasting using complex networks analysis of LSTM neural networks. Life Robot. 24(4), 445–451 (2019)

    Google Scholar 

  22. Yuan, Y., Lin, L., Huo, L.Z.: Using an attention-based LSTM encoder–decoder network for near real-time disturbance detection. IEEE J. Select. Top. Appl. Earth Observ. Remote Sens. 13, 1819–1832 (2020)

    Article  Google Scholar 

  23. Kim, J., Kwon, M., Kim, S.D.: Spatiotemporal neural network with attention mechanism for El Niño forecasts. Sci. Rep. 12(1), 7204 (2022)

    Article  Google Scholar 

  24. Gupta, M., Kodamana, H.: Prediction of ENSO beyond spring predictability barrier using deep convolutional LSTM networks. Remote Sense. 19, 1–5 (2020)

    Google Scholar 

  25. Yan, J., Mu, L., Wang, L.: Temporal convolutional networks for the advance prediction of ENSO. Sci. Rep. 10(1), 1–15 (2020)

    Article  Google Scholar 

  26. Cachay, S.R., Erickson, E.: The World as a Graph: Improving El Ni\~ no Forecasts with Graph Neural Networks. arXiv pre-print arXiv:2104.05089 (2021)

  27. Ham, Y.G., Kim, J.H.: Deep learning for multi-year ENSO forecasts. Nature 573(7775), 568–572 (2019)

    Article  Google Scholar 

  28. Srivastava, N., Mansimov, E., Salakhudinov, R.: Unsupervised learning of video representations using LSTMS. In: International Conference on Machine Learning, pp. 843–852. PMLR (2015)

    Google Scholar 

  29. Shi, X., Chen, Z., Wang, H.: Convolutional LSTM network: a machine learning approach for precipitation nowcasting. Adv. Neural. Inf. Process. Syst. 28(3), 802–810 (2015)

    Google Scholar 

  30. Wang, Y., Long, M., Wang, J.: Predrnn: recurrent neural net-works for predictive learning using spatiotemporal LSTMS. Adv. Neural. Inf. Process. Syst. 30(2), 879–888 (2017)

    Google Scholar 

  31. Wang, Y., Gao, Z., Long, M.: Predrnn++: Towards a resolution of the deep-in-time dilemma in spatiotemporal predictive learning. PMLR. 80(5), 5123–5132 (2018)

    Google Scholar 

  32. Wang, Y., Zhang, J., Zhu, H.: Memory in memory: a predictive neural network for learning higher-order non-stationarity from spatiotemporal dynamics. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9154–9162 (2019)

    Google Scholar 

  33. He, D., Lin, P.: Dlenso: A deep learning enso forecasting model. In: Pacific Rim International Conference on Artificial Intelligence, pp. 12–23. Springer, Cham (2019)

    Google Scholar 

  34. Hu, J., Weng, B.: Deep residual convolutional neural network combining dropout and transfer learning for ENSO forecasting. Geophys. Res. Let. 48, e2021GL093531 (2021)

    Google Scholar 

  35. Mu, B., Qin, B., Yuan, S.: ENSO-ASC 1.0. 0: ENSO deep learning forecast model with a multivariate air–sea coupler. Geoscientific Model Development. 14(11), 6977–6999 (2021)

    Google Scholar 

  36. Ye, F., Hu, J., Huang, T.Q.: Transformer for EI Niño-Southern oscillation prediction. IEEE Geosci. Remote Sens. Lett. 19, 1–5 (2021)

    Google Scholar 

  37. Hai, L.L., Wei, G.S., Jia, K.Z.: Spatiotemporal semantic decoupling network for improved ENSO forecasting. CLIVAR Exch. 81, 12–15 (2021)

    Google Scholar 

  38. Zhou, L., Zhang, R.H.: A self-attention–based neural network for three-dimensional multivariate modeling and its skillful ENSO predictions. Sci. Adv. 9(10), eadf2827 (2023)

    Google Scholar 

  39. Carton, J.A., Giese, B.S.: A reanalysis of ocean climate using Simple Ocean Data Assimilation (SODA). Mon. Weather Rev. 136(8), 2999–3017 (2008)

    Article  Google Scholar 

  40. Tian-Jun, Z., Li-Wei, Z.O.U., Xiao-Long, C.: Commentary on the coupled model intercomparison project phase 6 (CMIP6). Adv. Clim. Chang. Res. 15(5), 445 (2019)

    Google Scholar 

  41. Saha, S., Nadiga, S., Thiaw, C.: The NCEP climate forecast system. J. Clim. 19(15), 3483–3517 (2006)

    Article  Google Scholar 

Download references

Acknowledgement

This work is supported by the National Natural Science Foundation of China (Grant No.42075007), the Open Grants of the State Key Laboratory of Severe Weather (No.2021LASW-B19), and the Open Project of Provincial Key Laboratory for Computer Information Processing Technology under Grant KJS2275, Soochow University.

Author information

Authors and Affiliations

  1. School of Computer and Software, Engineering Research Center of Digital Forensics, Ministry of Education, Nanjing University of Information Science and Technology, Nanjing, 210044, China

    Wei Fang, Yu Sha & Xiaozhi Zhang

  2. Jiangsu Collaborative Innovation Center of Atmospheric Environment and Equipment Technology (CICAEET), Nanjing University of Information Science and Technology, Nanjing, 210044, China

    Wei Fang

  3. Provincial Key Laboratory for Computer Information Processing Technology, Soochow University, Suzhou, 215000, China

    Wei Fang

Authors
  1. Wei Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yu Sha
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaozhi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wei Fang .

Editor information

Editors and Affiliations

  1. Democritus University of Thrace, Xanthi, Greece

    Lazaros Iliadis

  2. Democritus University of Thrace, Xanthi, Greece

    Antonios Papaleonidas

  3. Lancaster University, Lancaster, UK

    Plamen Angelov

  4. Teesside University, Middlesbrough, UK

    Chrisina Jayne

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Fang, W., Sha, Y., Zhang, X. (2023). Spatiotemporal Model with Attention Mechanism for ENSO Predictions. In: Iliadis, L., Papaleonidas, A., Angelov, P., Jayne, C. (eds) Artificial Neural Networks and Machine Learning – ICANN 2023. ICANN 2023. Lecture Notes in Computer Science, vol 14262. Springer, Cham. https://doi.org/10.1007/978-3-031-44201-8_30

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-44201-8_30

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-44200-1

  • Online ISBN: 978-3-031-44201-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation