Skip to main content
SpringerLink
Log in

LSTM with spatiotemporal attention for IoT-based wireless sensor collected hydrological time-series forecasting

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

It is necessary to accurately assess the inflow and infiltration conditions in sewer systems if sewer overflows are to be avoided. In this regard, Long Short-Term Memory (LSTM) is widely utilized for hydrological time-series forecasting. However, hydrological time-series have been found to be highly nonlinear and dynamic, such that the original LSTM model cannot simultaneously consider the spatiotemporal correlations of the input sequences for water flow rate forecasting. To address this problem, we propose using an LSTM with spatiotemporal attention (LSTM-STA) model, one based on encoder-decoder architecture, as this will allow accurate forecasting of the water flow rate. The encoder incorporates a spatial attention mechanism module allowing it to adaptively capture the key spatial attributes from all related spatial attributes at each time step. The decoder also incorporates a temporal attention mechanism module for dynamically discovering the key encoder hidden states from all time steps in the window. Using the spatiotemporal attention mechanism, the LSTM-STA model comprehensively considers all the important factors influencing the water flow rate forecasting, in both temporal and spatial dimensions. We performed extensive experiments; applying the LSTM-STA model to real-world hydrological time-series datasets, each one containing 52,704 sampled data points while leveraging state-of-the-art SVR-rbf, MLP, CNN1D, GRU, LSTM, Encoder-Decoder, LSTM-SA, and LSTM-TA as baselines. The experimental results demonstrated that the LSTM-STA model outperforms the state-of-the-art baseline models. Specifically, the LSTM-STA model yields the lowest RMSE, MAE, MAPE, and the highest R2 in the test process, said values being 73.19, 33.37, 1.09, and 0.99858, respectively. We also verified the stability and hyperparameter sensitivity of the LSTM-STA model. Furthermore, we visualized the spatial attention weights and benefitted from spatial interpretability.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data availability

The raw/processed data required to reproduce these findings cannot be shared at this time as the data also forms part of an ongoing study.

References

  1. Tan P, Zhou Y, Zhang Y, Zhu DZ, Zhang T (2019) Assessment and pathway determination for rainfall-derived inflow and infiltration in sanitary systems: a case study. Urban Water J 16(8):600–607

    Article  Google Scholar 

  2. Lepot M, Makris KF, Clemens FH (2017) Detection and quantification of lateral, illicit connections and infiltration in sewers with infra-red camera: conclusions after a wide experimental plan. Water Res 122:678–691

    Article  Google Scholar 

  3. Kang H, Yang S, Huang J, Oh J (2020) Time series prediction of wastewater flow rate by bidirectional LSTM deep learning. Int J Control Autom Syst 18(12):3023–3030

    Article  Google Scholar 

  4. Zhang M, Liu Y, Cheng X, Zhu DZ, Shi H, Yuan Z (2018) Quantifying rainfall-derived inflow and infiltration in sanitary sewer systems based on conductivity monitoring. J Hydrol 558:174–183

    Article  Google Scholar 

  5. Rosin TR, Romano M, Keedwell E, Kapelan Z (2021) A committee evolutionary neural network for the prediction of combined sewer overflows. Water Resour Manage 35(4):1273–1289

    Article  Google Scholar 

  6. Box GE, Pierce DA (1970) Distribution of residual autocorrelations in autoregressive-integrated moving average time series models. J Am Stat Assoc 65(332):1509–1526

    Article  MathSciNet  MATH  Google Scholar 

  7. Box GE, Jenkins GM, Reinsel GC, Ljung GM (2015) Time series analysis: forecasting and control. John Wiley & Sons

    MATH  Google Scholar 

  8. Valipour M, Banihabib ME, Behbahani SMR (2013) Comparison of the ARMA, ARIMA, and the autoregressive artificial neural network models in forecasting the monthly inflow of Dez dam reservoir. J Hydrol 476:433–441

    Article  Google Scholar 

  9. Valipour M (2015) Long-term runoff study using SARIMA and ARIMA models in the United States. Meteorol Appl 22(3):592–598

    Article  MathSciNet  Google Scholar 

  10. Martínez-Acosta L, Medrano-Barboza JP, López-Ramos Á, Remolina López JF, López-Lambraño ÁA (2020) SARIMA approach to generating synthetic monthly rainfall in the Sinú river watershed in Colombia. Atmosphere 11(6):602

    Article  Google Scholar 

  11. Lin GF, Chen GR, Huang PY, Chou YC (2009) Support vector machine-based models for hourly reservoir inflow forecasting during typhoon-warning periods. J Hydrol 372(1–4):17–29

    Article  Google Scholar 

  12. Guo J, Zhou J, Qin H, Zou Q, Li Q (2011) Monthly streamflow forecasting based on improved support vector machine model. Expert Syst Appl 38(10):13073–13081

    Article  Google Scholar 

  13. Dawson CW, Wilby RL (2001) Hydrological modelling using artificial neural networks. Prog Phys Geogr 25(1):80–108

    Article  Google Scholar 

  14. Tokar AS, Johnson PA (1999) Rainfall-runoff modeling using artificial neural networks. J Hydrol Eng 4(3):232–239

    Article  Google Scholar 

  15. Gholami V, Sahour H (2022) Simulation of rainfall-runoff process using an artificial neural network (ANN) and field plots data. Theoret Appl Climatol 147(1):87–98

    Article  Google Scholar 

  16. Kawakami K (2008) Supervised sequence labelling with recurrent neural networks. Ph. D. thesis.

  17. Bengio Y, Simard P, Frasconi P (1994) Learning long-term dependencies with gradient descent is difficult. IEEE Trans Neural Networks 5(2):157–166

    Article  Google Scholar 

  18. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  19. Kratzert F, Klotz D, Brenner C, Schulz K, Herrnegger M (2018) Rainfall-runoff modelling using long short-term memory (LSTM) networks. Hydrol Earth Syst Sci 22(11):6005–6022

    Article  Google Scholar 

  20. Sahoo BB, Jha R, Singh A, Kumar D (2019) Long short-term memory (LSTM) recurrent neural network for low-flow hydrological time series forecasting. Acta Geophys 67(5):1471–1481

    Article  Google Scholar 

  21. Hu C, Wu Q, Li H, Jian S, Li N, Lou Z (2018) Deep learning with a long short-term memory networks approach for rainfall-runoff simulation. Water 10(11):1543

    Article  Google Scholar 

  22. Farhi N, Kohen E, Mamane H, Shavitt Y (2021) Prediction of wastewater treatment quality using LSTM neural network. Environ Technol Innov 23:101632

    Article  Google Scholar 

  23. Hunt K M, Matthews GR, Pappenberger F, Prudhomme C (2022) Using a long short-term memory (LSTM) neural network to boost river streamflow forecasts over the western United States. Hydrology and Earth System Sciences Discussions, 1–30.

  24. Cho K, Van Merriënboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H, Bengio Y (2014) Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078.

  25. Kalchbrenner N, Blunsom P (2013, October) Recurrent continuous translation models. In: Proceedings of the 2013 conference on empirical methods in natural language processing, pp 1700–1709

  26. Sutskever I, Vinyals O, Le QV (2014) Sequence to sequence learning with neural networks. In: Advances in neural information processing systems, pp 3104–3112

  27. Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473.

  28. Ding Y, Zhu Y, Feng J, Zhang P, Cheng Z (2020) Interpretable spatio-temporal attention LSTM model for flood forecasting. Neurocomputing 403:348–359

    Article  Google Scholar 

  29. Ran X, Shan Z, Fang Y, Lin C (2019) An LSTM-based method with attention mechanism for travel time prediction. Sensors 19(4):861

    Article  Google Scholar 

  30. Wang S, Wang X, Wang S, Wang D (2019) Bi-directional long short-term memory method based on attention mechanism and rolling update for short-term load forecasting. Int J Electr Power Energy Syst 109:470–479

    Article  Google Scholar 

  31. Cho K, Van Merriënboer B, Bahdanau D, Bengio Y (2014) On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint arXiv:1409.1259.

  32. Habimana O, Li Y, Li R, Gu X, Yan W (2020) Attentive convolutional gated recurrent network: a contextual model to sentiment analysis. Int J Mach Learn Cybern 11:2637–2651

    Article  Google Scholar 

  33. Ding G, Qin L (2020) Study on the prediction of stock price based on the associated network model of LSTM. Int J Mach Learn Cybern 11(6):1307–1317

    Article  Google Scholar 

  34. Rumelhart DE, Hinton GE, Williams RJ (1986) Learning representations by back-propagating errors. Nature 323(6088):533–536

    Article  MATH  Google Scholar 

  35. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.698.

  36. Bhosale YH, Patnaik KS (2022, May) IoT deployable lightweight deep learning application for COVID-19 detection with lung diseases using RaspberryPi. In: 2022 international conference on IoT and blockchain technology (ICIBT). IEEE, pp 1–6

Download references

Acknowledgements

This research was supported by the Chung-Ang University Young Scientist Scholarship in 2021, by the Chung-Ang University Research Scholarship Grants in 2022, and by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2021R1A2C1009735).

Author information

Authors and Affiliations

  1. School of Electrical and Electronics Engineering, Chung-Ang University, 84 Heukseok-ro, Dongjak-gu, 06974, Seoul, Republic of Korea

    Jianying Huang, Jinhui Li & Hoon Kang

  2. School of Civil and Environmental Engineering, Chung-Ang University, 84 Heukseok-ro, Dongjak-gu, 06974, Seoul, Republic of Korea

    Jeill Oh

Authors
  1. Jianying Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jinhui Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jeill Oh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hoon Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Jianying Huang or Hoon Kang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, J., Li, J., Oh, J. et al. LSTM with spatiotemporal attention for IoT-based wireless sensor collected hydrological time-series forecasting. Int. J. Mach. Learn. & Cyber. 14, 3337–3352 (2023). https://doi.org/10.1007/s13042-023-01836-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-023-01836-3

Keywords

Search

Navigation