Skip to main content

Advertisement

SpringerLink
Log in

An automatic hyperparameter optimization DNN model for precipitation prediction

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Deep neural networks (DNN) have gained remarkable success on many rainfall predictions tasks in recent years. However, the performance of DNN highly relies upon the hyperparameter setting. In order to design DNNs with the best performance, extensive expertise in both the DNN and the problem domain under investigation is required. But many DNN users have not met this requirement. Therefore, it is difficult for the users who have no extensive expertise in DNN to design optimal DNN architectures for their rainfall prediction problems that is to solve. In this paper, we proposed a novel automatic hyperparameters optimization method for DNN by using an improved Gene Expression Programming. The proposed method can automatically optimize the hyperparameters of DNN for precipitation modeling and prediction. Extensive experiments are conducted with three real precipitation datasets to verify the performance of the proposed algorithm in terms of four metrics, including MAE, MSE, RMSE, and R-Squared. The results show that: 1) the DNN optimized by the proposed method outperforms the existing precipitation prediction methods including Multiple Linear Regression (MLR), Back Propagation (BP), Support Vector Machine (SVM), Random Forest (RF) and DNN; 2) the proposed DNN hyperparameter optimization method outperforms state-of-the-art DNN hyperparameter optimization methods, including Genetic Algorithm, Bayes Search, Grid Search, Randomized Search, and Quasi Random Search.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. He X, Guan H, Zhang X, Simmons CT (2014) A wavelet-based multiple linear regression model for forecasting monthly rainfall. Int J Climatol 34(6):1898–1912

    Article  Google Scholar 

  2. Kumar A, Tyagi N (2016) Comparative analysis of backpropagation and RBF neural network on monthly rainfall prediction. In: IEEE international conference on inventive computation technologies (ICICT), pp 1-6

  3. Panaligan D, Razon JA, Caro J, et al (2016) Using machine learning to provide rapid rainfall forecasts based on radar-derived data. In: workshop on computing: Theory & Practice (WCTP), pp 117-131

  4. Yang TC, Yu PS, Lin KH, Kuo CM, Tseng HW (2018) Predictor selection method for the construction of support vector machine (SVM)-based typhoon rainfall forecasting models using a non-dominated sorting genetic algorithm. Meteorol Appl 25(4):510–522

    Article  Google Scholar 

  5. Meyer H, Kühnlein M, Appelhans T, Nauss T (2016) Comparison of four machine learning algorithms for their applicability in satellite-based optical rainfall retrievals. Atmos Res 169:424–433

    Article  Google Scholar 

  6. Jia Z, Guan Z, Liu Z, Yang DM (2020) Influence of short-term rainfall forecast error on flood forecast operation: a risk assessment based on Bayesian theory. Hum Ecol Risk Assess 2020(2):1–15

    Google Scholar 

  7. Kim S, Seo Y, Lee CJ (2016) Modeling of rainfall by combining neural computation and wavelet technique. Procedia Eng 154:1231–1236

    Article  Google Scholar 

  8. Dufek AS, Augusto DA, Dias PLS, Barbosa HJC (2017) Application of evolutionary computation on ensemble forecast of quantitative precipitation. Comput Geosci 106:139–149

    Article  Google Scholar 

  9. Young T, Hazarika D, Poria S, Cambria E (2018) Recent trends in deep learning based natural language processing. ieee. Computational intelligenCe magazine 13(3):55–75

    Article  Google Scholar 

  10. Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Petersen S (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529–533

    Article  Google Scholar 

  11. Zhang R, Isola P, Efros AA, et al (2016) Colorful image colorization. In: european conference on computer vision, pp 649-666

  12. Vinyals O, Toshev A, Bengio S, et al (2015) Show and tell: a neural image caption generator. In: computer vision and pattern recognition, pp 3156-3164

  13. Shen D, Wu G, Suk H et al (2017) Deep learning in medical image analysis. Annu Rev Biomed Eng 19(1):221–248

    Article  Google Scholar 

  14. Liu Y, Racah E, Correa J, Khosrowshahi A, Lavers D, Kunkel K, Collins W (2016) Application of deep convolutional neural networks for detecting extreme weather in climate datasets. Computer Vision and Pattern Recognition arXiv 1605:01156

    Google Scholar 

  15. Grover A, Kapoor A, Horvitz E, et al (2015) A deep hybrid model for weather forecasting. In: knowledge discovery and data mining, pp 379-386

  16. Dhar D, Bagchi S, Kayal CK, Mukherjee S, Chatterjee S (2019) Quantitative rainfall prediction: deep neural network-based approach. In: springer proceedings of international ethical hacking conference 2018 Singapore, pp. 455-463

  17. Cramer S, Kampouridis M, Freitas AA, Alexandridis AK (2017) An extensive evaluation of seven machine learning methods for rainfall prediction in weather derivatives. Expert Syst Appl 85(18):169–181

    Article  Google Scholar 

  18. Akiba T, Sano S, Yanase T, et al (2019) Optuna: a next-generation hyperparameter optimization framework. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp 2623-2631

  19. Feurer M, Springenberg JT, Hutter F, el at (2015) Initializing bayesian hyperparameter optimization via meta-learning. In: Twenty-Ninth AAAI Conference on Artificial Intelligence, pp. 1128–1135

  20. Koch P, Golovidov O, Gardner S, et al (2018) Autotune: a derivative-free optimization framework for hyperparameter tuning. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp 443-452

  21. Luo R, Tian F, Qin T, et al (2018) Neural architecture optimization. In: neural information processing systems, pp 7816-7827

  22. Ying, C, Klein, A, Real E, Christiansen E, Murphy K, Hutter F (2019) NAS-Bench-101: towards reproducible neural architecture search. https://doi.org/10.1007/978-3-030-05318-5_3

  23. Darwish A, Hassanien AE, Das S et al (2020) A survey of swarm and evolutionary computing approaches for deep learning. Artif Intell Rev 53(3):1767–1812

    Article  Google Scholar 

  24. Sun YN, Xue B, Zhang M, Yen GG (2020) Evolving deep convolutional neural networks for image classification. IEEE Trans Evol Comput 24(2):394–407

    Article  Google Scholar 

  25. Sun YN, Xue B, Zhang M, Yen GG (2020) Completely automated CNN architecture design based on blocks. IEEE transactions on neural networks and learning systems 31(4):1242–1254

    Article  MathSciNet  Google Scholar 

  26. Suganuma M, Shirakawa S, Nagao T, et al (2017) A genetic programming approach to designing convolutional neural network architectures. In: genetic and evolutionary computation conference, pp 497-504

  27. Wang B, Sun Y, Xue B, et al (2019) A hybrid GA-PSO method for evolving architecture and short connections of deep convolutional neural networks. In: pacific rim international conference on artificial intelligence, pp 650-663

  28. Xie L, Yuille A (2017) Genetic CNN. In: IEEE international conference on computer vision (ICCV), pp 1379-1388

  29. Sun Y, Xue B, Zhang M, et al. (2020) Automatically designing CNN architectures using the genetic Algorithm for image classification[J]. IEEE Transactions on Cybernetics, pp (99):1–15

  30. Hoang N, Tien Bui D (2018) Spatial prediction of rainfall-induced shallow landslides using gene expression programming integrated with GIS: a case study in Vietnam. Nat Hazards 92(3):1871–1887

    Article  Google Scholar 

  31. Peng Y, Changan Y, Jian WC (2010) Multicellular gene expression programming algorithm for function optimization. Control Theory & Applications 27(1):1585–1589

    Google Scholar 

  32. Zhong J, Ong Y, Cai W (2016) Self-learning gene expression programming. IEEE Trans Evol Comput 20(1):65–80

    Article  Google Scholar 

  33. Armaghani DJ, Faradonbeh RS, Rezaei H, Rashid ASA, Amnieh HB (2018) Settlement prediction of the rock-socketed piles through a new technique based on gene expression programming. Neural Comput & Applic 29(11):1115–1125

    Article  Google Scholar 

  34. Faradonbeh RS, Armaghani DJ, Amnieh HB, Mohamad ET (2018) Prediction and minimization of blast-induced flyrock using gene expression programming and firefly algorithm. Neural Comput & Applic 29(6):269–281

    Article  Google Scholar 

  35. Zhong J, Feng L, Cai W et al (2019) Multifactorial genetic programming for symbolic regression problems. IEEE Transactions on Systems, Man, and Cybernetics 25(7):1–14

    Google Scholar 

  36. Nazari A (2019) Application of gene expression programming to predict the compressive damage of lightweight aluminosilicate geopolymer. Neural Comput & Applic 31(2):767–776

    Article  Google Scholar 

  37. Ferreira C (2001) Gene expression programming: a new adaptive algorithm for solving problems. Complex Systems 13(2):87–129

    MathSciNet  MATH  Google Scholar 

  38. Deng CY, Peng YZ, Li HY, Gong DQ, Zhang H, Liu ZP (2019) A seft-adaptive multicellular GEP algorithm based on fuzzy control for function optimization. arXiv:2634575

  39. Peng Y, Deng C, Li H, et al (2019) Precipitation modeling and prediction based on fuzzy-control Multicellularular gene expression programming and wavelet transform. In: international conference on intelligent computing, pp 81-92

  40. Ramachandran P, Zoph B, Le QV (2018) Searching for activation functions. International conference on learning representations, arXiv:1710.05941

  41. Dong X, Shen J, Wang W, et al (2018) Hyperparameter optimization for tracking with continuous deep Q-Learning. In: computer vision and pattern recognition, pp 518-527

  42. Lorraine J, Duvenaud D (2018) Stochastic hyperparameter optimization through hypernetworks. arXiv preprint arXiv:1802.09419, 2018

  43. Srivastava N, Hinton GE, Krizhevsky A et al (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958

    MathSciNet  MATH  Google Scholar 

  44. Nazreen PM, Ramakrishnan AG (2018) DNN based speech enhancement for unseen noises using Monte Carlo dropout. In: international conference on signal processing and communication systems, pp 1-6

  45. Komer B, Bergstra J, Eliasmith C (2019) Hyperopt-Sklearn. Springer Cham, Automated Machine Learning, In, pp 97–111

    Google Scholar 

Download references

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China Grant #61562008 and #62006051, the Natural Science Foundation of Guangxi Zhuang Autonomous Region Grant #2017GXNSFAA198228 and #2017GXNSFBA198153 and #2018GXNSFAA294079, the Project of Scientific Research and Technology Development in Guangxi Grant #AA18118047 and #AD18126015. Thanks to the support by the BAGUI Scholar Program of Guangxi Zhuang Autonomous Region of China. Yuzhong Peng and Daoqing Gong contributed equally to this work and should be considered co-first authors. Yuzhong Peng and Hao Zhang are the co-corresponding author.

Author information

Authors and Affiliations

  1. School of Computer & Information Engineering, Nanning Normal University, Nanning, 530001, China

    Yuzhong Peng, Daoqing Gong & Chuyan Deng

  2. College of Computer Science & Technology, Fudan University, Shanghai, 200433, China

    Yuzhong Peng & Hao Zhang

  3. Department of Science, Shangqiu University Applied Science and Technology College, Kaifeng, 475000, China

    Hongya Li

  4. Department of Mathematics and Computer Science, The Guangxi College of Education, Nanning, 530023, China

    Hongguo Cai

Authors
  1. Yuzhong Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daoqing Gong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chuyan Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hongya Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hongguo Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Hao Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Yuzhong Peng or Hao Zhang.

Ethics declarations

Conflict of interests

There are no potential competing interests in our paper. And all authors have seen the manuscript and approved to submit to your journal. We confirm that the content of the manuscript has not been published or submitted for publication elsewhere.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

Table 9 The obtained hyperparameters optimized by FMCGEP-DNN On the dataset 1
Full size table
Table 10 The obtained hyperparameters optimized by FMCGEP-DNN on the dataset 2
Full size table
Table 11 The obtained hyperparameters optimized by FMCGEP-DNN on the dataset 3
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Peng, Y., Gong, D., Deng, C. et al. An automatic hyperparameter optimization DNN model for precipitation prediction. Appl Intell 52, 2703–2719 (2022). https://doi.org/10.1007/s10489-021-02507-y

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02507-y

Keywords

Search

Navigation