Abstract
Variational Data Assimilation (DA) is a technique aimed at mitigating the error in simulated states by integrating observations. Variational DA is widely employed in weather forecasting and hydrological modeling as an optimization technique for refining dynamic simulation states. However, when constructing the cost function in variational DA, it is necessary to establish a transformation function from simulated states to observations. When observations come from ground sensors or from remote sensing, representing such a transformation function with explicit expressions can sometimes be challenging or even impossible. Therefore, considering the strong mapping capabilities of Neural Network (NN)s in representing the relationship from simulated states to observations, this paper proposes a method utilizing a NN as the transformation function. We evaluate our method on a real dataset of river discharge in the UK and achieved a 39% enhancement in prediction accuracy, measured by Mean Square Error (MSE), compared to the results obtained without DA.
Supported by Resource Geophysics Academy, Imperial College London.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Lahoz, W.A., Schneider, P.: Data assimilation: making sense of earth observation. Front. Environ. Sci. 2, 16 (2014)
Cheng, S., et al.: Machine learning with data assimilation and uncertainty quantification for dynamical systems: a review. IEEE/CAA J. Automatica Sin. 10(6), 1361–1387 (2023)
Bannister, R.N.: A review of operational methods of variational and ensemble-variational data assimilation. Q. J. R. Meteorol. Soc. 143(703), 607–633 (2017)
Reichle, R.H., McLaughlin, D.B., Entekhabi, D.: Variational data assimilation of microwave radiobrightness observations for land surface hydrology applications. IEEE Trans. Geosci. Remote Sens. 39(8), 1708–1718 (2001)
Cheng, S., et al.: Error covariance tuning in variational data assimilation: application to an operating hydrological model. Stochast. Environ. Res. Risk Assess. 35(5), 1019–1038 (2021)
Alvarado-Montero, R., et al.: Multi-parametric variational data assimilation for hydrological forecasting. Adv. Water Resour. 110, 182–192 (2017)
Cheng, S., et al.: A graph clustering approach to localization for adaptive covariance tuning in data assimilation based on state-observation mapping. Math. Geosci. 53(8), 1751–1780 (2021)
Cheng, S., Qiu, M.: Observation error covariance specification in dynamical systems for data assimilation using recurrent neural networks. Neural Comput. Appl. 34(16), 13149–13167 (2022)
Cheng, S., et al.: Generalised latent assimilation in heterogeneous reduced spaces with machine learning surrogate models. J. Sci. Comput. 94(1), 11 (2023)
May, R., Dandy, G., Maier, H.: Review of input variable selection methods for artificial neural networks. Artif. Neural Netw.-Methodol. Adv. Biomed. Appl. 10(1), 19–45 (2011)
Matthews, G., et al.: Evaluating the impact of post-processing medium-range ensemble streamflow forecasts from the European flood awareness system. Hydrol. Earth Syst. Sci. 26(11), 2939–2968 (2022)
National river flow archive homepage. https://nrfa.ceh.ac.uk/. Accessed 7 Feb 2024
Li, M., et al.: Efficient mini-batch training for stochastic optimization. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2014)
Johnson, O.V., et al.: ps-CALR: periodic-shift cosine annealing learning rate for deep neural networks. IEEE Access 11, 139171–139186 (2023)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, K., D. Piggott, M., Wang, Y., Arcucci, R. (2024). Neural Network as Transformation Function in Data Assimilation. In: Franco, L., de Mulatier, C., Paszynski, M., Krzhizhanovskaya, V.V., Dongarra, J.J., Sloot, P.M.A. (eds) Computational Science – ICCS 2024. ICCS 2024. Lecture Notes in Computer Science, vol 14836. Springer, Cham. https://doi.org/10.1007/978-3-031-63775-9_23
Download citation
DOI: https://doi.org/10.1007/978-3-031-63775-9_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-63774-2
Online ISBN: 978-3-031-63775-9
eBook Packages: Computer ScienceComputer Science (R0)