A framework for estimating all-weather fine resolution soil moisture from the integration of physics-based and machine learning-based algorithms

Highlights

• A practical framework was proposed to obtain all-weather fine resolution soil moisture from the synergetic use of optical and microwave data.

• Physics-based and machine learning-based methods were integrated to estimate soil moisture.

• Acceptable accuracy with unbiased root mean square error of ∼ 0.06 m³/m³ can be achieved with the proposed approaches.

• Physics-based method can probably outperform machine learning-based method over regions with complicated underlying surfaces.

Abstract

Due to the effects of radio frequency interference and the limitations of algorithms under specific conditions, most of the currently available microwave-based soil moisture (SM) products are spatially discontinuous and have coarse spatial resolution, whereas optical observations also reveal various data gaps due to cloud contamination. Hence, the prediction of SM over invalid pixels and disaggregation from coarse to high scales are two main processes for obtaining SM at fine spatiotemporal resolution (e.g., daily/1-km). In the present study, two methods with respect to disaggregation-first or prediction-first were investigated from the synergetic use of the widely recognized European Space Agency-Climate Change Initiative (ESA-CCI) SM product and Moderate Resolution Imaging Spectroradiometer (MODIS) images over the Tibetan Plateau (TP) region. Specifically, the Disaggregation based on Physical And Theoretical scale Change (DisPATCh) algorithm and the generalized regression neural network (GRNN) were implemented in the disaggregation and prediction, respectively. In DisPATCh, spatially complete land surface temperature (LST), normalized difference vegetation index (NDVI) and digital elevation model (DEM) were provided as essential inputs to downscale the microwave-based ESA-CCI to a spatial resolution of 1 km, whereas MODIS-derived LST, NDVI, land surface albedo and DEM were considered in the GRNN prediction. Following the two methods, the daily/1-km SM dataset over a period of three years was finally estimated. Assessments with ground in-situ SM measurements over the TP region reveal an acceptable accuracy with unbiased root mean square errors of ∼ 0.06 m³/m³, indicating the potential to obtain operational daily/1-km spatially continuous SM products in future developments.

Introduction

As one of the fundamental components of the Earth system, soil moisture (SM) plays an important role in studies such as precipitation analysis, flood risk assessment and drought monitoring (Wang et al., 2019, Li et al., 2021). In general, two approaches have been frequently used to obtain SM: traditional ground in-situ measurements and satellite retrievals. Although the accuracy of ground in-situ measurements under field conditions is relatively higher, it commonly suffers from the disadvantages of limited observation area and high cost. As a consequence, an increasing number of studies have highlighted the feasibility of obtaining SM from satellite observations at regional or even global scales (Reichle et al., 2004, Qin et al., 2015, Srivastava, 2017).

At present, microwave is the main stream for obtaining SM and its dynamics across space and time (Babaeian et al., 2019, Li et al., 2021). Several widely used microwave-based SM products include the Soil Moisture Active Passive (SMAP), the Soil Moisture and Ocean Salinity (SMOS) and the blended dataset of the European Space Agency's Climate Change Initiative (ESA-CCI), which has counted both SMAP and SMOS as inputs. Among these SM products, the ESA-CCI can provide up to 40 years of daily SM data at a spatial resolution of 25 km, which has been widely used in a number of investigations with respect to the global water cycle, weather forecasting and agricultural systems (Burgin et al., 2017, Mishra et al., 2017, Dong et al., 2019, Oozeer et al., 2020, Zhu et al., 2019). Although these microwave-based SM products provide a large amount of data support for better understanding the response and feedback between the Earth and atmosphere, they also reveal several drawbacks, such as low spatial resolution (∼25–50 km) and various data gaps, due to the effects of radio frequency interference and the limitations of algorithms under specific conditions. As a consequence, most of the currently available microwave-based SM products can only provide spatially discontinuous data with coarse resolution.

To this end, an increasing number of studies have highlighted the feasibility of obtaining spatially continuous SM at fine spatiotemporal resolution from disaggregation and data gap prediction processes (Zhou et al., 2017, Chen et al., 2017, Kim et al., 2017, Ajami and Sharma, 2018, Ojha et al., 2019, Zhao and Li, 2013, Zhao et al., 2018). Among the disaggregation algorithms, the Disaggregation based on Physical And Theoretical scale Change (DisPATCh) proposed by Merlin et al. (2012) is one of the most widely used methods, and it has strong applicability in different climate types and soil textures (Djamai et al., 2015, Djamai et al., 2016). Specifically, a number of studies have highlighted the superiority of DisPATCh over other disaggregation approaches (Qu et al., 2021, Fontanet et al., 2018, Neuhauser et al., 2019). For the reconstruction of invalid pixels of SM products, three approaches were commonly implemented: a primary goal is to take advantage of atmospheric reanalysis to obtain SM proxies; another approach is to restore the optical measurements over the pixels contaminated by clouds in advance and to further estimate SM; and the third one is with the machine learning approach (Leng et al., 2019, Jia et al., 2020, He et al., 2021, Acar et al., 2020). The former two approaches general assume a theoretical clear-sky conditions for SSM estimation under actual cloudy pixels, which can probably lead to various uncertainties. Due to the complicated relationships between the SM dynamics and emitted/reflected radiation observed by satellites over cloudy conditions, traditional approaches may reveal little superiority. The motivation behind such matching learning approaches is that they can better simulate the complicated nonlinear relationships between the target variable and other auxiliary data, thus providing a prediction of the target variable with high accuracy (Liang et al., 2019, Liu et al., 2021). In a variety of machine learning methods, the generalized regression neural network (GRNN) has a strong nonlinear mapping ability and learning speed, which can maintain a good prediction with limited sample data and can handle unstable data better. Specifically, the GRNN has been frequently used to predict SM with generally better accuracy than other methods (Cui et al., 2020, Yuan et al., 2020, Liu et al., 2021).

Due to the unique geologic environment and climatic conditions of the Tibetan Plateau (TP) region, no remote sensing-based spatially continuous SM datasets with both high spatial resolution (∼1 km) and frequent temporal interval (∼daily) are currently available, whereas most of the microwave-based SM products reveal low spatial coverage over the TP regions (Zhang and Li, 2018, Du et al., 2020). To this end, the present study aims to provide a practical framework for obtaining daily/1-km spatially continuous SM datasets over the TP. The primary motivation for the proposed study is that the key land surface variables, such as land surface temperature (LST) and normalized difference vegetation index (NDVI), required in both the widely used disaggregation algorithms and matching learning approaches for estimating SM are currently available. Specifically, a number of studies have been dedicated to reconstructing the key land surface variables over cloudy conditions, thus making it possible to obtain spatially continuous parameters with reasonable accuracy (Zhou et al., 2017, Zhang et al., 2019, Gao et al., 2020, Qiu et al., 2021). It is noteworthy that these recent advances in optical-based land surface parameters have significant potential to recover the data gaps for microwave-based SM products, providing an unprecedented opportunity to derive spatially continuous SM datasets at high spatiotemporal resolution (e.g., daily/1-km). Moreover, because the prediction of SM over invalid pixels and disaggregation from coarse to high scales are two main processes for obtaining all-weather SM at fine spatiotemporal resolution, it is also interesting to investigate the performances of two different schemes (i.e., first prediction then disaggregation and first disaggregation then prediction), in consideration that the prediction process appears at different spatial scales in these two schemes. This is important for combing the machine learning-based and physics-based algorithms for estimating spatially continuous SM at high spatial resolution, especially for the TP region with complicated surface condition. The present study is organized as follows: Section 2 presents the materials and methods. The results and discussion are presented in Section 3 and Section 4, respectively. Section 5 concludes the paper.