Deep learning assisted continuous wavelet transform-based spectrogram for the detection of chlorophyll content in potato leaves

Highlights

• A CNN assisted CWT-based spectrogram strategy is proposed for chlorophyll content detection.

• The first-order derivative was performed to capture detailed spectral information.

• CWT was applied to obtain effective features from spectral data.

• CNN model was applied to explore deep features hidden in spectral data.

Abstract

Visible and near-infrared spectroscopy is a nondestructive method for the chlorophyll content detection of potato crops, in which effective feature extraction is a crucial issue for detecting accuracy in the field. This study aimed to explore comprehensive features to improve the accuracy of chlorophyll content detection in potato leaves. A feature extraction method was proposed by assisting continuous wavelet transform (CWT)-based spectrogram and convolutional neural network (CNN) of the deep learning algorithm. In the experiments, the spectral data in the range of 325–1075 nm were measured in the field. A total of 314 potato leaf samples were collected, and the chlorophyll content was determined in four growth periods. The spectral features in the spatial domain and time–frequency domain were considered in data processing. First, we compared features of first-order derivative (FOD) and Savitzky–Golay smoothing (SG) in the spatial domain to select the preprocessing method. Second, in the time–frequency domain, CWT decomposes spectral data into a series of wavelet coefficient curves at various scales and wavelengths. We supposed that combining both scales and wavelengths of the wavelet coefficients could benefit the detection of the chlorophyll content in potato crops. Thus, the spectrograms were established by transforming 1D wavelet coefficient curves into 2D wavelet coefficient spectrograms. The CNN model was applied to explore potential and comprehensive features in the spectrograms. Finally, partial least squares models were established to compare the detection capability of three features including the spatial domain, wavelet coefficients by CWT, and spectrogram features by CNN. The modeling performances showed that the FOD features obtained the coefficient of determination of 0.7856 in the prediction set (R_P²) and the root mean square error in the prediction set (RMSEP) with 3.9357; the FOD features performed better than the SG features (R_P² = 0.7734, RMSEP = 4.0460); The wavelet coefficients by CWT obtained R_P² = 0.8293 and RMSEP = 3.5117. The PLS model with spectrogram features by the CNN model performed best and achieved R_P² = 0.8749 and RMSEP = 3.0067. It preferably captured the deeper features of spectral data compared with features in the spatial domain. This research provides an effective method for leaf chlorophyll content detection in the precision management of potato crops.

Introduction

The chlorophyll content of potato crops is an important indicator of photosynthetic capacity and nitrogen status related to its underground tuber (Li et al., 2020a). Traditional chlorophyll content measurement relies on laboratory chemical analysis (e.g., ultraviolet spectrophotometry). Although this method can obtain accurate results, it involves disadvantages of destructive sampling and time-consuming operation (Song et al., 2021a). Visible and near-infrared (Vis-NIR) spectroscopy has proved to be a nondestructive and fast method to estimate the chlorophyll content (Li et al., 2020b, Liu et al., 2018, Zhang et al., 2012) due to the large quantities of features that spectral data contain (Liu et al., 2020a). In spectral analysis, extracting effective features from spectral data is one of the significant issues in chlorophyll content detection. This work may contribute efforts in this topic through monitoring physiological and growth status to indicate field management.

The detection of the chlorophyll content based on Vis-NIR spectroscopy generally depends on features in the bands of blue (400–450 nm), green (centered at 550 nm), red (650–700 nm), and red-edge (700–750 nm) regions, which present the strong absorption and reflection characteristics of chlorophyll (Zhao et al. 2021). The spectral reflectance contains complex information because it is not only determined by the concentration of biochemical components but also crop structural properties and field environment conditions (Ustin et al., 2009, Yu et al., 2014). Thus, finding effective features related to the chlorophyll content is confronted with many interference factors, and it is an obstacle in the accurate detection of the potato leaf chlorophyll content (Yao et al., 2013). Consequently, we focus on identifying effective features sensitive to the leaf chlorophyll content and improving the detection accuracy (Mirzaei et al., 2019).

Many attempts have been made to analyze spectrum features based on signal processing methods, mainly in the spatial domain and time–frequency domain (Berger et al., 2020). Considering spatial domain features, the construction of vegetation indices (VIs) and selection of sensitive wavelengths are used in vegetation monitoring. VI combines a group of fixed wavelengths, and each VI exhibits different suitability in practical applications, which can be classified into the simple ratio vegetation index, normalized difference vegetation index, and soil-adjusted vegetation index (Fu et al., 2021, Xue and Su, 2017). In addition, wavelength selection seeks to identify a subset of spectral variables by the removal of non-informative data when performing quantitative determinations between dissimilar samples (Fu et al., 2021). However, Liu et al. (2019) found that the red edge shifts progressively toward longer wavelengths as the chlorophyll content increases. The feature robustness of the selected wavelengths might be influenced by such a shifting phenomenon, so Li et al. (2014) found that normalized difference red-edge index calculated by reflectance at 720 and 790 nm contributes to the nitrogen estimation results with coefficient of determination (R²) of 0.49 in the V6–V7 corn growth period but 0.79 in the V10–V12 period. Similarly, small changes in the data can lead to different variable selection sets (Yang et al., 2019). Previous research indicated that the features at fixed wavelengths do not effectively present complex spectral differences caused by dynamic growth changes of crops, which might reduce their applicability (Qiao et al., 2020). Thus, identification of complicated features from spectral reflectance is critical to improve the modeling performance of chlorophyll content detection.

By extending the spatial domain into the time–frequency domain, continuous wavelet transform (CWT) has been demonstrated as a potential tool in feature selection and weak information extraction (Zhang et al., 2020). CWT utilizes wavelet functions to decompose the spectral data at different scales into wavelet coefficient curves, which represent the degree of similarity between signals and mother wavelet functions (He et al., 2018). Li et al. (2017) extracted red-edge positions from raw spectrum and wavelet-transformed spectrum to estimate the leaf content of cereal crops. The performance based on CWT achieved better validation results with R² of 0.86 than raw spectrum with R² of 0.79. We also applied CWT to detect the chlorophyll content of potato crops (Liu et al., 2020) in primary studies. Liu et al. (2020)) compared the relationship between the wavelet coefficients at different scales and the leaf chlorophyll content; the modeling accuracy at scale 3 performed better (R² = 0.85) than raw spectrum in the spatial domain (R² = 0.75). Although previous research proved that CWT can help provide more spectral features related to the chlorophyll content, the wavelengths of wavelet coefficients with higher correlation coefficients shifted among different scales. The changes among scales in CWT might also indicate the features related to the leaf chlorophyll content. However, most studies focused on features at the single scale and did not involve multi-scales when utilizing CWT. Some questions still need to be explored if features among the different scales are useful in the detection of the chlorophyll content. Therefore, it’s necessary to find complicated features considering both scales and wavelengths.

To present comprehensive information of wavelet coefficients at the single scale and changing phenomenon among different scales, we established spectrograms of wavelet coefficients based on CWT, in which we transformed 1D wavelet coefficient curves into 2D spectrograms. Meanwhile, the method of comprehensive feature exploration will be a challenge in spectrogram processing.

Numerous studies have reported that the convolutional neural network (CNN) model is a powerful tool for images or spectra analysis, especially in deeper feature extraction and recognition (Rong et al., 2020, Yang et al., 2019). Weng et al. (2020) applied the CNN model based on Vis-NIR spectroscopy and obtained the best discrimination results with accuracy of 99.00%, but random forest obtained an accuracy of 91.67% in type identification of minced beef adulteration. CNN can extract high-dimensional data features and reduce dimensionality by interleaving convolutional and pooling layers ((Chen et al., 2018;Kattenborn et al., 2021). In view of the spectrogram, Ng et al. (2019) transformed the 1D spectrum into the 2D spectrogram by Hann window to predict several soil properties; the modeling results showed that R² of the partial least squares model was in the range of 0.87–0.95, and R² of the CNN model was in the range of 0.90–0.95. These studies indicated that CNN has great potential in spectrogram feature extraction because the deep network helps find more hidden signals (Rong et al., 2020, Yang et al., 2019). Therefore, we proposed a strategy to combine the CNN model with spectrograms based on CWT to explore effective features among wavelengths and multi-scales related to the chlorophyll content.

This study aimed to extract comprehensive features by combining the CNN model and CWT-based spectrogram to improve the accuracy of chlorophyll content detection on the basis of Vis-NIR spectroscopy technology. The main procedures of this study were as follows: (1) compare different preprocessing methods to find appropriate features in the spatial domain; (2) perform CWT to extract robust features in the time–frequency domain; and (3) utilize the CNN model with 2D wavelet coefficient spectrogram to extract comprehensive features and construct the regression model for predicting the chlorophyll content of potato leaves.