Optimized ensemble modeling based on feature selection using simple sphere criterion for multi-scale mechanical frequency spectrum

Abstract

Several parameters of industrial processes are indirectly measured by multi-scale mechanical frequency spectrum. Selecting suitable mechanical sub-signals and relevant frequency spectral features for different process parameters remains an open issue. This study proposes a new optimized ensemble model based on feature selection using simple sphere criterion (SSC). Mechanical signals are adaptively decomposed and transformed into frequency spectral data with different timescales. These spectral data are fed into adaptive multi-scale spectral feature selection and modeling framework, in which local-scale frequency spectral features are adaptively selected with concurrent projection to latent structures and SSC based on unscaled data. The optimized ensemble model is constructed with selective information fusion strategy based on reduced frequency spectral data. The feature selection and model learning parameters are jointly selected. Simulation results based on the mechanical vibration and acoustic signals of an experimental laboratory-scale ball mill show the effectiveness of the proposed scheme.

Appendices

Appendix A: PLS algorithm

Projection to latent structures or partial least squares (PLS) constructs a linear multivariable regression model with the extracted latent variables (LVs) from the original input/output data space. It projects input data $\varvec{X} = \{\varvec{x}_{l} \}_{l = 1}^{k}$ and output data $\varvec{Y} = \{\varvec{y}_{l} \}_{l = 1}^{k}$ to a low-dimensional space defined by a small number of LVs $(\varvec{t}_{1}, \ldots,\varvec{t}_{h})$. PLS decomposes the input and output data into the following form:

$$ \left\{{\begin{array}{*{20}l} {\varvec{X} = \sum\nolimits_{i = 1}^{h} {{\varvec{t}}_{i} {\varvec{p}}_{i}} + {\varvec{E}} = {\varvec{TP}}^{\text{T}} + {\varvec{E}}} \hfill \\ {{\varvec{Y}} = \sum\nolimits_{i = 1}^{h} {{\varvec{u}}_{i} {\varvec{q}}_{i}} + {\varvec{F}} = {\varvec{UQ}}^{\text{T}} + {\varvec{F}}} \hfill \\ \end{array}} \right. $$

(A1)

where ${\varvec{T}} = \left[{{\varvec{t}}_{1}, \ldots,{\varvec{t}}_{h}} \right]$ and ${\varvec{U}} = \left[{{\varvec{u}}_{1}, \ldots,{\varvec{u}}_{h}} \right]$ are the latent scores; ${\varvec{P}} = \left[{{\varvec{p}}_{1}, \ldots,{\varvec{p}}_{h}} \right]$ and ${\varvec{Q}} = \left[{{\varvec{q}}_{1}, \ldots,{\varvec{q}}_{h}} \right]$ are the loadings for ${\varvec{X}}$ and ${\varvec{Y}}$, respectively; and ${\varvec{E}}$ and ${\varvec{F}}$ are the residual matrices corresponding to ${\varvec{X}}$ and ${\varvec{Y}}$, respectively. The number of LVs $h$ is usually determined with a cross-validation method. The prediction model based on PLS can be expressed as

$$ \left\{{\begin{array}{*{20}l} {{\varvec{Y}} = {\varvec{XB}} + {\varvec{G}}} \hfill \\ {{\varvec{B}} = {\varvec{X}}^{\text{T}} {\varvec{U}}\left({{\varvec{T}}^{\text{T}} {\varvec{XX}}^{\text{T}} {\varvec{U}}} \right)^{- 1} {\varvec{T}}^{\text{T}}} \hfill \\ \end{array}} \right.. $$

(A2)

However, unlike that with PCA, ${\varvec{T}}$ cannot be presented in terms of the original input data ${\varvec{X}}$ directly. We denote ${\varvec{W}} = [{\varvec{w}}_{ 1},{\varvec{w}}_{ 2}, \ldots,{\varvec{w}}_{\text{h}}]$. Let ${\varvec{R}} = [{\varvec{r}}_{ 1},{\varvec{r}}_{ 2}, \ldots,{\varvec{r}}_{\text{h}}]$, where ${\varvec{r}}_{ 1} = {\varvec{w}}_{ 1}$, for $i > 1$

$$ {\varvec{r}}_{i} = \prod\limits_{{j^{{\prime}} = 1}}^{i - 1} {\left({{\varvec{I}}_{p} - {\varvec{w}}_{{j^{{\prime}}}} {\varvec{p}}_{{j^{{\prime}}}}^{\text{T}}} \right)} {\varvec{w}}_{i} $$

(A3)

with

$$ {\varvec{R}} = ({\varvec{WP}}^{\text{T}} {\varvec{W}})^{- 1} $$

(A4)

The score matrix ${\varvec{T}}$ can be computed as follows:

$$ {\varvec{T}} = {\varvec{XR}} = {\varvec{X}}({\varvec{WP}}^{\text{T}} {\varvec{W}})^{- 1} $$

(A5)

Matrices ${\varvec{R}}$ and ${\varvec{P}}$ has the following relation:

$$ {\varvec{P}}^{T} {\varvec{R}} = {\varvec{R}}^{T} {\varvec{P}} = {\varvec{I}}_{k}. $$

(A6)

Appendix B: CPLS algorithm (Qin and Zheng 2013)

To maximize the covariance between input ${\varvec{X}}$ and output ${\varvec{Y}}$, the traditional PLS algorithm extracts the score ${\varvec{T}}$. However, the score ${\varvec{T}}$ containing variations that are both relative and orthogonal to the output data CPLS is proposed to provide a complete scheme for monitoring the quality and process operation of data.

At first, ${\varvec{T}}$, ${\varvec{Q}}$, and ${\varvec{R}}$ are obtained by using Eqs. (A1) and (A4). Subsequently, “predictable output” ${\hat{\varvec{Y}}}$ is calculated with

$$ {\hat{\varvec{Y}}} = T{\varvec{Q}}^{\text{T}} . $$

(B1)

Singular value decomposition is performed on ${\hat{\varvec{Y}}}$ with

$$ {\hat{\varvec{Y}}} = {\varvec{U}}_{\text{c}} {\varvec{D}}_{\text{c}} {\varvec{V}}_{\text{c}}^{\text{T}} \equiv {\varvec{U}}_{\text{c}} {\varvec{Q}}_{\text{c}}^{\text{T}} $$

(B2)

where ${\varvec{Q}}_{\text{c}} = {\varvec{V}}_{\text{c}} {\varvec{D}}_{\text{c}}$ includes all $h_{\text{c}}$ nonzero singular values in descending order and the corresponding right singular vectors. As ${\varvec{V}}_{\text{c}}$ is orthonormal, ${\varvec{U}}_{\text{c}}$ can be presented as

$$ {\varvec{U}}_{\text{c}} = {\hat{\varvec{Y}V}}_{\rm c} {\varvec{D}}_{\text{c}}^{- 1} = {\varvec{XRQ}}^{\text{T}} {\varvec{V}}_{\rm c} {\varvec{D}}_{\text{c}}^{- 1} = {\varvec{XR}}_{\text{c}} $$

(B3)

where

$$ {\varvec{R}}_{\text{c}} = {\varvec{RQ}}^{\text{T}} {\varvec{V}}_{\rm c} {\varvec{D}}_{\text{c}}^{- 1} $$

(B4)

The unpredictable output is formed and processed with PCA to yield the output-principal scores and output residuals. Finally, the output-irrelevant input is formed and processed with PCA to yield the input-principal scores and input residuals.

According to the above CPLS, the data matrices ${\varvec{X}}$ and ${\varvec{Y}}$ are decomposed as follows:

$$ \left\{{\begin{array}{*{20}c} {{\varvec{X}} = {\varvec{U}}_{\rm c} {\varvec{R}}_{\rm c}^{\diamondsuit} + {\varvec{T}}_{x} {\varvec{P}}_{x}^{\text{T}} + {\tilde{\varvec{X}}}} \\ {{\varvec{Y}} = {\varvec{U}}_{\rm c} {\varvec{Q}}_{\rm c}^{\text{T}} + {\varvec{T}}_{y} {\varvec{P}}_{y}^{\text{T}} + {\tilde{\varvec{Y}}}} \\ \end{array}} \right. $$

(B5)

where ${\varvec{R}}_{\rm c}^{\diamondsuit} = ({\varvec{R}}_{\rm c}^{\text{T}} {\varvec{R}}_{\rm c})^{- 1} {\varvec{R}}_{\rm c}^{T},\,{\varvec{P}}_{x},\,{\varvec{Q}}_{\rm c}$, and ${\varvec{P}}_{y}$ are the loading matrices; score ${\varvec{U}}_{\rm c}$ represents the covariations in the input data ${\varvec{X}}$ that are related to the predictable part ${\hat{\varvec{Y}}}$ of the output data ${\varvec{Y}}$; ${\varvec{T}}_{x}$ represents the variations in ${\varvec{X}}$ that are useless for predicting ${\varvec{Y}}$; ${\varvec{T}}_{y}$ represents the variations in ${\varvec{Y}}$ unpredicted by ${\varvec{X}}$; and ${\tilde{\varvec{X}}}$ and ${\tilde{\varvec{Y}}}$ denote the input residual and output residual, respectively.

Appendix C: List of abbreviations

Abbreviation	Meaning	Abbreviation	Meaning
FFT	Fast Fourier transformation	MBVR	Mill load parameter: material-to-ball volume ratio
EMD	Empirical mode decomposition	PD	Mill load parameter: pulp density
EEMD	Ensemble EMD	CVR	Mill load parameter: charge–volume ratio
IMF	Intrinsic mode function	BCVR	Mill load parameter: ball charge–volume ratio
VIMF	Vibration IMF	GPR	Grinding production ratio
AVIM	Acoustic IMF	MI	Mutual information
SC	Sphere criterion	BB	Branch and bound algorithm
SSC	Simple sphere criterion	AWF	Adaptive weighting fusion algorithm
PLS	Projection to latent structure/partial lest squares	LOOCV	Leave-one-out cross-validation
GA-PLS	Genetic algorithm-PLS	SVM	Support vector machines
CPLS	Concurrent PLS	LS-SVM	Least squares support vector machine
LSPLS	Local-scale PLS	PCA	Principal component analysis
LSCPLS	Local-scale CPLS	SEN	Selective ensemble