Adaptive mixtures: Recursive nonparametric pattern recognition

doi:10.1016/0031-3203(91)90145-U

Pattern Recognition

Volume 24, Issue 12, 1991, Pages 1197-1209

https://doi.org/10.1016/0031-3203(91)90145-U Get rights and content

Abstract

We develop a method of performing pattern recognition (discrimination and classification) using a recursive technique derived from mixture models, kernel estimation and stochastic approximation.

References (19)

D.M. Titterington et al.
Statistical Analysis of Finite Mixture Distributions
(1985)
R.A. Tapia et al.
Nonparametric Probability Density Estimation
(1978)
B.L.S. Prakasa Rao
Nonparametric Functional Estimation
(1983)
M.B. Nevel'son et al.
Stochastic Approximation and Recursive Estimation
T.Y. Young et al.
Classification, Estimation and Pattern Recognition
(1974)
M.A. Aizerman et al.
The probability problem of pattern recognition learning and the method of potential functions
Automn remote Control
(1966)
I.J. Good et al.
Nonparametric roughness penalties for probability densities
Biometrika
(1971)
K. Fukunaga et al.
The reduced parzen classifier
IEEE Trans. Pattern Anal. Mach. Intell.
(1989)
M.G. Kendall
Discrimination and classification

There are more references available in the full text version of this article.

Cited by (31)

Rectification of plant measurements using a statistical framework
2005, Computers and Chemical Engineering
Citation Excerpt :
It also describes the statistical characteristics, e.g. correlation among process variables and the distribution properties of these variables. To enhance the estimation of p(x), the method of adaptive mixtures (Priebe & Marchette, 1991) is used. This method broadens the estimation process resulting from the standard off-line training of p(x) and improves it with on-line updating of p(x).
Data rectification is the process of removing errors from the measured process data and estimating the true state of the plant. In this study, data rectification is posed in a probabilistic framework and historical plant data is utilized to learn the parameters of the plant model. This approach finds the most likely estimates of the true process states by maximizing the probability of the process states given the measurements. Using Bayes’ theorem, this maximization is redefined as the product of the prior probability density function (pdf) of process states and the probability distribution of measurements given the true process states. The technique exploits the existing trade off between these two terms to find the most likely values for the measured process variables.
The method of adaptive mixtures is used for both off-line and on-line estimating and updating of the pdf of the measured process variables. It is a recursive nonparametric method that fits a mixture of Gaussian pdf’s to the data. The changes in the process operating conditions are reflected by adding new components to the mixture.
The maximum likelihood data rectification objective function consists of two pdf’s. One represents the likely process states and the other characterizes the likely adjustments to the measured values. The first pdf, given the measured data condition, is estimated by the expectation–maximization (EM) algorithm or adaptive mixtures. The second pdf is modeled by the product of bimodal Gaussian distributions each representing a process sensor. The resultant complex objective function is then maximized by an iterative EM algorithm and the most likely values of the measured process variables are found. The changes in process operation are reflected in the objective function by updating its terms. The pdf of the measured process states is updated using the rectified points and a robust approach has been developed to update the pdf’s representing the plant sensors operation.
This new approach is implemented for numerical and chemical process examples. The capabilities of the new approach are demonstrated by producing robust estimates for covariance matrices, reliable estimates of mixture probability densities, rejecting the errors in the measurements and yielding reliable rectified values for the measured process variables in these examples.
Multidimensional Density Estimation
2005, Handbook of Statistics
Citation Excerpt :
Other sophisticated procedures for choosing the number of components in mixture models have also been explored. For example, Priebe and Marchette (1991, 1993) and Priebe (1994) discuss what the authors' refer to as “adaptive mixtures” that incorporate the ideas behind both kernel estimators and mixture models and that use a data-based method for adding new terms to a mixture. The adaptive mixture approach can at times overfit data and Solka et al. (1998) combine adaptive mixtures with a pruning step to yield more parsimonious models.
Modern data analysis requires a number of tools to undercover hidden structure. For initial exploration of data, animated scatter diagrams and nonparametric density estimation in many forms and varieties are the techniques of choice. This article focuses on the application of histograms and nonparametric kernel methods to explore data. The details of theory, computation, visualization, and presentation are all described.
Reducing bias in curve estimation by use of weights
1999, Computational Statistics and Data Analysis
A technique is suggested for reducing the order of bias of kernel estimators by weighting the contributions that different data values make to the estimator. The method is developed initially in the context of density estimation, where, unlike the “variable kernel” method proposed by Abramson, our approach does not involve using different bandwidths at different data values. Rather, it is a weighted-bootstrap version of the standard uniform-bootstrap method that is used to construct traditional kernel density estimators. The reduction in bias is achieved by biasing the bootstrap appropriately, in a global rather than local way. Our technique has a variety of different forms, each of which reduces the order of bias from the square to the fourth power of bandwidth, but does not alter the order of variance. It has immediate application to nonparametric regression, where it allows bias to be reduced without prejudicing the one sign of an estimator.
Tracking colour objects using adaptive mixture models
1999, Image and Vision Computing
The advanced distributed region of interest tool
1998, Pattern Recognition
This paper details recent work on the use of low-level features for the identification of regions of interest in images. Using-low-level features, the system classifies regions in the image via probability densities estimates for each class. These densities are estimated semi-parametrically, giving the system great flexibility in the functional form of the densities. This paper details the environment designed to allow easy implementation of pattern recognition algorithms.
Mammographic computer-assisted diagnosis using computational statistics pattern recognition
1995, Real-Time Imaging
Research begun for target identification utilizing pattern recognition has been applied to mammographic computer-assisted diagnosis. The research has utilized the discipline of computational statistics. Feature extraction based on fractals and incorporating segmentation boundaries led to probability density estimation and classification based on discriminant analysis. The results of applying these techniques to mammography are very promising and are reported herein. The results of these limited mammographic studies are discussed in their own light and in comparison with other's work.

View all citing articles on Scopus

View full text

Pattern Recognition

Adaptive mixtures: Recursive nonparametric pattern recognition

Abstract

Statistical Analysis of Finite Mixture Distributions

Nonparametric Probability Density Estimation

Nonparametric Functional Estimation

Stochastic Approximation and Recursive Estimation

Classification, Estimation and Pattern Recognition

The probability problem of pattern recognition learning and the method of potential functions

Automn remote Control

Nonparametric roughness penalties for probability densities

Biometrika

The reduced parzen classifier

IEEE Trans. Pattern Anal. Mach. Intell.

Discrimination and classification

Rectification of plant measurements using a statistical framework

Multidimensional Density Estimation

Reducing bias in curve estimation by use of weights

Tracking colour objects using adaptive mixture models

The advanced distributed region of interest tool

Mammographic computer-assisted diagnosis using computational statistics pattern recognition