A data driven procedure for density estimation with some applications

doi:10.1016/0031-3203(96)00028-3

Pattern Recognition

Volume 29, Issue 10, October 1996, Pages 1719-1736

https://doi.org/10.1016/0031-3203(96)00028-3 Get rights and content

Abstract

This paper deals with the probability density estimation using a kernel-based approach where the window size of the kernel is found by a data-driven procedure. It is theoretically shown that, under certain assumptions, the estimated densities on bounded sets can be asymptotically unbiased when the width of window is obtained from the minimal spanning tree of the observed data. The theoretical development initially carried out on R² is applicable to higher dimensional spaces. The results are experimentally verified on bounded sets with different types of distributions. The behaviour of the estimator in the case of the unbounded set as in that for Gaussian density is also experimentally seen to be good. Some applications of the proposed density estimation technique is demonstrated. One application is the representative point detection algorithm, which can be applied for data reduction and outlier rejection. Another application involves detection of border points of a dot pattern as well as finding a thinned version of the dot pattern.

References (35)

S.J. Sheather
A data-based algorithm for choosing the window width when estimating the density at a point
Comput. Statist. Data Anal.
(1983)
S.J. Sheather
An improved data-based algorithm for choosing the window width when estimating the density at a point
Comput. Statist. Data Anal.
(1986)
A.K. Jain et al.
Classifier design with Parzen windows
Pattern Recognition Artif. Intell.
(1988)
V. Ryzin
On a histogram method of density estimation
Commun. Statist.
(1973)
L.I. Boneva et al.
Spline transformations
J. R. Statist. Soc. Ser. B
(1971)
E. Parzen
On the estimation of a probability density function and the mode
Ann. Math. Statist.
(1962)
T. Cacoullos
Estimation of a multivariate density
Ann. Inst. Statist. Math.
(1966)
D.O. Loftsgaarden et al.
A non-parametric estimate of a multivariate density function
Ann. Math. Statist.
(1965)
K. Fukunaga et al.
Optimization of k-nearest neighbor density estimates
IEEE Trans. Inform. Theory
(1973)
N.N. Cencov
Evaluation of an unknown distribution density from observations
Soviet Math.
(1962)

J.B. Kruskal

On the shortest spanning subtree of a graph and the travelling salesman problem

B.B. Winter

Rate of strong consistency of two non-parametric density estimators

Ann. Statist.

(1975)

I.J. Good et al.

Nonparametric roughness penalties for probability densities

Biometrika

(1971)

R.L. Kashyap et al.

Estimation of probability density and distribution function

IEEE Trans. Inform. Theory

(1968)

W. Wertz

Statistical Density Estimation—a Survey

(1978)

B.L.S. Prakasa Rao

Nonparametric Functional Estimation

(1983)

C.A. Murthy

On consistent estimation of classes in S² in the context of cluster analysis

Cited by (7)

On bandwidth selection using minimal spanning tree for kernel density estimation
2016, Computational Statistics and Data Analysis
The use of kernel density estimation is quite well known in large variety of machine learning applications like classification, clustering, feature selection, etc. One of the major issues in the construction of kernel density estimators is the tuning of bandwidth parameter. Most of the bandwidth selection procedures optimize mean integrated squared or absolute error, which require huge computational time as the size of the data increases. Here, the bandwidth has been taken to be a function of inter-point distances of the data set. It is defined as a function of the length of Euclidean Minimal Spanning Tree of the given sample points. No rigorous theory about the asymptotic properties of the EMST based density estimator has been developed in the literature. Theoretical analysis of the asymptotic properties of the EMST based density estimator has been established and proved that the estimator is asymptotically unbiased to the original density at its every continuity point. Moreover, theoretical analysis has been provided for general kernel. Experiments are conducted using both synthetic and real-life data sets to compare the performance of the EMST bandwidth to those of conventional cross-validation and plug-in bandwidth selectors. It is found that the EMST based estimator achieves the comparative performance, while being simpler and faster than the conventional estimators.
Multivariate online kernel density estimation with Gaussian kernels
2011, Pattern Recognition
Citation Excerpt :
Non-parametric methods such as Parzen kernel density estimators (KDE) [4–6] alleviate this problem by treating each observation as a component in the mixture model. There have been several studies on how to efficiently estimate the bandwidth of each component (e.g., [7–12]) and to incorporate the measurement noise into the estimated bandwidths, e.g., [13]. Several researchers have recognized the drawbacks of using same bandwidth for all components.
We propose a novel approach to online estimation of probability density functions, which is based on kernel density estimation (KDE). The method maintains and updates a non-parametric model of the observed data, from which the KDE can be calculated. We propose an online bandwidth estimation approach and a compression/revitalization scheme which maintains the KDE's complexity low. We compare the proposed online KDE to the state-of-the-art approaches on examples of estimating stationary and non-stationary distributions, and on examples of classification. The results show that the online KDE outperforms or achieves a comparable performance to the state-of-the-art and produces models with a significantly lower complexity while allowing online adaptation.
Exploratory data analysis of evoked response single trials based on minimal spanning tree
2001, Clinical Neurophysiology
Objective: An exploratory data analysis framework, based on minimal spanning tree, is proposed as a means to support the analysis of single trial (ST) electrophysiological signals. The core of this framework is the compact description of the input ST sample in a form of content-dependent ordered lists. Based on the established hierarchies, efficient ways to increase the SNR, extract prototypical responses, visualize possible self-organization trends in the sample and track the course of evoked response along the trial-to-trial dimension, are proposed.
Method: Magnetoencephalographic auditory evoked responses were used for demonstrating and validating the introduced framework.
Results and conclusion: The results demonstrate the benefits, from this intelligent manipulation of STs, in understanding and enhancing the actual evoked signal. Specifically we find support for stimulus-induced phase-resetting hypothesis in the 3–20 Hz band, the existence of trials void of the prototypical evoked response, and an order across the single trial set hinting at an underlying process with long time scale.
A novel multiseed nonhierarchical data clustering technique
1997, IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Nonparametric density estimation based on self-organizing incremental neural network for large noisy data
2017, IEEE Transactions on Neural Networks and Learning Systems
Online discriminative kernel density estimator with gaussian kernels
2014, IEEE Transactions on Cybernetics

View all citing articles on Scopus

View full text

A data driven procedure for density estimation with some applications

Abstract

Comput. Statist. Data Anal.

Comput. Statist. Data Anal.

Pattern Recognition Artif. Intell.

On a histogram method of density estimation

Commun. Statist.

Spline transformations

J. R. Statist. Soc. Ser. B

On the estimation of a probability density function and the mode

Ann. Math. Statist.

Estimation of a multivariate density

Ann. Inst. Statist. Math.

A non-parametric estimate of a multivariate density function

Ann. Math. Statist.

Optimization of k-nearest neighbor density estimates

IEEE Trans. Inform. Theory

Evaluation of an unknown distribution density from observations

Soviet Math.

On the shortest spanning subtree of a graph and the travelling salesman problem

Rate of strong consistency of two non-parametric density estimators

Ann. Statist.

Nonparametric roughness penalties for probability densities

Biometrika

Estimation of probability density and distribution function

IEEE Trans. Inform. Theory

Statistical Density Estimation—a Survey

Nonparametric Functional Estimation

On consistent estimation of classes in S2 in the context of cluster analysis

On consistent estimation of classes in S² in the context of cluster analysis