Abstract
In Gaussian mixture modeling, it is crucial to select the number of Gaussians for a sample set, which becomes much more difficult when the overlap in the mixture is larger. Under regularization theory, we aim to solve this problem using a semi-supervised learning algorithm through incorporating pairwise constraints into entropy regularized likelihood (ERL) learning which can make automatic model selection for Gaussian mixture. The simulation experiments further demonstrate that the presented semi-supervised learning algorithm (i.e., the constrained ERL learning algorithm) can automatically detect the number of Gaussians with a good parameter estimation, even when two or more actual Gaussians in the mixture are overlapped at a high degree. Moreover, the constrained ERL learning algorithm leads to some promising results when applied to iris data classification and image database categorization.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Dattatreya GR (2002) Gaussian mixture parameter estimation with known means and unknown class-dependent variances. Pattern Recognit 35(7): 1611–1616
Render RA, Walker HF (1984) Mixture densities, maximum likelihood and the EM algorithm. SIAM Rev 26(2): 195–239
Akaike H (1974) A new look at the statistical model identification. IEEE Trans Auto Control 19(6): 716–723
Schwarz G (1978) Estimating the dimension of a model. Ann Stat 6(2): 461–464
Dennis DC, Finbarr OS (1990) Asymptotic analysis of penalized likelihood and related estimators. Ann Stat 18(6): 1676–1695
Lu Z (2006) A regularized minimum cross-entropy algorithm on mixtures of experts for time series prediction and curve detection. Pattern Recognit Lett 27(9): 947–955
Lu Z (2007) Entropy regularized likelihood learning on Gaussian mixture: two gradient implementations for automatic model selection. Neural Process Lett 25(1): 17–30
Lu Z (2006) An iterative algorithm for entropy regularized likelihood learning on Gaussian mixture with automatic model selection. Neurocomputing 69(13–15): 1674–1677
Ma J, Xu L (2005) Asymptotic convergence properties of the EM algorithm with respect to the overlap in the mixture. Neurocomputing 68(1–4): 105–129
Shental N, Bar-Hillel A, Hertz T, Weinshall D (2004) Computing Gaussian mixture models with EM using equivalence constraints. Adv Neural Inf Process Syst 16:465–472
Li Y, Shapiro LG, Bilmes JA (2005) A generative/discriminative learning algorithm for image classification. In: Proceedings of the tenth IEEE international conference on computer vision, pp 1605–1612
Manjunath BS, Ma WY (1996) Texture features for browsing and retrieval of image data. IEEE Trans Pattern Anal Mach Intell 18(8): 837–842
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Lu, Z., Peng, Y. A Semi-supervised Learning Algorithm on Gaussian Mixture with Automatic Model Selection. Neural Process Lett 27, 57–66 (2008). https://doi.org/10.1007/s11063-007-9059-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11063-007-9059-4