Skip to main content
SpringerLink
Log in

Stochastic Approximation Algorithm with Randomization at the Input for Unsupervised Parameters Estimation of Gaussian Mixture Model with Sparse Parameters

  • Stochastic Systems
  • Published:
Automation and Remote Control Aims and scope Submit manuscript

Abstract

We consider the possibilities of using stochastic approximation algorithms with randomization on the input under unknown but bounded interference in studying the clustering of data generated by a mixture of Gaussian distributions. The proposed algorithm, which is robust to external disturbances, allows us to process the data “on the fly” and has a high convergence rate. The operation of the algorithm is illustrated by examples of its use for clustering in various difficult conditions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Polyak, B.T., Vvedenie v optimizatsiyu, Moscow: Nauka, 1983. Translated under the title Introduction to Optimization, New York: Optimization Software, 1987.

    MATH  Google Scholar 

  2. Robbins, H. and Monro, S., A Stochastic Approximation Method, Ann. Math. Stat., 1951, pp. 400–407.

    Google Scholar 

  3. Kiefer, J. and Wolfowitz, J., Stochastic Estimation of the Maximum of a Regression Function, Ann. Math. Stat., 1952, 23, 3, pp. 462–466.

    Article  MATH  Google Scholar 

  4. Blum, J.R., Multidimensional Stochastic Approximation Methods, Ann. Math. Stat., 1954, 25, 4, pp. 737–744.

    Article  MATH  Google Scholar 

  5. Granichin, O.N., A Stochastic Recursive Procedure with Correlated Noises in the Observation, That Employs Trial Perturbations at the Input, Vestn. Leningr. Univ., Ser. 1, 1989, no. 1(4), pp. 19–21.

    MathSciNet  MATH  Google Scholar 

  6. Granichin, O.N., Procedure of Stochastic Approximation with Disturbances at the Input, Autom. Remote Control, 1992, 53, 2, pp. 232–237.

    MathSciNet  Google Scholar 

  7. Polyak, B.T. and Tsybakov, A.B., Optimal Orders of Accuracy for Search Algorithms of Stochastic Optimization, Probl. Peredachi Inf., 1990, 26, 2, pp. 45–53.

    Google Scholar 

  8. Rastrigin, L.A., Statisticheskie metody poiska (Statistical Search Methods), Moscow: Nauka, 1968.

    Google Scholar 

  9. Spall, J.C., Multivariate Stochastic Aproximation Using a Simultaneous Perturbation Gradient Aproximation, IEEE Trans. Autom. Control, 1992, vol. 37, no. 3, pp. 332–341.

    Article  Google Scholar 

  10. Granichin, O.N., Stochastic Approximation Search Algorithms with Randomization at the Input, Autom. Remote Control, 2015, 76, 5, pp. 762–775.

    Article  MathSciNet  MATH  Google Scholar 

  11. Granichin, O., Volkovich, Z., and Toledano-Kitai, D., Randomized Algorithms in Automatic Control and Data Mining, Berlin: Springer, 2015.

    Book  Google Scholar 

  12. Lloyd, S., Least Squares Quantization in PCM, IEEE Trans. Inform. Theory, 1982, 28, 2, pp. 129–136.

    Article  MathSciNet  MATH  Google Scholar 

  13. Shindler, M., Wong, A., and Meyerson, A., Fast and Accurate k-means For Large Datasets, Proc. 24th NIPS Conf., 2011.

    Google Scholar 

  14. Sculley, D., Web Scale K-Means Clustering, Proc. 19th WWW Conf., Raleigh, North Carolina, 2010.

    Google Scholar 

  15. Katselis, D., Beck, C.L., and van der Schaar, M., Ensemble Online Clustering through Decentralized Observations, Proc. 53rd IEEE CDC, Los Angeles, 2014, pp. 910–915.

    Google Scholar 

  16. Kaufman, L. and Rousseeuw, P., Finding Groups in Data: An Introduction to Cluster Analysis, New York: Wiley, 1990.

    Book  MATH  Google Scholar 

  17. Granichin, O.N., and Izmakova, O.A., A Randomized Stochastic Approximation Algorithm for Self-Learning, Autom. Remote Control, 2005, 66, 8, pp. 1239–1248.

    Article  MathSciNet  MATH  Google Scholar 

  18. Dempster, A., Laird, N., and Rubin, D., Maximum Likelihood from Incomplete Data via the EM Algorithm, J. Royal Stat. Soc. Ser. B, 1977, 39, 1, pp. 1–38.

    MathSciNet  MATH  Google Scholar 

  19. Bishop, C.M., Pattern Recognition and Machine Learning, New York: Springer, 2006.

    MATH  Google Scholar 

  20. Song, M. and Wang, H., Highly Efficient Incremental Estimation of GMM for Online Data Stream Clust, Proc. SPIE III, 2005, pp. 174–184.

    Google Scholar 

  21. Frey, B.J. and Dueck, D., Clustering by Passing Messages between Data Points, Sci., 2007, no. 315(5814), pp. 972–976.

    Article  MathSciNet  MATH  Google Scholar 

  22. Ester, M., Kriegel, H.P., Sander, J., and Xu, X., A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise, Proc. 2nd Int. Conf. on Knowledge Discovery and Data Mining, Portland, 1996, pp. 226–231.

    Google Scholar 

  23. Huber, P.J., Robust Statistics, New York: Wiley, 1981.

    Book  MATH  Google Scholar 

  24. Hubert, L. and Arabie, P., Comparing Partitions, J. Classif., 1985, vol. 2, no. 1, pp. 193–218.

    Article  MATH  Google Scholar 

  25. Dahlin, J., Wills, A., and Ninness, B., Sparse Bayesian ARX Models with Flexible Noise Distributions, Proc. 18th IFAC Sympos. on System Identification, Stockholm, 2018, pp. 25–30.

    Google Scholar 

Download references

Acknowledgments

This work was partially supported by the Russian Science Foundation, project no. 16-19-00057.

Author information

Authors and Affiliations

  1. St. Petersburg State University, St. Petersburg, Russia

    A. A. Boiarov & O. N. Granichin

  2. Institute for Problems of Mechanical Engineering, Russian Academy of Sciences, St. Petersburg, Russia

    A. A. Boiarov & O. N. Granichin

Authors
  1. A. A. Boiarov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. O. N. Granichin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to A. A. Boiarov or O. N. Granichin.

Additional information

Russian Text © The Author(s), 2019, published in Avtomatika i Telemekhanika, 2019, No. 8, pp. 44–63.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Boiarov, A.A., Granichin, O.N. Stochastic Approximation Algorithm with Randomization at the Input for Unsupervised Parameters Estimation of Gaussian Mixture Model with Sparse Parameters. Autom Remote Control 80, 1403–1418 (2019). https://doi.org/10.1134/S0005117919080034

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S0005117919080034

Keywords

Search

Navigation