Skip to main content
SpringerLink
Log in

Non-Gaussian Data Clustering via Expectation Propagation Learning of Finite Dirichlet Mixture Models and Applications

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Learning appropriate statistical models is a fundamental data analysis task which has been the topic of continuing interest. Recently, finite Dirichlet mixture models have proved to be an effective and flexible model learning technique in several machine learning and data mining applications. In this article, the problem of learning and selecting finite Dirichlet mixture models is addressed using an expectation propagation (EP) inference framework. Within the proposed EP learning method, for finite mixture models, all the involved parameters and the model complexity (i.e. the number of mixture components), can be evaluated simultaneously in a single optimization framework. Extensive simulations using synthetic data along with two challenging real-world applications involving automatic image annotation and human action videos categorization demonstrate that our approach is able to achieve better results than comparable techniques.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. The complete source code of this work is available upon request.

  2. Source code of PCA-SIFT: http://www.cs.cmu.edu/~yke/pcasift.

  3. WordNet is a large lexical database for English, which groups English words into sets of cognitive synonyms called synsets.

  4. http://labelme.csail.mit.edu/.

  5. Datasets are available at: http://vision.eecs.ucf.edu/datasetsActions.html.

References

  1. Blank M, Gorelick L, Shechtman E, Irani M, Basri R (2005) Actions as Space-Time Shapes. In: Proc. of the IEEE International Conference on Computer Vision (ICCV), pp 1395–1402.

  2. Bouguila N, Ziou D (2005) Using unsupervised learning of a finite dirichlet mixture model to improve pattern recognition applications. Pattern Recognit Lett 26(12):1916–1925

    Article  Google Scholar 

  3. Bouguila N, Ziou D (2006a) Online clustering via finite mixtures of dirichlet and minimum message length. Eng Appl Artif Intell 19(4):371–379

    Article  Google Scholar 

  4. Bouguila N, Ziou D (2006b) Unsupervised selection of a finite dirichlet mixture model: an MML-based approach. IEEE Trans Knowl Data Eng 18(8):993–1009

    Article  Google Scholar 

  5. Bouguila N, Ziou D, Vaillancourt J (2004) Unsupervised learning of a finite mixture model based on the dirichlet distribution and its application. IEEE Trans Image Process 13(11):1533–1543

    Article  Google Scholar 

  6. Chang E (2003) CBSA: content-based soft annotation for multinomial image retrieval using bayes point machines. IEEE Trans Circuit Syst Video Technol 13(1):26–38

    Article  Google Scholar 

  7. Chang S, Dasgupta N, Carin L (2005) A Bayesian approach to unsupervised feature selection and density estimation using expectation propagation. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp 1043–1050

  8. Datta R, Ge W, Li J, Wang JZ (2006) Toward bridging the annotation-retrieval gap in image search by a generative modeling approach. In: Proceedings of the 14th annual ACM international conference on multimedia (MM), ACM, pp 977–986

  9. Dollár P, Rabaud V, Cottrell G, Belongie S (2005) Behavior recognition via sparse spatio-temporal feature. In: Proceedings of the IEEE international workshop on visual surveillance and performance evaluation of tracking and surveillance (VS-PETS), pp 65–72

  10. Fan J, Gao Y, Luo H, Xu G (2005) Statistical modeling and conceptualization of natural images. Pattern Recognit 38:865–885

    Article  Google Scholar 

  11. Figueiredo M, Jain A (2002) Unsupervised learning of finite mixture models. IEEE Trans Pattern Anal Mach Intell 24(3):381–396

    Article  Google Scholar 

  12. Heskes T, Zoeter O (2002) Expectation propagation for approximate inference in dynamic Bayesian networks. In: Proceedings of the conference on uncertainty in artificial intelligence (UAI), pp 216–223

  13. Hofmann T (2001) Unsupervised learning by probabilistic latent semantic analysis. Mach Learn 42(1/2):177–196

    Article  MATH  Google Scholar 

  14. Ke Y, Sukthankar R (2004) PCA-SIFT: a more distinctive representation for local image descriptors. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 506–513

  15. Laptev I (2005) On space-time interest points. Int J Comput Vis 64(2/3):107–123

    Article  Google Scholar 

  16. Laptev I, Marszalek M, Schmid C, Rozenfeld B (2008) Learning realistic human actions from movies. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 1–8

  17. Law MHC, Figueiredo MAT, Jain AK (2004) Simultaneous feature selection and clustering using mixture models. IEEE Trans Pattern Anal Mach Intell 26(9):1154–1166

    Article  Google Scholar 

  18. Leacock C, Chodorow M (1998) In: Fellbaum C (Ed) WordNet: an electronic lexical database. MIT Press, pp 305–332

  19. Liu J, Luo J, Shah M (2009) Recognizing realistic actions from videos “In The Wild”. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp 1996–2003

  20. Luo J, Savakis AE, Singhal A (2005) A Bayesian network-based framework for semantic image understanding. Pattern Recognit 38:919–934

    Article  Google Scholar 

  21. Ma Z, Leijon A (2010) Expectation propagation for estimating the parameters of the beta distribution. In: Proceedings of IEEE international conference on acoustics speech and signal processing (ICASSP), pp 2082–2085

  22. Maybeck PS (1982) Stochastic models, estimation and control. Academic Press

  23. McLachlan G, Peel D (2000) Finite mixture models. Wiley, New York

    Book  MATH  Google Scholar 

  24. Mikolajczyk K, Schmid C (2004) Scale and affine invariant interest point detectors. Int J Comput Vis 60:63–86

    Article  Google Scholar 

  25. Miller GA (1995) WordNet: a lexical database for English. Commun ACM 38:39–41

    Article  Google Scholar 

  26. Minka T (2001) Expectation propagation for approximate Bayesian inference. In: Proceedings of the conference on uncertainty in artificial intelligence (UAI), pp 362–369

  27. Minka T, Lafferty J (2002) Expectation-propagation for the generative aspect model. In: Proceedings of the conference on uncertainty in artificial intelligence (UAI), pp 352–359

  28. Naphade MR, Huang TS (2001) A probabilistic framework for semantic video indexing, filtering, and retrieval. IEEE Trans Multimed 3:141–151

    Article  Google Scholar 

  29. Rodriguez M, Ahmed J, Shah M (2008) Action mach a spatio-temporal maximum average correlation height filter for action recognition. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp 1–8

  30. Russell B, Torralba A, Murphy K, Freeman W (2008) LabelMe: a database and Web-based tool for image annotation. Int J Comput Vis 77:157–173

    Article  Google Scholar 

  31. Schüldt C, Laptev I, Caputo B (2004) Recognizing human actions: a local SVM approach. In: Proceedings of the 17th international conference on pattern recognition (ICPR), pp 32–36

  32. Zhao R, Grosky WI (2000) From features to semantics: some preliminary results. In: Proceedings of the IEEE international conference on multimedia and expo (ICME). IEEE Computer Society, pp 679–682

  33. Zhong D, Zhang H, Chang SF (1997) Clustering methods for video browsing and annotation. In: Proceedings of the SPIE conference on storage and retrieval for video and image databases, pp 239–246

Download references

Acknowledgments

The completion of this research was made possible thanks to the Natural Sciences and Engineering Research Council of Canada (NSERC). The authors would like to thank the anonymous referees and the associate editor for their comments. The complete source code of this work is available upon request.

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, Concordia University, Montreal, QC, Canada

    Wentao Fan

  2. The Concordia Institute for Information Systems Engineering (CIISE), Concordia University, Montreal, QC, Canada

    Nizar Bouguila

Authors
  1. Wentao Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nizar Bouguila
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nizar Bouguila.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Fan, W., Bouguila, N. Non-Gaussian Data Clustering via Expectation Propagation Learning of Finite Dirichlet Mixture Models and Applications. Neural Process Lett 39, 115–135 (2014). https://doi.org/10.1007/s11063-013-9293-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-013-9293-x

Keywords

Search

Navigation