Skip to main content
SpringerLink
Log in

Model-Based Clustering Based on Variational Learning of Hierarchical Infinite Beta-Liouville Mixture Models

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

In this work, we develop a statistical framework for data clustering which uses hierarchical Dirichlet processes and Beta-Liouville distributions. The parameters of this framework are leaned using two variational Bayes approaches. The first one considers batch settings and the second one takes into account the dynamic nature of real data. Experimental results based on a challenging problem namely visual scenes categorization demonstrate the merits of the proposed framework.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. Source code of PCA-SIFT: http://www.cs.cmu.edu/~yke/pcasift.

  2. Database is available at: http://vision.princeton.edu/projects/2010/SUN/.

References

  1. Andrews JL, McNicholas PD, Subedi S (2011) Model-based classification via mixtures of multivariate t-distributions. Comput Stat Data Anal 55(1):520–529

    Article  MathSciNet  MATH  Google Scholar 

  2. Attias H (1999) A variational Bayes framework for graphical models. In: Proceedings of advances in neural information processing systems (NIPS), pp 209–215

  3. Banerjee A, Langford J (2004) An objective evaluation criterion for clustering. In: Proceedings of the ACM SIGKDD international conference on knowledge discovery and data mining (KDD). ACM, pp 515–520

  4. Bishop CM (2006) Pattern recognition and machine learning. Springer, Heidelberg

    MATH  Google Scholar 

  5. Blei DM, Jordan MI (2005) Variational inference for Dirichlet process mixtures. Bayesian Anal 1:121–144

    Article  MathSciNet  MATH  Google Scholar 

  6. Bouguila N (2011) Bayesian hybrid generative discriminative learning based on finite liouville mixture models. Pattern Recog 44(6):1183–1200

    Article  MATH  Google Scholar 

  7. Bouguila N (2012a) Hybrid generative/discriminative approaches for proportional data modeling and classification. IEEE Trans Knowl Data Eng 24(12):2184–2202

    Article  Google Scholar 

  8. Bouguila N (2012b) Infinite liouville mixture models with application to text and texture categorization. Pattern Recog Lett 33(2):103–110

    Article  Google Scholar 

  9. Bouguila N, Ziou D (2005) Using unsupervised learning of a finite Dirichlet mixture model to improve pattern recognition applications. Pattern Recog Lett 26(12):1916–1925

    Article  Google Scholar 

  10. Bouguila N, Ziou D, Vaillancourt J (2004) Unsupervised learning of a finite mixture model based on the Dirichlet distribution and its application. IEEE Trans Image Process 13(11):1533–1543

    Article  Google Scholar 

  11. Boutell MR, Luo J, Shen X, Brown CM (2004) Learning multi-label scene classification. Pattern Recog 37(9):1757–1771

    Article  Google Scholar 

  12. Cheng H, Jiang X, Sun Y, Wang J (2001) Color image segmentation: advances and prospects. Pattern Recog 34(12):2259–2281

    Article  MATH  Google Scholar 

  13. Cohn DA, Ghahramani Z, Jordan MI (1996) Active learning with statistical models. J Artif Intell Res 4:129–145

    MATH  Google Scholar 

  14. Corridoni JM, Bimbo AD, Pala P (1999) Image retrieval by color semantics. Multimed Syst 7(3):175–183

    Article  Google Scholar 

  15. Erdem C, Karabulut G, Yanmaz E, Anarim E (2001) Motion estimation in the frequency domain using fuzzy c-planes clustering. IEEE Trans Image Process 10(12):1873–1879

    Article  MATH  Google Scholar 

  16. Fan J, Gao Y, Luo H, Keim DA, Li Z (2008) A novel approach to enable semantic and visual image summarization for exploratory image search. In: Proceedings of the 1st ACM international conference on multimedia information retrieval (MIR). ACM, pp 358–365

  17. Fan W, Bouguila N (2013a) Learning finite Beta-liouville mixture models via variational Bayes for proportional data clustering. In: Proceedings of the 23rd international joint conference on artificial intelligence (IJCAI)

  18. Fan W, Bouguila N (2013b) Variational learning of finite Beta-liouville mixture models using component splitting. In: Proceedings of the international joint conference on neural networks (IJCNN), pp 1–8

  19. Fan W, Bouguila N (2014) Non-gaussian data clustering via expectation propagation learning of finite Dirichlet mixture models and applications. Neural Process Lett 39(2):115–135

    Article  Google Scholar 

  20. Fan W, Bouguila N, Ziou D (2012) Variational learning for finite Dirichlet mixture models and applications. IEEE Trans Neural Netw Learn Syst 23(5):762–774

    Article  Google Scholar 

  21. Ferguson TS (1983) Bayesian density estimation by mixtures of normal distributions. Recent Adv Stat 24:287–302

    Article  MathSciNet  MATH  Google Scholar 

  22. Graepel T, Herbrich R (2008) Large scale data analysis and modelling in online services and advertising. In: Proceedings of the ACM SIGKDD international conference on knowledge discovery and data mining (KDD), ACM, pp 2–2

  23. Hegerath A, Deselaers T, Ney H (2006) Patch-based object recognition using discriminatively trained gaussian mixtures. In: Proceedings of the British machine vision conference (BMVC), pp 519–528

  24. Ishwaran H, James LF (2001) Gibbs sampling methods for stick-breaking priors. J Am Stat Assoc 96:161–173

    Article  MathSciNet  MATH  Google Scholar 

  25. Ke Y, Sukthankar R (2004) Pca-sift: a more distinctive representation for local image descriptors. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition (CVPR), pp II–506–II–513 Vol 2

  26. Korwar RM, Hollander M (1973) Contributions to the theory of Dirichlet processes. Ann Probab 1:705–711

    Article  MathSciNet  MATH  Google Scholar 

  27. Kushner H, Yin G (1997) Stochastic approximation algorithms and applications, applications of mathematics. Springer, Berlin

    Book  MATH  Google Scholar 

  28. Laaksonen J, Koskela M, Oja E (2002) Picsom-self-organizing image retrieval with mpeg-7 content descriptors. IEEE Trans Neural Netw 13(4):841–853

    Article  MATH  Google Scholar 

  29. Lampert C, Nickisch H, Harmeling S (2009) Learning to detect unseen object classes by between-class attribute transfer. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 951–958

  30. Liu X, Fu H, Jia Y (2008) Gaussian mixture modeling and learning of neighboring characters for multilingual text extraction in images. Pattern Recog 41(2):484–493

    Article  MATH  Google Scholar 

  31. Liu Z, Song YQ, Chen JM, Xie CH, Zhu F (2012) Color image segmentation using nonparametric mixture models with multivariate orthogonal polynomials. Neural Comput Appl 21(4):801–811

    Article  Google Scholar 

  32. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Article  Google Scholar 

  33. Lu G (2002) Techniques and data structures for efficient multimedia retrieval based on similarity. IEEE Trans Multimed 4(3):372–384

    Article  Google Scholar 

  34. Luo J, Boutell M, Gray R, Brown C (2005) Image transform bootstrapping and its applications to semantic scene classification. IEEE Trans Syst Man Cybern Part B: Cybern 35(3):563–570

    Article  Google Scholar 

  35. Ma Z, Leijon A (2010) Expectation propagation for estimating the parameters of the Beta distribution. In: Proceedings IEEE international conference on acoustics speech and signal processing (ICASSP), pp 2082–2085

  36. Mancas-Thillou C, Gosselin B (2007) Color text extraction with selective metric-based clustering. Comput Vis Image Underst 107(1–2):97–107

    Article  Google Scholar 

  37. Maybeck PS (1982) Stochastic models, estimation and control. Academic Press, New York

    MATH  Google Scholar 

  38. McNicholas PD (2010) Model-based classification using latent gaussian mixture models. Stat Plan Inference 140(5):1175–1181

    Article  MathSciNet  MATH  Google Scholar 

  39. Minka T (2001) Expectation propagation for approximate Bayesian inference. In: Proceedings of the conference on uncertainty in artificial intelligence (UAI), pp 362–369

  40. Minka T, Lafferty J (2002) Expectation propagation for the generative aspect model. In: Proceedings of the conference on uncertainty in artificial intelligence (UAI), pp 352–359

  41. Mojsilovic A, Rogowitz B (2004) Semantic metric for image library exploration. IEEE Trans Multimed 6(6):828–838

    Article  Google Scholar 

  42. Mojsilovic A, Kovacevic J, Hu J, Safranek R, Ganapathy S (2000) Matching and retrieval based on the vocabulary and grammar of color patterns. IEEE Trans Image Process 9(1):38–54

    Article  Google Scholar 

  43. Nilsback ME, Zisserman A (2006) A visual vocabulary for flower classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), vol 2, pp 1447–1454

  44. Ozuysal M, Fua P, Lepetit V (2007) Fast keypoint recognition in ten lines of code. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 1–8

  45. Park SH, Yun ID, Lee SU (1998) Color image segmentation based on 3-d clustering: morphological approach. Pattern Recog 31(8):1061–1076

    Article  Google Scholar 

  46. Robert C, Casella G (1999) Monte Carlo statistical methods. Springer, New York

    Book  MATH  Google Scholar 

  47. Santago P, Gage H (1995) Statistical models of partial volume effect. IEEE Trans Image Process 4(11):1531–1540

    Article  Google Scholar 

  48. Sato M (2001) Online model selection based on the variational Bayes. Neural Comput 13:1649–1681

    Article  MATH  Google Scholar 

  49. Schweitzer H (1999) Organizing image databases as visual-content search trees. Image Vis Comput 17(7):501–511

    Article  MathSciNet  Google Scholar 

  50. Seemann E, Leibe B, Mikolajczyk K, Schiele B (2005) An evaluation of local shape-based features for pedestrian detection. In: Proceedings of the British machine vision conference (BMVC)

  51. Sethuraman J (1994) A constructive definition of Dirichlet priors. Statistica Sinica 4:639–650

    MathSciNet  MATH  Google Scholar 

  52. Souden M, Kinoshita K, Nakatani T (2013) An integration of source location cues for speech clustering in distributed microphone arrays. In: Proceedings of the IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 111–115

  53. Souden M, Kinoshita K, Delcroix M, Nakatani T (2014) Location feature integration for clustering-based speech separation in distributed microphone arrays. IEEE/ACM Trans Audio Speech Lang Process 22(2):354–367

    Article  Google Scholar 

  54. Teh YW, Jordan MI (2010) Hierarchical Bayesian nonparametric models. In: Hjort N, Holmes C, Müller P, Walker S (eds) Bayesian nonparametrics: principles and practice. Cambridge University Press, Cambridge

    Google Scholar 

  55. Teh YW, Jordan MI, Beal MJ, Blei DM (2006) Hierarchical Dirichlet processes. J Am Stat Assoc 101(476):1566–1581

    Article  MathSciNet  MATH  Google Scholar 

  56. Thureson J, Carlsson S (2004) Appearance based qualitative image description for object class recognition. In: Pajdla T, Matas J (eds) ECCV (2), Springer, Lecture notes in computer science, vol 3022, pp 518–529

  57. Wang C, Paisley JW, Blei DM (2011) Online variational inference for the hierarchical Dirichlet process. J Mach Learn Res—Proc Track 15:752–760

    Google Scholar 

  58. Wu Y, Huang TS (2000) Self-supervised learning for visual tracking and recognition of human hand. In: Proceedings of the 7th national conference on artificial intelligence and twelfth conference on on innovative applications of artificial intelligence (AAAI/IAAI), pp 243–248

  59. Xiao J, Hays J, Ehinger K, Oliva A, Torralba A (2010) Sun database: large-scale scene recognition from abbey to zoo. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 3485–3492

Download references

Acknowledgments

The completion of this work was supported by the Scientific Research Funds of Huaqiao University (600005-Z15Y0016). The authors would like to thank the anonymous referees and the associate editor for their comments.

Author information

Authors and Affiliations

  1. Department of Computer Science and Technology, Huaqiao University, Xiamen, China

    Wentao Fan

  2. The Concordia Institute for Information Systems Engineering (CIISE), Concordia University, Montreal, QC, Canada

    Nizar Bouguila

Authors
  1. Wentao Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nizar Bouguila
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wentao Fan.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fan, W., Bouguila, N. Model-Based Clustering Based on Variational Learning of Hierarchical Infinite Beta-Liouville Mixture Models. Neural Process Lett 44, 431–449 (2016). https://doi.org/10.1007/s11063-015-9466-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-015-9466-x

Keywords

Search

Navigation