Abstract
Discriminative models have been shown to be more advantageous for pattern recognition problem in machine learning. For this study, the main focus is developing a new hybrid model that combines the advantages of a discriminative technique namely the support vector machines (SVM) with the full efficiency offered through covariance multivariate generalized Gaussian mixture models (MGGMM). This new hybrid MGGMM applies the Fisher and Kullback–Leibler kernels derived from MGGMM to improve the kernel function of SVM. This approach is based on two different learning techniques explicitly: the Fisher scoring algorithm and the Bayes inference technique based on Markov Chain Monte Carlo and Metropolis–Hastings algorithm. These learning methods work with two model selection approaches (minimum message length and marginal likelihood) to determine the number of clusters. The effectiveness of the framework is demonstrated through extensive experiments including synthetic datasets, facial expression recognition and human activity recognition.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Adama DA, Lotfi A, Langensiepen C, Lee K, Trindade P (2018) Human activity learning for assistive robotics using a classifier ensemble. Soft Comput 22(21):7027–7039
Akaike H (1974) A new look at the statistical model identification. IEEE Trans Autom Control 19(6):716–723
Bartlett MS, Littlewort G, Fasel I, Movellan JR (2003) Real time face detection and facial expression recognition: development and applications to human computer interaction. In: Conference on computer vision and pattern recognition workshop, 2003. CVPRW’03, vol. 5. IEEE, pp 53–53
Baxter RA, Oliver JJ (2000) Finding overlapping components with mml. Stat Comput 10(1):5–16
Bouguila N (2011) Bayesian hybrid generative discriminative learning based on finite liouville mixture models. Pattern Recognit 44(6):1183–1200
Bouguila N (2012) Hybrid generative/discriminative approaches for proportional data modeling and classification. IEEE Trans Knowl Data Eng 24(12):2184–2202
Bouguila N, Ziou D (2007) High-dimensional unsupervised selection and estimation of a finite generalized Dirichlet mixture model based on minimum message length. IEEE Trans Pattern Anal Mach Intell 29(10):1716–1731
Boukouvalas Z, Fu GS, Adalı T (2015) An efficient multivariate generalized gaussian distribution estimator: Application to iva. In: 49th Annual conference on information sciences and systems (CISS), 2015. IEEE, pp 1–4
Bourouis S, Al-Osaimi FR, Bouguila N, Sallay H, Aldosari F, Al Mashrgy M (2019) Bayesian inference by reversible jump mcmc for clustering based on finite generalized inverted dirichlet mixtures. Soft Comput 23(14):5799–5813
Carlo M (1992) Comment: one long run with diagnostics: implementation strategies for Markov chain. Stat Sci 7(4):493–497
Chib S, Greenberg E (1995) Understanding the Metropolis–Hastings algorithm. Am Stat 49(4):327–335
Cohen I, Sebe N, Cozman FG, Huang TS (2003) Semi-supervised learning for facial expression recognition. In: Proceedings of the 5th ACM SIGMM international workshop on Multimedia information retrieval. ACM, pp 17–22
Conway JH, Sloane NJA (2013) Sphere packings, lattices and groups, vol 290. Springer, Berlin
Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the em algorithm. J R Stat Soc Ser B (Methodol) 39(1):1–22
Dollár P, Rabaud V, Cottrell G, Belongie S (2005) Behavior recognition via sparse spatio-temporal features. VS-PETS, Beijing
Elguebaly T, Bouguila N (2015) A hierarchical nonparametric Bayesian approach for medical images and gene expressions classification. Soft Comput 19(1):189–204
Fan W, Bouguila N (2013) Online facial expression recognition based on finite beta-liouville mixture models. In: 2013 International conference on computer and robot vision (CRV). IEEE, pp 37–44
Fan W, Bouguila N (2014) Variational learning for Dirichlet process mixtures of Dirichlet distributions and applications. Multimed Tools Appl 70(3):1685–1702
Fan W, Sallay H, Bouguila N, Bourouis S (2016) Variational learning of hierarchical infinite generalized Dirichlet mixture models and applications. Soft Comput 20(3):979–990
Figueiredo MAT, Jain AK (2002) Unsupervised learning of finite mixture models. IEEE Trans Pattern Anal Mach Intell 24(3):381–396
Gelman A, Stern HS, Carlin JB, Dunson DB, Vehtari A, Rubin DB (2013) Bayesian data analysis. Chapman and Hall/CRC, Boca Raton
Hershey JR, Olsen PA (2007) Approximating the kullback leibler divergence between gaussian mixture models. In: IEEE international conference on acoustics, speech and signal processing, 2007. ICASSP 2007. vol. 4. IEEE, pp IV–317
Jaakkola T, Haussler D (1999) Exploiting generative models in discriminative classifiers. In: Advances in neural information processing systems, pp 487–493
Kanade T, Tian Y, Cohn JF (2000) Comprehensive database for facial expression analysis. In: Proceedings fourth IEEE international conference on automatic face and gesture recognition. IEEE, p 46
Kass RE, Raftery AE (1995) Bayes factors. J Am Stat Assoc 90(430):773–795
Kelker D (1970) Distribution theory of spherical distributions and a location-scale parameter generalization. Sankhyā Indian J Stat Ser A 32:419–430
Kotz S (1975) Multivariate distributions at a cross-road. Stat Distrib Sci Work 1:247–270
Lajevardi SM, Hussain ZM (2009) Zernike moments for facial expression recognition. rn 2, 3
Lindley DV and Rao CR (1953) Advanced statistical methods in biometric research. J R Stat Soc 116(1):86–87
Marin JM, Robert C (2007) Bayesian core: a practical approach to computational Bayesian statistics. Springer, Berlin
Moreno PJ, Ho PP, Vasconcelos N (2004) A Kullback–Leibler divergence based kernel for svm classification in multimedia applications. In: Advances in neural information processing systems, pp 1385–1392
Najar F, Bourouis S, Bouguila N, Belghith S (2018) A fixed-point estimation algorithm for learning the multivariate ggmm: application to human action recognition. In: 2018 IEEE Canadian conference on electrical & computer engineering (CCECE). IEEE, pp 1–4
Najar F, Bourouis S, Bouguila N, Belghith S (2019) Unsupervised learning of finite full covariance multivariate generalized Gaussian mixture models for human activity recognition. Multimed Tools Appl 78:1–23
Neal RM (1992) Bayesian mixture modeling. In: Maximum entropy and Bayesian methods. Springer, pp. 197–211
Niebles JC, Wang H, Fei-Fei L (2008) Unsupervised learning of human action categories using spatial-temporal words. Int J Comput Vis 79(3):299–318
Pascal F, Bombrun L, Tourneret JY, Berthoumieu Y (2013) Parameter estimation for multivariate generalized gaussian distributions. IEEE Trans Signal Process 61(23):5960–5971
Rissanen J (1978) Modeling by shortest data description. Automatica 14(5):465–471
Robert C (2007) The Bayesian choice: from decision-theoretic foundations to computational implementation. Springer, Berlin
Robert C, Casella G (2000) Monte carlo statistical methods. Springer Text in Statistics, Springer. https://doi.org/10.1007/978-1-4757-4145-2
Roberts GO, Tweedie RL (1999) Bounds on regeneration times and convergence rates for Markov chains. Stoch Process Appl 80(2):211–229
Roh SB, Oh SK, Yoon JH, Seo K (2018) Design of face recognition system based on fuzzy transform and radial basis function neural networks. Soft Comput 23:1–17
Schuldt C, Laptev I, Caputo B (2004) Recognizing human actions: a local SVM approach. In: Proceedings of the 17th international conference on pattern recognition, 2004. ICPR 2004. vol. 3. IEEE, pp 32–36
Tsai HH, Chang YC (2018) Facial expression recognition using a combination of multiple facial features and support vector machine. Soft Comput 22(13):4389–4405
Verdoolaege G, Rosseel Y, Lambrechts M, Scheunders P (2009) Wavelet-based colour texture retrieval using the Kullback–Leibler divergence between bivariate generalized Gaussian models. In: 2009 16th IEEE international conference on image processing (ICIP). IEEE, pp 265–268
Verdoolaege G, Scheunders P (2012) On the geometry of multivariate generalized Gaussian models. J Math Imaging Vis 43(3):180–193
Vrigkas M, Nikou C, Kakadiaris IA (2015) A review of human activity recognition methods. Front Robot AI 2:28
Wallace CS, Boulton DM (1968) An information measure for classification. Comput J 11(2):185–194
Wong SF, Cipolla R (2007) Extracting spatiotemporal interest points using global information. In: 2007 IEEE 11th international conference on computer vision. Citeseer, pp 1–8
Yeasin M, Bullot B, Sharma R (2004) From facial expression to level of interest: a spatio-temporal approach. In: Proceedings of the 2004 IEEE computer society conference on computer vision and pattern recognition, 2004. CVPR 2004. vol. 2. IEEE, pp II–II
Zhao G, Pietikainen M (2007) Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans Pattern Anal Mach Intell 29(6):915–928
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest
Ethical approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Additional information
Communicated by V. Loia.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Najar, F., Bourouis, S., Bouguila, N. et al. A new hybrid discriminative/generative model using the full-covariance multivariate generalized Gaussian mixture models. Soft Comput 24, 10611–10628 (2020). https://doi.org/10.1007/s00500-019-04567-2
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-019-04567-2