Standard and Novel Model Selection Criteria in the Pairwise Likelihood Estimation of a Mixture Model for Ordinal Data

Abstract

The model selection in a mixture setting was extensively studied in literature in order to assess the number of components. There exist different classes of criteria; we focus on those penalizing the log-likelihood with a penalty term, that accounts for model complexity. However, a full likelihood is not always computationally feasible. To overcome this issue, the likelihood is replaced with a surrogate objective function. Thus, a question arises naturally: how the use of a surrogate objective function affects the definition of model selection criteria? The model selection and the model estimation are distinct issues. Even if it is not possible to establish a cause and effect relationship between them, they are linked to each other by the likelihood. In both cases, we need to approximate the likelihood; to this purpose, it is computationally efficient to use the same surrogate function. The aim of this paper is not to provide an exhaustive survey of model selection, but to show the main used criteria in a standard mixture setting and how they can be adapted to a non-standard context. In the last decade two criteria based on the observed composite likelihood were introduced. Here, we propose some new extensions of the standard criteria based on the expected complete log-likelihood to the non-standard context of a pairwise likelihood approach. The main advantage is a less demanding and more stable estimation. Finally, a simulation study is conducted to test and compare the performances of the proposed criteria with those existing in literature. As discussed in detail in Sect. 7, the novel criteria work very well in all scenarios considered.

Appendix

Maximizing the observed pairwise log-likelihood is equivalent to maximize the fuzzy classification pairwise log-likelihood. This partially justifies the behaviour of the criteria based on the expected complete pairwise log-likelihood. In this appendix we derive the pairwise EN term. This is useful to two things: if we define the pairwise EN, the criteria based on the expected complete pairwise log-likelihood can be seen as the observed pairwise likelihood penalized by the pairwise EN term. Moreover, it gives us an idea about the separation between the mixture components.

$$\displaystyle\begin{array}{rcl} p\ell(\boldsymbol{\psi };\mathbf{X})& =& \sum _{i=1}^{P-1}\sum _{ j=i+1}^{P}\sum _{ c_{i}=1}^{C_{i} }\sum _{c_{j}=1}^{C_{j} }n_{c_{i}c_{j}}^{(ij)}\log \left [\sum _{ h=1}^{G}p_{ h}\boldsymbol{\pi }_{c_{i}c_{j}}^{(ij)}(\boldsymbol{\mu }_{ h},\boldsymbol{\varSigma }_{h},\boldsymbol{\gamma })\right ] {}\\ & =& \sum _{i=1}^{P-1}\sum _{ j=i+1}^{P}\sum _{ c_{i}=1}^{C_{i} }\sum _{c_{j}=1}^{C_{j} }n_{c_{i}c_{j}}^{(ij)}\sum _{ g=1}^{G}p_{ c_{i}c_{j};g}^{(ij)}\log \left [\sum _{ h=1}^{G}p_{ h}\boldsymbol{\pi }_{c_{i}c_{j}}^{(ij)}(\boldsymbol{\mu }_{ h},\boldsymbol{\varSigma }_{h},\boldsymbol{\gamma })\right ] {}\\ & =& \sum _{i=1}^{P-1}\sum _{ j=i+1}^{P}\sum _{ c_{i}=1}^{C_{i} }\sum _{c_{j}=1}^{C_{j} }n_{c_{i}c_{j}}^{(ij)}\sum _{ g=1}^{G}p_{ c_{i}c_{j};g}^{(ij)}\log \left [\sum _{ h=1}^{G}p_{ h}\boldsymbol{\pi }_{c_{i}c_{j}}^{(ij)}(\boldsymbol{\mu }_{ h},\boldsymbol{\varSigma }_{h},\boldsymbol{\gamma })\right ] {}\\ & & +\sum _{i=1}^{P-1}\sum _{ j=i+1}^{P}\sum _{ c_{i}=1}^{C_{i} }\sum _{c_{j}=1}^{C_{j} }n_{c_{i}c_{j}}^{(ij)}\sum _{ g=1}^{G}p_{ c_{i}c_{j};g}^{(ij)}\log \left [p_{ g}\boldsymbol{\pi }_{c_{i}c_{j}}^{(ij)}(\boldsymbol{\mu }_{ g},\boldsymbol{\varSigma }_{g},\boldsymbol{\gamma })\right ] {}\\ & & -\sum _{i=1}^{P-1}\sum _{ j=i+1}^{P}\sum _{ c_{i}=1}^{C_{i} }\sum _{c_{j}=1}^{C_{j} }n_{c_{i}c_{j}}^{(ij)}\sum _{ g=1}^{G}p_{ c_{i}c_{j};g}^{(ij)}\log \left [p_{ g}\boldsymbol{\pi }_{c_{i}c_{j}}^{(ij)}(\boldsymbol{\mu }_{ g},\boldsymbol{\varSigma }_{g},\boldsymbol{\gamma })\right ] {}\\ & =& \sum _{i=1}^{P-1}\sum _{ j=i+1}^{P}\sum _{ c_{i}=1}^{C_{i} }\sum _{c_{j}=1}^{C_{j} }n_{c_{i}c_{j}}^{(ij)}\sum _{ g=1}^{G}p_{ c_{i}c_{j};g}^{(ij)}\log \left [p_{ g}\boldsymbol{\pi }_{c_{i}c_{j}}^{(ij)}(\boldsymbol{\mu }_{ g},\boldsymbol{\varSigma }_{g},\boldsymbol{\gamma })\right ] {}\\ & & -\sum _{i=1}^{P-1}\sum _{ j=i+1}^{P}\sum _{ c_{i}=1}^{C_{i} }\sum _{c_{j}=1}^{C_{j} }n_{c_{i}c_{j}}^{(ij)}\sum _{ g=1}^{G}p_{ c_{i}c_{j};g}^{(ij)}\biggl [\log \left (p_{ g}\boldsymbol{\pi }_{c_{i}c_{j}}^{(ij)}(\boldsymbol{\mu }_{ g},\boldsymbol{\varSigma }_{g},\boldsymbol{\gamma })\right ) {}\\ & & -\log \sum _{h=1}^{G}p_{ h}\boldsymbol{\pi }_{c_{i}c_{j}}^{(ij)}(\boldsymbol{\mu }_{ h},\boldsymbol{\varSigma }_{h},\boldsymbol{\gamma })\biggr ] {}\\ & =& \sum _{i=1}^{P-1}\sum _{ j=i+1}^{P}\sum _{ c_{i}=1}^{C_{i} }\sum _{c_{j}=1}^{C_{j} }n_{c_{i}c_{j}}^{(ij)}\sum _{ g=1}^{G}p_{ c_{i}c_{j};g}^{(ij)}\log \left [p_{ g}\boldsymbol{\pi }_{c_{i}c_{j}}^{(ij)}(\boldsymbol{\mu }_{ g},\boldsymbol{\varSigma }_{g},\boldsymbol{\gamma })\right ] {}\\ & & -\sum _{i=1}^{P-1}\sum _{ j=i+1}^{P}\sum _{ c_{i}=1}^{C_{i} }\sum _{c_{j}=1}^{C_{j} }n_{c_{i}c_{j}}^{(ij)}\sum _{ g=1}^{G}p_{ c_{i}c_{j};g}^{(ij)}\log \dfrac{p_{g}\boldsymbol{\pi }_{c_{i}c_{j}}^{(ij)}(\boldsymbol{\mu }_{g},\boldsymbol{\varSigma }_{g},\boldsymbol{\gamma })} {\sum _{h=1}^{G}p_{h}\boldsymbol{\pi }_{c_{i}c_{j}}^{(ij)}(\boldsymbol{\mu }_{h},\boldsymbol{\varSigma }_{h},\boldsymbol{\gamma })} {}\\ & =& p\ell_{c}(\boldsymbol{\psi };\mathbf{X}) - EN_{pl}(\mathbf{p}) {}\\ \end{array}$$