Skip to main content
SpringerLink
Log in

The Effect of Model Misspecification on Growth Mixture Model Class Enumeration

  • Published:
Journal of Classification Aims and scope Submit manuscript

Abstract

Multiple criteria have been proposed to aid in deciding how many latent classes to extract in growth mixture models; however, studies are just beginning to investigate the performance of these criteria under non-ideal conditions. We review these previous studies and conduct a simulation study to address the performance of fit criteria under two previously uninvestigated assumption violations: (1) linearity of covariates and (2) proper specification of the growth factor covariance matrix. Results show that, provided that estimation is carried out with a large number of random starts and final stage optimizations, BIC and the bootstrap likelihood ratio test perform exceedingly well at identifying whether the data are homogenous or whether latent classes may be present, even with misspecifications present. Results were far less favorable when software default estimation choices were selected. We discuss implications to empirical studies and speculate on the relation between estimation choices and fit criteria perform.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • AKAIKE, H. (1987), “Factor Analysis and AIC”’ Psychometrika, 52, 317–332.

    Article  MathSciNet  MATH  Google Scholar 

  • BAUER, D.J. (2007), “Observations on the Use of Growth Mixture Models in Psychological Research”, Multivariate Behavioral Research, 42, 757–786.

    Article  Google Scholar 

  • BAUER, D.J., and CURRAN, P.J. (2003a), “Distributional Assumptions of Growth Mixture Models: Implications for Overextraction of Latent Trajectory Classes”, Psychological Methods, 8, 338–363.

    Article  Google Scholar 

  • BAUER, D.J., and CURRAN, P.J. (2003b), “Overextraction of Latent Trajectory Classes: Much Ado About Nothing? Reply to Rindskopf (2003), Muthén (2003), and Cudeck and Henly (2003)”, Psychological Methods, 8, 384–393.

    Article  Google Scholar 

  • BAUER, D.J., and CURRAN, P. J. (2004), “The Integration of Continuous and Discrete Latent Variable Models: Potential Problems and Promising Opportunities”, Psychological Methods, 9, 3–29.

    Article  Google Scholar 

  • BIERNACKI, C., CELEUX, G., and GOVAERT, G. (2000), “Assessing a Mixture Model for Clustering with the Integrated Completed Likelihood”, Pattern Analysis and Machine Intelligence, IEEE Transactions, 22, 719–725.

    Article  Google Scholar 

  • BIERNACKI, C., and GOVAERT, G. (1997), “Using the Classification Likelihood to Choose the Number of Clusters”, Computing Science and Statistics, 29, 451–457.

    Google Scholar 

  • BURTON, C.L., GALATZER-LEVY, I.R., and BONANNO, G.A. (2015), “Treatment Type and Demographic Characteristics as Predictors for Cancer Adjustment: Prospective Trajectories of Depressive Symptoms in a Population Sample”, Health Psychology, 34, 602–609.

    Article  Google Scholar 

  • CHEN, Q., KWOK, O.M., LUO, W., and WILLSON, V.L. (2010), “The Impact of Ignoring a Level of Nesting Structure in Multilevel Growth Mixture Models: A Monte Carlo Study”, Structural Equation Modeling, 17, 570–589.

    Article  MathSciNet  Google Scholar 

  • COLDER, C.R., CAMPBELL, R.T., RUEL, E., RICHARDSON, J.L., and FLAY, B.R. (2002), “A Finite Mixture Model of Growth Trajectories of Adolescent Alcohol Use: Predictors and Consequences”, Journal of Consulting and Clinical Psychology, 70, 976.

    Article  Google Scholar 

  • CUDECK, R., and BROWNE, M.W. (1983), “Cross-Validation of Covariance Structures”, Multivariate Behavioral Research, 18, 147–167.

    Article  Google Scholar 

  • CUDECK, R., and HENLY, S.J. (2003), “A Realistic Perspective on Pattern Representation in Growth Data: Comment on Bauer and Curran (2003)”, Psychological Methods, 8, 378–383.

    Article  Google Scholar 

  • CURRAN, P.J. (2003), “Have Multilevel Models Been Structural Equation Models All Along?”, Multivariate Behavioral Research, 38, 529–569.

    Article  Google Scholar 

  • CURRAN, P.J., OBEIDAT, K., and LOSARDO, D. (2010), “Twelve Frequently Asked Questions About Growth Curve Modeling”, Journal of Cognition and Development, 11(2), 121–136.

    Article  Google Scholar 

  • DEROON-CASSINI, T.A., MANCINI, A.D., RUSCH, M.D., and BONANNO, G.A. (2010), “Psychopathology and Resilience Following Traumatic Injury: A Latent Growth Mixture Model Analysis”, Rehabilitation Psychology, 55, 1–11.

    Article  Google Scholar 

  • ENDERS, C.K., and TOFIGHI, D. (2008), “The Impact of Misspecifying Class-Specific Residual Variances in Growth Mixture Models”, Structural Equation Modeling, 15, 75–95.

    Article  MathSciNet  Google Scholar 

  • EVERITT, B.S. (1996), “An Introduction to Finite Mixture Distributions”, Statistical Methods in Medical Research, 5, 107–127.

    Article  Google Scholar 

  • GREEN, P.J. (1995), “Reversible Jump Markov Chain Monte Carlo Computation and Bayesian Model Determination”, Biometrika, 82, 711–732.

    Article  MathSciNet  MATH  Google Scholar 

  • HENSON, J.M., REISE, S.P., and KIM, K.H. (2007), “Detecting Mixtures from Structural Model Differences Using Latent Variable Mixture Modeling: A Comparison of Relative Model Fit Statistics”, Structural Equation Modeling, 14, 202–226.

    Article  MathSciNet  Google Scholar 

  • HIPP, J.R., and BAUER, D.J. (2006), “Local Solutions in the Estimation of Growth Mixture Models”, Psychological Methods, 11, 36–53.

    Article  Google Scholar 

  • HURVICH, C.M., and TSAI, C.L. (1989). “Regression and Time Series Model Selection in Small Samples”, Biometrika, 76, 297–307.

    Article  MathSciNet  MATH  Google Scholar 

  • LI, F., BARRERA JR, M., HOPS, H., and FISHER, K.J. (2002), “The Longitudinal Influence of Peers on the Development of Alcohol Use In Late Adolescence: A Growth Mixture Analysis”, Journal of Behavioral Medicine, 25, 293–315.

    Article  Google Scholar 

  • LI, M., HARRING, J.R., and MACREADY, G.B. (2014), “Investigating the Feasibility of Using Mplus in the Estimation of Growth Mixture Models”, Journal of Modern Applied Statistical Methods, 13, 484–513.

    Google Scholar 

  • LI, L., and HSER, Y.I. (2011), “On Inclusion of Covariates for Class Enumeration of Growth Mixture Models”, Multivariate Behavioral Research, 46, 266–302.

    Article  Google Scholar 

  • LIU, M., and HANCOCK, G.R. (2014), “Unrestricted Mixture Models for Class Identification in Growth Mixture Modeling”, Educational and Psychological Measurement, 74, 557–584.

    Article  Google Scholar 

  • LO, Y., MENDELL, N.R., and RUBIN, D.B. (2001), “Testing the Number of Components in a Normal Mixture”, Biometrika, 88, 767–778.

    Article  MathSciNet  MATH  Google Scholar 

  • JEFFRIES, N. (2003), “A Note on Testing the Number of Components in a Normal Mixture”, Biometrika, 90, 991–994.

    Article  MathSciNet  MATH  Google Scholar 

  • JONES, B.L., NAGIN, D.S., and ROEDER, K. (2001), “A SAS Procedure Based on Mixture Models for Estimating Developmental Trajectories”, Sociological Methods and Research, 29, 374–393.

    Article  MathSciNet  Google Scholar 

  • JUNG, T., and WICKRAMA, K.A.S. (2008), “An Introduction to Latent Class Growth Analysis and Growth Mixture Modeling”, Social and Personality Psychology Compass, 2, 302–317.

    Article  Google Scholar 

  • MACCALLUM, R.C., BROWNE, M.W., and SUGAWARA, H.M. (1996), “Power Analysis and Determination of Sample Size for Covariance Structure Modeling”, Psychological Methods, 1, 130–149.

    Article  Google Scholar 

  • MCARDLE, J.J. (1989), “Structural Modeling Experiments Using Multiple Growth Functions”, Learning and Individual Differences: Abilities, Motivation, and Methodology, eds. P. Ackerman, R. Kanfer, and R. Cudeck, Hillsdale, NJ: Erlbaum, pp. 71–117.

    Google Scholar 

  • MCLACHLAN, G.J., and PEEL, D. (2000), Finite Mixture Models, New York: Wiley.

    Book  MATH  Google Scholar 

  • MCLACHLAN, G.J. (1987), “On Bootstrapping the Likelihood Ratio Test Statistic for the Number of Components in a Normal Mixture”, Applied Statistics, 36, 318–324.

    Article  Google Scholar 

  • MEEHL, P.E. (1967), “Theory-Testing in Psychology and Physics: A Methodological Paradox”, Philosophy of Science, 34, 103–115.

    Article  Google Scholar 

  • MOERBEEK, M. (2004), “The Consequence of Ignoring a Level of Nesting in Multilevel Analysis,” Multivariate Behavioral Research, 39, 129–149.

    Article  Google Scholar 

  • MORGAN, G.B., HODGE, K.J., and BAGGETT, A.R. (2016), “Latent Profile Analysis with Nonnormal Mixtures: A Monte Carlo Examination of Model Selection Using Fit Indices”, Computational Statistics & Data Analysis, 93, 146–161.

    Article  MathSciNet  Google Scholar 

  • MUSU-GILLETTE, L.E., WIGFIELD, A., HARRING, J.R., and ECCLES, J.S. (2015), “Trajectories of Change in Students’ Self-Concepts of Ability and Values in Math and College Major Choice”, Educational Research and Evaluation, 21, 343–370.

    Article  Google Scholar 

  • MUTHÉN, B.O. (2003), “Statistical and Substantive Checking in Growth Mixture Modeling: Comment on Bauer and Curran (2003),” Psychological Methods, 8, 369–377.

    Article  Google Scholar 

  • MUTHÉN, B.O. (2001), “Second-Generation Structural Equation Modeling with a Combination of Categorical and Continuous Latent Variables: New Opportunities for Latent Class/Latent Growth Modeling”, in New Methods for the Analysis of Change, eds. L.M. Collins and A. Sayer, Washington, D.C.: American Psychological Association, pp. 291–322.

    Chapter  Google Scholar 

  • MUTHÉN, B.O., and CURRAN, P.J. (1997), “General Longitudinal Modeling of Individual Differences in Experimental Designs: A Latent Variable Framework for Analysis and Power Estimation”, Psychological Methods, 2, 371–402.

    Article  Google Scholar 

  • MUTHÉN, B.O., and SHEDDEN, K. (1999), “Finite Mixture Modeling with Mixture Outcomes Using the EM Algorithm”, Biometrics, 55, 463–469.

    Article  MATH  Google Scholar 

  • MEYERS, J.L., and BERETVAS, S.N. (2006), “The Impact of Inappropriate Modeling of Cross-Classified Data Structures”, Multivariate Behavioral Research, 41, 473–497.

    Article  Google Scholar 

  • MICCERI, T. (1989), “The Unicorn, the Normal Curve, and Other Improbable Creatures”, Psychological Bulletin, 105, 156–166.

    Article  Google Scholar 

  • NAGIN, D.S. (2005), Group-Based Modeling of Development, Cambridge, MA: Harvard University Press.

    Book  Google Scholar 

  • NAGIN, D.S. (1999), “Analyzing Developmental Trajectories: A Semiparametric, Group-Based Approach,” Psychological Methods, 4, 139–157.

    Article  Google Scholar 

  • NYLUND, K.L., ASPAROUHOV, T., and MUTHÉN, B.O. (2007), “Deciding on the Number of Classes in Latent Class Analysis and Growth Mixture Modeling: A Monte Carlo Simulation Study”, Structural Equation Modeling, 14, 535–569.

    Article  MathSciNet  Google Scholar 

  • PALARDY, G.J. (2008), “Differential School Effects Among Low, Middle, and High Social Class Composition Schools: A Multiple Group, Multilevel Latent Growth Curve Analysis”, School Effectiveness and School Improvement, 19, 21–49.

    Article  Google Scholar 

  • PETRAS, H., and MASYN, K. (2010), “General Growth Mixture Analysis with Antecedents and Consequences of Change”, in Handbook of quantitative criminology, eds. A. Piquero and D. Weisburd, New York: Springer, pp. 69–100.

    Chapter  Google Scholar 

  • PEUGH, J., and FAN, X. (2012), “How Well Does Growth Mixture Modeling Identify Heterogeneous Growth Trajectories? A Simulation Study Examining GMM's Performance Characteristics,” Structural Equation Modeling, 19, 204–226.

    Article  MathSciNet  Google Scholar 

  • PEUGH, J., and FAN, X. (2015), “Enumeration Index Performance in Generalized Growth Mixture Models: A Monte Carlo Test of Muthén’s (2003) Hypothesis”, Structural Equation Modeling, 22, 115–131.

    Article  MathSciNet  Google Scholar 

  • RAM, N., and GRIMM, K.J. (2009). “Growth Mixture Modeling: A Method for Identifying Differences in Longitudinal Change Among Unobserved Groups,” International Journal of Behavioral Development, 33, 565–576.

    Article  Google Scholar 

  • RINDSKOPF, D. (2003), “Mixture or Homogeneous? Comment on Bauer and Curran (2003)”, Psychological Methods, 8, 364–368.

    Article  Google Scholar 

  • ROEDER, K., LYNCH, K.G., and NAGIN, D.S. (1999), “Modeling Uncertainty in Latent Class Membership: A Case Study in Criminology”, Journal of the American Statistical Association, 94, 766–776.

    Article  Google Scholar 

  • SCHUMACKER, R., and MARCOULIDES, G. (Eds.). (1998). Interaction and Nonlinear Effects in Structural Equation Modeling. Mahwah, NJ: Erlbaum.

    Google Scholar 

  • SCHWARZ, G. (1978), “Estimating the Dimension of a Model”, The Annals of Statistics, 6, 461–464.

    Article  MathSciNet  MATH  Google Scholar 

  • SCLOVE, S.L. (1987), “Application of Model-Selection Criteria to Some Problems in Multivariate Analysis”, Psychometrika, 52, 333–343.

    Article  Google Scholar 

  • SHIREMAN, E., STEINLEY, D., and BRUSCO, M.J. (2015), “Examining the Effect of Initialization Strategies on the Performance of Mixture Modeling”, Behavior Research Methods. DOI: 10.3758/s13428-015-0697-6.

  • STEINLEY, D., and BRUSCO, M.J. (2011), “Evaluating Mixture Modeling for Clustering: Recommendations and Cautions”, Psychological Methods, 16, 63–79.

    Article  Google Scholar 

  • STEINLEY, D., and BRUSCO, M.J. (2007), “Initializing K-means Batch Clustering: A Critical Evaluation of Several Techniques”, Journal of Classification, 24, 99-121.

    Article  MathSciNet  MATH  Google Scholar 

  • STRAM, D.O., and LEE, J.W. (1994), “Variance Components Testing in the Longitudinal Mixed Effects Model”, Biometrics, 50, 1171–1177.

    Article  MATH  Google Scholar 

  • TITTERINGTON, D.M., SMITH A.F.M., and MAKOVM, U.E. (1985), Statistical Analysis of Finite Mixture Models, New York: Wiley.

    Google Scholar 

  • TOFIGHI, D., and ENDERS, C.K. (2008), “Identifying the Correct Number of Classes in a Growth Mixture Model”, in Mixture Models in Latent Variable Research, ed. G.R. Hancock, Greenwich, CT: Information Age, pp. 317–341.

    Google Scholar 

  • VAN LANDEGHEM, G., DE FRAINE, B., and VAN DAMME, J. (2005), “The Consequence of Ignoring a Level of Nesting in Multilevel Analysis: A Comment”, Multivariate Behavioral Research, 40, 423–434.

    Article  Google Scholar 

  • VICKERS, A.J. (2003), “How Many Repeated Measures In Repeated Measures Designs? Statistical Issues for Comparative Trials”, BMC Medical Research Methodology, 3, 22.

    Article  Google Scholar 

  • WALL, M.M., GUO, J., and AMEMIYA, Y. (2012), “Mixture Factor Analysis for Approximating a Nonnormally Distributed Continuous Latent Factor with Continuous and Dichotomous Observed Variables”, Multivariate Behavioral Research, 47, 276–313.

    Article  Google Scholar 

  • WANG, M., and BODNER, T.E. (2007), “Growth Mixture Modeling Identifying and Predicting Unobserved Subpopulations with Longitudinal Data”, Organizational Research Methods, 10, 635–656.

    Article  Google Scholar 

  • VUONG, Q.H. (1989), “Likelihood Ratio Tests for Model Selection and Non-Nested Hypotheses”, Econometrica, 57, 307–333.

    Article  MathSciNet  MATH  Google Scholar 

  • YUNG, Y.F. (1997), “Finite Mixtures in Confirmatory Factor-Analysis Models”, Psychometrika, 62, 297–330.

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Maryland, 100 E. Franklin Street Suite 200, Chapel Hill, NC, 27599, USA

    Daniel McNeish & Jeffrey R. Harring

Authors
  1. Daniel McNeish
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jeffrey R. Harring
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Daniel McNeish.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

McNeish, D., Harring, J.R. The Effect of Model Misspecification on Growth Mixture Model Class Enumeration. J Classif 34, 223–248 (2017). https://doi.org/10.1007/s00357-017-9233-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00357-017-9233-y

Keywords

Search

Navigation