Skip to main content
SpringerLink
Log in

Sampling schemes for generalized linear Dirichlet process random effects models

  • Published:
Statistical Methods & Applications Aims and scope Submit manuscript

Abstract

We evaluate MCMC sampling schemes for a variety of link functions in generalized linear models with Dirichlet process random effects. First, we find that there is a large amount of variability in the performance of MCMC algorithms, with the slice sampler typically being less desirable than either a Kolmogorov–Smirnov mixture representation or a Metropolis–Hastings algorithm. Second, in fitting the Dirichlet process, dealing with the precision parameter has troubled model specifications in the past. Here we find that incorporating this parameter into the MCMC sampling scheme is not only computationally feasible, but also results in a more robust set of estimates, in that they are marginalized-over rather than conditioned-upon. Applications are provided with social science problems in areas where the data can be difficult to model, and we find that the nonparametric nature of the Dirichlet process priors for the random effects leads to improved analyses with more reasonable inferences.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  • Abramowitz M, Stegun IA (1972) Stirling numbers of the second kind. Section 24.1.4. In: Handbook of mathematical functions with formulas, graphs, and mathematical tables, 9th printing. Dover, New York, pp 824–825.

  • Albert JH, Chib S (1993) Bayesian analysis of binary and polychotomous response data. J Am Stat Assoc 88: 669–679

    Article  MathSciNet  MATH  Google Scholar 

  • Andrews DF, Mallows CL (1974) Scale mixtures of normal distributions. J R Stat Soc Ser B 36: 99–102

    MathSciNet  MATH  Google Scholar 

  • Antoniak CE (1974) Mixtures of Dirichlet processes with applications to Bayesian nonparametric problems. Ann Stat 2: 1152–1174

    Article  MathSciNet  MATH  Google Scholar 

  • Balakrishnan N (1992) Handbook of the logistic distribution. CRC Press, Boca Raton

    MATH  Google Scholar 

  • Blackwell D, MacQueen JB (1973) Discreteness of Ferguson selections. Ann Stat 1: 358–365

    MathSciNet  Google Scholar 

  • Breslow NE, Clayton DG (1993) Approximate inference in generalized linear mixed models. J Am Stat Assoc 88: 9–25

    Article  MATH  Google Scholar 

  • Buonaccorsi JP (1996) Measurement error in the response in the general linear model. J Am Stat Assoc 91: 633–642

    Article  MathSciNet  MATH  Google Scholar 

  • Chib S, Greenberg E, Chen Y (1998) MCMC methods for fitting and comparing multinomial response models. Technical Report, Economics Working Paper Archive, Washington University at St. Louis, http://129.3.20.41/econ-wp/em/papers/9802/9802001.pdf

  • Chib S, Winkelmann R (2001) Markov chain Monte Carlo analysis of correlated count data. J Bus Econ Stat 19: 428–435

    Article  MathSciNet  Google Scholar 

  • Damien P, Wakefield J, Walker S (1999) Gibbs sampling for Bayesian non-conjugate and hierarchical models by using auxiliary variables. J R Stat Soc Ser B 61: 331–344

    Article  MathSciNet  MATH  Google Scholar 

  • Devroye L (1986) Non-uniform random variate generation. Springer, New York

    MATH  Google Scholar 

  • Dey DK, Ghosh SK, Mallick BK (2000) Generalized linear models: a Bayesian perspective. Marcel Dekker, New York

    MATH  Google Scholar 

  • Dorazio RM, Mukherjee B, Zhang L, Ghosh M, Jelks HL, Jordan F (2007) Modelling unobserved sources of heterogeneity in animal abundance using a Dirichlet process prior. Biometrics (online publication August 3, 2007)

  • Escobar MD, West M (1995) Bayesian density estimation and inference using mixtures. J Am Stat Assoc 90: 577–588

    Article  MathSciNet  MATH  Google Scholar 

  • Fahrmeir L, Tutz G (2001) Multivariate statistical modelling based on generalized linear models. 2. Springer, New York

    MATH  Google Scholar 

  • Ferguson TS (1973) A Bayesian analysis of some nonparametric problems. Ann Stat 1: 209–230

    Article  MathSciNet  MATH  Google Scholar 

  • Gill J, Casella G (2009) Nonparametric priors for ordinal Bayesian social science models: specification and estimation. J Am Stat Assoc 104: 453–464

    Article  MathSciNet  Google Scholar 

  • Gurr TR, Marshall MG, Jaggers K (2003) PolityIV, http://www.cidcm.umd.edu/inscr/polity/

  • Ishwaran H, James LF (2001) Gibbs sampling methods for stick-breaking priors. J Am Stat Assoc 96: 161–173

    Article  MathSciNet  MATH  Google Scholar 

  • Jiang J (2007) Linear and generalized linear mixed models and their applications. Springer, New York

    MATH  Google Scholar 

  • Koch MT, Cranmer S (2007) Terrorism than governments of the right? Testing the ‘Dick Cheney’ hypothesis: do governments of the left attract more than governments of the right?. Conflict Manage Peace Sci 24: 311–326

    Article  Google Scholar 

  • Korwar RM, Hollander M (1973) Contributions to the theory of Dirichlet processes. Ann Probab 1: 705–711

    Article  MathSciNet  MATH  Google Scholar 

  • Kyung M, Gill J, Casella G (2009) Characterizing the variance improvement in linear Dirichlet random effects models. Stat Probab Lett 79: 2343–2350

    Article  MathSciNet  MATH  Google Scholar 

  • Kyung M, Gill J, Casella G (2010) Estimation in Dirichlet random effects models. Ann Stat 38: 979–1009

    Article  MathSciNet  MATH  Google Scholar 

  • Kyung M, Gill J, Casella G (2011) New findings from terrorism data: Dirichlet process random effects models for latent groups. J R Stat Soc Ser C (Forthcoming)

  • Liu JS (1996) Nonparametric hierarchical Bayes via sequential imputations. Ann Stat 24: 911–930

    Article  MATH  Google Scholar 

  • Lo AY (1984) On a class of Bayesian nonparametric estimates: I. Density estimates. Ann Stat 12: 351–357

    Article  MATH  Google Scholar 

  • MacEachern SN, Müller P (1998) Estimating mixture of Dirichlet process model. J Comput Graph Stat 7: 223–238

    Article  Google Scholar 

  • McAuliffe JD, Blei DM, Jordan MI (2006) Nonparametric empirical Bayes for the Dirichlet process mixture model. Stat Comput 16: 5–14

    Article  MathSciNet  Google Scholar 

  • McCullagh P, Nelder JA (1989) Generalized linear models. 2. Chapman & Hall, New York

    MATH  Google Scholar 

  • McCullagh P, Yang J (2006) Stochastic classification models. Int Congr Math III: 669–686

    MathSciNet  Google Scholar 

  • McCulloch CE, Searle SR (2001) Generalized, linear, and mixed models. Wiley, New York

    MATH  Google Scholar 

  • Mira A, Tierney L (2002) Efficiency and convergence properties of slice samplers. Scand J Stat 29: 1–12

    Article  MathSciNet  MATH  Google Scholar 

  • Neal RM (2000) Markov chain sampling methods for Dirichlet process mixture models. J Comput Graph Stat 9: 249–265

    Article  MathSciNet  Google Scholar 

  • Neal RM (2003) Slice sampling. Ann Stat 31: 705–741

    Article  MathSciNet  MATH  Google Scholar 

  • Robert C, Casella G (2004) Monte Carlo statistical methods. Springer, New York

    MATH  Google Scholar 

  • Sethuraman J (1994) A constructive definition of Dirichlet priors. Statistica Sinica 4: 639–650

    MathSciNet  MATH  Google Scholar 

  • Teh YW, Jordan MI, Beal MJ, Blei DM (2006) Hierarchical Dirichlet processes. J Am Stat Assoc 101: 1566–1581

    Article  MathSciNet  MATH  Google Scholar 

  • Tierney L, Kadane JB (1986) Accurate approximations for posterior moments and marginal densities. J Am Stat Assoc 81: 82–86

    Article  MathSciNet  MATH  Google Scholar 

  • Wang N, Lin X, Gutierrez RG, Carroll RJ (1998) Bias analysis and SIMEX approach in generalized linear mixed measurement error models. J Am Stat Assoc 93: 249–261

    Article  MathSciNet  MATH  Google Scholar 

  • West M (1987) On scale mixtures of normal distributions. Biometrika 74: 646–648

    Article  MathSciNet  MATH  Google Scholar 

  • Wolfinger R, O’Connell M (1993) Generalized linear mixed models: a pseudolikelihood approach. J Stat Comput Simul 48: 233–243

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Statistics, Duksung Women’s University, 19 Geunhwagyo-Gil, Dobong Gu, Seoul, 132-714, Korea

    Minjung Kyung

  2. Department of Political Science, Washington University, One Brookings Dr., Seigle Hall, St. Louis, MO, USA

    Jeff Gill

  3. Department of Biostatistics, Washington University, One Brookings Dr., Seigle Hall, St. Louis, MO, USA

    Jeff Gill

  4. Department of Statistics, University of Florida, Gainesville, FL, 32611, USA

    George Casella

Authors
  1. Minjung Kyung
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jeff Gill
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. George Casella
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to George Casella.

Additional information

This study was supported by National Science Foundation Grants DMS-0631632, SES-0631588, DMS-04-05543.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kyung, M., Gill, J. & Casella, G. Sampling schemes for generalized linear Dirichlet process random effects models. Stat Methods Appl 20, 259–290 (2011). https://doi.org/10.1007/s10260-011-0168-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10260-011-0168-x

Keywords

Search

Navigation