Skip to main content

Advertisement

SpringerLink
Log in

Nonparametric Bayesian inference in applications

  • Original Paper
  • Published:
Statistical Methods & Applications Aims and scope Submit manuscript

Abstract

Nonparametric Bayesian (BNP) inference is concerned with inference for infinite dimensional parameters, including unknown distributions, families of distributions, random mean functions and more. Better computational resources and increased use of massive automated or semi-automated data collection makes BNP models more and more common. We briefly review some of the main classes of models, with an emphasis on how they arise from applied research questions, and focus in more depth only on BNP models for spatial inference as a good example of a class of inference problems where BNP models can successfully address limitations of parametric inference.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  • Argiento R, Bianchini I, Guglielmi A (2016) A blocked Gibbs sampler for NGG-mixture models via a priori truncation. Stat Comput 26(3):641–661

    Article  MathSciNet  MATH  Google Scholar 

  • Argiento R, Guglielmi A, Pievatolo A (2010) Bayesian density estimation and model selection using nonparametric hierarchical mixtures. Comput Stat Data Anal 54(4):816–832

    Article  MathSciNet  MATH  Google Scholar 

  • Baladandayuthapani V, Mallick BK, Carroll R (2005) Spatially adaptive Bayesian penalized regression splines (P-splines). J Comput Graph Stat 14:378–394

    Article  MathSciNet  Google Scholar 

  • Banerjee S, Carlin BP, Gelfand AE (2014) Hierarchical modeling and analysis for spatial data, 2nd edn. Chapman and Hall/CRC, Boca Raton

    MATH  Google Scholar 

  • Barrios E, Lijoi A, Nieto-Barajas LE, Prünster I (2013) Modeling with normalized random measure mixture models. Stat Sci 28(3):313–334

    Article  MathSciNet  MATH  Google Scholar 

  • Berger J, Guglielmi A (2001) Bayesian testing of a parametric model versus nonparametric alternatives. J Am Stat Assoc 96:174–184

    Article  MathSciNet  MATH  Google Scholar 

  • Brezger A, Kneib T, Lang S (2005) BayesX: analyzing Bayesian structural additive regression models. J Stat Softw 14(1):1–22

    Google Scholar 

  • Broderick T, Pitman J, Jordan MI (2013) Feature allocations, probability functions, and paintboxes. Bayesian Anal 8(4):801–836

    Article  MathSciNet  MATH  Google Scholar 

  • Camerlenghi F (2015) Hierarchical and nested random probability measures with statistical applications, PhD thesis, Universita degli Studi di Pavia, Pavia

  • Campbell T, Cai D, Broderick T (2016) Exchangeable trait allocations. ArXiv e-prints

  • Chipman HA, Kolaczyk ED, McCulloch RE (1997) Adaptive Bayesian wavelet shrinkage. J Am Stat Assocn 92:1413–1421

    Article  MATH  Google Scholar 

  • Clyde M, George E (2000) Flexible empirical Bayes estimation for wavelets. J R Stat Soc Ser B 62:681–698

    Article  MathSciNet  MATH  Google Scholar 

  • Crane H (2016) The ubiquitous Ewens sampling formula. Stat Sci Rev J Inst Math Stat 31(1):1–19

    MathSciNet  Google Scholar 

  • Dahl DB (2006) Model-based clustering for expression data via a Dirichlet process mixture model. In: Vannucci M, Do KA, Müller P (eds) Bayesian inference for gene expression and proteomics. Cambridge University Press, Cambridge, pp 201–218

    Chapter  Google Scholar 

  • Dalal SR (1979) Dirichlet invariant processes and applications to nonparametric estimation of symmetric distribution functions. Stoch Process Their Appl 9:99–108

    Article  MathSciNet  MATH  Google Scholar 

  • De Blasi P, Favaro S, Lijoi A, Mena R, Prünster I, Ruggiero M (2015) Are Gibbs-type priors the most natural generalization of the Dirichlet process? IEEE Trans Pattern Anal Mach Intell 37:212–229

    Article  Google Scholar 

  • Duan JA, Guindani M, Gelfand AE (2007) Generalized spatial Dirichlet process models. Biometrika 94(4):809–825

    Article  MathSciNet  MATH  Google Scholar 

  • Dykstra RL, Laud P (1981) A Bayesian nonparametric approach to reliability. Ann Stat 9:356–367

    Article  MathSciNet  MATH  Google Scholar 

  • Ewens WJ (1972) The sampling theory of selectively neutral alleles. Theor Popul Biol Int J 3:87–112; erratum, ibid. 3 (1972), 240; erratum, ibid. 3 (1972), 376

  • Fahrmeir L, Kneib T, Lang S (2004) Penalized structured additive regression for space-time data: a Bayesian perspective. Stat Sin 14:731–761

    MathSciNet  MATH  Google Scholar 

  • Favaro S, Teh YW (2013) MCMC for normalized random measure mixture models. Stat Sci 28(3):335–359

    Article  MathSciNet  MATH  Google Scholar 

  • Ferguson TS (1973) A Bayesian analysis of some nonparametric problems. Ann Stat 1:209–230

    Article  MathSciNet  MATH  Google Scholar 

  • Ferguson TS, Phadia EG (1979) Bayesian nonparametric estimation based on censored data. Ann Stat 7(1):163–186

    Article  MathSciNet  MATH  Google Scholar 

  • Foti NJ, Williamson SA (2015) A survey of non-exchangeable priors for Bayesian nonparametric models. IEEE Trans Pattern Anal Mach Intell 37:359–371

    Article  Google Scholar 

  • Fuentes M, Henry J, Reich B (2013) Nonparametric spatial models for extremes: application to extreme temperature data. Extremes 16(1):75–101

    Article  MathSciNet  MATH  Google Scholar 

  • Fuentes M, Reich B (2013) Multivariate spatial nonparametric modelling via kernel processes mixing. Stat Sin 23(1):75–97

    MathSciNet  MATH  Google Scholar 

  • Gelfand AE, Kottas A, MacEachern SN (2005) Bayesian nonparametric spatial modeling with Dirichlet process mixing. J Am Stat Assoc 100:1021–1035

    Article  MathSciNet  MATH  Google Scholar 

  • Ghosh S, Ungureanu AB, Sudderth EB, Blei DM (2011) Spatial distance dependent Chinese restaurant processes for image segmentation. In: Shawe-Taylor J, Zemel RS, Bartlett PL, Pereira F, Weinberger KQ (eds) Advances in neural information processing systems, vol 24. Curran Associates, New York, pp 1476–1484

    Google Scholar 

  • Ghoshal S (2010) The Dirichlet process, related priors and posterior asymptotics. In: Hjort et al. (2010), pp 22–34

  • Ghoshal S, van der Vaart A (2017) Fundamentals of nonparametric Bayesian inference. Cambridge University Press, Cambridge

    Book  MATH  Google Scholar 

  • Gramacy RB, Lee HKH (2008) Bayesian treed Gaussian process models with an application to computer modeling. J Am Stat Assoc 103:1119–1130

    Article  MathSciNet  MATH  Google Scholar 

  • Griffiths TL, Ghahramani Z (2006) Infinite latent feature models and the Indian buffet process. In: Weiss Y, Schölkopf B, Platt J (eds) Advances in neural information processing systems, vol 18. MIT Press, Cambridge, pp 475–482

    Google Scholar 

  • Hanson TE, Jara A (2013) Surviving fully Bayesian nonparametric regression models. In: Damien P, Dellaportas P, Polson NG, Stephens DA (eds) Bayesian theory and applications. Oxford University Press, Oxford, pp 593–615

    Chapter  Google Scholar 

  • Hanson T, Johnson WO (2002) Modeling regression error with a mixture of Polya trees. J Am Stat Assoc 97:1020–1033

    Article  MathSciNet  MATH  Google Scholar 

  • Hanson T, Johnson WO (2004) A Bayesian semiparametric AFT model for interval-censored data. J Comput Graph Stat 13:341–361

    Article  MathSciNet  Google Scholar 

  • Hartigan JA (1990) Partition models. Commun Stat Theory Methods 19(8):2745–2756

    Article  MathSciNet  Google Scholar 

  • Hegarty A, Barry D (2008) Bayesian disease mapping using product partition models. Stat Med 27(19):3868–3893

    Article  MathSciNet  Google Scholar 

  • Hjort NL (1990) Nonparametric Bayes estimators based on beta processes in models for life history data. Ann Stat 18:1259–1294

    Article  MathSciNet  MATH  Google Scholar 

  • Hjort NL, Holmes C, Müller P, Walker SG (eds) (2010) Bayesian nonparametrics. Cambridge University Press, Cambridge

    MATH  Google Scholar 

  • Iglesias PL, Orellana Y, Quintana FA (2009) Nonparametric Bayesian modelling using skewed Dirichlet processes. J Stat Plan Inference 139(3):1203–1214

    Article  MathSciNet  MATH  Google Scholar 

  • Ishwaran H, James LF (2001) Gibbs sampling methods for stick-breaking priors. J Am Stat Assoc 96(453):161–173

    Article  MathSciNet  MATH  Google Scholar 

  • James LF, Lijoi A, Prünster I (2009) Posterior analysis for normalized random measures with independent increments. Scand J Stat 36(1):76–97

    Article  MathSciNet  MATH  Google Scholar 

  • Jara A, Hanson T, Quintana F, Müller P, Rosner G (2011) DPpackage: Bayesian semi- and nonparametric modeling in R. J Stat Softw 40(5):1–30

    Article  Google Scholar 

  • Jo S, Lee J, Müller P, Quintana FA, Trippa L (2017) Dependent species sampling models for spatial density estimation. Bayesian Anal 12(2):379–406

    Article  MathSciNet  MATH  Google Scholar 

  • Jo S, Lee J, Page G, Quintana FA, Trippa L, Müller P (2015) Spatial species sampling and product partition models. In: Mitra R, Müller P (eds) Nonparametric Bayesian inference in biostatistics. Springer, New York, pp 359–375

    Chapter  Google Scholar 

  • Kingman JFC (1993) Poisson processes. Oxford University Press, Oxford

    MATH  Google Scholar 

  • Kottas A, Gelfand AE (2001) Bayesian semiparametric median regression modeling. J Am Stat Assoc 96:1458–1468

    Article  MATH  Google Scholar 

  • Lavine M (1992) Some aspects of Polya tree distributions for statistical modeling. Ann Stat 20:1222–1235

    Article  MATH  Google Scholar 

  • Lavine M (1994) More aspects of Polya tree distributions for statistical modelling. Ann Stat 22:1161–1176

    Article  MATH  Google Scholar 

  • Lee J, Müller P, Gulukota K, Ji Y (2015) A Bayesian feature allocation model for tumor heterogeneity. Ann Appl Stat 9(2):621–639

    Article  MathSciNet  MATH  Google Scholar 

  • Lee J, Müller P, Zhu Y, Ji Y (2013) A nonparametric Bayesian model for local clustering with application to proteomics. Stat Sci 28:209–22

    Article  Google Scholar 

  • Lee J, Quintana F, Müller P, Trippa L (2013) Defining predictive probability functions for species sampling models. Stat Sci 28(2):209–222

    Article  MathSciNet  MATH  Google Scholar 

  • Lijoi A, Mena RH, Prünster I (2007) Controlling the reinforcement in Bayesian non-parametric mixture models. J R Stat Soc Ser B (Statistical Methodology) 69(4):715–740

    Article  MathSciNet  Google Scholar 

  • Lijoi A, Nipoti B, Prünster I (2014) Bayesian inference with dependent normalized completely random measures. Bernoulli 20(3):1260–1291

    Article  MathSciNet  MATH  Google Scholar 

  • Lijoi A, Prünster I (2010) Models beyond the Dirichlet process. In: Hjort et al. (2010), pp 80–136

  • Lo AY (1984) On a class of Bayesian nonparametric estimates I: density estimates. Ann Stat 12:351–357

    Article  MathSciNet  MATH  Google Scholar 

  • MacEachern S (1999) Dependent nonparametric processes. In: ASA proceedings of the section on Bayesian Statistical Science. ASA, Alexandria, VA

  • Müller P, Erkanli A, West M (1996) Bayesian curve fitting using multivariate normal mixtures. Biometrika 83:67–79

    Article  MathSciNet  MATH  Google Scholar 

  • Müller P, Mitra R (2013) Bayesian nonparametric inference–why and how. Bayesian Anal 8(2):269–302

    Article  MathSciNet  MATH  Google Scholar 

  • Müller P, Quintana F (2010) Random partition models with regression on covariates. J Stat Plan Inference 140(10):2801–2808

    Article  MathSciNet  MATH  Google Scholar 

  • Müller P, Quintana FA (2004) Nonparametric Bayesian data analysis. Stat Sci Rev J Inst Math Stat 19(1):95–110

    MathSciNet  MATH  Google Scholar 

  • Müller P, Quintana F, Jara A, Hanson T (2015) Nonparametric Bayesian data analysis. Springer, New York

    Book  MATH  Google Scholar 

  • Müller P, Quintana F, Rosner GL (2011) A product partition model with regression on covariates. J Comput Graph Stat 20(1):260–278. Supplementary material available online

  • Nieto-Barajas L, Walker SG (2002) Markov beta and gamma processes for modelling hazard rates. Scand J Stat 29:413–424

    Article  MathSciNet  MATH  Google Scholar 

  • Page GL, Quintana FA (2016) Spatial product partition models. Bayesian Anal 11(1):265–298

    Article  MathSciNet  MATH  Google Scholar 

  • Papageorgiou G, Richardson S, Best N (2015) Bayesian non-parametric models for spatially indexed data of mixed type. J R Stat Soc Ser B Stat Methodol 77(5):973–999

    Article  MathSciNet  Google Scholar 

  • Petrone S, Guindani M, Gelfand AE (2009) Hybrid Dirichlet mixture models for functional data. J R Stat Soc Ser B 71(4):755–782

    Article  MathSciNet  MATH  Google Scholar 

  • Phadia EG (2013) Prior processes and their applications. Springer, New York

    Book  MATH  Google Scholar 

  • Pitman J (1996) Some developments of the Blackwell-MacQueen Urn scheme. In: Ferguson TS, Shapeley LS, MacQueen JB (eds) Statistics, probability and game theory. Papers in Honor of David Blackwell, Haywar, California, IMS Lecture Notes - Monograph Series, pp 245–268

  • Pitman J, Yor M (1997) The two-parameter Poisson-Dirichlet distribution derived from a stable subordinator. Ann Probab 25(2):855–900

    Article  MathSciNet  MATH  Google Scholar 

  • Quintana FA (2006) A predictive view of Bayesian clustering. J Stat Plan Inference 136(8):2407–2429

    Article  MathSciNet  MATH  Google Scholar 

  • Quintana FA, Iglesias PL (2003) Bayesian clustering and product partition models. J R Stat Soc Ser B 65:557–574

    Article  MathSciNet  MATH  Google Scholar 

  • Rasmussen CE, Williams CKI (2006) Gaussian processes for machine learning. In: Adaptive computation and machine learning. MIT Press, Cambridge, MA

  • Regazzini E, Lijoi A, Prünster I (2003) Distributional results for means of normalized random measures with independent increments. Ann Stat 31(2):560–585

    Article  MathSciNet  MATH  Google Scholar 

  • Reich BJ, Bandyopadhyay D, Bondell HD (2013) A nonparametric spatial model for periodontal data with nonrandom missingness. J Am Stat Assoc 108(503):820–831

    Article  MathSciNet  MATH  Google Scholar 

  • Reich BJ, Fuentes M (2007) A multivariate semiparametric Bayesian spatial modeling framework for hurricane surface wind fields. Ann Appl Stat 1:249–264

    Article  MathSciNet  MATH  Google Scholar 

  • Reich BJ, Fuentes M (2012) Nonparametric Bayesian models for a spatial covariance. Stat Methodol 9(1–2):265–274

    Article  MathSciNet  MATH  Google Scholar 

  • Rodríguez A, Dunson DB (2011) Nonparametric Bayesian models through probit stick-breaking processes. Bayesian Anal 6(1):145–177

    Article  MathSciNet  MATH  Google Scholar 

  • Rodríguez A, Dunson DB, Gelfand AE (2008) The nested Dirichlet process, with discussion. J Am Stat Assoc 103:1131–1144

    Article  MATH  Google Scholar 

  • Rodríguez A, Ghosh K (2012) Modeling relational data using nested infinite relational models, Technical report. Department of Applied Mathematics and Statistics, University of California, Santa Cruz

  • Rue H, Held L (2005) Gaussian Markov random fields. Monographs on statistics and applied probability, vol 104. Chapman & Hall/CRC, Boca Raton

  • Schörgendorfer A, Branscum A, Hanson T (2013) A Bayesian goodness of fit test and semiparametric generalization of logistic regression with measurement data. Biometrics 69:508–519

    Article  MathSciNet  MATH  Google Scholar 

  • Sethuraman J (1994) A constructive definition of Dirichlet priors. Stat Sin 4(2):639–650

    MathSciNet  MATH  Google Scholar 

  • Sudderth EB, Jordan MI (2009) Shared segmentation of natural scenes using dependent Pitman-Yor processes. In: Koller D, Schuurmans D, Bengio Y, Bottou L (eds) Advances in neural information processing systems, vol 21. Curran Associates, New York, pp 1585–1592

    Google Scholar 

  • Teh YW, Görür D, Ghahramani Z (2007) Stick-breaking construction for the Indian buffet process. In: Proceedings of the 11th conference on artificial intelligence and statistics

  • Teh YW, Jordan MI, Beal MJ, Blei DM (2006) Sharing clusters among related groups: hierarchical Dirichlet processes. J Am Stat Assoc 101:1566–1581

    Article  MATH  Google Scholar 

  • Thibaux R, Jordan M (2007) Hierarchical beta processes and the Indian buffet process. In: Proceedings of the 11th conference on artificial intelligence and statistics (AISTAT), Puerto Rico

  • Vidakovic B (1998) Nonlinear wavelet shrinkage with Bayes rules and Bayes factors. J Am Stat Assoc 93:173–179

    Article  MathSciNet  MATH  Google Scholar 

  • Wade S, Mongelluzzo S, Petrone S (2011) An enriched conjugate prior for Bayesian nonparametric inference. Bayesian Anal 6(3):359–385

    MathSciNet  MATH  Google Scholar 

  • Walker S (2013) Bayesian nonparametrics. In: Damien P, Dellaportas P, Polson NG, Stephens DA (eds) Bayesian theory and applications. Oxford University Press, Oxford, pp 249–270

    Chapter  Google Scholar 

  • Walker S, Mallick B (1999) A Bayesian semiparametric accelerated failure time model. Biometrics 55:477–483

    Article  MathSciNet  MATH  Google Scholar 

  • Walker S, Muliere P (1997) Beta-Stacy processes and a generalization of the Pólya-urn scheme. Ann Stat 25:1762–1780

    Article  MATH  Google Scholar 

  • Williams CKI (1997) Prediction with Gaussian processes: from linear regression to linear prediction and beyond. In: Learning and inference in graphical models. Kluwer, pp 599–621

  • Xu Y, Müller P, Wahed AS, Thall PF (2016) Bayesian nonparametric estimation for dynamic treatment regimes with sequential transition times. J Am Stat Assoc 111:921–950

    Article  MathSciNet  Google Scholar 

  • Xu Y, Scharpstein D, Müller P, Daniels M (2016) A Bayesian nonparametric approach for semi-competing risks, Technical report. Johns Hopkins University

  • Xu Y, Thall PF, Müller P, Reza MJA (2017) A decision-theoretic comparison of treatments to resolve air leaks after lung surgery based on nonparametric modeling. Bayesian Anal 12(3):639–652

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

Peter Müeller was partially funded by Grant NIH R01 CA132891-06A1. Fernando A. Quintana was partially funded by Grant FONDECYT 1141057.

Author information

Authors and Affiliations

  1. University of Texas at Austin, Austin, TX, USA

    Peter Müeller

  2. Pontificia Universidad Católica de Chile, Santiago, Chile

    Fernando A. Quintana

  3. Brigham Young University, Provo, UT, USA

    Garritt Page

Authors
  1. Peter Müeller
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fernando A. Quintana
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Garritt Page
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peter Müeller.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Müeller, P., Quintana, F.A. & Page, G. Nonparametric Bayesian inference in applications. Stat Methods Appl 27, 175–206 (2018). https://doi.org/10.1007/s10260-017-0405-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10260-017-0405-z

Keywords

Search

Navigation