Definition
Data augmentation is a Markov chain Monte Carlo algorithm for sampling from a Bayesian posterior distribution
Theory
Let y be the observed data and z be the missing data or latent variable. Let p(y, z | θ) be the probability distribution of the complete data (y, z), with θ being the unknown parameter. The marginal distribution of the observed data y is \(p(y\vert \theta ) = \int \nolimits \nolimits p(y,z\vert \theta )dz\). Let p(θ) be the prior distribution of θ. The goal is to draw Monte Carlo samples from the posterior distribution \(p(\theta \vert y) \propto p(\theta )p(y\vert \theta )\).
The data augmentation algorithm is an iterative algorithm. It starts from an initial value θ0. Let (θ t , z t ) be the values of θ...
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc B 39:1–38
Geman S, Geman D (1984) Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans Pattern Anal Mach Intell 6:721–741
Higdon DM (1998) Auxiliary variable methods for Markov chain Monte Carlo with applications. J Am Stat Assoc 93:585–595
Liu JS, Wu YN (1999) Parameter expansion for data augmentation. J Am Stat Assoc 94(448):1264–1274
Liu JS, Wong WH, Kong A (1994) Covariance structure of the Gibbs sampler with applications to the comparisons of estimators and augmentation schemes. Biometrika 81:27–40
Liu C, Rubin DB, Wu YN (1998) Parameter expansion to accelerate EM: the PX-EM algorithm. Biometrika 85(4): 755–770
Meng XL, van Dyk D (1997) The EM algorithm – an old folk-song sung to a fast new tune. J R Stat Soc B 59:511–567
Meng XL, van Dyk D (1999) Seeking efficient data augmentation schemes via conditional and marginal augmentation. Biometrika 86:301–320
Swendsen RH, Wang J (1987) Nonuniversal critical dynamics in Monte Carlo simulations. Phys Rev Lett 58:86–88
Tanner MA, Wong WH (1987) The calculation of posterior distributions by data augmentation. J Am Stat Assoc 82: 528–540
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer Science+Business Media New York
About this entry
Cite this entry
Wu, Y.N. (2014). Data Augmentation. In: Ikeuchi, K. (eds) Computer Vision. Springer, Boston, MA. https://doi.org/10.1007/978-0-387-31439-6_741
Download citation
DOI: https://doi.org/10.1007/978-0-387-31439-6_741
Published:
Publisher Name: Springer, Boston, MA
Print ISBN: 978-0-387-30771-8
Online ISBN: 978-0-387-31439-6
eBook Packages: Computer ScienceReference Module Computer Science and Engineering