Authors:
Kevin R. Keane
and
Jason J. Corso
Affiliation:
University at Buffalo, SUNY and University of Michigan, United States
Keyword(s):
Multivariate Normal Distributions, Gaussian Graphical Models, Degenerate Priors.
Related
Ontology
Subjects/Areas/Topics:
Bayesian Models
;
Graphical and Graph-Based Models
;
Kernel Methods
;
Model Selection
;
Pattern Recognition
;
Sparsity
;
Theory and Methods
Abstract:
Myopic reliance on a misleading first sentence in the abstract of Covariance Selectiona Dempster (1972)
spawned the computationally and mathematically dysfunctional Gaussian graphical model (GGM). In stark
contrast to the GGM approach, the actual (Dempster, 1972, § 3) algorithm facilitated elegant and powerful applications,
including a “texture model” developed two decades ago involving arbitrary distributions of 1000+
dimensions Zhu (1996). The “Covariance Selection” algorithm proposes a greedy sequence of increasingly
constrained maximum entropy hypotheses Good (1963), terminating when the observed data “fails to reject”
the last proposed probability distribution. We are mathematically critical of GGM methods that address a
continuous convex domain with a discrete domain “golden hammer”. Computationally, selection of the wrong
tool morphs polynomial-time algorithms into exponential-time algorithms. GGMs concepts are at odds with
the fundamental concept of the invariant sph
erical multivariate Gaussian distribution. We are critical of the
Bayesian GGM approach because the model selection process derails at the start when virtually all prior mass
is attributed to comically precise multi-dimensional geometric “configurations” (Dempster, 1969, Ch. 13). We
propose two Bayesian alternatives. The first alternative is based upon (Dempster, 1969, Ch. 15.3) and (Hoff,
2009, Ch. 7). The second alternative is based upon Bretthorst (2012), a recent paper placing maximum entropy
methods such as the “Covariance Selection” algorithm in a Bayesian framework.
(More)