Abstract
Several authors have recently proposed sparse estimation techniques for time-varying Markov networks, in which both graph structures and model parameters may change with time. In this study, we extend a previous approach with a low-rank assumption on the matrix of parameter sequence, utilizing a recent technique of nuclear norm regularization. This can potentially improve the estimation performance particularly in such cases that the local smoothness assumed in previous studies do not really hold. Then, we derive a simple algorithm based on the alternating direction method of multipliers (ADMM) which can effectively utilize the separable structure of our convex minimization problem. With an artificially-generated dataset, its superior performance in structure learning is demonstrated.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Banerjee, O., et al.: Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data. JMLR 9, 485–516 (2008)
Bertsekas, D.P., Tsitsiklis, J.N.: Parallel and distributed computation: Numerical methods. Prentice-Hall, Inc., Englewood Cliffs (1989)
Besag, J.: Statistical analysis of non-lattice data. J R Stat Soc Series D 24(3), 179–195 (1975)
Besag, J.: Efficiency of pseudo-likelihood estimation for simple Gaussian fields. Biometrika 64, 616–618 (1977)
Cai, J.-F., et al.: A singular value thresholding algorithm for matrix completion. SIAM Journal on Optimization 20(4), 1956–1982 (2010)
Eckstein, J., Bertsekas, D.: On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators. Mathematical Programming 5, 293–318 (1992)
Esser, E.: Applications of Lagrangian-based alternating direction methods and connections to split Bregman. CAM Reports 09-31, UCLA (2009)
Fazel, M., Hindi, H., Boyd, S.: Rank minimization and applications in system theory. In: Proceedings American Control Conference, pp. 3273–3278 (2004)
Figueiredo, M., Bioucas-Dias, J.: Restoration of Poissonian images using alternating direction optimization. IEEE Transactions on Image Processing (preprint) (2010), http://arxiv.org/abs/1001.2244
Goldstein, T., Osher, S.: The split Bregman method for L1-regularized problems. SIAM Journal on Imaging Sciences 2(2), 323–343 (2009)
Höfling, H., Tibshirani, R.: Estimation of sparse binary pairwise Markov networks using pseudo-likelihoods. JMLR 10, 883–906 (2009)
Hyvärinen, A.: Consistency of pseudolikelihood estimation of fully visible Boltzmann machines. Neural Comput. 18(10), 2283–2292 (2006)
Kolar, M., et al.: Estimating time-varying networks. Annals of Applied Statistics 4(1), 94–123 (2010)
Lee, S.-I., et al.: Efficient structure learning of Markov networks using L1-regularization. In: NIPS 2006 (2006)
Nocedal, J., Wright, S.J.: Numerical Optimization. Springer, Heidelberg (1999)
Pearl, J.: Probabilistic reasoning in intelligent systems: networks of plausible inference. Morgan Kaufmann Publishers Inc., San Francisco (1988)
Toh, K., Yun, S.: An accelerated proximal gradient algorithm for nuclear norm regularized least squares problems, National Univ. of Singapore (Preprint 2009)
Wainwright, M.J., et al.: High-dimensional graphical model selection using ℓ1-regularized logistic regression. In: NIPS (2006)
Xuan, X., Murphy, K.: Modeling changing dependency structure in multivariate time series. In: ICML 2007, pp. 1055–1062 (2007)
Yuan, X.M.: Alternating direction methods for sparse covariance selection, (2009), http://www.optimization-online.org/DB_FILE/2009/09/2390.pdf (Preprint)
Zhou, S., et al.: Time varying undirected graphs. Machine Learning (2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hirayama, Ji., Hyvärinen, A., Ishii, S. (2010). Sparse and Low-Rank Estimation of Time-Varying Markov Networks with Alternating Direction Method of Multipliers. In: Wong, K.W., Mendis, B.S.U., Bouzerdoum, A. (eds) Neural Information Processing. Theory and Algorithms. ICONIP 2010. Lecture Notes in Computer Science, vol 6443. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-17537-4_46
Download citation
DOI: https://doi.org/10.1007/978-3-642-17537-4_46
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-17536-7
Online ISBN: 978-3-642-17537-4
eBook Packages: Computer ScienceComputer Science (R0)