Abstract
In this paper, we propose a probabilistic sparse non-negative matrix factorization model that extends a recently proposed variational Bayesian non-negative matrix factorization model to explicitly account for sparsity. We assess the influence of imposing sparsity within a probabilistic framework on either the loading matrix, score matrix, or both and further contrast the influence of imposing an exponential or truncated normal distribution as prior. The probabilistic methods are compared to conventional maximum likelihood based NMF and sparse NMF on three image datasets; (1) A (synthetic) swimmer dataset, (2) The CBCL face dataset, and (3) The MNIST handwritten digits dataset. We find that the probabilistic sparse NMF is able to automatically learn the level of sparsity and find that the existing probabilistic NMF as well as the proposed probabilistic sparse NMF prunes inactive components and thereby automatically learns a suitable number of components. We further find that accounting for sparsity can provide more part based representations but for the probabilistic modeling the choice of priors and how sparsity is imposed can have a strong influence on the extracted representations.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
CBCL Face Database #1 MIT Center For Biological and Computation Learning http://www.ai.mit.edu/projects/cbcl. A copy is available at http://www.ai.mit.edu/projects/cbcl.old/software-datasets/faces.tar.gz.
References
Paatero, P., Tapper, U.: Positive matrix factorization: a non-negative factor model with optimal utilization of error estimates of data values. Environmetrics 5(2), 111–126 (1994)
Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negative matrix factorization. Nature 401(6755), 788–791 (1999)
Lee, D.D., Seung, H.S.: Algorithms for non-negative matrix factorization. In: Advances in Neural Information Processing Systems, pp. 556–562 (2001)
Bro, R., De Jong, S.: A fast non-negativity-constrained least squares algorithm. J. Chemom. 11(5), 393–401 (1997)
Lin, C.J.: Projected gradient methods for nonnegative matrix factorization. Neural Comput. 19(10), 2756–2779 (2007)
Bro, R.: Multi-way analysis in the food industry: models, algorithms, and applications. Ph.D. thesis, Amsterdam: Universiteit van Amsterdam (1998)
Cichocki, A., Zdunek, R., Amari, S.: Hierarchical ALS algorithms for nonnegative matrix and 3D tensor factorization. In: Davies, M.E., James, C.J., Abdallah, S.A., Plumbley, M.D. (eds.) ICA 2007. LNCS, vol. 4666, pp. 169–176. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-74494-8_22
Donoho, D., Stodden, V.: When does non-negative matrix factorization give a correct decomposition into parts? In: Advances in Neural Information Processing Systems, pp. 1141–1148 (2004)
Laurberg, H., Christensen, M.G., Plumbley, M.D., Hansen, L.K., Jensen, S.H.: Theorems on positive data: On the uniqueness of NMF. Comput. Intell. Neurosci. 2008, 10 (2008)
Hoyer, P.O.: Non-negative matrix factorization with sparseness constraints. J. Mach. Learn. Res. 5, 1457–1469 (2004)
Li, Y., Ngom, A.: The non-negative matrix factorization toolbox for biological data mining. Source Code Biol. Med. 8(1), 10 (2013)
Schmidt, M.N., Winther, O., Hansen, L.K.: Bayesian non-negative matrix factorization. In: Adali, T., Jutten, C., Romano, J.M.T., Barros, A.K. (eds.) ICA 2009. LNCS, vol. 5441, pp. 540–547. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-00599-2_68
Schmidt, M.N., Mohamed, S.: Probabilistic non-negative tensor factorization using Markov chain Monte Carlo. In: 2009 17th European Signal Processing Conference, pp. 1918–1922. IEEE (2009)
Brouwer, T., Frellsen, J., Lió, P.: Comparative study of inference methods for bayesian nonnegative matrix factorisation. In: Ceci, M., Hollmén, J., Todorovski, L., Vens, C., Džeroski, S. (eds.) ECML PKDD 2017. LNCS (LNAI), vol. 10534, pp. 513–529. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-71249-9_31
Zhong, M., Girolami, M.: Reversible jump mcmc for non-negative matrix factorization. In: International Conference on Artificial Intelligence and Statistics (2009)
Schmidt, M.N., Mørup, M.: Infinite non-negative matrix factorization. In: 2010 18th European Signal Processing Conference, pp. 905–909. IEEE (2010)
Guan, Y., Dy, J.: Sparse probabilistic principal component analysis. In: Artificial Intelligence and Statistics, pp. 185–192 (2009)
Nielsen, S.F.V., Mørup, M.: Non-negative tensor factorization with missing data for the modeling of gene expressions in the human brain. In: 2014 IEEE International Workshop on Machine Learning for Signal Processing (MLSP). IEEE (2014)
Bishop, C.M.: Variational principal components (1999)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Hinrich, J.L., Mørup, M. (2018). Probabilistic Sparse Non-negative Matrix Factorization. In: Deville, Y., Gannot, S., Mason, R., Plumbley, M., Ward, D. (eds) Latent Variable Analysis and Signal Separation. LVA/ICA 2018. Lecture Notes in Computer Science(), vol 10891. Springer, Cham. https://doi.org/10.1007/978-3-319-93764-9_45
Download citation
DOI: https://doi.org/10.1007/978-3-319-93764-9_45
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-93763-2
Online ISBN: 978-3-319-93764-9
eBook Packages: Computer ScienceComputer Science (R0)