Abstract
We propose a probabilistic model class for the analysis of three-way count data, motivated by studying the subjectivity of language. Our models are applicable for instance to a data tensor of how many times each subject used each term in each context, thus revealing individual variation in natural language use. As our main goal is exploratory analysis, we propose hybrid bilinear and trilinear models with zero-mean constraints, separating modeling the simpler and more complex phenomena. While helping exploratory analysis, this approach leads into a more involved model selection problem. Our solution by forward selection guided by cross-validation likelihood is shown to work reliably on experiments with synthetic data.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Carroll, J.D., Chang, J.J.: Analysis of individual differences in multidimensional scaling via an n-way generalization of ”eckart-young” decomposition. Psychometrika 35(3), 283–319 (1970), http://dx.doi.org/10.1007/BF02310791
Friedlander, M.P., Hatz, K.: Computing nonnegative tensor factorizations. Computational Optimization and Applications 23(4), 631–647 (2008)
Gärdenfors, P.: Conceptual Spaces. MIT Press (2000)
Harshman, R.: Foundations of the PARAFAC procedure: Model and conditions for an ’explanatory’ multi-mode factor analysis. In: UCLA Working Papers in phonetics (16) (1970)
Hitchcock, F.L.: The expression of a tensor or a polyadic as a sum of products. Journal of Mathematics and Physics 6, 164–189 (1927)
Honkela, T., Raitio, J., Nieminen, I., Lagus, K., Honkela, N., Pantzar, M.: Using GICA method to quantify epistemological subjectivity. In: Proceedings of IJCNN 2012, International Joint Conference on Neural Networks (2012)
Ilin, A., Raiko, T.: Practical approaches to principal component analysis in the presence of missing values. Journal of Machine Learning Research (JMLR) 11, 1957–2000 (2010)
Kolda, T.G., Bader, B.W.: Tensor decompositions and applications. SIAM Review 51(3), 455–500 (2009)
Kolda, T.G.: Orthogonal tensor decompositions. SIAM Journal on Matrix Analysis and Applications 23(1), 243–255 (2001)
Kolda, T.G., Bader, B.W., Kenny, J.P.: Higher-order web link analysis using multilinear algebra. In: ICDM 2005: Proceedings of the 5th IEEE International Conference on Data Mining, pp. 242–249 (November 2005)
Kruskal, J.B.: Three-way arrays: Rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics. Linear Algebra and its Applications 18, 95–138 (1977)
McCullagh, P., Nelder, J.A.: Generalized linear models, 2nd edn. Chapman & Hall, London (1989)
Mørup, M.: Applications of tensor (multiway array) factorizations and decompositions in data mining (2011), http://onlinelibrary.wiley.com/doi/10.1002/widm.1/full
Picard, R.R., Cook, R.D.: Cross-validation of regression models. Journal of the American Statistical Association 79(387), 575–583 (1984), http://www.jstor.org/stable/2288403
de Silva, V., Lim, L.H.: Tensor rank and the ill-posedness of the best low-rank approximation problem. SIAM J. Matrix Analysis Applications 30(3), 1084–1127 (2008), http://dblp.uni-trier.de/db/journals/siammax/siammax30.html#SilvaL08
Yilmaz, Y.K., Cemgil, A.T., Simsekli, U.: Generalised coupled tensor factorisation. In: Shawe-Taylor, J., Zemel, R.S., Bartlett, P.L., Pereira, F.C.N., Weinberger, K.Q. (eds.) NIPS, pp. 2151–2159 (2011), http://dblp.uni-trier.de/db/conf/nips/nips2011.html#YilmazCS11
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Raitio, J., Raiko, T., Honkela, T. (2012). Hybrid Bilinear and Trilinear Models for Exploratory Analysis of Three-Way Poisson Counts. In: Villa, A.E.P., Duch, W., Érdi, P., Masulli, F., Palm, G. (eds) Artificial Neural Networks and Machine Learning – ICANN 2012. ICANN 2012. Lecture Notes in Computer Science, vol 7553. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33266-1_59
Download citation
DOI: https://doi.org/10.1007/978-3-642-33266-1_59
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33265-4
Online ISBN: 978-3-642-33266-1
eBook Packages: Computer ScienceComputer Science (R0)