Abstract
In this paper, we propose a new probabilistic model of heterogeneously attributed multi-dimensional arrays. The model can manage heterogeneity by employing individual exponential family distributions for each attribute of the tensor array. Entries of the tensor are connected by latent variables and share information across the different attributes through the latent variables. The assumption of heterogeneity makes a Bayesian inference intractable, and we cast the EM algorithm approximated by the Laplace method and Gaussian process. We also extended the proposal algorithm for online learning. We apply our method to missing-values prediction and anomaly detection problems and show that our method outperforms conventional approaches that do not consider heterogeneity.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Andersson C, Bro R (2000) The n-way toolbox for MATLAB. Chemometr Intell Lab Syst 52(1): 1–4
Bertsekas DP, Bertsekas DP (1999) Nonlinear programming, 2nd edn. Athena Scientific, Belmont
Blei DM, McAuliffe JD (2007) Supervised topic models. Adv Neural Inf Process Syst 21
Bottou L, LeCun Y (2004) Large scale online learning. Adv Neural Inf Process Syst 16
Branch JW, Giannella C, Szymanski B, Wolff R, Kargupta H (2012) In-network outlier detection in wireless sensor networks. Knowl Inf Syst, pp 1–32
Chu W, Ghahramani Z (2009) Probabilistic models for incomplete multi-dimensional arrays. In: Proceedings of the 12th international conference on artificial intelligence and statistics
Cichocki A, Zdunek R, Choi S, Plemmons R Amari S (2007) Non-negative tensor factorization using alpha and beta divergences. In: Proceedings of IEEE international conference on acoustics, speech and signal processing, 2007, vol 3, pp III-1393–III-1396
Collins M, Dasgupta S, Schapire RE (2002) A generalization of principal components analysis to the exponential family. Adv Neural Inf Process Syst 14
De Lathauwer L, De Moor B, Vandewalle J (2000) A multilinear singular value decomposition. SIAM J Matrix Anal Appl 21(4): 1253–1278
De Lathauwer L, De Moor B, Vandewalle J (2000) On the best rank-1 and rank-(R_1,R_2,…,R_N) approximation of higher-order tensors. SIAM J Matrix Anal Appl 214: 1324–1342
Dunlavy DM, Kolda TG, Kegelmeyer WP (2006) Multilinear algebra for analyzing data with multiple linkages. Technical Report SAND2006-2079, Sandia National Laboratories
Gemulla R, Nijkamp E, Haas PJ, Sismanis Y (2011) Large-scale matrix factorization with distributed stochastic gradient descent. In: Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining, KDD ’11, pp 69–77
Getoor L, Taskar B (2007) Introduction to statistical relational learning (adaptive computation and machine learning). The MIT Press, Cambridge
Hall DL, Llinas J (1997) An introduction to multisensor data fusion. Proc IEEE 85(1): 6–23
Harshman RA (1970) Foundations of the PARAFAC procedure: models and conditions for an “explanatory” multi-modal factor analysis. UCLA Working Papers in phonetics, vol 16(1), p 84
Hayashi K, Takenouchi T, Shibata T, Kamiya Y, Kato D, Kunieda K, Yamada K, Ikeda K (2010) Exponential family tensor factorization for Missing-Values prediction and anomaly detection. In: The 10th IEEE international conference on data mining (ICDM’10)
Knorr EM, Ng RT, Tucakov V, Knorr EM, Ng RT, Tucakov V (2000) Distance-based outliers: algorithms and applications. The VLDB J 8(3): 237–253
Kolda TG, Bader BW (2009) Tensor decompositions and applications. SIAM Rev 51(3): 455–500
Koren Y, Bell R, Volinsky C (2009) Matrix factorization techniques for recommender systems. Computer 42(8): 30–37
Leskovec J, Faloutsos C (2007) Scalable modeling of real graphs using kronecker multiplication. In: Proceedings of the 24th international conference on machine learning, ICML ’07, pp 497–504
Liu J, Musialski P, Wonka P, Ye JP (2009) Tensor completion for estimating missing values in visual data. In: ICCV09, pp s2114–2121
Liu X, De Lathauwer L, Janssens F, De Moor B (2010) Hybrid clustering on multiple information sources via HOSVD. In: The seventh international symposium on neural networks (ISNN 2010)
Lu H, Vaidya J, Atluri V, Shin H, Jiang L (2011) Weighted rank-one binary matrix factorization. In: SDM, pp 283–294
Mavzgut J, Tivno P, Bodén, M, Yan H (2010) Multilinear decomposition and topographic mapping of binary tensors. In: Proceedings of the 20th international conference on Artificial neural networks: Part I, ICANN’10, pp 317–326
McCullagh P, Nelder JA (1989) Generalized linear models. Monographs on statistics and applied probability), 2nd edn. Chapman and Hall/CRC, London
Miettinen P (2011) Boolean tensor factorization. In: ICDM 2011, 11th IEEE international conference on data mining, pp 447–456
Mohamed S, Heller K, Ghahramani Z (2009) Bayesian exponential family PCA. Adv Neural Inf Process Syst 21
Mørup M (2011) Applications of tensor (multiway array) factorizations and decompositions in data mining. Wiley Interdiscip Rev Data Mining Knowl Discovery 1(1): 24–40
Mørup M, Hansen LK (2009) Automatic relevance determination for multiway models. J Chemometr Spec Issue Honor Professor Richard A. Harshman 23(7–8): 352–363
Nguyen TK, Cerf L, Plantevit M, Boulicaut J-F (2011) Multidimensional association rules in boolean tensors. In: 11th SIAM international conference on data mining SDM’11, pp 570–581
O’Hagan A (1991) Bayes-Hermite quadrature. J Stat Plan Inference 29: 245–260
O’Hagan A (1992) Some Bayesian numerical analysis. Bayesian Stat 4: 345–363
Pan SJ, Yang Q (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22(10): 1345–1359
Rasmussen CE, Ghahramani Z (2003) Bayesian Monte Carlo. Adv Neural Inf Process Syst 15(15): 505–512
Rasmussen CE, Williams CKI (2006) Gaussian processes for machine learning. The MIT Press, Cambridge
Rendle S, Thieme LS (2010) Pairwise interaction tensor factorization for personalized tag recommendation. In: WSDM ’10: proceedings of the third ACM international conference on Web search and data mining, pp 81–90
Shashua A, Hazan T (2005) Non-negative tensor factorization with applications to statistics and computer vision. In: Proceedings of the 22nd international conference on Machine learning, pp 792–799
Singh AP, Gordon GJ (2008) Relational learning via collective matrix factorization. In: Proceedings of the 14th ACM SIGKDD international conference on knowledge discovery and data mining, KDD ’08, pp 650–658
Sun J, Tao D, Faloutsos C (2006) Beyond streams and graphs: dynamic tensor analysis. In: KDD ’06: proceedings of the 12th ACM SIGKDD international conference on knowledge discovery and data mining, pp 374–383
Tomioka R, Hayashi K, Kashima H (2011) Estimation of low-rank tensors via convex optimization
Tucker L (1966) Some mathematical notes on three-mode factor analysis. Psychometrika 31(3): 279–311
Wedel M, Kamakura W (2001) Factor analysis with (mixed) observed and latent variables in the exponential family. Psychometrika 66(4): 515–530
Xiong L, Chen X, Huang TK, Schneider J, Carbonell JG (2010) Temporal collaborative filtering with bayesian probabilistic tensor factorization. In: SIAM data mining 2010 (SDM 10)
Zhang Z-Y, Li T, Ding C (2012) Non-negative tri-factor tensor decomposition with applications. Knowl Inf Syst, pp 1–23
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Hayashi, K., Takenouchi, T., Shibata, T. et al. Exponential family tensor factorization: an online extension and applications. Knowl Inf Syst 33, 57–88 (2012). https://doi.org/10.1007/s10115-012-0517-6
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10115-012-0517-6