Abstract
How can we generate sparse tensor decomposition results for better interpretability? Typical tensor decomposition results are dense. Dense results require additional postprocessing for data interpretation, especially when the data are large. Thus, we present a large-scale Tucker factorization method for sparse and accurate tensor decomposition, which we call the Very Sparse Tucker factorization (VeST) method. The proposed VeST outputs highly sparse decomposition results from a large-scale partially observable tensor data. The approach starts by decomposing the input tensor data, then iteratively determining unimportant elements, removing them, and updating the remaining elements until a terminal state is reached. We define ‘responsibility’ of each element on the reconstruction error to determine unimportant elements in the decomposition results. The decomposition results are updated iteratively in parallel using carefully constructed coordinate descent rules for scalable computation. Furthermore, the suggested method automatically looks for the optimal sparsity ratio, resulting in a balanced sparsity-accuracy trade-off. Extensive experiments using real-world datasets showed that our method produces more accurate results than that of the competitors. Experiments further showed that the proposed method is scalable in terms of the input dimensionality, the number of observable entries, and the thread count.









Similar content being viewed by others
References
The 100 greatest movies (2018). https://www.empireonline.com/movies/features/best-movies/
Allen G (2012) Sparse higher-order principal components analysis. Artif Intell Stat, pp 27–36
de Araujo MR, Ribeiro PMP, Faloutsos C (2017) Tensorcast: Forecasting with context using coupled tensors (best paper award). In: Raghavan V, Aluru S, Karypis G, Miele L, Wu X (eds) 2017 IEEE International Conference on Data Mining, ICDM 2017, New Orleans, LA, USA, 18–21 Nov 2017, pp 71–80. IEEE Computer Society
Austin W, Ballard G, Kolda TG (2016) Parallel tensor compression for large-scale scientific data. In: IPDPS, pp 912–922
Bader BW, Kolda TG, et la Tensor toolbox for matlab v. 3.0, version 00
Ballard G, Klinvex A, Kolda TG (2020) Tuckermpi: A parallel c++/mpi software package for large-scale data compression via the tucker tensor decomposition. ACM Trans Math Softw 46(2)
Chakaravarthy VT, Choi JW, Joseph DJ, Liu X, Murali P, Sabharwal Y, Sreedhar D (2017) On optimizing distributed tucker decomposition for dense tensors. In: IPDPS, pp 1038–1047
Choi D, Jang JG, Kang U (2019) S3cmtf: fast, accurate, and scalable method for incomplete coupled matrix-tensor factorization. PLOS One 14(6):1–20
Choi JW, Liu X, Chakaravarthy VT (2018) High-performance dense tucker decomposition on GPU clusters. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage, and Analysis, SC 2018, Dallas, TX, USA, 11–16 Nov 2018, pp 42:1–42:11. IEEE / ACM
Dagum L, Menon R (1998) Openmp: an industry-standard api for shared-memory programming. IEEE Comput Sci Eng 5(1):46–55
Gao J (2008) Robust L1 principal component analysis and its Bayesian variational inference. Neural Comput 20(2):555–572
Hao F, Park D, Yin X, Wang X, Phonexay V (2019) A location-sensitive over-the-counter medicines recommender based on tensor decomposition. J Supercomput 75(4):1953–1970
Jang J, Kang U (2020) D-tucker: Fast and memory-efficient tucker decomposition for dense tensors. In: 36th IEEE International Conference on Data Engineering, ICDE 2020, Dallas, TX, USA, 20–24 Apr 2020, pp 1850–1853. IEEE
Jang JG, Part M, Sael L (2021) \({\sf VEST}\): Very sparse tucker factorization of large-scale tensors. In: BigComp2021, pp 1–8
Jeon I, Papalexakis EE, Kang U, Faloutsos C (2015) Haten2: billion-scale tensor decompositions. In: Gehrke J, Lehner W, Shim K, Cha SK, Lohman GM (eds) 31st IEEE International Conference on Data Engineering, ICDE 2015, Seoul, South Korea, 13–17 Apr 2015, pp 1047–1058. IEEE Computer Society
Jeong S, Cho J (2013) A framework for online gait recognition based on multilinear tensor analysis. J Supercomput 65(1):106–121
Jiang F, Liu Xy, Lu H, Shen R (2018) Efficient multi-dimensional tensor sparse coding using t-linear combination. In: AAAI 2018, pp 3326–3333
Kaliyar RK, Goswami A, Narang P (2021) Deepfake: improving fake news detection using tensor decomposition-based deep neural network. J Supercomput 77(2):1015–1037
Kang U, Papalexakis EE, Harpale A, Faloutsos C (2012) Gigatensor: scaling tensor analysis up by 100 times—algorithms and discoveries. In: KDD, pp 316–324
Kim S, Lee S, Kim J, Yoon Y (2020) Mrtensorcube: tensor factorization with data reduction for context-aware recommendations. J Supercomput 76(10):7847–7857
Kim YD, Choi S (2007) Nonnegative tucker decomposition. In: CVPR’07, pp 1–8. IEEE
Kolda TG (2006) Multilinear operators for higher-order decompositions, vol 2. United States, Department of Energy
Kolda TG, Bader BW (2009) Tensor decompositions and applications. SIAM Rev 51(3):455–500
Lathauwer LD, Moor BD, Vandewalle J (2000) On the best rank-1 and rank-(\({R_{1}},{R_{2}}, \cdots ,{R_{N}}\)) approximation of higher-order tensors. SIAM J Matrix Anal Appl 21(4):1324–1342
Lee D, Lee J, Yu H (2018) Fast tucker factorization for large-scale tensor completion. In: ICDM 2018, pp 1098–1103
Lee J, Choi D, Sael L (2018) CTD: fast, accurate, and interpretable method for static and dynamic tensor decompositions. PloS One 13(7):e0200579
Lee J, Oh S, Sael L (2018) GIFT: guided and interpretable factorization for tensors with an application to large-scale multi-platform cancer analysis. Bioinformatics 34(24):4151–4158
Li J, Ma Y, Wu X, Li A, Barker KJ (2019) PASTA: a parallel sparse tensor algorithm benchmark suite. CCF Trans High Perform Comput 1:111–130
Madrid-Padilla OH, Scott J (2017) Tensor decomposition with generalized lasso penalties. J Comput Graph Stat 26(3):537–546
Mahoney MW, Maggioni M, Drineas P (2008) Tensor-CUR decompositions for tensor-based data. SIAM J Matrix Anal Appl 30(3):957–987
Malik OA, Becker S (2018) Low-rank Tucker decomposition of large tensors using tensorsketch. In: Bengio S, Wallach HM, Larochelle H, Grauman K, Cesa-Bianchi N, Garnett R (eds) Advances in neural information processing systems 31: annual conference on neural information processing systems 2018, NeurIPS 2018, 3–8 Dec 2018, Montréal, Canada, pp 10117–10127
Mørup M, Hansen LK, Arnfred SM (2008) Algorithms for sparse nonnegative Tucker decompositions. Neural comput 20(8):2112–2131
Oh J, Shin K, Papalexakis EE, Faloutsos C, Yu H (2017) S-HOT: scalable high-order Tucker decomposition. In: de Rijke M, Shokouhi M, Tomkins A, Zhang M (eds) Proceedings of the tenth ACM international conference on web search and data mining, WSDM 2017, Cambridge, United Kingdom, 6–10 Feb 2017, pp 761–770. ACM
Oh S, Park N, Jang J, Sael L, Kang U (2019) High-performance Tucker factorization on heterogeneous platforms. TPDS 30(10):2237–2248
Oh S, Park N, Sael L, Kang U (2018) Scalable Tucker factorization for sparse tensors—algorithms and discoveries. In: ICDE. IEEE Computer Society, Paris, France
Papalexakis EE, Faloutsos C, Sidiropoulos ND (2012) Parcube: Sparse parallelizable tensor decompositions. In: ECML-PKDD, Lecture Notes in Computer Science, vol 7523, pp 521–536. Springer
Pascual-Montano A, Carazo JM, Kochi K, Lehmann D, Pascual-Marqui RD (2006) Nonsmooth nonnegative matrix factorization (nsNMF). IEEE TPAMIT 28(3):403–415
Qi N, Shi Y, Sun X, Yin B(2016) TenSR: Multi-dimensional tensor sparse representation. In: 2016 IEEE CVPR, pp 5916–5925
Ribeiro MT, Singh S, Guestrin C (2016) “why should I trust you?”: Explaining the predictions of any classifier. In: ACM SIGKDD’16, pp 1135–1144
Sanderson C, Curtin R (2016) Armadillo: a template-based c++ library for linear algebra. J Open Source Softw
Smith S, Karypis G (2017) Accelerating the tucker decomposition with compressed sparse tensors. In: Euro-Par 2017, Lecture Notes in Computer Science, vol 10417, pp 653–668. Springer
Sun WW, Lu J, Liu H, Cheng G (2017) Provable sparse tensor decomposition. J R Stat Soc Ser B 79(3):899–916
Tsourakakis CE (2010) MACH: fast randomized tensor decompositions. In: Proceedings of the SIAM International Conference on Data Mining, SDM 2010, April 29–May 1, 2010, Columbus, Ohio, USA, pp 689–700. SIAM
Xu Y (2015) Alternating proximal gradient method for sparse nonnegative Tucker decomposition. Math Program Comput 7(1):39–70
Yi S, Lai Z, He Z, Cheung Y, Liu Y (2017) Joint sparse principal component analysis. Pattern Recog 61(2):524–536
Zhang M, Ding CHQ (2013) Robust tucker tensor decomposition for effective image representation. In: ICCV, pp 2448–2455
Zhang Z, Aeron S (2016) Denoising and completion of 3d data via multidimensional dictionary learning. In: IJCAI, pp 2371–2377. IJCAI/AAAI Press
Acknowledgements
The publication of this article has been funded by the Basic Science Research Program through the National Research Foundation of Korea (2018R1A1A3A0407953, 2018R1A5A1060031).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Jang, JG., Park, M., Lee, J. et al. Large-scale tucker Tensor factorization for sparse and accurate decomposition. J Supercomput 78, 17992–18022 (2022). https://doi.org/10.1007/s11227-022-04559-4
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11227-022-04559-4