Skip to main content
SpringerLink
Log in

Canonical polyadic decomposition (CPD) of big tensors with low multilinear rank

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Tensor decomposition methods have been widely applied to big data analysis as they bring multiple modes and aspects of data to a unified framework, which allows us to discover complex internal structures and correlations of data. Unfortunately most existing approaches are not designed to meet the challenges posed by big data dilemma. This paper attempts to improve the scalability of tensor decompositions and makes two contributions: A flexible and fast algorithm for the CP decomposition (FFCP) of tensors based on their Tucker compression; A distributed randomized Tucker decomposition approach for arbitrarily big tensors but with relatively low multilinear rank. These two algorithms can deal with huge tensors, even if they are dense. Extensive simulations provide empirical evidence of the validity and efficiency of the proposed algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. See www.mathworks.com/help/pdf_doc/mdce/mdce.pdf.

  2. We have assumed that each sub-tensor can be efficiently handled in by the associated worker, otherwise we increase the number of workers and decrease the size of each sub-tensor accordingly

  3. The performance index SIR is defined as \(\text {SIR}(\mathbf {a}_{r}^{(n)},{\hat {\mathbf {a}}}_{r}^{(n)})=20\log {\left \|\left \|{\mathbf {a}_{r}^{(n)}-{\hat {\mathbf {a}}}_{r}^{(n)}}\right \|\right \|}/{\left \|\left \|{\mathbf {a}_{r}^{(n)}}\right \|\right \|}\), where \(\mathbf {a}_{r}^{(n)}\) is an estimate of \(\mathbf {a}_{r}^{(n)}\), both \(\mathbf {a}_{r}^{(n)}\) and \(\mathbf {a}_{r}^{(n)}\) are normalized to be zero-mean and unit-variance.

References

  1. Andersson CA, Bro R (2000) The N-way toolbox for MATLAB. http://www.models.life.ku.dk/source/nwaytoolbox/

  2. Bader BW, Kolda TG (2012) MATLAB tensor toolbox version 2.5. http://csmr.ca.sandia.gov/tgkolda/TensorToolbox/

  3. Baraniuk R, Cevher V, Wakin M (2010) Low-dimensional models for dimensionality reduction and signal recovery: a geometric perspective. Proc IEEE 98 (6):959–971. https://doi.org/10.1109/JPROC.2009.2038076

    Article  Google Scholar 

  4. Bro R, Andersson CA (1998) Improving the speed of multiway algorithms: part ii: compression. Chemom Intell Lab Syst 42(1–2):105–113

    Article  Google Scholar 

  5. Cai D, He X, Han J, Huang T (2011) Graph regularized nonnegative matrix factorization for data representation. IEEE Trans Pattern Anal Mach Intell 33 (8):1548–1560. https://doi.org/10.1109/TPAMI.2010.231

    Article  Google Scholar 

  6. Caiafa CF, Cichocki A (2010) Generalizing the column-row matrix decomposition to multi-way arrays. Linear Algebra Appl 433(3):557–573

    Article  MathSciNet  Google Scholar 

  7. Cevher V, Becker S, Schmidt M (2014) Convex optimization for big data: scalable, randomized, and parallel algorithms for big data analytics. IEEE Signal Process Mag 31(5):32–43. https://doi.org/10.1109/MSP.2014.2329397

    Article  Google Scholar 

  8. Cichocki A, Phan AH (2009) Fast local algorithms for large scale nonnegative matrix and tensor factorizations. IEICE Trans Fund Electron Commun Comput Sci (Invited paper) E92-A(3):708–721

    Article  Google Scholar 

  9. Cichocki A, Zdunek R, Phan AH, Amari SI (2009) Nonnegative matrix and tensor factorizations: applications to exploratory multi-way data analysis and blind source separationpplications to exploratory multi-way data analysis and blind source separation. Wiley, Chichester

    Book  Google Scholar 

  10. Cohen J, Farias R, Comon P (2015) Fast decomposition of large nonnegative tensors. IEEE Signal Process Lett 22(7):862–866. https://doi.org/10.1109/LSP.2014.2374838

    Article  Google Scholar 

  11. Comon P (2014) Tensors: a brief introduction. IEEE Signal Process Mag 31 (3):44–53. https://doi.org/10.1109/MSP.2014.2298533

    Article  Google Scholar 

  12. De Lathauwer L, De Moor B, Vandewalle J (2000) A multilinear singular value decomposition. SIAM J Matrix Anal Appl 21:1253–1278

    Article  MathSciNet  Google Scholar 

  13. De Lathauwer L, Castaing J, Cardoso J (2007) Fourth-order cumulant-based blind identification of underdetermined mixtures. IEEE Trans Signal Process 55 (6):2965–2973. https://doi.org/10.1109/TSP.2007.893943

    Article  MathSciNet  MATH  Google Scholar 

  14. De Silva V, Lim LH (2008) Tensor rank and the ill-posedness of the best low-rank approximation problem. SIAM J Matrix Anal Appl 30(3):1084–1127

    Article  MathSciNet  Google Scholar 

  15. Drineas P, Mahoney MW (2007) A randomized algorithm for a tensor-based generalization of the singular value decomposition. Linear Algebra Appl 420 (2–3):553–571. https://doi.org/10.1016/j.laa.2006.08.023

    Article  MathSciNet  MATH  Google Scholar 

  16. Duchenne O, Bach F, Kweon IS, Ponce J (2011) A tensor-based algorithm for high-order graph matching. IEEE Trans Pattern Anal Mach Intell 33(12):2383–2395. https://doi.org/10.1109/TPAMI.2011.110

    Article  Google Scholar 

  17. Halko N, Martinsson P, Tropp J (2011) Finding structure with randomness: probabilistic algorithms for constructing approximate matrix decompositions. SIAM Rev 53(2):217–288. https://doi.org/10.1137/090771806

    Article  MathSciNet  MATH  Google Scholar 

  18. Kang U, Papalexakis E, Harpale A, Faloutsos C (2012) GigaTensor: scaling tensor analysis up by 100 times - algorithms and discoveries. In: Proceedings of the 18th ACM SIGKDD international conference on knowledge discovery and data mining, KDD ’12. ACM, New York, pp 316–324, DOI https://doi.org/10.1145/2339530.2339583

  19. Kolda TG, Bader BW (2009) Tensor decompositions and applications. SIAM Rev 51(3):455–500

    Article  MathSciNet  Google Scholar 

  20. Lim LH, Comon P (2009) Nonnegative approximations of nonnegative tensors. J Chemometr 23(7-8):432–441. https://doi.org/10.1002/cem.1244

    Article  Google Scholar 

  21. Nesterov Y (1983) A method of solving a convex programming problem with convergence rate o(1/k2). Soviet Math Doklady 27(2):372–376

    MATH  Google Scholar 

  22. Papalexakis EE, Faloutsos C, Sidiropoulos ND (2012) ParCube: sparse parallelizable tensor decompositions. In: European conference on machine learning and knowledge discovery in databases - volume part I, ECML PKDD’12. Springer, Berlin, pp 521–536, DOI https://doi.org/10.1007/978-3-642-33460-3_39

  23. Papalexakis E, Sidiropoulos N, Bro R (2013) From k-means to higher-way co-clustering: multilinear decomposition with sparse latent factors. IEEE Trans Signal Process 61(2):493–506. https://doi.org/10.1109/TSP.2012.2225052

    Article  Google Scholar 

  24. Phan A, Cichocki A (2009) Advances in PARAFAC using parallel block decomposition. In: Leung C, Lee M, Chan J (eds) Neural information processing, lecture notes in computer science, vol 5863. Springer, Berlin, pp 323–330, DOI https://doi.org/10.1007/978-3-642-10677-4_36

  25. Phan H, Tichavsky P, Cichocki A (2013) Fast alternating LS algorithms for high order CANDECOMP/PARAFAC tensor factorizations. IEEE Trans Signal Process 61(19):4834–4846. https://doi.org/10.1109/TSP.2013.2269903

    Article  Google Scholar 

  26. Rajwade A, Rangarajan A, Banerjee A (2013) Image denoising using the higher order singular value decomposition. IEEE Trans Pattern Anal Mach Intell 35 (4):849–862. https://doi.org/10.1109/TPAMI.2012.140

    Article  Google Scholar 

  27. Rodriguez M, Ahmed J, Shah M (2008) Action MACH: a spatio-temporal maximum average correlation height filter for action recognition. In: IEEE Conference on computer vision and pattern recognition, 2008. CVPR 2008, pp 1–8, DOI https://doi.org/10.1109/CVPR.2008.4587727

  28. Sidiropoulos ND, Bro R (2000) On the uniqueness of multilinear decomposition of N-way arrays. J Chemometr 14(3):229–239

    Article  Google Scholar 

  29. Sidiropoulos N, Papalexakis E, Faloutsos C (2014) A parallel algorithm for big tensor decomposition using randomly compressed cubes (PARACOMP). In: IEEE 39th International conference on acoustics, speech and signal processing (ICASSP), pp 1–5, DOI https://doi.org/10.1109/ICASSP.2014.6853546

  30. Tsourakakis C (2010) MACH: fast randomized tensor decompositions. In: Proceedings of the 2010 SIAM international conference on data mining, pp 689–700, DOI https://doi.org/10.1137/1.9781611972801.60

  31. Vasilescu M, Terzopoulos D (2002) Multilinear analysis of image ensembles: tensorfaces. In: Heyden A, Sparr G, Nielsen M, Johansen P (eds) Computer vision — ECCV 2002, lecture notes in computer science, vol 2350. Springer, Berlin, pp 447–460, DOI https://doi.org/10.1007/3-540-47969-4_30

  32. Vervliet N, Debals O, Sorber L, De Lathauwer L (2014) Breaking the curse of dimensionality using decompositions of incomplete tensors: tensor-based scientific computing in big data analysis. IEEE Signal Process Mag 31(5):71–79. https://doi.org/10.1109/MSP.2014.2329429

    Article  Google Scholar 

  33. Zhang Y, Zhou G, Jin J, Wang M, Wang X, Cichocki A (2013) L1-regularized multiway canonical correlation analysis for SSVEP-based BCI. IEEE Trans Neural Syst Rehabil Eng 21(6):887–896. https://doi.org/10.1109/TNSRE.2013.2279680

    Article  Google Scholar 

  34. Zhang Y, Zhou G, Jin J, Wang X, Cichocki A (2014) Frequency recognition in SSVEP-based BCI using multiset canonical correlation analysis. Int J Neur Syst 4(24):1450013(1–14)

    Google Scholar 

  35. Zhang Y, Zhou G, Jin J, Wang X, Cichocki A (2015) SSVEP recognition using common feature analysis in brain–computer interface. J Neurosci Methods 244 (0):8–15. https://doi.org/10.1016/j.jneumeth.2014.03.012

    Article  Google Scholar 

  36. Zhang Y, Zhou G, Zhao Q, Cichocki A, Wang X (2016) Fast nonnegative tensor factorization based on accelerated proximal gradient and low-rank approximation. Neurocomputing 198:148–154

    Article  Google Scholar 

  37. Zhang Y, Nam CS, Zhou G, Jin J, Wang X, Cichocki A (2019) Temporally constrained sparse group spatial patterns for motor imagery bci. IEEE Trans Cybern 49(9):3322–3332

    Article  Google Scholar 

  38. Zhang Y, Zhang H, Chen X, Liu M, Zhu X, Lee SW, Shen D (2019) Strength and similarity guided group-level brain functional network construction for mci diagnosis. Pattern Recogn 88:421–430

    Article  Google Scholar 

  39. Zhou G, Cichocki A (2012) Canonical polyadic decomposition based on a single mode blind source separation. IEEE Signal Process Lett 19(8):523–526. https://doi.org/10.1109/LSP.2012.2205237

    Article  Google Scholar 

  40. Zhou G, Cichocki A, Zhao Q, Xie S (2014) Nonnegative matrix and tensor factorizations: an algorithmic perspective. IEEE Signal Process Mag 31(3):54–65. https://doi.org/10.1109/MSP.2014.2298891

    Article  Google Scholar 

  41. Zhou G, Cichocki A, Zhao Q, Xie S (2015) Efficient nonnegative Tucker decompositions: algorithms and uniqueness. IEEE Trans Image Process 24(12):4990–5003

    Article  MathSciNet  Google Scholar 

  42. Zhou G, Cichocki A, Zhang Y, Mandic D (2016) Group component analysis from multiblock data: common and individual feature extraction. IEEE Transactions on Neural Networks and Learning Systems In print

  43. Zhou G, Zhao Q, Zhang Y, Adali T, Xie S, Cichocki A (2016) Linked component analysis from matrices to high-order tensors: applications to biomedical data. Proc IEEE 104(2):310–331. https://doi.org/10.1109/JPROC.2015.2474704

    Article  Google Scholar 

  44. Zhou T, Zhang C, Gong C, Bhaskar H, Yang J (2018) Multiview latent space learning with feature redundancy minimization. IEEE Transactions on Cybernetics

  45. Zhou T, Liu M, Thung KH, Shen D (2019) Latent representation learning for alzheimer’s disease diagnosis with incomplete multi-modality neuroimaging and genetic data. IEEE Transactions on Medical Imaging

  46. Zhou T, Zhang C, Peng X, Bhaskar H, Yang J (2019) Dual shared-specific multiview subspace clustering. IEEE Transactions on Cybernetics

Download references

Acknowledgements

This work was partially supported by the National Natural Science Foundation of China (NSFC) Grant 61673124, Grant 61973090, and Grant 61727810.

Author information

Authors and Affiliations

  1. School of Automation, Guangdong University of Technology, Guangzhou, China

    Yichun Qiu & Guoxu Zhou

  2. School of Automation, Guangdong-HongKong-Macao Joint Laboratory for Smart Discrete Manufacturing, Guangdong University of Technology, Guangzhou, 510006, China

    Guoxu Zhou

  3. Department of Psychiatry and Behavioral Sciences, Stanford University, Stanford, CA, USA

    Yu Zhang

  4. Skolkovo Institute of Science and Technology (SKOLTECH), 143026, Moscow, Russia

    Andrzej Cichocki

  5. System Research Institute, Polish Academy of Sciences, Warsaw, 00-901, Poland

    Andrzej Cichocki

Authors
  1. Yichun Qiu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guoxu Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yu Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Andrzej Cichocki
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guoxu Zhou.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Qiu, Y., Zhou, G., Zhang, Y. et al. Canonical polyadic decomposition (CPD) of big tensors with low multilinear rank. Multimed Tools Appl 80, 22987–23007 (2021). https://doi.org/10.1007/s11042-020-08711-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-08711-1

Keywords

Search

Navigation