Shuo Zhou
ABSTRACT
Tensors are a natural representation for multidimensional data. In recent years, CANDECOMP/PARAFAC (CP) decomposition, one of the most popular tools for analyzing multi-way data, has been extensively studied and widely applied. However, today's datasets are often dynamically changing over time. Tracking the CP decomposition for such dynamic tensors is a crucial but challenging task, due to the large scale of the tensor and the velocity of new data arriving. Traditional techniques, such as Alternating Least Squares (ALS), cannot be directly applied to this problem because of their poor scalability in terms of time and memory. Additionally, existing online approaches have only partially addressed this problem and can only be deployed on third-order tensors. To fill this gap, we propose an efficient online algorithm that can incrementally track the CP decompositions of dynamic tensors with an arbitrary number of dimensions. In terms of effectiveness, our algorithm demonstrates comparable results with the most accurate algorithm, ALS, whilst being computationally much more efficient. Specifically, on small and moderate datasets, our approach is tens to hundreds of times faster than ALS, while for large-scale datasets, the speedup can be more than 3,000 times. Compared to other state-of-the-art online approaches, our method shows not only significantly better decomposition quality, but also better performance in terms of stability, efficiency and scalability.
- E. Acar, et al. Scalable tensor factorizations for incomplete data. Chemometrics and Intelligent Laboratory Systems, 106(1):41--56, March 2011.Google Scholar
Cross Ref
- K. Altun, et al. Comparative study on classifying human activities with miniature inertial and magnetic sensors. Pattern Recognition, 43(10):3605--3620, 2010. Google ScholarDigital Library
- M. Bachlin, et al. Wearable assistant for Parkinson's disease patients with the freezing of gait symptom. IEEE Trans. Inf. Tech. Biomed., 14(2):436--446, 2010. Google ScholarDigital Library
- B. W. Bader, T. G. Kolda, et al. Matlab tensor toolbox version 2.6. Available online, February 2015.Google Scholar
- Y. Cai, et al. Facets: Fast comprehensive mining of coevolving high-order time series. In SIGKDD, 2015. Google ScholarDigital Library
- A. Cichocki. Era of big data processing: a new approach via tensor networks and tensor decompositions. arXiv preprint arXiv:1403.2048, 2014.Google Scholar
- A. Cichocki, et al. Tensor decompositions for signal processing applications: From two-way to multiway component analysis. Signal Processing Magazine, IEEE, 32(2):145--163, 2015.Google ScholarCross Ref
- P. Comon, X. Luciani, and A. L. De Almeida. Tensor decompositions, alternating least squares and other tales. J. Chemometrics, 23(7--8):393--405, 2009.Google ScholarCross Ref
- L. De Lathauwer, et al. On the best rank-1 and rank-(r 1, r 2,..., rn) approximation of higher-order tensors. SJMAEL, 21(4):1324--1342, 2000. Google ScholarDigital Library
- D. M. Dunlavy, T. G. Kolda, and E. Acar. Temporal link prediction using matrix and tensor factorizations. TKDD, 5(2):10, 2011. Google ScholarDigital Library
- H. Fanaee-T and J. Gama. Multi-aspect-streaming tensor analysis. Knowledge-Based Systems, 89:332--345, 2015. Google ScholarDigital Library
- J. Fonollosa, et al. Reservoir computing compensates slow response of chemosensor arrays exposed to fast varying gas concentrations in continuous monitoring. Sens. Actuator B-Chem., 215:618--629, 2015.Google ScholarCross Ref
- C. Hu, et al. Scalable bayesian non-negative tensor factorization for massive count data. In ECML-PKDD, 2015.Google ScholarCross Ref
- W. Hu, et al. Incremental tensor subspace learning and its applications to foreground segmentation and tracking. IJCV, 91(3):303--327, 2011. Google ScholarDigital Library
- T. G. Kolda. Multilinear operators for higher-order decompositions. Tech. Report SAND2006--2081, Sandia National Laboratories, April 2006.Google Scholar
- T. G. Kolda and B. W. Bader. Tensor decompositions and applications. SIAM review, 51(3):455--500, 2009. Google ScholarDigital Library
- W. Liu, et al. Utilizing common substructures to speedup tensor factorization for mining dynamic graphs. In CIKM, 2012. Google ScholarDigital Library
- X. Ma, et al. Dynamic updating and downdating matrix svd and tensor hosvd for adaptive indexing and retrieval of motion trajectories. In ICASSP, 2009. Google ScholarDigital Library
- S. A. Nene, et al. Columbia Object Image Library (COIL-20). Tech. report, Feb 1996.Google Scholar
- D. Nion, et al. Adaptive algorithms to track the parafac decomposition of a third-order tensor. IEEE Trans. Sig. Process., 57(6):2299--2310, 2009. Google ScholarDigital Library
- S. Papadimitriou, et al. Streaming pattern discovery in multiple time-series. In VLDB, 2005. Google ScholarDigital Library
- A. H. Phan, et al. Parafac algorithms for large-scale problems. Neurocomputing, 74(11):1970--1984, 2011. Google ScholarDigital Library
- F. S. Samaria, et al. Parameterisation of a stochastic model for human face identification. In WACV, 1994.Google ScholarCross Ref
- F. Schimbinschi, et al. Traffic forecasting in complex urban networks: Leveraging big data and machine learning. In Big Data, 2015. Google ScholarDigital Library
- L. Shi, et al. Stensr: Spatio-temporal tensor streams for anomaly detection and pattern discovery. KAIS, 43(2):333--353, 2015. Google ScholarDigital Library
- A. Sobral, et al. Incremental and multi-feature tensor subspace learning applied for background modeling and subtraction. In ICIAR, 2014.Google ScholarCross Ref
- J. Sun, D. Tao, and C. Faloutsos. Beyond streams and graphs: dynamic tensor analysis. In SIGKDD, 2006. Google ScholarDigital Library
- J. Sun, et al. Incremental tensor analysis: Theory and applications. TKDD, 2(3):11, 2008. Google ScholarDigital Library
- M. A. O. Vasilescu, et al. Multilinear analysis of image ensembles: Tensorfaces. In ECCV, 2002. Google ScholarDigital Library
- M. Zhang and A. A. Sawchuk. Usc-had: A daily activity dataset for ubiquitous activity recognition using wearable sensors. In Ubicomp-SAGAware, 2012. Google ScholarDigital Library
Index Terms
- Accelerating Online CP Decompositions for Higher Order Tensors
Recommendations
Decomposition-by-normalization (DBN): leveraging approximate functional dependencies for efficient CP and tucker decompositions
For many multi-dimensional data applications, tensor operations as well as relational operations both need to be supported throughout the data lifecycle. Tensor based representations (including two widely used tensor decompositions, CP and Tucker ...
Tensor Rank Estimation and Completion via CP-based Nuclear Norm
CIKM '17: Proceedings of the 2017 ACM on Conference on Information and Knowledge ManagementTensor completion (TC) is a challenging problem of recovering missing entries of a tensor from its partial observation. One main TC approach is based on CP/Tucker decomposition. However, this approach often requires the determination of a tensor rank a ...
On the Nuclear Norm and the Singular Value Decomposition of Tensors
Finding the rank of a tensor is a problem that has many applications. Unfortunately, it is often very difficult to determine the rank of a given tensor. Inspired by the heuristics of convex relaxation, we consider the nuclear norm instead of the rank of ...
Comments