Skip to main content
SpringerLink
Log in

Internet traffic tensor completion with tensor nuclear norm

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

The incomplete data is a common phenomenon in traffic network because of the high measurement cost, the failure of data collection systems and unavoidable transmission loss. Recovering the whole data from incomplete data is a very important task in internet engineering and management. In this paper, we adopt the low-rank tensor completion model equipped with tensor nuclear norm to reconstruct the internet traffic data. Besides using a low rank tensor to capture the global information of internet traffic data, we also utilize spatial correlation and periodicity to characterize the local information. The resulting model is a convex and separable optimization. Then, a proximal alternating direction method of multipliers is customized to solve the optimization problem, where all subproblems have closed-form solutions. Convergence analysis of the algorithm is given without any assumptions. Numerical experiments on Abilene and GÉANT datasets with random missing and structured loss show that the proposed model and algorithm perform better than other existing algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Algorithm 1
Algorithm 2
Algorithm 3
Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Data Availability

Abilene and GÉANT datasets used in this study are available in https://roughan.info/data/Abilene.tar.gz and https://totem.info.ucl.ac.be/dataset.html, respectively.

References

  1. Ouyang, Y., Xie, K., Wang, X., Wen, J., Zhang, G.: Lightweight trilinear pooling based tensor completion for network traffic monitoring. In: IEEE INFOCOM 2022—IEEE Conference on Computer Communications, pp. 2128–2137 (2022)

  2. Roughan, M., Zhang, Y., Willinger, W., Qiu, L.: Spatio-temporal compressive sensing and internet traffic matrices (extended version). IEEE/ACM Trans. Netw. 20(3), 662–676 (2012)

    Article  Google Scholar 

  3. Chen, Y. C., Qiu, L., Zhang, Y., Xue, G., Hu, Z.: Robust network compressive sensing. In: Proceedings of The 20th Annual International Conference on Mobile Computing and Networking, pp. 545–556 (2014)

  4. Gürsun, G., Crovella, M.: On traffic matrix completion in the internet. In: Proceedings of The 2012 Internet Measurement Conference, pp. 399–412 (2012)

  5. Kumar, A., Saradhi, V. V., Venkatesh, T.: Compressive sensing of internet traffic matrices using CUR decomposition. In: Proceedings of the 19th International Conference on Distributed Computing and Networking, pp. 1–7 (2018)

  6. Li, D., Xing, C., Zhang, G., Cao, H., Xu, B.: An online dynamic traffic matrix completion method in software defined networks. Comput. Commun. 145, 43–53 (2019)

    Article  Google Scholar 

  7. Lakhina, A., Papagiannaki, K., Crovella, M., Diot, C., Kolaczyk, E., Taft, N.: Structural analysis of network traffic flows. ACM Sigmetrics Perform. Eval. Rev. 32(1), 61–72 (2004)

    Article  Google Scholar 

  8. Xie, K., Wang, L., Wang, X., Xie, G., Wen, J., Zhang, G.: Accurate recovery of internet traffic data: a tensor completion approach. In: IEEE INFOCOM 2016—The 35th Annual IEEE International Conference on Computer Communications, pp. 1–9 (2016)

  9. Liu, J., Musialski, P., Wonka, P., Ye, J.: Tensor completion for estimating missing values in visual data. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 208–220 (2013)

    Article  Google Scholar 

  10. Zhang, Y., Silva, C.D., Kumar, R., Herrmann, F.J.: Massive 3D seismic data compression and inversion with hierarchical Tucker. SEG Tech. Program Expand. Abstr. 1347–1352, 2017 (2017)

    Google Scholar 

  11. Mørup, M.: Applications of tensor (multiway array) factorizations and decompositions in data mining. Wiley Interdiscip. Rev. Data Min. Knowl. Discov. 1(1), 24–40 (2011)

    Article  Google Scholar 

  12. Karatzoglou, A., Amatriain, X., Baltrunas, L., Oliver, N.: Multiverse recommendation: n-dimensional tensor factorization for context-aware collaborative filtering. In: Proceedings of The Fourth ACM Conference on Recommender Systems, pp. 79–86 (2010)

  13. Huang, Z., Qi, L.: Formulating an \(n\)-person noncooperative game as a tensor complementarity problem. Comput. Optim. Appl. 66, 557–576 (2017)

    Article  MathSciNet  Google Scholar 

  14. Carroll, J.D., Chang, J.J.: Analysis of individual differences in multidimensional scaling via an N-way generalization of “Eckart-Young’’ decomposition. Psychometrika 35, 283–319 (1970)

    Article  Google Scholar 

  15. Kolda, T.G., Bader, B.W.: Tensor decompositions and applications. SIAM Rev. 51(3), 455–500 (2009)

    Article  MathSciNet  Google Scholar 

  16. Acar, E., Dunlavy, D.M., Kolda, T.G., Mørup, M.: Scalable tensor factorizations for incomplete data. Chemom. Intell. Lab. Syst. 106(1), 41–56 (2011)

    Article  Google Scholar 

  17. Zhou, H., Zhang, D., Xie, K., Chen, Y.: Spatio-temporal tensor completion for imputing missing internet traffic data. In: 2015 IEEE 34th International Performance Computing and Communications Conference, pp. 1–7 (2015)

  18. Xie, K., Peng, C., Wang, X., Xie, G., Wen, J., Cao, J., Zhang, D., Qin, Z.: Accurate recovery of internet traffic data under variable rate measurements. IEEE/ACM Trans. Netw. 26(3), 1137–1150 (2018)

    Article  Google Scholar 

  19. Wang, L., Xie, K., Semong, T., Zhou, H.: Missing data recovery based on tensor-CUR decomposition. IEEE Access 6, 532–544 (2017)

    Article  Google Scholar 

  20. Qi, L., Chen, Y., Bakshi, M., Zhang, X.: Triple decomposition and tensor recovery of third order tensors. SIAM J. Matrix Anal. Appl. 42(1), 299–329 (2021)

    Article  MathSciNet  Google Scholar 

  21. Chen, Y., Zhang, X., Qi, L., Xu, Y.: A Barzilai–Borwein gradient algorithm for spatio-temporal internet traffic data completion via tensor triple decomposition. J. Sci. Comput. 88, 1–24 (2021)

    Article  MathSciNet  Google Scholar 

  22. Zhang, Z., Ling, C., He, H., Qi, L.: A tensor train approach for internet traffic data completion. Ann. Oper. Res. (2021). https://doi.org/10.1007/s10479-021-04147-4

    Article  Google Scholar 

  23. Oseledets, I.V.: Tensor-train decomposition. SIAM J. Sci. Comput. 33(5), 2295–2317 (2011)

    Article  MathSciNet  Google Scholar 

  24. Zhou, P., Lu, C., Lin, Z., Zhang, C.: Tensor factorization for low-rank tensor completion. IEEE Trans. Image Process. 27(3), 1152–1163 (2017)

    Article  MathSciNet  Google Scholar 

  25. Ling, C., Yu, G., Qi, L., Xu, Y.: T-product factorization method for internet traffic data completion with spatio-temporal regularization. Comput. Optim. Appl. 80(3), 883–913 (2021)

    Article  MathSciNet  Google Scholar 

  26. Yu, G., Wang, L., Wan, S., Qi, L., Xu, Y.: Tensor factorization with total variation for internet traffic data imputation. Pac. J. Optim. 17(3), 486–505 (2021)

    MathSciNet  Google Scholar 

  27. He, H., Ling, C., Xie, W.: Tensor completion via a generalized transformed tensor T-product decomposition without t-SVD. J. Sci. Comput. 93, 47 (2022)

    Article  MathSciNet  Google Scholar 

  28. Håstad, J.: Tensor rank is NP-complete. J. Algorithms 11(4), 644–654 (1990)

    Article  MathSciNet  Google Scholar 

  29. Liu, J., Musialski, P., Wonka, P., Ye, J.: Tensor completion for estimating missing values in visual data. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 208–220 (2013)

    Article  Google Scholar 

  30. Romera-Paredes, B., Pontil, M.: A new convex relaxation for tensor completion. In: Proceeding of the 26th International Conference on Neutral Information Processing Systems, pp. 2967–2975 (2013)

  31. Kilmer, M.E., Martin, C.D.: Factorization strategies for third-order tensors. Linear Algebra Appl. 435(3), 641–658 (2011)

    Article  MathSciNet  Google Scholar 

  32. Zhang, Z., Ely, G., Aeron, S., Hao, N., Kilmer, M.: Novel methods for multilinear data completion and de-noising based on tensor-SVD. In: Proceedings of The IEEE Conference on Computer Vision and Pattern Recognition, pp. 3842–3849 (2014)

  33. Kong, H., Lu, C., Lin, Z.: Tensor Q-rank: new data dependent definition of tensor rank. Mach. Learn. 110(7), 1867–1900 (2021)

    Article  MathSciNet  Google Scholar 

  34. Lu, C., Feng, J., Yan, S., Lin, Z.: A unified alternating direction method of multipliers by majorization minimization. IEEE Trans. Pattern Anal. Mach. Intell. 40(3), 527–541 (2018)

    Article  Google Scholar 

  35. Lu, C., Feng, J., Chen, Y., Lin, Z., Yan, S.: Tensor robust principal component analysis with a new tensor nuclear norm. IEEE Trans. Pattern Anal. Mach. Intell. 42(4), 925–938 (2020)

    Article  Google Scholar 

  36. Semerci, O., Hao, N., Kilmer, M.E., Mille, E.L.: Tensor-based formulation and nuclear norm regularization for multienergy computed tomography. IEEE Trans. Image Process. 23(4), 1678–1693 (2014)

    Article  MathSciNet  Google Scholar 

  37. Rockafellar, R.T.: Convex Analysis. Princeton University Press, Princeton, NJ (1970)

    Book  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Mathematical Sciences, South China Normal University, Guangzhou, 510631, China

    Can Li, Yannan Chen & Dong-Hui Li

  2. School of Mathematics and Statistics, Honghe University, Mengzi, 661199, China

    Can Li

Authors
  1. Can Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yannan Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dong-Hui Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yannan Chen.

Ethics declarations

Conflict of interest

The authors declare that they have no conflicts of interest to this work.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

C. Li: This author’s work was supported by Yunnan Natural Science Foundation (No. 202101BA070001-047). Y. Chen: This author’s work was supported by NSFC (Grant Nos. 12171168 and 12071159) and Guangdong Basis and Applied Basic Research Foundation (Nos. 2021A1515012032 and 2022A1515011123). D. Li: This author’s work was supported by NSFC (Grant No. 12271187).

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, C., Chen, Y. & Li, DH. Internet traffic tensor completion with tensor nuclear norm. Comput Optim Appl 87, 1033–1057 (2024). https://doi.org/10.1007/s10589-023-00545-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-023-00545-5

Keywords

Mathematics Subject Classification

Search

Navigation