skip to main content
10.1145/3459637.3481919acmconferencesArticle/Chapter ViewAbstractPublication PagescikmConference Proceedingsconference-collections
research-article

Flexible O&M for Telecom Networks at Huawei: A Language Model-based Approach

Published: 30 October 2021 Publication History

Abstract

Flexible operation and maintenance (O&M) is critical for telecommunication (telecom) service providers due to the ever-growing communication networks. Currently, most O&M operations still rely on rule-based strategies, which only cover limited scenarios and are costly to extend for novel applications as expert knowledge is intensively involved. To build a more flexible O&M system, we propose a language model to extract useful representations out of massive network signaling messages and use the representations to perform downstream O&M tasks. Given that a vanilla language model is not directly applicable for the structured signaling messages, we develop an expert-knowledge-inspired statistical approach to preprocess the messages and a hierarchical network architecture to extract message relations among different levels. Moreover, network messages in the real world are often contaminated, which can mislead the language model to learn incorrect message patterns. To mitigate data contamination, we propose a reverse training method that prevents the language model from learning the contaminated data. We collected hundreds of thousands of signaling message flows to train the proposed signaling language model and applied the trained model to O&M tasks. Offline experiments show that our proposed language model captures various signaling protocols and the extracted representations enable us to achieve expert-level performance in network anomaly detection and service recognition. Our language model has been deployed online at Huawei and significantly improved O&M efficiency.

Supplementary Material

MP4 File (CIKM21-afp0867.mp4)
Presentation video for the paper "Flexible OM for Telecom Networks at Huawei: A Language Model-based Approach" in Proceedings of the 30th ACM International Conference on Information and Knowledge Management.

References

[1]
Zeyuan Allen-Zhu, Yuanzhi Li, and Yingyu Liang. 2019 a. Learning and generalization in overparameterized neural networks, going beyond two layers. In Advances in Neural Information Processing Systems. Curran Associates, Inc., Vancouver, BC, Canada, 6158--6169.
[2]
Zeyuan Allen-Zhu, Yuanzhi Li, and Zhao Song. 2019 b. A convergence theory for deep learning via over-parameterization. In Proceedings of International Conference on Machine Learning. PMLR, Long Beach, CA, USA, 242--252.
[3]
Julien Audibert, Pietro Michiardi, Frédéric Guyard, Sébastien Marti, and Maria A Zuluaga. 2020. USAD: Unsupervised anomaly detection on multivariate time series. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, Virtual Event, 3395--3404.
[4]
Sara Ayoubi, Noura Limam, Mohammad A Salahuddin, Nashid Shahriar, Raouf Boutaba, Felipe Estrada-Solano, and Oscar M Caicedo. 2018. Machine learning for cognitive network management. IEEE Communications Magazine, Vol. 56, 1 (2018), 158--165.
[5]
Ricardo Baeza-Yates and Berthier Ribeiro-Neto. 2001. Modern Information Retrieval. Pearson, London, UK. 68--74 pages.
[6]
Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Jauvin. 2003. A neural probabilistic language model. Journal of Machine Learning Research, Vol. 3 (2003), 1137--1155.
[7]
Raouf Boutaba, Mohammad A Salahuddin, Noura Limam, Sara Ayoubi, Nashid Shahriar, Felipe Estrada-Solano, and Oscar M Caicedo. 2018. A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities. Journal of Internet Services and Applications, Vol. 9, 1 (2018), 1--99.
[8]
Kay Henning Brodersen, Cheng Soon Ong, Klaas Enno Stephan, and Joachim M Buhmann. 2010. The balanced accuracy and its posterior distribution. In Proceedings of the International Conference on Pattern Recognition. IEEE, Istanbul, Turkey, 3121--3124.
[9]
Andy Brown, Aaron Tuor, Brian Hutchinson, and Nicole Nichols. 2018. Recurrent neural network attention mechanisms for interpretable system log anomaly detection. In Proceedings of Workshop on Machine Learning for Computing Systems. ACM, Tempe, AZ, USA, 1--8.
[10]
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Mark Hesse, Christopher Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Advances in Neural Information Processing Systems. Curran Associates, Inc., Vancouver, BC, Canada.
[11]
Lars Buitinck, Gilles Louppe, Mathieu Blondel, Fabian Pedregosa, Andreas Mueller, Olivier Grisel, Vlad Niculae, Peter Prettenhofer, Alexandre Gramfort, Jaques Grobler, Robert Layton, Jake VanderPlas, Arnaud Joly, Brian Holt, and Gaë l Varoquaux. 2013. API design for machine learning software: Experiences from the scikit-learn project. In ECML PKDD Workshop: Languages for Data Mining and Machine Learning. Springer, Prague, Czech Republic, 108--122.
[12]
Zihang Dai, Zhilin Yang, Yiming Yang, Jaime G Carbonell, Quoc Le, and Ruslan Salakhutdinov. 2019. Transformer-XL: Attentive language models beyond a fixed-length context. In Proceedings of Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). ACL, Florence, Italy, 2978--2988.
[13]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. ACL, Minneapolis, MN, USA, 4171--4186.
[14]
Min Du, Feifei Li, Guineng Zheng, and Vivek Srikumar. 2017. Deeplog: Anomaly detection and diagnosis from system logs through deep learning. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. ACM, Dallas, TX, USA, 1285--1298.
[15]
Ana Gomez-Andrades, Raquel Barco, Immaculada Serrano, Patricia Delgado, Patricia Caro-Oliver, and Pablo Munoz. 2016. Automatic root cause analysis based on traces for LTE self-organizing networks. IEEE Wireless Communications, Vol. 23, 3 (2016), 20--28.
[16]
Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. 2018. Co-teaching: Robust training of deep neural networks with extremely noisy labels. In Advances in Neural Information Processing Systems. Curran Associates, Inc., Montréal, Canada, 8527--8537.
[17]
Zellig S Harris. 1954. Distributional structure. Word, Vol. 10, 2--3 (1954), 146--162.
[18]
Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of International Conference on Learning Representations. OpenReview.net, San Diego, CA, USA.
[19]
Stefan Kombrink, Tomávs Mikolov, Martin Karafiát, and Lukávs Burget. 2011. Recurrent neural network based language modeling in meeting recognition. In Proceedings of the Annual Conference of the International Speech Communication Association (INTERSPEECH). ISCA, Florence, Italy, 2877--2880.
[20]
Rui Lin, Shujie Liu, Muyun Yang, Mu Li, Ming Zhou, and Sheng Li. 2015. Hierarchical recurrent neural network for document modeling. In Proceedings of Conference on Empirical Methods in Natural Language Processing. ACL, Lisbon, Portugal, 899--907.
[21]
Weibin Meng, Ying Liu, Shenglin Zhang, Dan Pei, Hui Dong, Lei Song, and Xulong Luo. 2018. Device-agnostic log anomaly classification with partial labels. In Proceedings of IEEE/ACM International Symposium on Quality of Service (IWQoS). IEEE, Banff, AB, Canada, 1--10.
[22]
Tomas Mikolov and Geoffrey Zweig. 2012. Context dependent recurrent neural network language model. In IEEE Spoken Language Technology Workshop (SLT). IEEE, Miami, FL, USA, 234--239.
[23]
Jessica Moysen and Lorenza Giupponi. 2018. From 4G to 5G: Self-organized network management meets machine learning. Computer Communications, Vol. 129 (2018), 248--268.
[24]
Preetum Nakkiran, Gal Kaplun, Yamini Bansal, Tristan Yang, Boaz Barak, and Ilya Sutskever. 2019. Deep double descent: where bigger models and more data hurt. In Proceedings of International Conference on Learning Representations. OpenReview.net, New Orleans, LA, USA.
[25]
Behnam Neyshabur, Zhiyuan Li Li, Srinadh Bhojanapalli, Yann LeCun, and Nathan Srebro. 2019. The role of over-parametrization in generalization of neural networks. In Proceedings of International Conference on Learning Representations. OpenReview.net, New Orleans, LA, USA.
[26]
Ryuji Oda, Junji Tagane, Kazunori Umezaki, and Katsuyuki Fujiyoshi. 2009. Network quality monitoring device and method for internet services involving signaling. US Patent 8,300,633, Issued Oct. 30th., 2012.
[27]
Guansong Pang, Chunhua Shen, Longbing Cao, and Anton Van Den Hengel. 2021. Deep learning for anomaly detection: A review. ACM Computing Surveys (CSUR), Vol. 54, 2 (2021), 1--38.
[28]
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems. Curran Associates, Inc., Vancouver, BC, Canada, 8026--8037.
[29]
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf
[30]
Hansheng Ren, Bixiong Xu, Yujing Wang, Chao Yi, Congrui Huang, Xiaoyu Kou, Tony Xing, Mao Yang, Jie Tong, and Qi Zhang. 2019. Time-series anomaly detection service at Microsoft. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, Anchorage, AK, USA, 3009--3017.
[31]
Yanyao Shen and Sujay Sanghavi. 2019. Learning with bad training data via iterative trimmed loss minimization. In Proceedings of International Conference on Machine Learning. PMLR, Long Beach, CA, USA, 5739--5748.
[32]
Ya Su, Youjian Zhao, Chenhao Niu, Rong Liu, Wei Sun, and Dan Pei. 2019. Robust anomaly detection for multivariate time series through stochastic recurrent neural network. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, Anchorage, AK, USA, 2828--2837.
[33]
John G van Bosse and Fabrizio U Devetak. 2006. Signaling in Telecommunication Networks. John Wiley & Sons, New York.
[34]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems. Curran Associates, Inc., Long Beach, CA, USA, 5998--6008.
[35]
Eric Wallace, Yizhong Wang, Sujian Li, Sameer Singh, and Matt Gardner. 2019. Do NLP models know numbers? Probing numeracy in embeddings. In Proceedings of Conference on Empirical Methods in Natural Language Processing and International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). ACL, Hong Kong, SAR, China, 5310--5318.
[36]
Tian Wang and Kyunghyun Cho. 2016. Larger-Context Language Modelling with Recurrent Neural Network. In Proceedings of Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). ACL, Berlin, Germany, 1319--1329.
[37]
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. 2017. Understanding deep learning requires rethinking generalization. In Proceedings of International Conference on Learning Representations. OpenReview.net, Toulon, France.
[38]
Chuxu Zhang, Dongjin Song, Yuncong Chen, Xinyang Feng, Cristian Lumezanu, Wei Cheng, Jingchao Ni, Bo Zong, Haifeng Chen, and Nitesh V Chawla. 2019. A deep neural network for unsupervised anomaly detection and diagnosis in multivariate time series data. In Proceedings of the AAAI Conference on Artificial Intelligence. AAAI Press, Honolulu, HI, USA, 1409--1416.

Index Terms

  1. Flexible O&M for Telecom Networks at Huawei: A Language Model-based Approach

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      CIKM '21: Proceedings of the 30th ACM International Conference on Information & Knowledge Management
      October 2021
      4966 pages
      ISBN:9781450384469
      DOI:10.1145/3459637
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 30 October 2021

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. language model
      2. network o&m
      3. signaling

      Qualifiers

      • Research-article

      Conference

      CIKM '21
      Sponsor:

      Acceptance Rates

      Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

      Upcoming Conference

      CIKM '25

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • 0
        Total Citations
      • 216
        Total Downloads
      • Downloads (Last 12 months)55
      • Downloads (Last 6 weeks)6
      Reflects downloads up to 14 Feb 2025

      Other Metrics

      Citations

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media