Abstract
Nowadays, heterogeneous event sequence data is an inseparable and extremely important part for our daily life. The extraordinary nature of event sequence data is characterized by its existing complex long-term and short-term temporal dependencies. Most point process models based on a recurrent neural network fail to capture these dependencies and make accurate predictions. The Transformer Hawkes Process(THP) model, utilizes the self-attention mechanism to capture long-term dependencies, which is suitable and effective for the prediction of event sequence data. Graph contrastive learning (GCL) with adaptive reinforcement can enhance data by making the intra-class hidden features of the instances close while keeping the inter-class hidden features scattered away. Inspired by these, we propose the idea of combining the THP with adaptive enhanced GCL. The proposed Hawkes Process via Graph Contrastive Discriminant representation Learning and Transformer capturing long-term dependencies(GCDRLT) is the two-stage pipeline to enhance the capacity of hidden representation both on long-term dependencies and discriminant feature extraction. Experimental results on multiple datasets validate that the graph contrastive learning method can improve the accuracies of the Transformer-Hawkes process model for predicting heterogeneous event sequences.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Hawkes, A.G.: Spectra of some self-exciting and mutually exciting point processes. Biometrika 58, 83–90 (1971)
Isham, V., Westcott, M.: A self-correcting point process. Stoch. Process. App. 8, 335–347 (1979)
Xiao, S., et al.: Wasserstein learning of deep generative point process models. In: NIPS (2017)
Li, S., Xiao, S., Zhu, S., Du, N., Xie, Y., Song, L.: Learning temporal point processes via reinforcement learning. arXiv:abs/1811.05016 (2018)
Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv:abs/1810.04805 (2019)
Zuo, S., Jiang, H., Li, Z., Zhao, T., Zha, H.: Transformer Hawkes process. In: ICML (2020)
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. arXiv:abs/1409.0473 (2015)
Linsker, R.: Self-organization in a perceptual network. Computer 21, 105–117 (1988)
Wu, M., Zhuang, C., Mosse, M., Yamins, D.L.K., Goodman, N.D.: On mutual information in contrastive learning for visual representations. arXiv:abs/2005.13149 (2020)
Bachman, P., Hjelm, R.D., Buchwalter, W.: Learning representations by maximizing mutual information across views. In: NeurIPS (2019)
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I.: Language models are unsupervised multitask learners (2019)
Hjelm, R.D., Fedorov, A., Lavoie-Marchildon, S., Grewal, K., Trischler, A., Bengio, Y.: Learning deep representations by mutual information estimation and maximization. arXiv:abs/1808.06670 (2019)
Hénaff, O.J., et al.: Data-efficient image recognition with contrastive predictive coding. arXiv:abs/1905.09272 (2020)
van den Oord, A., Li, Y., Vinyals, O.: Representation learning with contrastive predictive coding. arXiv:abs/1807.03748 (2018)
Vaswani, A., et al.: Attention is all you need. arXiv:abs/1706.03762 (2017)
Poole, B., Ozair, S., van den Oord, A., Alemi, A.A., Tucker, G.: On variational bounds of mutual information. In: ICML (2019)
Tschannen, M., Djolonga, J., Rubenstein, P.K., Gelly, S., Lucic, M.: On mutual information maximization for representation learning. arXiv:abs/1907.13625 (2020)
Hassani, K., Ahmadi, A.H.K.: Contrastive multi-view representation learning on graphs. arXiv:abs/2006.05582 (2020)
Zhu, Y., Xu, Y., Yu, F., Liu, Q., Wu, S., Wang, L.: Deep graph contrastive representation learning. arXiv:abs/2006.04131 (2020)
Zhu, Y., Xu, Y., Yu, F., Liu, Q., Wu, S., Wang, L.: Graph contrastive learning with adaptive augmentation. In: Proceedings of the Web Conference 2021 (2021)
Du, N., Dai, H., Trivedi, R.S., Upadhyay, U., Gomez-Rodriguez, M., Song, L.: Recurrent marked temporal point processes: Embedding event history to vector. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2016)
Mei, H., Eisner, J.: The neural Hawkes process: a neurally self-modulating multivariate point process. In: NIPS (2017)
Johnson, A.E.W., et al.: MIMIC-III, a freely accessible critical care database. Sci. Data 3, 1–9 (2016)
Leskovec, J., Krevl, A.: SNAP datasets: Stanford large network dataset collection (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Cao, Z., Liu, Jw., Cheng, Zh. (2023). Hawkes Process via Graph Contrastive Discriminant Representation Learning and Transformer Capturing Long-Term Dependencies. In: Tanveer, M., Agarwal, S., Ozawa, S., Ekbal, A., Jatowt, A. (eds) Neural Information Processing. ICONIP 2022. Communications in Computer and Information Science, vol 1791. Springer, Singapore. https://doi.org/10.1007/978-981-99-1639-9_5
Download citation
DOI: https://doi.org/10.1007/978-981-99-1639-9_5
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-1638-2
Online ISBN: 978-981-99-1639-9
eBook Packages: Computer ScienceComputer Science (R0)