Skip to main content
SpringerLink
Log in

HGTHP: a novel hyperbolic geometric transformer hawkes process for event prediction

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Event sequences with spatiotemporal characteristics have been rapidly produced in various domains, such as earthquakes in seismology, electronic medical records in health care, and transactions in the financial market. These data are discrete events that are continuous and often continue for weeks, months, or years, and past events may trigger subsequent events. In this context, modeling spatiotemporal event sequences and forecasting the occurrence time and marker of the next event has become a hot topic. However, existing models either fail to capture the long-term temporal dependencies or ignore the essential spatial information between sequences. Moreover, existing models learn the influence of past events and predicted future events in Euclidean space, which has been shown to cause a significant distortion in hierarchical structure data. To correctly predict future events from historical events and design interventions and controls to guide the event dynamics to the desired results and inspired by the high capacity of modeling data in hyperbolic space, we proposed a novel hyperbolic graph transformer Hawkes process (HGTHP) model to capture the long-term temporal dependencies and spatial information from historical events with a hierarchical structure. The core concept of the HGTHP is to integrate the learned spatial information into the event embedding as auxiliary information and capture long-short term temporal dependencies from event sequences in non-Euclidean space by a hyperbolic self-attention mechanism. Numerous experiments on synthetic and real-world datasets proved that the proposed model obtains spatiotemporal information from hyperbolic space, and its predictions outperform those of state-of-the-art baselines in both time and marker, proving the proposed model’s effectiveness.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Algorithm 1
Algorithm 2
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Zhang L, Liu J, Zuo X (2023) Doubly time-dependent hawkes process and applications in failure sequence analysis. Comput Stat 38(2):1057–1093

    Article  MathSciNet  Google Scholar 

  2. Okawa M, Iwata T, Tanaka Y, Kurashima T, Toda H, Kashima H (2022) Context-aware spatio-temporal event prediction via convolutional hawkes processes. Mach Learn 111(8):2929–2950

    Article  MathSciNet  Google Scholar 

  3. Cui Z, Sun X, Pan L, Liu S, Xu G (2023) Event-based incremental recommendation via factors mixed hawkes process. Inf Sci 639:119007

    Article  Google Scholar 

  4. Jiang Y, Porter M.D (2022) Simulating fake news dissemination on twitter with multivariate hawkes processes. In: IEEE international conference on big data, big data 2022, Osaka, Japan, December 17-20, 2022, pp 3597–3606

  5. Shi Z, Cartlidge J (2022) State dependent parallel neural hawkes process for limit order book event stream prediction and simulation. In: KDD ’22: The 28th ACM SIGKDD conference on knowledge discovery and data mining, Washington, DC, USA, August 14 - 18, 2022, pp 1607–1615

  6. Ye F, Hu Z, Yilmaz E (2023) Modeling user satisfaction dynamics in dialogue via hawkes process. In: Proceedings of the 61st annual meeting of the association for computational linguistics (Volume 1: Long Papers), ACL 2023, Toronto, Canada, July 9-14, 2023, pp 8875–8889

  7. Bronstein MM, Bruna J, LeCun Y, Szlam A, Vandergheynst P (2017) Geometric deep learning: going beyond euclidean data. IEEE Signal Process Mag 34(4):18–42

    Article  Google Scholar 

  8. Khrulkov V, Mirvakhabova L, Ustinova E, Oseledets IV, Lempitsky VS (2020) Hyperbolic image embeddings. In: 2020 IEEE/CVF conference on computer vision and pattern recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, pp 6417–6427

  9. Sala F, Sa CD, Gu A, Ré C (2018) Representation tradeoffs for hyperbolic embeddings. In: Proceedings of the 35th international conference on machine learning, ICML 2018, vol 80. Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, pp 4457–4466

  10. Gülçehre Ç Denil M, Malinowski M, Razavi A, Pascanu R, Hermann KM, Battaglia PW, Bapst V, Raposo D, Santoro A, Freitas N (2019) Hyperbolic attention networks. In: 7th international conference on learning representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019

  11. Dhingra B, Shallue CJ, Norouzi M, Dai AM, Dahl GE (2018) Embedding text in hyperbolic spaces. In: Proceedings of the Twelfth Workshop on Graph-Based Methods for Natural Language Processing, TextGraphs@NAACL-HLT 2018, New Orleans, Louisiana, USA, June 6, 2018, pp 59–69

  12. Peng W, Varanka T, Mostafa A, Shi H, Zhao G (2022) Hyperbolic deep neural networks: a survey. IEEE Trans Pattern Anal Mach Intell 44(12):10023–10044

    Article  Google Scholar 

  13. Sarkar R (2011) Low distortion delaunay embedding of trees in hyperbolic plane. In: Graph drawing - 19th international symposium, GD 2011, vol 7034. Eindhoven, The Netherlands, September 21-23, 2011, Revised Selected Papers, pp 355–366

  14. Hawkes AG (1971) Spectra of some self-exciting and mutually exciting point processes. Biometrika 58(1):83–90

    Article  MathSciNet  Google Scholar 

  15. Ogata Y (1988) Statistical models for earthquake occurrences and residual analysis for point processes. J Am Stat Assoc 83(401):9–27

    Article  Google Scholar 

  16. Li L, Zha H (2013) Dyadic event attribution in social networks with mixtures of hawkes processes. In: 22nd ACM international conference on information and knowledge management, CIKM’13, San Francisco, CA, USA, October 27 - November 1, 2013, pp 1667–1672

  17. Kobayashi R, Lambiotte R (2016) Tideh: time-dependent hawkes process for predicting retweet dynamics. In: Proceedings of the 10th international conference on web and social media, Cologne, Germany, May 17-20, 2016, pp 191–200

  18. Li L, Zha H (2016) Household structure analysis via hawkes processes for enhancing energy disaggregation. In: Kambhampati S (ed), Proceedings of the twenty-fifth international joint conference on artificial intelligence, IJCAI 2016, New York, NY, USA, 9-15 July 2016, pp 2553–2559

  19. Salehi F, Trouleau W, Grossglauser M, Thiran P (2019) Learning hawkes processes from a handful of events. In: Advances in neural information processing systems 32: annual conference on neural information processing systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp 12694 –12704

  20. Du N, Dai H, Trivedi R, Upadhyay U, Gomez-Rodriguez M, Song L (2016) Recurrent marked temporal point processes: embedding event history to vector. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp 1555–1564

  21. Xiao S, Yan J, Yang X, Zha H, Chu SM (2017) Modeling the intensity function of point process via recurrent neural networks. In: Proceedings of the Thirty-First AAAI conference on artificial intelligence, February 4-9, 2017, San Francisco, California, USA, pp 1597–1603

  22. Mei H, Eisner J (2017) The neural hawkes process: a neurally self-modulating multivariate point process. In: Advances in neural information processing systems 30: annual conference on neural information processing systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp 6754–6764

  23. Zhang Q, Lipani A, Kirnap Ö, Yilmaz E (2020) Self-attentive hawkes process. In: Proceedings of the 37th international conference on machine learning, ICML 2020, 13-18 July 2020, Virtual Event, vol 119, pp 11183–11193

  24. Zuo S, Jiang H, Li Z, Zhao T, Zha H (2020) Transformer hawkes process. In: Proceedings of the 37th international conference on machine learning, ICML 2020, 13-18 July 2020, Virtual Event, vol 119, pp 11692–11702

  25. Liu Y, Yan T, Chen H (2018) Exploiting graph regularized multi-dimensional hawkes processes for modeling events with spatio-temporal characteristics. In: Lang J (ed) Proceedings of the twenty-seventh international joint conference on artificial intelligence, IJCAI 2018, July 13-19, 2018, Stockholm, Sweden, pp 2475–2482

  26. Yu L, Xu X, Trajcevski G, Zhou F (2022) Transformer-enhanced hawkes process with decoupling training for information cascade prediction. Knowl Based Syst 255:109740

    Article  Google Scholar 

  27. Shang J, Sun M (2019) Geometric hawkes processes with graph convolutional recurrent neural networks. In: The Thirty-Third AAAI conference on artificial intelligence, AAAI 2019, The thirty-first innovative applications of artificial intelligence conference, IAAI 2019, The Ninth AAAI symposium on educational advances in artificial intelligence, EAAI 2019, Honolulu, Hawaii, USA, January 27 - February 1, 2019, pp 4878–4885

  28. Du H, Zhou Y, Ma Y, Wang S (2021) Astrologer: exploiting graph neural hawkes process for event propagation prediction with spatio-temporal characteristics. Knowl Based Syst 228:107247

    Article  Google Scholar 

  29. Du H, Zhou Y (2022) Nostradamus: A novel event propagation prediction approach with spatio-temporal characteristics in non-euclidean space. Neural Netw 145:386–394

    Article  Google Scholar 

  30. Zhu S, Ding R, Zhang M, Hentenryck PV, Xie Y (2022) Spatio-temporal point processes with attention for traffic congestion event modeling. IEEE Trans Intell Transp Syst 23(7):7298–7309

    Article  Google Scholar 

  31. Chiang W-H, Liu X, Mohler G (2022) Hawkes process modeling of covid-19 with mobility leading indicators and spatial covariates. Int J Forecast 38(2):505–520

    Article  Google Scholar 

  32. Jiang L, Wang P, Cheng K, Liu K, Yin M, Jin B, Fu Y (2021) Eduhawkes: a neural hawkes process approach for online study behavior modeling. In: Proceedings of the 2021 SIAM international conference on data mining, SDM 2021, Virtual Event, April 29 - May 1, 2021, pp 567–575

  33. Gülçehre Ç, Denil M, Malinowski M, Razavi A, Pascanu R, Hermann KM, Battaglia PW, Bapst V, Raposo D, Santoro A, Freitas N (2019) Hyperbolic attention networks. In: 7th international conference on learning representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019

  34. Sawhney R, Agarwal S, Thakkar M, Wadhwa A, Shah RR (2021) Hyperbolic online time stream modeling. In: Proceedings of the 44th International ACM SIGIR conference on research and development in information retrieval, pp 1682–1686

  35. Chami I, Ying Z, Ré C, Leskovec J (2019) Hyperbolic graph convolutional neural networks. In: Advances in neural information processing systems 32: annual conference on neural information processing systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp 4869–4880

  36. Dai J, Wu Y, Gao Z, Jia Y (2021) A hyperbolic-to-hyperbolic graph convolutional network. In: IEEE conference on computer vision and pattern recognition, CVPR 2021, Virtual, June 19-25, 2021, pp 154–163

  37. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. In: Advances in neural information processing systems 30: annual conference on neural information processing systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp 5998–6008

  38. Li Y, Du N, Bengio S (2018) Time-dependent representation for neural event sequence prediction. In: 6th international conference on learning representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Workshop Track Proceedings

  39. Zhou K, Zha H, Song L (2013) Learning triggering kernels for multi-dimensional hawkes processes. In: Proceedings of the 30th International Conference on Machine Learning, ICML 2013, vol 28. Atlanta, GA, USA, 16-21 June 2013, pp 1301–1309

  40. Grant S, Betts B (2013) Encouraging user behaviour with achievements: an empirical study. In: Proceedings of the 10th working conference on mining software repositories, MSR ’13, San Francisco, CA, USA, May 18-19, 2013, pp 65–68

  41. Monti F, Bronstein MM, Bresson X (2017) Geometric matrix completion with recurrent multi-graph neural networks. In: Advances in neural information processing systems 30: annual conference on neural information processing systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp 3697–3707

  42. Shchur O, Bilos M, Günnemann S (2020) Intensity-free learning of temporal point processes. In: 8th international conference on learning representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020

Download references

Author information

Authors and Affiliations

  1. College of Mathematics and Computer Science, Zhejiang Normal University, Jinhua, 321004, Zhejiang, China

    Yiman Xie & Jianbin Wu

Authors
  1. Yiman Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jianbin Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jianbin Wu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xie, Y., Wu, J. HGTHP: a novel hyperbolic geometric transformer hawkes process for event prediction. Appl Intell 54, 357–374 (2024). https://doi.org/10.1007/s10489-023-05169-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-05169-0

Keywords

Search

Navigation