Abstract
Sequence prediction is a well-defined problem with a proliferation of applications, such as recommendation systems, social media monitor, economic analysis, etc. Recently, RNN-based methodologies have shown their superiority in time-series data analysis and sequence modeling. The question of which event would happen next is not difficult to answer anymore, but the prediction of when it would happen is still a mountain to climb. In this paper, we propose a Multi-task model to predict both event and their continuous timestamps at the same time. Specifically, (1) we design a two-layer RNN encoder for event sequences and a CNN encoder for time sequences, both equipped with multi-head self-attention to align history features; (2) we form multiple generative adversarial models for predicting future time sequences to solve the problem of multi-modal time distribution; (3) Mixture learning losses are adopted to conduct a 3-step learning strategy for training our model, the cross-entropy loss for events, Huber loss and adversarial classification loss which induces the Wasserstein distance for times. Due to these characteristics, we name it MM-CPred. The experiments on 4 real-life datasets confirmed its improvements compared with the baselines.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein GAN (2017). arXiv:abs/1701.07875
Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation (2014). arXiv preprint: arXiv:1406.1078
Fedus, W., Goodfellow, I., Dai, A.M.: MaskGAN: better text generation via filling in the blank. In: International Conference on Learning Representations (2018)
Goodfellow, I.J., et al.: Generative adversarial nets. In: NIPS (2014)
Granroth-Wilding, M., Clark, S.: What happens next? Event prediction using a compositional neural network model. In: Thirtieth AAAI Conference on Artificial Intelligence (2016)
Guo, J., Lu, S., Cai, H., Zhang, W., Yu, Y., Wang, J.: Long text generation via adversarial training with leaked information (2017). arXiv preprint: arXiv:abs/1709.08624
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Huber, P.J.: Robust estimation of a location parameter. In: Kotz, S., Johnson, N.L. (eds.) Breakthroughs in Statistics. Springer Series in Statistics (Perspectives in Statistics)Springer Series in Statistics (Perspectives in Statistics), pp. 492–518. Springer, New York (1992). https://doi.org/10.1007/978-1-4612-4380-9_35
Jang, E., Gu, S., Poole, B.: Categorical reparameterization with gumbel-softmax. In: ICLR (2017)
LeCun, Y., Bengio, Y., et al.: Convolutional networks for images, speech, and time series. The Handbook of Brain Theory and Neural Networks 3361(10), 1995 (1995)
Li, Y., Du, N., Bengio, S.: Time-dependent representation for neural event sequence prediction (2017). arXiv preprint arXiv:1708.00065
Li, Z., Ding, X., Liu, T.: Constructing narrative event evolutionary graph for script event prediction. In: IJCAI (2018)
Lin, K., Li, D., He, X., Zhang, Z., Sun, M.T.: Adversarial ranking for language generation. In: NIPS (2017)
Lin, L., Wen, L., Wang, J.: MM-Pred: a deep predictive model for multi-attribute event sequence. In: Proceedings of the 2019 SIAM International Conference on Data Mining, pp. 118–126. SIAM (2019)
Linden, G., Smith, B., Bullet, J.L.: Recommendations item-to-item collaborative filtering (2001)
Mei, H., Eisner, J.M.: The neural Hawkes process: a neurally self-modulating multivariate point process. In: Advances in Neural Information Processing Systems, pp. 6754–6764 (2017)
Musto, C., Semeraro, G., Degemmis, M., Lops, P.: Word embedding techniques for content-based recommender systems: an empirical evaluation. In: RecSys Posters (2015)
Neumann, L., Zisserman, A., Vedaldi, A.: Future event prediction: if and when. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (2019)
Pichl, M., Zangerle, E., Specht, G.: Now playing on spotify: leveraging spotify information on twitter for artist recommendations. In: ICWE Workshops (2015)
Sarwar, B.M., Karypis, G., Konstan, J.A., Riedl, J.: Item-based collaborative filtering recommendation algorithms. In: WWW (2001)
Semeniuta, S., Severyn, A., Gelly, S.: On accurate evaluation of GANs for language generation (2018). ArXiv: arXiv:abs/1806.04936
Tavakol, M., Brefeld, U.: Factored MDPs for detecting topics of user sessions. In: RecSys (2014)
Vaswani, A., et al.: Attention is all you need. In: Advances in neural information processing systems, pp. 5998–6008 (2017)
Xu, J., Ren, X., Lin, J., Sun, X.: Diversity-promoting GAN: a cross-entropy based generative adversarial network for diversified text generation. In: EMNLP (2018)
Yi, X., Hong, L., Zhong, E., Liu, N.N., Rajan, S.: Beyond clicks: dwell time for personalization. In: Proceedings of the 8th ACM Conference on Recommender Systems, pp. 113–120. ACM (2014)
Yu, L., Zhang, W., Wang, J., Yu, Y.: SeqGAN: sequence generative adversarial nets with policy gradient (2016). ArXiv:abs/1609.05473
Zhou, T., Qian, H., Shen, Z., Zhang, C., Wang, C., Liu, S., Ou, W.: Jump: a jointly predictor for user click and dwell time. In: IJCAI (2018)
Zhu, Y., et al.: What to do next: modeling user behaviors by time-LSTM. In: IJCAI, pp. 3602–3608 (2017)
Acknowledgment
The work was supported by the National Key Research and Development Program of China (No. 2019YFB1704003), the National Nature Science Foundation of China (No. 71690231), Tsinghua BNRist.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Lin, L., Zong, Z., Wen, L., Qian, C., Li, S., Wang, J. (2021). MM-CPred: A Multi-task Predictive Model for Continuous-Time Event Sequences with Mixture Learning Losses. In: Jensen, C.S., et al. Database Systems for Advanced Applications. DASFAA 2021. Lecture Notes in Computer Science(), vol 12681. Springer, Cham. https://doi.org/10.1007/978-3-030-73194-6_34
Download citation
DOI: https://doi.org/10.1007/978-3-030-73194-6_34
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-73193-9
Online ISBN: 978-3-030-73194-6
eBook Packages: Computer ScienceComputer Science (R0)