skip to main content
10.1145/3534678.3539422acmconferencesArticle/Chapter ViewAbstractPublication PageskddConference Proceedingsconference-collections
research-article
Public Access

Mining Spatio-Temporal Relations via Self-Paced Graph Contrastive Learning

Published: 14 August 2022 Publication History

Abstract

Modeling complex spatial and temporal dependencies are indispensable for location-bound time series learning. Existing methods, typically relying on graph neural networks (GNNs) and temporal learning modules based on recurrent neural networks, have achieved significant performance improvements. However, their representation capabilities and prediction results are limited when pre-defined graphs are unavailable. Unlike spatio-temporal GNNs focusing on designing complex architectures, we propose a novel adaptive graph construction strategy: Self-Paced Graph Contrastive Learning (SPGCL). It learns informative relations by maximizing the distinguishing margin between positive and negative neighbors and generates an optimal graph with a self-paced strategy. Specifically, the existing neighborhoods iteratively absorb more reliable nodes with the highest affinity scores as new neighbors to generate the next-round neighborhoods, and augmentations are applied to improve the transferability and robustness. As the adaptively self-paced graph approaches the optimized graph for prediction, the mutual information between nodes and the corresponding neighbors is maximized. Our work provides a new perspective of addressing spatio-temporal learning problems beyond information aggregation in Euclidean space and can be generalized to different tasks. Extensive experiments conducted on two typical spatio-temporal learning tasks (traffic forecasting and land displacement prediction) demonstrate the superior performance of SPGCL against the state-of-the-art.

Supplemental Material

MP4 File
Presentation video

References

[1]
Philip Bachman, R Devon Hjelm, and William Buchwalter. 2019. Learning Representations by Maximizing Mutual Information across Views. In NeurIPS. 15535--15545.
[2]
Lei Bai, Lina Yao, Can Li, Xianzhi Wang, and Can Wang. 2020. Adaptive graph convolutional recurrent network for traffic forecasting. In NeurIPS. 17804--17815.
[3]
Donald J. Berndt and James Clifford. 1994. Using Dynamic Time Warping to Find Patterns in Time Series. In SIGKDD. 359--370.
[4]
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020 b. A simple framework for contrastive learning of visual representations. In ICML. 1597--1607.
[5]
Xuxi Chen, Wuyang Chen, Tianlong Chen, Ye Yuan, Chen Gong, Kewei Chen, and Zhangyang Wang. 2020 a. Self-pu: Self boosted and calibrated positive-unlabeled training. In ICML. 1510--1519.
[6]
Yu Chen, Lingfei Wu, and Mohammed Zaki. 2020 c. Iterative Deep Graph Learning for Graph Neural Networks: Better and Robust Node Embeddings. In NeurIPS. 19314--19326.
[7]
Ching-Yao Chuang, Joshua Robinson, Yen-Chen Lin, Antonio Torralba, and Stefanie Jegelka. 2020. Debiased Contrastive Learning. In NeurIPS. 8765--8775.
[8]
Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. 2014. Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling. arxiv: 1412.3555
[9]
Pádraig Cunningham and Sarah Jane Delany. 2021. k-Nearest Neighbour Classifiers - A Tutorial. Comput. Surveys, Vol. 54, 6 (Jul 2021), 1--25.
[10]
Rumen Dangovski, Li Jing, Charlotte Loh, Seungwook Han, Akash Srivastava, Brian Cheung, Pulkit Agrawal, and Marin Soljacic. 2022. Equivariant Self-Supervised Learning: Encouraging Equivariance in Representations. In ICLR.
[11]
Marthinus C Du Plessis, Gang Niu, and Masashi Sugiyama. 2014. Analysis of learning from positive and unlabeled data. In NeurIPS. 703--711.
[12]
Charles Elkan and Keith Noto. 2008. Learning classifiers from only positive and unlabeled data. In SIGKDD. 213--220.
[13]
Zheng Fang, Qingqing Long, Guojie Song, and Kunqing Xie. 2021. Spatial-temporal graph ode networks for traffic flow forecasting. In SIGKDD. 364--373.
[14]
Luca Franceschi, Mathias Niepert, Massimiliano Pontil, and Xiao He. 2019. Learning discrete structures for graph neural networks. In ICML. 1972--1982.
[15]
Xu Geng, Yaguang Li, Leye Wang, Lingyu Zhang, Qiang Yang, Jieping Ye, and Yan Liu. 2019. Spatiotemporal Multi-Graph Convolution Network for Ride-Hailing Demand Forecasting. In AAAI. 3656--3663.
[16]
Shengnan Guo, Youfang Lin, Ning Feng, Chao Song, and Huaiyu Wan. 2019. Attention based spatial-temporal graph convolutional networks for traffic flow forecasting. In AAAI. 922--929.
[17]
Xiaojie Guo, Yuanqi Du, and Liang Zhao. 2021. Deep Generative Models for Spatial Networks. In SIGKDD. 505--515.
[18]
Michael Gutmann and Aapo Hyvärinen. 2010. Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In AISTATS. 297--304.
[19]
Ainaz Hajimoradlou, Gioachino Roberti, and David Poole. 2020. Predicting Landslides Using Locally Aligned Convolutional Neural Networks. In IJCAI. 3342--3348.
[20]
Jindong Han, Hao Liu, Hengshu Zhu, Hui Xiong, and Dejing Dou. 2021. Joint Air Quality and Weather Prediction Based on Multi-Adversarial Spatiotemporal Networks. In AAAI. 4081--4089.
[21]
Arman Hasanzadeh, Ehsan Hajiramezanali, Krishna Narayanan, Nick Duffield, Mingyuan Zhou, and Xiaoning Qian. 2019. Semi-Implicit Graph Variational Auto-Encoders. In NeurIPS. 10711--10722.
[22]
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. 2020. Momentum contrast for unsupervised visual representation learning. In CVPR. 9729--9738.
[23]
Thomas N. Kipf and Max Welling. 2016. Variational Graph Auto-Encoders. arxiv: 1611.07308
[24]
Ryuichi Kiryo, Gang Niu, Marthinus C du Plessis, and Masashi Sugiyama. 2017. Positive-unlabeled learning with non-negative risk estimator. In NeurIPS. 1674--1684.
[25]
Mengzhang Li and Zhanxing Zhu. 2021. Spatial-Temporal Fusion Graph Neural Networks for Traffic Flow Forecasting. In AAAI. 4189--4196.
[26]
Xiaoli Li and Bing Liu. 2003. Learning to classify texts using positive and unlabeled data. In IJCAI. 587--592.
[27]
Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. 2018. Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting. In ICLR.
[28]
Ishan Misra and Laurens van der Maaten. 2020. Self-supervised learning of pretext-invariant representations. In CVPR. 6707--6717.
[29]
Cheonbok Park, Chunggi Lee, Hyojin Bahng, Yunwon Tae, Seungmin Jin, Kihwan Kim, Sungahn Ko, and Jaegul Choo. 2020. ST-GRAT: A novel spatio-temporal graph attention networks for accurately forecasting dynamically changing road speed. In CIKM. 1215--1224.
[30]
Zhen Peng, Wenbing Huang, Minnan Luo, Qinghua Zheng, Yu Rong, Tingyang Xu, and Junzhou Huang. 2020. Graph representation learning via graphical mutual information maximization. In WWW. 259--270.
[31]
Ben Poole, Sherjil Ozair, Aaron Van Den Oord, Alex Alemi, and George Tucker. 2019. On variational bounds of mutual information. In ICML. 5171--5180.
[32]
Jiezhong Qiu, Qibin Chen, Yuxiao Dong, Jing Zhang, Hongxia Yang, Ming Ding, Kuansan Wang, and Jie Tang. 2020. Gcc: Graph contrastive coding for graph neural network pre-training. In SIGKDD. 1150--1160.
[33]
Joshua David Robinson, Ching-Yao Chuang, Suvrit Sra, and Stefanie Jegelka. 2020. Contrastive Learning with Hard Negative Samples. In ICLR.
[34]
Victor Garcia Satorras, Emiel Hoogeboom, and Max Welling. 2021. E (n) equivariant graph neural networks. In ICML. 9323--9332.
[35]
Chao Shang, Jie Chen, and Jinbo Bi. 2020. Discrete Graph Structure Learning for Forecasting Multiple Time Series. In ICLR.
[36]
Chao Song, Youfang Lin, Shengnan Guo, and Huaiyu Wan. 2020. Spatial-temporal synchronous graph convolutional networks: A new framework for spatial-temporal network data forecasting. In AAAI. 914--921.
[37]
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2019. Representation Learning with Contrastive Predictive Coding.
[38]
Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2018. Graph Attention Networks. In ICLR.
[39]
Xiaoyang Wang, Yao Ma, Yiqi Wang, Wei Jin, Xin Wang, Jiliang Tang, Caiyan Jia, and Jian Yu. 2020. Traffic Flow Prediction via Spatial Temporal Graph Neural Network. In WWW. 1082--1092.
[40]
Dongkuan Xu, Wei Cheng, Dongsheng Luo, Haifeng Chen, and Xiang Zhang. 2021. InfoGCL: Information-Aware Graph Contrastive Learning. In NeurIPS.
[41]
Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. 2020. Graph Contrastive Learning with Augmentations. In NeurIPS. 5812--5823.
[42]
Bing Yu, Haoteng Yin, and Zhanxing Zhu. 2018. Spatio-temporal graph convolutional networks: a deep learning framework for traffic forecasting. In IJCAI. 3634--3640.
[43]
Qi Zhang, Jianlong Chang, Gaofeng Meng, Shiming Xiang, and Chunhong Pan. 2020. Spatio-temporal graph structure learning for traffic forecasting. In AAAI. 1177--1185.
[44]
Cheng Zheng, Bo Zong, Wei Cheng, Dongjin Song, Jingchao Ni, Wenchao Yu, Haifeng Chen, and Wei Wang. 2020. Robust Graph Representation Learning via Neural Sparsification. In ICML. 11458--11468.
[45]
Fan Zhou, Rongfan Li, Kunpeng Zhang, and Goce Trajcevski. 2021. Land Deformation Prediction via Slope-Aware Graph Neural Networks. In AAAI. 15033-15040.
[46]
Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. 2021. Graph contrastive learning with adaptive augmentation. In WWW. 2069--2080.

Cited By

View all
  • (2025)Spatio-Temporal Contrastive Learning-Based Adaptive Graph Augmentation for Traffic Flow PredictionIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.348798226:1(1304-1318)Online publication date: Jan-2025
  • (2025)Data-Centric Graph Learning: A SurveyIEEE Transactions on Big Data10.1109/TBDATA.2024.348941211:1(1-20)Online publication date: Feb-2025
  • (2025)TSHDNet: temporal-spatial heterogeneity decoupling network for multi-mode traffic flow predictionApplied Intelligence10.1007/s10489-024-06218-y55:5Online publication date: 14-Jan-2025
  • Show More Cited By

Index Terms

  1. Mining Spatio-Temporal Relations via Self-Paced Graph Contrastive Learning

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    KDD '22: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining
    August 2022
    5033 pages
    ISBN:9781450393850
    DOI:10.1145/3534678
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 14 August 2022

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. contrastive learning
    2. graph neural networks
    3. self-paced learning
    4. spatio-temporal learning

    Qualifiers

    • Research-article

    Funding Sources

    Conference

    KDD '22
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

    Upcoming Conference

    KDD '25

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)1,369
    • Downloads (Last 6 weeks)108
    Reflects downloads up to 17 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)Spatio-Temporal Contrastive Learning-Based Adaptive Graph Augmentation for Traffic Flow PredictionIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.348798226:1(1304-1318)Online publication date: Jan-2025
    • (2025)Data-Centric Graph Learning: A SurveyIEEE Transactions on Big Data10.1109/TBDATA.2024.348941211:1(1-20)Online publication date: Feb-2025
    • (2025)TSHDNet: temporal-spatial heterogeneity decoupling network for multi-mode traffic flow predictionApplied Intelligence10.1007/s10489-024-06218-y55:5Online publication date: 14-Jan-2025
    • (2024)RCCNet: A Spatial-Temporal Neural Network Model for Logistics Delivery Timely Rate PredictionACM Transactions on Intelligent Systems and Technology10.1145/369064915:6(1-21)Online publication date: 29-Aug-2024
    • (2024)Enhancing Dependency Dynamics in Traffic Flow Forecasting via Graph Risk BootstrapProceedings of the 32nd ACM International Conference on Advances in Geographic Information Systems10.1145/3678717.3691237(147-159)Online publication date: 29-Oct-2024
    • (2024)SPContrastNet: A Self-Paced Contrastive Learning Model for Few-Shot Text ClassificationACM Transactions on Information Systems10.1145/365260042:5(1-25)Online publication date: 29-Apr-2024
    • (2024)Foundation Models for Time Series Analysis: A Tutorial and SurveyProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671451(6555-6565)Online publication date: 25-Aug-2024
    • (2024)Spatio-Temporal Graph Neural Networks for Predictive Learning in Urban Computing: A SurveyIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.333382436:10(5388-5408)Online publication date: Oct-2024
    • (2024)Contrastive-Learning-Based Adaptive Graph Fusion Convolution Network With Residual-Enhanced Decomposition Strategy for Traffic Flow ForecastingIEEE Internet of Things Journal10.1109/JIOT.2024.337075811:11(20246-20259)Online publication date: 1-Jun-2024
    • (2024)STS-CCL: Spatial-Temporal Synchronous Contextual Contrastive Learning for Urban Traffic ForecastingICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP48485.2024.10446624(6705-6709)Online publication date: 14-Apr-2024
    • Show More Cited By

    View Options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Login options

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media