skip to main content
10.1145/3442381.3449816acmconferencesArticle/Chapter ViewAbstractPublication PagesthewebconfConference Proceedingsconference-collections
research-article

AutoSTG: Neural Architecture Search for Predictions of Spatio-Temporal Graph✱

Published: 03 June 2021 Publication History

Abstract

Spatio-temporal graphs are important structures to describe urban sensory data, e.g., traffic speed and air quality. Predicting over spatio-temporal graphs enables many essential applications in intelligent cities, such as traffic management and environment analysis. Recently, many deep learning models have been proposed for spatio-temporal graph prediction and achieved significant results. However, designing neural networks requires rich domain knowledge and expert efforts. To this end, we study automated neural architecture search for spatio-temporal graphs with the application to urban traffic prediction, which meets two challenges: 1) how to define search space for capturing complex spatio-temporal correlations; and 2) how to learn network weight parameters related to the corresponding attributed graph of a spatio-temporal graph.
To tackle these challenges, we propose a novel framework, entitled AutoSTG, for automated spatio-temporal graph prediction. In our AutoSTG, spatial graph convolution and temporal convolution operations are adopted in our search space to capture complex spatio-temporal correlations. Besides, we employ the meta learning technique to learn the adjacency matrices of spatial graph convolution layers and kernels of temporal convolution layers from the meta knowledge of the attributed graph. And specifically, such meta knowledge is learned by a graph meta knowledge learner that iteratively aggregates knowledge on the attributed graph. Finally, extensive experiments were conducted on two real-world benchmark datasets to demonstrate that AutoSTG can find effective network architectures and achieve state-of-the-art results. To the best of our knowledge, we are the first to study neural architecture search for spatio-temporal graphs.

References

[1]
Andrew Brock, Theodore Lim, James M Ritchie, and Nick Weston. 2017. Smash: one-shot model architecture search through hypernetworks. arXiv preprint arXiv:1708.05344(2017).
[2]
Han Cai, Ligeng Zhu, and Song Han. 2018. Proxylessnas: Direct neural architecture search on target task and hardware. arXiv preprint arXiv:1812.00332(2018).
[3]
Weiqi Chen, Ling Chen, Yu Xie, Wei Cao, Yusong Gao, and Xiaojie Feng. 2019. Multi-range attentive bicomponent graph convolutional network for traffic forecasting. arXiv preprint arXiv:1911.12093(2019).
[4]
Rui Dai, Shenkun Xu, Qian Gu, Chenguang Ji, and Kaikui Liu. 2020. Hybrid Spatio-Temporal Graph Convolutional Network: Improving Traffic Prediction with Navigation Data. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 3074–3082.
[5]
Yang Gao, Hong Yang, Peng Zhang, Chuan Zhou, and Yue Hu. 2019. GraphNAS: Graph Neural Architecture Search with Reinforcement Learning. arXiv preprint arXiv:1904.09981(2019).
[6]
Shengnan Guo, Youfang Lin, Ning Feng, Chao Song, and Huaiyu Wan. 2019. Attention based spatial-temporal graph convolutional networks for traffic flow forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 922–929.
[7]
Suining He and Kang G Shin. 2020. Towards fine-grained flow forecasting: A graph attention approach for bike sharing systems. In Proceedings of The Web Conference 2020. 88–98.
[8]
Rongzhou Huang, Chuyin Huang, Yubao Liu, Genan Dai, and Weiyang Kong. [n.d.]. LSGCN: Long Short-Term Traffic Prediction with Graph Convolutional Networks. ([n. d.]).
[9]
Yoon Kim. 2014. Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882(2014).
[10]
Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907(2016).
[11]
Ting Li, Junbo Zhang, Kainan Bao, Yuxuan Liang, Yexin Li, and Yu Zheng. 2020. Autost: Efficient neural architecture search for spatio-temporal prediction. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.
[12]
Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. 2017. Diffusion convolutional recurrent neural network: Data-driven traffic forecasting. arXiv preprint arXiv:1707.01926(2017).
[13]
Yuxuan Liang, Songyu Ke, Junbo Zhang, Xiuwen Yi, and Yu Zheng. 2018. Geoman: Multi-level attention networks for geo-sensory time series prediction. In IJCAI. 3428–3434.
[14]
Chenxi Liu, Barret Zoph, Maxim Neumann, Jonathon Shlens, Wei Hua, Li-Jia Li, Li Fei-Fei, Alan Yuille, Jonathan Huang, and Kevin Murphy. 2018. Progressive neural architecture search. In Proceedings of the European Conference on Computer Vision (ECCV). 19–34.
[15]
Hanxiao Liu, Karen Simonyan, and Yiming Yang. 2018. Darts: Differentiable architecture search. arXiv preprint arXiv:1806.09055(2018).
[16]
Zheyi Pan, Yuxuan Liang, Weifeng Wang, Yong Yu, Yu Zheng, and Junbo Zhang. 2019. Urban traffic prediction from spatio-temporal data using deep meta learning. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1720–1730.
[17]
Zheyi Pan, Wentao Zhang, Yuxuan Liang, Weinan Zhang, Yong Yu, Junbo Zhang, and Yu Zheng. 2020. Spatio-Temporal Meta Learning for Urban Traffic Prediction. IEEE Transactions on Knowledge and Data Engineering (2020).
[18]
Hieu Pham, Melody Y Guan, Barret Zoph, Quoc V Le, and Jeff Dean. 2018. Efficient neural architecture search via parameter sharing. arXiv preprint arXiv:1802.03268(2018).
[19]
Xian Wu, Chao Huang, Chuxu Zhang, and Nitesh V Chawla. 2020. Hierarchically structured transformer networks for fine-grained spatial event forecasting. In Proceedings of The Web Conference 2020. 2320–2330.
[20]
Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, Xiaojun Chang, and Chengqi Zhang. 2020. Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks. arXiv preprint arXiv:2005.11650(2020).
[21]
Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, and Chengqi Zhang. 2019. Graph wavenet for deep spatial-temporal graph modeling. arXiv preprint arXiv:1906.00121(2019).
[22]
Sirui Xie, Hehui Zheng, Chunxiao Liu, and Liang Lin. 2018. SNAS: stochastic neural architecture search. arXiv preprint arXiv:1812.09926(2018).
[23]
Bing Yu, Haoteng Yin, and Zhanxing Zhu. 2017. Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. arXiv preprint arXiv:1709.04875(2017).
[24]
Chris Zhang, Mengye Ren, and Raquel Urtasun. 2018. Graph hypernetworks for neural architecture search. arXiv preprint arXiv:1810.05749(2018).
[25]
Junbo Zhang, Yu Zheng, and Dekang Qi. 2017. Deep spatio-temporal residual networks for citywide crowd flows prediction. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 31.
[26]
Junbo Zhang, Yu Zheng, Dekang Qi, Ruiyuan Li, Xiuwen Yi, and Tianrui Li. 2018. Predicting citywide crowd flows using deep spatio-temporal residual networks. Artificial Intelligence 259 (2018), 147–166.
[27]
Qi Zhang, Jianlong Chang, Gaofeng Meng, Shiming Xiang, and Chunhong Pan. 2020. Spatio-Temporal Graph Structure Learning for Traffic Forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 1177–1185.
[28]
Weijia Zhang, Hao Liu, Yanchi Liu, Jingbo Zhou, and Hui Xiong. 2020. Semi-Supervised Hierarchical Recurrent Graph Neural Network for City-Wide Parking Availability Prediction. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 1186–1193.
[29]
Chuanpan Zheng, Xiaoliang Fan, Cheng Wang, and Jianzhong Qi. 2020. Gman: A graph multi-attention network for traffic prediction. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 1234–1241.
[30]
Yu Zheng, Licia Capra, Ouri Wolfson, and Hai Yang. 2014. Urban computing: concepts, methodologies, and applications. ACM Transactions on Intelligent Systems and Technology (TIST) 5, 3(2014), 1–55.
[31]
Kaixiong Zhou, Qingquan Song, Xiao Huang, and Xia Hu. 2019. Auto-GNN: Neural Architecture Search of Graph Neural Networks. arXiv preprint arXiv:1909.03184(2019).
[32]
Barret Zoph and Quoc V Le. 2016. Neural architecture search with reinforcement learning. arXiv preprint arXiv:1611.01578(2016).

Cited By

View all
  • (2025)A systematic review for transformer-based long-term series forecastingArtificial Intelligence Review10.1007/s10462-024-11044-258:3Online publication date: 6-Jan-2025
  • (2024)Multi-Scale Non-Local Spatio-Temporal Information Fusion Networks for Multi-Step Traffic Flow ForecastingISPRS International Journal of Geo-Information10.3390/ijgi1303007113:3(71)Online publication date: 27-Feb-2024
  • (2024)A Survey on Knowledge Graph Related Research in Smart City DomainACM Transactions on Knowledge Discovery from Data10.1145/367261518:9(1-31)Online publication date: 19-Jul-2024
  • Show More Cited By
  1. AutoSTG: Neural Architecture Search for Predictions of Spatio-Temporal Graph✱

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    WWW '21: Proceedings of the Web Conference 2021
    April 2021
    4054 pages
    ISBN:9781450383127
    DOI:10.1145/3442381
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 03 June 2021

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Neural architecture search
    2. meta learning
    3. spatio-temporal graph

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Conference

    WWW '21
    Sponsor:
    WWW '21: The Web Conference 2021
    April 19 - 23, 2021
    Ljubljana, Slovenia

    Acceptance Rates

    Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)153
    • Downloads (Last 6 weeks)15
    Reflects downloads up to 06 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)A systematic review for transformer-based long-term series forecastingArtificial Intelligence Review10.1007/s10462-024-11044-258:3Online publication date: 6-Jan-2025
    • (2024)Multi-Scale Non-Local Spatio-Temporal Information Fusion Networks for Multi-Step Traffic Flow ForecastingISPRS International Journal of Geo-Information10.3390/ijgi1303007113:3(71)Online publication date: 27-Feb-2024
    • (2024)A Survey on Knowledge Graph Related Research in Smart City DomainACM Transactions on Knowledge Discovery from Data10.1145/367261518:9(1-31)Online publication date: 19-Jul-2024
    • (2024)A Comprehensive Survey on Automated Machine Learning for RecommendationsACM Transactions on Recommender Systems10.1145/36301042:2(1-38)Online publication date: 10-Apr-2024
    • (2024)MetaSTNet: Multimodal Meta-Learning for Cellular Traffic Conformal PredictionIEEE Transactions on Network Science and Engineering10.1109/TNSE.2023.333562611:2(1999-2011)Online publication date: Mar-2024
    • (2024)LightCTS*: Lightweight Correlated Time Series Forecasting Enhanced With Model DistillationIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.342445136:12(8695-8710)Online publication date: Dec-2024
    • (2024)Disentangling Structured Components: Towards Adaptive, Interpretable and Scalable Time Series ForecastingIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.337193136:8(3783-3800)Online publication date: Aug-2024
    • (2024)Spatio-Temporal Graph Neural Networks for Predictive Learning in Urban Computing: A SurveyIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.333382436:10(5388-5408)Online publication date: Oct-2024
    • (2024)Enhancing Deep Neural Network Architecture in Spatio- Temporal Forecasting Through Neural Architecture Search2024 International Conference on Recent Advances in Electrical, Electronics, Ubiquitous Communication, and Computational Intelligence (RAEEUCCI)10.1109/RAEEUCCI61380.2024.10547787(1-5)Online publication date: 17-Apr-2024
    • (2024)Forecasting Building Operation Dynamics Using a Physics-Informed Spatio-Temporal Graph Neural Network (PISTGNN) EnsembleEnergy and Buildings10.1016/j.enbuild.2024.115085(115085)Online publication date: Nov-2024
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media