Chunnan Wang
Abstract
In recent years, many spatial-temporal graph convolutional network (STGCN) models are proposed to deal with the spatial-temporal network data forecasting problem. These STGCN models have their own advantages, i.e., each of them puts forward many effective operations and achieves good prediction results in the real applications. If users can effectively utilize and combine these excellent operations integrating the advantages of existing models, then they may obtain more effective STGCN models thus create greater value using existing work. However, they fail to do so due to the lack of domain knowledge, and there is lack of automated system to help users to achieve this goal. In this article, we fill this gap and propose Auto-STGCN algorithm, which makes use of existing models to automatically explore high-performance STGCN model for specific scenarios. Specifically, we design Unified-STGCN framework, which summarizes the operations of existing architectures, and use parameters to control the usage and characteristic attributes of each operation, so as to realize the parameterized representation of the STGCN architecture and the reorganization and fusion of advantages. Then, we present Auto-STGCN, an optimization method based on reinforcement learning, to quickly search the parameter search space provided by Unified-STGCN, and generate optimal STGCN models automatically. Extensive experiments on real-world benchmark datasets show that our Auto-STGCN can find STGCN models superior to existing STGCN models used for search space construction, which demonstrates the effectiveness of our proposed method.
- [1] . 2018. Contextual graph Markov model: A deep and generative approach to graph processing. In Proceedings of the 35th International Conference on Machine Learning. Vol. 80, 304–313.Google Scholar
- [2] . 2019. Spatio-temporal graph convolutional and recurrent networks for citywide passenger demand prediction. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management. ACM, 2293–2296.Google ScholarDigital Library
- [3] . 2019. Passenger demand forecasting with multi-task convolutional recurrent neural networks. In Proceedings of the Pacific-Asia Conference on Knowledge Discovery and Data Mining. Vol. 11440, 29–42.Google ScholarDigital Library
- [4] . 1957. On the role of dynamic programming in statistical communication theory. IRE Transactions on Information Theory 3, 3 (1957), 197–203.Google ScholarCross Ref
- [5] . 2017. Neural optimizer search with reinforcement learning. In Proceedings of the 34th International Conference on Machine Learning. Vol. 70, PMLR, 459–468.Google Scholar
- [6] . 2001. Freeway performance measurement system: Mining loop detector data. Transportation Research Record 1748, 1 (2001), 96–102.Google ScholarCross Ref
- [7] . 2015. MXNet: A flexible and efficient machine learning library for heterogeneous distributed systems. arXiv:1512.01274. Retrieved from https://arxiv.org/abs/1512.01274.Google Scholar
- [8] . 2019. RENAS: Reinforced evolutionary neural architecture search. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4787–4796.Google ScholarCross Ref
- [9] . 2019. Cluster-GCN: An efficient algorithm for training deep and large graph convolutional networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 257–266.Google ScholarDigital Library
- [10] . 2021. The pulse of urban transport: Exploring the co-evolving pattern for spatio-temporal forecasting. ACM Transactions on Knowledge Discovery from Data 15, 6 (2021), 103:1–103:25.Google ScholarDigital Library
- [11] . 1996. Support vector regression machines. In Proceedings of the 9th International Conference on Neural Information Processing Systems. 155–161.Google ScholarDigital Library
- [12] . 2020. Graph neural architecture search. In Proceedings of the International Joint Conference on Artificial Intelligence. (Ed.), ijcai.org, 1403–1409.Google ScholarCross Ref
- [13] . 2019. Attention based spatial-temporal graph convolutional networks for traffic flow forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence. 922–929.Google ScholarDigital Library
- [14] . 2019. Deep spatial-temporal 3D convolutional neural networks for traffic data forecasting. IEEE Transactions on Intelligent Transportation Systems. 20, 10 (2019), 3913–3926.Google ScholarCross Ref
- [15] . 2017. Inductive representation learning on large graphs. In Proceedings of the International Conference on Neural Information Processing Systems. 1024–1034.Google Scholar
- [16] . 2021. DSRNA: Differentiable search of robust neural architectures. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Computer Vision Foundation/ IEEE, 6196–6205.Google ScholarCross Ref
- [17] . 1994. On the convergence of stochastic iterative dynamic programming algorithms. Neural Computation 6, 6 (1994), 1185–1201.Google ScholarDigital Library
- [18] . 2016. Structural-RNN: Deep learning on spatio-temporal graphs. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. 5308–5317.Google ScholarCross Ref
- [19] . 2020. STGAT: Spatial-temporal graph attention networks for traffic flow forecasting. IEEE Access 8 (2020), 134363–134372.Google ScholarCross Ref
- [20] . 1995. Learning from delayed rewards. Robotics and Autonomous Systems. 15, 4 (1995), 233–235.Google ScholarCross Ref
- [21] . 2018. Diffusion convolutional recurrent neural network: Data-driven traffic forecasting. In Proceedings of the International Conference on Learning Representations.Google Scholar
- [22] . 2019. DARTS: Differentiable architecture search. In Proceedings of the International Conference on Learning Representations.Google Scholar
- [23] . 2019. GeniePath: Graph neural networks with adaptive receptive paths. In Proceedings of the AAAI Conference on Artificial Intelligence. 4424–4431.Google ScholarDigital Library
- [24] . 2015. Human-level control through deep reinforcement learning. Nature 518, 7540 (2015), 529–533.Google ScholarCross Ref
- [25] . 1999. Policy invariance under reward transformations: Theory and application to reward shaping. In Proceedings of the 16th International Conference on Machine Learning. 278–287.Google ScholarDigital Library
- [26] . 2020. ASAP: Architecture search, anneal and prune. In
Proceedings of the International Conference on Artificial Intelligence and Statistics . Vol. 108, PMLR, 493–503.Google Scholar - [27] . 1994. Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley.Google ScholarCross Ref
- [28] . 2019. Regularized evolution for image classifier architecture search. In Proceedings of the AAAI Conference on Artificial Intelligence. AAAI Press, 4780–4789.Google ScholarDigital Library
- [29] . 2021. A comprehensive survey of neural architecture search: Challenges and solutions. ACM Computing Surveys 54, 4 (2021), 76:1–76:34.Google Scholar
- [30] . 2015. Convolutional LSTM network: A machine learning approach for precipitation nowcasting. In Proceedings of the International Conference on Neural Information Processing Systems. 802–810.Google Scholar
- [31] . 2020. Spatial-temporal synchronous graph convolutional networks: A new framework for spatial-temporal network data forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence. 914–921.Google ScholarCross Ref
- [32] . 2019. MnasNet: Platform-aware neural architecture search for mobile. In Proceedings of the Conference on Computer Vision and Pattern Recognition. 2820–2828.Google ScholarCross Ref
- [33] . 2018. Graph attention networks. In Proceedings of the International Conference on Learning Representations.Google Scholar
- [34] . 2003. Modeling and forecasting vehicular traffic flow as a seasonal ARIMA process: Theoretical basis and empirical results. Journal of Transportation Engineering 129, 6 (2003), 664–672.Google ScholarCross Ref
- [35] . 2019. How powerful are graph neural networks? In Proceedings of the International Conference on Learning Representations.Google Scholar
- [36] . 2020. Spatial-temporal transformer networks for traffic flow forecasting. arXiv:2001.02908. Retrieved from https://arxiv.org/abs/2001.02908.Google Scholar
- [37] . 2021. ZeroNAS: Differentiable generative adversarial networks search for zero-shot learning. IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 12 (
11 2021), 9733–9740.DOI: Google ScholarCross Ref - [38] . 2018. Spatial temporal graph convolutional networks for skeleton-based action recognition. In Proceedings of the AAAI Conference on Artificial Intelligence. 7444–7452.Google ScholarCross Ref
- [39] . 2020. CARS: Continuous evolution for efficient neural architecture search. In Proceedings of the Conference on Computer Vision and Pattern Recognition. IEEE.Google ScholarCross Ref
- [40] . 2018. Deep multi-view spatial-temporal network for taxi demand prediction. In Proceedings of the AAAI Conference on Artificial Intelligence. 2588–2595.Google ScholarCross Ref
- [41] . 2018. Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. In Proceedings of the International Joint Conference on Artificial Intelligence. 3634–3640.Google ScholarCross Ref
- [42] . 2021. Jointly modeling spatio-temporal dependencies and daily flow correlations for crowd flow prediction. ACM Transactions on Knowledge Discovery from Data 15, 4 (2021), 58:1–58:20.Google ScholarDigital Library
- [43] . 2020. Making sense of spatio-temporal preserving representations for EEG-based human intention recognition. IEEE Transactions on Cybernetics 50, 7 (2020), 3033–3044.
DOI: Google ScholarCross Ref - [44] . 2020. Efficient probabilistic logic reasoning with graph neural networks. In Proceedings of the International Conference on Learning Representations.Google Scholar
- [45] . 2018. Practical block-wise neural network architecture generation. In Proceedings of the Conference on Computer Vision and Pattern Recognition. 2423–2432.Google ScholarCross Ref
- [46] . 2018. Learning transferable architectures for scalable image recognition. In Proceedings of the Conference on Computer Vision and Pattern Recognition. 8697–8710.Google ScholarCross Ref
Index Terms
- Auto-STGCN: Autonomous Spatial-Temporal Graph Convolutional Network Search
Recommendations
Traffic Spatial-Temporal Prediction Based on Neural Architecture Search
SSTD '23: Proceedings of the 18th International Symposium on Spatial and Temporal DataTraffic spatial-temporal prediction is essential for intelligent transportation systems. However, the current approach relies heavily on expert knowledge and time-consuming manual modeling. Neural architecture search can build models adaptively, but it ...
Automated Search for Configurations of Convolutional Neural Network Architectures
SPLC '19: Proceedings of the 23rd International Systems and Software Product Line Conference - Volume AConvolutional Neural Networks (CNNs) are intensively used to solve a wide variety of complex problems. Although powerful, such systems require manual configuration and tuning. To this end, we view CNNs as configurable systems and propose an end-to-end ...
Auto-Keras: An Efficient Neural Architecture Search System
KDD '19: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data MiningNeural architecture search (NAS) has been proposed to automatically tune deep neural networks, but existing search algorithms, e.g., NASNet, PNAS, usually suffer from expensive computational cost. Network morphism, which keeps the functionality of a ...
Comments