skip to main content
introduction
Free access

Introduction to the Special Issue on Deep Learning for Spatio-Temporal Data: Part 2

Published: 26 March 2022 Publication History

1 Introduction

With the quick development of different position techniques, such as Global Position Systems (GPSs), mobile devices, and remote sensing, various spatio-temporal data have been generated nowadays. It is of great importance for many practical applications, including human mobility mining, urban planning, health care, and public safety, to mine valuable knowledge from spatio-temporal data. Recently, deep learning has achieved considerable success in many domains, such as computer vision, natural language processing, and medical analysis. It also gets remarkable performance gains in various spatio-temporal data mining (STDM) tasks including crowd flow prediction, origin-destination (OD) prediction, and health care. The aim of this special issue is to show the latest research findings and engineering experiences in developing and applying deep learning techniques for spatio-temporal data mining tasks and applications from the perspective of academia and industry.
The objective of this special issue is to highlight leading work in spatio-temporal data mining with deep learning techniques, to identify challenges, and to explore future topics in this area. The call for papers for this special issue was perceived very positively with 78 regular submissions, and 22 papers were finally accepted through a rigorous review process. The acceptance rate was therefore 28%. The accepted articles are divided into two parts for publication, and each part includes 11 articles. Part 1 covers the topics of spatio-temporal data modeling, spatio-temporal data representation learning, and various applications including traffic prediction, human trajectory prediction, and spatio-temporal anomaly detection. In part 2, the 11 papers discuss the topics of human mobility responses to COVID-19, earth imagery segmentation based on spatial-temporal data learning, urban dynamics analysis, and various applications including package pick-up route prediction and traffic flow prediciton. A brief introduction to the articles of part 2 is presented as follows.
In the article titled “Generative Adversarial Networks for Spatio-Temporal Data: A Survey,” Gao et al. conduct a comprehensive review on the recent developments of GANs for spatio-temporal data. They summarize the application of popular GAN architectures for spatio-temporal data and the common practices for evaluating the performance of spatio-temporal applications with GANs. At the end, they point out future research directions to benefit researchers in the area of spatio-temporal data analysis.
In the article titled “COVID-GAN+: Estimating Human Mobility Responses to COVID-19 through Spatio-Temporal Generative Adversarial Networks with Enhanced Features,” in response to the grand challenges posed by the COVID-19 pandemic to policymakers, Bao et al. formulate the human mobility estimation as a spatio-temporal data generation problem and propose COVID-GAN, a spatio-temporal Conditional Generative Adversarial Network, to estimate mobility (e.g., changes in POI visits) under various real-world conditions (e.g., COVID-19 severity, local policy interventions) integrated from multiple data sources. They also introduce a domain-constraint correction layer in the generator of COVID-GAN to reduce the difficulty of learning. Experiments with urban mobility data derived from cell phone records and census data show that COVID-GAN can well approximate real-world human mobility responses, and the proposed domain-constraint-based correction can greatly improve solution quality.
In the article titled “Weakly Supervised Spatial Deep Learning for Earth Image Segmentation Based on Imperfect Polyline Labels,,” Jiang et al. argue that existing research that overcomes imperfect training labels either focuses on errors in label class semantics or characterizes label location errors at the pixel level. These methods do not fully incorporate the geometric properties of label location errors in the vector representation. To fill the gap, they propose a weakly supervised learning framework to simultaneously update deep learning model parameters and infer hidden true vector label locations, and they model label location errors in the vector representation to partially reserve geometric properties (e.g., spatial contiguity within line segments). Evaluations on real-world datasets in the National Hydrography Dataset (NHD) refinement application illustrate that the proposed framework outperforms baseline methods in classification accuracy.
In the article titled “Earth Imagery Segmentation on Terrain Surface with Limited Training Labels: A Semi-supervised Approach Based on Physics-guided Graph Co-training,” He et al. study surface segmentation based on both explanatory features and surface topology. Since existing methods on semi-supervised and unsupervised learning for earth imagery often focus on learning representation without explicitly incorporating surface topology, they propose a novel framework that explicitly models the topological skeleton of a terrain surface with a contour tree from computational topology, which is guided by the physical constraint. Their framework consists of two neural networks: a convolutional neural network (CNN) to learn spatial contextual features on a 2D image grid and a graph neural network (GNN) to learn the statistical distribution of physics-guided spatial topological dependency on the contour tree. The two models are co-trained via variational EM. Evaluations on the real-world flood mapping datasets show that the proposed models outperform baseline methods in classification accuracy, especially when training labels are limited.
In the article titled “Make More Connections: Urban Traffic Flow Forecasting with Spatiotemporal Adaptive Gated Graph Convolution Network,” Lu et al. consider constructing the road network as a dynamic weighted graph through the attention mechanism to describe and capture the dynamic spatio-temporal correlation, and they aim to seek both spatial neighbors and semantic neighbors to make more connections between road nodes. They propose a novel Spatio-temporal Adaptive Gated Graph Convolution Network (STAG-GCN) to predict traffic conditions for several time steps ahead. STAG-GCN mainly consists of two major components: (1) multivariate self-attention Temporal Convolution Network (TCN) is utilized to capture local and long-range temporal dependencies across recent, daily periodic and weekly periodic observations, and (2) mix-hop AG-GCN extracts selective spatial and semantic dependencies within multi-layer stacking through an adaptive graph gating mechanism and mix-hop propagation mechanism. The outputs of different components are weighted fused to generate the final prediction results. Extensive experiments on two real-world large-scale urban traffic datasets have verified the effectiveness of their proposed approach.
In the article titled “Graph Sequence Neural Network with an Attention Mechanism for Traffic Speed Prediction,” Lu et al. propose a graph sequence neural network with an attention mechanism (GSeqAtt) for processing graph sequences, and they combine two attention mechanisms: a horizontal mechanism and a vertical mechanism. GTransformer, which is a horizontal attention mechanism for handling time series, is used to capture the correlations between graphs in the input time sequence. The vertical attention mechanism, a graph network (GN) block structure with an attention mechanism (GNAtt), acts within the graph structure in each frame of the time series. Experiments show that the proposed model is able to handle information propagation for graph sequences accurately and efficiently.
In the article titled “Urban Traffic Dynamics Prediction - A Continuous Spatial-Temporal Meta-Learning Approach,” Zhang et al. solve the traffic dynamics prediction problem from the Bayesian meta-learning perspective. They propose a novel continuous spatial-temporal meta-learner (cST-ML), which is trained on a distribution of traffic prediction tasks segmented by historical traffic data with the goal of learning a strategy that can be quickly adapted to related but unseen traffic prediction tasks. cST-ML tackles the traffic dynamics prediction challenges by advancing the Bayesian black-box meta-learning framework through the following new points: (1) cST-ML captures the dynamics of traffic prediction tasks using variational inference, and to better capture the temporal uncertainties within tasks, cST-ML performs as a rolling window within each task; (2) cST-ML has novel designs in architecture, where CNN and LSTM are embedded to capture the spatial-temporal dependencies between traffic status and traffic related features; and (3) novel training and testing algorithms for cST-ML are designed. Extensive experimental results verify that cSTML can significantly improve the urban traffic prediction performance and outperform all baseline models.
In the article titled “Predicting Citywide Crowd Dynamics at Big Events: A Deep Learning System,,” Jiang et al. study how to predict citywide crowd dynamics at big events. They aim to extract the “deep” trend only from the current momentary observations and generate an accurate prediction for the trend in the short future, which is considered to be an effective way to deal with the event situations. Motivated by this, they build an online system called DeepUrbanEvent, which can iteratively take citywide crowd dynamics from the current 1 hour as input and report the prediction results for the next 1 hour as output. A novel deep learning architecture built with recurrent neural networks is designed to effectively model these highly complex sequential data in an analogous manner to video prediction tasks. Experimental results demonstrate the superior performance of their proposed methodology to the existing approaches.
In the article titled “Deep Spatiotemporal Adaptive 3D Convolutional Neural Networks for Traffic Flow Prediction,,” Li et al. propose the Deep Spatiotemporal Adaptive 3D Convolution Neural Network (ST-A3DNet) to solve both spatiotemporal correlation and flexibility, and consider spatiotemporal complexity (complex external factors, such as weather and holidays). ST-A3DNet captures the spatio-temporal relationship at the same time through the Adaptive 3D convolution module, assigns different weights flexibly according to the influence of historical data, and obtains the impact of external factors on the flow through the Ex-Mask module. Extensive experiments are conducted on two datasets of Xi’an and Chengdu; the results show that ST-A3DNet achieves better results than the other 11 baselines.
In the article titled “DeepRoute+: Modeling Couriers’ Spatial-temporal Behaviors and Decision Preferences for Package Pick-up Route Prediction,” Wen et al. propose DeepRoute+ to predict couriers’ future package pick-up routes according to the couriers’ decision experience and preference learned from the historical behaviors. Specifically, DeepRoute+ consists of three layers: (1) the representation layer produces experience and preference-aware representations for the un-picked-up packages, in which a decision preference module can dynamically adjust the importance of factors that affect the courier’s decision under the current situation; (2) the transformer encoder layer encodes the representations of packages while considering the spatial-temporal correlations among them; and (3) the attention-based decoder layer uses the attention mechanism to generate the whole pick-up route recurrently. Experiments on a real-world logistics dataset demonstrate the state-of-the-art performance of DeepRoute+.
In the article titled “Multimodal Spatio-temporal Prediction with Stochastic Adversarial Networks,” Divya Saxena et al. propose a stochastic spatio-temporal generative model named D-GAN for more accurate ST prediction in multiple time steps. D-GAN consists of two components: (1) a spatio-temporal correlation network, which models the spatio-temporal joint distribution of pixels and supports a stochastic sampling of latent variables for multiple plausible futures, and (2) a stochastic adversarial network to jointly learn generation and variational inference of data through implicit distribution modeling. D-GAN also supports fusion of external factors through explicit objectives to improve the model learning. Extensive experiments performed on two real-world datasets show that D-GAN achieves significant performance improvement compared with baseline models.
By no means can one special issue scope all the exciting research in the field of spatio-temporal data mining with deep learning techniques. In addition, more challenges are emerging with more real-time data sensing, more spatio-temporal data-related applications, and more powerful processing capacities. We thank all the authors for their contributions. We appreciate the valuable comments from our reviewers, and we are grateful to the editorial board of ACM TIST for their help in coordinating the publication process. We hope that this collection will motivate further research in the exciting research area of deep-learning-based spatio-temporal data mining.
Senzhang Wang
Central South University, China
Junbo Zhang
JD Intelligent Cities Research; JD iCity, JD Tech, China
Yanjie Fu
University of Central Florida, U.S.A
Yong Li
Tsinghua University, China

Cited By

View all
  • (2024)TG-SPRED: Temporal Graph for Sensorial Data PREDictionACM Transactions on Sensor Networks10.1145/364989220:3(1-20)Online publication date: 13-Apr-2024
  • (2024)SiG: A Siamese-Based Graph Convolutional Network to Align Knowledge in Autonomous Transportation SystemsACM Transactions on Intelligent Systems and Technology10.1145/364386115:2(1-20)Online publication date: 28-Mar-2024
  • (2023)Isomorphic Graph Embedding for Progressive Maximal Frequent Subgraph MiningACM Transactions on Intelligent Systems and Technology10.1145/363063515:1(1-26)Online publication date: 19-Dec-2023
  • Show More Cited By

Index Terms

  1. Introduction to the Special Issue on Deep Learning for Spatio-Temporal Data: Part 2
        Index terms have been assigned to the content through auto-classification.

        Recommendations

        Comments

        Information & Contributors

        Information

        Published In

        cover image ACM Transactions on Intelligent Systems and Technology
        ACM Transactions on Intelligent Systems and Technology  Volume 13, Issue 2
        April 2022
        392 pages
        ISSN:2157-6904
        EISSN:2157-6912
        DOI:10.1145/3508464
        • Editor:
        • Huan Liu
        Issue’s Table of Contents

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        Published: 26 March 2022
        Published in TIST Volume 13, Issue 2

        Permissions

        Request permissions for this article.

        Check for updates

        Qualifiers

        • Introduction
        • Refereed

        Contributors

        Other Metrics

        Bibliometrics & Citations

        Bibliometrics

        Article Metrics

        • Downloads (Last 12 months)188
        • Downloads (Last 6 weeks)26
        Reflects downloads up to 15 Feb 2025

        Other Metrics

        Citations

        Cited By

        View all
        • (2024)TG-SPRED: Temporal Graph for Sensorial Data PREDictionACM Transactions on Sensor Networks10.1145/364989220:3(1-20)Online publication date: 13-Apr-2024
        • (2024)SiG: A Siamese-Based Graph Convolutional Network to Align Knowledge in Autonomous Transportation SystemsACM Transactions on Intelligent Systems and Technology10.1145/364386115:2(1-20)Online publication date: 28-Mar-2024
        • (2023)Isomorphic Graph Embedding for Progressive Maximal Frequent Subgraph MiningACM Transactions on Intelligent Systems and Technology10.1145/363063515:1(1-26)Online publication date: 19-Dec-2023
        • (2023)AutoCTS+: Joint Neural Architecture and Hyperparameter Search for Correlated Time Series ForecastingProceedings of the ACM on Management of Data10.1145/35889511:1(1-26)Online publication date: 30-May-2023
        • (2023)Towards more effective encoders in pre-training for sequential recommendationWorld Wide Web10.1007/s11280-023-01163-126:5(2801-2832)Online publication date: 12-May-2023

        View Options

        View options

        PDF

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        HTML Format

        View this article in HTML Format.

        HTML Format

        Login options

        Full Access

        Figures

        Tables

        Media

        Share

        Share

        Share this Publication link

        Share on social media