short-paper

Optimizing the Spatio-Temporal Resource Search Problem with Reinforcement Learning (GIS Cup)

Authors:

SIGSPATIAL '19: Proceedings of the 27th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems

Pages 628 - 631

https://doi.org/10.1145/3347146.3363351

Published: 05 November 2019 Publication History

Get Access

Abstract

Collecting spatio-temporal resources is an important goal in many real-world use cases such as finding customers for taxicabs. In this paper, we tackle the resource search problem posed by the GIS Cup 2019 where the objective is to minimize the average search time of taxicabs looking for customers. The main challenge is that the taxicabs may not communicate with each other and the only observation they have is the current time and position. Inspired by radial transit route structures in urban environments, our approach relies on round trips that are used as action space for a downstream reinforcement learning procedure. Our source code is publicly available at https://github.com/Fe18/TripBanditAgent.

References

[1]

Daniel Ayala, Ouri Wolfson, Bhaskar Dasgupta, Jie Lin, and Bo Xu. 2018. Spatio-temporal matching for urban transportation applications. ACM TSAS 3, 4 (2018), 11.

Google Scholar

[2]

Jago Dodson, Paul Mees, John Stone, and Matthew Burke. 2011. The Principles of Public Transport Network Planning: A review of the emerging literature with select examples. Issues paper 15 (2011).

Google Scholar

[3]

Gregor Jossé, Klaus Arthur Schmid, and Matthias Schubert. 2015. Probabilistic Resource Route Queries with Reappearance. In Proc. of EDBT. 445--456.

Google Scholar

[4]

Sebastian Schmoll, Sabrina Friedl, and Matthias Schubert. 2019. Scaling the Dynamic Resource Routing Problem. In Proc. of SSTD. 80--89.

Digital Library

Google Scholar

[5]

Sebastian Schmoll and Matthias Schubert. 2018. Vision paper: reinforcement learning in smart spatio-temporal environments. In Proc. of SIGSPATIAL GIS. 81--84.

Digital Library

Google Scholar

[6]

Ronald J Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning 8, 3-4 (1992), 229--256.

Google Scholar

Cited By

View all

Wang GHe SJiang LWang SMiao FZhang FDong ZZhang D(2024)FairMove: A Data-Driven Vehicle Displacement System for Jointly Optimizing Profit Efficiency and Fairness of Electric For-Hire VehiclesIEEE Transactions on Mobile Computing10.1109/TMC.2023.332667623:6(6785-6802)Online publication date: Jun-2024
https://doi.org/10.1109/TMC.2023.3326676
Wang GQin ZWang SSun HDong ZZhang D(2024)Towards Accessible Shared Autonomous Electric Mobility With Dynamic DeadlinesIEEE Transactions on Mobile Computing10.1109/TMC.2022.321312523:1(925-940)Online publication date: Jan-2024
https://doi.org/10.1109/TMC.2022.3213125
Bassem C(2024)Spatio-temporal Idle Routing for Green Mobility2024 25th IEEE International Conference on Mobile Data Management (MDM)10.1109/MDM61037.2024.00059(283-288)Online publication date: 24-Jun-2024
https://doi.org/10.1109/MDM61037.2024.00059
Show More Cited By

Index Terms

Optimizing the Spatio-Temporal Resource Search Problem with Reinforcement Learning (GIS Cup)
1. Computing methodologies
  1. Artificial intelligence
    1. Planning and scheduling
      1. Multi-agent planning

Recommendations

Reward Shaping in Episodic Reinforcement Learning
AAMAS '17: Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems

Recent advancements in reinforcement learning confirm that reinforcement learning techniques can solve large scale problems leading to high quality autonomous decision making. It is a matter of time until we will see large scale applications of ...
Multi-agent reinforcement learning based resource allocation for vehicular networks
PCCNT '23: Proceedings of the 2023 International Conference on Power, Communication, Computing and Networking Technologies

Vehicular communication becomes a hot research topic due to its advantage in congestion and traffic accident avoidance. To enhance the user experience quality and driving safety of vehicular networks, it is imperative to increase the capacity of each V2X ...
Advanced planning for autonomous vehicles using reinforcement learning and deep inverse reinforcement learning
Abstract
Autonomous vehicles promise to improve traffic safety while, at the same time, increase fuel efficiency and reduce congestion. They represent the main trend in future intelligent transportation systems. This paper concentrates on the planning ...
Highlights
- A new MDP is proposed to model highway traffic.
- The optimal policies for overtaking/tailgating are obtained using RL.
- Reward functions in the formulation of MaxEnt IRL can take any nonlinear form.
- New MaxEnt deep IRL algorithms ...

Comments

Information & Contributors

Information

Published In

SIGSPATIAL '19: Proceedings of the 27th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems

November 2019

648 pages

ISBN:9781450369091

DOI:10.1145/3347146

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 November 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper
Research
Refereed limited

Conference

SIGSPATIAL '19

Sponsor:

SIGSPATIAL

SIGSPATIAL '19: 27th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems

November 5 - 8, 2019

IL, Chicago, USA

Acceptance Rates

SIGSPATIAL '19 Paper Acceptance Rate 34 of 161 submissions, 21%;

Overall Acceptance Rate 257 of 1,238 submissions, 21%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
174
Total Downloads

Downloads (Last 12 months)11
Downloads (Last 6 weeks)2

Reflects downloads up to 25 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Wang GHe SJiang LWang SMiao FZhang FDong ZZhang D(2024)FairMove: A Data-Driven Vehicle Displacement System for Jointly Optimizing Profit Efficiency and Fairness of Electric For-Hire VehiclesIEEE Transactions on Mobile Computing10.1109/TMC.2023.332667623:6(6785-6802)Online publication date: Jun-2024
https://doi.org/10.1109/TMC.2023.3326676
Wang GQin ZWang SSun HDong ZZhang D(2024)Towards Accessible Shared Autonomous Electric Mobility With Dynamic DeadlinesIEEE Transactions on Mobile Computing10.1109/TMC.2022.321312523:1(925-940)Online publication date: Jan-2024
https://doi.org/10.1109/TMC.2022.3213125
Bassem C(2024)Spatio-temporal Idle Routing for Green Mobility2024 25th IEEE International Conference on Mobile Data Management (MDM)10.1109/MDM61037.2024.00059(283-288)Online publication date: 24-Jun-2024
https://doi.org/10.1109/MDM61037.2024.00059
Lin FHsieh H(2023)Exploiting Network Structure in Multi-criteria Distributed and Competitive Stationary-resource SearchingACM Transactions on Spatial Algorithms and Systems10.1145/35699379:4(1-33)Online publication date: 31-Dec-2023
https://dl.acm.org/doi/10.1145/3569937
Lin FHsieh H(2022)Multicriteria Route Planning for In-Operation Mass Transit under Urban DataApplied Sciences10.3390/app1206312712:6(3127)Online publication date: 18-Mar-2022
https://doi.org/10.3390/app12063127
Zheng BHu QMing LHu JChen LZheng KJensen C(2021)SOUP: Spatial-Temporal Demand Forecasting and Competitive SupplyIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2021.3110778(1-1)Online publication date: 2021
https://doi.org/10.1109/TKDE.2021.3110778
Wang GZhong SWang SMiao FDong ZZhang D(2021)Data-Driven Fairness-Aware Vehicle Displacement for Large-Scale Electric Taxi Fleets2021 IEEE 37th International Conference on Data Engineering (ICDE)10.1109/ICDE51399.2021.00108(1200-1211)Online publication date: Apr-2021
https://doi.org/10.1109/ICDE51399.2021.00108
O’Keeffe KAnklesaria SSanti PRatti C(2021)Using reinforcement learning to minimize taxi idle timesJournal of Intelligent Transportation Systems10.1080/15472450.2021.189780326:4(498-509)Online publication date: 16-Mar-2021
https://doi.org/10.1080/15472450.2021.1897803
Kim JPfoser DZufle A(2020)Vehicle Relocation for Ride-Hailing2020 IEEE 7th International Conference on Data Science and Advanced Analytics (DSAA)10.1109/DSAA49011.2020.00074(589-598)Online publication date: Oct-2020
https://doi.org/10.1109/DSAA49011.2020.00074

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Reward Shaping in Episodic Reinforcement Learning

Multi-agent reinforcement learning based resource allocation for vehicular networks

Advanced planning for autonomous vehicles using reinforcement learning and deep inverse reinforcement learning

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations