research-article

Augmented Vehicular Reality: Enabling Extended Vision for Future Vehicles

Authors:

Ramesh Govindan,

Marco Gruteser,

Gorkem KarAuthors Info & Claims

HotMobile '17: Proceedings of the 18th International Workshop on Mobile Computing Systems and Applications

Pages 67 - 72

https://doi.org/10.1145/3032970.3032976

Published: 21 February 2017 Publication History

Abstract

Like today's autonomous vehicle prototypes, vehicles in the future will have rich sensors to map and identify objects in the environment. For example, many autonomous vehicle prototypes today come with line-of-sight depth perception sensors like 3D cameras. These cameras are used for improving vehicular safety in autonomous driving, but have fundamentally limited visibility due to occlusions, sensing range, and extreme weather and lighting conditions. To improve visibility and performance, not just for autonomous vehicles but for other Advanced Driving Assistance Systems (ADAS), we explore a capability called Augmented Vehicular Reality (AVR). AVR broadens the vehicle's visual horizon by enabling it to share visual information with other nearby vehicles, but requires careful techniques to align coordinate frames of reference, and to detect dynamic objects. Preliminary evaluations hint at the feasibility of AVR and also highlight research challenges in achieving AVR's potential to improve autonomous vehicles and ADAS.

References

[1]

Google Self-Driving Car Project Monthly Report September 2016. https://static.googleusercontent.com/media/www.google.com/en//selfdrivingcar/files/reports/report-0916.pdf.

[2]

Here's How Tesla's Autopilot Works. http://www.businessinsider.com/how-teslas-autopilot-works-2016-7.

[3]

How Uber's First Self-Driving Car Works. http://www.businessinsider.com/how-ubers-driverless-cars-work-2016-9.

[4]

NVidia Drive PX 2. http://www.nvidia.com/object/drive-px.html.

[5]

ZED Stereo Camera. https://www.stereolabs.com/.

[6]

2010. IEEE Standard for Information technology-- Local and metropolitan area networks-- Specific requirements-- Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications Amendment 6: Wireless Access in Vehicular Environments. IEEE Std 802.11p-2010 (Amendment to IEEE Std 802.11-2007 as amended by IEEE Std 802.11k-2008, IEEE Std 802.11r-2008, IEEE Std 802.11y-2008, IEEE Std 802.11n-2009, and IEEE Std 802.11w-2009) (July 2010), 1--51.

[7]

David Astély, Erik Dahlman, Anders Furuskar, Ylva Jading, Magnus Lindstrom, and Stefan Parkvall. 2009. LTE: The Evolution of Mobile Broadband. Comm. Mag. 47, 4 (April 2009).

Digital Library

[8]

Cheng Bo, Xiang-Yang Li, Taeho Jung, Xufei Mao, Yue Tao, and Lan Yao. 2013. SmartLoc: Push the Limit of the Inertial Sensor Based Metropolitan Localization Using Smartphone. In Proceedings of the 19th Annual International Conference on Mobile Computing and Networking (MobiCom '13).

Digital Library

[9]

Kevin Boos, David Chu, and Eduardo Cuervo. FlashBack: Immersive Virtual Reality on Mobile Devices via Rendering Memoization. In Proceedings of the 14th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys '16).

Digital Library

[10]

Zhaowei Cai, Quanfu Fan, Rogério Schmidt Feris, and Nuno Vasconcelos. 2016. A Unified Multi-scale Deep Convolutional Neural Network for Fast Object Detection. In Computer Vision - ECCV 2016 - 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part IV. 354--370.

[11]

Tiffany Yu-Han Chen, Lenin S. Ravindranath, Shuo Deng, Paramvir Victor Bahl, and Hari Balakrishnan. 2015. Glimpse: Continuous, Real-Time Object Recognition on Mobile Devices. In 13th ACM Conference on Embedded Networked Sensor Systems (SenSys). Seoul, South Korea.

Digital Library

[12]

Andrew J. Davison, Ian D. Reid, Nicholas D. Molton, and Olivier Stasse. 2007. MonoSLAM: Real-Time Single Camera SLAM. IEEE Trans. Pattern Anal. Mach. Intell. 29, 6 (June 2007).

Digital Library

[13]

J. Engel, J. Stueckler, and D. Cremers. 2015. Large-Scale Direct SLAM with Stereo Cameras.

[14]

A. Gern, R. Moebus, and U. Franke. 2002. Vision-based lane recognition under adverse weather conditions using optical flow. In Intelligent Vehicle Symposium, 2002. IEEE, Vol. 2. 652--657 vol.2.

[15]

Mahanth Gowda, Justin Manweiler, Ashutosh Dhekne, Romit Roy Choudhury, and Justin D. Weisz. 2016. Tracking Drone Orientation with Multiple GPS Receivers. In Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking (MobiCom '16).

Digital Library

[16]

Santanu Guha, Kurt Plarre, Daniel Lissner, Somnath Mitra, Bhagavathy Krishna, Prabal Dutta, and Santosh Kumar. 2010. AutoWitness: Locating and Tracking Stolen Property While Tolerating GPS and Radio Outages. In Proceedings of the 8th ACM Conference on Embedded Networked Sensor Systems (SenSys '10).

Digital Library

[17]

Wolfgang Hess, Damon Kohler, Holger Rapp, and Daniel Andor. 2016. Real-Time Loop Closure in 2D LIDAR SLAM. In 2016 IEEE International Conference on Robotics and Automation (ICRA). 1271--1278.

Digital Library

[18]

Jing Huang and Suya You. 2012. Point cloud matching based on 3D self-similarity. In Computer Vision and Pattern Recognition Workshops (CVPRW), 2012 IEEE Computer Society Conference on. IEEE, 41--48.

[19]

Yurong Jiang, Hang Qiu, Matthew McCartney, Gaurav Sukhatme, Marco Gruteser, Fan Bai, Donald Grimm, and Ramesh Govindan. 2015. CARLOC: Precise Positioning of Automobiles. In Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems (SenSys '15).

Digital Library

[20]

Swarun Kumar, Shyamnath Gollakota, and Dina Katabi. 2012. A Cloud-assisted Design for Autonomous Driving. In Proceedings of the First Edition of the MCC Workshop on Mobile Cloud Computing (MCC '12).

Digital Library

[21]

Swarun Kumar, Lixin Shi, Nabeel Ahmed, Stephanie Gil, Dina Katabi, and Daniela Rus. 2012. CarSpeak: A Content-centric Network for Autonomous Driving. SIGCOMM Comput. Commun. Rev. 42, 4 (Aug. 2012).

Digital Library

[22]

Mathieu Labbé and François Michaud. 2014. Online global loop closure detection for large-scale multi-session graph-based slam. In 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2661--2666.

[23]

Raul MurArtal, J. M. M. Montiel, and Juan D. Tardos. 2015. ORB-SLAM: a Versatile and Accurate Monocular SLAM System. IEEE Transactions on Robotics (2015).

[24]

Richard A. Newcombe, Shahram Izadi, Otmar Hilliges, David Molyneaux, David Kim, Andrew J. Davison, Pushmeet Kohli, Jamie Shotton, Steve Hodges, and Andrew Fitzgibbon. 2011. KinectFusion: Real-time Dense Surface Mapping and Tracking. In Proceedings of the 2011 10th IEEE International Symposium on Mixed and Augmented Reality (ISMAR '11).

Digital Library

[25]

Joseph Redmon, Santosh Kumar Divvala, Ross B. Girshick, and Ali Farhadi. 2015. You Only Look Once: Unified, Real-Time Object Detection. CoRR abs/1506.02640 (2015). http://arxiv.org/abs/1506.02640

[26]

Miguel Angel Sotelo, Francisco Javier Rodriguez, Luis Magdalena, Luis Miguel Bergasa, and Luciano Boquete. 2004. A Color Vision-Based Lane Tracking System for Autonomous Driving on Unmarked Roads. Auton. Robots 16, 1 (Jan. 2004), 95--116.

Digital Library

[27]

Arvind Thiagarajan, Lenin Ravindranath, Katrina LaCurts, Samuel Madden, Hari Balakrishnan, Sivan Toledo, and Jakob Eriksson. 2009. VTrack: Accurate, Energy-aware Road Traffic Delay Estimation Using Mobile Phones. In Proceedings of the 7th ACM Conference on Embedded Networked Sensor Systems (SenSys '09).

Digital Library

[28]

Sebastian Thrun, Mike Montemerlo, Hendrik Dahlkamp, David Stavens, Andrei Aron, James Diebel, Philip Fong, John Gale, Morgan Halpenny, Gabriel Hoffmann, Kenny Lau, Celia Oakley, Mark Palatucci, Vaughan Pratt, Pascal Stang, Sven Strohband, Cedric Dupont, Lars-Erik Jendrossek, Christian Koelen, Charles Markey, Carlo Rummel, Joe van Niekerk, Eric Jensen, Philippe Alessandrini, Gary Bradski, Bob Davies, Scott Ettinger, Adrian Kaehler, Ara Nefian, and Pamela Mahoney. 2006. Stanley: The Robot That Won the DARPA Grand Challenge: Research Articles. J. Robot. Syst. 23, 9 (Sept. 2006), 661--692.

Digital Library

[29]

Chris Urmson, Joshua Anhalt, Drew Bagnell, Christopher Baker, Robert Bittner, M. N. Clark, John Dolan, Dave Duggins, Tugrul Galatali, Chris Geyer, Michele Gittleman, Sam Harbaugh, Martial Hebert, Thomas M. Howard, Sascha Kolski, Alonzo Kelly, Maxim Likhachev, Matt McNaughton, Nick Miller, Kevin Peterson, Brian Pilnick, Raj Rajkumar, Paul Rybski, Bryan Salesky, Young-Woo Seo, Sanjiv Singh, Jarrod Snider, Anthony Stentz, William "Red" Whittaker, Ziv Wolkowicki, Jason Ziglar, Hong Bae, Thomas Brown, Daniel Demitrish, Bakhtiar Litkouhi, Jim Nickolaou, Varsha Sadekar, Wende Zhang, Joshua Struble, Michael Taylor, Michael Darms, and Dave Ferguson. 2008. Autonomous Driving in Urban Environments: Boss and the Urban Challenge. J. Field Robot. 25, 8 (Aug. 2008), 425--466.

Digital Library

Cited By

Spampinato LFerretti DBuratti CMarini R(2025)Joint Trajectory Design and Radio Resource Management for UAV-Aided Vehicular NetworksIEEE Transactions on Vehicular Technology10.1109/TVT.2024.345495574:1(847-860)Online publication date: Jan-2025
https://doi.org/10.1109/TVT.2024.3454955
Mahmood AButler BJennings B(2024)Potential of Augmented Reality for Intelligent Transportation SystemsEncyclopedia of Computer Graphics and Games10.1007/978-3-031-23161-2_274(1455-1461)Online publication date: 5-Jan-2024
https://doi.org/10.1007/978-3-031-23161-2_274
Ding HShin K(2023)Context-Aware Beam Tracking for 5G mmWave V2I CommunicationsIEEE Transactions on Mobile Computing10.1109/TMC.2021.313795722:6(3257-3269)Online publication date: 1-Jun-2023
https://doi.org/10.1109/TMC.2021.3137957
Show More Cited By

Recommendations

AVR: Augmented Vehicular Reality
MobiSys '18: Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services

Autonomous vehicle prototypes today come with line-of-sight depth perception sensors like 3D cameras. These 3D sensors are used for improving vehicular safety in autonomous driving, but have fundamentally limited visibility due to occlusions, sensing ...
Augmented Vehicular Reality: Enabling Extended Vision for Future Automobiles

Autonomous vehicle prototypes today come with line-of-sight depth perception sensors like 3D cameras. These 3D sensors are used for improving vehicular safety in autonomous driving, but have fundamentally limited visibility due to occlusions, sensing ...
The Trouble with Autopilots: Assisted and Autonomous Driving on the Social Road
CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems

As self-driving cars have grown in sophistication and ability, they have been deployed on the road in both localised tests and as regular private vehicles. In this paper we draw upon publicly available videos of autonomous and assisted driving (...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

HotMobile '17: Proceedings of the 18th International Workshop on Mobile Computing Systems and Applications

February 2017

116 pages

ISBN:9781450349079

DOI:10.1145/3032970

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMOBILE: ACM Special Interest Group on Mobility of Systems, Users, Data and Computing

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 February 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

HotMobile '17

Sponsor:

SIGMOBILE

HotMobile '17: The 18th International Workshop on Mobile Computing Systems and Applications

February 21 - 22, 2017

CA, Sonoma, USA

Acceptance Rates

Overall Acceptance Rate 96 of 345 submissions, 28%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

38
Total Citations
View Citations
932
Total Downloads

Downloads (Last 12 months)29
Downloads (Last 6 weeks)4

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Spampinato LFerretti DBuratti CMarini R(2025)Joint Trajectory Design and Radio Resource Management for UAV-Aided Vehicular NetworksIEEE Transactions on Vehicular Technology10.1109/TVT.2024.345495574:1(847-860)Online publication date: Jan-2025
https://doi.org/10.1109/TVT.2024.3454955
Mahmood AButler BJennings B(2024)Potential of Augmented Reality for Intelligent Transportation SystemsEncyclopedia of Computer Graphics and Games10.1007/978-3-031-23161-2_274(1455-1461)Online publication date: 5-Jan-2024
https://doi.org/10.1007/978-3-031-23161-2_274
Ding HShin K(2023)Context-Aware Beam Tracking for 5G mmWave V2I CommunicationsIEEE Transactions on Mobile Computing10.1109/TMC.2021.313795722:6(3257-3269)Online publication date: 1-Jun-2023
https://doi.org/10.1109/TMC.2021.3137957
Njoku JNwakanma CKim D(2023)Building a Metaverse for Transportation Systems: A Brief Review and Demonstration2023 Fourteenth International Conference on Ubiquitous and Future Networks (ICUFN)10.1109/ICUFN57995.2023.10199405(544-549)Online publication date: 4-Jul-2023
https://doi.org/10.1109/ICUFN57995.2023.10199405
Zhou PKortoci PYau YFinley BWang XBraud TLee LTarkoma SKangasharju JHui P(2022)AICP: Augmented Informative Cooperative PerceptionIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2022.315517523:11(22505-22518)Online publication date: Nov-2022
https://doi.org/10.1109/TITS.2022.3155175
Zhang JShi HCui YQian FWang WZheng KWu J(2022)To Punctuality and Beyond: Meeting Application Deadlines with DTP2022 IEEE 30th International Conference on Network Protocols (ICNP)10.1109/ICNP55882.2022.9940391(1-11)Online publication date: 30-Oct-2022
https://doi.org/10.1109/ICNP55882.2022.9940391
Hwang SLee JKang S(2022)Enabling Product Recognition and Tracking Based on Text Detection for Mobile Augmented RealityIEEE Access10.1109/ACCESS.2022.320534410(98769-98782)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3205344
Ghoshal MDash PKong ZXu QHu YKoutsonikolas DLi Y(2022)Can 5G mmWave Support Multi-user AR?Passive and Active Measurement10.1007/978-3-030-98785-5_8(180-196)Online publication date: 22-Mar-2022
https://doi.org/10.1007/978-3-030-98785-5_8
An QShen Y(2021)On the Information Coupling and Propagation of Visual 3D Perception in Vehicular Networks With Position UncertaintyIEEE Transactions on Vehicular Technology10.1109/TVT.2021.312180170:12(13325-13339)Online publication date: Dec-2021
https://doi.org/10.1109/TVT.2021.3121801
Antonioli RFodor GSoldati PMaciel T(2021)Decentralized Joint Beamforming, User Scheduling, and QoS Management in 5G and Beyond SystemsIEEE Communications Standards Magazine10.1109/MCOMSTD.001.20000295:1(62-69)Online publication date: Mar-2021
https://doi.org/10.1109/MCOMSTD.001.2000029
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten