skip to main content
10.1145/3032970.3032976acmconferencesArticle/Chapter ViewAbstractPublication PageshotmobileConference Proceedingsconference-collections
research-article

Augmented Vehicular Reality: Enabling Extended Vision for Future Vehicles

Published: 21 February 2017 Publication History

Abstract

Like today's autonomous vehicle prototypes, vehicles in the future will have rich sensors to map and identify objects in the environment. For example, many autonomous vehicle prototypes today come with line-of-sight depth perception sensors like 3D cameras. These cameras are used for improving vehicular safety in autonomous driving, but have fundamentally limited visibility due to occlusions, sensing range, and extreme weather and lighting conditions. To improve visibility and performance, not just for autonomous vehicles but for other Advanced Driving Assistance Systems (ADAS), we explore a capability called Augmented Vehicular Reality (AVR). AVR broadens the vehicle's visual horizon by enabling it to share visual information with other nearby vehicles, but requires careful techniques to align coordinate frames of reference, and to detect dynamic objects. Preliminary evaluations hint at the feasibility of AVR and also highlight research challenges in achieving AVR's potential to improve autonomous vehicles and ADAS.

References

[1]
Google Self-Driving Car Project Monthly Report September 2016. https://static.googleusercontent.com/media/www.google.com/en//selfdrivingcar/files/reports/report-0916.pdf.
[2]
Here's How Tesla's Autopilot Works. http://www.businessinsider.com/how-teslas-autopilot-works-2016-7.
[3]
How Uber's First Self-Driving Car Works. http://www.businessinsider.com/how-ubers-driverless-cars-work-2016-9.
[4]
NVidia Drive PX 2. http://www.nvidia.com/object/drive-px.html.
[5]
ZED Stereo Camera. https://www.stereolabs.com/.
[6]
2010. IEEE Standard for Information technology-- Local and metropolitan area networks-- Specific requirements-- Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications Amendment 6: Wireless Access in Vehicular Environments. IEEE Std 802.11p-2010 (Amendment to IEEE Std 802.11-2007 as amended by IEEE Std 802.11k-2008, IEEE Std 802.11r-2008, IEEE Std 802.11y-2008, IEEE Std 802.11n-2009, and IEEE Std 802.11w-2009) (July 2010), 1--51.
[7]
David Astély, Erik Dahlman, Anders Furuskar, Ylva Jading, Magnus Lindstrom, and Stefan Parkvall. 2009. LTE: The Evolution of Mobile Broadband. Comm. Mag. 47, 4 (April 2009).
[8]
Cheng Bo, Xiang-Yang Li, Taeho Jung, Xufei Mao, Yue Tao, and Lan Yao. 2013. SmartLoc: Push the Limit of the Inertial Sensor Based Metropolitan Localization Using Smartphone. In Proceedings of the 19th Annual International Conference on Mobile Computing and Networking (MobiCom '13).
[9]
Kevin Boos, David Chu, and Eduardo Cuervo. FlashBack: Immersive Virtual Reality on Mobile Devices via Rendering Memoization. In Proceedings of the 14th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys '16).
[10]
Zhaowei Cai, Quanfu Fan, Rogério Schmidt Feris, and Nuno Vasconcelos. 2016. A Unified Multi-scale Deep Convolutional Neural Network for Fast Object Detection. In Computer Vision - ECCV 2016 - 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part IV. 354--370.
[11]
Tiffany Yu-Han Chen, Lenin S. Ravindranath, Shuo Deng, Paramvir Victor Bahl, and Hari Balakrishnan. 2015. Glimpse: Continuous, Real-Time Object Recognition on Mobile Devices. In 13th ACM Conference on Embedded Networked Sensor Systems (SenSys). Seoul, South Korea.
[12]
Andrew J. Davison, Ian D. Reid, Nicholas D. Molton, and Olivier Stasse. 2007. MonoSLAM: Real-Time Single Camera SLAM. IEEE Trans. Pattern Anal. Mach. Intell. 29, 6 (June 2007).
[13]
J. Engel, J. Stueckler, and D. Cremers. 2015. Large-Scale Direct SLAM with Stereo Cameras.
[14]
A. Gern, R. Moebus, and U. Franke. 2002. Vision-based lane recognition under adverse weather conditions using optical flow. In Intelligent Vehicle Symposium, 2002. IEEE, Vol. 2. 652--657 vol.2.
[15]
Mahanth Gowda, Justin Manweiler, Ashutosh Dhekne, Romit Roy Choudhury, and Justin D. Weisz. 2016. Tracking Drone Orientation with Multiple GPS Receivers. In Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking (MobiCom '16).
[16]
Santanu Guha, Kurt Plarre, Daniel Lissner, Somnath Mitra, Bhagavathy Krishna, Prabal Dutta, and Santosh Kumar. 2010. AutoWitness: Locating and Tracking Stolen Property While Tolerating GPS and Radio Outages. In Proceedings of the 8th ACM Conference on Embedded Networked Sensor Systems (SenSys '10).
[17]
Wolfgang Hess, Damon Kohler, Holger Rapp, and Daniel Andor. 2016. Real-Time Loop Closure in 2D LIDAR SLAM. In 2016 IEEE International Conference on Robotics and Automation (ICRA). 1271--1278.
[18]
Jing Huang and Suya You. 2012. Point cloud matching based on 3D self-similarity. In Computer Vision and Pattern Recognition Workshops (CVPRW), 2012 IEEE Computer Society Conference on. IEEE, 41--48.
[19]
Yurong Jiang, Hang Qiu, Matthew McCartney, Gaurav Sukhatme, Marco Gruteser, Fan Bai, Donald Grimm, and Ramesh Govindan. 2015. CARLOC: Precise Positioning of Automobiles. In Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems (SenSys '15).
[20]
Swarun Kumar, Shyamnath Gollakota, and Dina Katabi. 2012. A Cloud-assisted Design for Autonomous Driving. In Proceedings of the First Edition of the MCC Workshop on Mobile Cloud Computing (MCC '12).
[21]
Swarun Kumar, Lixin Shi, Nabeel Ahmed, Stephanie Gil, Dina Katabi, and Daniela Rus. 2012. CarSpeak: A Content-centric Network for Autonomous Driving. SIGCOMM Comput. Commun. Rev. 42, 4 (Aug. 2012).
[22]
Mathieu Labbé and François Michaud. 2014. Online global loop closure detection for large-scale multi-session graph-based slam. In 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2661--2666.
[23]
Raul MurArtal, J. M. M. Montiel, and Juan D. Tardos. 2015. ORB-SLAM: a Versatile and Accurate Monocular SLAM System. IEEE Transactions on Robotics (2015).
[24]
Richard A. Newcombe, Shahram Izadi, Otmar Hilliges, David Molyneaux, David Kim, Andrew J. Davison, Pushmeet Kohli, Jamie Shotton, Steve Hodges, and Andrew Fitzgibbon. 2011. KinectFusion: Real-time Dense Surface Mapping and Tracking. In Proceedings of the 2011 10th IEEE International Symposium on Mixed and Augmented Reality (ISMAR '11).
[25]
Joseph Redmon, Santosh Kumar Divvala, Ross B. Girshick, and Ali Farhadi. 2015. You Only Look Once: Unified, Real-Time Object Detection. CoRR abs/1506.02640 (2015). http://arxiv.org/abs/1506.02640
[26]
Miguel Angel Sotelo, Francisco Javier Rodriguez, Luis Magdalena, Luis Miguel Bergasa, and Luciano Boquete. 2004. A Color Vision-Based Lane Tracking System for Autonomous Driving on Unmarked Roads. Auton. Robots 16, 1 (Jan. 2004), 95--116.
[27]
Arvind Thiagarajan, Lenin Ravindranath, Katrina LaCurts, Samuel Madden, Hari Balakrishnan, Sivan Toledo, and Jakob Eriksson. 2009. VTrack: Accurate, Energy-aware Road Traffic Delay Estimation Using Mobile Phones. In Proceedings of the 7th ACM Conference on Embedded Networked Sensor Systems (SenSys '09).
[28]
Sebastian Thrun, Mike Montemerlo, Hendrik Dahlkamp, David Stavens, Andrei Aron, James Diebel, Philip Fong, John Gale, Morgan Halpenny, Gabriel Hoffmann, Kenny Lau, Celia Oakley, Mark Palatucci, Vaughan Pratt, Pascal Stang, Sven Strohband, Cedric Dupont, Lars-Erik Jendrossek, Christian Koelen, Charles Markey, Carlo Rummel, Joe van Niekerk, Eric Jensen, Philippe Alessandrini, Gary Bradski, Bob Davies, Scott Ettinger, Adrian Kaehler, Ara Nefian, and Pamela Mahoney. 2006. Stanley: The Robot That Won the DARPA Grand Challenge: Research Articles. J. Robot. Syst. 23, 9 (Sept. 2006), 661--692.
[29]
Chris Urmson, Joshua Anhalt, Drew Bagnell, Christopher Baker, Robert Bittner, M. N. Clark, John Dolan, Dave Duggins, Tugrul Galatali, Chris Geyer, Michele Gittleman, Sam Harbaugh, Martial Hebert, Thomas M. Howard, Sascha Kolski, Alonzo Kelly, Maxim Likhachev, Matt McNaughton, Nick Miller, Kevin Peterson, Brian Pilnick, Raj Rajkumar, Paul Rybski, Bryan Salesky, Young-Woo Seo, Sanjiv Singh, Jarrod Snider, Anthony Stentz, William "Red" Whittaker, Ziv Wolkowicki, Jason Ziglar, Hong Bae, Thomas Brown, Daniel Demitrish, Bakhtiar Litkouhi, Jim Nickolaou, Varsha Sadekar, Wende Zhang, Joshua Struble, Michael Taylor, Michael Darms, and Dave Ferguson. 2008. Autonomous Driving in Urban Environments: Boss and the Urban Challenge. J. Field Robot. 25, 8 (Aug. 2008), 425--466.

Cited By

View all
  • (2025)Joint Trajectory Design and Radio Resource Management for UAV-Aided Vehicular NetworksIEEE Transactions on Vehicular Technology10.1109/TVT.2024.345495574:1(847-860)Online publication date: Jan-2025
  • (2024)Potential of Augmented Reality for Intelligent Transportation SystemsEncyclopedia of Computer Graphics and Games10.1007/978-3-031-23161-2_274(1455-1461)Online publication date: 5-Jan-2024
  • (2023)Context-Aware Beam Tracking for 5G mmWave V2I CommunicationsIEEE Transactions on Mobile Computing10.1109/TMC.2021.313795722:6(3257-3269)Online publication date: 1-Jun-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
HotMobile '17: Proceedings of the 18th International Workshop on Mobile Computing Systems and Applications
February 2017
116 pages
ISBN:9781450349079
DOI:10.1145/3032970
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 February 2017

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. ADAS
  2. Autonomous Cars
  3. Collaborative Sensing
  4. Extended Vision

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

HotMobile '17
Sponsor:

Acceptance Rates

Overall Acceptance Rate 96 of 345 submissions, 28%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)29
  • Downloads (Last 6 weeks)4
Reflects downloads up to 28 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Joint Trajectory Design and Radio Resource Management for UAV-Aided Vehicular NetworksIEEE Transactions on Vehicular Technology10.1109/TVT.2024.345495574:1(847-860)Online publication date: Jan-2025
  • (2024)Potential of Augmented Reality for Intelligent Transportation SystemsEncyclopedia of Computer Graphics and Games10.1007/978-3-031-23161-2_274(1455-1461)Online publication date: 5-Jan-2024
  • (2023)Context-Aware Beam Tracking for 5G mmWave V2I CommunicationsIEEE Transactions on Mobile Computing10.1109/TMC.2021.313795722:6(3257-3269)Online publication date: 1-Jun-2023
  • (2023)Building a Metaverse for Transportation Systems: A Brief Review and Demonstration2023 Fourteenth International Conference on Ubiquitous and Future Networks (ICUFN)10.1109/ICUFN57995.2023.10199405(544-549)Online publication date: 4-Jul-2023
  • (2022)AICP: Augmented Informative Cooperative PerceptionIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2022.315517523:11(22505-22518)Online publication date: Nov-2022
  • (2022)To Punctuality and Beyond: Meeting Application Deadlines with DTP2022 IEEE 30th International Conference on Network Protocols (ICNP)10.1109/ICNP55882.2022.9940391(1-11)Online publication date: 30-Oct-2022
  • (2022)Enabling Product Recognition and Tracking Based on Text Detection for Mobile Augmented RealityIEEE Access10.1109/ACCESS.2022.320534410(98769-98782)Online publication date: 2022
  • (2022)Can 5G mmWave Support Multi-user AR?Passive and Active Measurement10.1007/978-3-030-98785-5_8(180-196)Online publication date: 22-Mar-2022
  • (2021)On the Information Coupling and Propagation of Visual 3D Perception in Vehicular Networks With Position UncertaintyIEEE Transactions on Vehicular Technology10.1109/TVT.2021.312180170:12(13325-13339)Online publication date: Dec-2021
  • (2021)Decentralized Joint Beamforming, User Scheduling, and QoS Management in 5G and Beyond SystemsIEEE Communications Standards Magazine10.1109/MCOMSTD.001.20000295:1(62-69)Online publication date: Mar-2021
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media