skip to main content
10.1145/3447993.3483242acmconferencesArticle/Chapter ViewAbstractPublication PagesmobicomConference Proceedingsconference-collections
research-article
Public Access

EMP: edge-assisted multi-vehicle perception

Published: 25 October 2021 Publication History

Abstract

Connected and Autonomous Vehicles (CAVs) heavily rely on 3D sensors such as LiDARs, radars, and stereo cameras. However, 3D sensors from a single vehicle suffer from two fundamental limitations: vulnerability to occlusion and loss of details on far-away objects. To overcome both limitations, in this paper, we design, implement, and evaluate EMP, a novel edge-assisted multi-vehicle perception system for CAVs. In EMP, multiple nearby CAVs share their raw sensor data with an edge server which then merges CAVs' individual views to form a more complete view with a higher resolution. The merged view can drastically enhance the perception quality of the participating CAVs. Our core methodological contribution is to make the sensor data sharing scalable, adaptive, and resource-efficient over oftentimes highly fluctuating wireless links through a series of novel algorithms, which are then integrated into a full-fledged cooperative sensing pipeline. Extensive evaluations demonstrate that EMP can achieve real-time processing at 24 FPS and end-to-end latency of 93 ms on average. EMP reduces the end-to-end latency by 49% to 65% compared to the traditional vehicle-to-vehicle (V2V) sharing approach without edge support. Our case studies show that cooperative sensing powered by EMP can detect hazards such as blind spots faster by 0.5 to 1.1 seconds, compared to a single vehicle's perception.

References

[1]
2001. Linux Traffic Control. https://man7.org/linux/man-pages/man8/tc.8.html.
[2]
2013. Vehicle Stopping Distance and Time. https://nacto.org/docs/usdg/vehicle_stopping_distance_and_time_upenn.pdf.
[3]
2016. Annual Traffic Report - WSDOT. https://www.wsdot.wa.gov/mapsdata/travel/pdf/Annual_Traffic_Report_2016.pdf.
[4]
2018. How autonomous vehicles could save over 350K lives in the US and millions worldwide. https://zdnet.com/article/how-autonomous-vehicles-could-save-over-350k-lives-in-the-us-and-millions-worldwide/.
[5]
2019. AT&T integrating 5G with Microsoft cloud to enable next-generation solutions on the edge. https://news.microsoft.com/2019/11/26/att-integrating-5g-with-microsoft-cloud-to-enable-next-generation-solutions-on-the-edge/.
[6]
2019. Study shows autonomous vehicles can help improve traffic flow. https://phys.org/news/2018-02-autonomous-vehicles-traffic.html.
[7]
2019. Verizon and AWS announce 5G Edge computing partnership. https://techcrunch.com/2019/12/03/verizon-and-aws-announce-5g-edge-computing-partnership/.
[8]
2020. Apollo. https://apollo.auto/.
[9]
2020. INRIX Global Traffic Scorecard - Last-Mile Speed. https://inrix.com/scorecard/.
[10]
2020. Microsoft partners with the industry to unlock new 5G scenarios with Azure Edge Zones. https://azure.microsoft.com/en-us/blog/microsoft-partners-with-the-industry-to-unlock-new-5g-scenarios-with-azure-edge-zones/.
[11]
2020. Snow and Ice Pose a Vexing Obstacle for Self-Driving Cars. https://www.wired.com/story/snow-ice-pose-vexing-obstacle-self-driving-cars/.
[12]
2020. Unprotected Turns - The Right Way To Navigate Complex Intersections. https://www.epermittest.com/drivers-education/unprotected-turns.
[13]
2021. Autoware. https://www.autoware.org/.
[14]
2021. Draco 3D Graphics Compression. https://google.github.io/draco/.
[15]
2021. EMP GitHub repository. https://github.com/Shawnxm/EMP.
[16]
2021. High performance INS for ADAS and autonomous vehicle testing. https://www.oxts.com/products/rt3000/.
[17]
2021. Lidar --- Wikipedia. https://en.wikipedia.org/wiki/Lidar.
[18]
2021. Netgear nighthawk x10 ad7200 smart wifi router (r9000). https://www.netgear.com/home/wifi/routers/ad7200-fastest-router/.
[19]
2021. TrustZone - Arm Developer. https://developer.arm.com/ip-products/security-ip/trustzone.
[20]
2021. Velodyne LiDAR HDL-64E. https://www.velodynelidar.com/hdl-64e.html.
[21]
2021. Verizon Hyper Precise Location. https://thingspace.verizon.com/services/hyper-precise-location/.
[22]
Eduardo Arnold, Mehrdad Dianati, Robert de Temple, and Saber Fallah. 2020. Cooperative perception for 3D object detection in driving scenarios using infrastructure sensors. IEEE Transactions on Intelligent Transportation Systems (2020).
[23]
Alireza Asvadi, Pedro Girao, Paulo Peixoto, and Urbano Nunes. 2016. 3D object tracking using RGB and LIDAR data. In 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC). IEEE, 1255--1260.
[24]
Franz Aurenhammer. 1987. Power diagrams: properties, algorithms and applications. SIAM J. Comput. 16, 1 (1987), 78--96.
[25]
Franz Aurenhammer. 1991. Voronoi diagrams---a survey of a fundamental geometric data structure. ACM Computing Surveys (CSUR) 23, 3 (1991), 345--405.
[26]
Franz Aurenhammer, Rolf Klein, and Der-Tsai Lee. 2013. Voronoi diagrams and Delaunay triangulations. World Scientific Publishing Company.
[27]
Yulong Cao, Chaowei Xiao, Benjamin Cyr, Yimeng Zhou, Won Park, Sara Rampazzi, Qi Alfred Chen, Kevin Fu, and Z Morley Mao. 2019. Adversarial sensor attack on lidar-based perception in autonomous driving. In Proceedings of the 2019 ACM SIGSAC conference on computer and communications security. 2267--2281.
[28]
Qi Chen, Xu Ma, Sihai Tang, Jingda Guo, Qing Yang, and Song Fu. 2019. F-cooper: feature based cooperative perception for autonomous vehicle edge computing system using 3D point clouds. In Proceedings of the 4th ACM/IEEE Symposium on Edge Computing. 88--100.
[29]
Qi Chen, Sihai Tang, Qing Yang, and Song Fu. 2019. Cooper: Cooperative perception for connected autonomous vehicles based on 3d point clouds. In 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS). IEEE, 514--524.
[30]
Junil Choi, Vutha Va, Nuria Gonzalez-Prelcic, Robert Daniels, Chandra R Bhat, and Robert W Heath. 2016. Millimeter-wave vehicular communication to support massive automotive sensing. IEEE Communications Magazine 54, 12 (2016), 160--167.
[31]
Tanmoy Das, Lu Chen, Rupam Kundu, Arjun Bakshi, Prasun Sinha, Kannan Srinivasan, Gaurav Bansal, and Takayuki Shimizu. 2018. Corecast: Collision resilient broadcasting in vehicular networks. In Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services. 217--229.
[32]
Haowen Deng, Tolga Birdal, and Slobodan Ilic. 2018. Ppfnet: Global context aware local features for robust 3d point matching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 195--205.
[33]
Martin A Fischler and Robert C Bolles. 1981. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24, 6 (1981), 381--395.
[34]
Andreas Geiger, Philip Lenz, and Raquel Urtasun. 2012. Are we ready for Autonomous Driving? The KITTI Vision Benchmark Suite. In Conference on Computer Vision and Pattern Recognition (CVPR).
[35]
Zan Gojcic, Caifa Zhou, Jan D Wegner, and Andreas Wieser. 2019. The perfect match: 3d point cloud matching with smoothed densities. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5545--5554.
[36]
Shengtuo Hu, Qi Alfred Chen, Jiwon Joung, Can Carlak, Yiheng Feng, Z Morley Mao, and Henry X Liu. 2020. Cvshield: Guarding sensor data in connected vehicle with trusted execution environment. In Proceedings of the Second ACM Workshop on Automotive and Aerial Vehicle Security. 1--4.
[37]
Yun Chao Hu, Milan Patel, Dario Sabella, Nurit Sprecher, and Valerie Young. 2015. Mobile Edge Computing---A Key Technology towards 5G. ETSI white paper 11, 11 (2015), 1--16.
[38]
Braden Hurl, Krzysztof Czarnecki, and Steven Waslander. 2019. Precise synthetic image and lidar (presil) dataset for autonomous vehicle perception. In 2019 IEEE Intelligent Vehicles Symposium (IV). IEEE, 2522--2529.
[39]
John B Kenney. 2011. Dedicated short-range communications (DSRC) standards in the United States. Proc. IEEE 99, 7 (2011), 1162--1182.
[40]
Swarun Kumar, Lixin Shi, Nabeel Ahmed, Stephanie Gil, Dina Katabi, and Daniela Rus. 2012. Carspeak: a content-centric network for autonomous driving. ACM SIGCOMM Computer Communication Review 42, 4 (2012), 259--270.
[41]
Alex H Lang, Sourabh Vora, Holger Caesar, Lubing Zhou, Jiong Yang, and Oscar Beijbom. 2019. PointPillars: Fast encoders for object detection from point clouds. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 12697--12705.
[42]
Adam Langley, Alistair Riddoch, Alyssa Wilk, Antonio Vicente, Charles Krasic, Dan Zhang, Fan Yang, Fedor Kouranov, Ian Swett, Janardhan Iyengar, et al. 2017. The quic transport protocol: Design and internet-scale deployment. In Proceedings of the conference of the ACM special interest group on data communication. 183--196.
[43]
Jinsung Lee, Sungyong Lee, Jongyun Lee, Sandesh Dhawaskar Sathyanarayana, Hyoyoung Lim, Jihoon Lee, Xiaoqing Zhu, Sangeeta Ramakrishnan, Dirk Grunwald, Kyunghan Lee, et al. 2020. PERCEIVE: deep learning-based cellular uplink prediction using real-time scheduling patterns. In Proceedings of the 18th International Conference on Mobile Systems, Applications, and Services. 377--390.
[44]
Kyungjin Lee, Juheon Yi, Youngki Lee, Sunghyun Choi, and Young Min Kim. 2020. GROOT: a real-time streaming system of high-fidelity volumetric videos. In Proceedings of the 26th Annual International Conference on Mobile Computing and Networking. 1--14.
[45]
Jianqiang Li, Genqiang Deng, Chengwen Luo, Qiuzhen Lin, Qiao Yan, and Zhong Ming. 2016. A hybrid path planning method in unmanned air/ground vehicle (UAV/UGV) cooperative systems. IEEE Transactions on Vehicular Technology 65, 12 (2016), 9585--9596.
[46]
Shih-Chieh Lin, Yunqi Zhang, Chang-Hong Hsu, Matt Skach, Md E Haque, Lingjia Tang, and Jason Mars. 2018. The architectural implications of autonomous driving: Constraints and acceleration. In Proceedings of the Twenty-Third International Conference on Architectural Support for Programming Languages and Operating Systems. 751--766.
[47]
Hansi Liu, Pengfei Ren, Shubham Jain, Mohannad Murad, Marco Gruteser, and Fan Bai. 2019. FusionEye: Perception Sharing for Connected Vehicles and its Bandwidth-Accuracy Trade-offs. In 2019 16th Annual IEEE International Conference on Sensing, Communication, and Networking (SECON). IEEE, 1--9.
[48]
Luyang Liu, Hongyu Li, and Marco Gruteser. 2019. Edge assisted real-time object detection for mobile augmented reality. In The 25th Annual International Conference on Mobile Computing and Networking. 1--16.
[49]
Martin Magnusson, Achim Lilienthal, and Tom Duckett. 2007. Scan registration for autonomous mining vehicles using 3D-NDT. Journal of Field Robotics 24, 10 (2007), 803--827.
[50]
Gaurang Naik, Biplav Choudhury, and Jung-Min Park. 2019. IEEE 802.11 bd & 5G NR V2X: Evolution of radio access technologies for V2X communications. IEEE Access 7 (2019), 70169--70184.
[51]
Arvind Narayanan, Eman Ramadan, Jason Carpenter, Qingxu Liu, Yu Liu, Feng Qian, and Zhi-Li Zhang. 2020. A first look at commercial 5G performance on smartphones. In Proceedings of The Web Conference 2020. 894--905.
[52]
Arvind Narayanan, Eman Ramadan, Rishabh Mehta, Xinyue Hu, Qingxu Liu, Rostand AK Fezeu, Udhaya Kumar Dayalan, Saurabh Verma, Peiqi Ji, Tao Li, et al. 2020. Lumos5g: Mapping and predicting commercial mmwave 5g throughput. In Proceedings of the ACM Internet Measurement Conference. 176--193.
[53]
Arvind Narayanan, Xumiao Zhang, Ruiyang Zhu, Ahmad Hassan, Shuowei Jin, Xiao Zhu, Xiaoxuan Zhang, Denis Rybkin, Zhengxuan Yang, Z Morley Mao, et al. 2021. A Variegated Look at 5G in the Wild: Performance, Power, and QoE Implications. In Proceedings of the Annual conference of the ACM Special Interest Group on Data Communication on the applications, technologies, architectures, and protocols for computer communication.
[54]
Cristina Olaverri-Monreal, Pedro Gomes, Ricardo Fernandes, Fausto Vieira, and Michel Ferreira. 2010. The See-Through System: A VANET-enabled assistant for overtaking maneuvers. In 2010 IEEE Intelligent Vehicles Symposium. IEEE, 123--128.
[55]
Apostolos Papathanassiou and Alexey Khoryaev. 2017. Cellular V2X as the essential enabler of superior global connected transportation services. IEEE 5G Tech Focus 1, 2 (2017), 1--2.
[56]
Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, and Ramesh Govindan. 2018. AVR: Augmented vehicular reality. In Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 81--95.
[57]
Andreas Rauch, Felix Klanner, Ralph Rasshofer, and Klaus Dietmayer. 2012. Car2x-based perception in a high-level fusion architecture for cooperative perception systems. In 2012 IEEE Intelligent Vehicles Symposium. IEEE, 270--275.
[58]
Radu Bogdan Rusu, Nico Blodow, and Michael Beetz. 2009. Fast point feature histograms (FPFH) for 3D registration. In 2009 IEEE international conference on robotics and automation. IEEE, 3212--3217.
[59]
Vinit Sarode, Xueqian Li, Hunter Goforth, Yasuhiro Aoki, Rangaprasad Arun Srivatsan, Simon Lucey, and Howie Choset. 2019. PCRNet: Point cloud registration network using PointNet encoding. arXiv preprint arXiv:1908.07906 (2019).
[60]
Mahadev Satyanarayanan. 2017. The Emergence of Edge Computing. Computer 50, 1 (2017), 30--39.
[61]
Shaoshuai Shi, Xiaogang Wang, and Hongsheng Li. 2019. Pointrcnn: 3d object proposal generation and detection from point cloud. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 770--779.
[62]
Weisong Shi, Jie Cao, Quan Zhang, Youhuizi Li, and Lanyu Xu. 2016. Edge Computing: Vision and Challenges. IEEE internet of things journal 3, 5 (2016), 637--646.
[63]
Jiachen Sun, Yulong Cao, Qi Alfred Chen, and Z Morley Mao. 2020. Towards robust lidar-based perception in autonomous driving: General black-box adversarial sensor attack and countermeasures. In 29th {USENIX} Security Symposium ({USENIX} Security 20). 877--894.
[64]
James Tu, Tsunhsuan Wang, Jingkang Wang, Sivabalan Manivasagam, Mengye Ren, and Raquel Urtasun. 2021. Adversarial Attacks On Multi-Agent Communication. arXiv preprint arXiv:2101.06560 (2021).
[65]
Junjue Wang, Ziqiang Feng, Zhuo Chen, Shilpa George, Mihir Bala, Padmanabhan Pillai, Shao-Wen Yang, and Mahadev Satyanarayanan. 2018. Bandwidth-efficient live video analytics for drones via edge computing. In 2018 IEEE/ACM Symposium on Edge Computing (SEC). IEEE, 159--173.
[66]
Tsun-Hsuan Wang, Sivabalan Manivasagam, Ming Liang, Bin Yang, Wenyuan Zeng, and Raquel Urtasun. 2020. V2vnet: Vehicle-to-vehicle communication for joint perception and prediction. In European Conference on Computer Vision. Springer, 605--621.
[67]
IEEE 1609 WG. 2016. IEEE Standard for Wireless Access in Vehicular Environments-Security Services for Applications and Management Messages. IEEE Std 1609.2-2016 (Revision of IEEE Std 1609.2-2013) (2016).
[68]
Pengxiang Wu, Siheng Chen, and Dimitris N Metaxas. 2020. MotionNet: Joint Perception and Motion Prediction for Autonomous Driving Based on Bird's Eye View Maps. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 11385--11395.
[69]
Xiufeng Xie and Kyu-Han Kim. 2019. Source Compression with Bounded DNN Perception Loss for IoT Edge Computer Vision. In The 25th Annual International Conference on Mobile Computing and Networking. 1--16.
[70]
Dongzhu Xu, Anfu Zhou, Xinyu Zhang, Guixian Wang, Xi Liu, Congkai An, Yiming Shi, Liang Liu, and Huadong Ma. 2020. Understanding operational 5g: A first measurement study on its coverage, performance and energy consumption. In Proceedings of the Annual conference of the ACM Special Interest Group on Data Communication on the applications, technologies, architectures, and protocols for computer communication. 479--494.
[71]
Zhigang Xu, Xiaochi Li, Xiangmo Zhao, Michael H Zhang, and Zhongren Wang. 2017. DSRC versus 4G-LTE for connected vehicle applications: A study on field experiments of vehicular communication performance. Journal of Advanced Transportation 2017 (2017).
[72]
Huacheng Zeng, Hossein Pirayesh, Pedram Kheirkhah Sangdeh, and Adnan Quadri. 2021. VehCom: Delay-Guaranteed Message Broadcast for Large-Scale Vehicular Networks. IEEE Transactions on Wireless Communications (2021).
[73]
Yiming Zeng, Yu Hu, Shice Liu, Jing Ye, Yinhe Han, Xiaowei Li, and Ninghui Sun. 2018. Rt3d: Real-time 3-d vehicle detection in lidar point cloud for autonomous driving. IEEE Robotics and Automation Letters 3, 4 (2018), 3434--3440.
[74]
Wenwei Zhang, Hui Zhou, Shuyang Sun, Zhe Wang, Jianping Shi, and Chen Change Loy. 2019. Robust multi-modality multi-object tracking. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 2365--2374.
[75]
Pengyuan Zhou, Tristan Braud, Aleksandr Zavodovski, Zhi Liu, Xianfu Chen, Pan Hui, and Jussi Kangasharju. 2020. Edge-facilitated Augmented Vision in Vehicle-to-Everything Networks. (2020).
[76]
Xiao Zhu, Jiachen Sun, Xumiao Zhang, Y Ethan Guo, Feng Qian, and Z Morley Mao. 2020. MPBond: efficient network-level collaboration among personal mobile devices. In Proceedings of the 18th International Conference on Mobile Systems, Applications, and Services. 364--376.

Cited By

View all
  • (2025)Edge-Assisted Collaborative Perception Against Jamming and Interference in Vehicular NetworksIEEE Transactions on Wireless Communications10.1109/TWC.2024.351060124:1(860-874)Online publication date: Jan-2025
  • (2025)V2I-Coop: Accurate Object Detection for Connected Automated Vehicles at Accident Black Spots With V2I Cross-Modality CooperationIEEE Transactions on Mobile Computing10.1109/TMC.2024.348675824:3(2043-2055)Online publication date: Mar-2025
  • (2024)On data fabrication in collaborative vehicular perceptionProceedings of the 33rd USENIX Conference on Security Symposium10.5555/3698900.3699253(6309-6326)Online publication date: 14-Aug-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
MobiCom '21: Proceedings of the 27th Annual International Conference on Mobile Computing and Networking
October 2021
887 pages
ISBN:9781450383424
DOI:10.1145/3447993
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 October 2021

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. LiDAR
  2. autonomous cars
  3. cooperative sensing
  4. edge computing

Qualifiers

  • Research-article

Funding Sources

Conference

ACM MobiCom '21
Sponsor:

Acceptance Rates

Overall Acceptance Rate 440 of 2,972 submissions, 15%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)904
  • Downloads (Last 6 weeks)107
Reflects downloads up to 13 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Edge-Assisted Collaborative Perception Against Jamming and Interference in Vehicular NetworksIEEE Transactions on Wireless Communications10.1109/TWC.2024.351060124:1(860-874)Online publication date: Jan-2025
  • (2025)V2I-Coop: Accurate Object Detection for Connected Automated Vehicles at Accident Black Spots With V2I Cross-Modality CooperationIEEE Transactions on Mobile Computing10.1109/TMC.2024.348675824:3(2043-2055)Online publication date: Mar-2025
  • (2024)On data fabrication in collaborative vehicular perceptionProceedings of the 33rd USENIX Conference on Security Symposium10.5555/3698900.3699253(6309-6326)Online publication date: 14-Aug-2024
  • (2024)VulcanProceedings of the 21st USENIX Symposium on Networked Systems Design and Implementation10.5555/3691825.3691902(1385-1402)Online publication date: 16-Apr-2024
  • (2024)Cooperative Perception System for Aiding Connected and Automated Vehicle Navigation and Improving SafetyTransportation Research Record: Journal of the Transportation Research Board10.1177/036119812412527792678:12(1498-1510)Online publication date: 18-Jun-2024
  • (2024)Artificial Intelligence of Things: A SurveyACM Transactions on Sensor Networks10.1145/369063921:1(1-75)Online publication date: 30-Aug-2024
  • (2024)Stealthy Data Fabrication in Collaborative Vehicular PerceptionProceedings of the Sixth Workshop on CPS&IoT Security and Privacy10.1145/3690134.3694822(142-149)Online publication date: 19-Nov-2024
  • (2024)Boosting Collaborative Vehicular Perception on the Edge with Vehicle-to-Vehicle CommunicationProceedings of the 22nd ACM Conference on Embedded Networked Sensor Systems10.1145/3666025.3699328(141-154)Online publication date: 4-Nov-2024
  • (2024)VRF: Vehicle Road-side Point Cloud FusionProceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services10.1145/3643832.3661874(547-560)Online publication date: 3-Jun-2024
  • (2024)RECAP: 3D Traffic ReconstructionProceedings of the 30th Annual International Conference on Mobile Computing and Networking10.1145/3636534.3690691(1252-1267)Online publication date: 4-Dec-2024
  • Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media