Sri Hrushikesh Varma Bhupathiraju
ABSTRACT
Autonomous Vehicles (AVs) using LiDAR-based object detection systems are rapidly improving and becoming an increasingly viable method of transportation. While effective at perceiving the surrounding environment, these detection systems are shown to be vulnerable to attacks using lasers which can cause obstacle misclassifications or removal. These laser attacks, however, are challenging to perform, requiring precise aiming and accuracy. Our research exposes a new threat in the form of Intentional Electro-Magnetic-Interference (IEMI), which affects the time-of-flight (TOF) circuits that make up modern LiDARs. We show that these vulnerabilities can be exploited to force the AV Perception system to misdetect, misclassify objects, and perceive non-existent obstacles. We evaluate the vulnerability in three AV perception modules (PointPillars, PointRCNN, and Apollo) and show how the classification rate drops below 50%. We also analyze the impact of the IEMI injection on two fusion models (AVOD and Frustum-ConvNet) and in real-world scenarios. Finally, we discuss potential countermeasures and propose two strategies to detect signal injection.
Supplemental Material
- 2017. KITTI Vision Benchmark: 3D Object Detection. http://www.cvlibs.net/dat asets/kitti/eval_object.php?obj_benchmark=3d. Accessed: 2021-08--17.Google Scholar
- 2019. FCC - Radio Frequency Safety. https://www.fcc.gov/general/radiofrequency- safety-0.Google Scholar
- 2022. Navigant Research Names Waymo, Ford Autonomous Vehicles, Cruise, and Baidu the Leading Developers of Automated Driving Systems. https://ww w.businesswire.com/news/home/20200407005119/en/Navigant-Research- Names-Waymo-Ford-Autonomous-Vehicles-Cruise-and-Baidu-the-Leading- Developers-of-Automated-Driving-Systems. Accessed: 2023-01-08.Google Scholar
- Mazen Abdelfattah, Kaiwen Yuan, Z. Jane Wang, and Rabab Ward. 2021. Adversarial Attacks on Camera-LiDAR Models for 3D Car Detection. In IROS 2021.Google ScholarDigital Library
- Ron Amadeo. 2017. Google's Waymo invests in LIDAR technology, cuts costs by 90 percent. Ars Technica, Jan 10 (2017).Google Scholar
- Baidu Inc. 2017. Apollo. http://apollo.auto. Accessed: 2021--10-08.Google Scholar
- Benewake. 2018. CE30-A Solid State Array LiDAR Specification. http://statics3 .seeedstudio.com/assets/file/bazaar/product/DE-LiDAR-CE30-A-Datasheet- V010-EN.pdf.Google Scholar
- Yulong Cao, S. Hrushikesh Bhupathiraju, Pirouz Naghavi, Takeshi Sugawara, Z. Morley Mao, and Sara Rampazzi. 2023. You Can't See Me: Physical Removal Attacks on LiDAR-based Autonomous Vehicles Driving Frameworks. In USENIX Security 23.Google Scholar
- Yulong Cao, Jiaxiang Ma, Kevin Fu, Sara Rampazzi, and Morley Mao. 2021. Automated Tracking System For LiDAR Spoofing Attacks On Moving Targets. In AutoSec Workshop, 2021.Google Scholar
- Y. Cao, N. Wang, C. Xiao, D. Yang, J. Fang, R. Yang, Q. Chen, M. Liu, and B. Li. 2021. Invisible for both Camera and LiDAR: Security of Multi-Sensor Fusion based Perception in Autonomous Driving Under Physical-World Attacks. In IEEE S&P 2021.Google Scholar
- Yulong Cao, Chaowei Xiao, Benjamin Cyr, Yimeng Zhou, Won Park, Sara Rampazzi, Qi Alfred Chen, Kevin Fu, and Zhuoqing Morley Mao. 2019. Adversarial Sensor Attack on LiDAR-based Perception in Autonomous Driving. In CCS '19.Google Scholar
- Yulong Cao, Chaowei Xiao, Dawei Yang, Jing Fang, Ruigang Yang, Mingyan Liu, and Bo Li. 2019. Adversarial objects against LiDAR-based autonomous driving systems. arXiv:1907.05418 (2019).Google Scholar
- DDL Chung. 2000. Materials for electromagnetic interference shielding. Journal of Materials Engineering and performance 9 (2000), 350--354.Google ScholarCross Ref
- Cygbot. 2019. 2D 3D Dual Solid State TOF LiDAR. https://www.cygbot.com/2d- 3d-dual-solid-state-tof-lidar.Google Scholar
- F. Fiori and P.S. Crovetti. 2002. Nonlinear effects of radio-frequency interference in operational amplifiers. IEEE Trans. Circuits and Systems I 49, 3 (2002), 367--372. https://doi.org/10.1109/81.989173Google Scholar
- Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. 2014. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 (2014).Google Scholar
- David S Hall. 2020. High definition lidar system. US Patent App. 15/700,543.Google Scholar
- David S Hall, Pieter J Kerstens, Yupeng Cui, Mathew Noel Rekow, and Stephen S Nestinger. 2018. LIDAR based 3-D imaging with varying pulse repetition. US Patent 10,048,374.Google Scholar
- R Spencer Hallyburton, Yupei Liu, Yulong Cao, Z Morley Mao, and Miroslav Pajic. 2022. Security analysis of camera-LiDAR fusion against black-box attacks on autonomous vehicles. In USENIX Security 22.Google Scholar
- Hamamatsu Photonics K.K. 2019. APD Modules C12703 series. Hamamatsu Photonics K.K.Google Scholar
- Abdullah Hamdi, Sara Rojas, Ali Thabet, and Bernard Ghanem. 2020. AdvPC: Transferable adversarial perturbations on 3D point clouds. In ECCV 2020.Google ScholarDigital Library
- Mark Harris. 2017. GM Cruise Snaps Up Solid-State Lidar Pioneer Strobe Inc. IEEE Spectrum 11 (2017).Google Scholar
- Zhongyuan Hau, T Kenneth, Soteris Demetriou, and Emil C Lupu. 2021. Object Removal Attacks on LiDAR-based 3D Object Detectors. In Workshop on Automotive and Autonomous Vehicle Security (AutoSec), Vol. 2021. 25.Google ScholarCross Ref
- Yuhang He, Wentao Yu, Jie Han, Xing Wei, Xiaopeng Hong, and Yihong Gong. 2021. Know your surroundings: Panoramic multi-object tracking by multimodality collaboration. In CVPR 2021. 2969--2980.Google ScholarCross Ref
- Chao Huang, Ruihui Li, Xianzhi Li, and Chi-Wing Fu. 2020. Non-local part-aware point cloud denoising. arXiv preprint arXiv:2003.06631 (2020).Google Scholar
- Velodyne LiDAR Inc. 2019. USER'S MANUAL AND PROGRAMMING GUIDE: VLP-16 Velodyne LiDAR Puck Rev E. https://velodynelidar.com/wp-content/up loads/2019/12/63--9243-Rev-E-VLP-16-User-Manual.pdf.Google Scholar
- J.Schesser. 2017. Sampling and Aliasing. https://web.njit.edu/~joelsd/Fundamen tals/coursework/BME310computingcw6.pdf.Google Scholar
- Chaouki Kasmi and Jose Lopes Esteves. 2015. IEMI threats for information security: Remote command injection on modern smartphones. IEEE Trans. Electromagnetic Compatibility 57, 6 (2015), 1752--1755.Google ScholarCross Ref
- Mandeep Kaur, Shikha Kakar, and Danvir Mandal. 2011. Electromagnetic interference. In 2011 3rd International Conference on Electronics Computer Technology.Google ScholarCross Ref
- Jason Ku, Melissa Mozifian, Jungwook Lee, Ali Harakeh, and Steven LWaslander. 2018. Joint 3d proposal generation and object detection from view aggregation. In IROS 2018. IEEE.Google ScholarDigital Library
- Denis Foo Kune, John Backes, Shane S Clark, Daniel Kramer, Matthew Reynolds, Kevin Fu, Yongdae Kim, and Wenyuan Xu. 2013. Ghost Talk: Mitigating EMI signal injection attacks against analog sensors. In IEEE S&P 2013. 145--159.Google Scholar
- Alex H Lang, Sourabh Vora, Holger Caesar, Lubing Zhou, Jiong Yang, and Oscar Beijbom. 2019. Pointpillars: Fast Encoders for Object Detection from Point Clouds. In CVPR 2019.Google ScholarCross Ref
- Timothy B. Lee. 2018. Why Spinning Lidar Sensors Might be Around for Another Decade. Ars Technica.Google Scholar
- Xingyi Li, Wenxuan Wu, Xiaoli Z Fern, and Li Fuxin. 2021. The devils in the point clouds: Studying the robustness of point cloud convolutions. arXiv preprint arXiv:2101.07832 (2021).Google Scholar
- Shitong Luo andWei Hu. 2021. Score-based point cloud denoising. In Proceedings of the IEEE/CVF International Conference on Computer Vision.Google ScholarCross Ref
- Roberto Martin-Martin, Mihir Patel, Hamid Rezatofighi, Abhijeet Shenoi, Jun- Young Gwak, Eric Frankel, Amir Sadeghian, and Silvio Savarese. 2021. Jrdb: A dataset and benchmark of egocentric robot visual perception of humans in built environments. IEEE transactions on pattern analysis and machine intelligence (2021).Google Scholar
- Mini-Circuits. 2019. High Power Amplifier ZHL-5W-202-S. https://www.minici rcuits.com/pdfs/ZHL-5W-202-S.pdf.Google Scholar
- Ehsan Nezhadarya, Ehsan Taghavi, Ryan Razani, Bingbing Liu, and Jun Luo. 2020. Adaptive hierarchical down-sampling for point cloud classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Google ScholarCross Ref
- Seong Joon Oh, Bernt Schiele, and Mario Fritz. 2019. Towards reverse-engineering black-box neural networks. Explainable AI: Interpreting, Explaining and Visualizing Deep Learning (2019), 121--144.Google Scholar
- Soham Pal, Yash Gupta, Aditya Shukla, Aditya Kanade, Shirish Shevade, and Vinod Ganapathy. 2019. A framework for the extraction of deep neural networks by leveraging public data. arXiv preprint arXiv:1905.09165 (2019).Google Scholar
- Youngseok Park, Yunmok Son, Hocheol Shin, Dohyun Kim, and Yongdae Kim. 2016. This ain't your dose: Sensor spoofing attack on medical infusion pump. In WOOT 16.Google Scholar
- Scott Drew Pendleton, Hans Andersen, Xinxin Du, Xiaotong Shen, Malika Meghjani, You Hong Eng, Daniela Rus, and Marcelo H Ang Jr. 2017. Perception, planning, control, and coordination for autonomous vehicles. Machines 5, 1 (2017), 6.Google ScholarCross Ref
- Francesca Pistilli, Giulia Fracastoro, Diego Valsesia, and Enrico Magli. 2020. Learning graph-convolutional representations for point cloud denoising. In Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part XX 16. Springer, 103--118.Google Scholar
- Milica Popovic, BD Popovic, and Z Popovic. 2006. Electromagnetic induction. Fundamentals of Engineering Electromagnetics (2006).Google Scholar
- Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. 2017. Pointnet: Deep learning on point sets for 3D classification and segmentation. In CVPR 2017.Google Scholar
- Rui Qian, Xin Lai, and Xirong Li. 2022. 3d object detection for autonomous driving: a survey. Pattern Recognition 130 (2022), 108796.Google ScholarDigital Library
- Ettus Research. 2012. USRP N200/N210 Networked Series. https://www.ettus.co m/wp-content/uploads/2019/01/07495_Ettus_N200--210_DS_Flyer_HR_1.pdf.Google Scholar
- Ferran Reverter, Xiujun Li, and Gerard C M Meijer. 2006. Stability and accuracy of active shielding for grounded capacitive sensors. Measurement Science and Technology 17, 11 (sep 2006), 2884. https://doi.org/10.1088/0957-0233/17/11/004Google ScholarCross Ref
- JM Roe. 1978. Integrated Circuit Electromagnetic Susceptibility Handbook. Phase III.Google Scholar
- Sowmya Sankaran, Kalim Deshmukh, M. Basheer Ahamed, and S.K. Khadheer Pasha. 2018. Recent advances in electromagnetic interference shielding properties of metal and carbon filler reinforced flexible polymer composites: A review. Composites Part A: Applied Science and Manufacturing (2018). https://doi.org/10 .1016/j.compositesa.2018.08.006Google Scholar
- Shanghai Slamtec Co., Ltd. 2019. RPLiDAR S2. https://www.slamtec.com/en/S2.Google Scholar
- Shaoshuai Shi, Xiaogang Wang, and Hongsheng Li. 2019. PointRCNN: 3D object proposal generation and detection from point cloud. In CVPR 2019.Google ScholarCross Ref
- Giuseppe Schirripa Spagnolo, Adriana Postiglione, and Ilaria De Angelis. 2020. Simple equipment for teaching internal photoelectric effect. Physics Education (2020).Google Scholar
- Enrique Mario Spinelli and Ferran Reverter. 2009. On the stability of shield-driver circuits. IEEE Transactions on Instrumentation and Measurement (2009).Google Scholar
- Stanford Vision and Learning Lab. 2017. JackRabbot. https://svl.stanford.edu/pro jects/jackrabbot/.Google Scholar
- Takeshi Sugawara, Benjamin Cyr, Sara Rampazzi, Daniel Genkin, and Kevin Fu. 2020. Light Commands: Laser-Based Audio Injection Attacks on Voice- Controllable Systems. In 29th USENIX Security Symposium (USENIX Security 20). USENIX Association, 2631--2648. https://www.usenix.org/conference/usenixse curity20/presentation/sugawaraGoogle Scholar
- Jiachen Sun, Yulong Cao, Qi Alfred Chen, and Z Morley Mao. 2020. Towards robust LiDAR-based perception in autonomous driving: General black-box adversarial sensor attack and countermeasures. In USENIX Security 20. 877--894.Google Scholar
- Christian Szegedy,Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2013. Intriguing properties of neural networks. arXiv:1312.6199 (2013).Google Scholar
- James Tu, Mengye Ren, Sivabalan Manivasagam, Ming Liang, Bin Yang, Richard Du, Frank Cheng, and Raquel Urtasun. 2020. Physically Realizable Adversarial Examples for LiDAR Object Detection. In CVPR 2020.Google Scholar
- Yazhou Tu, Sara Rampazzi, Bin Hao, Angel Rodriguez, Kevin Fu, and Xiali Hei. 2019. Trick or heat? Manipulating critical temperature-based control systems using rectification attacks. In CCS '19. 2301--2315.Google ScholarDigital Library
- Yazhou Tu, Vijay Srinivas Tida, Zhongqi Pan, and Xiali Hei. 2021. Transduction Shield: A Low-Complexity Method to Detect and Correct the Effects of EMI Injection Attacks on Sensors (ASIA CCS '21). Association for Computing Machinery.Google Scholar
- ZhixinWang and Kui Jia. 2019. Frustum ConvNet: Sliding Frustums to Aggregate Local Point-Wise Features for Amodal 3D Object Detection. IROS 2019 (2019).Google Scholar
- David AWare. 2017. Effects of intentional electromagnetic interference on analog to digital converter measurements of sensor outputs and general purpose input output pins. Ph.D. Dissertation. Utah State University.Google Scholar
- Yuxin Wen, Jiehong Lin, Ke Chen, CL Philip Chen, and Kui Jia. 2020. Geometryaware generation of adversarial point clouds. IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 6 (2020), 2984--2999.Google ScholarCross Ref
- Chong Xiang, Charles R Qi, and Bo Li. 2019. Generating 3D adversarial point clouds. In CVPR 2019. 9136--9144.Google ScholarCross Ref
- Zhifei Xu, Runbing Hua, Jack Juang, Shengxuan Xia, Jun Fan, and Chulsoon Hwang. 2021. Inaudible Attack on Smart Speakers With Intentional Electromagnetic Interference. IEEE Transactions on Microwave Theory and Techniques (2021).Google ScholarCross Ref
- Bin Yang, Wenjie Luo, and Raquel Urtasun. 2018. Pixor: Real-time 3D object detection from point clouds. In CVPR 2018. 7652--7660.Google ScholarCross Ref
- Jiancheng Yang, Qiang Zhang, Rongyao Fang, Bingbing Ni, Jinxian Liu, and Qi Tian. 2019. Adversarial attack and defense on point sets. arXiv preprint arXiv:1902.10899 (2019).Google Scholar
- Zetong Yang, Yanan Sun, Shu Liu, and Jiaya Jia. 2020. 3dssd: Point-based 3d single stage object detector. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 11040--11048.Google ScholarCross Ref
- Kentaro Yoshioka. 2022. A Tutorial and Review of Automobile Direct ToF LiDAR SoCs: Evolution of Next-Generation LiDARs. IEICE Transactions on Electronics (2022).Google Scholar
- Guoming Zhang, Chen Yan, Xiaoyu Ji, Tianchen Zhang, Taimin Zhang, and Wenyuan Xu. 2017. Dolphinattack: Inaudible voice commands. In CCS '17.Google ScholarDigital Library
- Min Zhang. 2021. A CAPACITIVELY COUPLED PIN INJECTION METHOD -- AN ALTERNATIVE TO BCI TEST.Google Scholar
- Qifan Zhang, Junjie Shen, Mingtian Tan, Zhe Zhou, Zhou Li, Qi Alfred Chen, and Haipeng Zhang. 2022. Play the Imitation Game: Model Extraction Attack against Autonomous Driving Localization. In Proceedings of the 38th Annual Computer Security Applications Conference. 56--70.Google ScholarDigital Library
- Qian-Yi Zhou, Jaesik Park, and Vladlen Koltun. 2018. Open3D: A Modern Library for 3D Data Processing. arXiv:1801.09847 (2018).Google Scholar
- Yin Zhou and Oncel Tuzel. 2018. Voxelnet: End-to-end learning for point cloud based 3D object detection. In CVPR 2018. 4490--4499.Google ScholarCross Ref
- Yi Zhu, Chenglin Miao, Tianhang Zheng, Foad Hajiaghajani, Lu Su, and Chunming Qiao. 2021. Can We Use Arbitrary Objects to Attack LiDAR Perception in Autonomous Driving?. In CCS '21.Google ScholarDigital Library
Index Terms
- EMI-LiDAR: Uncovering Vulnerabilities of LiDAR Sensors in Autonomous Driving Setting using Electromagnetic Interference
Recommendations
Towards Backdoor Attacks against LiDAR Object Detection in Autonomous Driving
SenSys '22: Proceedings of the 20th ACM Conference on Embedded Networked Sensor SystemsDue to the great advantage of LiDAR sensors in perceiving complex driving environments, LiDAR-based 3D object detection has recently drawn significant attention in autonomous driving. Although many advanced LiDAR object detection models have been ...
TileMask: A Passive-Reflection-based Attack against mmWave Radar Object Detection in Autonomous Driving
CCS '23: Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications SecurityIn autonomous driving, millimeter wave (mmWave) radar has been widely adopted for object detection because of its robustness and reliability under various weather and lighting conditions. For radar object detection, deep neural networks (DNNs) are ...
Adversarial Attacks against LiDAR Semantic Segmentation in Autonomous Driving
SenSys '21: Proceedings of the 19th ACM Conference on Embedded Networked Sensor SystemsToday, most autonomous vehicles (AVs) rely on LiDAR (Light Detection and Ranging) perception to acquire accurate information about their immediate surroundings. In LiDAR-based perception systems, semantic segmentation plays a critical role as it can ...
Comments