research-article

Multi-view Correlation based Black-box Adversarial Attack for 3D Object Detection

Authors:

Weihong DengAuthors Info & Claims

KDD '21: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining

Pages 1036 - 1044

https://doi.org/10.1145/3447548.3467432

Published: 14 August 2021 Publication History

Abstract

Deep neural networks have made tremendous progress in 3D object detection, which is an important task especially in autonomous driving scenarios. Benefited from the breakthroughs in deep learning and sensor technologies, 3D object detection methods based on different sensors, such as camera and LiDAR, have developed rapidly. Meanwhile, more and more researches notice that the abundant information contained in the multi-view data can be used to obtain more accurate understanding of the 3D surrounding environment. Therefore, many sensor-fusion 3D object detection methods have been proposed. As safety is critical in autonomous driving and the deep neural networks are known to be vulnerable to adversarial examples with visually imperceptible perturbations, it is significant to investigate adversarial attacks for 3D object detection. Recent works have shown that both image-based and LiDAR-based networks can be attacked by the adversarial examples while the attacks to the sensor-fusion models, which tend to be more robust, haven't been studied. To this end, we propose a simple multi-view correlation based adversarial attack method for the camera-LiDAR fusion 3D object detection models and focus on the black-box attack setting which is more practical in real-world systems. Specifically, we first design a generative network to generate image adversarial examples based on an auxiliary image semantic segmentation network. Then, we develop a cross-view perturbation projection method by exploiting the camera-LiDAR correlations to map each image adversarial example to the space of the point cloud data to form the point cloud adversarial examples in the LiDAR view. Extensive experiments on the KITTI dataset demonstrate the effectiveness of the proposed method.

References

[1]

Hervé Abdi and Lynne J Williams. 2010. Principal component analysis. Wiley Interdisciplinary Reviews: Computational Statistics, Vol. 2, 4 (2010), 433--459.

Digital Library

[2]

Shotaro Akaho. 2006. A kernel method for canonical correlation analysis. arXiv preprint cs/0609071 (2006).

[3]

Francis R Bach, Gert RG Lanckriet, and Michael I Jordan. 2004. Multiple kernel learning, conic duality, and the SMO algorithm. In ICML.

[4]

Avrim Blum and Tom Mitchell. 1998. Combining labeled and unlabeled data with co-training. In COLT.

[5]

Nicholas Carlini and David Wagner. 2017. Towards evaluating the robustness of neural networks. In IEEE Symposium on Security and Privacy (SP).

[6]

Dongdong Chen, Jing Liao, Lu Yuan, Nenghai Yu, and Gang Hua. 2017b. Coherent online video style transfer. In ICCV.

[7]

Pin-Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsieh. 2017 d. Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models. In Proc. of the 10th ACM Workshop on Artificial Intelligence and Security.

Digital Library

[8]

Xiaozhi Chen, Kaustav Kundu, Ziyu Zhang, Huimin Ma, Sanja Fidler, and Raquel Urtasun. 2016. Monocular 3d object detection for autonomous driving. In CVPR.

[9]

Xiaozhi Chen, Kaustav Kundu, Yukun Zhu, Huimin Ma, Sanja Fidler, and Raquel Urtasun. 2017a. 3d object proposals using stereo imagery for accurate object class detection. IEEE TPAMI, Vol. 40, 5 (2017), 1259--1272.

[10]

Xiaozhi Chen, Huimin Ma, Ji Wan, Bo Li, and Tian Xia. 2017c. Multi-view 3d object detection network for autonomous driving. In CVPR.

[11]

Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. 2016. The cityscapes dataset for semantic urban scene understanding. In CVPR.

[12]

Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. 2010. The pascal visual object classes (voc) challenge. IJCV, Vol. 88, 2 (2010), 303--338.

Digital Library

[13]

Andreas Geiger, Philip Lenz, and Raquel Urtasun. 2012. Are we ready for Autonomous Driving? The KITTI Vision Benchmark Suite. In CVPR.

[14]

Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. 2014. Rich feature hierarchies for accurate object detection and semantic segmentation. In CVPR.

[15]

Mehmet Gönen and Ethem Alpaydin. 2011. Multiple kernel learning algorithms. JMLR, Vol. 12 (2011), 2211--2268.

Digital Library

[16]

Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and harnessing adversarial examples. In ICLR.

[17]

Chuan Guo, Jacob Gardner, Yurong You, Andrew Gordon Wilson, and Kilian Weinberger. 2019. Simple black-box adversarial attacks. In ICML.

[18]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In CVPR.

[19]

Arthur E Hoerl and Robert W Kennard. 1970. Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, Vol. 12, 1 (1970), 55--67.

[20]

Harold Hotelling. 1936. Relations between two sets of variates. Biometrika, Vol. 28 (1936), 321--377.

[21]

Tengteng Huang, Zhe Liu, Xiwu Chen, and Xiang Bai. 2020. EPNet: Enhancing point features with image semantics for 3d object detection. In ECCV.

[22]

Diederik P Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In ICLR.

[23]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In NeurIPS.

[24]

Shan Li and Weihong Deng. 2020. Deep facial expression recognition: A survey. IEEE Transactions on Affective Computing (2020).

[25]

Jonathan Long, Evan Shelhamer, and Trevor Darrell. 2015. Fully convolutional networks for semantic segmentation. In CVPR.

[26]

Mieczysław Mastyło. 2013. Bilinear interpolation theorems and applications. Journal of Functional Analysis, Vol. 265, 2 (2013), 185--207.

[27]

Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard. 2016. Deepfool: a simple and accurate method to fool deep neural networks. In CVPR.

[28]

Kamal Nigam and Rayid Ghani. 2000. Analyzing the effectiveness and applicability of co-training. In CIKM.

[29]

Marin Orsic, Ivan Kreso, Petra Bevandic, and Sinisa Segvic. 2019. In defense of pre-trained imagenet architectures for real-time semantic segmentation of road-driving images. In CVPR.

[30]

Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z Berkay Celik, and Ananthram Swami. 2017. Practical black-box attacks against machine learning. In ACM ASIACCS.

[31]

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et almbox. 2019. Pytorch: An imperative style, high-performance deep learning library. In NeurIPS.

[32]

Charles R Qi, Wei Liu, Chenxia Wu, Hao Su, and Leonidas J Guibas. 2018. Frustum pointnets for 3d object detection from rgb-d data. In CVPR.

[33]

Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster r-cnn: Towards real-time object detection with region proposal networks. In NeurIPS.

[34]

Shaoshuai Shi, Xiaogang Wang, and Hongsheng Li. 2019. Pointrcnn: 3d object proposal generation and detection from point cloud. In CVPR.

[35]

Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2015. Going deeper with convolutions. In CVPR.

[36]

Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2014. Intriguing properties of neural networks. In ICLR.

[37]

James Tu, Mengye Ren, Sivabalan Manivasagam, Ming Liang, Bin Yang, Richard Du, Frank Cheng, and Raquel Urtasun. 2020. Physically Realizable Adversarial Examples for LiDAR Object Detection. In CVPR.

[38]

Sourabh Vora, Alex H Lang, Bassam Helou, and Oscar Beijbom. 2020. Pointpainting: Sequential fusion for 3d object detection. In CVPR.

[39]

Mei Wang and Weihong Deng. 2021. Deep face recognition: A survey. Neurocomputing, Vol. 429 (2021), 215--244.

[40]

Xingxing Wei, Siyuan Liang, Ning Chen, and Xiaochun Cao. 2019. Transferable adversarial attacks for image and video object detection. In IJCAI.

[41]

Chong Xiang, Charles R Qi, and Bo Li. 2019. Generating 3d adversarial point clouds. In CVPR.

[42]

Chaowei Xiao, Bo Li, Jun-Yan Zhu, Warren He, Mingyan Liu, and Dawn Song. 2018. Generating adversarial examples with adversarial networks. In IJCAI.

[43]

Cihang Xie, Jianyu Wang, Zhishuai Zhang, Yuyin Zhou, Lingxi Xie, and Alan Yuille. 2017. Adversarial examples for semantic segmentation and object detection. In CVPR.

[44]

Bin Xu and Zhenzhong Chen. 2018. Multi-level fusion based 3d object detection from monocular images. In CVPR.

[45]

Bin Yang, Wenjie Luo, and Raquel Urtasun. 2018. Pixor: Real-time 3d object detection from point clouds. In CVPR.

[46]

Hang Zhou, Dongdong Chen, Jing Liao, Kejiang Chen, Xiaoyi Dong, Kunlin Liu, Weiming Zhang, Gang Hua, and Nenghai Yu. 2020. LG-GAN: Label Guided Adversarial Network for Flexible Targeted Attack of Point Cloud Based Deep Networks. In CVPR.

[47]

Yin Zhou and Oncel Tuzel. 2018. Voxelnet: End-to-end learning for point cloud based 3d object detection. In CVPR.

Cited By

Wu JYao WJia SJiang TZhou WMa CChen X(2025)Gradient-based sparse voxel attacks on point cloud object detectionPattern Recognition10.1016/j.patcog.2024.111156160(111156)Online publication date: Apr-2025
https://doi.org/10.1016/j.patcog.2024.111156
Yasas Mahima KPerera AAnavatti SGarratt M(2024)Toward Robust 3D Perception for Autonomous Vehicles: A Review of Adversarial Attacks and CountermeasuresIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.345629325:12(19176-19202)Online publication date: Dec-2024
https://doi.org/10.1109/TITS.2024.3456293
Chen HYan HYang XSu HZhao SQian F(2024)Efficient Adversarial Attack Strategy Against 3D Object Detection in Autonomous Driving SystemsIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.341003825:11(16118-16132)Online publication date: Nov-2024
https://doi.org/10.1109/TITS.2024.3410038
Show More Cited By

Index Terms

Multi-view Correlation based Black-box Adversarial Attack for 3D Object Detection
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Object detection
2. Security and privacy

Recommendations

Unrestricted Black-Box Adversarial Attack Using GAN with Limited Queries
Computer Vision – ECCV 2022 Workshops
Abstract
Adversarial examples are inputs intentionally generated for fooling a deep neural network. Recent studies have proposed unrestricted adversarial attacks that are not norm-constrained. However, the previous unrestricted attack methods still have ...
Black-box adversarial attacks on XSS attack detection model
Abstract
Cross-site scripting (XSS) has been extensively studied, although mitigating such attacks in web applications remains challenging. While there is an increasing number of XSS attack detection approaches designed based on machine learning and deep ...
Singular Value Manipulating: An Effective DRL-Based Adversarial Attack on Deep Convolutional Neural Network
Abstract
In recent years, deep convolutional neural networks (DCNNs) have become increasingly prevalent in image processing applications. However, DCNNs are vulnerable to adversarial attacks, which are generated by adding imperceptible perturbations to the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

KDD '21: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining

August 2021

4259 pages

ISBN:9781450383325

DOI:10.1145/3447548

General Chairs:
Feida Zhu
Singapore Management University
,
Beng Chin Ooi
National University of Singapore
,
Chunyan Miao
Nanyang Technology University
,
Program Chairs:
Haixun Wang,
Iryna Skrypnyk,
Wynne Hsu,
Sanjay Chawla

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 August 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

KDD '21

Sponsor:

KDD '21: The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 14 - 18, 2021

Virtual Event, Singapore

Acceptance Rates

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Upcoming Conference

KDD '25

Sponsor:
sigkdd
sigkdd

The 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 3 - 7, 2025

Toronto , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
507
Total Downloads

Downloads (Last 12 months)87
Downloads (Last 6 weeks)7

Reflects downloads up to 08 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wu JYao WJia SJiang TZhou WMa CChen X(2025)Gradient-based sparse voxel attacks on point cloud object detectionPattern Recognition10.1016/j.patcog.2024.111156160(111156)Online publication date: Apr-2025
https://doi.org/10.1016/j.patcog.2024.111156
Yasas Mahima KPerera AAnavatti SGarratt M(2024)Toward Robust 3D Perception for Autonomous Vehicles: A Review of Adversarial Attacks and CountermeasuresIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.345629325:12(19176-19202)Online publication date: Dec-2024
https://doi.org/10.1109/TITS.2024.3456293
Chen HYan HYang XSu HZhao SQian F(2024)Efficient Adversarial Attack Strategy Against 3D Object Detection in Autonomous Driving SystemsIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.341003825:11(16118-16132)Online publication date: Nov-2024
https://doi.org/10.1109/TITS.2024.3410038
Huang QGu CWang YHu D(2024)SpotAttack: Covering Spots on Surface to Attack LiDAR-Based Autonomous Driving SystemsIEEE Internet of Things Journal10.1109/JIOT.2024.345269411:24(40634-40644)Online publication date: 15-Dec-2024
https://doi.org/10.1109/JIOT.2024.3452694
Wang YSun TLi SYuan XNi WHossain EVincent Poor H(2023)Adversarial Attacks and Defenses in Machine Learning-Empowered Communication Systems and Networks: A Contemporary SurveyIEEE Communications Surveys & Tutorials10.1109/COMST.2023.331949225:4(2245-2298)Online publication date: 26-Sep-2023
https://dl.acm.org/doi/10.1109/COMST.2023.3319492

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten