skip to main content
research-article

DT-DS: CAN Intrusion Detection with Decision Tree Ensembles

Published: 22 March 2023 Publication History

Abstract

The controller area network (CAN) protocol, used in many modern vehicles for real-time inter-device communications, is known to have cybersecurity vulnerabilities, putting passengers at risk for data exfiltration and control system sabotage. To address this issue, researchers have proposed to utilize security measures based on cryptography and message authentication; unfortunately, such approaches are often too computationally expensive to be deployed in real time on CAN devices. Additionally, they have developed machine learning (ML) techniques to detect anomalies in CAN traffic and thereby prevent attacks. The main disadvantage of existing ML-based techniques is that they either depend on additional computational hardware or they heuristically assume that all communication anomalies are malicious.
In this article, we show that tree-based learning ensembles outperform anomaly-based techniques like AutoRegressive Integrated Moving Average (ARIMA) and Z-Score when used to detect attacks that result in increased bus utilization. We evaluated the detection capacity of three tree-based ensembles, Adaboost, gradient boosting, and random forests, and collectively refer to these as DT-DS. We conclude that the decision tree ensemble with Adaboost performs best with an area under curve (AUC) score of 0.999, closely followed by gradient boosting and random forests with 0.997 and 0.991 AUC scores, respectively, when trained using message profiles. We observe that with an increase in the observation window, the DT-DS models present an average AUC score of 0.999, and offer a nearly perfect detection of attacks, at the cost of increased latency in detection of attacked messages. We evaluate the performance of the IDS for Aeronautical Radio, Incorporated– (ARINC) encoded CAN communication traffic in avionic systems, generated using an aerospace testbench, ARINC-825TBv2. The IDS has been evaluated against the active attacks of a state-of-the-art predictive attacker model. Additionally, we observed that the performance of IDS approaches such as ARIMA and Z-Score degrade considerably with a decrease in the size of the observation time window. In contrast, the performance of DT-DS models is consistent, with only an average drop of 0.005 in the AUC score.

References

[1]
ARINC Specification 825. Standard. ARINC.
[2]
FlightGear Flight Simulator. Retrieved from https://www.flightgear.org/about/.
[3]
2018. White paper using CAN bus serial communications in space flight applications.
[4]
Adeeb Alhomoud, Rashid Munir, Jules Pagna Disso, Irfan Awan, and A. Al-Dhelaan. 2011. Performance evaluation study of intrusion detection systems. Proc. Comput. Sci. 5 (2011), 173–180. DOI:
[5]
M. R. Ansari, S. Yu, and Q. Yu. 2015. IntelliCAN: Attack-resilient controller area network (CAN) for secure automobiles. In Proceedings of the IEEE International Symposium on Defect and Fault Tolerance in VLSI and Nanotechnology Systems (DFTS’15). 233–236. DOI:
[6]
O. Avatefipour, A. Hafeez, M. Tayyab, and H. Malik. 2017. Linking received packet to the transmitter through physical-fingerprinting of controller area network. In Proceedings of the IEEE Workshop on Information Forensics and Security (WIFS’17). 1–6. DOI:
[7]
Aymen Boudguiga, Witold Klaudel, Antoine Boulanger, and Pascal Chiron. 2016. A simple intrusion detection method for controller area network. In Proceedings of the IEEE International Conference on Communications (ICC’16). 1–7. DOI:
[8]
Mehmet Bozdal, Mohammad Samie, Sohaib Aslam, and I.K. Jennions. 2020. Evaluation of CAN bus security challenges. Sensors 20 (042020), 16–17. DOI:
[9]
Robert Buttigieg, Mario Farrugia, and Clyde Meli. 2017. Security issues in controller area networks in automobiles. In Proceedings of the 18th International Conference on Sciences and Techniques of Automatic Control and Computer Engineering (STA). 93–98. DOI:
[10]
Ashwin Chandwani, Saikat Dey, and Ayan Mallik. 2020. Cybersecurity of onboard charging systems for electric vehicles—Review, challenges and countermeasures. IEEE Access (122020), 1–1. DOI:
[11]
Stephen Checkoway, Damon McCoy, Brian Kantor, Danny Anderson, Hovav Shacham, Stefan Savage, Karl Koscher, Alexei Czeskis, Franziska Roesner, and Tadayoshi Kohno. 2011. Comprehensive experimental analyses of automotive attack surfaces. In Proceedings of the 20th USENIX Conference on Security (SEC’11). USENIX Association, Berkeley, CA, 6.
[12]
Weiwei Chen, Fangang Kong, Feng Mei, Guiqin Yuan, and Bo Li. 2017. A novel unsupervised anomaly detection approach for intrusion detection system. In Proceedings of the IEEE 3rd International Conference on Big Data Security on Cloud (Bigdatasecurity’17), IEEE International Conference on High Performance and Smart Computing (HPSC’17), and IEEE International Conference on Intelligent Data and Security (IDS’17). 69–73. DOI:
[13]
Lilly Cheng. 2019. Basic Ensemble Learning (Random Forest, AdaBoost, Gradient Boosting): Step by Step Explained. Retrieved from https://towardsdatascience.com/basic-ensemble-learning-random-forest-adaboost-gradient-boosting-step-by-step-explained-95d49d1e2725.
[14]
Kyong-Tak Cho and Kang G. Shin. 2016. Error handling of in-vehicle networks makes them vulnerable. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (CCS’16). Association for Computing Machinery, New York, NY, 1044–1055. DOI:
[15]
W. Choi, K. Joo, H. J. Jo, M. C. Park, and D. H. Lee. 2018. VoltageIDS: Low-level communication characteristics for automotive intrusion detection system. IEEE Trans. Inf. Forens. Secur. 13, 8 (2018), 2114–2129. DOI:
[16]
Miro Enev, Alex Takakuwa, Karl Koscher, and Tadayoshi Kohno. 2016. Automobile driver fingerprinting. Proc. Priv. Enhanc. Technol. 2016 (012016). DOI:
[17]
Cyrus Farivar. 2015. FBI: Researcher Admitted to Hacking Plane In-flight, Causing it to “Climb.” Retrieved from https://arstechnica.com/information-technology/2015/05/fbi-researcher-admitted-to-hacking-plane-in-flight-causing-it-to-climb/.
[18]
Yoav Freund and Robert E. Schapire. 1997. A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci. 55, 1 (August1997), 119–139. DOI:
[19]
M. Gmiden, M. H. Gmiden, and H. Trabelsi. 2016. An intrusion detection method for securing in-vehicle CAN bus. In Proceedings of the 17th International Conference on Sciences and Techniques of Automatic Control and Computer Engineering (STA’16). 176–180. DOI:
[20]
Bogdan Groza and Pal-Stefan Murvay. 2019. Efficient intrusion detection with bloom filtering in controller area networks. IEEE Trans. Inf. Forens. Secur. 14, 4 (2019), 1037–1051. DOI:
[21]
Bogdan Groza, Stefan Murvay, Anthony Van Herrewege, and Ingrid Verbauwhede. 2017. LiBrA-CAN: Lightweight broadcast authentication for controller area networks. ACM Trans. Embed. Comput. Syst. 16, 3, Article 90 (April2017), 28 pages. DOI:
[22]
Jon C. Haass, J. Philip Craiger, and Gary C. Kessler. 2018. A framework for aviation cybersecurity. In Proceedings of the IEEE National Aerospace and Electronics Conference (NAESON’18). 132–136. DOI:
[23]
CAN in Automation. 1999. CANopen application layer and communication profile.
[25]
Kvaser Inc.An Overview of ARINC. Retrieved fromhttps://www.kvaser.com/arinc/.
[26]
National Instruments. Controller Area Network (CAN) Overview. Technical Report.
[27]
Ki-Dong Kang, Youngmi Baek, Seonghun Lee, and Sang Hyuk Son. 2017. An attack-resilient source authentication protocol in controller area network. In Proceedings of the Symposium on Architectures for Networking and Communications Systems (ANCS’17). IEEE Press, 109–118. DOI:
[28]
M. Kang and J. Kang. 2016. A novel intrusion detection method using deep neural network for in-vehicle network security. In Proceedings of the IEEE 83rd Vehicular Technology Conference (VTC Spring’16). 1–5. DOI:
[29]
Patrick Kiley. 2019. Investigating CAN Bus Network Integrity in Avionics Systems. Retrieved fromhttps://www.rapid7.com/research/report/investigating-can-bus-network-integrity-in-avionics-systems/.
[30]
Ralph Knueppel. 2012. Standardization of CAN networks for airborne use through ARINC 825.
[31]
Karl Koscher, Alexei Czeskis, Franziska Roesner, Shwetak Patel, Tadayoshi Kohno, Stephen Checkoway, Damon McCoy, Brian Kantor, Danny Anderson, Hovav Shacham, et al. 2010. Experimental security analysis of a modern automobile. In Proceedings of the IEEE Symposium on Security and Privacy (SP’10). IEEE, 447–462.
[32]
Vipin Kumar Kukkala, Sooryaa Vignesh Thiruloga, and Sudeep Pasricha. 2020. INDRA: Intrusion detection using recurrent autoencoders in automotive embedded systems. arxiv:cs.CR/2007.08795. Retrieved from https://arxiv.org/abs/2007.08795.
[33]
Gulshan Kumar. 2014. Evaluation metrics for intrusion detection systems—A study. Int. J. Comput. Sci. Mobile Appl. (2014).
[34]
T. Kuwahara, Yukino Baba, H. Kashima, T. Kishikawa, Jun’ichi Tsurumi, T. Haga, Y. Ujiie, Takamitsu Sasaki, and Hideki Matsushima. 2018. Supervised and unsupervised intrusion detection based on CAN message frequencies for in-vehicle network. J. Inf. Process. 26 (2018), 306–313.
[35]
Donghwoon Kwon, Suwoo Park, and Jeong-Tak Ryu. 2017. A study on big data thinking of the internet of things-based smart-connected car in conjunction with controller area network bus and 4G-long term evolution. Symmetry 9, 8 (2017). DOI:
[36]
Ulf E. Larson, Dennis K. Nilsson, and Erland Jonsson. 2008. An approach to specification-based attack detection for in-vehicle networks. In Proceedings of the IEEE Intelligent Vehicles Symposium. 220–225. DOI:
[37]
C. Lin and A. Sangiovanni-Vincentelli. 2012. Cyber-security for the controller area network (CAN) communication protocol. In Proceedings of the International Conference on Cyber Security. 1–7. DOI:
[38]
Congli Ling and Dongqin Feng. 2012/11. An algorithm for detection of malicious messages on CAN buses. In Proceedings of the National Conference on Information Technology and Computer Science. Atlantis Press, 627–630. DOI:
[39]
S. Longari, D. H. N. Valcarcel, M. Zago, M. Carminati, and S. Zanero. 2020. CANnolo: An anomaly detection system based on LSTM autoencoders for controller area network. IEEE Trans. Netw. Serv. Manage. (2020), 1–1. DOI:
[40]
Rouhollah Mahfouzi, Amir Aminifar, Petru Eles, Zebo Peng, and Mattias Villani. 2016. Intrusion-damage assessment and mitigation in cyber-physical systems for control applications. In Proceedings of the 24th International Conference on Real-Time Networks and Systems (RTNS’16). 141–150.
[41]
Rouhollah Mahfouzi, Amir Aminifar, Soheil Samii, Mathias Payer, Petru Eles, and Zebo Peng. 2019. Butterfly attack: Adversarial manipulation of temporal properties of cyber-physical systems. In Proceedings of the IEEE Real-Time Systems Symposium (RTSS’19).
[42]
M. Marchetti and D. Stabili. 2017. Anomaly detection of CAN bus messages through analysis of ID sequences. In Proceedings of the IEEE Intelligent Vehicles Symposium (IV’17). 1577–1583. DOI:
[43]
John McHugh. 2000. Testing intrusion detection systems: A critique of the 1998 and 1999 darpa intrusion detection system evaluations as performed by lincoln laboratory. ACM Trans. Inf. Syst. Secur. 3, 4 (November2000), 262–294. DOI:
[44]
Charlie Miller and Chris Valasek. A Survey of Remote Automotive Attack Surfaces. Technical Report.
[45]
Francesco Mola. 1998. Classification and regression trees software and new developments. In Advances in Data Science and Classification, Alfredo Rizzi, Maurizio Vichi, and Hans-Hermann Bock (Eds.). Springer, Berlin, 311–318.
[46]
Michael R. Moore, Robert A. Bridges, Frank L. Combs, Michael S. Starr, and Stacy J. Prowell. 2017. Modeling inter-signal arrival times for accurate detection of CAN bus signal injection attacks: A data-driven approach to in-vehicle intrusion detection. In Proceedings of the 12th Annual Conference on Cyber and Information Security Research (CISRC’17). Association for Computing Machinery, New York, NY, Article 11, 4 pages. DOI:
[47]
Subhojeet Mukherjee, Hossein Shirazi, Indrakshi Ray, Jeremy Daily, and Rose Gamble. 2016. Practical DoS attacks on embedded networks in commercial vehicles. In Information Systems Security, Indrajit Ray, Manoj Singh Gaur, Mauro Conti, Dheeraj Sanghi, and V. Kamakoti (Eds.). Springer International Publishing, Cham, 23–42.
[48]
M. Müter and N. Asaj. 2011. Entropy-based anomaly detection for in-vehicle networks. In Proceedings of the IEEE Intelligent Vehicles Symposium (IV’11). 1110–1115. DOI:
[49]
G. V. Nadiammai and M. Hemalatha. 2014. Effective approach toward intrusion detection system using data mining techniques. Egypt. Inf. J. 15, 1 (2014), 37–50. DOI:
[50]
Dennis K. Nilsson, Ulf E. Larson, and Erland Jonsson. 2008. Efficient in-vehicle delayed data authentication based on compound message authentication codes. In Proceedings of the IEEE 68th Vehicular Technology Conference. 1–5. DOI:
[51]
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine learning in Python. J. Mach. Learn. Res. 12 (2011), 2825–2830.
[52]
Adrian Perrig, Robert Szewczyk, Victor Wen, David Culler, and J. D. Tygar. 2001. SPINS: Security protocols for sensor networks. In Proceedings of the 7th Annual International Conference on Mobile Computing and Networking (MobiCom’01). Association for Computing Machinery, New York, NY, 189–199. DOI:
[53]
David Powers. 2008. Evaluation: From precision, recall and F-factor to ROC, informedness, markedness & correlation. Mach. Learn. Technol. 2 (012008).
[54]
Florian Sagstetter, Sidharta Andalam, Peter Waszecki, Martin Lukasiewycz, Hauke Stähle, Samarjit Chakraborty, and Alois Knoll. 2014. Schedule integration framework for time-triggered automotive architectures. In Proceedings of the 51st ACM/EDAC/IEEE Design Automation Conference (DAC’14).
[55]
Jaydip Sen and Sidra Mehtab. 2020. Machine learning applications in misuse and anomaly detection. In Security and Privacy From a Legal, Ethical, and Technical Perspective. IntechOpen. DOI:
[56]
Y. Shoukry, P. Martin, P. Tabuada, and M. Srivastava. 2013. Non-invasive spoofing attacks for anti-lock braking systems. In Cryptographic Hardware and Embedded Systems. Lecture Notes in Computer Science, Vol. 8086. Springer, Berlin.
[57]
Taylor G. Smith et al. 2017–. pmdarima: ARIMA estimators for Python. Retrieved fromhttp://www.alkaline-ml.com/pmdarima.
[58]
H. M. Song, H. R. Kim, and H. K. Kim. 2016. Intrusion detection system based on the analysis of time intervals of CAN messages for in-vehicle network. In Proceedings of the International Conference on Information Networking (ICOIN’16). 63–68. DOI:
[59]
Jason Staggs. 2013. How to Hack Your Mini Cooper: Reverse Engineering Controller Area Network CAN Messages on Passenger Automobiles. Retrieved May 13, 2021 from https://doi.org/10.5446/38934
[60]
Ivan Studnia, Vincent Nicomette, Eric Alata, Yves Deswarte, Mohamed Kaâniche, and Youssef Laarouchi. 2013. Survey on security threats and protection mechanisms in embedded automotive networks. In Proceedings of the 43rd Annual IEEE/IFIP Conference on Dependable Systems and Networks Workshop (DSN-W’13). 1–12. DOI:
[61]
Stock Flight Systems. CANaerospace. Retrieved from https://www.stockflightsystems.com/canaerospace.html.
[62]
A. Taylor, N. Japkowicz, and S. Leblanc. 2015. Frequency-based anomaly detection for the automotive CAN bus. In Proceedings of the World Congress on Industrial Control Systems Security (WCICSS’15). 45–49. DOI:
[63]
Texas Instruments 2016. Introduction to the Controller Area Network(CAN). Retrieved fromhttps://www.ti.com/lit/an/sloa101b/sloa101b.pdf.
[64]
A. Tomlinson, J. Bryans, S. A. Shaikh, and H. K. Kalutarage. 2018. Detection of automotive CAN cyber-attacks by identifying packet timing anomalies in time windows. In Proceedings of the 48th Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshops (DSN-W’18). 231–238. DOI:
[65]
Clinton Young, Habeeb Olufowobi, Gedare Bloom, and Joseph Zambreno. 2019. Automotive intrusion detection based on constant CAN message frequencies across vehicle driving modes. In Proceedings of the ACM Workshop on Automotive Cybersecurity (AutoSec’19). Association for Computing Machinery, New York, NY, 9–14. DOI:
[66]
Clinton Young, Joseph Zambreno, Habeeb Olufowobi, and Gedare Bloom. 2019. Survey of automotive controller area network intrusion detection systems. IEEE Des. Test 36, 6 (2019), 48–55. DOI:
[67]
Derek Yu, Michael Vaquier, Evan Laflamme, Gabrielle Doucette-Poirier, Justin Tremblay, and Brett H. Meyer. 2019. ARINC-825TBv2: A hardware-in-the-ioop simulation platform for aerospace security research. In Proceedings of the 30th International Workshop on Rapid System Prototyping (RSP’19). Association for Computing Machinery, New York, NY, 29–35.DOI:
[68]
Jiong Zhang and Mohammad Zulkernine. 2006. Anomaly based network intrusion detection with unsupervised outlier detection. In Proceedings of the IEEE International Conference on Communications, Vol. 5. DOI:

Cited By

View all
  • (2024)can-fp: An Attack-Aware Analysis of False Alarms in Automotive Intrusion Detection Models2024 21st Annual International Conference on Privacy, Security and Trust (PST)10.1109/PST62714.2024.10788039(1-12)Online publication date: 28-Aug-2024

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Cyber-Physical Systems
ACM Transactions on Cyber-Physical Systems  Volume 7, Issue 1
January 2023
187 pages
ISSN:2378-962X
EISSN:2378-9638
DOI:10.1145/3582896
  • Editor:
  • Chenyang Lu
Issue’s Table of Contents

Publisher

Association for Computing Machinery

New York, NY, United States

Journal Family

Publication History

Published: 22 March 2023
Online AM: 21 January 2023
Accepted: 14 September 2022
Revised: 18 April 2022
Received: 01 July 2021
Published in TCPS Volume 7, Issue 1

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Controller area network
  2. avionic security
  3. ARINC-825
  4. intrusion detection system
  5. ensemble learning

Qualifiers

  • Research-article

Funding Sources

  • Natural Science and Engineering Research Council of Canada (NSERC)

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)121
  • Downloads (Last 6 weeks)9
Reflects downloads up to 20 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)can-fp: An Attack-Aware Analysis of False Alarms in Automotive Intrusion Detection Models2024 21st Annual International Conference on Privacy, Security and Trust (PST)10.1109/PST62714.2024.10788039(1-12)Online publication date: 28-Aug-2024

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Full Text

View this article in Full Text.

Full Text

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media