research-article

Automated security test generation for MQTT using attack patterns

Authors:

Flavio Ferrarotti,

Rudolf RamlerAuthors Info & Claims

ARES '20: Proceedings of the 15th International Conference on Availability, Reliability and Security

Article No.: 97, Pages 1 - 9

https://doi.org/10.1145/3407023.3407078

Published: 25 August 2020 Publication History

Abstract

The dramatic increase of attacks and malicious activities has made security a major concern in the development of interconnected cyber-physical systems and raised the need to address this concern also in testing. The goal of security testing is to discover vulnerabilities in the system under test so that they can be fixed before an attacker finds and abuses them. However, testing for security issues faces the challenge of systematically exploring a potentially non-tractable number of interaction scenarios that have to include also invalid inputs and possible harmful interaction attempts. In this paper, we describe an approach for automated generation of test cases for security testing, which are based on attack patterns. These patterns are blueprints that can be used for exploiting common vulnerabilities. The approach combines random test case generation with attack patterns implemented for the Message Queuing Telemetry Transport (MQTT) protocol. We have applied the proposed testing approach to five popular and widely available MQTT brokers, generating 1,804 interaction sequences in form of executable test cases which resulted in numerous test failures, unhandled exceptions and crashes. A detailed manual analysis of these cases have revealed 28 security-relevant issues and critical shortcomings in the tested MQTT broker implementations.

References

[1]

Ala I. Al-Fuqaha, Mohsen Guizani, Mehdi Mohammadi, Mohammed Aledhari, and Moussa Ayyash. 2015. Internet of Things: A Survey on Enabling Technologies, Protocols, and Applications. IEEE Communications Surveys and Tutorials 17, 4 (2015), 2347--2376.

Digital Library

[2]

Saswat Anand, Edmund K Burke, Tsong Yueh Chen, John Clark, Myra B Cohen, Wolfgang Grieskamp, Mark Harman, Mary Jean Harrold, Phil Mcminn, Antonia Bertolino, et al. 2013. An orchestrated survey of methodologies for automated software test case generation. Journal of Systems and Software 86, 8 (2013), 1978--2001.

Digital Library

[3]

Andrew Banks, Ed Briggs, Ken Borgendale, and Rahul Gupta. [n.d.]. MQTT Version 5.0. OASIS Standard. https://docs.oasis-open.org/mqtt/mqtt/v5.0/mqtt-v5.0.html

[4]

Andrew Banks and Rahul Gupta. [n.d.]. MQTT Version 3.1.1. OASIS Standard. http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/mqtt-v3.1.1.html

[5]

Josip Bozic and Franz Wotawa. 2014. Security Testing Based on Attack Patterns. In Seventh IEEE International Conference on Software Testing, Verification and Validation, ICST 2014 Workshops Proceedings, March 31 - April 4, 2014, Cleveland, Ohio, USA. IEEE Computer Society, 4--11.

[6]

Alireza Esfahani, Georgios Mantas, Rainer Matischek, Firooz B. Saghezchi, Jonathan Rodriguez, Ani Bicaku, Silia Maksuti, Markus Gerhard Tauber, Christoph Schmittner, and Joaquim Bastos. 2019. A Lightweight Authentication Mechanism for M2M Communications in Industrial IoT Environment. IEEE Internet of Things Journal 6, 1 (2019), 288--296.

[7]

Patrick Th. Eugster, Pascal Felber, Rachid Guerraoui, and Anne-Marie Kermarrec. 2003. The many faces of publish/subscribe. ACM Comput. Surv. 35, 2 (2003), 114--131.

Digital Library

[8]

Tobias Heer, Oscar García Morchon, René Hummen, Sye Loong Keoh, Sandeep S. Kumar, and Klaus Wehrle. 2011. Security Challenges in the IP-based Internet of Things. Wireless Personal Communications 61, 3 (2011), 527--542.

Digital Library

[9]

HiveMQ. [n.d.]. Enterprise MQTT Broker. https://www.hivemq.com/downloads/hivemq-data-sheet-4.2.pdf.

[10]

HiveMQ. [n.d.]. Security Fundamentals. https://www.hivemq.com/blog/mqtt-security-fundamentals-authentication-username-password/.

[11]

Greg Hoglund and Gary McGraw. 2004. Exploiting Software: How to Break Code. Addison Wesley.

Digital Library

[12]

M. Houimli, L. Kahloul, and S. Benaoun. 2017. Formal specification, verification and evaluation of the MQTT protocol in the Internet of Things. In 2017 International Conference on Mathematics and Information Technology (ICMIT). IEEE Computer Society, 214--221.

[13]

Rizwan Khan and Santosh Kumar. 2013. Using Exploit Patterns to Develop Secure Software. VSRD International Journal of Computer Science & Information Technology 3 (01 2013), 257--260.

[14]

Amir Manzoor. 2016. Securing Device Connectivity in the Industrial Internet of Things (IoT). In Connectivity Frameworks for Smart Devices. Computer Communications and Networks. Springer, Cham, 3--22.

[15]

Vasileios Mavroeidis and Siri Bromander. 2017. Cyber threat intelligence model: An evaluation of taxonomies, sharing standards, and ontologies within cyber threat intelligence. In 2017 European Intelligence and Security Informatics Conference (EISIC). IEEE, 91--98.

[16]

Kristiyan Mladenov. 2017. Formal verification of the implementation of the MQTT protocol in IoT devices. Technical Report. University of Amsterdam, Faculty of Physics, Mathematics and Informatics.

[17]

Andrew Moore, Robert Ellison, and Rick Linger. 2001. Attack Modeling for Information Security and Survivability. Technical Report. Technical Note CMU/SEI-2001-TN-001, Carnegie Mellon University.

[18]

Ricardo Neisse, Gary Steri, and Gianmarco Baldini. 2014. Enforcement of security policy rules for the Internet of Things. In IEEE 10th International Conference on Wireless and Mobile Computing, Networking and Communications, WiMob 2014. IEEE Computer Society, 165--172.

[19]

Carlos Pacheco and Michael D Ernst. 2007. Randoop: feedback-directed random testing for Java. In Companion to the 22nd ACM SIGPLAN conference on Object-oriented programming systems and applications companion. 815--816.

[20]

Rudolf Ramler, Georg Buchgeher, and Claus Klammer. 2018. Adapting automated test generation to GUI testing of industry applications. Information and Software Technology 93 (2018), 248--263.

Digital Library

[21]

Santiago Hernández Ramos, M. Teresa Villalba, and Raquel Lacuesta. 2018. MQTT Security: A Novel Fuzzing Approach. Wireless Communications and Mobile Computing 2018 (2018).

[22]

SeongHan Shin, Kazukuni Kobara, Chia-Chuan Chuang, and Weicheng Huang. 2016. A security framework for MQTT. In 2016 IEEE Conference on Communications and Network Security, CNS 2016, Philadelphia, PA, USA, October 17-19, 2016. IEEE, 432--436.

[23]

M. Singh, M. A. Rajan, V. L. Shivraj, and P. Balamuralidhar. 2015. Secure MQTT for Internet of Things (IoT). In Fifth International Conference on Communication Systems and Network Technologies. IEEE, 746--751.

[24]

Hannes Sochor, Flavio Ferrarotti, and Rudolf Ramler. 2020. An Architecture for Automated Security Test Case Generation for MQTT Systems. In 4th International Workshop on Cyber-Security and Functional Safety in Cyber-Physical Systems (IWCFS), co-located with DEXA 2020, Bratislava, Slovakia. Springer.

[25]

Wei-Tsung Su, Wei-Cheng Chen, and Chao-Chun Chen. 2019. An Extensible and Transparent Thing-to-Thing Security Enhancement for MQTT Protocol in IoT Environment. In 2019 Global IoT Summit, GIoTS 2019, Aarhus, Denmark, June 17-21, 2019. IEEE, 1--4.

[26]

Ari Takanen, Jared D Demott, Charles Miller, and Atte Kettunen. 2018. Fuzzing for software security testing and quality assurance. Artech House.

[27]

Martin Tappler, Bernhard K. Aichernig, and Roderick Bloem. 2017. Model-Based Testing IoT Communication via Active Automata Learning. In 2017 IEEE International Conference on Software Testing, Verification and Validation, ICST 2017, Tokyo, Japan, March 13-17, 2017. IEEE Computer Society, 276--287.

[28]

James A Whittaker. 2002. How to Break Software: A Practical Guide to Testing. Addison-Wesley Longman Publishing.

Cited By

Quincozes VQuincozes SKazienko JGama SCheikhrouhou OKoubaa A(2024)A survey on IoT application layer protocols, security challenges, and the role of explainable AI in IoT (XAIoT)International Journal of Information Security10.1007/s10207-024-00828-w23:3(1975-2002)Online publication date: 13-Mar-2024
https://doi.org/10.1007/s10207-024-00828-w
Rodriguez LBatista D(2023)Resource-Intensive Fuzzing for MQTT Brokers: State of the Art, Performance Evaluation, and Open IssuesIEEE Networking Letters10.1109/LNET.2023.32635565:2(100-104)Online publication date: Jun-2023
https://doi.org/10.1109/LNET.2023.3263556
Wan CMehmood ACarsten MEpiphaniou GLloret J(2022)A Blockchain Based Forensic System for IoT Sensors using MQTT Protocol2022 9th International Conference on Internet of Things: Systems, Management and Security (IOTSMS)10.1109/IOTSMS58070.2022.10062190(1-8)Online publication date: 29-Nov-2022
https://doi.org/10.1109/IOTSMS58070.2022.10062190
Show More Cited By

Index Terms

Automated security test generation for MQTT using attack patterns
1. Security and privacy
  1. Network security
  2. Software and application security

Recommendations

Automated Security Test Generation with Formal Threat Models

Security attacks typically result from unintended behaviors or invalid inputs. Security testing is labor intensive because a real-world program usually has too many invalid inputs. It is highly desirable to automate or partially automate security-...
Attack pattern-based combinatorial testing
AST 2014: Proceedings of the 9th International Workshop on Automation of Software Test

The number of potential security threats rises with the increasing number of web applications, which cause tremendous financial and existential implications for developers and users as well. The biggest challenge for security testing is to specify and ...
An orchestrated survey of methodologies for automated software test case generation

Test case generation is among the most labour-intensive tasks in software testing. It also has a strong impact on the effectiveness and efficiency of software testing. For these reasons, it has been one of the most active research topics in software ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ARES '20: Proceedings of the 15th International Conference on Availability, Reliability and Security

August 2020

1073 pages

ISBN:9781450388337

DOI:10.1145/3407023

Program Chairs:
Melanie Volkamer
Karlsruhe Institute of Technologie (KIT), Competence Center for Applied Security Technology (KASTEL)
,
Christian Wressnegger
Karlsruhe Institute of Technologie (KIT), Competence Center for Applied Security Technology (KASTEL)

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 August 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Federal Ministry for Climate Action, Environment, Energy, Mobility, Innovation and Technology (BMK)
Province of Upper Austria
COMET - Competence Centers for Excellent Technologies Programme
Federal Ministry for Digital and Economic Affairs (BMDW)
Austrian Research Promotion Agency (FFG)

Conference

ARES 2020

ARES 2020: The 15th International Conference on Availability, Reliability and Security

August 25 - 28, 2020

Virtual Event, Ireland

Acceptance Rates

Overall Acceptance Rate 228 of 451 submissions, 51%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
316
Total Downloads

Downloads (Last 12 months)29
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Quincozes VQuincozes SKazienko JGama SCheikhrouhou OKoubaa A(2024)A survey on IoT application layer protocols, security challenges, and the role of explainable AI in IoT (XAIoT)International Journal of Information Security10.1007/s10207-024-00828-w23:3(1975-2002)Online publication date: 13-Mar-2024
https://doi.org/10.1007/s10207-024-00828-w
Rodriguez LBatista D(2023)Resource-Intensive Fuzzing for MQTT Brokers: State of the Art, Performance Evaluation, and Open IssuesIEEE Networking Letters10.1109/LNET.2023.32635565:2(100-104)Online publication date: Jun-2023
https://doi.org/10.1109/LNET.2023.3263556
Wan CMehmood ACarsten MEpiphaniou GLloret J(2022)A Blockchain Based Forensic System for IoT Sensors using MQTT Protocol2022 9th International Conference on Internet of Things: Systems, Management and Security (IOTSMS)10.1109/IOTSMS58070.2022.10062190(1-8)Online publication date: 29-Nov-2022
https://doi.org/10.1109/IOTSMS58070.2022.10062190
Araujo Rodriguez LBatista D(2021)Towards Improving Fuzzer Efficiency for the MQTT Protocol2021 IEEE Symposium on Computers and Communications (ISCC)10.1109/ISCC53001.2021.9631520(1-7)Online publication date: 5-Sep-2021
https://doi.org/10.1109/ISCC53001.2021.9631520
Aichernig BMuskardin EPferscher A(2021)Learning-Based Fuzzing of IoT Message Brokers2021 14th IEEE Conference on Software Testing, Verification and Validation (ICST)10.1109/ICST49551.2021.00017(47-58)Online publication date: Apr-2021
https://doi.org/10.1109/ICST49551.2021.00017
Sochor HFerrarotti FRamler R(2021)An automated evaluation of broker compatibility for the Message Queuing Telemetry Transport protocolJournal of Software: Evolution and Process10.1002/smr.2410Online publication date: 6-Dec-2021
https://doi.org/10.1002/smr.2410

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten