Hannes Sochor
ABSTRACT
The dramatic increase of attacks and malicious activities has made security a major concern in the development of interconnected cyber-physical systems and raised the need to address this concern also in testing. The goal of security testing is to discover vulnerabilities in the system under test so that they can be fixed before an attacker finds and abuses them. However, testing for security issues faces the challenge of systematically exploring a potentially non-tractable number of interaction scenarios that have to include also invalid inputs and possible harmful interaction attempts. In this paper, we describe an approach for automated generation of test cases for security testing, which are based on attack patterns. These patterns are blueprints that can be used for exploiting common vulnerabilities. The approach combines random test case generation with attack patterns implemented for the Message Queuing Telemetry Transport (MQTT) protocol. We have applied the proposed testing approach to five popular and widely available MQTT brokers, generating 1,804 interaction sequences in form of executable test cases which resulted in numerous test failures, unhandled exceptions and crashes. A detailed manual analysis of these cases have revealed 28 security-relevant issues and critical shortcomings in the tested MQTT broker implementations.
- Ala I. Al-Fuqaha, Mohsen Guizani, Mehdi Mohammadi, Mohammed Aledhari, and Moussa Ayyash. 2015. Internet of Things: A Survey on Enabling Technologies, Protocols, and Applications. IEEE Communications Surveys and Tutorials 17, 4 (2015), 2347--2376.Google Scholar
Digital Library
- Saswat Anand, Edmund K Burke, Tsong Yueh Chen, John Clark, Myra B Cohen, Wolfgang Grieskamp, Mark Harman, Mary Jean Harrold, Phil Mcminn, Antonia Bertolino, et al. 2013. An orchestrated survey of methodologies for automated software test case generation. Journal of Systems and Software 86, 8 (2013), 1978--2001.Google ScholarDigital Library
- Andrew Banks, Ed Briggs, Ken Borgendale, and Rahul Gupta. [n.d.]. MQTT Version 5.0. OASIS Standard. https://docs.oasis-open.org/mqtt/mqtt/v5.0/mqtt-v5.0.htmlGoogle Scholar
- Andrew Banks and Rahul Gupta. [n.d.]. MQTT Version 3.1.1. OASIS Standard. http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/mqtt-v3.1.1.htmlGoogle Scholar
- Josip Bozic and Franz Wotawa. 2014. Security Testing Based on Attack Patterns. In Seventh IEEE International Conference on Software Testing, Verification and Validation, ICST 2014 Workshops Proceedings, March 31 - April 4, 2014, Cleveland, Ohio, USA. IEEE Computer Society, 4--11.Google Scholar
- Alireza Esfahani, Georgios Mantas, Rainer Matischek, Firooz B. Saghezchi, Jonathan Rodriguez, Ani Bicaku, Silia Maksuti, Markus Gerhard Tauber, Christoph Schmittner, and Joaquim Bastos. 2019. A Lightweight Authentication Mechanism for M2M Communications in Industrial IoT Environment. IEEE Internet of Things Journal 6, 1 (2019), 288--296.Google ScholarCross Ref
- Patrick Th. Eugster, Pascal Felber, Rachid Guerraoui, and Anne-Marie Kermarrec. 2003. The many faces of publish/subscribe. ACM Comput. Surv. 35, 2 (2003), 114--131.Google ScholarDigital Library
- Tobias Heer, Oscar García Morchon, René Hummen, Sye Loong Keoh, Sandeep S. Kumar, and Klaus Wehrle. 2011. Security Challenges in the IP-based Internet of Things. Wireless Personal Communications 61, 3 (2011), 527--542.Google ScholarDigital Library
- HiveMQ. [n.d.]. Enterprise MQTT Broker. https://www.hivemq.com/downloads/hivemq-data-sheet-4.2.pdf.Google Scholar
- HiveMQ. [n.d.]. Security Fundamentals. https://www.hivemq.com/blog/mqtt-security-fundamentals-authentication-username-password/.Google Scholar
- Greg Hoglund and Gary McGraw. 2004. Exploiting Software: How to Break Code. Addison Wesley.Google ScholarDigital Library
- M. Houimli, L. Kahloul, and S. Benaoun. 2017. Formal specification, verification and evaluation of the MQTT protocol in the Internet of Things. In 2017 International Conference on Mathematics and Information Technology (ICMIT). IEEE Computer Society, 214--221.Google Scholar
- Rizwan Khan and Santosh Kumar. 2013. Using Exploit Patterns to Develop Secure Software. VSRD International Journal of Computer Science & Information Technology 3 (01 2013), 257--260.Google Scholar
- Amir Manzoor. 2016. Securing Device Connectivity in the Industrial Internet of Things (IoT). In Connectivity Frameworks for Smart Devices. Computer Communications and Networks. Springer, Cham, 3--22.Google Scholar
- Vasileios Mavroeidis and Siri Bromander. 2017. Cyber threat intelligence model: An evaluation of taxonomies, sharing standards, and ontologies within cyber threat intelligence. In 2017 European Intelligence and Security Informatics Conference (EISIC). IEEE, 91--98.Google ScholarCross Ref
- Kristiyan Mladenov. 2017. Formal verification of the implementation of the MQTT protocol in IoT devices. Technical Report. University of Amsterdam, Faculty of Physics, Mathematics and Informatics.Google Scholar
- Andrew Moore, Robert Ellison, and Rick Linger. 2001. Attack Modeling for Information Security and Survivability. Technical Report. Technical Note CMU/SEI-2001-TN-001, Carnegie Mellon University.Google Scholar
- Ricardo Neisse, Gary Steri, and Gianmarco Baldini. 2014. Enforcement of security policy rules for the Internet of Things. In IEEE 10th International Conference on Wireless and Mobile Computing, Networking and Communications, WiMob 2014. IEEE Computer Society, 165--172.Google ScholarCross Ref
- Carlos Pacheco and Michael D Ernst. 2007. Randoop: feedback-directed random testing for Java. In Companion to the 22nd ACM SIGPLAN conference on Object-oriented programming systems and applications companion. 815--816.Google Scholar
- Rudolf Ramler, Georg Buchgeher, and Claus Klammer. 2018. Adapting automated test generation to GUI testing of industry applications. Information and Software Technology 93 (2018), 248--263.Google ScholarDigital Library
- Santiago Hernández Ramos, M. Teresa Villalba, and Raquel Lacuesta. 2018. MQTT Security: A Novel Fuzzing Approach. Wireless Communications and Mobile Computing 2018 (2018).Google Scholar
- SeongHan Shin, Kazukuni Kobara, Chia-Chuan Chuang, and Weicheng Huang. 2016. A security framework for MQTT. In 2016 IEEE Conference on Communications and Network Security, CNS 2016, Philadelphia, PA, USA, October 17-19, 2016. IEEE, 432--436.Google ScholarCross Ref
- M. Singh, M. A. Rajan, V. L. Shivraj, and P. Balamuralidhar. 2015. Secure MQTT for Internet of Things (IoT). In Fifth International Conference on Communication Systems and Network Technologies. IEEE, 746--751.Google Scholar
- Hannes Sochor, Flavio Ferrarotti, and Rudolf Ramler. 2020. An Architecture for Automated Security Test Case Generation for MQTT Systems. In 4th International Workshop on Cyber-Security and Functional Safety in Cyber-Physical Systems (IWCFS), co-located with DEXA 2020, Bratislava, Slovakia. Springer.Google ScholarCross Ref
- Wei-Tsung Su, Wei-Cheng Chen, and Chao-Chun Chen. 2019. An Extensible and Transparent Thing-to-Thing Security Enhancement for MQTT Protocol in IoT Environment. In 2019 Global IoT Summit, GIoTS 2019, Aarhus, Denmark, June 17-21, 2019. IEEE, 1--4.Google Scholar
- Ari Takanen, Jared D Demott, Charles Miller, and Atte Kettunen. 2018. Fuzzing for software security testing and quality assurance. Artech House.Google Scholar
- Martin Tappler, Bernhard K. Aichernig, and Roderick Bloem. 2017. Model-Based Testing IoT Communication via Active Automata Learning. In 2017 IEEE International Conference on Software Testing, Verification and Validation, ICST 2017, Tokyo, Japan, March 13-17, 2017. IEEE Computer Society, 276--287.Google ScholarCross Ref
- James A Whittaker. 2002. How to Break Software: A Practical Guide to Testing. Addison-Wesley Longman Publishing.Google Scholar
Index Terms
- Automated security test generation for MQTT using attack patterns
Recommendations
Automated Security Test Generation with Formal Threat Models
Security attacks typically result from unintended behaviors or invalid inputs. Security testing is labor intensive because a real-world program usually has too many invalid inputs. It is highly desirable to automate or partially automate security-...
Attack pattern-based combinatorial testing
AST 2014: Proceedings of the 9th International Workshop on Automation of Software TestThe number of potential security threats rises with the increasing number of web applications, which cause tremendous financial and existential implications for developers and users as well. The biggest challenge for security testing is to specify and ...
An orchestrated survey of methodologies for automated software test case generation
Test case generation is among the most labour-intensive tasks in software testing. It also has a strong impact on the effectiveness and efficiency of software testing. For these reasons, it has been one of the most active research topics in software ...
Comments