Skip to main content
SpringerLink
Log in

Modeling Obfuscation Stealth Through Code Complexity

  • Conference paper
  • First Online:
Computer Security. ESORICS 2023 International Workshops (ESORICS 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14399))

Included in the following conference series:

  • 145 Accesses

Abstract

Code obfuscation is often utilized by authors of malware to protect it from detection or to hide its maliciousness from code analysis. Obfuscation stealth describes how difficult it is to determine which protection technique has been applied to a program and which parts of the code have been protected. In previous literature, most of the presented obfuscation identification methods analyze the program code itself (for example, the frequency and distribution of opcodes). However, simple countermeasures such as instruction substitution can have a negative impact on the identification rate. In this paper, we present a novel approach for an accurate obfuscation identification model based on a combination of multiple code complexity metrics. An evaluation with 4124 samples protected with 11 different obfuscations, combinations of obfuscations, and various compiler configurations demonstrates an overall classification accuracy of 86.5%.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://irdeto.com.

  2. 2.

    https://tigress.wtf.

  3. 3.

    https://myjit.sourceforge.net.

References

  1. Abran, A., Lopez, M., Habra, N.: An analysis of the mccabe cyclomatic complexity number. In: Proceedings of the 14th International Workshop on Software Measurement (IWSM) IWSM-Metrikon, pp. 391–405 (2004)

    Google Scholar 

  2. Bacci, A., Bartoli, A., Martinelli, F., Medvet, E., Mercaldo, F.: Detection of obfuscation techniques in android applications. In: Proceedings of the 13th International Conference on Availability, Reliability and Security, pp. 1–9 (2018)

    Google Scholar 

  3. Basili, V.R., Perricone, B.T.: Software errors and complexity: an empirical investigation0. Commun. ACM 27(1), 42–52 (1984)

    Article  Google Scholar 

  4. Brosch, T., Morgenstern, M.: Runtime packers: the hidden problem. Black Hat USA (2006)

    Google Scholar 

  5. Canavese, D., Regano, L., Basile, C., Viticchié, A.: Estimating software obfuscation potency with artificial neural networks. In: Livraga, G., Mitchell, C. (eds.) STM 2017. LNCS, vol. 10547, pp. 193–202. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-68063-7_13

    Chapter  Google Scholar 

  6. Coleman, D., Oman, P., Ash, D., Lowther, B.: Using metrics to evaluate software system maintainability. Computer 27(08), 44–49 (1994)

    Google Scholar 

  7. Collberg, C., Thomborson, C., Low, D.: A taxonomy of obfuscating transformations. Technical report, Department of Computer Science, The University of Auckland, New Zealand (1997)

    Google Scholar 

  8. Collberg, C., Thomborson, C., Low, D.: Manufacturing cheap, resilient, and stealthy opaque constructs. In: Proceedings of the 25th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, pp. 184–196 (1998)

    Google Scholar 

  9. Ebad, S.A., Darem, A.A., Abawajy, J.H.: Measuring software obfuscation quality - a systematic literature review. IEEE Access 9, 99024–99038 (2021)

    Article  Google Scholar 

  10. Ebert, C., Cain, J., Antoniol, G., Counsell, S., Laplante, P.: Cyclomatic complexity. IEEE Softw. 33(6), 27–29 (2016)

    Article  Google Scholar 

  11. Fitzpatrick, J.: Applying the ABC metric to C, C++, and Java. Technical report, C++ report (1997)

    Google Scholar 

  12. Gibert, D., Mateu, C., Planes, J., Vicens, R.: Classification of malware by using structural entropy on convolutional neural networks. Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32, no. 1, April 2018. https://doi.org/10.1609/aaai.v32i1.11409, https://ojs.aaai.org/index.php/AAAI/article/view/11409

  13. Halstead, M.H.: Elements of Software Science. Operating and Programming Systems Series. Elsevier Science Inc., USA (1977)

    Google Scholar 

  14. Hatton, L.: Re-examining the defect-density versus component size distribution. IEEE Softw. 110 (1997)

    Google Scholar 

  15. Honglei, T., Wei, S., Yanan, Z.: The research on software metrics and software complexity metrics. In: 2009 International Forum on Computer Science-Technology and Applications, vol. 1, pp. 131–136. IEEE (2009)

    Google Scholar 

  16. Ikerionwu, C.: Cyclomatic complexity as a software metric. Int. J. Acad. Res. 2(3) (2010)

    Google Scholar 

  17. Junod, P., Rinaldini, J., Wehrli, J., Michielin, J.: Obfuscator-LLVM-software protection for the masses. In: 2015 IEEE/ACM 1st International Workshop on Software Protection, pp. 3–9. IEEE (2015)

    Google Scholar 

  18. Kanzaki, Y., Monden, A., Collberg, C.: Code artificiality: a metric for the code stealth based on an n-gram model. In: 2015 IEEE/ACM 1st International Workshop on Software Protection, pp. 31–37. IEEE (2015)

    Google Scholar 

  19. Khan, A.A., Mahmood, A., Amralla, S.M., Mirza, T.H.: Comparison of software complexity metrics. Int. J. Comput. Netw. Technol. 4(01) (2016)

    Google Scholar 

  20. Kim, J., Kang, S., Cho, E.-S., Paik, J.-Y.: LOM: lightweight classifier for obfuscation methods. In: Kim, H. (ed.) WISA 2021. LNCS, vol. 13009, pp. 3–15. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-89432-0_1

    Chapter  Google Scholar 

  21. Kurtukova, A., Romanov, A., Shelupanov, A.: Source code authorship identification using deep neural networks. Symmetry 12(12) (2020)

    Google Scholar 

  22. Madi, A., Zein, O.K., Kadry, S.: On the improvement of cyclomatic complexity metric. Int. J. Softw. Eng. Appl. 7(2), 67–82 (2013)

    Google Scholar 

  23. Madou, M., Anckaert, B., De Bus, B., De Bosschere, K., Cappaert, J., Preneel, B.: On the effectiveness of source code transformations for binary obfuscation. In: Proceedings of the International Conference on Software Engineering Research and Practice (SERP06), pp. 527–533. CSREA Press (2006)

    Google Scholar 

  24. Mason, J., Small, S., Monrose, F., MacManus, G.: English shellcode. In: Proceedings of the 16th ACM Conference on Computer and Communications Security, pp. 524–533 (2009)

    Google Scholar 

  25. McCabe, T.J.: A complexity measure. IEEE Trans. Softw. Eng. SE-2(4), 308–320 (1976). https://doi.org/10.1109/TSE.1976.233837

  26. Morgenstern, M., Pilz, H.: Useful and useless statistics about viruses and anti-virus programs. In: Proceedings of the CARO Workshop (2010)

    Google Scholar 

  27. Myers, G.J.: An extension to the cyclomatic measure of program complexity. SIGPLAN Not. 12(10), 61–64 (1977)

    Google Scholar 

  28. Nagra, J., Collberg, C.: Surreptitious Software: Obfuscation, Watermarking, and Tamperproofing for Software Protection: Obfuscation, Watermarking, and Tamperproofing for Software Protection. Pearson Education (2009)

    Google Scholar 

  29. Necula, G.C., McPeak, S., Weimer, W.: Cil: intermediate language and tools for analysis and transformation of C programs (2002)

    Google Scholar 

  30. Oman, P., Hagemeister, J.: Metrics for assessing a software system’s maintainability. In: Proceedings Conference on Software Maintenance 1992, pp. 337–344 (1992)

    Google Scholar 

  31. Oman, P., Hagemeister, J.: Construction and testing of polynomials predicting software maintainability. J. Syst. Softw. 24(3), 251–266 (1994). Oregon Workshop on Software Metrics

    Google Scholar 

  32. Rahbarinia, B., Balduzzi, M., Perdisci, R.: Exploring the long tail of (malicious) software downloads. In: 2017 47th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), pp. 391–402. IEEE (2017)

    Google Scholar 

  33. Sarwar, M.M.S., Shahzad, S., Ahmad, I.: Cyclomatic complexity: the nesting problem. In: Eighth International Conference on Digital Information Management (ICDIM 2013), pp. 274–279. IEEE (2013)

    Google Scholar 

  34. Sebastian, S.A., Malgaonkar, S., Shah, P., Kapoor, M., Parekhji, T.: A study & review on code obfuscation. In: 2016 World Conference on Futuristic Trends in Research and Innovation for Social Welfare, pp. 1–6. IEEE (2016)

    Google Scholar 

  35. Sellers, B.H.: Modularization and Mccabe’s Cyclomatic complexity. Commun. ACM 35(12), 17–20 (1992)

    Google Scholar 

  36. Shen, V.Y., Yu, T.J., Thebaut, S.M., Paulsen, L.R.: Identifying error-prone software-an empirical study. IEEE Trans. Softw. Eng. (4), 317–324 (1985)

    Google Scholar 

  37. Snoek, J., Larochelle, H., Adams, R.P.: Practical Bayesian optimization of machine learning algorithms. In: Advances in Neural Information Processing Systems, vol. 25 (2012)

    Google Scholar 

  38. Wang, Y., Rountev, A.: Who changed you? Obfuscator identification for android. In: 2017 IEEE/ACM 4th International Conference on Mobile Software Engineering and Systems (MOBILESoft), pp. 154–164. IEEE (2017)

    Google Scholar 

  39. Withrow, C.: Error density and size in ADA software. IEEE Softw. 7(1), 26–30 (1990)

    Article  Google Scholar 

  40. Wu, Z., Gianvecchio, S., Xie, M., Wang, H.: Mimimorphism: a new approach to binary code obfuscation. In: Proceedings of the 17th ACM Conference on Computer and Communications Security, pp. 536–546 (2010)

    Google Scholar 

  41. Yu, S., Zhou, S.: A survey on metric of software complexity. In: 2010 2nd IEEE International Conference on Information Management and Engineering, pp. 352–356. IEEE (2010)

    Google Scholar 

Download references

Acknowledgments

This research was funded in whole, or in part, by the Austrian Science Fund (FWF) I 3646-N31. For the purpose of open access, the author has applied a CC BY public copyright license to any Author Accepted Manuscript version arising from this submission.

Author information

Authors and Affiliations

  1. University of Vienna, Vienna, Austria

    Sebastian Schrittwieser, Kevin Mallinger, Patrick Kochberger, Caroline Lawitschka & Edgar R. Weippl

  2. SBA Research, Vienna, Austria

    Elisabeth Wimmer & Sebastian Raubitzek

  3. St. Pölten University of Applied Sciences, St. Pölten, Austria

    Patrick Kochberger

Authors
  1. Sebastian Schrittwieser
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Elisabeth Wimmer
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kevin Mallinger
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Patrick Kochberger
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Caroline Lawitschka
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Sebastian Raubitzek
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Edgar R. Weippl
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sebastian Schrittwieser .

Editor information

Editors and Affiliations

  1. Norwegian University of Science and Technology, Gjøvik, Norway

    Sokratis Katsikas

  2. Norwegian Computing Center, Oslo, Norway

    Habtamu Abie

  3. University of Trento, Trento, Italy

    Silvio Ranise

  4. University of Genoa, Genoa, Italy

    Luca Verderame

  5. Consiglio Nazionale delle Ricerche (CNR), Genoa, Italy

    Enrico Cambiaso

  6. SINTEF A.S., Oslo, Norway

    Rita Ugarelli

  7. Instituto Superior de Engenharia do Porto, Porto, Portugal

    Isabel Praça

  8. Hong Kong Polytechnic University, Hong Kong, China

    Wenjuan Li

  9. Technical University of Denmark, Kongens Lyngby, Denmark

    Weizhi Meng

  10. University of Nottingham, Nottingham, UK

    Steven Furnell

  11. Norwegian University of Science and Technology, Gjøvik, Norway

    Basel Katt

  12. Norwegian Computing Center, Oslo, Norway

    Sandeep Pirbhulal

  13. Institute for Energy Technology (IFE), Halden, Norway

    Ankur Shukla

  14. University of Calabria, Rende, Italy

    Michele Ianni

  15. University of Verona, Verona, Italy

    Mila Dalla Preda

  16. The University of Texas at San Antonio, San Antonio, TX, USA

    Kim-Kwang Raymond Choo

  17. University of Lisbon, Lisbon, Portugal

    Miguel Pupo Correia

  18. University of Twente, Enschede, The Netherlands

    Abhishta Abhishta

  19. University of Amsterdam, Amsterdam, The Netherlands

    Giovanni Sileno

  20. Open University in the Netherlands, Heerlen, The Netherlands

    Mina Alishahi

  21. Robert Gordon University, Aberdeen, UK

    Harsha Kalutarage

  22. Osaka University, Osaka, Japan

    Naoto Yanai

A Specifications

A Specifications

Table 4. Best parameter combinations found per classifier and feature set.
Full size table

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Schrittwieser, S. et al. (2024). Modeling Obfuscation Stealth Through Code Complexity. In: Katsikas, S., et al. Computer Security. ESORICS 2023 International Workshops. ESORICS 2023. Lecture Notes in Computer Science, vol 14399. Springer, Cham. https://doi.org/10.1007/978-3-031-54129-2_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-54129-2_23

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-54128-5

  • Online ISBN: 978-3-031-54129-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation