Skip to main content
SpringerLink
Log in

Empirical assessment of the effort needed to attack programs protected with client/server code splitting

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

Context

Code hardening is meant to fight malicious tampering with sensitive code executed on client hosts. Code splitting is a hardening technique that moves selected chunks of code from client to server. Although widely adopted, the effective benefits of code splitting are not fully understood and thoroughly assessed.

Objective

The objective of this work is to compare non protected code vs. code splitting protected code, considering two levels of the chunk size parameter, in order to assess the effectiveness of the protection - in terms of both attack time and success rate - and to understand the attack strategy and process used to overcome the protection.

Method

We conducted an experiment with master students performing attack tasks on a small application hardened with different levels of protection. Students carried out their task working at the source code level.

Results

We observed a statistically significant effect of code splitting on the attack success rate that, on the average, was reduced from 89% with unprotected clear code to 52% with the most effective protection. The protection variant that moved some small-sized code chunks turned out to be more effective than the alternative moving fewer but larger chunks. Different strategies were identified yielding different success rates. Moreover we discovered that successful attacks exhibited different process w.r.t. failed ones.

Conclusions

We found empirical evidence of the effect of code splitting, assessed the relative magnitude, and evaluated the influence of the chunk size parameter. Moreover we extracted the process used to overcome such obfuscation technique.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

Notes

  1. SpaceGame can be obtained from SourceForge at https://sourceforge.net/projects/game/.

  2. Interested readers can find details about these functions in the file gamespace.c.

  3. GrammaTech CodeSurfer https://www.grammatech.com/products/codesurfer

  4. https://fluxicon.com/disco/

  5. Two examples of excellent reverse engineering and exploitation tools are IDA-Pro https://www.hex-rays.com/products/ida/ and radare2 https://rada.re/

  6. The syllabus reports the following course that “Computer and System Security” students have followed: Computer Science (introduction to C programming), Algorithms and Programming, Object oriented programming, Operating systems (scripting languages), at Bachelor level, and System programming, Distributed Programming I and for the Master together with the optional course named Distributed Programming II.

  7. http://security.polito.it/lioy/02krq/

  8. https://www.proprofs.com/quiz-school/story.php?title=test-your-c-skills

  9. http://www.pskills.org/c.jsp

  10. https://github.com/radare/radare2

  11. https://fluxicon.com/disco/

  12. In the Italian University system, grades are assigned on a 30 values scale.

  13. https://github.com/torsec/splitting-experiments/tree/master/analysis

  14. https://github.com/torsec/splitting-experiments

  15. https://www.antlr.org/

  16. https://jgrapht.org/

References

  • Anckaert B., Madou M., De Sutter B., De Bus B., De Bosschere K., Preneel B. (2007) Program obfuscation: a quantitative approach. In: Proceedings of ACM workshop on quality of protection, pp 15–20

  • Barak B., Goldreich O., Impagliazzo R., Rudich S., Sahai A., Vadhan S., Yang K. (2001) On the (im) possibility of obfuscating programs. Lect Notes Comput Sci 2139:19–23

    Article  MathSciNet  Google Scholar 

  • Canavese D., Regano L., Basile C., Viticchié A (2017) Estimating software obfuscation potency with artificial neural networks. In: International workshop on security and trust management, pp 193–202. Springer

  • Ceccato M., Capiluppi A., Falcarin P., Boldyreff C. (2015) A large study on the effect of code obfuscation on the quality of java code. Empir Softw Eng 20 (6):1486–1524

    Article  Google Scholar 

  • Ceccato M., Dalla Preda M., Nagra J., Collberg C., Tonella P. (2009) Trading-off security and performance in barrier slicing for remote software entrusting. Autom Softw Eng 16(2):235–261

    Article  Google Scholar 

  • Ceccato M., Dalla Preda M., Nagra J., Collberg C. S., Tonella P. (2007) Barrier slicing for remote software trusting. In: SCAM, pp 27–36

  • Ceccato M., Di Penta M., Falcarin P., Ricca F., Torchiano M., Tonella P. (2014) A family of experiments to assess the effectiveness and efficiency of source code obfuscation techniques. Empir Softw Eng 19(4):1040–1074

    Google Scholar 

  • Ceccato M., Di Penta M., Nagra J., Falcarin P., Ricca F., Torchiano M., Tonella P. (2009) The effectiveness of source code obfuscation: An experimental assessment. In: IEEE 17th international conference on program comprehension (ICPC), pp 178–187. https://doi.org/10.1109/ICPC.2009.5090041

  • Ceccato M., Tonella P., Basile C., Coppens B., De Sutter B., Falcarin P., Torchiano M. (2017) How professional hackers understand protected code while performing attack tasks. In: Proceedings of the 25th IEEE international conference on program comprehension (ICPC)

  • Ceccato M., Tonella P., Basile C., Falcarin P., Torchiano M., Coppens B., De Sutter B. (2018) Understanding the behaviour of hackers while performing attack tasks in a professional setting and in a public challenge. Empirical Software Engineering (EMSE), pp 1–47

  • Collberg C., Thomborson C., Low D. (1997) A taxonomy of obfuscating transformations. Technical report 148, Dept. of Computer Science, The Univ. of Auckland

  • Collberg C. S., Thomborson C. (2002) Watermarking, tamper-proofing, and obfuscation - tools for software protection. IEEE Trans. Softw. Eng. 28(8):735–746. https://doi.org/10.1109/TSE.2002.1027797

    Article  Google Scholar 

  • Falcarin P., Collberg C., Atallah M., Jakubowski M. (2011) Guest editors’ introduction: Software protection. IEEE Softw 28(2):24–27. 10.1109/MS.2011.34

    Article  Google Scholar 

  • Feldt R., Zimmermann T., Bergersen G. R., Falessi D., Jedlitschka A., Juristo N., Münch J., Oivo M., Runeson P., Shepperd M., Sjøberg DIK, Turhan B (2018) Four commentaries on the use of students and professionals in empirical software engineering experiments. Empir Softw Eng 23(6):3801–3820

    Article  Google Scholar 

  • Futcher L., von Solms R. (2008) Guidelines for secure software development. In: Proceedings of SAICSIT. ACM, New York, pp 56–65

  • Goto H., Mambo M., Matsumura K., Shizuya H. (2000) An approach to the objective and quantitative evaluation of tamper-resistant software. In: Third Int. workshop on information security, pp 82–96. Springer

  • Hänsch N., Schankin A., Protsenko M., Freiling F., Benenson Z. (2018) Programming experience might not help in comprehending obfuscated source code efficiently. In: 14th symposium on usable privacy and security ({SOUPS} 2018), pp 341–356

  • Krinke J. (2003) Barrier slicing and chopping. In: SCAM, pp 81–87

  • Linn C., Debray S. (2003) Obfuscation of executable code to improve resistance to static disassembly. In: Proc. ACM conf. computer and communications security, pp 290–299

  • Oppenheim A. N. (1992) Questionnaire Design, Interviewing and Attitude Measurement. Pinter, London

    Google Scholar 

  • R Core Team (2016) R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. https://www.R-project.org/

  • Sutherland I., Kalb G. E., Blyth A., Mulley G. (2006) An empirical examination of the reverse engineering process for binary files. Comput Secur 25(3):221–228

    Article  Google Scholar 

  • Visaggio C. A., Pagin G. A., Canfora G. (2013) An empirical study of metric-based methods to detect obfuscated code. International Journal of Security & Its Applications 7(2)

  • Viticchié A., Basile C., Avancini A., Ceccato M., Abrath B., Coppens B. (2016) Reactive attestation: Automatic detection and reaction to software tampering attacks. In: Proceedings of the 2016 ACM workshop on software PROtection, pp 73–84. ACM

  • Viticchié A., Regano L., Torchiano M., Basile C., Ceccato M., Tonella P., Tiella R. (2016) Assessment of source code obfuscation techniques. In: 2016 IEEE 16th international working conference on source code analysis and manipulation (SCAM), pp 11–20. IEEE

  • Weiser M. (1981) Program slicing. In: Proceedings of the 5th international conference on software engineering, ICSE ’81. http://dl.acm.org/citation.cfm?id=800078.802557. IEEE Press, Piscataway, pp 439–449

  • Wheeler B, Torchiano M (2016) lmPerm: Permutation tests for linear models. http://CRAN.R-project.org/package=lmPerm. R package version 2.1.0

  • Wohlin C., Runeson P., Hst M., Ohlsson M. C., Regnell B., Wessln A. (2012) Experimentation in software engineering, Springer Publishing Company, Incorporated

Download references

Author information

Authors and Affiliations

  1. Politecnico di Torino, Torino, Italy

    Alessio Viticchié, Leonardo Regano, Cataldo Basile & Marco Torchiano

  2. Fondazione Bruno Kessler, Trento, Italy

    Mariano Ceccato

  3. Università della Svizzera Italiana (USI), Lugano, Switzerland

    Paolo Tonella

Authors
  1. Alessio Viticchié
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Leonardo Regano
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Cataldo Basile
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Marco Torchiano
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mariano Ceccato
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Paolo Tonella
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mariano Ceccato.

Additional information

Communicated by: Saurabh Sinha

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix A: Additional Analyses

Two additional analyses are reported for the sake of completeness, though they are not central for the main findings of this work. The first analysis concerns the distribution of the time spent to complete the task and the second reports the correlation between attack task correctness and the self-claimed experience with the C language, optionally expressed by a subset of the subjects.

1.1 A.1 Elapsed Time

The distribution of elapsed time required to complete the task is reported in Fig. 18 for both successful and failed attacks. By looking at Fig. 18, we can see that the tasks with a wrong solution are all very high and compressed against the 120 min time limitation. Such phenomenon is probably due to the participant trying to use all the allotted time to attack the code without succeeding.

1.2 A.2 Experience of Participants with the C Language

Since the questionnaire about personal experience was not mandatory for the participants, we were able to collect information concerning only 35 students (out of 87).

Figure 19 reports the average correctness (with the relative 95% confidence interval) for the subjects reporting different levels of experience with the C language. We cannot observe any clear and statistically significant trend.

Fig. 19
figure 19

Experience vs. Overall Correcteness

Full size image

A more detailed view can be observed by looking at the success rate when controlling for the treatment. Figure 20 reports the average correctness by experience and treatment. Also in this case the effect of experience is difficult to identify.

Fig. 20
figure 20

Correcteness vs. Experience by Treatment

Full size image

Figure 21 reports the effect of experience on the time to complete the attack task, divided by treatment. We observe a general reduction in time as experience increases though such effect is not statistically significant (too few data points and too much variability).

Fig. 21
figure 21

Experience vs. Correcteness by Treatment

Full size image

Finally, we report the relationship between the experience and the strategy selected, divided by treatment. This is reported as a heat map in Fig. 22. Also in this case we do not observe any clear effect of experience on the strategy selection.

Fig. 22
figure 22

Experience vs. Strategy by Treatment

Full size image

Appendix B: Call Graph of the SpaceGame Application

We report in Fig. 23 for the sake of completeness, the call graph of the object of our experiment, the SpaceGame program, extracted with an ad hoc java program based on antlrFootnote 15 and JGraphT.Footnote 16

Fig. 23
figure 23

Call graph of the clear version of the SpaceGame program

Full size image

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Viticchié, A., Regano, L., Basile, C. et al. Empirical assessment of the effort needed to attack programs protected with client/server code splitting. Empir Software Eng 25, 1–48 (2020). https://doi.org/10.1007/s10664-019-09738-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10664-019-09738-1

Keywords

Search

Navigation