Skip to main content
SpringerLink
Log in

SoK: A Tale of Reduction, Security, and Correctness - Evaluating Program Debloating Paradigms and Their Compositions

  • Conference paper
  • First Online:
Computer Security – ESORICS 2023 (ESORICS 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14347))

Included in the following conference series:

  • 307 Accesses

Abstract

Automated software debloating of program source or binary code has tremendous potential to improve both application performance and security. Unfortunately, measuring and comparing the effectiveness of various debloating methods is challenging due to the absence of a universal benchmarking platform that can accommodate diverse approaches. In this paper, we first present \({\textsc {DebloatBench}}_{\textrm{A}}\) (Debloating benchmark for applications), an extensible and sustainable benchmarking platform that enables comparison of different research techniques. Then, we perform a holistic comparison of the techniques to assess the current progress.

In the current version, we integrated four software debloating research tools: Chisel, Occam, Razor, and Piece-wise. Each tool is representative of a different class of debloaters: program source, compiler intermediate representation, executable binary, and external library. Our evaluation revealed interesting insights (i.e., hidden and explicit tradeoffs) about existing techniques, which might inspire future research. For example, all the binaries produced by Occam and Piece-Wise were correct, while Chisel significantly outperformed others in binary size and Gadget class reductions. In a first-of-its-kind composition, we also combined multiple debloaters to debloat a single binary. Our performance evaluation showed that, in both ASLR-proof and Turing-complete gadget expressively cases, several compositions (e.g., Chisel-Occam, Chisel-Occam-Razor) significantly outperformed the best-performing single tool (i.e., Chisel).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Some of the test cases are taken from Razor Benchmarks [37].

References

  1. Busy box. https://busybox.net/

  2. Depclean. https://github.com/castor-software/depclean

  3. Ropgadget tool. https://github.com/JonathanSalwan/ROPgadget

  4. Abubakar, M., Ahmad, A., Fonseca, P., Xu, D.: Shard: fine-grained kernel specialization with context-aware hardening. In: 28th USENIX Security Symposium (2019)

    Google Scholar 

  5. Agadakos, I., Jin, D., Williams-King, D., Kemerlis, V.P., Portokalidis, G.: Nibbler: debloating binary shared libraries. In: ACSAC, pp. 70–83 (2019)

    Google Scholar 

  6. Ahmad, A., Anwar, M., Sharif, H., Gehani, A., Zaffar, F.: Trimmer: context-specific code reduction. In: 37th IEEE/ACM Conference on Automated Software Engineering (ASE) (2022)

    Google Scholar 

  7. Ahmad, A., et al.: Trimmer: an automated system for configuration-based software debloating. IEEE Trans. Softw. Eng. (TSE) 48(9) (2022)

    Google Scholar 

  8. Alhanahnah, M., Jain, R., Rastogi, V., Jha, S., Reps, T.: Lightweight, multi-stage, compiler-assisted application specialization. In: 7th European Symposium on Security and Privacy. IEEE (2022)

    Google Scholar 

  9. Azad, B.A., Laperdrix, P., Nikiforakis, N.: Less is more: quantifying the security benefits of debloating web applications. In: 28th USENIX Security Symposium (2019)

    Google Scholar 

  10. Bessey, A., et al.: A few billion lines of code later: using static analysis to find bugs in the real world. Commun. ACM 53(2), 66–75 (2010)

    Article  Google Scholar 

  11. Bhattacharya, S., Rajamani, K., Gopinath, K., Gupta, M.: The interplay of software bloat, hardware energy proportionality and system bottlenecks. In: HotPower’11, pp. 1–5 (2011)

    Google Scholar 

  12. Bierbaumer, B., Kirsch, J., Kittel, T., Francillon, A., Zarras, A.: Smashing the stack protector for fun and profit. In: Janczewski, L.J., Kutyłowski, M. (eds.) SEC 2018. IAICT, vol. 529, pp. 293–306. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-99828-2_21

    Chapter  Google Scholar 

  13. Biswas, P., Burow, N., Payer, M.: Code specialization through dynamic feature observation. In: Joshi, A., Carminati, B., Verma, R.M. (eds.) CODASPY ’21, pp. 257–268 (2021)

    Google Scholar 

  14. Brown, M.D., Pande, S.: Is less really more? Towards better metrics for measuring security improvements realized through software debloating. In: 12th USENIX Workshop (CSET 19) (2019)

    Google Scholar 

  15. Brown, M.D., Pruett, M., Bigelow, R., Mururu, G., Pande, S.: Not so fast: understanding and mitigating negative impacts of compiler optimizations on code reuse gadget sets. Proc. ACM Program. Lang. 5(OOPSLA) (2021)

    Google Scholar 

  16. Bruce, B.R., Zhang, T., Arora, J., Xu, G.H., Kim, M.: JShrink: in-depth investigation into debloating modern Java applications. In: Devanbu, P., Cohen, M.B., Zimmermann, T. (eds.) ESEC/FSE, pp. 135–146. ACM (2020)

    Google Scholar 

  17. Chaqfeh, M., Zaki, Y., Hu, J., Subramanian, L.: JScleaner: de-cluttering mobile webpages through Javascript cleanup. In: Huang, Y., King, I., Liu, T., van Steen, M. (eds.) WWW, pp. 763–773. ACM/IW3C2 (2020)

    Google Scholar 

  18. Dewan, A., Rao, P., Sodhi, B., Kapur, R.: BloatLibD: detecting bloat libraries in Java applications. In: 16th Conference on the Evaluation of Novel Approaches to Software Engineering (2021)

    Google Scholar 

  19. GuardSquare: Proguard. https://github.com/Guardsquare/proguard

  20. Guo, P.J., Engler, D.R.: CDE: using system call interposition to automatically create portable software packages. In: Nieh, J., Waldspurger, C.A. (eds.) USENIX ATC (2011)

    Google Scholar 

  21. Hassan, M., et al.: Evaluating container debloaters. In: IEEE Secure Development Conference, SecDev 2023, Atlanta, GA, USA, 18–20 October 2023. IEEE (2023)

    Google Scholar 

  22. Heo, K., Lee, W., Pashakhanloo, P., Naik, M.: Effective program debloating via reinforcement learning. In: 2018 ACM CCS, pp. 380–394 (2018)

    Google Scholar 

  23. Holzmann, G.J.: Code inflation. IEEE Softw. 32(2), 10–13 (2015)

    Article  MathSciNet  Google Scholar 

  24. Homescu, A., Stewart, M., Larsen, P., Brunthaler, S., Franz, M.: Microgadgets: size does matter in Turing-Complete Return-Oriented programming. In: USENIX WOOT ’12 (2012)

    Google Scholar 

  25. Javed, F., Afzal, M.K., Sharif, M., Kim, B.S.: Internet of things (IoT) operating systems support, networking technologies, applications, and challenges: a comparative review. IEEE CS &T 20(3), 2062–2100 (2018)

    Google Scholar 

  26. Jiang, Y., Wu, D., Liu, P.: JRed: program customization and bloatware mitigation based on static analysis. In: IEEE COMPSAC, pp. 12–21 (2016)

    Google Scholar 

  27. Jones, N.D.: An introduction to partial evaluation. ACM Comput. Surv. 28(3), 480–503 (1996)

    Article  Google Scholar 

  28. Kalhauge, C.G., Palsberg, J.: Logical bytecode reduction. In: ACM SIGPLAN PLDI, pp. 1003–1016. ACM (2021)

    Google Scholar 

  29. Kuo, H., et al.: Multik: a framework for orchestrating multiple specialized kernels. CoRR abs/1903.06889 (2019)

    Google Scholar 

  30. Kupoluyi, T., Chaqfeh, M., Varvello, M., Hashmi, W., Subramanian, L., Zaki, Y.: Muzeel: a dynamic Javascript analyzer for dead code elimination in today’s web. arXiv preprint arXiv:2106.08948 (2021)

  31. Malecha, G., Gehani, A., Shankar, N.: Automated software winnowing. In: 30th ACM Symposium on Applied Computing (SAC) (2015)

    Google Scholar 

  32. Martin, R.C.: The open-closed principle. More C++ Gems 19(96) (1996)

    Google Scholar 

  33. Navas, J., Gehani, A.: OCCAMv2: combining static and dynamic analysis for effective and efficient whole program specialization. Commun. ACM 66(4) (2023)

    Google Scholar 

  34. Necula, G.C., McPeak, S., Rahul, S.P., Weimer, W.: CIL: intermediate language and tools for analysis and transformation of C programs. In: Horspool, R.N. (ed.) Conference on Compiler Construction (2002)

    Google Scholar 

  35. Obbink, N.G., Malavolta, I., Scoccia, G.L., Lago, P.: An extensible approach for taming the challenges of Javascript dead code elimination. In: Oliveto, R., Penta, M.D., Shepherd, D.C. (eds.) Conference on Software Analysis, Evolution and Reengineering (2018)

    Google Scholar 

  36. Porter, C., Mururu, G., Barua, P., Pande, S.: Blankit library debloating: getting what you want instead of cutting what you don’t. In: ACM SIGPLAN PLDI, pp. 164–180 (2020)

    Google Scholar 

  37. Qian, C., Hu, H., Alharthi, M., Chung, P.H., Kim, T., Lee, W.: Razor: a framework for post-deployment software debloating. In: USENIX Security (2019)

    Google Scholar 

  38. Quach, A., Erinfolami, R., Demicco, D., Prakash, A.: A multi-OS cross-layer study of bloating in user programs, kernel and managed execution environments. In: Kim, T., Wang, C., Wu, D. (eds.) Workshop on Forming an Ecosystem Around Software Transformation (2017)

    Google Scholar 

  39. Quach, A., Prakash, A., Yan, L.: Debloating software through piece-wise compilation and loading. In: USENIX Security, pp. 869–886 (2018)

    Google Scholar 

  40. Ramanathan, M.K., Clapp, L., Barik, R., Sridharan, M.: Piranha: reducing feature flag debt at UBER. In: Rothermel, G., Bae, D. (eds.) ICSE-SEIP, pp. 221–230. ACM (2020)

    Google Scholar 

  41. Rastogi, V., Davidson, D., Carli, L.D., Jha, S., McDaniel, P.D.: Cimplifier: automatically debloating containers. In: Bodden, E., Schäfer, W., van Deursen, A., Zisman, A. (eds.) European Software Engineering Conference/Foundations of Software Engineering (2017)

    Google Scholar 

  42. Regehr, J., Chen, Y., Cuoq, P., Eide, E., Ellison, C., Yang, X.: Test-case reduction for C compiler bugs. In: ACM PLDI, pp. 335–346 (2012)

    Google Scholar 

  43. Shacham, H.: The geometry of innocent flesh on the bone: return-into-libc without function calls (on the x86). In: Ning, P., di Vimercati, S.D.C., Syverson, P.F. (eds.) ACM CCS 2007, pp. 552–561. ACM (2007)

    Google Scholar 

  44. Sharif, H., Abubakar, M., Gehani, A., Zaffar, F.: Trimmer: application specialization for code debloating. In: 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE) (2018)

    Google Scholar 

  45. Smowton, C.S.: I/O Optimisation and elimination via partial evaluation. Technical report, UC, CL, December 2014

    Google Scholar 

  46. Sun, C., Li, Y., Zhang, Q., Gu, T., Su, Z.: Perses: syntax-guided program reduction. In: ICSE 2018, pp. 361–371 (2018)

    Google Scholar 

  47. Tip, F., Laffra, C., Sweeney, P.F., Streeter, D.: Practical experience with an application extractor for Java. SIGPLAN Not. 34(10), 292–305 (1999)

    Google Scholar 

  48. Turcotte, A., Arteca, E., Mishra, A., Alimadadi, S., Tip, F.: Stubbifier: debloating dynamic server-side Javascript applications. CoRR abs/2110.14162 (2021)

    Google Scholar 

  49. Vázquez, H.C., Bergel, A., Vidal, S.A., Pace, J.A.D., Marcos, C.A.: Slimming Javascript applications: an approach for removing unused functions from Javascript libraries. Inf. Softw. Technol. 107, 18–29 (2019)

    Article  Google Scholar 

  50. Wu, J., et al.: LightBlue: automatic profile-aware debloating of Bluetooth stacks. In: 30th USENIX Security Symposium (2021)

    Google Scholar 

  51. Xin, Q., Kim, M., Zhang, Q., Orso, A.: Program debloating via stochastic optimization. In: ICSE-NIER ’20, pp. 65–68 (2020)

    Google Scholar 

  52. Xin, Q., Kim, M., Zhang, Q., Orso, A.: Subdomain-based generality-aware debloating. In: 35th IEEE/ACM ASE (2020)

    Google Scholar 

  53. Xu, G., Mitchell, N., Arnold, M., Rountev, A., Sevitsky, G.: Software bloat analysis: finding, removing, and preventing performance problems in modern large-scale object-oriented applications. In: FSE/SDP, pp. 421–426 (2010)

    Google Scholar 

Download references

Acknowledgements

This material is based upon work supported by the National Science Foundation (NSF) under Grant ACI-1440800 and the Office of Naval Research (ONR) under Contracts N68335-17-C-0558 and N00014-18-1-2660. Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of NSF or ONR. We thank Muhammad Hassan, Abdullah Naveed, Talha Tahir, Muhammad Farrukh, and Ahsan Amin for their help in preparing and testing the large application suite.

Author information

Authors and Affiliations

  1. University of Arizona, Tucson, USA

    Muaz Ali & Sazzadur Rahaman

  2. Stony Brook University, Stony Brook, USA

    Muhammad Muzammil

  3. Georgia Institute of Technology, Atlanta, USA

    Faraz Karim

  4. Boston University, Boston, USA

    Ayesha Naeem

  5. Tufts University, Medford, USA

    Rukhshan Haroon

  6. LUMS, Lahore, Pakistan

    Muhammad Haris, Waseem Sabir & Fareed Zaffar

  7. University of Pittsburgh, Pittsburgh, USA

    Huzaifah Nadeem

  8. Data Security Technologies, Richardson, USA

    Fahad Shaon

  9. SRI International, Menlo Park, USA

    Vinod Yegneswaran & Ashish Gehani

Authors
  1. Muaz Ali
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Muhammad Muzammil
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Faraz Karim
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ayesha Naeem
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Rukhshan Haroon
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Muhammad Haris
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Huzaifah Nadeem
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Waseem Sabir
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Fahad Shaon
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Fareed Zaffar
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Vinod Yegneswaran
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Ashish Gehani
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Sazzadur Rahaman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Muaz Ali .

Editor information

Editors and Affiliations

  1. University of California, Irvine, CA, USA

    Gene Tsudik

  2. University of Padua, Padua, Italy

    Mauro Conti

  3. Delft University of Technology, Delft, The Netherlands

    Kaitai Liang

  4. Delft University of Technology, Delft, The Netherlands

    Georgios Smaragdakis

Appendix

Appendix

Table 3. The number of train and correctness testcases we used during our experiment for each of the input programs.
Full size table
Fig. 10.
figure 10

For coreutils, (a) average fractions of test passed for different heuristics in Razor and (b) relationship between the time taken to train Razor and the number of train cases. For non-coreutils, (c) ASLR-proof ROP gadget expressivity and (d) Turing-complete ROP gadget expressivity reduction.

Full size image

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ali, M. et al. (2024). SoK: A Tale of Reduction, Security, and Correctness - Evaluating Program Debloating Paradigms and Their Compositions. In: Tsudik, G., Conti, M., Liang, K., Smaragdakis, G. (eds) Computer Security – ESORICS 2023. ESORICS 2023. Lecture Notes in Computer Science, vol 14347. Springer, Cham. https://doi.org/10.1007/978-3-031-51482-1_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-51482-1_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-51481-4

  • Online ISBN: 978-3-031-51482-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation