Skip to main content
SpringerLink
Log in

On the Soundness of Call Graph Construction in the Presence of Dynamic Language Features - A Benchmark and Tool Evaluation

  • Conference paper
  • First Online:
Programming Languages and Systems (APLAS 2018)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 11275))

Included in the following conference series:

Abstract

Static program analysis is widely used to detect bugs and vulnerabilities early in the life cycle of software. It models possible program executions without executing a program, and therefore has to deal with both false positives (precision) and false negatives (soundness). A particular challenge for sound static analysis is the presence of dynamic language features, which are prevalent in modern programming languages, and widely used in practice.

We catalogue these features for Java and present a micro-benchmark that can be used to study the recall of static analysis tools. In many cases, we provide examples of real-world usage of the respective feature. We then study the call graphs constructed with soot, wala and doop using the benchmark. We find that while none of the tools can construct a sound call graph for all benchmark programs, they all offer some support for dynamic language features.

We also discuss the notion of possible program execution that serves as the ground truth used to define both precision and soundness. It turns out that this notion is less straight-forward than expected as there are corner cases where the (language, JVM and standard library) specifications do not unambiguously define possible executions.

This work was supported by the Science for Technological Innovation (SfTI) National Science Challenge (NSC) of New Zealand (PROP-52515-NSCSEED-MAU). The work of the second author was supported by a faculty gift by Oracle Inc.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Meaning here a combination of the JVM Specification [2] and the documentation of the classes of the standard library.

  2. 2.

    https://goo.gl/1LxbSd and https://goo.gl/qYeVTd, both accessed 10 June 2018.

  3. 3.

    https://maven.apache.org/, accessed 30 August 2018.

  4. 4.

    https://goo.gl/JG9qD2, accessed 24 May 2018.

  5. 5.

    https://github.com/eclipse/openj9/pull/2240, accessed 16 August 2018.

  6. 6.

    https://xalan.apache.org, accessed 4 June 2018.

  7. 7.

    As doop does not release versions, we used a version built from commit 4a94ae3bab4edcdba068b35a6c0b8774192e59eb.

  8. 8.

    https://groups.google.com/forum/m/#!topic/soot-list/xQwsU7DlmqM, accessed 5 June 2018.

References

  1. Invokedynamic rectifier/project serializer. http://www.opal-project.de/DeveloperTools.html

  2. The Java language specification. https://docs.oracle.com/javase/specs

  3. SPECjvm2008 benchmark. www.spec.org/jvm2008

  4. Andreasen, E.S., Møller, A., Nielsen, B.B.: Systematic approaches for increasing soundness and precision of static analyzers. In: Proceedings of SOAP 2017. ACM (2017)

    Google Scholar 

  5. Bacon, D.F., Sweeney, P.F.: Fast static analysis of c++ virtual function calls. In: Proceedings of the OOPSLA 1996. ACM (1996)

    Google Scholar 

  6. Blackburn, S.M., et al.: The DaCapo benchmarks: Java benchmarking development and analysis. In: Proceedings of the OOPSLA 2006. ACM (2006)

    Google Scholar 

  7. Bodden, E.: Invokedynamic support in soot. In: Proceedings of the SOAP 2012. ACM (2012)

    Google Scholar 

  8. Bodden, E., Sewe, A., Sinschek, J., Oueslati, H., Mezini, M.: Taming reflection: aiding static analysis in the presence of reflection and custom class loaders. In: Proceedings of the ICSE 2011. ACM (2011)

    Google Scholar 

  9. Dietrich, J., Jezek, K., Rasheed, S., Tahir, A., Potanin, A.: Evil pickles: DoS attacks based on object-graph engineering. In: Proceedings of the ECOOP 2017. LZI (2017)

    Google Scholar 

  10. Dietrich, J., Schole, H., Sui, L., Tempero, E.: XCorpus-an executable corpus of Java programs. JOT 16(4), 1:1–24 (2017)

    Article  Google Scholar 

  11. Dietrich, J., Sui, L., Rasheed, S., Tahir, A.: On the construction of soundness oracles. In: Proceedings of the SOAP 2017. ACM (2017)

    Google Scholar 

  12. Dolby, J., Fink, S.J., Sridharan, M.: T.J. Watson Libraries for Analysis (2015). http://wala.sourceforge.net

  13. Ernst, M.D.: Static and dynamic analysis: synergy and duality. In: Proceedings of the WODA 2003 (2003)

    Google Scholar 

  14. Foote, B., Johnson, R.E.: Reflective facilities in Smalltalk-80. In: Proceedings of the OOPSLA 1989. ACM (1989)

    Google Scholar 

  15. Fourtounis, G., Kastrinis, G., Smaragdakis, Y.: Static analysis of Java dynamic proxies. In: Proceedings of the ISSTA 2018. ACM (2018)

    Google Scholar 

  16. Gosling, J., Joy, B., Steele, G., Bracha, G., Buckley, A.: The Java Language Specification. Java Series, Java SE 8 edn. Addison-Wesley Professional, Boston (2014)

    Google Scholar 

  17. Grech, N., Fourtounis, G., Francalanza, A., Smaragdakis, Y.: Heaps don’t lie: countering unsoundness with heap snapshots. In: Proceedings of the OOPSLA 2017. ACM (2017)

    Google Scholar 

  18. Grove, D., DeFouw, G., Dean, J., Chambers, C.: Call graph construction in object-oriented languages. In: Proceedings of the OOPSLA 1997. ACM (1997)

    Google Scholar 

  19. Holzinger, P., Triller, S., Bartel, A., Bodden, E.: An in-depth study of more than ten years of Java exploitation. In: Proceedings of the CCS 2016. ACM (2016)

    Google Scholar 

  20. Jezek, K., Dietrich, J.: Magic with dynamo-flexible cross-component linking for Java with invokedynamic. In: Proceedings of the ECOOP 2016. LZI (2016)

    Google Scholar 

  21. Kiczales, G., Hilsdale, E., Hugunin, J., Kersten, M., Palm, J., Griswold, W.G.: An overview of AspectJ. In: Knudsen, J.L. (ed.) ECOOP 2001. LNCS, vol. 2072, pp. 327–354. Springer, Heidelberg (2001). https://doi.org/10.1007/3-540-45337-7_18

    Chapter  Google Scholar 

  22. Landman, D., Serebrenik, A., Vinju, J.J.: Challenges for static analysis of Java reflection-literature review and empirical study. In: Proceedings of the ICSE 2017. IEEE (2017)

    Google Scholar 

  23. Lhoták, O.: Comparing call graphs. In: Proceedings of the PASTE 2007. ACM (2007)

    Google Scholar 

  24. Li, Y., Tan, T., Sui, Y., Xue, J.: Self-inferencing reflection resolution for Java. In: Jones, R. (ed.) ECOOP 2014. LNCS, vol. 8586, pp. 27–53. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-662-44202-9_2

    Chapter  Google Scholar 

  25. Liu, J., Li, Y., Tan, T., Xue, J.: Reflection analysis for Java: uncovering more reflective targets precisely. In: Proceedings of the ISSRE 2017. IEEE (2017)

    Google Scholar 

  26. Livshits, B., Sridharan, M., Smaragdakis, Y., Lhoták, O., Amaral, J.N., Chang, B.Y.E., Guyer, S.Z., Khedker, U.P., Møller, A., Vardoulakis, D.: In defense of soundiness: a manifesto. CACM 58(2), 44–46 (2015)

    Article  Google Scholar 

  27. Livshits, B., Whaley, J., Lam, M.S.: Reflection analysis for Java. In: Yi, K. (ed.) APLAS 2005. LNCS, vol. 3780, pp. 139–160. Springer, Heidelberg (2005). https://doi.org/10.1007/11575467_11

    Chapter  Google Scholar 

  28. Mastrangelo, L., Ponzanelli, L., Mocci, A., Lanza, M., Hauswirth, M., Nystrom, N.: Use at your own risk: the Java unsafe API in the wild. In: Proceedings of the OOPSLA 2015. ACM (2015)

    Google Scholar 

  29. Murphy, G.C., Notkin, D., Griswold, W.G., Lan, E.S.: An empirical study of static call graph extractors. ACM TOSEM 7(2), 158–191 (1998)

    Article  Google Scholar 

  30. Reif, M., Kübler, F., Eichberg, M., Mezini, M.: Systematic evaluation of the unsoundness of call graph construction algorithms for Java. In: Proceedings of the SOAP 2018. ACM (2018)

    Google Scholar 

  31. Rountev, A., Kagan, S., Gibas, M.: Evaluating the imprecision of static analysis. In: Proceedings of the PASTE 2004. ACM (2004)

    Google Scholar 

  32. Ryder, B.G.: Constructing the call graph of a program. IEEE TSE 3, 216–226 (1979)

    MathSciNet  MATH  Google Scholar 

  33. Shipilev, A.: JEP 280: indify string concatenation. http://openjdk.java.net/jeps/280

  34. Shivers, O.: Control-flow analysis of higher-order languages. Ph.D. thesis, Carnegie Mellon University (1991)

    Google Scholar 

  35. Smaragdakis, Y., Balatsouras, G., Kastrinis, G., Bravenboer, M.: More sound static handling of Java reflection. In: Feng, X., Park, S. (eds.) APLAS 2015. LNCS, vol. 9458, pp. 485–503. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-26529-2_26

    Chapter  Google Scholar 

  36. Smith, B.C.: Reflection and semantics in LISP. In: Proceedings of the POPL 1984. ACM (1984)

    Google Scholar 

  37. Sui, L., Dietrich, J., Tahir, A.: On the use of mined stack traces to improve the soundness of statically constructed call graphs. In: Proceedings of the APSEC 2017. IEEE (2017)

    Google Scholar 

  38. Sundaresan, V., et al.: Practical virtual method call resolution for Java. In: Proceedings of the OOPSLA 2000. ACM (2000)

    Google Scholar 

  39. Tempero, E., Anslow, C., Dietrich, J., Han, T., Li, J., Lumpe, M., Melton, H., Noble, J.: Qualitas corpus: a curated collection of Java code for empirical studies. In: Proceedings of the APSEC 2010 (2010)

    Google Scholar 

  40. Tip, F., Palsberg, J.: Scalable propagation-based call graph construction algorithms. In: Proceedings of the OOPSLA 2000. ACM (2000)

    Google Scholar 

Download references

Acknowledgement

We thank Paddy Krishnan, Francois Gauthier and Michael Eichberg for their comments.

Author information

Authors and Affiliations

  1. Massey University Institute of Fundamental Sciences, 4410, Palmerston North, New Zealand

    Li Sui, Michael Emery, Shawn Rasheed & Amjed Tahir

  2. Victoria University of Wellington School of Engineering and Computer Science, 6012, Wellington, New Zealand

    Jens Dietrich

Authors
  1. Li Sui
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jens Dietrich
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michael Emery
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shawn Rasheed
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Amjed Tahir
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Li Sui .

Editor information

Editors and Affiliations

  1. Korea Advanced Institute of Science and Technology, Daejeon, Korea (Republic of)

    Sukyoung Ryu

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sui, L., Dietrich, J., Emery, M., Rasheed, S., Tahir, A. (2018). On the Soundness of Call Graph Construction in the Presence of Dynamic Language Features - A Benchmark and Tool Evaluation. In: Ryu, S. (eds) Programming Languages and Systems. APLAS 2018. Lecture Notes in Computer Science(), vol 11275. Springer, Cham. https://doi.org/10.1007/978-3-030-02768-1_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-02768-1_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-02767-4

  • Online ISBN: 978-3-030-02768-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation