ABSTRACT
We leverage combined concrete and symbolic execution and several fault-localization techniques to create a uniquely powerful tool for localizing faults in PHP applications. The tool automatically generates tests that expose failures, and then automatically localizes the faults responsible for those failures, thus overcoming the limitation of previous fault-localization techniques that a test suite be available upfront. The fault-localization techniques we employ combine variations on the Tarantula algorithm with a technique based on maintaining a mapping between statements and the fragments of output they produce. We implemented these techniques in a tool called Apollo, and evaluated them by localizing 75 randomly selected faults that were exposed by automatically generated tests in four PHP applications. Our findings indicate that, using our best technique, 87.7% of the faults under consideration are localized to within 1% of all executed statements, which constitutes an almost five-fold improvement over the Tarantula algorithm.
- R. Abreu, P. Zoeteweij, and A. J. C. van Gemund. An evaluation of similarity coefficients for software fault localization. In PRDC 2006, pages 39--46, 2006. Google ScholarDigital Library
- H. Agrawal, J. R. Horgan, S. London, and W. E. Wong. Fault localization using execution slices and dataflow tests. In ISSRE, Toulouse, France, 1995.Google ScholarCross Ref
- S. Artzi, A. Kiežun, J. Dolby, F. Tip, D. Dig, A. Paradkar, and M. D. Ernst. Finding bugs in dynamic web applications. In ISSTA, pages 261--272, 2008. Google ScholarDigital Library
- S. Artzi, A. Kiežun, J. Dolby, F. Tip, D. Dig, A. Paradkar, and M. D. Ernst. Finding bugs in web applications using dynamic test generation and explicit state model checking. IEEE Transactions on Software Engineering, 2010. To appear. Google ScholarDigital Library
- C. Cadar, V. Ganesh, P. M. Pawlowski, D. L. Dill, and D. R. Engler. EXE: automatically generating inputs of death. In CCS, 2006. Google ScholarDigital Library
- T. M. Chilimbi, B. Liblit, K. K. Mehra, A. V. Nori, and K. Vaswani. Holmes: Effective statistical debugging via efficient path profiling. In ICSE, 2009. Google ScholarDigital Library
- H. Cleve and A. Zeller. Locating causes of program failures. In ICSE, pages 342--351, May 2005. Google ScholarDigital Library
- V. Dallmeier, C. Lindig, and A. Zeller. Lightweight defect localization for java. In ECOOP, pages 528--550, 2005. Google ScholarDigital Library
- P. Godefroid, N. Klarlund, and K. Sen. DART: Directed automated random testing. In PLDI, 2005. Google ScholarDigital Library
- P. Godefroid, M. Y. Levin, and D. Molnar. Automated whitebox fuzz testing. In NDSS, 2008.Google Scholar
- S. Horwitz, T. Reps, and D. Binkley. Interprocedural slicing using dependence graphs. ACM Trans. Program. Lang. Syst., 12(1):26--60, 1990. Google ScholarDigital Library
- M. Hutchins, H. Foster, T. Goradia, and T. Ostrand. Experiments of the effectiveness of dataflow- and controlflow-based test adequacy criteria. In ICSE, pages 191--200, 1994. Google ScholarDigital Library
- D. Jeffrey, N. Gupta, and R. Gupta. Fault localization using value replacement. In ISSTA, pages 167--178, 2008. Google ScholarDigital Library
- J. A. Jones and M. J. Harrold. Empirical evaluation of the tarantula automatic fault-localization technique. In ASE, pages 273--282, 2005. Google ScholarDigital Library
- J. A. Jones, M. J. Harrold, and J. Stasko. Visualization of test information to assist fault localization. In ICSE, pages 467--477, 2002. Google ScholarDigital Library
- B. Liblit, A. Aiken, A. X. Zheng, and M. I. Jordan. Bug isolation via remote program sampling. In PLDI, pages 141--154, 2003. Google ScholarDigital Library
- B. Liblit, M. Naik, A. X. Zheng, A. Aiken, and M. I. Jordan. Scalable statistical bug isolation. In PLDI'05, pages 15--26, 2005. Google ScholarDigital Library
- C. Liu, X. Yan, L. Fei, J. Han, and S. P. Midkiff. Sober: statistical model-based bug localization. In FSE, pages 286--295, 2005. Google ScholarDigital Library
- J. Lyle and M. Weiser. Automatic bug location by program slicing. In ICCEA, pages 877--883, Beijing (Peking), China, 1987.Google Scholar
- Y. Minamide. Static approximation of dynamically generated Web pages. In WWW, 2005. Google ScholarDigital Library
- H. Pan and E. H. Spafford. Heuristics for automatic localization of software faults. Technical Report SERC-TR-116-P, Purdue University, July 1992.Google Scholar
- X. Ren and B. G. Ryder. Heuristic ranking of java program edits for fault localization. In ISSTA, pages 239--249, 2007. Google ScholarDigital Library
- M. Renieris and S. P. Reiss. Fault localization with nearest neighbor queries. In ASE, pages 30--39, 2003.Google ScholarCross Ref
- R. Santelices, J. A. Jones, Y. Yu, and M. J. Harrold. Lightweight fault-localization using multiple coverage types. In ICSE, pages 56--66, 2009. Google ScholarDigital Library
- K. Sen, D. Marinov, and G. Agha. CUTE: A concolic unit testing engine for C. In FSE, 2005. Google ScholarDigital Library
- M. Stoerzer, B. G. Ryder, X. Ren, and F. Tip. Finding Failure-inducing Changes in Java Programs Using Change Classification. In FSE, pages 57--68, Portland, OR, USA, Nov. 7--9, 2006. Google ScholarDigital Library
- F. Tip. A survey of program slicing techniques. Journal of Programming Languages, 3(3):121--189, 1995.Google Scholar
- G. Wassermann, D. Yu, A. Chander, D. Dhurjati, H. Inamura, and Z. Su. Dynamic test input generation for web applications. In ISSTA, 2008. Google ScholarDigital Library
- C. Yilmaz, A. M. Paradkar, and C. Williams. Time will tell: fault localization using time spectra. In ICSE, pages 81--90, 2008. Google ScholarDigital Library
- A. Zeller. Isolating cause-effect chains from computer programs. In FSE, pages 1--10. ACM Press, November 2002. Google ScholarDigital Library
- X. Zhang, N. Gupta, and R. Gupta. Locating faults through automated predicate switching. In ICSE, pages 272--281, 2006. Google ScholarDigital Library
- Z. Zhang, W. K. Chan, T. H. Tse, B. Jiang, and X. Wang. Capturing propagation of infected program states. In ESEC/FSE, pages 43--52, 2009. Google ScholarDigital Library
Recommendations
Finding bugs in dynamic web applications
ISSTA '08: Proceedings of the 2008 international symposium on Software testing and analysisWeb script crashes and malformed dynamically-generated Web pages are common errors, and they seriously impact usability of Web applications. Current tools for Web-page validation cannot handle the dynamically-generated pages that are ubiquitous on today'...
Directed test generation for effective fault localization
ISSTA '10: Proceedings of the 19th international symposium on Software testing and analysisFault-localization techniques that apply statistical analyses to execution data gathered from multiple tests are quite effective when a large test suite is available. However, if no test suite is available, what is the best approach to generate one? ...
Empirical evaluation of the tarantula automatic fault-localization technique
ASE '05: Proceedings of the 20th IEEE/ACM International Conference on Automated Software EngineeringThe high cost of locating faults in programs has motivated the development of techniques that assist in fault localization by automating part of the process of searching for faults. Empirical studies that compare these techniques have reported the ...
Comments