Skip to main content
Springer Nature Link
Log in

Why and what happened? Aiding bug comprehension with automated category and causal link identification

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

When a new bug report is assigned to developers, they first need to understand what the bug report expresses (what) and why this bug occurs (why). To do so, developers usually explore different bug related data sources to investigate whether there are historical bugs with similar symptoms and causes related to the bug at hand. Automatic bug classification with respect to what and why information of bugs would enable developers to narrow down their search of bug resources and improve the bug fixing productivity. To achieve this goal, we propose an approach, BugClass, which applies a deep neural network classification approach based on Hierarchical Attention Networks (HAN) to automatically classify the bugs into different what and why categories by exploiting the bug repository and commit repository. Then, we explore the causal link relationship between what and why categories to further improve the accuracy of the bug classification. Experimental results demonstrate that BugClass is effective to classify the given bug reports into what and why categories, and can be also effectively used for identifying the why category for new bugs based on the causal link relations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. https://bugzilla.mozilla.org/

  2. https://bugs.eclipse.org/bugs/

  3. https://www.bugzilla.org/

  4. https://www.atlassian.com/software/jira

  5. The definitions and taxonomy of ODC v5.2 attributes are accessible at http://researcher.watson.ibm.com/researcher/files/us-pasanth/ODC-5-2.pdf.

  6. https://standards.ieee.org/standard/1044-1993.html

  7. http://www.nltk.org/

  8. http://code.google.com/p/word2vec/

  9. https://github.com/GumTreeDiff/gumtree

  10. http://svmlight.joachims.org/svmmulticlass.html

  11. https://github.com/LogPAI/bugrepo

References

  • 2010 IEEE standard classification for software anomalies. IEEE Std 1044-2009 (Revision of IEEE Std 1044-1993) pp 1–23

  • Adamson G W, Boreham J (1974) The use of an association measure based on character structure to identify semantically related pairs of words and document titles. Information Storage and Retrieval 10(7-8):253–260

    Article  Google Scholar 

  • Agrawal R, Srikant R (1994) Fast algorithms for mining association rules in large databases. In: Bocca J B, Jarke M, Zaniolo C (eds) VLDB’94, Proceedings of 20th International Conference on Very Large Data Bases, September 12–15, 1994, Santiago de Chile, Chile, Morgan Kaufmann, pp 487–499

  • Avizienis A, Laprie J, Randell B, Landwehr C E (2004) Basic concepts and taxonomy of dependable and secure computing. IEEE Trans Dependable Sec Comput 1(1):11–33

    Article  Google Scholar 

  • Azmi M, Runger G C, Berrado A (2019) Interpretable regularized class association rules algorithm for classification in a categorical data space. Inf Sci 483:313–331

    Article  Google Scholar 

  • Basili V R, Selby R W (1987) Comparing the effectiveness of software testing strategies. IEEE Trans Software Eng 13(12):1278–1296

    Article  Google Scholar 

  • Beizer B (1990) Software testing techniques. 2 edn, Van Nostrand Reinhold

  • Bian P, Liang B, Shi W, Huang J, Cai Y (2018) Nar-miner: discovering negative association rules from code for bug detection. In: Leavens G T, Garcia A, Pasareanu C S (eds) Proceedings of the 2018 ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/SIGSOFT FSE 2018, Lake Buena Vista, FL, USA, November 04-09, ACM, pp 411–422

  • Bȯhme M, Soremekun E O, Chattopadhyay S, Ugherughe E, Zeller A (2017) Where is the bug and how is it fixed? an experiment with practitioners. In: Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering, ESEC/FSE 2017, Paderborn, Germany, September 4-8, 2017, pp 117–128

  • Bugde S, Nagappan N, Rajamani S K, Ramalingam G (2008) Global software servicing: Observational experiences at microsoft. In: 3rd IEEE International Conference on Global Software Engineering, ICGSE 2008, Bangalore, India, 17-20 August, vol 2008, pp 182–191

  • Campos E C, de Almeida Maia M (2017) Common bug-fix patterns: A large-scale observational study. In: 2017 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM 2017, Toronto, ON, Canada, November 9-10, 2017, pp 404–413

  • Catolino G, Palomba F, Zaidman A, Ferrucci F (2019) Not all bugs are the same: Understanding, characterizing, and classifying bug types. J Syst Softw 152:165–181

    Article  Google Scholar 

  • Chaparro O, Lu J, Zampetti F, Moreno L, Penta M D, Marcus A, Bavota G, Ng V (2017) Detecting missing information in bug descriptions. In: Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering, ESEC/FSE 2017, Paderborn, Germany, September 4-8, 2017, pp 396–407

  • Chawathe SS, Rajaraman A, Garcia-Molina H, Widom J (1996) Change detection in hierarchically structured information. In: Proceedings of the 1996 ACM SIGMOD International Conference on Management of Data, Montreal, Quebec, Canada, June 4-6, 1996, pp 493–504

  • Chillarege R, Bhandari I S, Chaar J K, Halliday M J, Moebus D S, Ray B K, Wong M (1992) Orthogonal defect classification - A concept for in-process measurements. IEEE Trans Software Eng 18(11):943–956

    Article  Google Scholar 

  • Davies S, Roper M (2014) What’s in a bug report?. In: 2014 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM ’14, Torino, Italy, September 18-19, 2014, pp 26:1–26:10

  • Dey R, Salem F M (2017) Gate-variants of gated recurrent unit (GRU) neural networks. arXiv:1701.05923

  • Falleri J, Morandat F, Blanc X, Martinez M, Monperrus M (2014) Fine-grained and accurate source code differencing. In: ACM/IEEE International Conference on Automated Software Engineering, ASE ’14, Vasteras, Sweden - September 15 - 19, 2014, pp 313–324

  • Gao S, Young M T, Qiu J X, Yoon H, Christian J B, Fearn P A, Tourassi G D, Ramanthan A (2018) Hierarchical attention networks for information extraction from cancer pathology reports. JAMIA 25(3):321–330

    Google Scholar 

  • Ge R, Huang F, Jin C, Yuan Y (2015) Escaping from saddle points - online stochastic gradient for tensor decomposition. In: Proceedings of The 28th Conference on Learning Theory, COLT 2015, Paris, France, July, 3-6, 2015, pp 797–842

  • Hamill M (2015) Exploring fault types, detection activities, and failure severity in an evolving safety-critical software system. Soft Quality J 23(2):229–265

    Article  Google Scholar 

  • Han J, Kamber M, Pei J (2011) Data Mining: Concepts and Techniques, 3rd edn. Morgan Kaufmann

  • He J, Wang B, Fu M, Yang T, Zhao X (2019) Hierarchical attention and knowledge matching networks with information enhancement for end-to-end task-oriented dialog systems. IEEE Access 7:18871–18883

    Article  Google Scholar 

  • Hermann KM, Blunsom P (2014) Multilingual models for compositional distributed semantics. In: Proceedings of the 52nd annual meeting of the association for computational linguistics, ACL 2014, June 22-27, 2014, Baltimore, MD, USA, vol 1. Long Papers, pp 58–68

  • Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  • Hu X, Li G, Xia X, Lo D, Jin Z (2018) Deep code comment generation. In: Proceedings of the 26th Conference on Program Comprehension, ICPC 2018, Gothenburg, Sweden, May 27–28, 2018, pp 200–210

  • Huang L, Ng V, Persing I, Chen M, Li Z, Geng R, Tian J (2015) Autoodc: Automated generation of orthogonal defect classifications. Autom Softw Eng 22(1):3–46

    Article  Google Scholar 

  • Humphrey W S (1995) A discipline for software engineering. Series in software engineering. Addison-Wesley

  • Jiang J, Xiong Y, Zhang H, Gao Q, Chen X (2018) Shaping program repair space with existing patches and similar code. In: Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2018, Amsterdam, The Netherlands, July 16-21, 2018, pp 298–309

  • Joachims T (1998) Text categorization with support vector machines: Learning with many relevant features. In: Machine Learning: ECML-98, 10th European Conference on Machine Learning, Chemnitz, Germany, April 21-23, 1998, Proceedings, pp 137–142

  • Kalchbrenner N, Grefenstette E, Blunsom P (2014) A convolutional neural network for modelling sentences. In: Proceedings of the 52nd annual meeting of the association for Computational Linguistics, ACL 2014, June 22-27, 2014, Baltimore, MD, USA, Volume 1: Long Papers, pp 655–665

  • Kim S, Whitehead Jr. E J W (2006) How long did it take to fix bugs?. In: Proceedings of the 2006 International Workshop on Mining Software Repositories, MSR 2006, Shanghai, China, May 22-23, 2006, pp 173–174

  • Kim Y (2014) Convolutional neural networks for sentence classification. In: Moschitti A, Pang B, Daelemans W, Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing EMNLP 2014 October 25-29 (eds). ACL, pp 1746–1751

  • Le X D, Chu D, Lo D, Le Goues C, Visser W (2017) S3: syntax- and semantic-guided repair synthesis via programming by examples. In: Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering, ESEC/FSE 2017, Paderborn, Germany, September 4-8, 2017, pp 593–604

  • Le X D, Thung F, Lo D, Le Goues C (2018) Overfitting in semantics-based automated program repair. Empir Softw Eng 23(5):3007–3033

    Article  Google Scholar 

  • Le Goues C, Holtschulte N, Smith EK, Brun Y, Devanbu PT, Forrest S, Weimer W (2015) The manybugs and introclass benchmarks for automated repair of C programs . IEEE Trans Software Eng 41(12):1236–1256

    Article  Google Scholar 

  • Li Z, Zhou Y (2005) Pr-miner: automatically extracting implicit programming rules and detecting violations in large software code. In: Wermelinger M, Gall H C (eds) Proceedings of the 10th European Software Engineering Conference held jointly with 13th ACM SIGSOFT International Symposium on Foundations of Software Engineering, 2005, Lisbon, Portugal, September 5-9, 2005, ACM, pp 306–315

  • Liang B, Bian P, Zhang Y, Shi W, You W, Cai Y (2016) Antminer: mining more bugs by reducing noise interference. In: Dillon L K, Visser W, Williams L A (eds) Proceedings of the 38th International Conference on Software Engineering, ICSE 2016, Austin, TX, USA, May 14-22, 2016. ACM, pp 333–344

  • Liu L, Yu S, Wei X, Ning Z (2018) An improved apriori-based algorithm for friends recommendation in microblog. Int J Commun Sys 31(2)

  • Mäntylä M, Lassenius C (2009) What types of defects are really discovered in code reviews? IEEE Trans Software Eng 35(3):430–448

    Article  Google Scholar 

  • Motwani M, Sankaranarayanan S, Just R, Brun Y (2018) Do automated program repair techniques repair hard and important bugs? Empir Softw Eng 23(5):2901–2947

    Article  Google Scholar 

  • urphy-Hill ER, Zimmermann T, Bird C, Nagappan N (2015) The design space of bug fixes and how developers navigate it. IEEE Trans Software Eng 41(1):65–81

    Article  Google Scholar 

  • Nayrolles M, Hamou-Lhadj A (2018) Towards a classification of bugs to facilitate software maintainability tasks. In: Proceedings of the 1st International Workshop on Software Qualities and Their Dependencies, SQUADE@ICSE 2018, Gothenburg, Sweden, May 28, 2018, pp 25–32

  • Ni Z, Li B, Sun X, Chen T, Tang B, Shi X (2020) Analyzing bug fix for automatic bug cause classification. J Syst Softw 163:110538

    Article  Google Scholar 

  • Ocariza FS, Bajaj K, Pattabiraman K, Mesbah A (2017) A study of causes and consequences of client-side javascript bugs. IEEE Trans Software Eng 43(2):128–144

    Article  Google Scholar 

  • Pappas N (2017) Popescu-Belis A Multilingual hierarchical attention networks for document classification. arXiv:.....

  • Patil S (2017) Concept-based classification of software defect reports. In: Proceedings of the 14th International Conference on Mining Software Repositories, MSR 2017, Buenos Aires, Argentina, May 20-28, 2017, pp 182–186

  • Podgurski A, Leon D, Francis P, Masri W, Minch M, Sun J, Wang B (2003) Automated support for classifying software failure reports. In: Proceedings of the 25th International Conference on Software Engineering, May 3-10. Portland, Oregon USA, 465–477, vol 2003

  • Qin H, Sun X (2018) Classifying bug reports into bugs and non-bugs using LSTM Proceedings of the Tenth Asia-Pacific Symposium on Internetware, Internetware 2018, Beijing, China, September 16-16, 2018. ACM, pp 20:1–20:4

  • Shan L, Li Z, Feng Q, Lin T, Zhou P, Zhou Y (2005) Bugbench: Benchmarks for evaluating bug detection tools. Workshop on the Evaluation of Software Defect Detection Tools

  • Shao Y, Liu B, Wang S, Li G (2020) Software defect prediction based on correlation weighted class association rule mining. Knowl Based Syst 196:105742

    Article  Google Scholar 

  • Sharma M, Tandon A, Kumari M, Singh V B (2018) Reduction of redundant rules in association rule mining-based bug assignment. arXiv:1807.08906

  • Shepperd M J (1993) Practical software metrics for project management and process improvement: R grady prentice-hall (1992) £30.95 282 pp ISBN 0 13 720384 5. Inf Softw Technol 35(11-12):701

    Article  Google Scholar 

  • Shu R, Xia T, Williams L, Menzies T (2019) Better security bug report classification via hyperparameter optimization. arXiv:1905.06872

  • Song Q, Shepperd M J, Cartwright M, Mair C (2006) Software defect association mining and defect correction effort prediction. IEEE Trans Software Eng 32(2):69–82

    Article  Google Scholar 

  • Soto M, Le Goues C (2018) Using a probabilistic model to predict bug fixes. In: 25th International Conference on Software Analysis, Evolution and Reengineering, SANER 2018, Campobasso, Italy, March 20-23, 2018, pp 221–231

  • Su Y, Luarn P, Lee Y, Yen S (2017) Creating an invalid defect classification model using text mining on server development. J Syst Softw 125:197–206

    Article  Google Scholar 

  • Sun X, Peng X, Zhang K, Liu Y, Cai Y (2019) How security bugs are fixed and what can be improved: an empirical study with mozilla. Sci China Inf Sci 62(1):19102:1–19102:3

    Article  Google Scholar 

  • Tan L, Liu C, Li Z, Wang X, Zhou Y, Zhai C (2014) Bug characteristics in open source software. Empir Softw Eng 19(6):1665–1705

    Article  Google Scholar 

  • Tarnpradab S, Liu F, Hua K A (2018) Toward extractive summarization of online forum discussions via hierarchical attention networks. arXiv:1805.10390

  • Terdchanakul P, Hata H, Phannachitta P, Matsumoto K (2017) Bug or not? bug report classification using n- gram IDF. In: 2017 IEEE International Conference on Software Maintenance and Evolution, ICSME 2017, Shanghai, China, September 17–22, 2017, pp 534–538

  • Thung F, Lo D, Jiang L (2012) Automatic defect categorization. In: 19th Working Conference on Reverse Engineering, WCRE 2012, Kingston, ON, Canada, October 15-18, 2012, pp 205–214

  • Thung F, Le X D, Lo D (2015) Active semi-supervised defect categorization. In: Proceedings of the 2015 IEEE 23rd International Conference on Program Comprehension, ICPC 2015, Florence/Firenze, Italy, May 16-24, 2015, pp 60–70

  • Timperley C S, Stepney S, Le Goues C (2018) Bugzoo: a platform for studying software bugs. In: Proceedings of the 40th International Conference on Software Engineering: Companion Proceeedings, ICSE 2018, Gothenburg, Sweden, May 27 - June 03, 2018, pp 446–447

  • van Tonder R, Le Goues C (2018) Static automated program repair for heap properties. In: Proceedings of the 40th International Conference on Software Engineering, ICSE 2018, Gothenburg, Sweden, May 27 - June, 03, 2018, pp 151–162

  • Vieira S M, Kaymak U, Sousa J M C (2010) Cohen’s kappa coefficient as a performance measure for feature selection. In: FUZZ-IEEE 2010, IEEE International Conference on Fuzzy Systems, Barcelona, Spain, 18-23 July, 2010 Proceedings. IEEE, pp 1–8

  • Wang L, Sun X, Wang J, Duan Y, Li B (2017) Construct bug knowledge graph for bug resolution: poster. In: Proceedings of the 39th international conference on software engineering, ICSE 2017, Buenos Aires, Argentina, May 20-28, 2017 - Companion, pp 189–191

  • Wang S, Khomh F, Zou Y (2016) Improving bug management using correlations in crash reports. Empir Softw Eng 21(2):337–367

    Article  Google Scholar 

  • Wang SI, Manning CD (2012) Baselines and bigrams: Simple, good sentiment and topic classification. In: The 50th annual meeting of the association for computational linguistics, proceedings of the conference, July 8-14, 2012, Jeju Island, Korea - Volume 2: Short Papers, pp 90–94

  • Wen M, Chen J, Wu R, Hao D, Cheung S (2018) Context-aware patch generation for better automated program repair. In: Proceedings of the 40th International Conference on Software Engineering, ICSE 2018, Gothenburg, Sweden, May 27 - June 03, 2018, pp 1–11

  • Xia X, Lo D (2017) An effective change recommendation approach for supplementary bug fixes. Autom Softw Eng 24(2):455–498

    Article  Google Scholar 

  • Xie Q, Wen Z, Zhu J, Gao C, Zheng Z (2018) Detecting duplicate bug reports with convolutional neural networks. In: 25th Asia-Pacific software engineering conference, APSEC 2018, Nara, Japan, December 4-7, 2018, pp 416–425

  • Xiong Y, Wang J, Yan R, Zhang J, Han S, Huang G, Zhang L (2017) Precise condition synthesis for program repair. In: Proceedings of the 39th international conference on software engineering, ICSE 2017, Buenos Aires, Argentina, May 20-28, 2017, pp 416–426

  • Yang Z, Yang D, Dyer C, He X, Smola A J, Hovy E H (2016) Hierarchical attention networks for document classification. In: NAACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego California, USA, June 12-17, 2016, pp 1480–1489

  • Ye X, Fang F, Wu J, Bunescu R C, Liu C (2018) Bug report classification using LSTM architecture for more accurate software defect locating. In: 17th IEEE International conference on machine learning and applications, ICMLA 2018, Orlando, FL, USA, December 17-20, 2018, pp 1438–1445

  • Zafar S, Malik MZ, Walia GS (2019) Towards standardizing and improving classification of bug-fix commits. In: 219 ACM/IEEE international symposium on empirical software engineering and measurement, ESEM 2019, Porto de Galinhas, Recife, Brazil, September 19–20, 2019. IEEE, pp 1–6

  • Zhang Y, Chen Y, Cheung S, Xiong Y, Zhang L (2018) An empirical study on tensorflow program bugs. In: Tip F, Bodden E (eds) Proceedings of the 27th ACM SIGSOFT international symposium on software testing and analysis, ISSTA 2018, Amsterdam, The Netherlands, July 16-21, 2018. ACM, pp 129–140

  • Zhong H, Mei H (2018) Mining repair model for exception-related bug. J Syst Softw 141:16–31

    Article  Google Scholar 

  • Zhou C, Li B, Sun X, Guo H (2018) Recognizing software bug-specific named entity in software bug repository. In: Proceedings of the 26th Conference on Program Comprehension, ICPC 2018, Gothenburg, Sweden, May 27-28, 2018, pp 108–119

  • Zhou C, Li B, Sun X (2020) Improving software bug-specific named entity recognition with deep neural network. J Syst Softw 165:110572

    Article  Google Scholar 

  • Zhou T, Sun X, Xia X, Li B, Chen X (2019) Improving defect prediction with deep forest. Inf Softw Technol 114:204–216

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China (No.61872312, No.61972335, No.62002309); the Yangzhou city-Yangzhou University Science and Technology Cooperation Fund Project (No.YZU201803, No. YZU201902); the Six Talent Peaks Project in Jiangsu Province (No. RJFW-053), the Jiangsu “333” Project; the Natural Science Foundation of the Jiangsu Higher Education Institutions of China (No. 20KJB520016); the Open Funds of State Key Laboratory for Novel Software Technology of Nanjing University (No.KFKT2020B15, No.KFKT2020B16) and Yangzhou University Top-level Talents Support Program (2019) .

Author information

Authors and Affiliations

  1. School of Information Engineering, Yangzhou University, Yangzhou, China

    Cheng Zhou, Bin Li, Xiaobing Sun & Lili Bo

  2. Taizhou University, Taizhou, China

    Cheng Zhou

  3. Nanjing University, State Key Laboratory for Novel Software Technology, Nanjing, Jiangsu, China

    Xiaobing Sun & Lili Bo

Authors
  1. Cheng Zhou
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Bin Li
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Xiaobing Sun
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Lili Bo
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Xiaobing Sun.

Additional information

Communicated by: Shaowei Wang, Tse-Hsun (Peter) Chen, Sebastian Baltes, Ivano Malavolta, Christoph Treude, and Alexander Serebrenik

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article belongs to the Topical Collection: Collective Knowledge in Software Engineering

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, C., Li, B., Sun, X. et al. Why and what happened? Aiding bug comprehension with automated category and causal link identification. Empir Software Eng 26, 118 (2021). https://doi.org/10.1007/s10664-021-10010-8

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10664-021-10010-8

Keywords