Skip to main content
SpringerLink
Log in

The impact of imperfect change rules on framework API evolution identification: an empirical study

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

Software frameworks keep evolving. It is often time-consuming for developers to keep their client code up-to-date. Not all frameworks have documentation about the upgrading process. Many approaches have been proposed to ease the impact of non-documented framework evolution on developers by identifying change rules between two releases of a framework, but these change rules are imperfect, i.e., not 100 % correct. To the best of our knowledge, there is no empirical study to show the usefulness of these imperfect change rules. Therefore, we design and conduct an experiment to evaluate their impact. In the experiment, the subjects must find the replacements of 21 missing methods in the new releases of three open-source frameworks with the help of (1) all-correct, (2) imperfect, and (3) no change rules. The statistical analysis results show that the precision of the replacements found by the subjects with the three sets of change rules are significantly different. The precision with all-correct change rules is the highest while that with no change rules is the lowest, while imperfect change rules give a precision in between. The effect size of the difference between the subjects with no and imperfect change rules is large and that between the subjects with imperfect and correct change rules is moderate. The results of this study show that the change rules generated by framework API evolution approaches do help developers, even they are not always correct. The imperfect change rules can be used by developers upgrading their code when documentation is not available or as a complement to partial documentation. The moderate difference between results from subjects with imperfect and all-correct change rules also suggests that improving precision of change rules will still help developers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. http://developer.android.com/sdk/api_diff/8/changes.html

  2. http://wiki.eclipse.org/Provisional_API_Guidelines#Before_the_API_freeze

  3. http://docs.python.org/2/library/os.html#os.system

  4. http://www.jedit.org

  5. http://www.jfree.org/jfreechart

  6. http://www.jhotdraw.org

  7. http://www.ptidej.net/download/experiments/emse13a

  8. http://web.soccerlab.polymtl.ca/serveaua/experiment/ with password: ptidejexp

  9. http://www.ptidej.net/download/experiments/emse13a

  10. http://www.ptidej.net/download/experiments/emse13a

References

  • Abbes M, Khomh F, Guéhéneuc Y-G, Antoniol G (2011) An empirical study of the impact of two antipatterns, blob and spaghetti code, on program comprehension. In: Proceedings of the 2011 15th European conference on software maintenance and reengineering, ser. CSMR ’11. IEEE Computer Society, Washington, DC, USA, pp 181–190

  • Ali N, Sharafi Z, Guéhéneuc Y-G, Antoniol G (2012) An empirical study on requirements traceability using eye-tracking. In: Proceedings of the international conference on software maintenance, ser. ICSM 2012

  • Antoniol G, Penta MD, Merlo E (2004) An automatic approach to identify class evolution discontinuities. In: IWPSE ’04: Proceedings of the principles of software evolution, 7th international workshop. IEEE Computer Society, pp 31–40

  • Baker R D (1995) Modern permutation test software. In: Edgington E (ed) Randomization tests. Marcel Dekker Incorporated

  • Beckwith L, Burnett M, Wiedenbeck S, Cook C, Sorte S, Hastings M (2005) Effectiveness of end-user debugging software features: are there gender issues? In: Proceedings of the SIGCHI conference on human factors in computing systems, ser. CHI ’05, New York, NY, USA, pp 869–878

  • Businge J, Serebrenik A, van den Brand M (2013) Analyzing the eclipse api usage: putting the developer in the loop. In: CSMR, pp 37–46

  • Chow K, Notkin D (1996) Semi-automatic update of applications in response to library changes. In: Proceedings of the 1996 international conference on software maintenance. ser. ICSM 1996. IEEE Computer Society, Washington, DC, USA, p 359

  • Dagenais B, Robillard MP (2011) Recommending adaptive changes for framework evolution. ACM Trans Softw Eng Methodol 20(4):19:1–19:35

    Article  Google Scholar 

  • Demeyer S, Ducasse S, Nierstrasz O (2000) Finding refactorings via change metrics. In: OOPSLA ’00: Proceedings of the 15th ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications. ACM, New York, NY, USA, pp 166–177

  • Dig D, Comertoglu C, Marinov D, Johnson R (2006) Automated detection of refactorings in evolving components. In: ECOOP ’06: Proceedings of the 20th European conference on object-oriented programming. Springer, Berlin

  • Dig D, Manzoor K, Johnson R, Nguyen TN (2007) Refactoring-aware configuration management for object-oriented programs. In: ICSE ’07: Proceedings of the 29th international conference on Software Engineering. IEEE Computer Society, Washington, DC, USA, pp 427–436

  • Fagard RH, Staessen JA, Thijs L (1996) Advantages and disadvantages of the meta-analysis approach. J Hypertens 14(2)

  • Fluri B, Gall HC (2006) Classifying change types for qualifying change couplings. In: ICPC ’06: Proceedings of the 14th IEEE international conference on program comprehension. IEEE Computer Society, Washington, DC, USA, pp 35–45

  • Gamma E, Helm R, Johnson RE, Vlissides J (1995) Design patterns: elements of reusable object-oriented software. Addison-Wesley

  • German DM, Hassan AE (2009) License integration patterns: addressing license mismatches in component-based development. In: ICSE ’09: Proceedings of the 2009 IEEE 31st international conference on software engineering. IEEE Computer Society, Washington, DC, USA, pp 188–198

  • Godfrey MW, Zou L (2005) Using origin analysis to detect merging and splitting of source code entities. IEEE Trans Softw Eng 31(2):166–181

    Article  Google Scholar 

  • Grissom R, Kim J (2005) Effect sizes for research: a broad practical approach. Lawrence Erlbaum Associates

  • Hart SG, Stavenland LE (1988) Development of NASA-TLX (Task Load Index): results of empirical and theoretical research, pp 139–183

  • Hart SG, Staveland LE (1988) Nasa task load index (tlx) v 1.0. http://humansystems.arc.nasa.gov/groups/TLX/downloads/TLX.pdf

  • Henkel J, Diwan A (2005) Catchup: capturing and replaying refactorings to support api evolution. In: ICSE ’05: Proceedings of the 27th international conference on Software engineering. ACM, New York, NY, USA, pp 274–283

  • Kataoka Y, Ernst MD, Griswold WG, Notkin D (2001) Automated support for program refactoring using invariants. In: Proceedings of the IEEE international conference on software maintenance, ser. ICSM 2001. IEEE Computer Society, Washington, DC, USA, p 736

  • Kemper C, Overbeck C (2005) What’s new with jbuilder. In: JavaOne Sun’s 2005 worldwide java developer conference

  • Kim M, Notkin D, Grossman D (2007) Automatic inference of structural changes for matching across program versions. In: ICSE ’07: Proceedings of the 29th international conference on software engineering. IEEE Computer Society, Washington, DC, USA, pp 333–343

  • Kim S, Pan K, Whitehead EJ Jr. (2005) When functions change their names: automatic detection of origin relationships. In: WCRE ’05: Proceedings of the 12th working conference on reverse engineering. IEEE Computer Society, Washington, DC, USA, pp 143–152

  • Lawrie D, Morrell C, Feild H, Binkley DW (2007) Effective identifier names for comprehension and memory. ISSE 3(4):303–318

    Google Scholar 

  • Likert R (1932) A technique for the measurement of attitudes. Arch Psychol 22(140):1–55

    Google Scholar 

  • Malpohl G, Hunt JJ, Tichy WE (2000) Renaming detection. In: ASE ’00: Proceedings of the 15th IEEE international conference on Automated software engineering. IEEE Computer Society, Washington, DC, USA, p 73

  • Meng S, Wang X, Zhang L, Mei H (2012) A history-based matching approach to identification of framework evolution. In: Proceedings of 34th international conference on software engineering, ser. ICSE 2012, pp 353–363

  • Meyers-Levy J (1989) Gender differences in information processing: a selectivityinterpretation. In: Cafferata P, Tybout A (eds) Cognitive and affective responses to advertising. Lexington Books

  • Miller R G J (1981) Simultaneous statistical inference, 2nd ed. Springer

  • O’Donnell E, Johnson E (2001) The effects of auditor gender and task complexity on information processing efficiency. Int J Audit 5(2):91–105

    Article  Google Scholar 

  • Schäfer T, Jonas J, Mezini M (2008) Mining framework usage changes from instantiation code. In: ICSE ’08: Proceedings of the 30th international conference on software engineering. ACM, New York, NY, USA, pp 471–480

  • Sharif B, Maletic JI (2010) An eye tracking study on camelcase and underscore identifier styles. In: Proceedings of the 2010 IEEE 18th international conference on program comprehension, ser. ICPC ’10. IEEE Computer Society, Washington, DC, USA, pp 196–205

  • Sharafi Z, Soh Z , Guéhéneuc Y-G, Antoniol G (2012) Women and men - different but equal: on the impact of identifier style on source code reading. In: ICPC, pp 27–36

  • Weißgerber P, Diehl S (2006) Identifying refactorings from source-code changes. In: ASE ’06: Proceedings of the 21st IEEE/ACM international conference on automated software engineering. IEEE Computer Society, Washington, DC, USA, pp 231–240

  • Wohlin C, Runeson P, Höst M (1999) Experimentation in software engineering: an introduction. Springer

  • Wu W, Guéhéneuc Y-G, Antoniol G, Kim M (2010) Aura: a hybrid approach to identify framework evolution. In: Proceedings of the 32nd ACM/IEEE international conference on software engineering, vol 1, ser. ICSE ’10. ACM, New York, NY, USA, pp 325–334. [Online]. Available: doi:10.1145/1806799.1806848

  • Xing Z, Stroulia E (2006) Refactoring detection based on umldiff change-facts queries. In: WCRE ’06: Proceedings of the 13th Working Conference on Reverse Engineering. IEEE Computer Society, Washington, DC, USA, pp 263–274

  • Xing Z, Stroulia E (2007) API-evolution support with diff-CatchUp. IEEE Trans Softw Eng 33(12): 818–836

    Article  Google Scholar 

  • Yusuf S, Kagdi H, Maletic JI (2007) Assessing the comprehension of uml class diagrams via eye tracking. In: Proceedings of the 15th IEEE international conference on program comprehension, ser. ICPC ’07. IEEE Computer Society, Washington, DC, USA, pp 113–122

  • Zhang C, Budgen D (2012) What do we know about the effectiveness of software design patterns? Trans Softw Eng 38(5): 1213–1231. [Online]. Available: http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=5975176

    Article  Google Scholar 

Download references

Acknowledgments

The authors would like to thank all the anonymous subjects, who contributed their effort and time for this work, and the anonymous reviewers for their comments that greatly improve the quality of this article.

Author information

Authors and Affiliations

  1. Ptidej Team, DGIGL, École Polytechnique de Montréal, Montréal, Canada

    Wei Wu, Adrien Serveaux & Yann-Gaël Guéhéneuc

  2. Soccer Lab, DGIGL, École Polytechnique de Montréal, Montréal, Canada

    Wei Wu & Giuliano Antoniol

Authors
  1. Wei Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Adrien Serveaux
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yann-Gaël Guéhéneuc
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Giuliano Antoniol
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wei Wu.

Additional information

Communicated by: Laurie William

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, W., Serveaux, A., Guéhéneuc, YG. et al. The impact of imperfect change rules on framework API evolution identification: an empirical study. Empir Software Eng 20, 1126–1158 (2015). https://doi.org/10.1007/s10664-014-9317-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10664-014-9317-9

Keywords

Search

Navigation