Skip to main content

Advertisement

SpringerLink
Log in

Refactoring practices in the context of data-intensive systems

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

Developers often refactor code to improve the maintainability and comprehension of the software. There are many studies on refactoring activities in traditional software systems. However, refactoring in data-intensive systems is not well explored. Understanding the refactoring practices of developers is important to develop efficient tool support. We conducted a longitudinal study of refactoring activities in data-access classes using 29 SQL and NoSQL database based data-intensive systems. We investigated the prevalence, co-occurrence, and evolution of data-access refactorings, and the association of data-access refactorings with data-access smells. We also conducted a manual analysis of 500 samples of data-access refactoring instances to identify the functionalities of the code that are targeted by such refactorings. Furthermore, we analyzed 500 sample data-access refactoring commits to understand the context behind the applied refactorings and explored the characteristics and contribution of developers involved in the refactorings. We also conducted a developer survey to complement our analysis on the subject systems. Our results show that data-access refactorings are prevalent and different in type. Most of the data-access refactorings target codes that implement data fetching and insertion, but they mostly do not modify data-access queries. Most of the data-access refactorings are done when adding or modifying features and during bug fixes. data-access refactoring is often performed by developers with higher development and refactoring experience. Overall, the results show that data-access refactorings focus on improving the code quality but not optimizing the underlying data-access operations by fixing data-access smells. Hence, more work is needed from the research community on providing awareness and support to practitioners on the benefits of addressing data-access smells with refactorings.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Listing 1
Listing 2
Listing 3
Listing 4
Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. https://www.mongodb.com/developer/products/mongodb/schema-design-anti-pattern-separating-data/

  2. https://riak.com/posts/technical/riak-development-anti-patterns/index.html

  3. https://bit.ly/39RXcPI

  4. https://github.com/bio2rdf/bio2rdf-scripts

  5. http://www.jnosql.org/docs/introduction.html

  6. https://github.com/tsantalis/RefactoringMiner

  7. https://dwheeler.com/sloccount/

  8. https://bit.ly/36XxUyk

  9. https://bit.ly/3EzrZ0P

  10. https://bit.ly/3z7dXCg

  11. https://bit.ly/3Jhq0Si

  12. https://bit.ly/3qyWf75

  13. https://bit.ly/3ewi3dU

  14. https://bit.ly/3HhviLN

  15. https://www.red-gate.com/hub/product-learning/sql-prompt

  16. https://sqlenlight.com/

References

  • Agrawal R, Imielinski T, Swami A (1993) Mining associations between sets of items in large databases. In: Proceedings of the ACM SIGMOD international conference on management of data, pp 207–216

  • Agrawal R, Srikant R, et al. (1994) Fast algorithms for mining association rules. In: Proc. 20th int. conf. very large data bases, VLDB, vol 1215. Citeseer, pp 487–499

  • de Almeida Filho FG, Martins ADF, Vinuto TdS, Monteiro JM, de Sousa ÍP, de Castro Machado J, Rocha LS (2019) Prevalence of bad smells in PL/SQL projects. In: Proceedings of the 27th international conference on program comprehension. IEEE Press, pp 116–121

  • Alomar EA, Peruma A, Mkaouer MW, Newman C, Ouni A, Kessentini M (2021) How we refactor and how we document it? On the use of supervised machine learning algorithms to classify refactoring documentation. Expert Syst Appl 167(114):176

    Google Scholar 

  • Arzamasova N, Schäler M, Böhm K (2018) Cleaning antipatterns in an SQL query log. IEEE Trans Knowl Data Eng 30(3):421–434

    Article  Google Scholar 

  • Brin S, Motwani R, Ullman JD, Tsur S (1997) Dynamic itemset counting and implication rules for market basket data. In: Proceedings of the 1997 ACM SIGMOD international conference on Management of data, pp 255–264

  • Chávez A, Ferreira I, Fernandes E, Cedrim D, Garcia A (2017) How does refactoring affect internal quality attributes? a multi-project study. In: Proceedings of the 31st Brazilian symposium on software engineering, pp 74–83

  • Chen CP, Zhang CY (2014) Data-intensive applications, challenges, techniques and technologies: A survey on big data. Inf Sci 275:314–347

    Article  Google Scholar 

  • Cramer H (1946) Mathematical methods of statistics. Princeton U, Press, Princeton, p 500

    MATH  Google Scholar 

  • Dig D, Comertoglu C, Marinov D, Johnson R (2006) Automated detection of refactorings in evolving components. In: European conference on object-oriented programming. Springer, pp 404–428

  • Falleri JR, Morandat F, Blanc X, Martinez M, Monperrus M (2014) Fine-grained and accurate source code differencing. In: Proceedings of the 29th ACM/IEEE international conference on Automated software engineering, pp 313–324

  • Ferreira I, Fernandes E, Cedrim D, Uchôa A, Bibiano AC, Garcia A, Correia JL, Santos F, Nunes G, Barbosa C et al (2018) The buggy side of code refactoring: Understanding the relationship between refactorings and bugs. In: Proceedings of the 40th international conference on software engineering: companion proceeedings, pp 406–407

  • Foidl H, Felderer M, Biffl S (2019) Technical debt in data-intensive software systems. In: 2019 45th Euromicro conference on software engineering and advanced applications (SEAA). https://doi.org/10.1109/SEAA.2019.00058, pp 338–341

  • Foidl H, Felderer M, Ramler R (2022) Data smells: categories, causes and consequences, and detection of suspicious data in ai-based systems. In: Crnkovic I (ed) Proceedings of the 1st international conference on AI engineering: software engineering for AI, CAIN 2022, Pittsburgh, Pennsylvania, May 16-24, 2022. https://doi.org/10.1145/3522664.3528590. ACM, pp 229–239

  • Fowler M (2002) Refactoring: Improving the design of existing code. Extreme Program Agile Methods–XP/Agil Universe 2002:256

    Article  MATH  Google Scholar 

  • Hummel O, Eichelberger H, Giloj A, Werle D, Schmid K (2018) A collection of software engineering challenges for big data system development. In: 2018 44th Euromicro conference on software engineering and advanced applications (SEAA). https://doi.org/10.1109/SEAA.2018.00066, pp 362–369

  • Hummel O, Eichelberger H, Giloj A, Werle D, Schmid K (2018) A collection of software engineering challenges for big data system development. In: 2018 44th euromicro conference on software engineering and advanced applications (SEAA). IEEE, pp 362–369

  • Iammarino M, Zampetti F, Aversano L, Di Penta M (2019) Self-admitted technical debt removal and refactoring actions: Co-occurrence or more?. In: 2019 IEEE international conference on software maintenance and evolution (ICSME). IEEE, pp 186–190

  • Karwin B (2010) SQL Antipatterns: Avoiding the pitfalls of database programming Pragmatic Bookshelf

  • Khumnin P, Senivongse T (2017) SQL antipatterns detection and database refactoring process. In: 2017 18th IEEE/ACIS international conference on software engineering, artificial intelligence, networking and parallel/distributed computing (SNPD), pp 199–205

  • Kim M, Gee M, Loh A, Rachatasumrit N (2010) Ref-finder: a refactoring reconstruction tool based on logic query templates. In: Proceedings of the 18th ACM SIGSOFT international symposium on Foundations of software engineering, pp 371–372

  • Kurtanović Z, Maalej W (2018) On user rationale in software engineering. Requir Eng 23(3):357–379

    Article  Google Scholar 

  • Mahmoudi M, Nadi S, Tsantalis N (2019) Are refactorings to blame? an empirical study of refactorings in merge conflicts. In: 2019 IEEE 26th international conference on software analysis, evolution and reengineering (SANER). IEEE, pp 151–162

  • McDonald N, Goggins S (2013) Performance and participation in open source software on Github. In: CHI’13 extended abstracts on human factors in computing systems, pp 139–144

  • Meurice L, Nagy C, Cleve A (2016) Static analysis of dynamic database usage in Java systems. In: International conference on advanced information systems engineering. Springer, pp 491–506

  • Murphy GC, Kersten M, Findlater L (2006) How are java software developers using the elipse ide? IEEE Softw 23(4):76–83

    Article  Google Scholar 

  • Muse BA, Khomh F, Antoniol G (2022) Do developers refactor data access code? an empirical study. In: the 29th IEEE international conference on software analysis, evolution and reengineering(SANER)

  • Muse BA, Khomh F, Antoniol G (2022) Replication package: Refactoring Practices in the Context of Data-intensive Systems. https://doi.org/10.5281/zenodo.7140854

  • Muse BA, Nagy C, Cleve A, Khomh F, Antoniol G (2022) FIXME: synchronize with database! an empirical study of data access self-admitted technical debt. Empir Softw Eng 27(6):130. https://doi.org/10.1007/s10664-022-10119-4

    Article  Google Scholar 

  • Muse BA, Rahman MM, Nagy C, Cleve A, Khomh F, Antoniol G (2020) On the prevalence, impact, and evolution of SQL code smells in data-intensive systems. In: Proceedings of the 17th international conference on mining software repositories, pp 327–338

  • Nagy C, Cleve A (2017) A static code smell detector for SQL queries embedded in Java code. In: 2017 IEEE 17th international working conference on source code analysis and manipulation (SCAM). IEEE, pp 147–152

  • Nagy C, Cleve A (2018) SQLInspect: A static analyzer to inspect database usage in Java applications. In: Proceedings of the 40th international conference on software engineering: companion proceedings. ACM, pp 93–96

  • Park B, Rao DL, Gudivada VN (2021) Dangers of bias in data-intensive information systems. In: Deshpande P, Abraham A, Iyer B, Ma K (eds) Next generation information processing system. Springer Singapore, Singapore, pp 259–271

  • Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: Machine learning in Python. J Mach Learn Res 12:2825–2830

    MathSciNet  MATH  Google Scholar 

  • Perez-Palacin D, Merseguer J, Requeno JI, Guerriero M, Di Nitto E, Tamburri DA (2019) A UML profile for the design, quality assessment and deployment of data-intensive applications. Softw Syst Model 18(6):3577–3614

    Article  Google Scholar 

  • Peruma A (2019) A preliminary study of android refactorings. In: 2019 IEEE/ACM 6th international conference on mobile software engineering and systems (MOBILESoft). IEEE, pp 148–149

  • Peruma A, Mkaouer MW, Decker MJ, Newman CD (2018) An empirical investigation of how and why developers rename identifiers. In: Proceedings of the 2nd international workshop on refactoring, pp 26–33

  • Peruma A, Mkaouer MW, Decker MJ, Newman CD (2020) Contextualizing rename decisions using refactorings, commit messages, and data types. J Syst Softw 169(110):704

    Google Scholar 

  • Piatetsky S, Frawley G, William J (1991) Discovery, analysis and presentation of strong rules knowledge discovery in databases

  • Sharma T, Fragkoulis M, Rizou S, Bruntink M, Spinellis D (2018) Smelly relations: Measuring and understanding database schema quality. In: 2018 IEEE/ACM 40th international conference on software engineering: software engineering in practice track (ICSE-SEIP), pp 55–64

  • Shome A, Cruz L, van Deursen A (2022) Data smells in public datasets. In: Crnkovic I (ed) Proceedings of the 1st international conference on AI engineering: software engineering for AI, CAIN 2022, Pittsburgh, Pennsylvania, May 16-24, 2022. https://doi.org/10.1145/3522664.3528621. ACM, pp 205–216

  • Silva D, Silva J, Santos GJDS, Terra R, Valente MTO (2020) Refdiff 2.0: A multi-language refactoring detection tool. IEEE Trans Softw Eng

  • Silva D, Tsantalis N, Valente MT (2016) Why we refactor? confessions of Github contributors. In: Proceedings of the 2016 24th ACM SIGSOFT international symposium on foundations of software engineering, pp 858–870

  • Spadini D, Aniche M, Bacchelli A (2018) PyDriller: Python Framework for Mining Software Repositories. https://doi.org/10.1145/3236024.3264598

  • Tsantalis N, Ketkar A, Dig D (2020) Refactoringminer 2.0. IEEE Transactions on Software Engineering. https://doi.org/10.1109/TSE.2020.3007722

  • Vassallo C, Grano G, Palomba F, Gall HC, Bacchelli A (2019) A large-scale empirical exploration on refactoring activities in open source software projects. Sci Comput Program 180:1–15

    Article  Google Scholar 

  • Wohlin C, Runeson P, Höst M, Ohlsson MC, Regnell B, Wesslén A (2012) Experimentation in software engineering. Springer Science & Business Media, Berlin

    Book  MATH  Google Scholar 

  • Zhou C, Kuttal SK, Ahmed I (2018) What makes a good developer? an empirical study of developers’ technical and social competencies. In: 2018 IEEE symposium on visual languages and human-centric computing (VL/HCC). IEEE, pp 319–321

  • Zhou J, Wang S, Bezemer CP, Zou Y, Hassan AE (2021) Studying the association between bountysource bounties and the issue-addressing likelihood of Github issue reports. IEEE Trans Softw Eng 47(12):2919–2933. https://doi.org/10.1109/TSE.2020.2974469

    Article  Google Scholar 

Download references

Funding

This work is partly funded by the Natural Sciences and Engineering Research Council of Canada (NSERC) and the Fonds de Recherché du Québec (FRQ).

Author information

Authors and Affiliations

  1. Polytechnique Montréal, Quebec, QC, Canada

    Biruk Asmare Muse, Foutse Khomh & Giuliano Antoniol

Authors
  1. Biruk Asmare Muse
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Foutse Khomh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Giuliano Antoniol
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Biruk Asmare Muse.

Ethics declarations

Conflict of Interests

The authors have no competing interests to declare that are relevant to the content of this article.

Additional information

Communicated by: Mehrdad Sabetzadeh

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Muse, B.A., Khomh, F. & Antoniol, G. Refactoring practices in the context of data-intensive systems. Empir Software Eng 28, 46 (2023). https://doi.org/10.1007/s10664-022-10271-x

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10664-022-10271-x

Keywords

Search

Navigation