Drift: Fine-Grained Prediction of the Co-Evolution of Production and Test Code via Machine Learning

As production code evolves, test code can quickly become outdated. When test code is outdated, it may fail to capture errors in the programs under test and can lead to serious software bugs that result in significant losses for both developers and users. To ensure high software quality, it is crucial to promptly update the test code after making changes to the production code. This practice ensures that the test code and production code evolve together, reducing the likelihood of errors and ensuring the software remains reliable. However, maintaining test code can be challenging and time-consuming. To automate the identification of outdated test code, recent research has proposed Sitar, a machine learning-based method. Despite Sitar’s usefulness, it has major limitations, including its coarse prediction granularity (at class level), reliance on naming conventions to discover test code, and dependence on manually summarized features to construct machine learning models.

In this paper, we address the limitations of Sitar and propose a new machine learning-based approach Drift. Drift predicts outdated test cases at the method level. It leverages method-calling relationships to accurately infer the links between production and test code, and automatically learns features via code analysis. We evaluate Drift using 40 open-source Java projects in both within-project and cross-project scenarios, and find that Drift can achieve satisfactory prediction performances in both scenarios. We also compare Drift with existing methods for outdated test code prediction and find that Drift can significantly outperform them. For example, compared with Sitar, the accuracy of Drift is increased by about 8.5%, the F1-score is increased by about 8.3%, and more importantly, the number of test cases that developers need to check is reduced by about 75%. Therefore, our method, Drift, can predict outdated test cases more accurately at a fine-grained level, and thus better facilitate the co-evolution of production and test code.