Inferring Metamorphic Relations from JavaDocs: A Deep Dive into the MeMo Approach

Abstract

Identifying and selecting suitable Metamorphic Relations is a complex process since it necessitates a thorough grasp of the system under test and its problem domain. Recently, an approach supporting unit testing at the method level called MeMo was proposed. Through a module called MR-Finder, MeMo infers Equivalent Metamorphic Relations (EMRs) by identifying sentences in Javadoc’s comments that describe equivalent behaviours between different methods of the same class. MR-Finder has three main components: (i) a predefined set of 10 words that express an equivalence (S10W). (ii) A mechanism that measures the semantic similarity between two sentences by using Word Move Distance (WDM). (iii) A binary classifier that decides whether a given sentence points to an EMR. The goal of our research is to determine if MeMo’s MR-Finder module can be improved. For that purpose, we first re-build the MR-Finder module and use the same dataset provided by MeMo’s authors to verify the reported results in the original study and establish the basis for further experiments. Second, we explore two strategies, STRTG No.1 and STRTG No.2, to improve the MR-Finder. In STRTG No.1, we increase the set S10W. In STRTG No.2, we keep S10W unchanged but add a second template sentence to the MR-Finder module. We successfully re-implemented the MR-Finder module and achieved comparable results using the same S10W. Our results indicate that the overall performance of MR-Finder is very likely to improve when the initial set of equivalent words increases, i.e., with STRTG No.1.