Given a problem reported in the Problem View or in the Analysis View, the quick fix proposals can be generated in three ways.
We have evaluated our quick fixes by generating mutants of four transformations in order to create a wide range of problems. Then, we use our speculative analysis to check whether each quick fix actually solves its targeted problem and its impact (i.e., which new problems produces or fixes as a side-effect).
Input: Mutated transformation T Step 0: Run the static analyser on T, obtaining its list of problems Lp Step 1: Confirm or discard potential problems by finding a model witness If discarded, remove problem from Lp Step 2: Foreach P in Lp Retrieve the set of available quick fixes for P Foreach applicable quick fix Q Discard quick fix if it requires user intervention Count Q as applicable for the problem type of P Apply Q on T speculatively Copy T into Tq Apply Q on Tq Run the static analyser on Tq Confirm or discard problems for Tq, as in step 1 Compute the impact obtaining fixedProblems and newProblems If P belongs to newProblems Mark the application of Q as valid
This script has been carried out using a custom-made facility to automatically process transformations and meta-models and export the results in textual form or in Excel format. The detailed results and the data to replicate the experiments are available in the following files.
The anatlyzer.evaluation.quickfixes.sosym2015 project includes four .exp files to easily run the experiments. For example, open ant2maven.exp, select the Result tab, and press Execute. After a few minutes, the results will appear in the text editor and can be exported to Excel. The experiments may take between 10 minutes and 60 minutes.
To run the experiments Eclipse must be configured to run with
a heap space of at least 4 GB. You must change eclipse.ini adding