Suggesting model transformation repairs for rule-based languages using a contract-based testing approach

Abstract

Model transformations play an essential role in most model-driven software projects. As the size and complexity of model transformations increase, their reuse, evolution and maintenance become a challenge. This work further details the ModelTransformation TEst Specification (MoTES) approach, which leverages contract-based model testing techniques to assist engineers in model transformation evolution and repairing. The main novelty of our approach is to use contract-based model transformation testing as a foundation to derive suggestions of concrete adaptation actions. MoTES uses contracts to specify the expected behaviour of the model transformation under test. These contracts are transformed into model transformations which act as oracles on input–output model pairs, previously generated by executing the transformation under test on provided input models. By further processing, the oracles’ output model, precision and recall metrics are calculated for every output pattern (testing results). These metrics are then categorised to increase the user’s ability to interpret and act on them. TheMoTES approach defines 8 cases for precision and recall values classification (test result cases). As traceability information is retained from transformation rules to each output pattern, it is possible to classify each transformation rule involved according to its impact on the metrics, e.g. the number of true positives generated. The MoTES approach defines 37 cases for these classifications, with each one linked to a particular (abstract) action suggested on a rule, such as relaxation of the rules.A comprehensive evaluation of this approach is also presented, consisting of three case studies. Two previous case studies performed over two model transformations (UML2ER and E2M) are replicated to contrast MoTES with an existing model transformation fault localisation approach. An additional case study presents how MoTES helps with the evolution of an existing model transformation in the context of a reverse engineering project. Main evaluation results show that our approach can not only detect the errors introduced in the transformations but also localise the faulty rule and suggest the proper repair actions, which significantly reduce testers’ effort. From a quantitative perspective, in the third case study, MoTES was able to indicate one faulty rule from 19 possibilities for each result case and suggest one or two repair actions from 6 possibilities for each faulty rule