Fuzzy Testing

PRELIMINARY DOCUMENTATION

Hudson integration

To integrate the fuzzy testing into hudson three jobs are needed:

• the explorer job
• the differ job
• and the normal job

First, a fuzzy.properties file has to be added in order to specify the parameters like the hudson URL. This file has to be in the test plugin in a fuzzy/ folder:

fuzzy.hudson.url=https://hudson.eclipse.org/hudson
fuzzy.hudson.job=emf-emfstore-fuzzytests-explorer
fuzzy.hudson.diffjob=emf.emfstore-fuzzytests-differ
fuzzy.hudson.artifact.folder=/artifact/git/org.eclipse.emf.emfstore.other/

• fuzzy.hudson.url specifies the hudson url.
• fuzzy.hudson.job specifies the hudson explorer job.
• fuzzy.hudson.diffjob specifies the hudson diff job.
• fuzzy.hudson.artifact.folder specifies the folder where hudson places the artifacted results

The execution of the tests on the server depends on the building system. For buckminster a launch config for the tests has to be added and executed as JUnit tests. The explorer executes all tests in a normal way. Then the TestReports contained in the fuzzy folder of the execution root can be artifacted. In the differ job you have to execute some code to iterate over all available configuration and build the DiffReports, which is done by the following code:

DiffGenerator diffGenerator = new DiffGenerator() ;
HudsonTestRunProvider runProvider = new HudsonTestRunProvider() ;

for (TestConfig config : runProvider.getAllConfigs()) {
 runProvider.setConfig(config) ;
 TestRun[] runs = runProvider.getTestRuns();
 diffGenerator.createDiff(runs[0], runs[1]) ;
}

Again the result of this differencing has to artifacted for usage in the normal job. The same launch config used before in the explorer job has to be added, but with a special system property. To only execute the tests selected by the differ job the flag

-DfilterTests=true

is required.