This repository contains nodejs script for detecting Implementation-Dependent flaky tests using NonDex tool and Order-Dependent flaky tests using iDFlakies tool. This repository contains projects
directory for cloning github repositories. Directory unfixedFlakyTests
contains list of unfixed flaky tests according to the http://mir.cs.illinois.edu/flakytests/unfixed.html. Directory mockedLogs
contains mocked logs of commands mvn edu.illinois:nondex-maven-plugin:1.1.2:nondex
and mvn testrunner:testplugin -Ddetector.detector_type=random-class-method -Ddt.randomize.rounds=10 -Ddt.detector.original_order.all_must_pass=false
for faster development. CSV report contains list of executed tests with a column that indicates whether a test is deleted, not flaky anymore, still flaky, or newly found.
- Node v9.11.2
- Java SDK 8
Run the following commands:
npm install && node script.js
(base) Miloss-MBP:flakytests miloskotlar$ node script.js
Cloning into 'iDFlakies'...
*****************************************************************
Running scripts for fastjson project
Project URL: https://github.com/alibaba/fastjson
*****************************************************************
Latest commit from master branch is a7f1f5c4ac52503cc95bf1d27498098b3093a0df
Current commit from master branch is e05e9c5e4be580691cc55a59f3256595393203a1
Cloning https://github.com/alibaba/fastjson
Cloning into 'fastjson'...
=================================================================
Running NonDex script for ID tests
Project Name: fastjson
=================================================================
Runnung projects/fastjson module. This may take up to several hours.
Analysing results, this may take up to several minutes.
=================================================================
Running iDFlakies script for OD tests
Project Name: fastjson
=================================================================
This may take up to several hours.
Runnung projects/fastjson module. This may take up to several hours.
Analysing results, this may take up to several minutes.
Detected changes in 26 tests so far...
.....
Excel report is available at report.csv
This script doesn't compile downloaded projects and doesn't have all necessary dependencies which can result in missing dependencies and errors in modules during the test executions.