Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MuBench pipeline check dataset checks non passed datasets too #429

Open
akwick opened this issue May 15, 2019 · 1 comment
Open

MuBench pipeline check dataset checks non passed datasets too #429

akwick opened this issue May 15, 2019 · 1 comment

Comments

@akwick
Copy link
Member

akwick commented May 15, 2019

Done:

  • Start the MUBench Interactive Shell with the data folder passed via -v
  • Execute pipeline check dataset --datasets jca-param-all

Observed output:

...
[WARNING] Unknown dataset entry "testng.677302c.grouminer-4" in dataset "FSE18-ExPrecision-TruePositives".
[WARNING] Unknown dataset entry "testng.677302c.grouminer-17" in dataset "FSE18-ExPrecision-TruePositives".
[WARNING] Unknown dataset entry "alibaba-druid.e10f28.1" in dataset "JCA-All".
[WARNING] Unknown dataset entry "alibaba-druid.e10f28.2" in dataset "JCA-All".
[WARNING] Unknown dataset entry "andoid-rcs-rcsjta.04d847.1" in dataset "JCA-All".
...

Expected output:
No output for datasets which aren't passed.

@akwick
Copy link
Member Author

akwick commented May 15, 2019

I observed the same behavior for projects.
Execute pipeline check dataset --only public-cms

Observed output:

mubench> pipeline check dataset --only public-cms
[INFO   ] Starting benchmark...
[INFO   ] All requirements satisfied. You're good to go.
[WARNING] Invalid dataset entry "closure.319" in "TSE17-ExPrecision". Entries must be "project.version.misuse".
[WARNING] Invalid dataset entry "itext.5091" in "TSE17-ExPrecision". Entries must be "project.version.misuse".
[WARNING] Invalid dataset entry "jmrtd.51" in "TSE17-ExPrecision". Entries must be "project.version.misuse".
....

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant