-
Notifications
You must be signed in to change notification settings - Fork 100
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: add performance assessment to acceptance tests #1771
Conversation
📝 Acceptance Test Report📋 Summary✅ The rule acceptance has passed for commit 838fa46 📊 Notices ComparisonNew Errors (1 out of 1519 datasets, ~0%) ✅Details of new errors due to code change, which is less than the provided threshold of 1%.
Dropped Errors (0 out of 1519 datasets, ~0%) ✅No changes were detected due to the code change. New Warnings (2 out of 1519 datasets, ~0%) ✅Details of new errors due to code change, which is less than the provided threshold of 1%.
Dropped Warnings (1 out of 1519 datasets, ~0%) ✅Details of new errors due to code change, which is less than the provided threshold of 1%.
🛡️ Corruption Check1 out of 1520 sources (~0 %) are corrupted.
⏱️ Performance Assessment📈 Validation TimeAssess the performance in terms of seconds taken for the validation process.
|
📝 Acceptance Test Report📋 Summary✅ The rule acceptance has passed for commit ead9f46 📊 Notices ComparisonNew Errors (0 out of 1523 datasets, ~0%) ✅No changes were detected due to the code change. Dropped Errors (0 out of 1523 datasets, ~0%) ✅No changes were detected due to the code change. New Warnings (0 out of 1523 datasets, ~0%) ✅No changes were detected due to the code change. Dropped Warnings (0 out of 1523 datasets, ~0%) ✅No changes were detected due to the code change. 🛡️ Corruption Check0 out of 1523 sources (~0 %) are corrupted. ⏱️ Performance Assessment📈 Validation TimeAssess the performance in terms of seconds taken for the validation process.
|
@cka-y |
@emmambd exactly -- this follows the naming convention of the reports i.e. |
.../main/java/org/mobilitydata/gtfsvalidator/outputcomparator/io/CorruptedSourcesCollector.java
Show resolved
Hide resolved
main/src/main/java/org/mobilitydata/gtfsvalidator/runner/ValidationRunner.java
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LVGTM
📝 Acceptance Test Report📋 Summary✅ The rule acceptance has passed for commit ec6ba68 📊 Notices ComparisonNew Errors (0 out of 1523 datasets, ~0%) ✅No changes were detected due to the code change. Dropped Errors (0 out of 1523 datasets, ~0%) ✅No changes were detected due to the code change. New Warnings (0 out of 1523 datasets, ~0%) ✅No changes were detected due to the code change. Dropped Warnings (0 out of 1523 datasets, ~0%) ✅No changes were detected due to the code change. 🛡️ Corruption Check0 out of 1523 sources (~0 %) are corrupted. ⏱️ Performance Assessment📈 Validation TimeAssess the performance in terms of seconds taken for the validation process.
|
Summary:
This PR introduces several key improvements and changes:
latest
andreference
reports.latest
andreference
are included in the JSON acceptance report. Each element in the list is formatted as follows:Expected Behavior:
Example of the markdown report for modified reports (posted in the comments of the PR here)
Example of the JSON acceptance test report
Please make sure these boxes are checked before submitting your pull request - thanks!
gradle test
to make sure you didn't break anything