-
Notifications
You must be signed in to change notification settings - Fork 81
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make it easier to run CI tests locally #69
Comments
You could also look into something like Toast. |
Some interesting tools in this space are |
One thing to keep in mind is that Act and I think Toast as well don't support Windows images (or at least didn't last time I tried IIRC). |
I struggled specifically with formatting and forgetting to make stderr match in tests. Adding a way to format test files and an easier way to update all the stderr files for tests might be helpful. 🙂 |
Awesome, added that to the issue description. This'll definitely be important to solve moving forward. |
@djkoloski I took a stab at pulling the scripts out of |
Move some CI test bash infrastructure into `util.sh`. Implement a rudimentary test runner in `test.sh`. Eventually, the intention is that `test.sh` will mimic as closely as possible the tests that run during CI so that developers can run those tests locally. Part of #69
Move some CI test bash infrastructure into `util.sh`. Implement a rudimentary test runner in `test.sh`. Eventually, the intention is that `test.sh` will mimic as closely as possible the tests that run during CI so that developers can run those tests locally. Part of #69
@djkoloski do you feel like #728 and #1171 combined are sufficient to close this issue? |
Yes, this issue can be closed. |
We currently run a large suite of tests in CI, which includes testing with different tools across different toolchains and different targets. It also involves miscellaneous things like confirming that
README.md
is up to date, that code is properly formatted withrustfmt
, etc. In other words, it's not as simple as just runningcargo test
.As a result, it's easy to make changes that appear fine locally but break CI, requiring an edit-commit-push cycle just to see if all of the CI tests will pass. We should make it easier to run most or all of these tests locally. For especially fast tests (like
README.md
and code formatting), we could even provide them as commit hooks.@djkoloski also reports finding this to be painful (#69 (comment)):
The challenge here is going to be making these tests available without either a) moving everything out of our
ci.yml
and into a script (which means we don't get nice job output and error messages) or, b) duplicating everything (which is brittle).The text was updated successfully, but these errors were encountered: