-
Notifications
You must be signed in to change notification settings - Fork 279
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
catkin_make run_tests returns success even when tests fail. #576
Comments
You have to run |
OK, cakin_test_results does seem to do the right thing. What is all the other junk for in that ros_comm .travis.yml file? Maybe it predates catkin_test_results? |
That might have been from a point where |
OK, I'm satisfied. I'll close this. It would be helpful if the catkin docs and/or the "USAGE" line for catkin_test_result described the result code behavior. Thanks for your help! |
The result code returned by "catkin_make run_tests" and by "cd build; make run_tests" in a catkin workspace is 0 (success) even when tests have failed.
This makes it difficult to script automated testing, since Travis (for one) runs every line in its script until one fails, as determined by the result code.
In ros_comm's .travis.yml, I found this pile of stuff, which I believe is a workaround for the problem I'm describing:
I notice that there exists a "test" target in the Makefile, and when I run "make test" it does indeed return a failure code when any of the tests fail. Yay! However, it says it ran 12 tests, and when I run "make run_tests" it says it ran 39 tests. So that makes me think that "make test" is not running all of my tests. Also the output formatting and result reporting is totally different between "make test" and "make run_tests", so I'm not sure what is going on there.
The text was updated successfully, but these errors were encountered: