-
Notifications
You must be signed in to change notification settings - Fork 87
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Guide to Docs for ARMI Testing Tools #1746
Comments
RE: Self-Cleaning DirectoriesA lot of the use cases from Some possible side benefits of this alternative approach:
Aside from historical inertia, is there a strong reason why ARMI would like to continue to maintain |
RE: Load a Test ReactorI'm working on something for this. It'll be able to cache test reactors for quick reloading between test runs. It'll also be fully compatible with xdist -- NO RACE CONDITIONS! 🎉 It'll be ready for show-time in a week or two. I'm just writing-up documentation and developing realistic benchmark cases. If it's accepted by ARMI, then |
Thanks for the help on this @andfranklin! |
Plugins often use a temporary directory changer outside of testing to run external programs that produce their own temporary files. When the temporary directory is closed after the work is done only some a subset of files are returned. |
It's used in the code as well as the tests |
I think refactoring the logging used for hundreds of tests is probably out of scope for this ticket. |
This sounds awesome! |
Sorry, what I was saying is we already HAVE a solution for this. |
This ticket is NOT about features we don't have. If you have a feature request or idea, that would be a different ticket. The point of this ticket is to write documentation for testing tools we already have in ARMI. |
Understood, just trying to provide a different perspective of what could be. To me it seems like ARMI could relieve some of its maintenance burden if different practices were adopted. Maybe there's a reason why things are the way that they are for reasons that I'm not aware of 🤷♂️ |
That's absolutely fair 😆 I wasn't trying to suggest that. |
It will be! I'm excited to get feedback from y'all. We'll get to it soon. |
Ah that makes sense, thanks @jakehader and @opotowsky. Sounds like #1780 actually needs to be addressed. I'll see if I can help with that. |
@opotowsky Do you want to take this ticket, or should I? I figure it should be one of us. |
I can take it but I'm probably full up for work this week. So it would be later. (I'm happy to do it though) |
There's no rush. I wouldn't get to it this week either. If you want it, self-assign. Otherwise, just assign it to me. #shrug NBD either way. |
Based on recent questions I've gotten, I think it would be useful to add a guide to the ARMI-specific testing tools to the "dev" section of the ARMI docs. Below is a first-draft that @opotowsky wrote previously.
ARMI Testing Tools
ARMI has many useful tools to streamline tests in the plugins. Included here are some popular ones, or at least ones that should be popular.
If you're trying to write a new unit test, chances are something LIKE it has been done before and you don't need to design it from scratch. Look around ARMI and other plugins for examples of tests.
Testing with runLog
Use Case: Test code that prints to stdout
While there are some other mocking examples in ARMI, none are as heavily used as
mockRunLogs
.mockRunLogs.BufferLog()
is used to capture therunLog
output instead of printing it.In test_comparedb3.py, there is a (simplified here) use case. A portion of the test for
_diffSpecialData
wants to confirm the below printout has happened, so it uses thegetStdout()
method to check that the expected printout exists.There are examples of this throughout ARMI, but sparsely (so far) in plugins. Search for BufferLog or getStdout in the code to find examples.
Self-Cleaning Directories
Use Case: Automatically cleans up tests that create files
Two main uses of this class in testing:
armi/armi/operators/tests/test_operators.py
Lines 237 to 242 in 2bcb036
armi/armi/bookkeeping/db/tests/test_comparedb3.py
Lines 36 to 52 in 2bcb036
Note that sometimes it's necessary to give the temporary directory change object a non-default root path:
Load a Test Reactor
Use Case: need a full reactor.
Warning: Expensive, and possibly over-used. Consider whether mocking or BYO components (below) can be used instead.
from armi.reactor.tests.test_reactors import loadTestReactor
Returns an operator and a reactor:
armi/armi/reactor/tests/test_reactors.py
Lines 142 to 146 in 342d2c9
So many interfaces and methods require an operator, a reactor, or both. Check out this example in
test_fluxReconstructor.py
:And a test in that class using fuel blocks from the loaded reactor (and a FluxReconstructor method from the operator):
Sidebar: speed up tests using test reactor
If using, maybe you can speed up unit tests by reducing the number of rings.
The database interface needs a reactor to load, so the setUp of a class in test_database3.py reduces the size of the reactor ahead of the tests:
Empty Reactors
from armi import tests
getEmptyHexReactor()
getEmptyCartesianReactor()
BYO Blocks & Assemblies
Use Case: likes speedy tests and doesn't need a full reactor
from armi.reactor.tests import test_blocks
buildSimpleFuelBlock() : returns a block with no reactor/assembly lineage
armi/armi/reactor/tests/test_blocks.py
Lines 46 to 47 in 2bcb036
loadtestBlock()
: returns a block in an assembly in a reactorarmi/armi/reactor/tests/test_blocks.py
Lines 79 to 80 in 2bcb036
... block.add(stuff) ...
armi/armi/reactor/tests/test_blocks.py
Lines 238 to 243 in 2bcb036
from armi.reactor.tests import test_assemblies
buildTestAssemblies()
armi/armi/reactor/tests/test_assemblies.py
Lines 54 to 61 in 2bcb036
makeTestAssembly()
armi/armi/reactor/tests/test_assemblies.py
Lines 182 to 190 in 2bcb036
Miscellaneous Miscellany
Here are some test files we felt might be useful for people to search to help with test development in plugins.
from armi.materials.tests import test_material
from armi.nuclearDataIO.cccc.tests import test_labels
from armi.nuclearDataIO.tests import test_xsLibraries
from armi.physics.neutronics.fissionProductModel.tests import test_lumpedFissionProduct
from armi.physics.neutronics.tests import test_crossSectionManager
from armi.reactor.blueprints.tests import test_customIsotopics
from armi.tests import ArmiTestHelper
# compareFilesLineByLine & compareLinesThe text was updated successfully, but these errors were encountered: