Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Relax tolerance for truncating small snocan values in CanopyFluxes #2457

Open
wants to merge 599 commits into
base: master
Choose a base branch
from

Conversation

billsacks
Copy link
Member

Description of changes

The default of 1e-13 was letting through some state updates that look like they should have been truncated to 0: each time step, snocan was decreasing by about 12 orders of magnitude.

It looks like what's happening is that snocan is periodically getting set to a very small but non-zero value – maybe coming from a small value of liqcan (which probably needs its own truncate_small_values, which might solve this issue, but that's a matter for another time). Then the model seems to be "trying" to evaporate all of this small snocan in CanopyFluxes in the next time step: the fluxes have values that reduce snocan by something like 12 orders of magnitude. But that reduction isn't quite enough to trigger the truncate_small_values call that @slevis-lmwg introduced, since that uses the default relative epsilon of 1e-13. By loosening the tolerance on the epsilon value, we allow the state update to achieve what seems to be its "desired" reduction of the state to exactly 0 in a single time step.

I think the reason this was causing a failure in the tracer test is that eventually the snocan state got close to the smallest representable double precision value: the test failed when snocan reached 4e-296; the tracer in question is set up to be 10 orders of magnitude smaller, so it should have had a value of 4e-306 at this point. But that's getting very close to limit of representation of double precision, which is about 1e-308, so I think the tracer value lost precision and was rounded to 0, while the bulk value was still very slightly non-zero.

My hypothesis for why this just started appearing with the change that @ekluzek found in #2444 (comment) is that the new freezing soil temperatures probably cause veg temperatures to be below freezing, which causes snocan to come into play where it maybe didn't before.

Specific notes

Contributors other than yourself, if any:

CTSM Issues Fixed (include github issue #):

Are answers expected to change (and if so in what way)? YES - roundoff level changes expected for some (maybe many) tests (even those without water tracers)

Any User Interface Changes (namelist or namelist defaults changes)? No

Testing performed, if any:
Ran SMS_D_Ld10.f10_f10_mg37.I2000Clm50BgcCrop.izumi_intel.clm-tracer_consistency, verified that it PASSes now

ekluzek and others added 30 commits August 25, 2023 17:06
…for the checkvarlist, and increase the size of TK_ROOF in the config file to the now needed 10
…mask used for the mesh sys test to integer in ncap2 so that it will work
…h file to compare to, that can be seen that it looks like it's expected to for the fsurdat landmask. A question now is why the identity test needs to subtract 1, when the new test doesn't
…ntical mask rather than -1, because the file already had the -1 taken out
…d grids are being run, there are a few grids that don't test both 1850 and 2000 for example, but most cases are covered
ekluzek and others added 11 commits March 27, 2024 19:34
Add a couple features from multi over to single, so that multi can use the same code.
…le not having to test both, which allows the SystemTest to be valid for just using the single script
Merge ctsm5.2 branch up to ctsm5.1.dev174
Merge b4bdev 20240321

 Conflicts:
	.git-blame-ignore-revs
…tests for ctsm5.2, that were removed in the update
Update ctsm5.2 branch to ctsm5.1.dev175
The default of 1e-13 was letting through some state updates that look
like they should have been truncated to 0: each time step, snocan was
decreasing by about 12 orders of magnitude.

Addresses ESCOMP#2444
@ekluzek ekluzek added the type: enhancement new capability or improved behavior of existing capability label Apr 5, 2024
Copy link
Contributor

@ekluzek ekluzek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome. Thanks for figuring this out @billsacks.

I'm thinking for visibility this should go to master rather than ctsm5.2 since it's an answer change, and I'd like it to be more obvious than bound up in all the ctsm5.2 changes. But, I'll ask other opinions...

@billsacks
Copy link
Member Author

I'm thinking for visibility this should go to master rather than ctsm5.2 since it's an answer change, and I'd like it to be more obvious than bound up in all the ctsm5.2 changes. But, I'll ask other opinions...

That makes sense to me. I branched off of the 5.2 branch so that I could reproduce the issue and verify that this resolves it, but I agree with your approach here.

@ekluzek ekluzek changed the base branch from ctsm5.2.mksurfdata to master April 5, 2024 04:48
@ekluzek
Copy link
Contributor

ekluzek commented Apr 5, 2024

I rebased to master, which means it now contains all of ctsm5.2, but that should be cleared up when ctsm5.2.0 is made on master.

@ekluzek ekluzek added this to the 2024 CESM June workshop milestone Apr 25, 2024
@ekluzek ekluzek added the tag: next this issue should get some attention in the next week or two label Apr 25, 2024
@wwieder wwieder removed the tag: next this issue should get some attention in the next week or two label Apr 25, 2024
@samsrabin samsrabin added this to In progress - master in Upcoming tags Apr 25, 2024
@wwieder wwieder added the type: tests additions or changes to tests label Jun 20, 2024
@wwieder wwieder moved this from In progress - master/b4b-dev to Ready to start (or start again) in Upcoming tags Jun 20, 2024
@slevis-lmwg
Copy link
Contributor

slevis-lmwg commented Jul 6, 2024

@billsacks, could you give me collaborator permissions, so that I may update this PR to the latest ctsm tag in preparation for merging to master?

Submitted on derecho:
./run_sys_tests -s aux_clm -c ctsm5.2.008 --skip-generate

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: enhancement new capability or improved behavior of existing capability type: tests additions or changes to tests
Projects
Upcoming tags
Ready to start (or start again)
Status: In Progress
Development

Successfully merging this pull request may close these issues.

Failing water isotope test on the ctsm5.2 branch
5 participants