Fill halo regions for Face-Face-Any fields on ConformalCubedSphereGrid #3280
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
(a) Added tests for verifying the correct filling of halos at face-face
locations of the orthogonal spherical shell grids comprising the six panels
of the conformal cubed sphere;
(b) Currently no test exists for verifying the halo filling of the points of
intersection of (i) the northwest corners of panels 1, 3, and 5, and
(ii) the southeast corners of panels 2, 4, and 6. At these special points,
the aforementioned corners of the intersecting panels are all halo corners.
In other words, no interior corner with pre-defined value exists against
which the remaining halo corners can be verified.
(c) The tests fail at the intersection of panels with non-trivial connectivity.
(d) The number of horizontal halos is specified to be 3 everywhere.
An error occurs because of the incorrect implementation of @apply_regionally.
navidcy
reviewed
Sep 21, 2023
navidcy
reviewed
Sep 21, 2023
navidcy
reviewed
Sep 21, 2023
Also, replaced ζ with ψ.
(a) Added a script named test/test_multi_region_cubed_sphere_scalar_halos.jl to
verify the correct halo filling for scalar fields at locations cca, fca,
cfa, and ffa;
(b) Manually filled out the halos of the parent array and compared them against
the halos of the array filled out with the fill_halo_regions! function;
(c) The tests fail for all halos at the ffa location, and for the north and west
halos at the cca, fca, and cfa locations;
(d) Compacted the script test/test_multi_region_cubed_sphere.jl.
|
Let me summarize my findings in the commit 4841c96. I wrote a new script test/test_multi_region_cubed_sphere_scalar_halos.jl to verify the correct halo filling for scalar fields at locations cca, fca, cfa, and ffa. I manually filled out the halos of the parent (off-set) array and compared them against the halos of the array filled out with the fill_halo_regions! function. At the ffa locations, the tests fail for all halos. At the cca, fca, and cfa locations, the tests fail for (a) the north and west halos of the odd-numbered panels 1, 3, 5 (with non-trivial connectivities), and (b) the south and east halos of the even-numbered panels 2, 4, 6 (with non-trivial connectivities). The tests which fail are currently commented out in the script test/test_multi_region_cubed_sphere_scalar_halos.jl. After some digging, I found out that both the latitude and longitude values are not correctly defined on these halos. To delve into the root of the issue, I first replace the line Ψᵣ(λ, φ, z) = - U * R * (sind(φ) * cosd(α) - cosd(λ) * cosd(φ) * sind(α))with Ψᵣ(λ, φ, z) = λ in test/test_multi_region_cubed_sphere_scalar_halos.jl, and run the script: julia> include("test/test_multi_region_cubed_sphere_scalar_halos.jl")Then I type the following commands over the terminal and study the output: julia> Nx, Ny, Nz = 3, 3, 1
(3, 3, 1)
julia> grid = ConformalCubedSphereGrid(panel_size = (Nx, Ny, Nz), z = (0, 1), radius = 1, horizontal_direction_halo = 3, z_halo = 1)
ConformalCubedSphereGrid{Float64, FullyConnected, FullyConnected, Bounded} partitioned on CPU():
├── grids: 3×3×1 OrthogonalSphericalShellGrid{Float64, FullyConnected, FullyConnected, Bounded} on CPU with 3×3×1 halo and with precomputed metrics
├── partitioning: CubedSpherePartition with (1 region in each panel)
├── connectivity: CubedSphereConnectivity
└── devices: (CPU(), CPU(), CPU(), CPU(), CPU(), CPU())
julia> data_1 = create_c_test_data(grid)
CubedSphereField{Center, Center, Center}
├── grid: 3×3×1 ConformalCubedSphereGrid{Float64, FullyConnected, FullyConnected, Bounded} on CPU with 3×3×1 halo
├── boundary conditions: FieldBoundaryConditions
└── data: MultiRegionObject{NTuple{6, OffsetArray{Float64, 3, Array{Float64, 3}}}, NTuple{6, CPU}}
└── max=180.0, min=-149.552, mean=16.6667
julia> parent_data_1 = create_c_parent_test_data(grid)
CubedSphereField{Center, Center, Center}
├── grid: 3×3×1 ConformalCubedSphereGrid{Float64, FullyConnected, FullyConnected, Bounded} on CPU with 3×3×1 halo
├── boundary conditions: FieldBoundaryConditions
└── data: MultiRegionObject{NTuple{6, OffsetArray{Float64, 3, Array{Float64, 3}}}, NTuple{6, CPU}}
└── max=180.0, min=-149.552, mean=16.6667
julia> fill_halo_regions!(data_1)
julia> rotl90(getregion(data_1, 1)[:, :, 1])
9×9 OffsetArray(::Matrix{Float64}, -2:6, -2:6) with eltype Float64 with indices -2:6×-2:6:
0.0 0.0 0.0 -135.0 180.0 135.0 -45.0 -90.0 -135.0
0.0 0.0 0.0 -90.0 0.0 90.0 -7.01671e-15 0.0 180.0
0.0 0.0 0.0 -45.0 -7.01671e-15 45.0 45.0 90.0 135.0
-121.901 -90.0 -58.0992 -31.9008 -1.44609e-15 31.9008 58.0992 90.0 121.901
-120.448 -90.0 -59.5524 -30.4476 -3.50835e-15 30.4476 59.5524 90.0 120.448
-121.901 -90.0 -58.0992 -31.9008 -5.57062e-15 31.9008 58.0992 90.0 121.901
0.0 0.0 0.0 -45.0 -3.50835e-15 45.0 45.0 90.0 135.0
0.0 0.0 0.0 -90.0 -3.50835e-15 90.0 -3.50835e-15 -3.50835e-15 180.0
0.0 0.0 0.0 -135.0 180.0 135.0 -45.0 -90.0 -135.0
julia> rotl90(getregion(parent_data_1, 1)[:, :, 1])
9×9 OffsetArray(::Matrix{Float64}, -2:6, -2:6) with eltype Float64 with indices -2:6×-2:6:
0.0 -121.901 -120.448 -121.901 48.6426 59.5524 48.6426 27.8542 -6.58025e-15
0.0 -90.0 -90.0 -90.0 59.5524 90.0 59.5524 30.4476 -3.50835e-15
0.0 -58.0992 -59.5524 -58.0992 48.6426 59.5524 48.6426 27.8542 5.31196e-16
27.8542 30.4476 27.8542 -31.9008 -1.44609e-15 31.9008 58.0992 90.0 121.901
-1.77786e-15 2.14825e-31 1.77786e-15 -30.4476 -3.50835e-15 30.4476 59.5524 90.0 120.448
-27.8542 -30.4476 -27.8542 -31.9008 -5.57062e-15 31.9008 58.0992 90.0 121.901
-48.6426 -59.5524 -48.6426 -45.0 -3.50835e-15 45.0 45.0 90.0 135.0
-59.5524 -90.0 -59.5524 -90.0 -3.50835e-15 90.0 -3.50835e-15 -3.50835e-15 180.0
-48.6426 -59.5524 -48.6426 -135.0 180.0 135.0 -45.0 -90.0 -135.0
julia> rotl90(getregion(data_1, 2)[:, :, 1])
9×9 OffsetArray(::Matrix{Float64}, -2:6, -2:6) with eltype Float64 with indices -2:6×-2:6:
-135.0 180.0 135.0 -45.0 -90.0 -135.0 0.0 0.0 0.0
-90.0 0.0 90.0 -7.01671e-15 0.0 180.0 0.0 0.0 0.0
-45.0 -7.01671e-15 45.0 45.0 90.0 135.0 0.0 0.0 0.0
-31.9008 -1.44609e-15 31.9008 58.0992 90.0 121.901 148.099 180.0 -148.099
-30.4476 -3.50835e-15 30.4476 59.5524 90.0 120.448 149.552 180.0 -149.552
-31.9008 -5.57062e-15 31.9008 58.0992 90.0 121.901 148.099 180.0 -148.099
-45.0 -3.50835e-15 45.0 45.0 90.0 135.0 0.0 0.0 0.0
-90.0 -3.50835e-15 90.0 -3.50835e-15 -3.50835e-15 180.0 0.0 0.0 0.0
-135.0 180.0 135.0 -45.0 -90.0 -135.0 0.0 0.0 0.0
julia> rotl90(getregion(parent_data_1, 2)[:, :, 1])
9×9 OffsetArray(::Matrix{Float64}, -2:6, -2:6) with eltype Float64 with indices -2:6×-2:6:
-135.0 180.0 135.0 -45.0 -90.0 -135.0 48.6426 59.5524 48.6426
-90.0 0.0 90.0 -7.01671e-15 0.0 180.0 59.5524 90.0 59.5524
-45.0 -7.01671e-15 45.0 45.0 90.0 135.0 48.6426 59.5524 48.6426
-31.9008 -1.44609e-15 31.9008 58.0992 90.0 121.901 27.8542 30.4476 27.8542
-30.4476 -3.50835e-15 30.4476 59.5524 90.0 120.448 -6.58025e-15 -3.50835e-15 5.31196e-16
-31.9008 -5.57062e-15 31.9008 58.0992 90.0 121.901 -27.8542 -30.4476 -27.8542
-45.0 27.8542 -1.77786e-15 -27.8542 -48.6426 -59.5524 -48.6426 148.099 149.552
-90.0 30.4476 2.14825e-31 -30.4476 -59.5524 -90.0 -59.5524 180.0 180.0
-135.0 27.8542 1.77786e-15 -27.8542 -48.6426 -59.5524 -48.6426 -148.099 -149.552So, apparently, the main culprit in this case is not the fill_halos! function, but rather the incorrectly defined latitudes and longitudes at the halo locations used to specify the values of the parent data. @navidcy and @simone-silvestri, do you concur? The above discrepancy has prompted me to look into the script src/MultiRegion/multi_region_cubed_sphere_grid.jl. I did discover that one layer of halos are filled out for the latitudes, longitudes, and the metric terms, and only at the ffa locations. So, now, I am working on filling out the entire width of halos and at other locations as well. |
|
How did the tracer advection test pass then? |
Generalized halo-filling for latitude and longitude at face-face locations to handle multi-layer halos. After initially hard-coding the halo fills for each panel, I transitioned to a more streamlined approach using new functions and a loop over all panels. The expanded version is currently commented out. Upcoming tasks include extending this generalization to center-center, center-face, and face-center locations.
…/Oceananigans.jl into cubed-sphere-halo-tests
Deleted the expanded version of the multi-layer halo-fills for latitude and longitude at face-face locations mentioned in commit e8923e9.
|
I'm closing this; superseded by #3488 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces test functions to verify the accurate halo filling at face-face locations of the orthogonal spherical shell grids comprising the six panels of the conformal cubed sphere. Additionally, it provides comments and docstrings for these test functions, thereby closing #3242.