-
Notifications
You must be signed in to change notification settings - Fork 191
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fill halo regions for Face-Face-Any fields on ConformalCubedSphereGrid #3280
Conversation
(a) Added tests for verifying the correct filling of halos at face-face locations of the orthogonal spherical shell grids comprising the six panels of the conformal cubed sphere; (b) Currently no test exists for verifying the halo filling of the points of intersection of (i) the northwest corners of panels 1, 3, and 5, and (ii) the southeast corners of panels 2, 4, and 6. At these special points, the aforementioned corners of the intersecting panels are all halo corners. In other words, no interior corner with pre-defined value exists against which the remaining halo corners can be verified. (c) The tests fail at the intersection of panels with non-trivial connectivity. (d) The number of horizontal halos is specified to be 3 everywhere.
An error occurs because of the incorrect implementation of @apply_regionally.
Also, replaced ζ with ψ.
(a) Added a script named test/test_multi_region_cubed_sphere_scalar_halos.jl to verify the correct halo filling for scalar fields at locations cca, fca, cfa, and ffa; (b) Manually filled out the halos of the parent array and compared them against the halos of the array filled out with the fill_halo_regions! function; (c) The tests fail for all halos at the ffa location, and for the north and west halos at the cca, fca, and cfa locations; (d) Compacted the script test/test_multi_region_cubed_sphere.jl.
Let me summarize my findings in the commit 4841c96. I wrote a new script test/test_multi_region_cubed_sphere_scalar_halos.jl to verify the correct halo filling for scalar fields at locations cca, fca, cfa, and ffa. I manually filled out the halos of the parent (off-set) array and compared them against the halos of the array filled out with the fill_halo_regions! function. At the ffa locations, the tests fail for all halos. At the cca, fca, and cfa locations, the tests fail for (a) the north and west halos of the odd-numbered panels 1, 3, 5 (with non-trivial connectivities), and (b) the south and east halos of the even-numbered panels 2, 4, 6 (with non-trivial connectivities). The tests which fail are currently commented out in the script test/test_multi_region_cubed_sphere_scalar_halos.jl. After some digging, I found out that both the latitude and longitude values are not correctly defined on these halos. To delve into the root of the issue, I first replace the line Ψᵣ(λ, φ, z) = - U * R * (sind(φ) * cosd(α) - cosd(λ) * cosd(φ) * sind(α)) with Ψᵣ(λ, φ, z) = λ in test/test_multi_region_cubed_sphere_scalar_halos.jl, and run the script: julia> include("test/test_multi_region_cubed_sphere_scalar_halos.jl") Then I type the following commands over the terminal and study the output: julia> Nx, Ny, Nz = 3, 3, 1
(3, 3, 1)
julia> grid = ConformalCubedSphereGrid(panel_size = (Nx, Ny, Nz), z = (0, 1), radius = 1, horizontal_direction_halo = 3, z_halo = 1)
ConformalCubedSphereGrid{Float64, FullyConnected, FullyConnected, Bounded} partitioned on CPU():
├── grids: 3×3×1 OrthogonalSphericalShellGrid{Float64, FullyConnected, FullyConnected, Bounded} on CPU with 3×3×1 halo and with precomputed metrics
├── partitioning: CubedSpherePartition with (1 region in each panel)
├── connectivity: CubedSphereConnectivity
└── devices: (CPU(), CPU(), CPU(), CPU(), CPU(), CPU())
julia> data_1 = create_c_test_data(grid)
CubedSphereField{Center, Center, Center}
├── grid: 3×3×1 ConformalCubedSphereGrid{Float64, FullyConnected, FullyConnected, Bounded} on CPU with 3×3×1 halo
├── boundary conditions: FieldBoundaryConditions
└── data: MultiRegionObject{NTuple{6, OffsetArray{Float64, 3, Array{Float64, 3}}}, NTuple{6, CPU}}
└── max=180.0, min=-149.552, mean=16.6667
julia> parent_data_1 = create_c_parent_test_data(grid)
CubedSphereField{Center, Center, Center}
├── grid: 3×3×1 ConformalCubedSphereGrid{Float64, FullyConnected, FullyConnected, Bounded} on CPU with 3×3×1 halo
├── boundary conditions: FieldBoundaryConditions
└── data: MultiRegionObject{NTuple{6, OffsetArray{Float64, 3, Array{Float64, 3}}}, NTuple{6, CPU}}
└── max=180.0, min=-149.552, mean=16.6667
julia> fill_halo_regions!(data_1)
julia> rotl90(getregion(data_1, 1)[:, :, 1])
9×9 OffsetArray(::Matrix{Float64}, -2:6, -2:6) with eltype Float64 with indices -2:6×-2:6:
0.0 0.0 0.0 -135.0 180.0 135.0 -45.0 -90.0 -135.0
0.0 0.0 0.0 -90.0 0.0 90.0 -7.01671e-15 0.0 180.0
0.0 0.0 0.0 -45.0 -7.01671e-15 45.0 45.0 90.0 135.0
-121.901 -90.0 -58.0992 -31.9008 -1.44609e-15 31.9008 58.0992 90.0 121.901
-120.448 -90.0 -59.5524 -30.4476 -3.50835e-15 30.4476 59.5524 90.0 120.448
-121.901 -90.0 -58.0992 -31.9008 -5.57062e-15 31.9008 58.0992 90.0 121.901
0.0 0.0 0.0 -45.0 -3.50835e-15 45.0 45.0 90.0 135.0
0.0 0.0 0.0 -90.0 -3.50835e-15 90.0 -3.50835e-15 -3.50835e-15 180.0
0.0 0.0 0.0 -135.0 180.0 135.0 -45.0 -90.0 -135.0
julia> rotl90(getregion(parent_data_1, 1)[:, :, 1])
9×9 OffsetArray(::Matrix{Float64}, -2:6, -2:6) with eltype Float64 with indices -2:6×-2:6:
0.0 -121.901 -120.448 -121.901 48.6426 59.5524 48.6426 27.8542 -6.58025e-15
0.0 -90.0 -90.0 -90.0 59.5524 90.0 59.5524 30.4476 -3.50835e-15
0.0 -58.0992 -59.5524 -58.0992 48.6426 59.5524 48.6426 27.8542 5.31196e-16
27.8542 30.4476 27.8542 -31.9008 -1.44609e-15 31.9008 58.0992 90.0 121.901
-1.77786e-15 2.14825e-31 1.77786e-15 -30.4476 -3.50835e-15 30.4476 59.5524 90.0 120.448
-27.8542 -30.4476 -27.8542 -31.9008 -5.57062e-15 31.9008 58.0992 90.0 121.901
-48.6426 -59.5524 -48.6426 -45.0 -3.50835e-15 45.0 45.0 90.0 135.0
-59.5524 -90.0 -59.5524 -90.0 -3.50835e-15 90.0 -3.50835e-15 -3.50835e-15 180.0
-48.6426 -59.5524 -48.6426 -135.0 180.0 135.0 -45.0 -90.0 -135.0
julia> rotl90(getregion(data_1, 2)[:, :, 1])
9×9 OffsetArray(::Matrix{Float64}, -2:6, -2:6) with eltype Float64 with indices -2:6×-2:6:
-135.0 180.0 135.0 -45.0 -90.0 -135.0 0.0 0.0 0.0
-90.0 0.0 90.0 -7.01671e-15 0.0 180.0 0.0 0.0 0.0
-45.0 -7.01671e-15 45.0 45.0 90.0 135.0 0.0 0.0 0.0
-31.9008 -1.44609e-15 31.9008 58.0992 90.0 121.901 148.099 180.0 -148.099
-30.4476 -3.50835e-15 30.4476 59.5524 90.0 120.448 149.552 180.0 -149.552
-31.9008 -5.57062e-15 31.9008 58.0992 90.0 121.901 148.099 180.0 -148.099
-45.0 -3.50835e-15 45.0 45.0 90.0 135.0 0.0 0.0 0.0
-90.0 -3.50835e-15 90.0 -3.50835e-15 -3.50835e-15 180.0 0.0 0.0 0.0
-135.0 180.0 135.0 -45.0 -90.0 -135.0 0.0 0.0 0.0
julia> rotl90(getregion(parent_data_1, 2)[:, :, 1])
9×9 OffsetArray(::Matrix{Float64}, -2:6, -2:6) with eltype Float64 with indices -2:6×-2:6:
-135.0 180.0 135.0 -45.0 -90.0 -135.0 48.6426 59.5524 48.6426
-90.0 0.0 90.0 -7.01671e-15 0.0 180.0 59.5524 90.0 59.5524
-45.0 -7.01671e-15 45.0 45.0 90.0 135.0 48.6426 59.5524 48.6426
-31.9008 -1.44609e-15 31.9008 58.0992 90.0 121.901 27.8542 30.4476 27.8542
-30.4476 -3.50835e-15 30.4476 59.5524 90.0 120.448 -6.58025e-15 -3.50835e-15 5.31196e-16
-31.9008 -5.57062e-15 31.9008 58.0992 90.0 121.901 -27.8542 -30.4476 -27.8542
-45.0 27.8542 -1.77786e-15 -27.8542 -48.6426 -59.5524 -48.6426 148.099 149.552
-90.0 30.4476 2.14825e-31 -30.4476 -59.5524 -90.0 -59.5524 180.0 180.0
-135.0 27.8542 1.77786e-15 -27.8542 -48.6426 -59.5524 -48.6426 -148.099 -149.552 So, apparently, the main culprit in this case is not the fill_halos! function, but rather the incorrectly defined latitudes and longitudes at the halo locations used to specify the values of the parent data. @navidcy and @simone-silvestri, do you concur? The above discrepancy has prompted me to look into the script src/MultiRegion/multi_region_cubed_sphere_grid.jl. I did discover that one layer of halos are filled out for the latitudes, longitudes, and the metric terms, and only at the ffa locations. So, now, I am working on filling out the entire width of halos and at other locations as well. |
How did the tracer advection test pass then? |
Generalized halo-filling for latitude and longitude at face-face locations to handle multi-layer halos. After initially hard-coding the halo fills for each panel, I transitioned to a more streamlined approach using new functions and a loop over all panels. The expanded version is currently commented out. Upcoming tasks include extending this generalization to center-center, center-face, and face-center locations.
…/Oceananigans.jl into cubed-sphere-halo-tests
Deleted the expanded version of the multi-layer halo-fills for latitude and longitude at face-face locations mentioned in commit e8923e9.
I'm closing this; superseded by #3488 |
This PR introduces test functions to verify the accurate halo filling at face-face locations of the orthogonal spherical shell grids comprising the six panels of the conformal cubed sphere. Additionally, it provides comments and docstrings for these test functions, thereby closing #3242.