Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Passes grid argument to NetCDFOutputWriter #3576

Merged
merged 25 commits into from
May 10, 2024
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
48aa15c
Update netcdf_output_writer.jl
tomchor May 2, 2024
f0143e1
Add grid as property to NetCDFOutputWriter and use it during file ini…
glwagner May 3, 2024
fdee401
Merge branch 'main' into tc/grid-to-netcdfwriter
glwagner May 3, 2024
296ab22
paragraph explaining that netcdf writer also accepts an output grid
iuryt May 3, 2024
1ce9908
change wording and add jldoctest
iuryt May 6, 2024
fa97a72
Update docs/src/model_setup/output_writers.md
iuryt May 6, 2024
077fea6
Do a little better with function output for NetCDF output writer
glwagner May 6, 2024
c61729b
Fixes interpolate! plus some improvements to NetCDFOutputWriter
glwagner May 7, 2024
4134b5b
add new example for docs
iuryt May 7, 2024
ed5ce3e
simplify example
iuryt May 7, 2024
373f15a
simplify example in the docstring
iuryt May 7, 2024
6f0b9fb
Update docs/src/model_setup/output_writers.md
iuryt May 8, 2024
42ccd03
Make grid a kwarg and update language
glwagner May 8, 2024
8a7d7c0
Merge branch 'main' into tc/grid-to-netcdfwriter
glwagner May 8, 2024
14c94d8
Merge branch 'main' into tc/grid-to-netcdfwriter
tomchor May 8, 2024
70e459a
Update src/OutputWriters/netcdf_output_writer.jl
iuryt May 8, 2024
9403243
Update docs/src/model_setup/output_writers.md
iuryt May 8, 2024
45f0293
Update netcdf_output_writer.jl
iuryt May 8, 2024
8c141de
Bugfix plus better error message
glwagner May 8, 2024
c4357e5
Correctly define_output_variable for LagrangianParticles
glwagner May 8, 2024
4a3bd34
Merge branch 'main' into tc/grid-to-netcdfwriter
glwagner May 8, 2024
aa5e65c
Fix znodes in output writers docs
glwagner May 9, 2024
ab73354
fix znodes
iuryt May 9, 2024
a3a2f7e
fix double semicolon
iuryt May 9, 2024
4850ddc
14.5 kB
glwagner May 9, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions docs/src/model_setup/output_writers.md
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,37 @@ NetCDFOutputWriter scheduled on IterationInterval(1):
└── file size: 17.8 KiB
```

`NetCDFOutputWriter` can also be configured for `outputs` that are interpolated or regridded to a different grid than `model.grid`.
To use this functionality, the `output_grid` must be passed explicitly when constructing `NetCDFOutputWriter` along with the regridded / interpolated `outputs`.
Copy link
Collaborator Author

@tomchor tomchor May 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we mention that once NetCDFWriter has output_grid passed it can only write outputs on that grid? In other words, you can't have outputs that are the original/full grid and the interpolated/coarse grid in the same NetCDFWriter file (for now at least). I feel this can be a source of confusion for users in the future.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also we don't pass output_grid. We pass the keyword grid, correct?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also we don't pass output_grid. We pass the keyword grid, correct?

We made it a positional argument. But we can change that and make a keyword argument for sure. What do you think?

Also, I agree 100% that making that more explicit is a good idea. The grid and outputs must be consistent, otherwise output writing will fail.

Also, you often write NetCDFWriter. I like that name because it's a little more concise ("output" is obvious). Should we change this and JLD2?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that NetCDFWriter is better and that we should also change that for JLD.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, after this is merged we will get on that.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also we don't pass output_grid. We pass the keyword grid, correct?

We made it a positional argument. But we can change that and make a keyword argument for sure. What do you think?

Ah, true. I have a bit of a preference for keyword arguments, but I don't feel strongly about it.

Also, I agree 100% that making that more explicit is a good idea. The grid and outputs must be consistent, otherwise output writing will fail.

Also, you often write NetCDFWriter. I like that name because it's a little more concise ("output" is obvious). Should we change this and JLD2?

Ah, yeah, sorry. I write that for the exact reason you mentioned, and the fact that NetCDFOutputWriter is kinda long lol

I often thought abut opening an issue to suggest that change myself, but haven't gotten around to it. So I'm very much in favor of making that change!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm ok I agree it should be a kwarg.

Copy link
Member

@glwagner glwagner May 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed that the docs examples are identical to the doc string examples. Ideally these would be different... there's no point in copy pasting here. It's more efficient to use Documenter to insert the docstring if we decide that we don't have the bandwidth to come up with unique examples.

But I personally think that the docstring and docs should have different examples. The docstring should have short concise examples and the docs should go into more detail.

Anyways, task for the future I guess.


```jldoctest
using Oceananigans
using Oceananigans.Fields: interpolate!

grid = RectilinearGrid(size=(1, 1, 8), extent=(1, 1, 1));
model = NonhydrostaticModel(; grid)
simulation = Simulation(model, Δt=1, stop_iteration=1)

coarse_grid = RectilinearGrid(size=(grid.Nx, grid.Ny, grid.Nz÷2), extent=(grid.Lx, grid.Ly, grid.Lz))
coarse_u = Field{Face, Center, Center}(coarse_grid)

interpolate_u(model) = interpolate!(coarse_u, model.velocities.u)
outputs = (; u = interpolate_u)

simulation.output_writers[:coarse_u] = NetCDFOutputWriter(model, outputs, coarse_grid;
filename = "coarse_u.nc",
schedule = IterationInterval(1))
glwagner marked this conversation as resolved.
Show resolved Hide resolved

# output
NetCDFOutputWriter scheduled on IterationInterval(1):
├── filepath: ./coarse_u.nc
├── dimensions: zC(4), zF(5), xC(1), yF(1), xF(1), yC(1), time(0)
├── 1 outputs: u
└── array type: Array{Float64}
├── file_splitting: NoFileSplitting
└── file size: 14.6 KiB
```

See [`NetCDFOutputWriter`](@ref) for more information.

## JLD2 output writer
Expand Down
2 changes: 1 addition & 1 deletion src/Fields/interpolate.jl
Original file line number Diff line number Diff line change
Expand Up @@ -354,5 +354,5 @@ function interpolate!(to_field::Field, from_field::AbstractField)

fill_halo_regions!(to_field)

return nothing
return to_field
end
122 changes: 92 additions & 30 deletions src/OutputWriters/netcdf_output_writer.jl
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,8 @@ using Oceananigans.Utils: versioninfo_with_gpu, oceananigans_versioninfo, pretty
using Oceananigans.TimeSteppers: float_or_date_time
using Oceananigans.Fields: reduced_dimensions, reduced_location, location, validate_indices

mutable struct NetCDFOutputWriter{D, O, T, A, FS} <: AbstractOutputWriter
mutable struct NetCDFOutputWriter{G, D, O, T, A, FS} <: AbstractOutputWriter
grid :: G
filepath :: String
dataset :: D
outputs :: O
Expand Down Expand Up @@ -162,7 +163,7 @@ function add_schedule_metadata!(global_attributes, schedule::AveragedTimeInterva
end

"""
NetCDFOutputWriter(model, outputs; filename, schedule
NetCDFOutputWriter(model, outputs, grid; filename, schedule
dir = ".",
array_type = Array{Float64},
indices = nothing,
Expand All @@ -180,7 +181,7 @@ Construct a `NetCDFOutputWriter` that writes `(label, output)` pairs in `outputs
be a `Dict`) to a NetCDF file, where `label` is a string that labels the output and `output` is
either a `Field` (e.g. `model.velocities.u`) or a function `f(model)` that
returns something to be written to disk. Custom output requires the spatial `dimensions` (a
`Dict`) to be manually specified (see examples).
`Dict`) or `grid` to be manually specified (see examples).

Keyword arguments
=================
Expand Down Expand Up @@ -354,20 +355,53 @@ NetCDFOutputWriter scheduled on IterationInterval(1):
├── file_splitting: NoFileSplitting
└── file size: 17.8 KiB
```

`NetCDFOutputWriter` can also be configured for `outputs` that are interpolated or regridded to a different grid than `model.grid`.
To use this functionality, the `output_grid` must be passed explicitly when constructing `NetCDFOutputWriter` along with the regridded / interpolated `outputs`.

```jldoctest
using Oceananigans
using Oceananigans.Fields: interpolate!

grid = RectilinearGrid(size=(1, 1, 8), extent=(1, 1, 1));
model = NonhydrostaticModel(; grid)
simulation = Simulation(model, Δt=1, stop_iteration=1)

coarse_grid = RectilinearGrid(size=(grid.Nx, grid.Ny, grid.Nz÷2), extent=(grid.Lx, grid.Ly, grid.Lz))
coarse_u = Field{Face, Center, Center}(coarse_grid)

interpolate_u(model) = interpolate!(coarse_u, model.velocities.u)
outputs = (; u = interpolate_u)

simulation.output_writers[:coarse_u] = NetCDFOutputWriter(model, outputs, coarse_grid;
filename = "coarse_u.nc",
schedule = IterationInterval(1))

# output
NetCDFOutputWriter scheduled on IterationInterval(1):
├── filepath: ./coarse_u.nc
├── dimensions: zC(4), zF(5), xC(1), yF(1), xF(1), yC(1), time(0)
├── 1 outputs: u
└── array type: Array{Float64}
├── file_splitting: NoFileSplitting
└── file size: 14.6 KiB
```
"""
function NetCDFOutputWriter(model, outputs; filename, schedule,
dir = ".",
array_type = Array{Float64},
indices = (:, :, :),
with_halos = false,
function NetCDFOutputWriter(model, outputs, grid=model.grid;
filename,
schedule,
dir = ".",
array_type = Array{Float64},
indices = (:, :, :),
with_halos = false,
global_attributes = Dict(),
output_attributes = Dict(),
dimensions = Dict(),
overwrite_existing = nothing,
deflatelevel = 0,
part = 1,
file_splitting = NoFileSplitting(),
verbose = false)
dimensions = Dict(),
overwrite_existing = nothing,
deflatelevel = 0,
part = 1,
file_splitting = NoFileSplitting(),
verbose = false)
mkpath(dir)
filename = auto_extension(filename, ".nc")
filepath = joinpath(dir, filename)
Expand All @@ -394,7 +428,7 @@ function NetCDFOutputWriter(model, outputs; filename, schedule,
# with LagrangianParticles output (see the end of the file).
# We shouldn't support this in the future; we should require users to 'name' LagrangianParticles output.
outputs = dictify(outputs)
outputs = Dict(string(name) => construct_output(outputs[name], model.grid, indices, with_halos) for name in keys(outputs))
outputs = Dict(string(name) => construct_output(outputs[name], grid, indices, with_halos) for name in keys(outputs))

output_attributes = dictify(output_attributes)
global_attributes = dictify(global_attributes)
Expand All @@ -414,9 +448,11 @@ function NetCDFOutputWriter(model, outputs; filename, schedule,
dimensions,
overwrite_existing,
deflatelevel,
grid,
model)

return NetCDFOutputWriter(filepath,
return NetCDFOutputWriter(grid,
filepath,
dataset,
outputs,
schedule,
Expand Down Expand Up @@ -446,22 +482,34 @@ get_default_dimension_attributes(grid::ImmersedBoundaryGrid) =
##### Variable definition
#####

materialize_output(func, model) = func(model)
materialize_output(field::AbstractField, model) = field
materialize_output(particles::LagrangianParticles, model) = particles

""" Defines empty variables for 'custom' user-supplied `output`. """
function define_output_variable!(dataset, output, name, array_type, deflatelevel, output_attributes, dimensions)
function define_output_variable!(dataset, output, name, array_type,
deflatelevel, attrib, dimensions)

name ∉ keys(dimensions) && error("Custom output $name needs dimensions!")

defVar(dataset, name, eltype(array_type), (dimensions[name]..., "time"),
deflatelevel=deflatelevel, attrib=output_attributes)
dims = dimensions[name]
FT = eltype(array_type)
defVar(dataset, name, FT, (dims..., "time"); deflatelevel, attrib)

return nothing
end


""" Defines empty field variable. """
define_output_variable!(dataset, output::AbstractField, name, array_type, deflatelevel, output_attributes, dimensions) =
defVar(dataset, name, eltype(array_type),
(netcdf_spatial_dimensions(output)..., "time"),
deflatelevel=deflatelevel, attrib=output_attributes)
function define_output_variable!(dataset, output::AbstractField, name, array_type,
deflatelevel, attrib, dimensions)

dims = netcdf_spatial_dimensions(output)
FT = eltype(array_type)
defVar(dataset, name, FT, (dims..., "time"); deflatelevel, attrib)

return nothing
end

""" Defines empty field variable for `WindowedTimeAverage`s over fields. """
define_output_variable!(dataset, output::WindowedTimeAverage{<:AbstractField}, args...) =
Expand Down Expand Up @@ -580,12 +628,17 @@ end
#####

""" Defines empty variable for particle trackting. """
function define_output_variable!(dataset, output::LagrangianParticles, name, array_type, deflatelevel, output_attributes, dimensions)
function define_output_variable!(dataset, output::LagrangianParticles, name, array_type,
deflatelevel, output_attributes, dimensions)

particle_fields = eltype(output.properties) |> fieldnames .|> string
T = eltype(array_type)

for particle_field in particle_fields
defVar(dataset, particle_field, eltype(array_type),
("particle_id", "time"), deflatelevel=deflatelevel)
defVar(dataset, particle_field, T, ("particle_id", "time"); deflatelevel)
end

return nothing
end

dictify(outputs::LagrangianParticles) = Dict("particles" => outputs)
Expand Down Expand Up @@ -633,6 +686,7 @@ function initialize_nc_file!(filepath,
dimensions,
overwrite_existing,
deflatelevel,
grid,
model)

mode = overwrite_existing ? "c" : "a"
Expand All @@ -648,12 +702,12 @@ function initialize_nc_file!(filepath,
# schedule::AveragedTimeInterval
schedule, outputs = time_average_outputs(schedule, outputs, model)

dims = default_dimensions(outputs, model.grid, indices, with_halos)
dims = default_dimensions(outputs, grid, indices, with_halos)

# Open the NetCDF dataset file
dataset = NCDataset(filepath, mode, attrib=global_attributes)

default_dimension_attributes = get_default_dimension_attributes(model.grid)
default_dimension_attributes = get_default_dimension_attributes(grid)

# Define variables for each dimension and attributes if this is a new file.
if mode == "c"
Expand All @@ -669,7 +723,7 @@ function initialize_nc_file!(filepath,

# Creates an unlimited dimension "time"
defDim(dataset, "time", Inf)
defVar(dataset, "time", eltype(model.grid), ("time",), attrib=time_attrib)
defVar(dataset, "time", eltype(grid), ("time",), attrib=time_attrib)

# Use default output attributes for known outputs if the user has not specified any.
# Unknown outputs get an empty tuple (no output attributes).
Expand All @@ -681,7 +735,14 @@ function initialize_nc_file!(filepath,

for (name, output) in outputs
attributes = try output_attributes[name]; catch; Dict(); end
define_output_variable!(dataset, output, name, array_type, deflatelevel, attributes, dimensions)
materialized = materialize_output(output, model)
define_output_variable!(dataset,
materialized,
name,
array_type,
deflatelevel,
attributes,
dimensions)
end

sync(dataset)
Expand All @@ -704,4 +765,5 @@ initialize_nc_file!(ow::NetCDFOutputWriter, model) =
ow.dimensions,
ow.overwrite_existing,
ow.deflatelevel,
ow.grid,
model)