Skip to content

Commit

Permalink
TKEDissipationVerticalDiffusivity (aka k-epsilon) turbulence closure …
Browse files Browse the repository at this point in the history
…(take 2) (#3605)

* Generalize definition of Ri

* Non-negative Ri

* Use quasi-conservative reconstruction of shear production and buoyancy flux

* Add convective mixing length for momentum

* Update minimum TKE

* Major reorganizations to get CATKE time-stepping to work

* Working on comparison

* Some updates

* Heroics to get things to wokrk

* Update tke time stepping

* Bit of cleanup

* Figure out the right temporal stencil

* Updated default convective buoyancy flux and delete ccc methods

* Add back convective mixing length for ccc

* Grr also add back stability function for ccc

* Better error message when setting fails for field time series

* Better notation for surface buoyancy flux;

* Implement split-explicit time-stepping for CATKE

* Optimize CATKE substepping a bit more

* Performance enhancements

* Add third method to CATKE comparison

* Minor improvements

* Clean up

* Update

* Move around some inbounds

* Delete unused code

* Clean up time stepping

* Little things

* Improve show and consistently use surface-ignoring reconstruction

* Update definition of Ri and try a new stability funciton

* Delete show

* Queue up change to sheared convection

* Update scale

* Another tweak

* Add four more parameters for shear turbulence in unstable stratification (#3600)

* Add special negative Ri branch to the stability function

* Update parameters

* Improve show for CATKE

* Skip test for tuples-with-CATKE for now

* Try to fix checkpointing / regression test issues

* Its a special test so we have to comment it out

* Make room for TKEDissipationVerticalDiffusivity

* Things are more or less wokring

* Trying to get k-epsilon to work

* Success?

* Wow its working

* Updates to permit ensembles of closures

* Change TimeInterval to avoid roundoff error issue plus some cleanup

* fix typo

* Import fix

* Potentially address overflow problem

* Minor improvements

* Bug fix for safety factor

* Add a script to inspect stability functions

* Make schedule work even with fake models

* Fix tests for TimeInterval

* Fix JLD2 output writer tests

* Fix NetCDF writer test?

* Rm reference to CATKEVerticalDiffusivities

* fix typo

Co-authored-by: Simone Silvestri <[email protected]>

* Update src/TurbulenceClosures/turbulence_closure_implementations/TKEBasedVerticalDiffusivities/catke_vertical_diffusivity.jl

Co-authored-by: Navid C. Constantinou <[email protected]>

* Update src/TurbulenceClosures/turbulence_closure_implementations/TKEBasedVerticalDiffusivities/tke_dissipation_equations.jl

Co-authored-by: Navid C. Constantinou <[email protected]>

* Restore convective adjustment notation

* Address a few review comments

* Get rid of background diffusivities

* Update src/TurbulenceClosures/turbulence_closure_implementations/TKEBasedVerticalDiffusivities/tke_dissipation_vertical_diffusivity.jl

Co-authored-by: Navid C. Constantinou <[email protected]>

* Fix docstring

* Add reference

* Better docstrings and comments

---------

Co-authored-by: Navid C. Constantinou <[email protected]>
Co-authored-by: Simone Silvestri <[email protected]>
  • Loading branch information
3 people committed Jun 20, 2024
1 parent 56c6dd9 commit 5661358
Show file tree
Hide file tree
Showing 22 changed files with 2,112 additions and 81 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ using Oceananigans.Utils: work_layout, KernelParameters
using Oceananigans.Grids: halo_size
using Oceananigans.Fields: immersed_boundary_condition
using Oceananigans.Biogeochemistry: update_tendencies!
using Oceananigans.TurbulenceClosures.CATKEVerticalDiffusivities: FlavorOfCATKE
using Oceananigans.TurbulenceClosures.TKEBasedVerticalDiffusivities: FlavorOfCATKE, FlavorOfTD

using Oceananigans.ImmersedBoundaries: active_interior_map, ActiveCellsIBG,
InteriorMap, active_linear_index_to_tuple
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,10 @@ function ab2_step_tracers!(tracers, model, Δt, χ)
# TODO: do better than this silly criteria, also need to check closure tuples
if closure isa FlavorOfCATKE && tracer_name == :e
@debug "Skipping AB2 step for e"
elseif closure isa FlavorOfTD && tracer_name ==
@debug "Skipping AB2 step for ϵ"
elseif closure isa FlavorOfTD && tracer_name == :e
@debug "Skipping AB2 step for e"
else
Gⁿ = model.timestepper.Gⁿ[tracer_name]
G⁻ = model.timestepper.G⁻[tracer_name]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ using Oceananigans.Grids: Flat, Bounded
using Oceananigans.Fields: ZeroField
using Oceananigans.Coriolis: AbstractRotation
using Oceananigans.TurbulenceClosures: AbstractTurbulenceClosure
using Oceananigans.TurbulenceClosures.CATKEVerticalDiffusivities: CATKEVDArray
using Oceananigans.TurbulenceClosures.TKEBasedVerticalDiffusivities: CATKEVDArray

import Oceananigans.Grids: validate_size, validate_halo
import Oceananigans.Models: validate_tracer_advection
Expand Down
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
using Oceananigans.Grids: Flat, Bounded, y_domain
using Oceananigans.TurbulenceClosures: AbstractTurbulenceClosure
using Oceananigans.TurbulenceClosures.CATKEVerticalDiffusivities: _top_tke_flux, CATKEVDArray
using Oceananigans.TurbulenceClosures.TKEBasedVerticalDiffusivities: _top_tke_flux, CATKEVDArray

import Oceananigans.Grids: validate_size, validate_halo, XYRegularRG
import Oceananigans.TurbulenceClosures: time_discretization, compute_diffusivities!, with_tracers
import Oceananigans.TurbulenceClosures: ∂ⱼ_τ₁ⱼ, ∂ⱼ_τ₂ⱼ, ∂ⱼ_τ₃ⱼ, ∇_dot_qᶜ
import Oceananigans.TurbulenceClosures.CATKEVerticalDiffusivities: top_tke_flux
import Oceananigans.TurbulenceClosures.TKEBasedVerticalDiffusivities: top_tke_flux
import Oceananigans.Coriolis: x_f_cross_U, y_f_cross_U, z_f_cross_U

#####
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,10 @@ function store_tendencies!(model::HydrostaticFreeSurfaceModel)

if closure isa FlavorOfCATKE && field_name == :e
@debug "Skipping store tendencies for e"
elseif closure isa FlavorOfTD && field_name ==
@debug "Skipping store tendencies for ϵ"
elseif closure isa FlavorOfTD && field_name == :e
@debug "Skipping store tendencies for e"
else
launch!(model.architecture, model.grid, :xyz,
store_field_tendencies!,
Expand Down
2 changes: 1 addition & 1 deletion src/Models/NonhydrostaticModels/nonhydrostatic_model.jl
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ using Oceananigans.Models: AbstractModel, NaNChecker, extract_boundary_condition
using Oceananigans.Solvers: FFTBasedPoissonSolver
using Oceananigans.TimeSteppers: Clock, TimeStepper, update_state!, AbstractLagrangianParticles
using Oceananigans.TurbulenceClosures: validate_closure, with_tracers, DiffusivityFields, time_discretization, implicit_diffusion_solver
using Oceananigans.TurbulenceClosures.CATKEVerticalDiffusivities: FlavorOfCATKE
using Oceananigans.TurbulenceClosures.TKEBasedVerticalDiffusivities: FlavorOfCATKE
using Oceananigans.Utils: tupleit
using Oceananigans.Grids: topology

Expand Down
4 changes: 2 additions & 2 deletions src/TurbulenceClosures/TurbulenceClosures.jl
Original file line number Diff line number Diff line change
Expand Up @@ -165,15 +165,15 @@ include("turbulence_closure_implementations/scalar_biharmonic_diffusivity.jl")
include("turbulence_closure_implementations/smagorinsky_lilly.jl")
include("turbulence_closure_implementations/anisotropic_minimum_dissipation.jl")
include("turbulence_closure_implementations/convective_adjustment_vertical_diffusivity.jl")
include("turbulence_closure_implementations/CATKEVerticalDiffusivities/CATKEVerticalDiffusivities.jl")
include("turbulence_closure_implementations/TKEBasedVerticalDiffusivities/TKEBasedVerticalDiffusivities.jl")
include("turbulence_closure_implementations/ri_based_vertical_diffusivity.jl")

# Special non-abstracted diffusivities:
# TODO: introduce abstract typing for these
include("turbulence_closure_implementations/isopycnal_skew_symmetric_diffusivity.jl")
include("turbulence_closure_implementations/leith_enstrophy_diffusivity.jl")

using .CATKEVerticalDiffusivities: CATKEVerticalDiffusivity
using .TKEBasedVerticalDiffusivities: CATKEVerticalDiffusivity, TKEDissipationVerticalDiffusivity

# Miscellaneous utilities
include("diffusivity_fields.jl")
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,186 @@
module TKEBasedVerticalDiffusivities

using Adapt
using CUDA
using KernelAbstractions: @kernel, @index

using Oceananigans.Architectures
using Oceananigans.Grids
using Oceananigans.Utils
using Oceananigans.Units
using Oceananigans.Fields
using Oceananigans.Operators

using Oceananigans.Utils: prettysummary
using Oceananigans.Grids: peripheral_node, inactive_node, inactive_cell
using Oceananigans.Fields: ZeroField
using Oceananigans.BoundaryConditions: default_prognostic_bc, DefaultBoundaryCondition
using Oceananigans.BoundaryConditions: BoundaryCondition, FieldBoundaryConditions
using Oceananigans.BoundaryConditions: DiscreteBoundaryFunction, FluxBoundaryCondition
using Oceananigans.BuoyancyModels: ∂z_b, top_buoyancy_flux
using Oceananigans.Grids: inactive_cell

using Oceananigans.TurbulenceClosures:
getclosure,
time_discretization,
AbstractScalarDiffusivity,
VerticallyImplicitTimeDiscretization,
VerticalFormulation

import Oceananigans.BoundaryConditions: getbc
import Oceananigans.Utils: with_tracers
import Oceananigans.TurbulenceClosures:
validate_closure,
shear_production,
buoyancy_flux,
dissipation,
add_closure_specific_boundary_conditions,
compute_diffusivities!,
DiffusivityFields,
implicit_linear_coefficient,
viscosity,
diffusivity,
viscosity_location,
diffusivity_location,
diffusive_flux_x,
diffusive_flux_y,
diffusive_flux_z

const c = Center()
const f = Face()
const VITD = VerticallyImplicitTimeDiscretization

@inline ϕ²(i, j, k, grid, ϕ, args...) = ϕ(i, j, k, grid, args...)^2

@inline function shearᶜᶜᶠ(i, j, k, grid, u, v)
∂z_u² = ℑxᶜᵃᵃ(i, j, k, grid, ϕ², ∂zᶠᶜᶠ, u)
∂z_v² = ℑyᵃᶜᵃ(i, j, k, grid, ϕ², ∂zᶜᶠᶠ, v)
= ∂z_u² + ∂z_v²
return
end

@inline function shearᶜᶜᶜ(i, j, k, grid, u, v)
∂z_u² = ℑxᶜᵃᵃ(i, j, k, grid, ℑbzᵃᵃᶜ, ϕ², ∂zᶠᶜᶠ, u)
∂z_v² = ℑyᵃᶜᵃ(i, j, k, grid, ℑbzᵃᵃᶜ, ϕ², ∂zᶜᶠᶠ, v)
= ∂z_u² + ∂z_v²
return
end

@inline Riᶜᶜᶜ(i, j, k, grid, velocities, tracers, buoyancy) =
ℑbzᵃᵃᶜ(i, j, k, grid, Riᶜᶜᶠ, velocities, tracers, buoyancy)

@inline function Riᶜᶜᶠ(i, j, k, grid, velocities, tracers, buoyancy)
u = velocities.u
v = velocities.v
= shearᶜᶜᶠ(i, j, k, grid, u, v)
= ∂z_b(i, j, k, grid, buoyancy, tracers)
Ri =/
return ifelse(N² == 0, zero(grid), Ri)
end

# @inline ℑbzᵃᵃᶜ(i, j, k, grid, fᵃᵃᶠ, args...) = ℑzᵃᵃᶜ(i, j, k, grid, fᵃᵃᶠ, args...)

# A particular kind of reconstruction that ignores peripheral nodes
@inline function ℑbzᵃᵃᶜ(i, j, k, grid, fᵃᵃᶠ, args...)
k⁺ = k + 1
k⁻ = k

f⁺ = fᵃᵃᶠ(i, j, k⁺, grid, args...)
f⁻ = fᵃᵃᶠ(i, j, k⁻, grid, args...)

p⁺ = peripheral_node(i, j, k⁺, grid, c, c, f)
p⁻ = peripheral_node(i, j, k⁻, grid, c, c, f)

f⁺ = ifelse(p⁺, f⁻, f⁺)
f⁻ = ifelse(p⁻, f⁺, f⁻)

return (f⁺ + f⁻) / 2
end

# To reconstruct buoyancy flux "conservatively" (ie approximately corresponding to production/destruction
# of mean potential energy):
@inline function buoyancy_fluxᶜᶜᶠ(i, j, k, grid, tracers, buoyancy, diffusivities)
κc = @inbounds diffusivities.κc[i, j, k]
= ∂z_b(i, j, k, grid, buoyancy, tracers)
return - κc *
end

@inline explicit_buoyancy_flux(i, j, k, grid, closure, velocities, tracers, buoyancy, diffusivities) =
ℑbzᵃᵃᶜ(i, j, k, grid, buoyancy_fluxᶜᶜᶠ, tracers, buoyancy, diffusivities)

# Note special attention paid to averaging the vertical grid spacing correctly
@inline Δz_νₑ_az_bzᶠᶜᶠ(i, j, k, grid, νₑ, a, b) = ℑxᶠᵃᵃ(i, j, k, grid, νₑ) * ∂zᶠᶜᶠ(i, j, k, grid, a) *
Δzᶠᶜᶠ(i, j, k, grid) * ∂zᶠᶜᶠ(i, j, k, grid, b)

@inline Δz_νₑ_az_bzᶜᶠᶠ(i, j, k, grid, νₑ, a, b) = ℑyᵃᶠᵃ(i, j, k, grid, νₑ) * ∂zᶜᶠᶠ(i, j, k, grid, a) *
Δzᶜᶠᶠ(i, j, k, grid) * ∂zᶜᶠᶠ(i, j, k, grid, b)

@inline function shear_production_xᶠᶜᶜ(i, j, k, grid, νₑ, uⁿ, u⁺)
Δz_Pxⁿ = ℑbzᵃᵃᶜ(i, j, k, grid, Δz_νₑ_az_bzᶠᶜᶠ, νₑ, uⁿ, u⁺)
Δz_Px⁺ = ℑbzᵃᵃᶜ(i, j, k, grid, Δz_νₑ_az_bzᶠᶜᶠ, νₑ, u⁺, u⁺)
return (Δz_Pxⁿ + Δz_Px⁺) / (2 * Δzᶠᶜᶜ(i, j, k, grid))
end

@inline function shear_production_yᶜᶠᶜ(i, j, k, grid, νₑ, vⁿ, v⁺)
Δz_Pyⁿ = ℑbzᵃᵃᶜ(i, j, k, grid, Δz_νₑ_az_bzᶜᶠᶠ, νₑ, vⁿ, v⁺)
Δz_Py⁺ = ℑbzᵃᵃᶜ(i, j, k, grid, Δz_νₑ_az_bzᶜᶠᶠ, νₑ, v⁺, v⁺)
return (Δz_Pyⁿ + Δz_Py⁺) / (2 * Δzᶜᶠᶜ(i, j, k, grid))
end

@inline function shear_production(i, j, k, grid, νₑ, uⁿ, u⁺, vⁿ, v⁺)
# Reconstruct the shear production term in an "approximately conservative" manner
# (ie respecting the spatial discretization and using a stencil commensurate with the
# loss of mean kinetic energy due to shear production --- but _not_ respecting the
# the temporal discretization. Note that also respecting the temporal discretization, would
# require storing the velocity field at n and n+1):

return ℑxᶜᵃᵃ(i, j, k, grid, shear_production_xᶠᶜᶜ, νₑ, uⁿ, u⁺) +
ℑyᵃᶜᵃ(i, j, k, grid, shear_production_yᶜᶠᶜ, νₑ, vⁿ, v⁺)
end

@inline function turbulent_velocityᶜᶜᶜ(i, j, k, grid, closure, e)
eᵢ = @inbounds e[i, j, k]
eᵐⁱⁿ = closure.minimum_tke
return sqrt(max(eᵐⁱⁿ, eᵢ))
end

@inline function mask_diffusivity(i, j, k, grid, κ★)
on_periphery = peripheral_node(i, j, k, grid, c, c, f)
within_inactive = inactive_node(i, j, k, grid, c, c, f)
nan = convert(eltype(grid), NaN)
return ifelse(on_periphery, zero(grid), ifelse(within_inactive, nan, κ★))
end

@inline clip(x) = max(zero(x), x)

function get_time_step(closure_array::AbstractArray)
# assume they are all the same
closure = CUDA.@allowscalar closure_array[1, 1]
return get_time_step(closure)
end

include("tke_top_boundary_condition.jl")

include("catke_vertical_diffusivity.jl")
include("catke_mixing_length.jl")
include("catke_equation.jl")
include("time_step_catke_equation.jl")

include("tke_dissipation_vertical_diffusivity.jl")
include("tke_dissipation_stability_functions.jl")
include("tke_dissipation_equations.jl")

for S in (:CATKEMixingLength,
:CATKEEquation,
:StratifiedDisplacementScale,
:ConstantStabilityFunctions,
:VariableStabilityFunctions)

@eval @inline convert_eltype(::Type{FT}, s::$S) where FT =
$S{FT}(; Dict(p => getproperty(s, p) for p in propertynames(s))...)

@eval @inline convert_eltype(::Type{FT}, s::$S{FT}) where FT = s
end

end # module

Loading

0 comments on commit 5661358

Please sign in to comment.