Diagnostics

[odunbar]

We should perhaps remove the somewhat arbitrary Warning for covariance collapse, and replace it with some more sensible diagnostics.

(Initial) Suggestions

• At start: Configuration (inflation/localization/EKP method etc.)
• During run: Loss error, ensemble failures, timestep, current covariance determinant,

[eviatarbach]

I have an initial implementation. As an example, running it on the sinusoid calibration returns the following:

┌ Info: Initializing ensemble Kalman process of type Inversion().
│ Number of ensemble members: 5.
│ Localization: EnsembleKalmanProcesses.Localizers.NoLocalization().
└ Failure handler: IgnoreFailures().
[ Info: Iteration 1
[ Info: Covariance-weighted error: 1096.5094081602947
[ Info: Covariance trace: 0.2025190802232435
[ Info: Covariance trace ratio (current/previous): 0.03370770919010306
[ Info: Iteration 2
[ Info: Covariance-weighted error: 2.2109668012208337
[ Info: Covariance trace: 0.04825934401887812
[ Info: Covariance trace ratio (current/previous): 0.23829529526640275
[ Info: Iteration 3
[ Info: Covariance-weighted error: 1.366460650411799
[ Info: Covariance trace: 0.04891196022172665
[ Info: Covariance trace ratio (current/previous): 1.013523105548083
[ Info: Iteration 4
[ Info: Covariance-weighted error: 1.5064956186916068
[ Info: Covariance trace: 0.03304993985520381
[ Info: Covariance trace ratio (current/previous): 0.6757026237628289
[ Info: Iteration 5
[ Info: Covariance-weighted error: 1.5273731577766525
[ Info: Covariance trace: 0.025561593323362435
[ Info: Covariance trace ratio (current/previous): 0.7734232932147889

Any suggestions about what else would be useful?

[odunbar]

Awesome thanks!

• Perhaps we can add an iteration 0 with the prior error / trace? (often this first step is quite important)
• Could you block the info together for each iteration? rather than each line being a new "Info"
• Could you add a verbose-flag type thing to remove this information if not needed.
• I was wondering if it makes sense also to add in the current "time" based on sum of stored Delta T e.g. in your setting with Delta T = 0.1,0.1,0.3 it would show
  Iteration 1, (T=0.1)
  Iteration 2, (T=0.2)
  Iteration 3, (T=0.5)
  this may be useful when we start adding more adaptive timesteppers

[eviatarbach]

Sounds good, doing this now! The only problem with iteration 0 is that the prior error will be the same as that for iteration 1, since iteration 1 uses the G evaluated at the prior u. So I will only print the covariance trace.