UKI improperly tested - and other test issues...

[odunbar]

I have noticed during other investigations that the implemented UKI we have is only running unit tests on a linear case!

Moreover, when providing it with nonlinear problems e.g. learning x through G(x) = sqrt(||x - x_{true}||^2), data and produced with G + additive noise, and priors with small bias, it seems quite unstable. Given it has been used in other investigations effectively I suspect a bug.

EDIT: I'm now making this around other bugs found during testing/inverse problems suite

• [PARTIALLY RESOLVED: (1) the adaptive learning rate seems to help a bit, (2) Using the alpha_ref = 0.85 prevents divergence but changes where the method converges] UKI seems unstable on current test suite (nonlinear problems) Resolve why. Fix, and uncomment UKI test for schedulers in when performance has been thoroughly assessed.
• [RESOLVED: We were applying it in low parameter - large ensemble regime (inappropriate for localization) Localization (now back to EKI) causes blow up in on even the linear problem. Somehow it does not effect the final mean, and thus we did not notice, (1) add test to check tr(cov) of ensemble to detect some form of blow up in one dimension, and resolve why this occurs.

[odunbar]

Here is the result of learning with additive normal noise on several evaluations of G(x) = sqrt(||x - x_{true}||^2). The prior is bounded by [-2,2] in the x axis.

After 30 iterations of all methods. EKI has converged, EKS is approaching the posterior in each iteration, but UKI is diverging (in the transformed space, it is approaching the left boundary)