Extending to RJMCMC

[mhycheung]

Reversible-jump MCMC (https://en.wikipedia.org/wiki/Reversible-jump_Markov_chain_Monte_Carlo) is an extension to standard MCMC, where the Markov chain can transverse between the parameter spaces of different models (possibly of different dimensions). This allows sampling from the posterior of different models and estimating the odds ratio between them by a single RJMCMC run.

The goal of RJMCMC is to avoid wasting time on models with low evidences, which would save computational resources when the number of competing models is high. However, this will require a good proposal for the trans-model jump moves. In fact, the proposal is often highly problem specific, and even a "well-designed" proposal does not give a good acceptance rate for the jump moves.

To solve the low-jump-rate problem, FlowMC might be able to help a lot. For example, a naive implementation could be as follows:

1. Run an initial standard MCMC within each model, and estimate the posterior and evidence with normalizing flow.
2. Use the estimated evidence of each model as a guide for constructing the probability of trans-model jumps, and use the estimated posteriors as proposals for each model.
3. Iteratively refine the evidence and posteriors until the run converges

Step 1 could be replaced by something smarter, such as regular RJMCMC, but that could be problem specific. The above procedure can naturally be implemented on GPUs.