Some notes and resources on computing techniques for Applied Bayesian Modelling.
Motivation
Bayesian modelling is now ubiquitous. And it's wonderful. It does, however, come with its own set of computational difficulties. In these notes we will try and collect tips and tricks for making fitting, diagnosing and using models under the Bayesian paradigm easier and more stable.
Modelling
- Prior and posterior pushforward checking. See Bayesian Workflow and Gabry et al. (2019). See also the Stan guide.
Computational
- Generating truncated random variables Stan manual;
- Goldberg's 1991 gem "What every computer scientist should know about floating-point arithmetic." is great;
- The relevant section of the Stan guide is also of great value.
- Gelman (2004) is a good source for a discussion on parametrisations.
- This blog post by Thomas Twiecki explains nicely how parametrisation matters, especially for hierarchical models, as does this excellent video by Ben Lambert.
- The QR is a great tool to alleviate computational problems with regression models.
- Hamiltonian Monte Carlo for Hierarchical Models by Mike Betancourt and Mark Girolami is a great technical resource.
- A General Framework for the Parametrization of Hierarchical Models By Papaspiliopoulos et al.
- Efficient parametrisations for normal linear mixed models from Alan E. Gelfand, Sujit K. Sahu and Bradley P. Carlin.
Some of this stuff was suggested by Lucas Moschen in this issue.
- My course on Computational Statistics is a good source of in-depth material. In particular, slides for a crash course on MCMC are here.
- If you know what's good for you, you will check out Mike Betancourt's writtings.
- A good resource in Portuguese is Marco Inácio's Apostila de Stan. But beware it was written some time ago and the language has moved forward quite a bit.
- Stan's documentation page is also a great place to find tips and tricks.