Amortized Variational Transdimensional Inference
Laurence Davies, Daniel MacKinlay, Rafael Oliveira, Scott SIsson
The expressiveness of flow-based models combined with stochastic variational inference (SVI) has, in recent years, expanded the application of optimization-based Bayesian inference to include problems with complex data relationships. However, until now, SVI using flow-based models has been limited to problems of fixed dimension. We introduce CoSMIC normalizing flows (COntextually-Specified Masking for Identity-mapped Components), an extension to neural autoregressive conditional normalizing flow architectures that enables using a single amortized variational density for inference over a transdimensional target distribution. We propose a combined stochastic variational transdimensional inference approach to training CoSMIC flows using techniques from Bayesian optimization and Monte Carlo gradient estimation. Numerical examples are provided to demonstrate the proposed methodology on challenging problems that scale to high-cardinality model spaces.