Faster Uncertainty Quantification for Inverse Problems with Conditional Normalizing Flows
In inverse problems, we often have access to data consisting of paired samples (x,y)∼pX,Y(x,y) where y are partial observations of a physical system, and x represents the unknowns of the problem. Under these circumstances, we can employ supervised training to learn a solution x and its uncertainty from the observations y. We refer to this problem as the "supervised" case. However, the data y∼pY(y) collected at one point could be distributed differently than observations y′∼pY′(y′), relevant for a current set of problems. In the context of Bayesian inference, we propose a two-step scheme, which makes use of normalizing flows and joint data to train a conditional generator qθ(x∣y) to approximate the target posterior density pX∣Y(x∣y). Additionally, this preliminary phase provides a density function qθ(x∣y), which can be recast as a prior for the "unsupervised" problem, e.g.~when only the observations y′∼pY′(y′), a likelihood model y′∣x, and a prior on x′ are known. We then train another invertible generator with output density qϕ′(x∣y′) specifically for y′, allowing us to sample from the posterior pX∣Y′(x∣y′). We present some synthetic results that demonstrate considerable training speedup when reusing the pretrained network qθ(x∣y′) as a warm start or preconditioning for approximating pX∣Y′(x∣y′), instead of learning from scratch. This training modality can be interpreted as an instance of transfer learning. This result is particularly relevant for large-scale inverse problems that employ expensive numerical simulations.
