BZPEERPapers, people, evidence and research paths
Menu

Discover

SearchFind papers, people, topics, institutions and opportunities from one place.GraphSee how papers, authors, topics and evidence connect.

Research tools

HomeContinue from the papers, people and topics that matter now.CollectionsTurn saved papers into reading lists, evidence maps and shareable context.ResearchReconstruct the path that led you from search to paper to question.CollaborationMove from a paper, topic or expert request into a focused thread.PublishCreate full-text research articles with metadata, formulas and versions.

Network

ExpertsFind specialists whose work and reviews explain why they can help.CommunitiesJoin research groups organized around papers, topics and shared evidence.

Opportunities

ListingsBrowse roles, calls and collaborations connected to real research context.

Account

Log inReturn to your saved papers, graph views and active threads.Create accountStart saving papers, building a research trail and joining teams.
Log inCreate account

Source author record

Colin Fox

Colin Fox appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record
ComputationMethodologymath.NAmath.PRPopulations and Evolution

Catalog footprint

What is connected

12works
5topics
4close collaborators

Actions

Connect this record

Log in to claim
Open graphBrowse works

Research graph

See the researcher in context

Open full explorer

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

12 published item(s)

preprint2022arXiv

Multilevel Delayed Acceptance MCMC

We develop a novel Markov chain Monte Carlo (MCMC) method that exploits a hierarchy of models of increasing complexity to efficiently generate samples from an unnormalized target distribution. Broadly, the method rewrites the Multilevel MCMC approach of Dodwell et al. (2015) in terms of the Delayed Acceptance (DA) MCMC of Christen & Fox (2005). In particular, DA is extended to use a hierarchy of models of arbitrary depth, and allow subchains of arbitrary length. We show that the algorithm satisfies detailed balance, hence is ergodic for the target distribution. Furthermore, multilevel variance reduction is derived that exploits the multiple levels and subchains, and an adaptive multilevel correction to coarse-level biases is developed. Three numerical examples of Bayesian inverse problems are presented that demonstrate the advantages of these novel methods. The software and examples are available in PyMC3.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Randomized Reduced Forward Models for Efficient Metropolis--Hastings MCMC, with Application to Subsurface Fluid Flow and Capacitance Tomography

Bayesian modelling and computational inference by Markov chain Monte Carlo (MCMC) is a principled framework for large-scale uncertainty quantification, though is limited in practice by computational cost when implemented in the simplest form that requires simulating an accurate computer model at each iteration of the MCMC. The delayed acceptance Metropolis--Hastings MCMC leverages a reduced model for the forward map to lower the compute cost per iteration, though necessarily reduces statistical efficiency that can, without care, lead to no reduction in the computational cost of computing estimates to a desired accuracy. Randomizing the reduced model for the forward map can dramatically improve computational efficiency, by maintaining the low cost per iteration but also avoiding appreciable loss of statistical efficiency. Randomized maps are constructed by a posteriori adaptive tuning of a randomized and locally-corrected deterministic reduced model. Equivalently, the approximated posterior distribution may be viewed as induced by a modified likelihood function for use with the reduced map, with parameters tuned to optimize the quality of the approximation to the correct posterior distribution. Conditions for adaptive MCMC algorithms allow practical approximations and algorithms that have guaranteed ergodicity for the target distribution. Good statistical and computational efficiencies are demonstrated in examples of calibration of large-scale numerical models of geothermal reservoirs and electrical capacitance tomography.

0 citations0 reviewsNo code linkedOpen access
preprint2018arXiv

Adaptive Smoothing for Trajectory Reconstruction

Trajectory reconstruction is the process of inferring the path of a moving object between successive observations. In this paper, we propose a smoothing spline -- which we name the V-spline -- that incorporates position and velocity information and a penalty term that controls acceleration. We introduce a particular adaptive V-spline designed to control the impact of irregularly sampled observations and noisy velocity measurements. A cross-validation scheme for estimating the V-spline parameters is given and we detail the performance of the V-spline on four particularly challenging test datasets. Finally, an application of the V-spline to vehicle trajectory reconstruction in two dimensions is given, in which the penalty term is allowed to further depend on known operational characteristics of the vehicle.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Fast sampling in a linear-Gaussian inverse problem

We solve the inverse problem of deblurring a pixelized image of Jupiter using regularized deconvolution and by sample-based Bayesian inference. By efficiently sampling the marginal posterior distribution for hyperparameters, then the full conditional for the deblurred image, we find that we can evaluate the posterior mean faster than regularized inversion, when selection of the regularizing parameter is considered. To our knowledge, this is the first demonstration of sampling and inference that takes less compute time than regularized inversion in an inverse problems. Comparison to random-walk Metropolis-Hastings and block Gibbs MCMC shows that marginal then conditional sampling also outperforms these more common sampling algorithms, having better scaling with problem size. When problem-specific computations are feasible the asymptotic cost of an independent sample is one linear solve, implying that sample-based Bayesian inference may be performed directly over function spaces, when that limit exists.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Metropolis-Hastings algorithms with autoregressive proposals, and a few examples

We analyse computational efficiency of Metropolis-Hastings algorithms with stochastic AR(1) process proposals. These proposals include, as a subclass, discretized Langevin diffusion (e.g. MALA) and discretized Hamiltonian dynamics (e.g. HMC). We derive expressions for the expected acceptance rate and expected jump size for MCMC methods with general stochastic AR(1) process proposals for the case where the target distribution is absolutely continuous with respect to a Gaussian and the covariance of the Gaussian is allowed to have off-diagonal terms. This allows us to extend what is known about several MCMC methods as well as determining the efficiency of new MCMC methods of this type. In the special case of Hybrid Monte Carlo, we can determine the optimal integration time and the effect of the choice of mass matrix. By including the effect of Metropolis-Hastings we also extend results by Fox and Parker, who used matrix splitting techniques to analyse the performance and improve efficiency of stochastic AR(1) processes for sampling from Gaussian distributions.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Numerical approximation of the Frobenius-Perron operator using the finite volume method

We develop a finite-dimensional approximation of the Frobenius-Perron operator using the finite volume method applied to the continuity equation for the evolution of probability. A Courant-Friedrichs-Lewy condition ensures that the approximation satisfies the Markov property, while existing convergence theory for the finite volume method guarantees convergence of the discrete operator to the continuous operator as mesh size tends to zero. Properties of the approximation are demonstrated in a computed example of sequential inference for the state of a low-dimensional mechanical system when observations give rise to multi-modal distributions.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Sampling hyperparameters in hierarchical models: improving on Gibbs for high-dimensional latent fields and large data sets

We consider posterior sampling in the very common Bayesian hierarchical model in which observed data depends on high-dimensional latent variables that, in turn, depend on relatively few hyperparameters. When the full conditional over the latent variables has a known form, the marginal posterior distribution over hyperparameters is accessible and can be sampled using a Markov chain Monte Carlo (MCMC) method on a low-dimensional parameter space. This may improve computational efficiency over standard Gibbs sampling since computation is not over the high-dimensional space of latent variables and correlations between hyperparameters and latent variables become irrelevant. When the marginal posterior over hyperparameters depends on a fixed-dimensional sufficient statistic, precomputation of the sufficient statistic renders the cost of the low-dimensional MCMC independent of data size. Then, when the hyperparameters are the primary variables of interest, inference may be performed in big-data settings at modest cost. Moreover, since the form of the full conditional for the latent variables does not depend on the form of the hyperprior distribution, the method imposes no restriction on the hyperprior, unlike Gibbs sampling that typically requires conjugate distributions. We demonstrate these efficiency gains in four computed examples.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Accelerated Gibbs sampling of normal distributions using matrix splittings and polynomials

Standard Gibbs sampling applied to a multivariate normal distribution with a specified precision matrix is equivalent in fundamental ways to the Gauss-Seidel iterative solution of linear equations in the precision matrix. Specifically, the iteration operators, the conditions under which convergence occurs, and geometric convergence factors (and rates) are identical. These results hold for arbitrary matrix splittings from classical iterative methods in numerical linear algebra giving easy access to mature results in that field, including existing convergence results for antithetic-variable Gibbs sampling, REGS sampling, and generalizations. Hence, efficient deterministic stationary relaxation schemes lead to efficient generalizations of Gibbs sampling. The technique of polynomial acceleration that significantly improves the convergence rate of an iterative solver derived from a \emph{symmetric} matrix splitting may be applied to accelerate the equivalent generalized Gibbs sampler. Identicality of error polynomials guarantees convergence of the inhomogeneous Markov chain, while equality of convergence factors ensures that the optimal solver leads to the optimal sampler. Numerical examples are presented, including a Chebyshev accelerated SSOR Gibbs sampler applied to a stylized demonstration of low-level Bayesian image reconstruction in a large 3-dimensional linear inverse problem.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Efficiency and computability of MCMC with Langevin, Hamiltonian, and other matrix-splitting proposals

We analyse computational efficiency of Metropolis-Hastings algorithms with AR(1) process proposals. These proposals include, as a subclass, discretized Langevin diffusion (e.g. MALA) and discretized Hamiltonian dynamics (e.g. HMC). By including the effect of Metropolis-Hastings we extend earlier work by Fox and Parker, who used matrix splitting techniques to analyse the performance and improve efficiency of AR(1) processes for targeting Gaussian distributions. Our research enables analysis of MCMC methods that draw samples from non-Gaussian target distributions by using AR(1) process proposals in Metropolis-Hastings algorithms, by analysing the matrix splitting of the precision matrix for a local Gaussian approximation of the non-Gaussian target.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Efficient recycled algorithms for quantitative trait models on phylogenies

We present an efficient and flexible method for computing likelihoods of phenotypic traits on a phylogeny. The method does not resort to Monte-Carlo computation but instead blends Felsenstein's discrete character pruning algorithm with methods for numerical quadrature. It is not limited to Gaussian models and adapts readily to model uncertainty in the observed trait values. We demonstrate the framework by developing efficient algorithms for likelihood calculation and ancestral state reconstruction under Wright's threshold model, applying our methods to a dataset of trait data for extrafloral nectaries (EFNs) across a phylogeny of 839 Labales species.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Coupled MCMC with a randomized acceptance probability

We consider Metropolis Hastings MCMC in cases where the log of the ratio of target distributions is replaced by an estimator. The estimator is based on m samples from an independent online Monte Carlo simulation. Under some conditions on the distribution of the estimator the process resembles Metropolis Hastings MCMC with a randomized transition kernel. When this is the case there is a correction to the estimated acceptance probability which ensures that the target distribution remains the equilibrium distribution. The simplest versions of the Penalty Method of Ceperley and Dewing (1999), the Universal Algorithm of Ball et al. (2003) and the Single Variable Exchange algorithm of Murray et al. (2006) are special cases. In many applications of interest the correction terms cannot be computed. We consider approximate versions of the algorithms. We show that on average O(m) of the samples realized by a simulation approximating a randomized chain of length n are exactly the same as those of a coupled (exact) randomized chain. Approximation biases Monte Carlo estimates with terms O(1/m) or smaller. This should be compared to the Monte Carlo error which is O(1/sqrt(n)).

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

On optimal direction gibbs sampling

Generalized Gibbs kernels are those that may take any direction not necessarily bounded to each axis along the parameters of the objective function. We study how to optimally choose such directions in a Directional, random scan, Gibbs sampler setting. The optimal direction is chosen by minimizing to the mutual information (Kullback-Leibler divergence) of two steps of the MCMC for a truncated Normal objective function. The result is generalized to be used when a Multivariate Normal (local) approximation is available for the objective function. Three Gibbs direction distributions are tested in highly skewed non-normal objective functions.

0 citations0 reviewsNo code linkedOpen access