BZPEERPapers, people, evidence and research paths
Menu

Discover

SearchFind papers, people, topics, institutions and opportunities from one place.GraphSee how papers, authors, topics and evidence connect.

Research tools

HomeContinue from the papers, people and topics that matter now.CollectionsTurn saved papers into reading lists, evidence maps and shareable context.ResearchReconstruct the path that led you from search to paper to question.CollaborationMove from a paper, topic or expert request into a focused thread.PublishCreate full-text research articles with metadata, formulas and versions.

Network

ExpertsFind specialists whose work and reviews explain why they can help.CommunitiesJoin research groups organized around papers, topics and shared evidence.

Opportunities

ListingsBrowse roles, calls and collaborations connected to real research context.

Account

Log inReturn to your saved papers, graph views and active threads.Create accountStart saving papers, building a research trail and joining teams.
Log inCreate account

Source author record

Benedict Leimkuhler

Benedict Leimkuhler appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record
physics.comp-phcond-mat.stat-mechMachine Learningmath.NAComputationmath.STMethodologyphysics.chem-phphysics.flu-dynStatistics Theorycond-mat.soft

Catalog footprint

What is connected

13works
11topics
4close collaborators

Actions

Connect this record

Log in to claim
Open graphBrowse works

Research graph

See the researcher in context

Open full explorer

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

13 published item(s)

preprint2022arXiv

Randomized Time Riemannian Manifold Hamiltonian Monte Carlo

Hamiltonian Monte Carlo (HMC) algorithms which combine numerical approximation of Hamiltonian dynamics on finite intervals with stochastic refreshment and Metropolis correction are popular sampling schemes, but it is known that they may suffer from slow convergence in the continuous time limit. A recent paper of Bou-Rabee and Sanz-Serna (Ann. Appl. Prob., 27:2159-2194, 2017) demonstrated that this issue can be addressed by simply randomizing the duration parameter of the Hamiltonian paths. In this article, we use the same idea to enhance the sampling efficiency of a constrained version of HMC, with potential benefits in a variety of application settings. We demonstrate both the conservation of the stationary distribution and the ergodicity of the method. We also compare the performance of various schemes in numerical studies of model problems, including an application to high-dimensional covariance estimation.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Assessing numerical methods for molecular and particle simulation

We discuss the design of state-of-the-art numerical methods for molecular dynamics, focusing on the demands of soft matter simulation, where the purposes include sampling and dynamics calculations both in and out of equilibrium. We discuss the characteristics of different algorithms, including their essential conservation properties, the convergence of averages, and the accuracy of numerical discretizations. Formulations of the equations of motion which are suited to both equilibrium and nonequilibrium simulation include Langevin dynamics, dissipative particle dynamics (DPD), and the more recently proposed "pairwise adaptive Langevin" (PAdL) method, which, like DPD but unlike Langevin dynamics, conserves momentum and better matches the relaxation rate of orientational degrees of freedom. PAdL is easy to code and suitable for a variety of problems in nonequilibrium soft matter modeling, our simulations of polymer melts indicate that this method can also provide dramatic improvements in computational efficiency. Moreover we show that PAdL gives excellent control of the relaxation rate to equilibrium. In the nonequilibrium setting, we further demonstrate that while PAdL allows the recovery of accurate shear viscosities at higher shear rates than are possible using the DPD method at identical timestep, it also outperforms Langevin dynamics in terms of stability and accuracy at higher shear rates.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Covariance-Controlled Adaptive Langevin Thermostat for Large-Scale Bayesian Sampling

Monte Carlo sampling for Bayesian posterior inference is a common approach used in machine learning. The Markov Chain Monte Carlo procedures that are used are often discrete-time analogues of associated stochastic differential equations (SDEs). These SDEs are guaranteed to leave invariant the required posterior distribution. An area of current research addresses the computational benefits of stochastic gradient methods in this setting. Existing techniques rely on estimating the variance or covariance of the subsampling error, and typically assume constant variance. In this article, we propose a covariance-controlled adaptive Langevin thermostat that can effectively dissipate parameter-dependent noise while maintaining a desired target distribution. The proposed method achieves a substantial speedup over popular alternative schemes for large-scale machine learning applications.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Pairwise adaptive thermostats for improved accuracy and stability in dissipative particle dynamics

We examine the formulation and numerical treatment of dissipative particle dynamics (DPD) and momentum-conserving molecular dynamics. We show that it is possible to improve both the accuracy and the stability of DPD by employing a pairwise adaptive Langevin thermostat that precisely matches the dynamical characteristics of DPD simulations (e.g., autocorrelation functions) while automatically correcting thermodynamic averages using a negative feedback loop. In the low friction regime, it is possible to replace DPD by a simpler momentum-conserving variant of the Nosé--Hoover--Langevin method based on thermostatting only pairwise interactions; we show that this method has an extra order of accuracy for an important class of observables (a superconvergence result), while also allowing larger timesteps than alternatives. All the methods mentioned in the article are easily implemented. Numerical experiments are performed in both equilibrium and nonequilibrium settings; using Lees--Edwards boundary conditions to induce shear flow.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Partitioned integrators for thermodynamic parameterization of neural networks

Traditionally, neural networks are parameterized using optimization procedures such as stochastic gradient descent, RMSProp and ADAM. These procedures tend to drive the parameters of the network toward a local minimum. In this article, we employ alternative "sampling" algorithms (referred to here as "thermodynamic parameterization methods") which rely on discretized stochastic differential equations for a defined target distribution on parameter space. We show that the thermodynamic perspective already improves neural network training. Moreover, by partitioning the parameters based on natural layer structure we obtain schemes with very rapid convergence for data sets with complicated loss landscapes. We describe easy-to-implement hybrid partitioned numerical algorithms, based on discretized stochastic differential equations, which are adapted to feed-forward neural networks, including a multi-layer Langevin algorithm, AdLaLa (combining the adaptive Langevin and Langevin algorithms) and LOL (combining Langevin and Overdamped Langevin); we examine the convergence of these methods using numerical studies and compare their performance among themselves and in relation to standard alternatives such as stochastic gradient descent and ADAM. We present evidence that thermodynamic parameterization methods can be (i) faster, (ii) more accurate, and (iii) more robust than standard algorithms used within machine learning frameworks.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

TATi-Thermodynamic Analytics ToolkIt: TensorFlow-based software for posterior sampling in machine learning applications

With the advent of GPU-assisted hardware and maturing high-efficiency software platforms such as TensorFlow and PyTorch, Bayesian posterior sampling for neural networks becomes plausible. In this article we discuss Bayesian parametrization in machine learning based on Markov Chain Monte Carlo methods, specifically discretized stochastic differential equations such as Langevin dynamics and extended system methods in which an ensemble of walkers is employed to enhance sampling. We provide a glimpse of the potential of the sampling-intensive approach by studying (and visualizing) the loss landscape of a neural network applied to the MNIST data set. Moreover, we investigate how the sampling efficiency itself can be significantly enhanced through an ensemble quasi-Newton preconditioning method. This article accompanies the release of a new TensorFlow software package, the Thermodynamic Analytics ToolkIt, which is used in the computational experiments.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Adaptive Thermostats for Noisy Gradient Systems

We study numerical methods for sampling probability measures in high dimension where the underlying model is only approximately identified with a gradient system. Extended stochastic dynamical methods are discussed which have application to multiscale models, nonequilibrium molecular dynamics, and Bayesian sampling techniques arising in emerging machine learning applications. In addition to providing a more comprehensive discussion of the foundations of these methods, we propose a new numerical method for the adaptive Langevin/stochastic gradient Nosé--Hoover thermostat that achieves a dramatic improvement in numerical efficiency over the most popular stochastic gradient methods reported in the literature. We also demonstrate that the newly established method inherits a superconvergence property (fourth order convergence to the invariant measure for configurational quantities) recently demonstrated in the setting of Langevin dynamics. Our findings are verified by numerical experiments.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Ensemble preconditioning for Markov chain Monte Carlo simulation

We describe parallel Markov chain Monte Carlo methods that propagate a collective ensemble of paths, with local covariance information calculated from neighboring replicas. The use of collective dynamics eliminates multiplicative noise and stabilizes the dynamics thus providing a practical approach to difficult anisotropic sampling problems in high dimensions. Numerical experiments with model problems demonstrate that dramatic potential speedups, compared to various alternative schemes, are attainable.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

The computation of averages from equilibrium and nonequilibrium Langevin molecular dynamics

We consider numerical methods for thermodynamic sampling, i.e. computing sequences of points distributed according to the Gibbs-Boltzmann distribution, using Langevin dynamics and overdamped Langevin dynamics (Brownian dynamics). A wide variety of numerical methods for Langevin dynamics may be constructed based on splitting the stochastic differential equations into various component parts, each of which may be propagated exactly in the sense of distributions. Each such method may be viewed as generating samples according to an associated invariant measure that differs from the exact canonical invariant measure by a stepsize-dependent perturbation. We provide error estimates a la Talay-Tubaro on the invariant distribution for small stepsize, and compare the sampling bias obtained for various choices of splitting method. We further investigate the overdamped limit and apply the methods in the context of driven systems where the goal is sampling with respect to a nonequilibrium steady state. Our analyses are illustrated by numerical experiments.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Direct control of the small-scale energy balance in 2D fluid dynamics

We explore the direct modification of the pseudo-spectral truncation of 2D, incompressible fluid dynamics to maintain a prescribed kinetic energy spectrum. The method provides a means of simulating fluid states with defined spectral properties, for the purpose of matching simulation statistics to given information, arising from observations, theoretical prediction or high fidelity simulation. In the scheme outlined here, Nosé-Hoover thermostats, commonly used in molecular dynamics, are introduced as feedback controls applied to energy shells of the Fourier-discretized Navier-Stokes equations. As we demonstrate in numerical experiments, the dynamical properties (quantified using autocorrelation functions) are only modestly perturbed by our device, while ensemble dispersion is significantly enhanced in comparison with simulations of a corresponding truncation incorporating hyperviscosity.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Least-biased correction of extended dynamical systems using observational data

We consider dynamical systems evolving near an equilibrium statistical state where the interest is in modelling long term behavior that is consistent with thermodynamic constraints. We adjust the distribution using an entropy-optimizing formulation that can be computed on-the- fly, making possible partial corrections using incomplete information, for example measured data or data computed from a different model (or the same model at a different scale). We employ a thermostatting technique to sample the target distribution with the aim of capturing relavant statistical features while introducing mild dynamical perturbation (thermostats). The method is tested for a point vortex fluid model on the sphere, and we demonstrate both convergence of equilibrium quantities and the ability of the formulation to balance stationary and transient- regime errors.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

On the numerical treatment of dissipative particle dynamics and related systems

We review and compare numerical methods that simultaneously control temperature while preserving the momentum, a family of particle simulation methods commonly used for the modelling of complex fluids and polymers. The class of methods considered includes dissipative particle dynamics (DPD) as well as extended stochastic-dynamics models incorporating a generalized pairwise thermostat scheme in which stochastic forces are eliminated and the coefficient of dissipation is treated as an additional auxiliary variable subject to a feedback (kinetic energy) control mechanism. In the latter case, we consider the addition of a coupling of the auxiliary variable, as in the Nosé-Hoover-Langevin (NHL) method, with stochastic dynamics to ensure ergodicity, and find that the convergence of ensemble averages is substantially improved. To this end, splitting methods are developed and studied in terms of their thermodynamic accuracy, two-point correlation functions, and convergence. In terms of computational efficiency as measured by the ratio of thermodynamic accuracy to CPU time, we report significant advantages in simulation for the pairwise NHL method compared to popular alternative schemes (up to an 80\% improvement), without degradation of convergence rate. The momentum-conserving thermostat technique described here provides a consistent hydrodynamic model in the low-friction regime, but it will also be of use in both equilibrium and nonequilibrium molecular simulation applications owing to its efficiency and simple numerical implementation.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

Robust and efficient configurational molecular sampling via Langevin Dynamics

A wide variety of numerical methods are evaluated and compared for solving the stochastic differential equations encountered in molecular dynamics. The methods are based on the application of deterministic impulses, drifts, and Brownian motions in some combination. The Baker-Campbell-Hausdorff expansion is used to study sampling accuracy following recent work by the authors, which allows determination of the stepsize-dependent bias in configurational averaging. For harmonic oscillators, configurational averaging is exact for certain schemes, which may result in improved performance in the modelling of biomolecules where bond stretches play a prominent role. For general systems, an optimal method can be identified that has very low bias compared to alternatives. In simulations of the alanine dipeptide reported here (both solvated and unsolvated), higher accuracy is obtained without loss of computational efficiency, while allowing large timestep, and with no impairment of the conformational exploration rate (the effective diffusion rate observed in simulation). The optimal scheme is a uniformly better performing algorithm for molecular sampling, with overall efficiency improvements of 25% or more in practical timestep size achievable in vacuum, and with reductions in the error of configurational averages of a factor of ten or more attainable in solvated simulations at large timestep.

0 citations0 reviewsNo code linkedOpen access