Source author record

Frederik Heber

Frederik Heber appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record

Machine Learning math.NA math.ST Statistics Theory

Catalog footprint

What is connected

2works

4topics

4close collaborators

Actions

Connect this record

Open graph Browse works

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

preprint2020arXiv

TATi-Thermodynamic Analytics ToolkIt: TensorFlow-based software for posterior sampling in machine learning applications

With the advent of GPU-assisted hardware and maturing high-efficiency software platforms such as TensorFlow and PyTorch, Bayesian posterior sampling for neural networks becomes plausible. In this article we discuss Bayesian parametrization in machine learning based on Markov Chain Monte Carlo methods, specifically discretized stochastic differential equations such as Langevin dynamics and extended system methods in which an ensemble of walkers is employed to enhance sampling. We provide a glimpse of the potential of the sampling-intensive approach by studying (and visualizing) the loss landscape of a neural network applied to the MNIST data set. Moreover, we investigate how the sampling efficiency itself can be significantly enhanced through an ensemble quasi-Newton preconditioning method. This article accompanies the release of a new TensorFlow software package, the Thermodynamic Analytics ToolkIt, which is used in the computational experiments.

preprint2016arXiv

A CG-type method in Banach spaces with an application to computerized tomography

Conjugate Gradient (CG) methods are one of the most effective iterative methods to solve linear equations in Hilbert spaces. So far, they have been inherently bound to these spaces since they make use of the inner product structure. In more general Banach spaces one of the most prominent iterative solvers are Landweber-type methods that essentially resemble the Steepest Descent method applied to the normal equation. More advanced are subspace methods that take up the idea of a Krylov-type search space, wherein an optimal solution is sought. However, they do not share the conjugacy property with CG methods. In this article we propose that the Sequential Subspace Optimization (SESOP) method can be considered as an extension of CG methods to Banach spaces. We employ metric projections to orthogonalize the current search direction with respect to the search space from the last iteration. For the l2-space our method then exactly coincides with the Polak-Ribière type of the CG method when applied to the normal equation. We show that such an orthogonalized search space still leads to weak convergence of the subspace method. Moreover, numerical experiments on a random matrix toy problem and 2D computerized tomography on lp-spaces show superior convergence properties over all p compared to non-orthogonalized search spaces. This especially holds for lp-spaces with small p. We see that the closer we are to an l2-space, the more we recover of the conjugacy property that holds in these spaces, i. e., as expected, the more the convergence behaves independently of the size of the truncated search space.