BZPEERPapers, people, evidence and research paths
Menu

Discover

SearchFind papers, people, topics, institutions and opportunities from one place.GraphSee how papers, authors, topics and evidence connect.

Research tools

HomeContinue from the papers, people and topics that matter now.CollectionsTurn saved papers into reading lists, evidence maps and shareable context.ResearchReconstruct the path that led you from search to paper to question.CollaborationMove from a paper, topic or expert request into a focused thread.PublishCreate full-text research articles with metadata, formulas and versions.

Network

ExpertsFind specialists whose work and reviews explain why they can help.CommunitiesJoin research groups organized around papers, topics and shared evidence.

Opportunities

ListingsBrowse roles, calls and collaborations connected to real research context.

Account

Log inReturn to your saved papers, graph views and active threads.Create accountStart saving papers, building a research trail and joining teams.
Log inCreate account

Source author record

Benoit Dherin

Benoit Dherin appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record
math-phmath.MPmath.SGmath.QAmath.RTMachine Learningmath.ATmath.DG

Catalog footprint

What is connected

14works
8topics
4close collaborators

Actions

Connect this record

Log in to claim
Open graphBrowse works

Research graph

See the researcher in context

Open full explorer

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

14 published item(s)

preprint2022arXiv

Implicit Gradient Regularization

Gradient descent can be surprisingly good at optimizing deep neural networks without overfitting and without explicit regularization. We find that the discrete steps of gradient descent implicitly regularize models by penalizing gradient descent trajectories that have large loss gradients. We call this Implicit Gradient Regularization (IGR) and we use backward error analysis to calculate the size of this regularization. We confirm empirically that implicit gradient regularization biases gradient descent toward flat minima, where test errors are small and solutions are robust to noisy parameter perturbations. Furthermore, we demonstrate that the implicit gradient regularization term can be used as an explicit regularizer, allowing us to control this gradient regularization directly. More broadly, our work indicates that backward error analysis is a useful theoretical approach to the perennial question of how learning rate, model size, and parameter regularization interact to determine the properties of overparameterized models optimized with gradient descent.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

On the Origin of Implicit Regularization in Stochastic Gradient Descent

For infinitesimal learning rates, stochastic gradient descent (SGD) follows the path of gradient flow on the full batch loss function. However moderately large learning rates can achieve higher test accuracies, and this generalization benefit is not explained by convergence bounds, since the learning rate which maximizes test accuracy is often larger than the learning rate which minimizes training loss. To interpret this phenomenon we prove that for SGD with random shuffling, the mean SGD iterate also stays close to the path of gradient flow if the learning rate is small and finite, but on a modified loss. This modified loss is composed of the original loss function and an implicit regularizer, which penalizes the norms of the minibatch gradients. Under mild assumptions, when the batch size is small the scale of the implicit regularization term is proportional to the ratio of the learning rate to the batch size. We verify empirically that explicitly including the implicit regularizer in the loss can enhance the test accuracy when the learning rate is small.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Formal symplectic realizations

We study the relationship between several constructions of symplectic realizations of a given Poisson manifold. Our main result is a general formula for a formal symplectic realization in the case of an arbitrary Poisson structure on $\R^n$. This formula is expressed in terms of rooted trees and elementary differentials, building on the work of Butcher, and the coefficients are shown to be a generalization of Bernoulli numbers appearing in the linear Poisson case. We also show that this realization coincides with a formal version of the original construction of Weinstein, when suitably put in global Darboux form, and with the realization coming from tree-level part of Kontsevich's star product. We provide a simple iterated integral expression for the relevant coefficients and show that they coincide with underlying Kontsevich weights.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Deformation quantization of Leibniz algebras

This paper has two parts. The first part is a review and extension of the methods of integration of Leibniz algebras into Lie racks, including as new feature a new way of integrating 2-cocycles (see Lemma 3.9). In the second part, we use the local integration of a Leibniz algebra h using a Baker-Campbell-Hausdorff type formula in order to deformation quantize its linear dual h^*. More precisely, we define a natural rack product on the set of exponential functions which extends to a rack action on C^{\infty}(h^*).

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

G-Systems and Deformation of G-Actions On Rd

Given a (smooth) action of a Lie group G on Rd we construct a DGA whose Maurer-Cartan elements are in one to one correspondence with some class of defomations of the (induced) G-action on the ring of formal power series with coefficients in the ring of smooth functions on Rd. In the final part of this note we discuss the cohomological obstructions to the existence and to the uniqueness (in a sense to be clarified) of such deformations.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Integration of Lie Algebroid Comorphisms

We show that the path construction integration of Lie algebroids by Lie groupoids is an actual equivalence from the category of integrable Lie algebroids and complete Lie algebroid comorphisms to the category of source 1-connected Lie groupoids and Lie groupoid comorphisms. This allows us to construct an actual symplectization functor in Poisson geometry. We include examples to show that the integrability of comorphisms and Poisson maps may not hold in the absence of a completeness assumption.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Quantization of (volume-preserving) actions on R^d

We associate a space of (formal) representations on the space of h-formal power series with coefficients in the space of smooth functions on Rd (which we call quantizations) with an action of a group on Rd by smooth diffeomorphisms. If the action is further volume preserving, these quantizations can be realized as unitary representations on square summable functions on Rd by bounded h-dependent Fourier integral operators, the formal case corresponding to the asymptotics in the limit h going to zero. We construct DGAs controlling these quantizations and prove existence and rigidity results for them.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Quantizations of Momentum Maps and G-Systems

In this note, we give an explicit formula for a family of deformation quantizations for the momentum map associated with the cotangent lift of a Lie group action on Rd. This family of quantizations is parametrized by the formal G-systems introduced by the authors in a previous paper and allows us to obtain classical invariant Hamiltonians that quantize without anomalies with respect to the quantizations of the action prescribed by the formal G-systems.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Symplectic Microgeometry II: Generating functions

We adapt the notion of generating functions for lagrangian submanifolds to symplectic microgeometry. We show that a symplectic micromorphism always admits a global generating function. As an application, we describe hamiltonian flows as special symplectic micromorphisms whose local generating functions are the solutions of Hamilton-Jacobi equations. We obtain a purely categorical formulation of the temporal evolution in classical mechanics.

0 citations0 reviewsNo code linkedOpen access
preprint2005arXiv

Formal Lagrangian Operad

Given a symplectic manifold $M$, we may define an operad structure on the the spaces $\op^k$ of the Lagrangian submanifolds of $(\bar{M})^k\times M$ via symplectic reduction. If $M$ is also a symplectic groupoid, then its multiplication space is an associative product in this operad. Following this idea, we provide a deformation theory for symplectic groupoids analog to the deformation theory of algebras. It turns out that the semi-classical part of Kontsevich's deformation of $C^\infty(\R^d)$ is a deformation of the trivial symplectic groupoid structure of $T^*\R^d$.

0 citations0 reviewsNo code linkedOpen access
preprint2003arXiv

Formal symplectic groupoid

The multiplicative structure of the trivial symplectic groupoid over $\mathbb R^d$ associated to the zero Poisson structure can be expressed in terms of a generating function. We address the problem of deforming such a generating function in the direction of a non-trivial Poisson structure so that the multiplication remains associative. We prove that such a deformation is unique under some reasonable conditions and we give the explicit formula for it. This formula turns out to be the semi-classical approximation of Kontsevich's deformation formula. For the case of a linear Poisson structure, the deformed generating function reduces exactly to the CBH formula of the associated Lie algebra. The methods used to prove existence are interesting in their own right as they come from an at first sight unrelated domain of mathematics: the Runge--Kutta theory of the numeric integration of ODE's.

0 citations0 reviewsNo code linkedOpen access