BZPEERReading, trust and collaboration for research
Menu

Discover

GraphFollow connections around a paper, topic or saved view.

Workspaces

HomePick up where your research flow left off.InboxHandle requests, replies and notifications from one place.CollectionsCurate reading lists, evidence sets and shareable dossiers.ResearchSee your private provenance graph, trail and imports.CollaborationMove from discovery into focused discussion rooms.PublishDraft, preview and publish full-text research articles.

Network

ExpertsFind specialists worth following or asking for help.CommunitiesJoin research circles organized around shared work.PartnersSee external organizations active around the same topics.

Opportunities

ListingsBrowse roles, calls and collaborations with clearer fit signals.

Account

Log inReturn to your reading and collaboration workspace.Create accountStart saving work, following people and joining teams.
Log inCreate account

Researcher profile

Benoit Dherin

Benoit Dherin contributes to research discovery and scholarly infrastructure.

ResearcherAffiliation not importedOpen to collaborate
math.SGmath-phmath.MPMachine Learningmath.DG

Trust snapshot

Quick read

Trust 21 - EmergingVerification L1Unclaimed author
7works
0followers
5topics
4close collaborators

Actions

Decide how to stay connected

Follow researcher0

Identity and collaboration

How to connect with this researcher

Claiming links this public author record to a researcher profile and unlocks direct collaboration workflows.

Log in to claim

Direct collaboration

Open a focused conversation when the fit is right

Claim this author entity first to unlock direct invitations.

Research graph

See the researcher in context

Open full explorer

Inspect adjacent work, topics, institutions and collaborators without jumping out to a separate graph page.

Building this graph slice

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

7 published item(s)

preprint2022arXiv

Implicit Gradient Regularization

Gradient descent can be surprisingly good at optimizing deep neural networks without overfitting and without explicit regularization. We find that the discrete steps of gradient descent implicitly regularize models by penalizing gradient descent trajectories that have large loss gradients. We call this Implicit Gradient Regularization (IGR) and we use backward error analysis to calculate the size of this regularization. We confirm empirically that implicit gradient regularization biases gradient descent toward flat minima, where test errors are small and solutions are robust to noisy parameter perturbations. Furthermore, we demonstrate that the implicit gradient regularization term can be used as an explicit regularizer, allowing us to control this gradient regularization directly. More broadly, our work indicates that backward error analysis is a useful theoretical approach to the perennial question of how learning rate, model size, and parameter regularization interact to determine the properties of overparameterized models optimized with gradient descent.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

On the Origin of Implicit Regularization in Stochastic Gradient Descent

For infinitesimal learning rates, stochastic gradient descent (SGD) follows the path of gradient flow on the full batch loss function. However moderately large learning rates can achieve higher test accuracies, and this generalization benefit is not explained by convergence bounds, since the learning rate which maximizes test accuracy is often larger than the learning rate which minimizes training loss. To interpret this phenomenon we prove that for SGD with random shuffling, the mean SGD iterate also stays close to the path of gradient flow if the learning rate is small and finite, but on a modified loss. This modified loss is composed of the original loss function and an implicit regularizer, which penalizes the norms of the minibatch gradients. Under mild assumptions, when the batch size is small the scale of the implicit regularization term is proportional to the ratio of the learning rate to the batch size. We verify empirically that explicitly including the implicit regularizer in the loss can enhance the test accuracy when the learning rate is small.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Integration of Lie Algebroid Comorphisms

We show that the path construction integration of Lie algebroids by Lie groupoids is an actual equivalence from the category of integrable Lie algebroids and complete Lie algebroid comorphisms to the category of source 1-connected Lie groupoids and Lie groupoid comorphisms. This allows us to construct an actual symplectization functor in Poisson geometry. We include examples to show that the integrability of comorphisms and Poisson maps may not hold in the absence of a completeness assumption.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Symplectic Microgeometry II: Generating functions

We adapt the notion of generating functions for lagrangian submanifolds to symplectic microgeometry. We show that a symplectic micromorphism always admits a global generating function. As an application, we describe hamiltonian flows as special symplectic micromorphisms whose local generating functions are the solutions of Hamilton-Jacobi equations. We obtain a purely categorical formulation of the temporal evolution in classical mechanics.

0 citations0 reviewsNo code linkedOpen access
preprint2005arXiv

Formal Lagrangian Operad

Given a symplectic manifold $M$, we may define an operad structure on the the spaces $\op^k$ of the Lagrangian submanifolds of $(\bar{M})^k\times M$ via symplectic reduction. If $M$ is also a symplectic groupoid, then its multiplication space is an associative product in this operad. Following this idea, we provide a deformation theory for symplectic groupoids analog to the deformation theory of algebras. It turns out that the semi-classical part of Kontsevich's deformation of $C^\infty(\R^d)$ is a deformation of the trivial symplectic groupoid structure of $T^*\R^d$.

0 citations0 reviewsNo code linkedOpen access