Source author record

Pierfrancesco Urbani

Pierfrancesco Urbani appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record

Catalog footprint

What is connected

26works
6topics
4close collaborators

Actions

Connect this record

Log in to claim

Research graph

See the researcher in context

Open full explorer

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

26 published item(s)

preprint2022arXiv

Field theory for zero temperature soft anharmonic spin glasses in a field

We introduce a finite dimensional anharmonic soft spin glass in a field and show how it allows the construction a field theory at zero temperature and the corresponding loop expansion. The mean field level of the model coincides with a recently introduced fully connected model, the KHGPS model, and it has a spin glass transition in a field at zero temperature driven by the appearance of pseudogapped non-linear excitations. We analyze the zero temperature limit of the theory and the behavior of the bare masses and couplings on approaching the mean field zero temperature critical point. Focusing on the so called replicon sector of the field theory, we show that the bare mass corresponding to fluctuations in this sector is strictly positive at the transition in a certain region of control parameter space. At the same time the two relevant cubic coupling constants $g_1$ and $g_2$ show a non-analytic behavior in their bare values: approaching the critical point at zero temperature, $g_1\to \infty$ while $g_2\propto T$ with a prefactor diverging at the transition. Along the same lines we also develop the field theory to study the density of states of the model in finite dimension. We show that in the mean field limit the density of states converges to the one of the KHGPS model. However the construction allows a treatment of finite dimensional effects in perturbation theory.

preprint2022arXiv

The effective noise of Stochastic Gradient Descent

Stochastic Gradient Descent (SGD) is the workhorse algorithm of deep learning technology. At each step of the training phase, a mini batch of samples is drawn from the training dataset and the weights of the neural network are adjusted according to the performance on this specific subset of examples. The mini-batch sampling procedure introduces a stochastic dynamics to the gradient descent, with a non-trivial state-dependent noise. We characterize the stochasticity of SGD and a recently-introduced variant, \emph{persistent} SGD, in a prototypical neural network model. In the under-parametrized regime, where the final training error is positive, the SGD dynamics reaches a stationary state and we define an effective temperature from the fluctuation-dissipation theorem, computed from dynamical mean-field theory. We use the effective temperature to quantify the magnitude of the SGD noise as a function of the problem parameters. In the over-parametrized regime, where the training error vanishes, we measure the noise magnitude of SGD by computing the average distance between two replicas of the system with the same initialization and two different realizations of SGD noise. We find that the two noise measures behave similarly as a function of the problem parameters. Moreover, we observe that noisier algorithms lead to wider decision boundaries of the corresponding constraint satisfaction problem.

preprint2021arXiv

Dynamical mean-field theory for stochastic gradient descent in Gaussian mixture classification

We analyze in a closed form the learning dynamics of stochastic gradient descent (SGD) for a single-layer neural network classifying a high-dimensional Gaussian mixture where each cluster is assigned one of two labels. This problem provides a prototype of a non-convex loss landscape with interpolating regimes and a large generalization gap. We define a particular stochastic process for which SGD can be extended to a continuous-time limit that we call stochastic gradient flow. In the full-batch limit, we recover the standard gradient flow. We apply dynamical mean-field theory from statistical physics to track the dynamics of the algorithm in the high-dimensional limit via a self-consistent stochastic process. We explore the performance of the algorithm as a function of the control parameters shedding light on how it navigates the loss landscape.

preprint2021arXiv

High dimensional optimization under non-convex excluded volume constraints

We consider high dimensional random optimization problems where the dynamical variables are subjected to non-convex excluded volume constraints. We focus on the case in which the cost function is a simple quadratic cost and the excluded volume constraints are modeled by a perceptron constraint satisfaction problem. We show that depending on the density of constraints, one can have different situations. If the number of constraints is small, one typically has a phase where the ground state of the cost function is unique and sits on the boundary of the island of configurations allowed by the constraints. In this case, there is a hypostatic number of marginally satisfied constraints. If the number of constraints is increased one enters a glassy phase where the cost function has many local minima sitting again on the boundary of the regions of allowed configurations. At the phase transition point, the total number of marginally satisfied constraints becomes equal to the number of degrees of freedom in the problem and therefore we say that these minima are isostatic. We conjecture that by increasing further the constraints the system stays isostatic up to the point where the volume of available phase space shrinks to zero. We derive our results using the replica method and we also analyze a dynamical algorithm, the Karush-Kuhn-Tucker algorithm, through dynamical mean-field theory and we show how to recover the results of the replica approach in the replica symmetric phase.

preprint2021arXiv

Stochasticity helps to navigate rough landscapes: comparing gradient-descent-based algorithms in the phase retrieval problem

In this paper we investigate how gradient-based algorithms such as gradient descent, (multi-pass) stochastic gradient descent, its persistent variant, and the Langevin algorithm navigate non-convex loss-landscapes and which of them is able to reach the best generalization error at limited sample complexity. We consider the loss landscape of the high-dimensional phase retrieval problem as a prototypical highly non-convex example. We observe that for phase retrieval the stochastic variants of gradient descent are able to reach perfect generalization for regions of control parameters where the gradient descent algorithm is not. We apply dynamical mean-field theory from statistical physics to characterize analytically the full trajectories of these algorithms in their continuous-time limit, with a warm start, and for large system sizes. We further unveil several intriguing properties of the landscape and the algorithms such as that the gradient descent can obtain better generalization properties from less informed initializations.

preprint2020arXiv

Complex Dynamics in Simple Neural Networks: Understanding Gradient Flow in Phase Retrieval

Despite the widespread use of gradient-based algorithms for optimizing high-dimensional non-convex functions, understanding their ability of finding good minima instead of being trapped in spurious ones remains to a large extent an open problem. Here we focus on gradient flow dynamics for phase retrieval from random measurements. When the ratio of the number of measurements over the input dimension is small the dynamics remains trapped in spurious minima with large basins of attraction. We find analytically that above a critical ratio those critical points become unstable developing a negative direction toward the signal. By numerical experiments we show that in this regime the gradient flow algorithm is not trapped; it drifts away from the spurious critical points along the unstable direction and succeeds in finding the global minimum. Using tools from statistical physics we characterize this phenomenon, which is related to a BBP-type transition in the Hessian of the spurious minima.

preprint2020arXiv

Critical energy landscape of linear soft spheres

We show that soft spheres interacting with a linear ramp potential when overcompressed beyond the jamming point fall in an amorphous solid phase which is critical, mechanically marginally stable and share many features with the jamming point itself. In the whole phase, the relevant local minima of the potential energy landscape display an isostatic contact network of perfectly touching spheres whose statistics is controlled by an infinite lengthscale. Excitations around such energy minima are non-linear, system spanning, and characterized by a set of non-trivial critical exponents. We perform numerical simulations to measure their values and show that, while they coincide, within numerical precision, with the critical exponents appearing at jamming, the nature of the corresponding excitations is richer. Therefore, linear soft spheres appear as a novel class of finite dimensional systems that self-organize into new, critical, marginally stable, states.

preprint2020arXiv

Marvels and Pitfalls of the Langevin Algorithm in Noisy High-dimensional Inference

Gradient-descent-based algorithms and their stochastic versions have widespread applications in machine learning and statistical inference. In this work we perform an analytic study of the performances of one of them, the Langevin algorithm, in the context of noisy high-dimensional inference. We employ the Langevin algorithm to sample the posterior probability measure for the spiked matrix-tensor model. The typical behaviour of this algorithm is described by a system of integro-differential equations that we call the Langevin state evolution, whose solution is compared with the one of the state evolution of approximate message passing (AMP). Our results show that, remarkably, the algorithmic threshold of the Langevin algorithm is sub-optimal with respect to the one given by AMP. We conjecture this phenomenon to be due to the residual glassiness present in that region of parameters. Finally we show how a landscape-annealing protocol, that uses the Langevin algorithm but violate the Bayes-optimality condition, can approach the performance of AMP.

preprint2020arXiv

Passed & Spurious: Descent Algorithms and Local Minima in Spiked Matrix-Tensor Models

In this work we analyse quantitatively the interplay between the loss landscape and performance of descent algorithms in a prototypical inference problem, the spiked matrix-tensor model. We study a loss function that is the negative log-likelihood of the model. We analyse the number of local minima at a fixed distance from the signal/spike with the Kac-Rice formula, and locate trivialization of the landscape at large signal-to-noise ratios. We evaluate in a closed form the performance of a gradient flow algorithm using integro-differential PDEs as developed in physics of disordered systems for the Langevin dynamics. We analyze the performance of an approximate message passing algorithm estimating the maximum likelihood configuration via its state evolution. We conclude by comparing the above results: while we observe a drastic slow down of the gradient flow dynamics even in the region where the landscape is trivial, both the analyzed algorithms are shown to perform well even in the part of the region of parameters where spurious local minima are present.

preprint2020arXiv

Proliferation of non-linear excitations in the piecewise-linear perceptron

We investigate the properties of local minima of the energy landscape of a continuous non-convex optimization problem, the spherical perceptron with piecewise linear cost function and show that they are critical, marginally stable and displaying a set of pseudogaps, singularities and non-linear excitations whose properties appear to be in the same universality class of jammed packings of hard spheres. The piecewise linear perceptron problem appears as an evolution of the purely linear perceptron optimization problem that has been recently investigated in [1]. Its cost function contains two non-analytic points where the derivative has a jump. Correspondingly, in the non-convex/glassy phase, these two points give rise to four pseudogaps in the force distribution and this induces four power laws in the gap distribution as well. In addition one can define an extended notion of isostaticity and show that local minima appear again to be isostatic in this phase. We believe that our results generalize naturally to more complex cases with a proliferation of non-linear excitations as the number of non-analytic points in the cost function is increased.

preprint2020arXiv

Searching for the Gardner transition in glassy glycerol

We search for a Gardner transition in glassy glycerol, a standard molecular glass, measuring the third harmonics cubic susceptibility $χ_3^{(3)}$ from slightly below the usual glass transition temperature down to $10K$. According to the mean field picture, if local motion within the glass were becoming highly correlated due to the emergence of a Gardner phase then $χ_3^{(3)}$, which is analogous to the dynamical spin-glass susceptibility, should increase and diverge at the Gardner transition temperature $T_G$. We find instead that upon cooling $| χ_3^{(3)} |$ decreases by several orders of magnitude and becomes roughly constant in the regime $100K-10K$. We rationalize our findings by assuming that the low temperature physics is described by localized excitations weakly interacting via a spin-glass dipolar pairwise interaction in a random magnetic field. Our quantitative estimations show that the spin-glass interaction is twenty to fifty times smaller than the local random field contribution, thus rationalizing the absence of the spin-glass Gardner phase. This hints at the fact that a Gardner phase may be suppressed in standard molecular glasses, but it also suggests ways to favor its existence in other amorphous solids and by changing the preparation protocol.

preprint2016arXiv

Breakdown of Elasticity in Amorphous Solids

What characterises a solid is its way to respond to external stresses. Ordered solids, such crystals, display an elastic regime followed by a plastic one, both well understood microscopically in terms of lattice distortion and dislocations. For amorphous solids the situation is instead less clear, and the microscopic understanding of the response to deformation and stress is a very active research topic. Several studies have revealed that even in the elastic regime the response is very jerky at low temperature, resembling very much the one of disordered magnetic materials. Here we show that in a very large class of amorphous solids this behaviour emerges by decreasing the temperature as a phase transition where standard elastic behaviour breaks down. At the transition all non-linear elastic modulii diverge and standard elasticity theory does not hold anymore. Below the transition the response to deformation becomes history and time-dependent.

preprint2015arXiv

Following the evolution of glassy states under external perturbations: the full replica symmetry breaking solution

The state-following technique allows the study of metastable glassy states under external perturbations. Here we show how this construction can be used to study the behavior of glassy states of Hard Spheres in infinite dimensions under compression or shear strain. In a preceding work it has been shown that in both cases, when the external perturbation is sufficiently strong, glassy states undergo a second-order transition, called the Gardner transition, whereupon a hierarchical structure of marginal micro-states manifests within the original glass state. The purpose of this work is to study the solution of the state-following construction in this marginal phase. We show that upon compression, close to the jamming transition, the metastable states are described by a scaling solution characterized by a set of non-trivial critical exponents that agree with the results obtained in a preceding work, and we compute the value of the final jamming density for various glassy states. Moreover we show that under the action of the shear strain, beyond the Gardner point, the metastable states can be followed in the marginal phase and we detect an overshoot in the stress-strain curve in agreement with numerical and experimental observations. Finally we further characterize the Gardner transition point by computing both the critical 4-point susceptibility and the exponents that characterize the critical slowing down of the dynamics within a glassy state close to the transition.

preprint2015arXiv

Nonperturbative fluctuations and metastability in a simple model: from observables to microscopic theory and back

Slow dynamics in glassy systems is often interpreted as due to thermally activated events between "metastable" states. This emphasizes the role of nonperturbative fluctuations, which is especially dramatic when these fluctuations destroy a putative phase transition predicted at the mean-field level. To gain insight into such hard problems, we consider the implementation of a generic back-and-forth process, between microscopic theory and observable behavior via effective theories, in a toy model that is simple enough to allow for a thorough investigation: the one-dimensional $φ^4$ theory at low temperature. We consider two ways of restricting the extent of the fluctuations, which both lead to a nonconvex effective potential (or free energy) : either through a finite-size system or by means of a running infrared cutoff within the nonperturbative Renormalization Group formalism. We discuss the physical insight one can get and the ways to treat strongly nonperturbative fluctuations in this context.

preprint2015arXiv

Quasi equilibrium construction for the long time limit of glassy dynamics

In this paper we review a recent proposal to understand the long time limit of glassy dynamics in terms of an appropriate Markov Chain. [1]. The advantages of the resulting construction are many. The first one is that it gives a quasi equilibrium description on how glassy systems explore the phase space in the slow relaxation part of their dynamics. The second one is that it gives an alternative way to obtain dynamical equations starting from a dynamical rule that is static in spirit. This provides a way to overcome the difficulties encountered in the short time part of the dynamics where current conservation must be enforced. We study this approach in detail in a prototypical mean field disordered spin system, namely the p-spin spherical model, showing how we can obtain the well known equations that describes its dynamics. Then we apply the same approach to structural glasses. We first derive a set of dynamical Ornstein-Zernike equations which are very general in nature. Finally we consider two possible closure schemes for them, namely the Hypernetted Chain approximation of liquid theory and a closure of the BBGKY hierarchy that has been recently introduced by G. Szamel. From both approaches we finally find a set of dynamical Mode-Coupling like equations that are supposed to describe the system in the long time/slow dynamics regime.

preprint2015arXiv

Quasi-equilibrium in glassy dynamics: a liquid theory approach

We introduce a quasi-equilibrium formalism in the theory of liquids in order to obtain a set of coarse grained long time dynamical equations for the two point density correlation functions. Our scheme allows to use typical approximations devised for equilibrium to study long time glassy dynamics. We study the Hypernetted Chain (HNC) approximation and a recent closure scheme by Szamel. In both cases we get dynamical equations that have the structure of the Mode-Coupling (MCT) equations in the long time region. The HNC approach, that was so far used to get equilibrium quantities is thus generalized to a fully consistent scheme where long-time dynamic quantities can also be computed. In the context of this approximation we get an asymptotic description of both equilibrium glassy dynamics at high temperature and of aging dynamics at low temperature. The Szamel approximation on the other hand is shown to lead to the exact Mode Coupling equation of Götze for equilibrium dynamics. We clarify the way phase space is sampled according to MCT during dynamical relaxation.

preprint2015arXiv

Universal Spectrum of Normal Modes in Low-Temperature Glasses: an Exact Solution

We report an analytical study of the vibrational spectrum of the simplest model of jamming, the soft perceptron. We identify two distinct classes of soft modes. The first kind of modes are related to isostaticity and appear only in the close vicinity of the jamming transition. The second kind of modes instead are present everywhere in the glass phase and are related to the hierarchical structure of the potential energy landscape. Our results highlight the universality of the spectrum of normal modes in disordered systems, and open the way towards a detailed analytical understanding of the vibrational spectrum of low-temperature glasses.

preprint2014arXiv

Exact theory of dense amorphous hard spheres in high dimension. III. The full RSB solution

In the first part of this paper, we derive the general replica equations that describe infinite-dimensional hard spheres at any level of replica symmetry breaking (RSB) and in particular in the fullRSB scheme. We show that these equations are formally very similar to the ones that have been derived for spin glass models, thus showing that the analogy between spin glasses and structural glasses conjectured by Kirkpatrick, Thirumalai, and Wolynes is realized in a strong sense in the mean field limit. We also suggest how the computation could be generalized in an approximate way to finite dimensional hard spheres. In the second part of the paper, we discuss the solution of these equations and we derive from it a number of physical predictions. We show that, below the Gardner transition where the 1RSB solution becomes unstable, a fullRSB phase exists and we locate the boundary of the fullRSB phase. Most importantly, we show that the fullRSB solution predicts correctly that jammed packings are isostatic, and allows one to compute analytically the critical exponents associated with the jamming transition, which are missed by the 1RSB solution. We show that these predictions compare very well with numerical results.

preprint2014arXiv

Following the evolution of glassy states under external perturbations: compression and shear-strain

We consider the adiabatic evolution of glassy states under external perturbations. Although the formalism we use is very general, we focus here on infinite-dimensional hard spheres where an exact analysis is possible. We consider perturbations of the boundary, i.e. compression or (volume preserving) shear-strain, and we compute the response of glassy states to such perturbations: pressure and shear-stress. We find that both quantities overshoot before the glass state becomes unstable at a spinodal point where it melts into a liquid (or yields). We also estimate the yield stress of the glass. Finally, we study the stability of the glass basins towards breaking into sub-basins, corresponding to a Gardner transition. We find that close to the dynamical transition, glasses undergo a Gardner transition after an infinitesimal perturbation.

preprint2014arXiv

Fractal free energy landscapes in structural glasses

Glasses are amorphous solids whose constituent particles are caged by their neighbors and thus cannot flow. This sluggishness is often ascribed to the free energy landscape containing multiple minima (basins) separated by high barriers. Here we show, using theory and numerical simulation, that the landscape is much rougher than is classically assumed. Deep in the glass, it undergoes a "roughness transition" to fractal basins. This brings about isostaticity at jamming and marginality of glassy states near jamming. Critical exponents for the basin width, the weak force distribution, and the spatial spread of quasi-contacts at jamming can be analytically determined. Their value is found to be compatible with numerical observations. This advance therefore incorporates the jamming transition of granular materials into the framework of glass theory. Because temperature and pressure control which features of the landscape are experienced, glass mechanics and transport are expected to reflect the features of the topology we discuss here. Hitherto mysterious properties of low-temperature glasses could be explained by this approach.

preprint2014arXiv

Statistical mechanics of the spherical hierarchical model with random fields

We study analytically the equilibrium properties of the spherical hierarchical model in the presence of random fields. The expression for the critical line separating a paramagnetic from a ferromagnetic phase is derived. The critical exponents characterising this phase transition are computed analytically and compared with those of the corresponding $D$-dimensional short-range model, leading to conclude that the usual mapping between one dimensional long-range models and $D$-dimensional short-range models holds exactly for this system, in contrast to models with Ising spins. Moreover, the critical exponents of the pure model and those of the random field model satisfy a relationship that mimics the dimensional reduction rule. The absence of a spin-glass phase is strongly supported by the local stability analysis of the replica symmetric saddle-point as well as by an independent computation of the free-energy using a renormalization-like approach. This latter result enlarges the class of random field models for which the spin-glass phase has been recently ruled out.

preprint2014arXiv

The Gardner transition in finite dimensions

Recent works on hard spheres in the limit of infinite dimensions revealed that glass states, envisioned as meta-basins in configuration space, can break up in a multitude of separate basins at low enough temperature or high enough pressure, leading to the emergence of new kinds of soft-modes and unusual properties. In this paper we study by perturbative renormalisation group techniques the critical properties of this transition, which has been discovered in disordered mean-field models in the '80s. We find that the upper critical dimension $d_u$ above which mean-field results hold is strictly larger than six and apparently non-universal, i.e. system dependent. Below $d_u$, we do not find any perturbative attractive fixed point (except for a tiny region of the 1RSB breaking parameter), thus showing that the transition in three dimensions either is governed by a non-perturbative fixed point unrelated to the Gaussian mean-field one or becomes first order or does not exist. We also discuss possible relationships with the behavior of spin glasses in a field.

preprint2013arXiv

Exact theory of dense amorphous hard spheres in high dimension. II. The high density regime and the Gardner transition

We consider the theory of the glass phase and jamming of hard spheres in the large space dimension limit. Building upon the exact expression for the free-energy functional obtained previously, we find that the Random First Order Transition (RFOT) scenario is realized here with two thermodynamic transitions: the usual Kauzmann point associated with entropy crisis, and a further transition at higher pressures in which a glassy structure of micro-states is developed within each amorphous state. This kind of glass-glass transition into a phase dominating the higher densities was described years ago by Elisabeth Gardner, and may well be a generic feature of RFOT. Micro states that are small excitations of an amorphous matrix -- separated by low entropic or energetic barriers -- thus emerge naturally, and modify the high pressure (or low temperature) limit of the thermodynamic functions.

preprint2013arXiv

Quantitative field theory of the glass transition

We develop a full microscopic replica field theory of the dynamical transition in glasses. By studying the soft modes that appear at the dynamical temperature we obtain an effective theory for the critical fluctuations. This analysis leads to several results: we give expressions for the mean field critical exponents, and we study analytically the critical behavior of a set of four-points correlation functions from which we can extract the dynamical correlation length. Finally, we can obtain a Ginzburg criterion that states the range of validity of our analysis. We compute all these quantities within the Hypernetted Chain Approximation (HNC) for the Gibbs free energy and we find results that are consistent with numerical simulations.

preprint2013arXiv

Static replica approach to critical correlations in glassy systems

We discuss the slow relaxation phenomenon in glassy systems by means of replicas by constructing a static field theory approach to the problem. At the mean field level we study how criticality in the four point correlation functions arises because of the presence of soft modes and we derive an effective replica field theory for these critical fluctuations. By using this at the Gaussian level we obtain many physical quantities: the correlation length, the exponent parameter that controls the Mode-Coupling dynamical exponents for the two-point correlation functions, and the prefactor of the critical part of the four point correlation functions. Moreover we perform a one-loop computation in order to identify the region in which the mean field Gaussian approximation is valid. The result is a Ginzburg criterion for the glass transition. We define and compute in this way a proper Ginzburg number. Finally, we present numerical values of all these quantities obtained from the Hypernetted Chain approximation for the replicated liquid theory.