Source author record

Walid Hachem

Walid Hachem appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record

Catalog footprint

What is connected

25works
14topics
4close collaborators

Actions

Connect this record

Log in to claim

Research graph

See the researcher in context

Open full explorer

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

25 published item(s)

preprint2022arXiv

Convergence of constant step stochastic gradient descent for non-smooth non-convex functions

This paper studies the asymptotic behavior of the constant step Stochastic Gradient Descent for the minimization of an unknown function F , defined as the expectation of a non convex, non smooth, locally Lipschitz random function. As the gradient may not exist, it is replaced by a certain operator: a reasonable choice is to use an element of the Clarke subdifferential of the random function; an other choice is the output of the celebrated backpropagation algorithm, which is popular amongst practionners, and whose properties have recently been studied by Bolte and Pauwels [7]. Since the expectation of the chosen operator is not in general an element of the Clarke subdifferential BF of the mean function, it has been assumed in the literature that an oracle of BF is available. As a first result, it is shown in this paper that such an oracle is not needed for almost all initialization points of the algorithm. Next, in the small step size regime, it is shown that the interpolated trajectory of the algorithm converges in probability (in the compact convergence sense) towards the set of solutions of the differential inclusion. Finally, viewing the iterates as a Markov chain whose transition kernel is indexed by the step size, it is shown that the invariant distribution of the kernel converge weakly to the set of invariant distribution of this differential inclusion as the step size tends to zero. These results show that when the step size is small, with large probability, the iterates eventually lie in a neighborhood of the critical points of the mean function F .

preprint2022arXiv

Spectral measure of empirical autocovariance matrices of high dimensional Gaussian stationary processes

Consider the empirical autocovariance matrix at a given non-zero time lag based on observations from a multivariate complex Gaussian stationary time series. The spectral analysis of these autocovariance matrices can be useful in certain statistical problems, such as those related to testing for white noise. We study the behavior of their spectral measures in the asymptotic regime where the time series dimension and the observation window length both grow to infinity, and at the same rate. Following a general framework in the field of the spectral analysis of large random non-Hermitian matrices, at first the probabilistic behavior of the small singular values of the shifted versions of the autocovariance matrix are obtained. This is then used to infer about the large sample behaviour of the empirical spectral measure of the autocovariance matrices at any lag. Matrix orthogonal polynomials on the unit circle play a crucial role in our study.

preprint2020arXiv

A Fully Stochastic Primal-Dual Algorithm

A new stochastic primal--dual algorithm for solving a composite optimization problem is proposed. It is assumed that all the functions/operators that enter the optimization problem are given as statistical expectations. These expectations are unknown but revealed across time through i.i.d. realizations. The proposed algorithm is proven to converge to a saddle point of the Lagrangian function. In the framework of the monotone operator theory, the convergence proof relies on recent results on the stochastic Forward Backward algorithm involving random monotone operators. An example of convex optimization under stochastic linear constraints is considered.

preprint2020arXiv

Non-Hermitian random matrices with a variance profile (I): Deterministic equivalents and limiting ESDs

For each $n$, let $A_n=(σ_{ij})$ be an $n\times n$ deterministic matrix and let $X_n=(X_{ij})$ be an $n\times n$ random matrix with i.i.d. centered entries of unit variance. We study the asymptotic behavior of the empirical spectral distribution $μ_n^Y$ of the rescaled entry-wise product \[ Y_n = \left(\frac1{\sqrt{n}} σ_{ij}X_{ij}\right). \] For our main result we provide a deterministic sequence of probability measures $μ_n$, each described by a family of Master Equations, such that the difference $μ^Y_n - μ_n$ converges weakly in probability to the zero measure. A key feature of our results is to allow some of the entries $σ_{ij}$ to vanish, provided that the standard deviation profiles $A_n$ satisfy a certain quantitative irreducibility property. An important step is to obtain quantitative bounds on the solutions to an associate system of Schwinger--Dyson equations, which we accomplish in the general sparse setting using a novel graphical bootstrap argument.

preprint2020arXiv

Non-Hermitian random matrices with a variance profile (II): properties and examples

For each $n$, let $A_n=(σ_{ij})$ be an $n\times n$ deterministic matrix and let $X_n=(X_{ij})$ be an $n\times n$ random matrix with i.i.d. centered entries of unit variance. In the companion article Cook et al., we considered the empirical spectral distribution $μ_n^Y$ of the rescaled entry-wise product \[ Y_n = \frac 1{\sqrt{n}} A_n\odot X_n = \left(\frac1{\sqrt{n}} σ_{ij}X_{ij}\right) \] and provided a deterministic sequence of probability measures $μ_n$ such that the difference $μ^Y_n - μ_n$ converges weakly in probability to the zero measure. A key feature in Cook et al. was to allow some of the entries $σ_{ij}$ to vanish, provided that the standard deviation profiles $A_n$ satisfy a certain quantitative irreducibility property. In the present article, we provide more information on the sequence $(μ_n)$, described by a family of Master Equations. We consider these equations in important special cases such as separable variance profiles $σ^2_{ij}=d_i \widetilde d_j$ and sampled variance profiles $σ^2_{ij} = σ^2\left(\frac in, \frac jn \right)$ where $(x,y)\mapsto σ^2(x,y)$ is a given function on $[0,1]^2$. Associate examples are provided where $μ_n^Y$ converges to a genuine limit. We study $μ_n$'s behavior at zero and provide examples where $μ_n$'s density is bounded, blows up, or vanishes while an atom appears. As a consequence, we identify the profiles that yield the circular law. Finally, building upon recent results from Alt et al., we prove that except maybe in zero, $μ_n$ admits a positive density on the centered disc of radius $\sqrt{ρ(V_n)}$, where $V_n=(\frac 1n σ_{ij}^2)$ and $ρ(V_n)$ is its spectral radius.

preprint2016arXiv

Dynamical behavior of a stochastic forward-backward algorithm using random monotone operators

The purpose of this paper is to study the dynamical behavior of the sequence produced by a forward-backward algorithm involving two random maximal monotone operators and a sequence of decreasing step sizes. Defining a mean monotone operator as an Aumann integral, and assuming that the sum of the two mean operators is maximal (sufficient maximality conditions are provided), it is shown that with probability one, the interpolated process obtained from the iterates is an asymptotic pseudo trajectory in the sense of Bena\"ım and Hirsch of the differential inclusion involving the sum of the mean operators. The convergence of the empirical means of the iterates towards a zero of the sum of the mean operators is shown, as well as the convergence of the sequence itself to such a zero under a demipositivity assumption. These results find applications in a wide range of optimization or variational inequality problems in random environments.

preprint2016arXiv

Large complex correlated Wishart matrices: Fluctuations and asymptotic independence at the edges

We study the asymptotic behavior of eigenvalues of large complex correlated Wishart matrices at the edges of the limiting spectrum. In this setting, the support of the limiting eigenvalue distribution may have several connected components. Under mild conditions for the population matrices, we show that for every generic positive edge of that support, there exists an extremal eigenvalue which converges almost surely toward that edge and fluctuates according to the Tracy-Widom law at the scale $N^{2/3}$. Moreover, given several generic positive edges, we establish that the associated extremal eigenvalue fluctuations are asymptotically independent. Finally, when the leftmost edge is the origin (hard edge), the fluctuations of the smallest eigenvalue are described by mean of the Bessel kernel at the scale $N^2$.

preprint2015arXiv

A Coordinate Descent Primal-Dual Algorithm and Application to Distributed Asynchronous Optimization

Based on the idea of randomized coordinate descent of $α$-averaged operators, a randomized primal-dual optimization algorithm is introduced, where a random subset of coordinates is updated at each iteration. The algorithm builds upon a variant of a recent (deterministic) algorithm proposed by Vũ and Condat that includes the well known ADMM as a particular case. The obtained algorithm is used to solve asynchronously a distributed optimization problem. A network of agents, each having a separate cost function containing a differentiable term, seek to find a consensus on the minimum of the aggregate objective. The method yields an algorithm where at each iteration, a random subset of agents wake up, update their local estimates, exchange some data with their neighbors, and go idle. Numerical results demonstrate the attractive performance of the method. The general approach can be naturally adapted to other situations where coordinate descent convex optimization algorithms are used with a random choice of the coordinates.

preprint2015arXiv

A Survey on the Eigenvalues Local Behavior of Large Complex Correlated Wishart Matrices

The aim of this note is to provide a pedagogical survey of the recent works by the authors ( arXiv:1409.7548 and arXiv:1507.06013) concerning the local behavior of the eigenvalues of large complex correlated Wishart matrices at the edges and cusp points of the spectrum: Under quite general conditions, the eigenvalues fluctuations at a soft edge of the limiting spectrum, at the hard edge when it is present, or at a cusp point, are respectively described by mean of the Airy kernel, the Bessel kernel, or the Pearcey kernel. Moreover, the eigenvalues fluctuations at several soft edges are asymptotically independent. In particular, the asymptotic fluctuations of the matrix condition number can be described. Finally, the next order term of the hard edge asymptotics is provided.

preprint2015arXiv

Analysis of the limiting spectral measure of large random matrices of the separable covariance type

Consider the random matrix $Σ= D^{1/2} X \widetilde D^{1/2}$ where $D$ and $\widetilde D$ are deterministic Hermitian nonnegative matrices with respective dimensions $N \times N$ and $n \times n$, and where $X$ is a random matrix with independent and identically distributed centered elements with variance $1/n$. Assume that the dimensions $N$ and $n$ grow to infinity at the same pace, and that the spectral measures of $D$ and $\widetilde D$ converge as $N,n \to\infty$ towards two probability measures. Then it is known that the spectral measure of $ΣΣ^*$ converges towards a probability measure $μ$ characterized by its Stieltjes Transform. In this paper, it is shown that $μ$ has a density away from zero, this density is analytical wherever it is positive, and it behaves in most cases as $\sqrt{|x - a|}$ near an edge $a$ of its support. A complete characterization of the support of $μ$ is also provided. \\ Beside its mathematical interest, this analysis finds applications in a certain class of statistical estimation problems.

preprint2015arXiv

Large Complex Correlated Wishart Matrices: The Pearcey Kernel and Expansion at the Hard Edge

We study the eigenvalue behaviour of large complex correlated Wishart matrices near an interior point of the limiting spectrum where the density vanishes (cusp point), and refine the existing results at the hard edge as well. More precisely, under mild assumptions for the population covariance matrix, we show that the limiting density vanishes at generic cusp points like a cube root, and that the local eigenvalue behaviour is described by means of the Pearcey kernel if an extra decay assumption is satisfied. As for the hard edge, we show that the density blows up like an inverse square root at the origin. Moreover, we provide an explicit formula for the $1/N$ correction term for the fluctuation of the smallest random eigenvalue.

preprint2015arXiv

The Shannon's mutual information of a multiple antenna time and frequency dependent channel: an ergodic operator approach

Consider a random non-centered multiple antenna radio transmission channel. Assume that the deterministic part of the channel is itself frequency selective, and that the random multipath part is represented by an ergodic stationary vector process. In the Hilbert space $l^2({\mathbb Z})$, one can associate to this channel a random ergodic self-adjoint operator having a so-called Integrated Density of States (IDS). Shannon's mutual information per receive antenna of this channel coincides then with the integral of a $\log$ function with respect to the IDS. In this paper, it is shown that when the numbers of antennas at the transmitter and at the receiver tend to infinity at the same rate, the mutual information per receive antenna tends to a quantity that can be identified and, in fact, is closely related to that obtained within the random matrix approach. This result can be obtained by analyzing the behavior of the Stieltjes transform of the IDS in the regime of the large numbers of antennas.

preprint2014arXiv

Estimation of Toeplitz Covariance Matrices in Large Dimensional Regime with Application to Source Detection

In this article, we derive concentration inequalities for the spectral norm of two classical sample estimators of large dimensional Toeplitz covariance matrices, demonstrating in particular their asymptotic almost sure consistence. The consistency is then extended to the case where the aggregated matrix of time samples is corrupted by a rank one (or more generally, low rank) matrix. As an application of the latter, the problem of source detection in the context of large dimensional sensor networks within a temporally correlated noise environment is studied. As opposed to standard procedures, this application is performed online, i.e. without the need to possess a learning set of pure noise samples.

preprint2014arXiv

Explicit Convergence Rate of a Distributed Alternating Direction Method of Multipliers

Consider a set of N agents seeking to solve distributively the minimization problem $\inf_{x} \sum_{n = 1}^N f_n(x)$ where the convex functions $f_n$ are local to the agents. The popular Alternating Direction Method of Multipliers has the potential to handle distributed optimization problems of this kind. We provide a general reformulation of the problem and obtain a class of distributed algorithms which encompass various network architectures. The rate of convergence of our method is considered. It is assumed that the infimum of the problem is reached at a point $x_\star$, the functions $f_n$ are twice differentiable at this point and $\sum \nabla^2 f_n(x_\star) > 0$ in the positive definite ordering of symmetric matrices. With these assumptions, it is shown that the convergence to the consensus $x_\star$ is linear and the exact rate is provided. Application examples where this rate can be optimized with respect to the ADMM free parameter $ρ$ are also given.

preprint2014arXiv

Statistical Inference in Large Antenna Arrays under Unknown Noise Pattern

In this article, a general information-plus-noise transmission model is assumed, the receiver end of which is composed of a large number of sensors and is unaware of the noise pattern. For this model, and under reasonable assumptions, a set of results is provided for the receiver to perform statistical eigen-inference on the information part. In particular, we introduce new methods for the detection, counting, and the power and subspace estimation of multiple sources composing the information part of the transmission. The theoretical performance of some of these techniques is also discussed. An exemplary application of these methods to array processing is then studied in greater detail, leading in particular to a novel MUSIC-like algorithm assuming unknown noise covariance.

preprint2013arXiv

Asynchronous Distributed Optimization using a Randomized Alternating Direction Method of Multipliers

Consider a set of networked agents endowed with private cost functions and seeking to find a consensus on the minimizer of the aggregate cost. A new class of random asynchronous distributed optimization methods is introduced. The methods generalize the standard Alternating Direction Method of Multipliers (ADMM) to an asynchronous setting where isolated components of the network are activated in an uncoordinated fashion. The algorithms rely on the introduction of randomized Gauss-Seidel iterations of a Douglas-Rachford operator for finding zeros of a sum of two monotone operators. Convergence to the sought minimizers is provided under mild connectivity conditions. Numerical results sustain our claims.

preprint2013arXiv

Performance of a Distributed Stochastic Approximation Algorithm

In this paper, a distributed stochastic approximation algorithm is studied. Applications of such algorithms include decentralized estimation, optimization, control or computing. The algorithm consists in two steps: a local step, where each node in a network updates a local estimate using a stochastic approximation algorithm with decreasing step size, and a gossip step, where a node computes a local weighted average between its estimates and those of its neighbors. Convergence of the estimates toward a consensus is established under weak assumptions. The approach relies on two main ingredients: the existence of a Lyapunov function for the mean field in the agreement subspace, and a contraction property of the random matrices of weights in the subspace orthogonal to the agreement subspace. A second order analysis of the algorithm is also performed under the form of a Central Limit Theorem. The Polyak-averaged version of the algorithm is also considered.

preprint2013arXiv

The outliers among the singular values of large rectangular random matrices with additive fixed rank deformation

Consider the matrix $Σ_n = n^{-1/2} X_n D_n^{1/2} + P_n$ where the matrix $X_n \in \C^{N\times n}$ has Gaussian standard independent elements, $D_n$ is a deterministic diagonal nonnegative matrix, and $P_n$ is a deterministic matrix with fixed rank. Under some known conditions, the spectral measures of $Σ_n Σ_n^*$ and $n^{-1} X_n D_n X_n^*$ both converge towards a compactly supported probability measure $μ$ as $N,n\to\infty$ with $N/n\to c>0$. In this paper, it is proved that finitely many eigenvalues of $Σ_nΣ_n^*$ may stay away from the support of $μ$ in the large dimensional regime. The existence and locations of these outliers in any connected component of $\R - \support(μ)$ are studied. The fluctuations of the largest outliers of $Σ_nΣ_n^*$ are also analyzed. The results find applications in the fields of signal processing and radio communications.

preprint2012arXiv

A CLT on the SNR of Diagonally Loaded MVDR Filters

This paper studies the fluctuations of the signal-to-noise ratio (SNR) of minimum variance distorsionless response (MVDR) filters implementing diagonal loading in the estimation of the covariance matrix. Previous results in the signal processing literature are generalized and extended by considering both spatially as well as temporarily correlated samples. Specifically, a central limit theorem (CLT) is established for the fluctuations of the SNR of the diagonally loaded MVDR filter, under both supervised and unsupervised training settings in adaptive filtering applications. Our second-order analysis is based on the Nash-Poincaré inequality and the integration by parts formula for Gaussian functionals, as well as classical tools from statistical asymptotic theory. Numerical evaluations validating the accuracy of the CLT confirm the asymptotic Gaussianity of the fluctuations of the SNR of the MVDR filter.

preprint2012arXiv

A Subspace Estimator for Fixed Rank Perturbations of Large Random Matrices

This paper deals with the problem of parameter estimation based on certain eigenspaces of the empirical covariance matrix of an observed multidimensional time series, in the case where the time series dimension and the observation window grow to infinity at the same pace. In the area of large random matrix theory, recent contributions studied the behavior of the extreme eigenvalues of a random matrix and their associated eigenspaces when this matrix is subject to a fixed-rank perturbation. The present work is concerned with the situation where the parameters to be estimated determine the eigenspace structure of a certain fixed-rank perturbation of the empirical covariance matrix. An estimation algorithm in the spirit of the well-known MUSIC algorithm for parameter estimation is developed. It relies on an approach recently developed by Benaych-Georges and Nadakuditi, relating the eigenspaces of extreme eigenvalues of the empirical covariance matrix with eigenspaces of the perturbation matrix. First and second order analyses of the new algorithm are performed.

preprint2012arXiv

Analysis of Sum-Weight-like algorithms for averaging in Wireless Sensor Networks

Distributed estimation of the average value over a Wireless Sensor Network has recently received a lot of attention. Most papers consider single variable sensors and communications with feedback (e.g. peer-to-peer communications). However, in order to use efficiently the broadcast nature of the wireless channel, communications without feedback are advocated. To ensure the convergence in this feedback-free case, the recently-introduced Sum-Weight-like algorithms which rely on two variables at each sensor are a promising solution. In this paper, the convergence towards the consensus over the average of the initial values is analyzed in depth. Furthermore, it is shown that the squared error decreases exponentially with the time. In addition, a powerful algorithm relying on the Sum-Weight structure and taking into account the broadcast nature of the channel is proposed.

preprint2012arXiv

Fluctuations of spiked random matrix models and failure diagnosis in sensor networks

In this article, the joint fluctuations of the extreme eigenvalues and eigenvectors of a large dimensional sample covariance matrix are analyzed when the associated population covariance matrix is a finite-rank perturbation of the identity matrix, corresponding to the so-called spiked model in random matrix theory. The asymptotic fluctuations, as the matrix size grows large, are shown to be intimately linked with matrices from the Gaussian unitary ensemble (GUE). When the spiked population eigenvalues have unit multiplicity, the fluctuations follow a central limit theorem. This result is used to develop an original framework for the detection and diagnosis of local failures in large sensor networks, for known or unknown failure magnitude.

preprint2011arXiv

A CLT for Information-theoretic statistics of Non-centered Gram random matrices

In this article, we study the fluctuations of the random variable: $$ {\mathcal I}_n(ρ) = \frac 1N \log\det(Σ_n Σ_n^* + ρI_N),\quad (ρ>0) $$ where $Σ_n= n^{-1/2} D_n^{1/2} X_n\tilde D_n^{1/2} +A_n$, as the dimensions of the matrices go to infinity at the same pace. Matrices $X_n$ and $A_n$ are respectively random and deterministic $N\times n$ matrices; matrices $D_n$ and $\tilde D_n$ are deterministic and diagonal, with respective dimensions $N\times N$ and $n\times n$; matrix $X_n=(X_{ij})$ has centered, independent and identically distributed entries with unit variance, either real or complex. We prove that when centered and properly rescaled, the random variable ${\mathcal I}_n(ρ)$ satisfies a Central Limit Theorem and has a Gaussian limit. The variance of ${\mathcal I}_n(ρ)$ depends on the moment $\E X_{ij}^2$ of the variables $X_{ij}$ and also on its fourth cumulant $κ= \E|X_{ij}|^4 - 2 - |\E X_{ij}^2|^2$. The main motivation comes from the field of wireless communications, where ${\mathcal I}_n(ρ)$ represents the mutual information of a multiple antenna radio channel. This article closely follows the companion article "A CLT for Information-theoretic statistics of Gram random matrices with a given variance profile", {\em Ann. Appl. Probab. (2008)} by Hachem et al., however the study of the fluctuations associated to non-centered large random matrices raises specific issues, which are addressed here.

preprint2011arXiv

Large information plus noise random matrix models and consistent subspace estimation in large sensor networks

In array processing, a common problem is to estimate the angles of arrival of $K$ deterministic sources impinging on an array of $M$ antennas, from $N$ observations of the source signal, corrupted by gaussian noise. The problem reduces to estimate a quadratic form (called "localization function") of a certain projection matrix related to the source signal empirical covariance matrix. Recently, a new subspace estimation method (called "G-MUSIC") has been proposed, in the context where the number of available samples $N$ is of the same order of magnitude than the number of sensors $M$. In this context, the traditional subspace methods tend to fail because the empirical covariance matrix of the observations is a poor estimate of the source signal covariance matrix. The G-MUSIC method is based on a new consistent estimator of the localization function in the regime where $M$ and $N$ tend to $+\infty$ at the same rate. However, the consistency of the angles estimator was not adressed. The purpose of this paper is to prove the consistency of the angles of arrival estimator in the previous asymptotic regime. To prove this result, we show the property that the singular values of M x N Gaussian information plus noise matrix escape from certain intervals is an event of probability decreasing at rate O(1/N^p) for all p. A regularization trick is also introduced, which allows to confine these singular values into certain intervals and to use standard tools as Poincaré inequality to characterize any moments of the estimator. These results are believed to be of independent interest.

preprint2011arXiv

On bilinear forms based on the resolvent of large random matrices

Consider a matrix $Σ_n$ with random independent entries, each non-centered with a separable variance profile. In this article, we study the limiting behavior of the random bilinear form $u_n^* Q_n(z) v_n$, where $u_n$ and $v_n$ are deterministic vectors, and Q_n(z) is the resolvent associated to $Σ_n Σ_n^*$ as the dimensions of matrix $Σ_n$ go to infinity at the same pace. Such quantities arise in the study of functionals of $Σ_n Σ_n^*$ which do not only depend on the eigenvalues of $Σ_n Σ_n^*$, and are pivotal in the study of problems related to non-centered Gram matrices such as central limit theorems, individual entries of the resolvent, and eigenvalue separation.