BZPEERPapers, people, evidence and research pathsDiscover
Research tools
Network
Opportunities
Account
Source author record
Mark Rudelson appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.
Catalog footprint
Research graph
Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.
BZPEER is loading the nearby papers, people, topics and institutions for this page.
Published work
Let $P$ be a bounded polyhedron defined as the intersection of the non-negative orthant ${\Bbb R}^n_+$ and an affine subspace of codimension $m$ in ${\Bbb R}^n$. We show that a simple and computationally efficient formula approximates the volume of $P$ within a factor of $γ^m$, where $γ>0$ is an absolute constant. The formula provides the best known estimate for the volume of transportation polytopes from a wide family.
This paper deals with the problem of graph matching or network alignment for Erdős--Rényi graphs, which can be viewed as a noisy average-case version of the graph isomorphism problem. Let $G$ and $G'$ be $G(n, p)$ Erdős--Rényi graphs marginally, identified with their adjacency matrices. Assume that $G$ and $G'$ are correlated such that $\mathbb{E}[G_{ij} G'_{ij}] = p(1-α)$. For a permutation $π$ representing a latent matching between the vertices of $G$ and $G'$, denote by $G^π$ the graph obtained from permuting the vertices of $G$ by $π$. Observing $G^π$ and $G'$, we aim to recover the matching $π$. In this work, we show that for every $\varepsilon \in (0,1]$, there is $n_0>0$ depending on $\varepsilon$ and absolute constants $α_0, R > 0$ with the following property. Let $n \ge n_0$, $(1+\varepsilon) \log n \le np \le n^{\frac{1}{R \log \log n}}$, and $0 < α< \min(α_0,\varepsilon/4)$. There is a polynomial-time algorithm $F$ such that $\mathbb{P}\{F(G^π,G')=π\}=1-o(1)$. This is the first polynomial-time algorithm that recovers the exact matching between vertices of correlated Erdős--Rényi graphs with constant correlation with high probability. The algorithm is based on comparison of partition trees associated with the graph vertices.
We consider three different models of sparse random graphs:~undirected and directed Erdős-Rényi graphs, and random bipartite graph with an equal number of left and right vertices. For such graphs we show that if the edge connectivity probability $p \in (0,1)$ satisfies $n p \ge \log n + k(n)$ with $k(n) \to \infty$ as $n \to \infty$, then the adjacency matrix is invertible with probability approaching one (here $n$ is the number of vertices in the two former cases and the number of left and right vertices in the latter case). If $np \le \log n -k(n)$ then these matrices are invertible with probability approaching zero, as $n \to \infty$. In the intermediate region, when $np=\log n + k(n)$, for a bounded sequence $k(n) \in \mathbb{R}$, the event $Ω_0$ that the adjacency matrix has a zero row or a column and its complement both have non-vanishing probability. For such choices of $p$ our results show that conditioned on the event $Ω_0^c$ the matrices are again invertible with probability tending to one. This shows that the primary reason for the non-invertibility of such matrices is the existence of a zero row or a column. The bounds on the probability of the invertibility of these matrices are a consequence of quantitative lower bounds on their smallest singular values. Combining this with an upper bound on the largest singular value of the centered version of these matrices we show that the (modified) condition number is $O(n^{1+o(1)})$ on the event that there is no zero row or column, with large probability. This matches with von Neumann's prediction about the condition number of random matrices up to a factor of $n^{o(1)}$, for the entire range of $p$.
Consider an eigenvector of the adjacency matrix of a G(n, p) graph. A nodal domain is a connected component of the set of vertices where this eigenvector has a constant sign. It is known that with high probability, there are exactly two nodal domains for each eigenvector corresponding to a non-leading eigenvalue. We prove that with high probability, the sizes of these nodal domains are approximately equal to each other.
We analyze the behavior of the Barvinok estimator of the hafnian of even dimension, symmetric matrices with nonnegative entries. We introduce a condition under which the Barvinok estimator achieves subexponential errors, and show that this condition is almost optimal. Using that hafnians count the number of perfect matchings in graphs, we conclude that Barvinok's estimator gives a polynomial-time algorithm for the approximate (up to subexponential errors) evaluation of the number of perfect matchings.
Suppose that we observe $y \in \mathbb{R}^f$ and $X \in \mathbb{R}^{f \times m}$ in the following errors-in-variables model: \begin{eqnarray*} y & = & X_0 β^* + ε\\ X & = & X_0 + W \end{eqnarray*} where $X_0$ is a $f \times m$ design matrix with independent subgaussian row vectors, $ε\in \mathbb{R}^f$ is a noise vector and $W$ is a mean zero $f \times m$ random noise matrix with independent subgaussian column vectors, independent of $X_0$ and $ε$. This model is significantly different from those analyzed in the literature in the sense that we allow the measurement error for each covariate to be a dependent vector across its $f$ observations. Such error structures appear in the science literature when modeling the trial-to-trial fluctuations in response strength shared across a set of neurons. Under sparsity and restrictive eigenvalue type of conditions, we show that one is able to recover a sparse vector $β^* \in \mathbb{R}^m$ from the model given a single observation matrix $X$ and the response vector $y$. We establish consistency in estimating $β^*$ and obtain the rates of convergence in the $\ell_q$ norm, where $q = 1, 2$ for the Lasso-type estimator, and for $q \in [1, 2]$ for a Dantzig-type conic programming estimator. We show error bounds which approach that of the regular Lasso and the Dantzig selector in case the errors in $W$ are tending to 0.
We prove that with high probability, every eigenvector of a random matrix is delocalized in the sense that any subset of its coordinates carries a non-negligible portion of its $\ell_2$ norm. Our results pertain to a wide class of random matrices, including matrices with independent entries, symmetric and skew-symmetric matrices, as well as some other naturally arising ensembles. The matrices can be real and complex; in the latter case we assume that the real and imaginary parts of the entries are independent.
We show that for any $t>1$, the set of unconditional convex bodies in $\mathbb{R}^n$ contains a $t$-separated subset of cardinality at least $\exp \exp (C(t) n)$. This implies that there exists an unconditional convex body in $\mathbb{R}^n$ which cannot be approximated within the distance $d$ by a projection of a polytope with $N$ faces unless $N > \exp(c(d)n)$. We also show that for $t>2$, the cardinality of a $t$-separated set of completely symmetric bodies in $\mathbb{R}^n$ does not exceed $\exp \exp (c(t) \log^2 n)$.
Kernel methods are an extremely popular set of techniques used for many important machine learning and data analysis applications. In addition to having good practical performances, these methods are supported by a well-developed theory. Kernel methods use an implicit mapping of the input data into a high dimensional feature space defined by a kernel function, i.e., a function returning the inner product between the images of two data points in the feature space. Central to any kernel method is the kernel matrix, which is built by evaluating the kernel function on a given sample dataset. In this paper, we initiate the study of non-asymptotic spectral theory of random kernel matrices. These are n x n random matrices whose (i,j)th entry is obtained by evaluating the kernel function on $x_i$ and $x_j$, where $x_1,...,x_n$ are a set of n independent random high-dimensional vectors. Our main contribution is to obtain tight upper bounds on the spectral norm (largest eigenvalue) of random kernel matrices constructed by commonly used kernel functions based on polynomials and Gaussian radial basis. As an application of these results, we provide lower bounds on the distortion needed for releasing the coefficients of kernel ridge regression under attribute privacy, a general privacy notion which captures a large class of privacy definitions. Kernel ridge regression is standard method for performing non-parametric regression that regularly outperforms traditional regression approaches in various domains. Our privacy distortion lower bounds are the first for any kernel technique, and our analysis assumes realistic scenarios for the input, unlike all previous lower bounds for other release problems which only hold under very restrictive input settings.
We prove that an n by n random matrix G with independent entries is completely delocalized. Suppose the entries of G have zero means, variances uniformly bounded below, and a uniform tail decay of exponential type. Then with high probability all unit eigenvectors of G have all coordinates of magnitude O(n^{-1/2}), modulo logarithmic corrections. This comes a consequence of a new, geometric, approach to delocalization for random matrices.
We present estimates on the small singular values of a class of matrices with independent Gaussian entries and inhomogeneous variance profile, satisfying a broad-connectedness condition. Using these estimates and concentration of measure for the spectrum of Gaussian matrices with independent entries, we prove that for a large class of graphs satisfying an appropriate expansion property, the Barvinok--Godsil-Gutman estimator for the permanent achieves sub-exponential errors with high probability.
We study concentration properties of random vectors of the form $AX$, where $X = (X_1, ..., X_n)$ has independent coordinates and $A$ is a given matrix. We show that the distribution of $AX$ is well spread in space whenever the distributions of $X_i$ are well spread on the line. Specifically, assume that the probability that $X_i$ falls in any given interval of length $T$ is at most $p$. Then the probability that $AX$ falls in any given ball of radius $T \|A\|_{HS}$ is at most $(Cp)^{0.9 r(A)}$, where $r(A)$ denotes the stable rank of $A$ and $C$ is an absolute constant.
In this expository note, we give a modern proof of Hanson-Wright inequality for quadratic forms in sub-gaussian random variables. We deduce a useful concentration inequality for sub-gaussian random vectors. Two examples are given to illustrate these results: a concentration of distances between random vectors and subspaces, and a bound on the norms of products of random and deterministic matrices.
We show that a perturbation of any fixed square matrix D by a random unitary matrix is well invertible with high probability. A similar result holds for perturbations by random orthogonal matrices; the only notable exception is when D is close to orthogonal. As an application, these results completely eliminate a hard-to-check condition from the Single Ring Theorem by Guionnet, Krishnapur and Zeitouni.
Non-asymptotic theory of random matrices strives to investigate the spectral properties of random matrices, which are valid with high probability for matrices of a large fixed size. Results obtained in this framework find their applications in high-dimensional convexity, analysis of convergence of algorithms, as well as in random matrix theory itself. In these notes we survey some recent results in this area and describe the techniques aimed for obtaining explicit probability bounds.
We provide an affirmative answer to a problem posed by Barvinok and Veomett, showing that in general an n-dimensional convex body cannot be approximated by a projection of a section of a simplex of a sub-exponential dimension. Moreover, we establish a lower bound of the Banach-Mazur distance between n-dimensional projections of sections of an N-dimensional simplex and a certain convex symmetric body, which is sharp up to a logarithmic factor for all N>n.
We define the row product of K matrices of size d by n as a matrix of size d^K by n, whose row are entry-wise products of rows of these matrices. This construction arises in certain computer science problems. We study the question, to which extent the spectral and geometric properties of the row product of independent random matrices resemble those properties for a d^K by n matrix with independent random entries. In particular, we show that the largest and the smallest singular values of these matrices are of the same order, as long as n is significantly smaller than d^K. We also consider a problem of privately releasing the summary information about a database, and use the previous results to obtain a bound for the minimal amount of noise, which has to be added to the released data to avoid a privacy breach.
We consider the power of linear reconstruction attacks in statistical data privacy, showing that they can be applied to a much wider range of settings than previously understood. Linear attacks have been studied before (Dinur and Nissim PODS'03, Dwork, McSherry and Talwar STOC'07, Kasiviswanathan, Rudelson, Smith and Ullman STOC'10, De TCC'12, Muthukrishnan and Nikolov STOC'12) but have so far been applied only in settings with releases that are obviously linear. Consider a database curator who manages a database of sensitive information but wants to release statistics about how a sensitive attribute (say, disease) in the database relates to some nonsensitive attributes (e.g., postal code, age, gender, etc). We show one can mount linear reconstruction attacks based on any release that gives: a) the fraction of records that satisfy a given non-degenerate boolean function. Such releases include contingency tables (previously studied by Kasiviswanathan et al., STOC'10) as well as more complex outputs like the error rate of classifiers such as decision trees; b) any one of a large class of M-estimators (that is, the output of empirical risk minimization algorithms), including the standard estimators for linear and logistic regression. We make two contributions: first, we show how these types of releases can be transformed into a linear format, making them amenable to existing polynomial-time reconstruction algorithms. This is already perhaps surprising, since many of the above releases (like M-estimators) are obtained by solving highly nonlinear formulations. Second, we show how to analyze the resulting attacks under various distributional assumptions on the data. Specifically, we consider a setting in which the same statistic (either a) or b) above) is released about how the sensitive attribute relates to all subsets of size k (out of a total of d) nonsensitive boolean attributes.
Random matrices are widely used in sparse recovery problems, and the relevant properties of matrices with i.i.d. entries are well understood. The current paper discusses the recently introduced Restricted Eigenvalue (RE) condition, which is among the most general assumptions on the matrix, guaranteeing recovery. We prove a reduction principle showing that the RE condition can be guaranteed by checking the restricted isometry on a certain family of low-dimensional subspaces. This principle allows us to establish the RE condition for several broad classes of random matrices with dependent entries, including random matrices with subgaussian rows and non-trivial covariance structure, as well as matrices with independent rows, and uniformly bounded entries.
The classical random matrix theory is mostly focused on asymptotic spectral properties of random matrices as their dimensions grow to infinity. At the same time many recent applications from convex geometry to functional analysis to information theory operate with random matrices in fixed dimensions. This survey addresses the non-asymptotic theory of extreme singular values of random matrices with independent entries. We focus on recently developed geometric methods for estimating the hard edge of random matrices (the smallest singular value).
We prove an optimal estimate on the smallest singular value of a random subgaussian matrix, valid for all fixed dimensions. For an N by n matrix A with independent and identically distributed subgaussian entries, the smallest singular value of A is at least of the order \sqrt{N} - \sqrt{n-1} with high probability. A sharp estimate on the probability is also obtained.
Let A be a matrix whose entries are real i.i.d. centered random variables with unit variance and suitable moment assumptions. Then the smallest singular value of A is of order n^{-1/2} with high probability. The lower estimate of this type was proved recently by the authors; in this note we establish the matching upper estimate.
We prove two basic conjectures on the distribution of the smallest singular value of random n times n matrices with independent entries. Under minimal moment assumptions, we show that the smallest singular value is of order n^{-1/2}, which is optimal for Gaussian matrices. Moreover, we give a optimal estimate on the tail probability. This comes as a consequence of a new and essentially sharp estimate in the Littlewood-Offord problem: for i.i.d. random variables X_k and real numbers a_k, determine the probability P that the sum of a_k X_k lies near some number v. For arbitrary coefficients a_k of the same order of magnitude, we show that they essentially lie in an arithmetic progression of length 1/p.
We study random submatrices of a large matrix A. We show how to approximately compute A from its random submatrix of the smallest possible size O(r log r) with a small error in the spectral norm, where r = ||A||_F^2 / ||A||_2^2 is the numerical rank of A. The numerical rank is always bounded by, and is a stable relaxation of, the rank of A. This yields an asymptotically optimal guarantee in an algorithm for computing low-rank approximations of A. We also prove asymptotically optimal estimates on the spectral norm and the cut-norm of random submatrices of A. The result for the cut-norm yields a slight improvement on the best known sample complexity for an approximation algorithm for MAX-2CSP problems. We use methods of Probability in Banach spaces, in particular the law of large numbers for operator-valued random variables.
We want to exactly reconstruct a sparse signal f (a vector in R^n of small support) from few linear measurements of f (inner products with some fixed vectors). A nice and intuitive reconstruction by Linear Programming has been advocated since 80-ies by Dave Donoho and his collaborators. Namely, one can relax the reconstruction problem, which is highly nonconvex, to a convex problem -- and, moreover, to a linear program. However, when is exactly the reconstruction problem equivalent to its convex relaxation is an open question. Recent work of many authors shows that the number of measurements k(r,n) needed to exactly reconstruct any r-sparse signal f of length n (a vector in R^n of support r) from its linear measurements with the convex relaxation method is usually O(r polylog(n)). However, known estimates of the number of measurements k(r,n) involve huge constants, in spite of very good performance of the algorithms in practice. In this paper, we consider random Gaussian measurements and random Fourier measurements (a frequency sample of f). For Gaussian measurements, we prove the first guarantees with reasonable constants: k(r,n) < 12 r (2 + log(n/r)), which is optimal up to constants. For Fourier measurements, we prove the best known bound k(r,n) = O(r log(n) . log^2(r) log(r log n)), which is optimal within the log log n and log^3 r factors. Our arguments are based on the technique of Geometric Functional Analysis and Probability in Banach spaces.
We develop an approach through geometric functional analysis to error correcting codes and to reconstruction of signals from few linear measurements. An error correcting code encodes an n-letter word x into an m-letter word y in such a way that x can be decoded correctly when any r letters of y are corrupted. We prove that most linear orthogonal transformations Q from R^n into R^m form efficient and robust robust error correcting codes over reals. The decoder (which corrects the corrupted components of y) is the metric projection onto the range of Q in the L_1 norm. An equivalent problem arises in signal processing: how to reconstruct a signal that belongs to a small class from few linear measurements? We prove that for most sets of Gaussian measurements, all signals of small support can be exactly reconstructed by the L_1 norm minimization. This is a substantial improvement of recent results of Donoho and of Candes and Tao. An equivalent problem in combinatorial geometry is the existence of a polytope with fixed number of facets and maximal number of lower-dimensional facets. We prove that most sections of the cube form such polytopes.
We find a sharp combinatorial bound for the metric entropy of sets in R^n and general classes of functions. This solves two basic combinatorial conjectures on the empirical processes. 1. A class of functions satisfies the uniform Central Limit Theorem if the square root of its combinatorial dimension is integrable. 2. The uniform entropy is equivalent to the combinatorial dimension under minimal regularity. Our method also constructs a nicely bounded coordinate section of a symmetric convex body in R^n. In the operator theory, this essentially proves for all normed spaces the restricted invertibility principle of Bourgain and Tzafriri.
In the paper "Isoperimetry of waists and local versus global asymptotic convex geometries", it was proved that the existence of nicely bounded sections of two symmetric convex bodies K and L implies that the intersection of randomly rotated K and L is nicely bounded. In this appendix, we achieve a polynomial bound on the diameter of that intersection (in the ratio of the dimensions of the sections).
Let $y$ be a random vector in \rn, satisfying $$ \Bbb E \, \tens{y} = id. $$ Let $M$ be a natural number and let $y_1 \etc y_M$ be independent copies of $y$. We prove that for some absolute constant $C$ $$ \enor{\frac{1}{M} \sum_i \tens{y_i} - id} \le C \cdot \frac{\sqrt{\log M}}{\sqrt{M}} \cdot \left ( \enor{y}^{\log M} \right )^{1/ \log M}, $$ provided that the last expression is smaller than 1. We apply this estimate to obtain a new proof of a result of Bourgain concerning the number of random points needed to bring a convex body into a nearly isotropic position.