BZPEERPapers, people, evidence and research paths
Menu

Discover

SearchFind papers, people, topics, institutions and opportunities from one place.GraphSee how papers, authors, topics and evidence connect.

Research tools

HomeContinue from the papers, people and topics that matter now.CollectionsTurn saved papers into reading lists, evidence maps and shareable context.ResearchReconstruct the path that led you from search to paper to question.CollaborationMove from a paper, topic or expert request into a focused thread.PublishCreate full-text research articles with metadata, formulas and versions.

Network

ExpertsFind specialists whose work and reviews explain why they can help.CommunitiesJoin research groups organized around papers, topics and shared evidence.

Opportunities

ListingsBrowse roles, calls and collaborations connected to real research context.

Account

Log inReturn to your saved papers, graph views and active threads.Create accountStart saving papers, building a research trail and joining teams.
Log inCreate account

Source author record

Helmut Bölcskei

Helmut Bölcskei appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record
Information Theorymath.ITMachine Learningmath.STStatistics Theorymath.FAArtificial IntelligenceComputational ComplexityComputer Visionmath.AGmath.COmath.CVNeural and Evolutionary Computing

Catalog footprint

What is connected

44works
13topics
4close collaborators

Actions

Connect this record

Log in to claim
Open graphBrowse works

Research graph

See the researcher in context

Open full explorer

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

44 published item(s)

preprint2022arXiv

High-Dimensional Distribution Generation Through Deep Neural Networks

We show that every $d$-dimensional probability distribution of bounded support can be generated through deep ReLU networks out of a $1$-dimensional uniform input distribution. What is more, this is possible without incurring a cost - in terms of approximation error measured in Wasserstein-distance - relative to generating the $d$-dimensional target distribution from $d$ independent random variables. This is enabled by a vast generalization of the space-filling approach discovered in (Bailey & Telgarsky, 2018). The construction we propose elicits the importance of network depth in driving the Wasserstein distance between the target distribution and its neural network approximation to zero. Finally, we find that, for histogram target distributions, the number of bits needed to encode the corresponding generative network equals the fundamental limit for encoding probability distributions as dictated by quantization theory.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Canonical Conditions for K/2 Degrees of Freedom

We present a necessary and sufficient condition for $1/2$ degree of freedom for each user in constant $K$-user single-antenna interference channels. This condition applies to all channel topologies, i.e., to fully-connected channels as well as channels that have individual links absent, reflected by corresponding zeros in the channel matrix. Moreover, it captures the essence of interference alignment by virtue of being expressed in terms of a generic injectivity condition that guarantees separability of signal and interference. Finally, we provide codebook constructions achieving $1/2$ degree of freedom for each user for all channel matrices satisfying our condition.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Neural network identifiability for a family of sigmoidal nonlinearities

This paper addresses the following question of neural network identifiability: Does the input-output map realized by a feed-forward neural network with respect to a given nonlinearity uniquely specify the network architecture, weights, and biases? Existing literature on the subject Sussman 1992, Albertini, Sontag et al. 1993, Fefferman 1994 suggests that the answer should be yes, up to certain symmetries induced by the nonlinearity, and provided the networks under consideration satisfy certain "genericity conditions". The results in Sussman 1992 and Albertini, Sontag et al. 1993 apply to networks with a single hidden layer and in Fefferman 1994 the networks need to be fully connected. In an effort to answer the identifiability question in greater generality, we derive necessary genericity conditions for the identifiability of neural networks of arbitrary depth and connectivity with an arbitrary nonlinearity. Moreover, we construct a family of nonlinearities for which these genericity conditions are minimal, i.e., both necessary and sufficient. This family is large enough to approximate many commonly encountered nonlinearities to within arbitrary precision in the uniform norm.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Uncertainty relations and sparse signal recovery

This chapter provides a principled introduction to uncertainty relations underlying sparse signal recovery. We start with the seminal work by Donoho and Stark, 1989, which defines uncertainty relations as upper bounds on the operator norm of the band-limitation operator followed by the time-limitation operator, generalize this theory to arbitrary pairs of operators, and then develop -- out of this generalization -- the coherence-based uncertainty relations due to Elad and Bruckstein, 2002, as well as uncertainty relations in terms of concentration of $1$-norm or $2$-norm. The theory is completed with the recently discovered set-theoretic uncertainty relations which lead to best possible recovery thresholds in terms of a general measure of parsimony, namely Minkowski dimension. We also elaborate on the remarkable connection between uncertainty relations and the "large sieve", a family of inequalities developed in analytic number theory. It is finally shown how uncertainty relations allow to establish fundamental limits of practical signal recovery problems such as inpainting, declipping, super-resolution, and denoising of signals corrupted by impulse noise or narrowband interference. Detailed proofs are provided throughout the chapter.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Degrees of freedom in vector interference channels

This paper continues the Wu-Shamai-Verdu program [3] on characterizing the degrees of freedom (DoF) of interference channels (ICs) through Renyi information dimension. Specifically, we find a single-letter formula for the DoF of vector ICs, encompassing multiple-input multiple-output (MIMO) ICs, time- and/or frequency-selective ICs, and combinations thereof, as well as scalar ICs as considered in [3]. The DoF-formula we obtain lower-bounds the DoF of all channels--with respect to the choice of the channel matrix--and upper-bounds the DoF of almost all channels. It applies to a large class of noise distributions, and its proof is based on an extension of a result by Guionnet and Shlyakthenko [3] to the vector case in combination with the Ruzsa triangle inequality for differential entropy introduced by Kontoyiannis and Madiman [4]. As in scalar ICs, achieving full DoF requires the use of singular input distributions. Strikingly, in the vector case it suffices to enforce singularity on the joint distribution of each individual transmit vector. This can be realized through signaling in subspaces of the ambient signal space, which is in accordance with the idea of interference alignment, and, most importantly, allows the scalar entries of the transmit vectors to have non-singular distributions. The DoF-formula for vector ICs we obtain enables a unified treatment of "classical" interference alignment a la Cadambe and Jafar [5], and Maddah-Ali et al. [6], and the number-theoretic schemes proposed in [7], [8]. Moreover, it allows to calculate the DoF achieved by new signaling schemes for vector ICs. We furthermore recover the result by Cadambe and Jafar on the non-separability of parallel ICs [9] and we show that almost all parallel ICs are separable in terms of DoF. Finally, our results apply to complex vector ICs, thereby extending the main findings of [2] to the complex case.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Deterministic Performance Analysis of Subspace Methods for Cisoid Parameter Estimation

Performance analyses of subspace algorithms for cisoid parameter estimation available in the literature are predominantly of statistical nature with a focus on asymptotic$-$either in the sample size or the SNR$-$statements. This paper presents a deterministic, finite sample size, and finite-SNR performance analysis of the ESPRIT algorithm and the matrix pencil method. Our results are based, inter alia, on a new upper bound on the condition number of Vandermonde matrices with nodes inside the unit disk. This bound is obtained through a generalization of Hilbert's inequality frequently used in large sieve theory.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Discrete Deep Feature Extraction: A Theory and New Architectures

First steps towards a mathematical theory of deep convolutional neural networks for feature extraction were made---for the continuous-time case---in Mallat, 2012, and Wiatowski and Bölcskei, 2015. This paper considers the discrete case, introduces new convolutional neural network architectures, and proposes a mathematical framework for their analysis. Specifically, we establish deformation and translation sensitivity results of local and global nature, and we investigate how certain structural properties of the input signal are reflected in the corresponding feature vectors. Our theory applies to general filters and general Lipschitz-continuous non-linearities and pooling operators. Experiments on handwritten digit classification and facial landmark detection---including feature importance evaluation---complement the theoretical findings.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Information-Theoretic Limits of Matrix Completion

We propose an information-theoretic framework for matrix completion. The theory goes beyond the low-rank structure and applies to general matrices of "low description complexity". Specifically, we consider $m\times n$ random matrices $\mathbf{X}$ of arbitrary distribution (continuous, discrete, discrete-continuous mixture, or even singular). With $\mathcal{S}$ an $\varepsilon$-support set of $\mathbf{X}$, i.e., $\mathrm{P}[\mathbf{X}\in\mathcal{S}]\geq 1-\varepsilon$, and $\underline{\mathrm{dim}}_\mathrm{B}(\mathcal{S})$ denoting the lower Minkowski dimension of $\mathcal{S}$, we show that $k> \underline{\mathrm{dim}}_\mathrm{B}(\mathcal{S})$ trace inner product measurements with measurement matrices $A_i$, suffice to recover $\mathbf{X}$ with probability of error at most $\varepsilon$. The result holds for Lebesgue a.a. $A_i$ and does not need incoherence between the $A_i$ and the unknown matrix $\mathbf{X}$. We furthermore show that $k> \underline{\mathrm{dim}}_\mathrm{B}(\mathcal{S})$ measurements also suffice to recover the unknown matrix $\mathbf{X}$ from measurements taken with rank-one $A_i$, again this applies to a.a. rank-one $A_i$. Rank-one measurement matrices are attractive as they require less storage space than general measurement matrices and can be applied faster. Particularizing our results to the recovery of low-rank matrices, we find that $k>(m+n-r)r$ measurements are sufficient to recover matrices of rank at most $r$. Finally, we construct a class of rank-$r$ matrices that can be recovered with arbitrarily small probability of error from $k<(m+n-r)r$ measurements.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Lossless Linear Analog Compression

We establish the fundamental limits of lossless linear analog compression by considering the recovery of random vectors ${\boldsymbol{\mathsf{x}}}\in{\mathbb R}^m$ from the noiseless linear measurements ${\boldsymbol{\mathsf{y}}}=\boldsymbol{A}{\boldsymbol{\mathsf{x}}}$ with measurement matrix $\boldsymbol{A}\in{\mathbb R}^{n\times m}$. Specifically, for a random vector ${\boldsymbol{\mathsf{x}}}\in{\mathbb R}^m$ of arbitrary distribution we show that ${\boldsymbol{\mathsf{x}}}$ can be recovered with zero error probability from $n>\inf\underline{\operatorname{dim}}_\mathrm{MB}(U)$ linear measurements, where $\underline{\operatorname{dim}}_\mathrm{MB}(\cdot)$ denotes the lower modified Minkowski dimension and the infimum is over all sets $U\subseteq{\mathbb R}^{m}$ with $\mathbb{P}[{\boldsymbol{\mathsf{x}}}\in U]=1$. This achievability statement holds for Lebesgue almost all measurement matrices $\boldsymbol{A}$. We then show that $s$-rectifiable random vectors---a stochastic generalization of $s$-sparse vectors---can be recovered with zero error probability from $n>s$ linear measurements. From classical compressed sensing theory we would expect $n\geq s$ to be necessary for successful recovery of ${\boldsymbol{\mathsf{x}}}$. Surprisingly, certain classes of $s$-rectifiable random vectors can be recovered from fewer than $s$ measurements. Imposing an additional regularity condition on the distribution of $s$-rectifiable random vectors ${\boldsymbol{\mathsf{x}}}$, we do get the expected converse result of $s$ measurements being necessary. The resulting class of random vectors appears to be new and will be referred to as $s$-analytic random vectors.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Characterizing degrees of freedom through additive combinatorics

We establish a formal connection between the problem of characterizing degrees of freedom (DoF) in constant single-antenna interference channels (ICs), with general channel matrix, and the field of additive combinatorics. The theory we develop is based on a recent breakthrough result by Hochman in fractal geometry. Our first main contribution is an explicit condition on the channel matrix to admit full, i.e., $K/2$ DoF; this condition is satisfied for almost all channel matrices. We also provide a construction of corresponding DoF-optimal input distributions. The second main result is a new DoF-formula exclusively in terms of Shannon entropies. This formula is more amenable to both analytical statements and numerical evaluations than the DoF-formula by Wu et al., which is in terms of Rényi information dimension. We then use the new DoF-formula to shed light on the hardness of finding the exact number of DoF in ICs with rational channel coefficients, and to improve the best known bounds on the DoF of a well-studied channel matrix.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Deep Convolutional Neural Networks Based on Semi-Discrete Frames

Deep convolutional neural networks have led to breakthrough results in practical feature extraction applications. The mathematical analysis of these networks was pioneered by Mallat, 2012. Specifically, Mallat considered so-called scattering networks based on identical semi-discrete wavelet frames in each network layer, and proved translation-invariance as well as deformation stability of the resulting feature extractor. The purpose of this paper is to develop Mallat's theory further by allowing for different and, most importantly, general semi-discrete frames (such as, e.g., Gabor frames, wavelets, curvelets, shearlets, ridgelets) in distinct network layers. This allows to extract wider classes of features than point singularities resolved by the wavelet transform. Our generalized feature extractor is proven to be translation-invariant, and we develop deformation stability results for a larger class of deformations than those considered by Mallat. For Mallat's wavelet-based feature extractor, we get rid of a number of technical conditions. The mathematical engine behind our results is continuous frame theory, which allows us to completely detach the invariance and deformation stability proofs from the particular algebraic structure of the underlying frames.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Density Criteria for the Identification of Linear Time-Varying Systems

This paper addresses the problem of identifying a linear time-varying (LTV) system characterized by a (possibly infinite) discrete set of delays and Doppler shifts. We prove that stable identifiability is possible if the upper uniform Beurling density of the delay-Doppler support set is strictly smaller than 1/2 and stable identifiability is impossible for densities strictly larger than 1/2. The proof of this density theorem reveals an interesting relation between LTV system identification and interpolation in the Bargmann-Fock space. Finally, we introduce a subspace method for solving the system identification problem at hand.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Dimensionality-reduced subspace clustering

Subspace clustering refers to the problem of clustering unlabeled high-dimensional data points into a union of low-dimensional linear subspaces, whose number, orientations, and dimensions are all unknown. In practice one may have access to dimensionality-reduced observations of the data only, resulting, e.g., from undersampling due to complexity and speed constraints on the acquisition device or mechanism. More pertinently, even if the high-dimensional data set is available it is often desirable to first project the data points into a lower-dimensional space and to perform clustering there; this reduces storage requirements and computational cost. The purpose of this paper is to quantify the impact of dimensionality reduction through random projection on the performance of three subspace clustering algorithms, all of which are based on principles from sparse signal recovery. Specifically, we analyze the thresholding based subspace clustering (TSC) algorithm, the sparse subspace clustering (SSC) algorithm, and an orthogonal matching pursuit variant thereof (SSC-OMP). We find, for all three algorithms, that dimensionality reduction down to the order of the subspace dimensions is possible without incurring significant performance degradation. Moreover, these results are order-wise optimal in the sense that reducing the dimensionality further leads to a fundamentally ill-posed clustering problem. Our findings carry over to the noisy case as illustrated through analytical results for TSC and simulations for SSC and SSC-OMP. Extensive experiments on synthetic and real data complement our theoretical findings.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Nonparametric Nearest Neighbor Random Process Clustering

We consider the problem of clustering noisy finite-length observations of stationary ergodic random processes according to their nonparametric generative models without prior knowledge of the model statistics and the number of generative models. Two algorithms, both using the L1-distance between estimated power spectral densities (PSDs) as a measure of dissimilarity, are analyzed. The first algorithm, termed nearest neighbor process clustering (NNPC), to the best of our knowledge, is new and relies on partitioning the nearest neighbor graph of the observations via spectral clustering. The second algorithm, simply referred to as k-means (KM), consists of a single k-means iteration with farthest point initialization and was considered before in the literature, albeit with a different measure of dissimilarity and with asymptotic performance results only. We show that both NNPC and KM succeed with high probability under noise and even when the generative process PSDs overlap significantly, all provided that the observation length is sufficiently large. Our results quantify the tradeoff between the overlap of the generative process PSDs, the noise variance, and the observation length. Finally, we present numerical performance results for synthetic and real data.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Robust Subspace Clustering via Thresholding

The problem of clustering noisy and incompletely observed high-dimensional data points into a union of low-dimensional subspaces and a set of outliers is considered. The number of subspaces, their dimensions, and their orientations are assumed unknown. We propose a simple low-complexity subspace clustering algorithm, which applies spectral clustering to an adjacency matrix obtained by thresholding the correlations between data points. In other words, the adjacency matrix is constructed from the nearest neighbors of each data point in spherical distance. A statistical performance analysis shows that the algorithm exhibits robustness to additive noise and succeeds even when the subspaces intersect. Specifically, our results reveal an explicit tradeoff between the affinity of the subspaces and the tolerable noise level. We furthermore prove that the algorithm succeeds even when the data points are incompletely observed with the number of missing entries allowed to be (up to a log-factor) linear in the ambient dimension. We also propose a simple scheme that provably detects outliers, and we present numerical results on real and synthetic data.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Compressive Nonparametric Graphical Model Selection For Time Series

We propose a method for inferring the conditional indepen- dence graph (CIG) of a high-dimensional discrete-time Gaus- sian vector random process from finite-length observations. Our approach does not rely on a parametric model (such as, e.g., an autoregressive model) for the vector random process; rather, it only assumes certain spectral smoothness proper- ties. The proposed inference scheme is compressive in that it works for sample sizes that are (much) smaller than the number of scalar process components. We provide analytical conditions for our method to correctly identify the CIG with high probability.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Explicit and almost sure conditions for K/2 degrees of freedom

It is well known that in K-user constant single-antenna interference channels K/2 degrees of freedom (DoF) can be achieved for almost all channel matrices. Explicit conditions on the channel matrix to admit K/2 DoF are, however, not available. The purpose of this paper is to identify such explicit conditions, which are satisfied for almost all channel matrices. We also provide a construction of corresponding asymptotically DoF-optimal input distributions. The main technical tool used is a recent breakthrough result by Hochman in fractal geometry.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Neighborhood Selection for Thresholding-based Subspace Clustering

Subspace clustering refers to the problem of clustering high-dimensional data points into a union of low-dimensional linear subspaces, where the number of subspaces, their dimensions and orientations are all unknown. In this paper, we propose a variation of the recently introduced thresholding-based subspace clustering (TSC) algorithm, which applies spectral clustering to an adjacency matrix constructed from the nearest neighbors of each data point with respect to the spherical distance measure. The new element resides in an individual and data-driven choice of the number of nearest neighbors. Previous performance results for TSC, as well as for other subspace clustering algorithms based on spectral clustering, come in terms of an intermediate performance measure, which does not address the clustering error directly. Our main analytical contribution is a performance analysis of the modified TSC algorithm (as well as the original TSC algorithm) in terms of the clustering error directly.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Subspace clustering of dimensionality-reduced data

Subspace clustering refers to the problem of clustering unlabeled high-dimensional data points into a union of low-dimensional linear subspaces, assumed unknown. In practice one may have access to dimensionality-reduced observations of the data only, resulting, e.g., from "undersampling" due to complexity and speed constraints on the acquisition device. More pertinently, even if one has access to the high-dimensional data set it is often desirable to first project the data points into a lower-dimensional space and to perform the clustering task there; this reduces storage requirements and computational cost. The purpose of this paper is to quantify the impact of dimensionality-reduction through random projection on the performance of the sparse subspace clustering (SSC) and the thresholding based subspace clustering (TSC) algorithms. We find that for both algorithms dimensionality reduction down to the order of the subspace dimensions is possible without incurring significant performance degradation. The mathematical engine behind our theorems is a result quantifying how the affinities between subspaces change under random dimensionality reducing projections.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Super-Resolution from Short-Time Fourier Transform Measurements

While spike trains are obviously not band-limited, the theory of super-resolution tells us that perfect recovery of unknown spike locations and weights from low-pass Fourier transform measurements is possible provided that the minimum spacing, $Δ$, between spikes is not too small. Specifically, for a cutoff frequency of $f_c$, Donoho [2] shows that exact recovery is possible if $Δ> 1/f_c$, but does not specify a corresponding recovery method. On the other hand, Candès and Fernandez-Granda [3] provide a recovery method based on convex optimization, which provably succeeds as long as $Δ> 2/f_c$. In practical applications one often has access to windowed Fourier transform measurements, i.e., short-time Fourier transform (STFT) measurements, only. In this paper, we develop a theory of super-resolution from STFT measurements, and we propose a method that provably succeeds in recovering spike trains from STFT measurements provided that $Δ> 1/f_c$.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

Almost Lossless Analog Signal Separation

We propose an information-theoretic framework for analog signal separation. Specifically, we consider the problem of recovering two analog signals from a noiseless sum of linear measurements of the signals. Our framework is inspired by the groundbreaking work of Wu and Verdú (2010) on almost lossless analog compression. The main results of the present paper are a general achievability bound for the compression rate in the analog signal separation problem, an exact expression for the optimal compression rate in the case of signals that have mixed discrete-continuous distributions, and a new technique for showing that the intersection of generic subspaces with subsets of sufficiently small Minkowski dimension is empty. This technique can also be applied to obtain a simplified proof of a key result in Wu and Verdú (2010).

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

Capacity Pre-Log of Noncoherent SIMO Channels via Hironaka's Theorem

We find the capacity pre-log of a temporally correlated Rayleigh block-fading SIMO channel in the noncoherent setting. It is well known that for block-length L and rank of the channel covariance matrix equal to Q, the capacity pre-log in the SISO case is given by 1-Q/L. Here, Q/L can be interpreted as the pre-log penalty incurred by channel uncertainty. Our main result reveals that, by adding only one receive antenna, this penalty can be reduced to 1/L and can, hence, be made to vanish in the large-L limit, even if Q/L remains constant as L goes to infinity. Intuitively, even though the SISO channels between the transmit antenna and the two receive antennas are statistically independent, the transmit signal induces enough statistical dependence between the corresponding receive signals for the second receive antenna to be able to resolve the uncertainty associated with the first receive antenna's channel and thereby make the overall system appear coherent. The proof of our main theorem is based on a deep result from algebraic geometry known as Hironaka's Theorem on the Resolution of Singularities.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

Identification of Sparse Linear Operators

We consider the problem of identifying a linear deterministic operator from its response to a given probing signal. For a large class of linear operators, we show that stable identifiability is possible if the total support area of the operator's spreading function satisfies D<=1/2. This result holds for an arbitrary (possibly fragmented) support region of the spreading function, does not impose limitations on the total extent of the support region, and, most importantly, does not require the support region to be known prior to identification. Furthermore, we prove that stable identifiability of almost all operators is possible if D<1. This result is surprising as it says that there is no penalty for not knowing the support region of the spreading function prior to identification. Algorithms that provably recover all operators with D<=1/2, and almost all operators with D<1 are presented.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

Noisy Subspace Clustering via Thresholding

We consider the problem of clustering noisy high-dimensional data points into a union of low-dimensional subspaces and a set of outliers. The number of subspaces, their dimensions, and their orientations are unknown. A probabilistic performance analysis of the thresholding-based subspace clustering (TSC) algorithm introduced recently in [1] shows that TSC succeeds in the noisy case, even when the subspaces intersect. Our results reveal an explicit tradeoff between the allowed noise level and the affinity of the subspaces. We furthermore find that the simple outlier detection scheme introduced in [1] provably succeeds in the noisy case.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

Subspace Clustering via Thresholding and Spectral Clustering

We consider the problem of clustering a set of high-dimensional data points into sets of low-dimensional linear subspaces. The number of subspaces, their dimensions, and their orientations are unknown. We propose a simple and low-complexity clustering algorithm based on thresholding the correlations between the data points followed by spectral clustering. A probabilistic performance analysis shows that this algorithm succeeds even when the subspaces intersect, and when the dimensions of the subspaces scale (up to a log-factor) linearly in the ambient dimension. Moreover, we prove that the algorithm also succeeds for data points that are subject to erasures with the number of erasures scaling (up to a log-factor) linearly in the ambient dimension. Finally, we propose a simple scheme that provably detects outliers.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

Time-Frequency Foundations of Communications

In the tradition of Gabor's 1946 landmark paper [1], we advocate a time-frequency (TF) approach to communications. TF methods for communications have been proposed very early (see the box History). While several tutorial papers and book chapters on the topic are available (see, e.g., [2]-[4] and references therein), the goal of this paper is to present the fundamental aspects in a coherent and easily accessible manner. Specifically, we establish the role of TF methods in communications across a range of subject areas including TF dispersive channels, orthogonal frequency division multiplexing (OFDM), information-theoretic limits, and system identification and channel estimation. Furthermore, we present fundamental results that are stated in the literature for the continuous-time case in simple linear algebra terms.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Joint Sparsity with Different Measurement Matrices

We consider a generalization of the multiple measurement vector (MMV) problem, where the measurement matrices are allowed to differ across measurements. This problem arises naturally when multiple measurements are taken over time, e.g., and the measurement modality (matrix) is time-varying. We derive probabilistic recovery guarantees showing that---under certain (mild) conditions on the measurement matrices---l2/l1-norm minimization and a variant of orthogonal matching pursuit fail with a probability that decays exponentially in the number of measurements. This allows us to conclude that, perhaps surprisingly, recovery performance does not suffer from the individual measurements being taken through different measurement matrices. What is more, recovery performance typically benefits (significantly) from diversity in the measurement matrices; we specify conditions under which such improvements are obtained. These results continue to hold when the measurements are subject to (bounded) noise.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

On the Sensitivity of Continuous-Time Noncoherent Fading Channel Capacity

The noncoherent capacity of stationary discrete-time fading channels is known to be very sensitive to the fine details of the channel model. More specifically, the measure of the support of the fading-process power spectral density (PSD) determines if noncoherent capacity grows logarithmically in SNR or slower than logarithmically. Such a result is unsatisfactory from an engineering point of view, as the support of the PSD cannot be determined through measurements. The aim of this paper is to assess whether, for general continuous-time Rayleigh-fading channels, this sensitivity has a noticeable impact on capacity at SNR values of practical interest. To this end, we consider the general class of band-limited continuous-time Rayleigh-fading channels that satisfy the wide-sense stationary uncorrelated-scattering (WSSUS) assumption and are, in addition, underspread. We show that, for all SNR values of practical interest, the noncoherent capacity of every channel in this class is close to the capacity of an AWGN channel with the same SNR and bandwidth, independently of the measure of the support of the scattering function (the two-dimensional channel PSD). Our result is based on a lower bound on noncoherent capacity, which is built on a discretization of the channel input-output relation induced by projecting onto Weyl-Heisenberg (WH) sets. This approach is interesting in its own right as it yields a mathematically tractable way of dealing with the mutual information between certain continuous-time random signals.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Sparse Signal Recovery in Hilbert Spaces

This paper reports an effort to consolidate numerous coherence-based sparse signal recovery results available in the literature. We present a single theory that applies to general Hilbert spaces with the sparsity of a signal defined as the number of (possibly infinite-dimensional) subspaces participating in the signal's representation. Our general results recover uncertainty relations and coherence-based recovery thresholds for sparse signals, block-sparse signals, multi-band signals, signals in shift-invariant spaces, and signals in finite unions of (possibly infinite-dimensional) subspaces. Moreover, we improve upon and generalize several of the existing results and, in many cases, we find shortened and simplified proofs.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Sparse Signal Separation in Redundant Dictionaries

We formulate a unified framework for the separation of signals that are sparse in "morphologically" different redundant dictionaries. This formulation incorporates the so-called "analysis" and "synthesis" approaches as special cases and contains novel hybrid setups. We find corresponding coherence-based recovery guarantees for an l1-norm based separation algorithm. Our results recover those reported in Studer and Baraniuk, ACHA, submitted, for the synthesis setting, provide new recovery guarantees for the analysis setting, and form a basis for comparing performance in the analysis and synthesis settings. As an aside our findings complement the D-RIP recovery results reported in Candès et al., ACHA, 2011, for the "analysis" signal recovery problem: minimize_x ||Ψx||_1 subject to ||y - Ax||_2 \leq ε, by delivering corresponding coherence-based recovery results.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Compressive Identification of Linear Operators

We consider the problem of identifying a linear deterministic operator from an input-output measurement. For the large class of continuous (and hence bounded) operators, under additional mild restrictions, we show that stable identifiability is possible if the total support area of the operator's spreading function satisfies D <= 1/2. This result holds for arbitrary (possibly fragmented) support regions of the spreading function, does not impose limitations on the total extent of the support region, and, most importantly, does not require the support region of the spreading function to be known prior to identification. Furthermore, we prove that asking for identifiability of only almost all operators, stable identifiability is possible if D <= 1. This result is surprising as it says that there is no penalty for not knowing the support region of the spreading function prior to identification.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

High-SNR Capacity of Wireless Communication Channels in the Noncoherent Setting: A Primer

This paper, mostly tutorial in nature, deals with the problem of characterizing the capacity of fading channels in the high signal-to-noise ratio (SNR) regime. We focus on the practically relevant noncoherent setting, where neither transmitter nor receiver know the channel realizations, but both are aware of the channel law. We present, in an intuitive and accessible form, two tools, first proposed by Lapidoth & Moser (2003), of fundamental importance to high-SNR capacity analysis: the duality approach and the escape-to-infinity property of capacity-achieving distributions. Furthermore, we apply these tools to refine some of the results that appeared previously in the literature and to simplify the corresponding proofs.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Noncoherent SIMO Pre-Log via Resolution of Singularities

We establish a lower bound on the noncoherent capacity pre-log of a temporally correlated Rayleigh block-fading single-input multiple-output (SIMO) channel. Our result holds for arbitrary rank Q of the channel correlation matrix, arbitrary block-length L > Q, and arbitrary number of receive antennas R, and includes the result in Morgenshtern et al. (2010) as a special case. It is well known that the capacity pre-log for this channel in the single-input single-output (SISO) case is given by 1-Q/L, where Q/L is the penalty incurred by channel uncertainty. Our result reveals that this penalty can be reduced to 1/L by adding only one receive antenna, provided that L \geq 2Q - 1 and the channel correlation matrix satisfies mild technical conditions. The main technical tool used to prove our result is Hironaka's celebrated theorem on resolution of singularities in algebraic geometry.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Recovery of Sparsely Corrupted Signals

We investigate the recovery of signals exhibiting a sparse representation in a general (i.e., possibly redundant or incomplete) dictionary that are corrupted by additive noise admitting a sparse representation in another general dictionary. This setup covers a wide range of applications, such as image inpainting, super-resolution, signal separation, and recovery of signals that are impaired by, e.g., clipping, impulse noise, or narrowband interference. We present deterministic recovery guarantees based on a novel uncertainty relation for pairs of general dictionaries and we provide corresponding practicable recovery algorithms. The recovery guarantees we find depend on the signal and noise sparsity levels, on the coherence parameters of the involved dictionaries, and on the amount of prior knowledge about the signal and noise support sets.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Uncertainty Relations and Sparse Signal Recovery for Pairs of General Signal Sets

We present an uncertainty relation for the representation of signals in two different general (possibly redundant or incomplete) signal sets. This uncertainty relation is relevant for the analysis of signals containing two distinct features each of which can be described sparsely in a suitable general signal set. Furthermore, the new uncertainty relation is shown to lead to improved sparsity thresholds for recovery of signals that are sparse in general dictionaries. Specifically, our results improve on the well-known $(1+1/d)/2$-threshold for dictionaries with coherence $d$ by up to a factor of two. Furthermore, we provide probabilistic recovery guarantees for pairs of general dictionaries that also allow us to understand which parts of a general dictionary one needs to randomize over to "weed out" the sparsity patterns that prohibit breaking the square-root bottleneck.

0 citations0 reviewsNo code linkedOpen access
preprint2010arXiv

Where is Randomness Needed to Break the Square-Root Bottleneck?

As shown by Tropp, 2008, for the concatenation of two orthonormal bases (ONBs), breaking the square-root bottleneck in compressed sensing does not require randomization over all the positions of the nonzero entries of the sparse coefficient vector. Rather the positions corresponding to one of the two ONBs can be chosen arbitrarily. The two-ONB structure is, however, restrictive and does not reveal the property that is responsible for allowing to break the bottleneck with reduced randomness. For general dictionaries we show that if a sub-dictionary with small enough coherence and large enough cardinality can be isolated, the bottleneck can be broken under the same probabilistic model on the sparse coefficient vector as in the two-ONB case.

0 citations0 reviewsNo code linkedOpen access
preprint2009arXiv

Compressed Sensing of Block-Sparse Signals: Uncertainty Relations and Efficient Recovery

We consider compressed sensing of block-sparse signals, i.e., sparse signals that have nonzero coefficients occurring in clusters. An uncertainty relation for block-sparse signals is derived, based on a block-coherence measure, which we introduce. We then show that a block-version of the orthogonal matching pursuit algorithm recovers block $k$-sparse signals in no more than $k$ steps if the block-coherence is sufficiently small. The same condition on block-coherence is shown to guarantee successful recovery through a mixed $\ell_2/\ell_1$-optimization approach. This complements previous recovery results for the block-sparse case which relied on small block-restricted isometry constants. The significance of the results presented in this paper lies in the fact that making explicit use of block-sparsity can provably yield better reconstruction properties than treating the signal as being sparse in the conventional sense, thereby ignoring the additional structure in the problem.

0 citations0 reviewsNo code linkedOpen access
preprint2009arXiv

Improved Sparsity Thresholds Through Dictionary Splitting

Known sparsity thresholds for basis pursuit to deliver the maximally sparse solution of the compressed sensing recovery problem typically depend on the dictionary's coherence. While the coherence is easy to compute, it can lead to rather pessimistic thresholds as it captures only limited information about the dictionary. In this paper, we show that viewing the dictionary as the concatenation of two general sub-dictionaries leads to provably better sparsity thresholds--that are explicit in the coherence parameters of the dictionary and of the individual sub-dictionaries. Equivalently, our results can be interpreted as sparsity thresholds for dictionaries that are unions of two general (i.e., not necessarily orthonormal) sub-dictionaries.

0 citations0 reviewsNo code linkedOpen access
preprint2009arXiv

On the Sensitivity of Noncoherent Capacity to the Channel Model

The noncoherent capacity of stationary discrete-time fading channels is known to be very sensitive to the fine details of the channel model. More specifically, the measure of the set of harmonics where the power spectral density of the fading process is nonzero determines if capacity grows logarithmically in SNR or slower than logarithmically. An engineering-relevant problem is to characterize the SNR value at which this sensitivity starts to matter. In this paper, we consider the general class of continuous-time Rayleigh-fading channels that satisfy the wide-sense stationary uncorrelated-scattering (WSSUS) assumption and are, in addition, underspread. For this class of channels, we show that the noncoherent capacity is close to the AWGN capacity for all SNR values of practical interest, independently of whether the scattering function is compactly supported or not. As a byproduct of our analysis, we obtain an information-theoretic pulse-design criterion for orthogonal frequency-division multiplexing systems.

0 citations0 reviewsNo code linkedOpen access
preprint2009arXiv

Tail Behavior of Sphere-Decoding Complexity in Random Lattices

We analyze the (computational) complexity distribution of sphere-decoding (SD) for random infinite lattices. In particular, we show that under fairly general assumptions on the statistics of the lattice basis matrix, the tail behavior of the SD complexity distribution is solely determined by the inverse volume of a fundamental region of the underlying lattice. Particularizing this result to NxM, N>=M, i.i.d. Gaussian lattice basis matrices, we find that the corresponding complexity distribution is of Pareto-type with tail exponent given by N-M+1. We furthermore show that this tail exponent is not improved by lattice-reduction, which includes layer-sorting as a special case.

0 citations0 reviewsNo code linkedOpen access
preprint2008arXiv

Noncoherent Capacity of Underspread Fading Channels

We derive bounds on the noncoherent capacity of wide-sense stationary uncorrelated scattering (WSSUS) channels that are selective both in time and frequency, and are underspread, i.e., the product of the channel's delay spread and Doppler spread is small. For input signals that are peak constrained in time and frequency, we obtain upper and lower bounds on capacity that are explicit in the channel's scattering function, are accurate for a large range of bandwidth and allow to coarsely identify the capacity-optimal bandwidth as a function of the peak power and the channel's scattering function. We also obtain a closed-form expression for the first-order Taylor series expansion of capacity in the limit of large bandwidth, and show that our bounds are tight in the wideband regime. For input signals that are peak constrained in time only (and, hence, allowed to be peaky in frequency), we provide upper and lower bounds on the infinite-bandwidth capacity and find cases when the bounds coincide and the infinite-bandwidth capacity is characterized exactly. Our lower bound is closely related to a result by Viterbi (1967). The analysis in this paper is based on a discrete-time discrete-frequency approximation of WSSUS time- and frequency-selective channels. This discretization explicitly takes into account the underspread property, which is satisfied by virtually all wireless communication channels.

0 citations0 reviewsNo code linkedOpen access
preprint2007arXiv

Capacity of Underspread Noncoherent WSSUS Fading Channels under Peak Signal Constraints

We characterize the capacity of the general class of noncoherent underspread wide-sense stationary uncorrelated scattering (WSSUS) time-frequency-selective Rayleigh fading channels, under peak constraints in time and frequency and in time only. Capacity upper and lower bounds are found which are explicit in the channel's scattering function and allow to identify the capacity-maximizing bandwidth for a given scattering function and a given peak-to-average power ratio.

0 citations0 reviewsNo code linkedOpen access
preprint2007arXiv

Distributed Transmit Diversity in Relay Networks

We analyze fading relay networks, where a single-antenna source-destination terminal pair communicates through a set of half-duplex single-antenna relays using a two-hop protocol with linear processing at the relay level. A family of relaying schemes is presented which achieves the entire optimal diversity-multiplexing (DM) tradeoff curve. As a byproduct of our analysis, it follows that delay diversity and phase-rolling at the relay level are optimal with respect to the entire DM-tradeoff curve, provided the delays and the modulation frequencies, respectively, are chosen appropriately.

0 citations0 reviewsNo code linkedOpen access