BZPEERPapers, people, evidence and research paths
Menu

Discover

SearchFind papers, people, topics, institutions and opportunities from one place.GraphSee how papers, authors, topics and evidence connect.

Research tools

HomeContinue from the papers, people and topics that matter now.CollectionsTurn saved papers into reading lists, evidence maps and shareable context.ResearchReconstruct the path that led you from search to paper to question.CollaborationMove from a paper, topic or expert request into a focused thread.PublishCreate full-text research articles with metadata, formulas and versions.

Network

ExpertsFind specialists whose work and reviews explain why they can help.CommunitiesJoin research groups organized around papers, topics and shared evidence.

Opportunities

ListingsBrowse roles, calls and collaborations connected to real research context.

Account

Log inReturn to your saved papers, graph views and active threads.Create accountStart saving papers, building a research trail and joining teams.
Log inCreate account

Source author record

Saikat Chatterjee

Saikat Chatterjee appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record
Information Theorymath.ITMachine Learningmath.CTmath-phmath.MPmath.STStatistics Theorymath.DGeess.SPhep-thmath.OCNumerical AnalysisSystems and ControlApplicationsastro-ph.SRComputer Visioncond-mat.dis-nneess.SYGenomicsmath.RTquant-phRobotics

Catalog footprint

What is connected

49works
23topics
4close collaborators

Actions

Connect this record

Log in to claim
Open graphBrowse works

Research graph

See the researcher in context

Open full explorer

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

49 published item(s)

preprint2022arXiv

DeepBayes -- an estimator for parameter estimation in stochastic nonlinear dynamical models

Stochastic nonlinear dynamical systems are ubiquitous in modern, real-world applications. Yet, estimating the unknown parameters of stochastic, nonlinear dynamical models remains a challenging problem. The majority of existing methods employ maximum likelihood or Bayesian estimation. However, these methods suffer from some limitations, most notably the substantial computational time for inference coupled with limited flexibility in application. In this work, we propose DeepBayes estimators that leverage the power of deep recurrent neural networks in learning an estimator. The method consists of first training a recurrent neural network to minimize the mean-squared estimation error over a set of synthetically generated data using models drawn from the model set of interest. The a priori trained estimator can then be used directly for inference by evaluating the network with the estimation data. The deep recurrent neural network architectures can be trained offline and ensure significant time savings during inference. We experiment with two popular recurrent neural networks -- long short term memory network (LSTM) and gated recurrent unit (GRU). We demonstrate the applicability of our proposed method on different example models and perform detailed comparisons with state-of-the-art approaches. We also provide a study on a real-world nonlinear benchmark problem. The experimental evaluations show that the proposed approach is asymptotically as good as the Bayes estimator.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Multi-modal curb detection and filtering

Reliable knowledge of road boundaries is critical for autonomous vehicle navigation. We propose a robust curb detection and filtering technique based on the fusion of camera semantics and dense lidar point clouds. The lidar point clouds are collected by fusing multiple lidars for robust feature detection. The camera semantics are based on a modified EfficientNet architecture which is trained with labeled data collected from onboard fisheye cameras. The point clouds are associated with the closest curb segment with $L_2$-norm analysis after projecting into the image space with the fisheye model projection. Next, the selected points are clustered using unsupervised density-based spatial clustering to detect different curb regions. As new curb points are detected in consecutive frames they are associated with the existing curb clusters using temporal reachability constraints. If no reachability constraints are found a new curb cluster is formed from these new points. This ensures we can detect multiple curbs present in road segments consisting of multiple lanes if they are in the sensors' field of view. Finally, Delaunay filtering is applied for outlier removal and its performance is compared to traditional RANSAC-based filtering. An objective evaluation of the proposed solution is done using a high-definition map containing ground truth curb points obtained from a commercial map supplier. The proposed system has proven capable of detecting curbs of any orientation in complex urban road scenarios comprising straight roads, curved roads, and intersections with traffic isles.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

Robust Classification using Hidden Markov Models and Mixtures of Normalizing Flows

We test the robustness of a maximum-likelihood (ML) based classifier where sequential data as observation is corrupted by noise. The hypothesis is that a generative model, that combines the state transitions of a hidden Markov model (HMM) and the neural network based probability distributions for the hidden states of the HMM, can provide a robust classification performance. The combined model is called normalizing-flow mixture model based HMM (NMM-HMM). It can be trained using a combination of expectation-maximization (EM) and backpropagation. We verify the improved robustness of NMM-HMM classifiers in an application to speech recognition.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

A model-free, data-based forecast for sunspot cycle 25

The dynamic activity of the Sun, governed by its cycle of sunspots -- strongly magnetized regions that are observed on its surface -- modulate our solar system space environment creating space weather. Severe space weather leads to disruptions in satellite operations, telecommunications, electric power grids and air-traffic on polar routes. Forecasting the cycle of sunspots, however, has remained a challenging problem. We use reservoir computing -- a model-free, neural--network based machine-learning technique -- to forecast the upcoming solar cycle, sunspot cycle 25. The standard algorithm forecasts that solar cycle 25 is going to last about ten years, the maxima is going to appear in the year 2024 and the maximum number of sunspots is going to be 113 ($\pm15$). We also develop a novel variation of the standard algorithm whose forecasts for duration and peak timing matches that of the standard algorithm, but whose peak amplitude forecast is 124 ($\pm2$) -- within the upper bound of the standard reservoir computing algorithm. We conclude that sunspot cycle 25 is likely to be a weak, lower than average solar cycle, somewhat similar in strength to sunspot cycle 24.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Asynchronous Decentralized Learning of a Neural Network

In this work, we exploit an asynchronous computing framework namely ARock to learn a deep neural network called self-size estimating feedforward neural network (SSFN) in a decentralized scenario. Using this algorithm namely asynchronous decentralized SSFN (dSSFN), we provide the centralized equivalent solution under certain technical assumptions. Asynchronous dSSFN relaxes the communication bottleneck by allowing one node activation and one side communication, which reduces the communication overhead significantly, consequently increasing the learning speed. We compare asynchronous dSSFN with traditional synchronous dSSFN in the experimental results, which shows the competitive performance of asynchronous dSSFN, especially when the communication network is sparse.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Neural Network based Explicit Mixture Models and Expectation-maximization based Learning

We propose two neural network based mixture models in this article. The proposed mixture models are explicit in nature. The explicit models have analytical forms with the advantages of computing likelihood and efficiency of generating samples. Computation of likelihood is an important aspect of our models. Expectation-maximization based algorithms are developed for learning parameters of the proposed models. We provide sufficient conditions to realize the expectation-maximization based learning. The main requirements are invertibility of neural networks that are used as generators and Jacobian computation of functional form of the neural networks. The requirements are practically realized using a flow-based neural network. In our first mixture model, we use multiple flow-based neural networks as generators. Naturally the model is complex. A single latent variable is used as the common input to all the neural networks. The second mixture model uses a single flow-based neural network as a generator to reduce complexity. The single generator has a latent variable input that follows a Gaussian mixture distribution. We demonstrate efficiency of proposed mixture models through extensive experiments for generating samples and maximum likelihood based classification.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

On two notions of a Gerbe over a stack

Let $\mathcal{G}$ be a Lie groupoid. The category $B\mathcal{G}$ of principal $\mathcal{G}$-bundles defines a differentiable stack. On the other hand, given a differentiable stack $\mathcal{D}$, there exists a Lie groupoid $\mathcal{H}$ such that $B\mathcal{H}$ is isomorphic to $\mathcal{D}$. Define a gerbe over a stack as a morphism of stacks $F\colon \mathcal{D}\rightarrow \mathcal{C}$, such that $F$ and the diagonal map $Δ_F\colon \mathcal{D}\rightarrow \mathcal{D}\times_{\mathcal{C}}\mathcal{D}$ are epimorphisms. This paper explores the relationship between a gerbe defined above and a Morita equivalence class of a Lie groupoid extension.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Powering Hidden Markov Model by Neural Network based Generative Models

Hidden Markov model (HMM) has been successfully used for sequential data modeling problems. In this work, we propose to power the modeling capacity of HMM by bringing in neural network based generative models. The proposed model is termed as GenHMM. In the proposed GenHMM, each HMM hidden state is associated with a neural network based generative model that has tractability of exact likelihood and provides efficient likelihood computation. A generative model in GenHMM consists of mixture of generators that are realized by flow models. A learning algorithm for GenHMM is proposed in expectation-maximization framework. The convergence of the learning GenHMM is analyzed. We demonstrate the efficiency of GenHMM by classification tasks on practical sequential data. Code available at https://github.com/FirstHandScientist/genhmm.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Predictive Analysis of COVID-19 Time-series Data from Johns Hopkins University

We provide a predictive analysis of the spread of COVID-19, also known as SARS-CoV-2, using the dataset made publicly available online by the Johns Hopkins University. Our main objective is to provide predictions of the number of infected people for different countries in the next 14 days. The predictive analysis is done using time-series data transformed on a logarithmic scale. We use two well-known methods for prediction: polynomial regression and neural network. As the number of training data for each country is limited, we use a single-layer neural network called the extreme learning machine (ELM) to avoid over-fitting. Due to the non-stationary nature of the time-series, a sliding window approach is used to provide a more accurate prediction.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

SSFN -- Self Size-estimating Feed-forward Network with Low Complexity, Limited Need for Human Intervention, and Consistent Behaviour across Trials

We design a self size-estimating feed-forward network (SSFN) using a joint optimization approach for estimation of number of layers, number of nodes and learning of weight matrices. The learning algorithm has a low computational complexity, preferably within few minutes using a laptop. In addition the algorithm has a limited need for human intervention to tune parameters. SSFN grows from a small-size network to a large-size network, guaranteeing a monotonically non-increasing cost with addition of nodes and layers. The learning approach uses judicious a combination of `lossless flow property' of some activation functions, convex optimization and instance of random matrix. Consistent performance -- low variation across Monte-Carlo trials -- is found for inference performance (classification accuracy) and estimation of network size.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Analysis of Regularized LS Reconstruction and Random Matrix Ensembles in Compressed Sensing

Performance of regularized least-squares estimation in noisy compressed sensing is analyzed in the limit when the dimensions of the measurement matrix grow large. The sensing matrix is considered to be from a class of random ensembles that encloses as special cases standard Gaussian, row-orthogonal, geometric and so-called T-orthogonal constructions. Source vectors that have non-uniform sparsity are included in the system model. Regularization based on l1-norm and leading to LASSO estimation, or basis pursuit denoising, is given the main emphasis in the analysis. Extensions to l2-norm and "zero-norm" regularization are also briefly discussed. The analysis is carried out using the replica method in conjunction with some novel matrix integration results. Numerical experiments for LASSO are provided to verify the accuracy of the analytical results. The numerical experiments show that for noisy compressed sensing, the standard Gaussian ensemble is a suboptimal choice for the measurement matrix. Orthogonal constructions provide a superior performance in all considered scenarios and are easier to implement in practical applications. It is also discovered that for non-uniform sparsity patterns the T-orthogonal matrices can further improve the mean square error behavior of the reconstruction when the noise level is not too high. However, as the additive noise becomes more prominent in the system, the simple row-orthogonal measurement matrix appears to be the best choice out of the considered ensembles.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Categorified presheaves and sieves

Let $\mathcal C$ be a category of a set of (small) categories. This paper concerns with the ${\mathbf {Cat}}$-valued presheaves and sieves over category $\mathcal C.$ Since ${\mathbf {Cat}}$ is not a concrete category, existing definition of presheaves can not deal with the situation. This paper proposes a new framework for the purpose. The main result is a version of Yoneda embedding for ${\mathbf {Cat}}$-valued presheaves, and construction of the ${\mathbf {Cat}}$-valued sieves over the category ${\mathcal O}(\mathbf B)$ of subcategories of a given topological category $\mathbf B.$

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

On $\mathbf {Cat}$-valued sheaves

Let ${\widetilde {\mathcal O}}(\mathbf B)$ be the category of (open) subcategories of a topological groupoid ${\mathbf B}.$ This paper concerns with the ${\mathbf {Cat}}$-valued sheaves over category ${\widetilde {\mathcal O}}(\mathbf B).$ Since ${\mathbf {Cat}}$ is not a concrete category, traditional definition of presheaf can not deal with the situation. [13] proposes a new framework for the purpose. Starting from the definition given in [13], we build-up the frame work for ${\mathbf {Cat}}$-valued sheaves. For that purpose we introduce a notion of categorical union, such that categorical union of subcategories is a subcategory, which is required for a meaningful definition of a categorical cover of a topological category. The main result is the following. For a fixed category $\mathbf C,$ the categories of local functorial sections from $\mathbf B$ to $\mathbf C$ define a ${\mathbf {Cat}}$-valued sheaf on ${\widetilde {\mathcal O}}(\mathbf B).$ Replacing $\mathbf C$ with a categorical group $\mathcal G,$ we find a ${\mathbf {CatGrp}}$-valued sheaf on ${\widetilde {\mathcal O}}(\mathbf B).$

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Bayesian Learning for Low-Rank matrix reconstruction

We develop latent variable models for Bayesian learning based low-rank matrix completion and reconstruction from linear measurements. For under-determined systems, the developed methods are shown to reconstruct low-rank matrices when neither the rank nor the noise power is known a-priori. We derive relations between the latent variable models and several low-rank promoting penalty functions. The relations justify the use of Kronecker structured covariance matrices in a Gaussian based prior. In the methods, we use evidence approximation and expectation-maximization to learn the model parameters. The performance of the methods is evaluated through extensive numerical simulations.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Combined modeling of sparse and dense noise for improvement of Relevance Vector Machine

Using a Bayesian approach, we consider the problem of recovering sparse signals under additive sparse and dense noise. Typically, sparse noise models outliers, impulse bursts or data loss. To handle sparse noise, existing methods simultaneously estimate the sparse signal of interest and the sparse noise of no interest. For estimating the sparse signal, without the need of estimating the sparse noise, we construct a robust Relevance Vector Machine (RVM). In the RVM, sparse noise and ever present dense noise are treated through a combined noise model. The precision of combined noise is modeled by a diagonal matrix. We show that the new RVM update equations correspond to a non-symmetric sparsity inducing cost function. Further, the combined modeling is found to be computationally more efficient. We also extend the method to block-sparse signals and noise with known and unknown block structures. Through simulations, we show the performance and computation efficiency of the new RVM in several applications: recovery of sparse and block sparse signals, housing price prediction and image denoising.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Design and Analysis of a Greedy Pursuit for Distributed Compressed Sensing

We consider a distributed compressed sensing scenario where many sensors measure correlated sparse signals and the sensors are connected through a network. Correlation between sparse signals is modeled by a partial common support-set. For such a scenario, the main objective of this paper is to develop a greedy pursuit algorithm. We develop a distributed parallel pursuit (DIPP) algorithm based on exchange of information about estimated support-sets at sensors. The exchange of information helps to improve estimation of the partial common support-set, that in turn helps to gradually improve estimation of support-sets in all sensors, leading to a better quality reconstruction performance. We provide restricted isometry property (RIP) based theoretical analysis on the algorithm's convergence and reconstruction performance. Under certain theoretical requirements on the quality of information exchange over network and RIP parameters of sensor nodes, we show that the DIPP algorithm converges to a performance level that depends on a scaled additive measurement noise power (convergence in theory) where the scaling coefficient is a function of RIP parameters and information processing quality parameters. Using simulations, we show practical reconstruction performance of DIPP vis-a-vis amount of undersampling, signal-to-measurement-noise ratios and network-connectivity conditions.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Distributed Quantization for Measurement of Correlated Sparse Sources over Noisy Channels

In this paper, we design and analyze distributed vector quantization (VQ) for compressed measurements of correlated sparse sources over noisy channels. Inspired by the framework of compressed sensing (CS) for acquiring compressed measurements of the sparse sources, we develop optimized quantization schemes that enable distributed encoding and transmission of CS measurements over noisy channels followed by joint decoding at a decoder. The optimality is addressed with respect to minimizing the sum of mean-square error (MSE) distortions between the sparse sources and their reconstruction vectors at the decoder. We propose a VQ encoder-decoder design via an iterative algorithm, and derive a lower-bound on the end-to-end MSE of the studied distributed system. Through several simulation studies, we evaluate the performance of the proposed distributed scheme.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Geodesics on path spaces and double category

Let $M$ be a Riemannian manifold and ${\mathcal P}M$ be the space of all smooth paths on $M$. We describe geodesics on path space ${\mathcal P}M$. Normal neighbourhood structure on ${\mathcal P}M$ has been discussed. We identify paths on $M$ under "back-track" equivalence. Under this identification we show that if $M$ is complete, then geodesics on path space yield a double category.We gave a physical interpretation of this double category.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Twisted-Product Categorical Bundles

Categorical bundles provide a natural framework for gauge theories involving multiple gauge groups. Unlike the case of traditional bundles there are distinct notions of triviality, and hence also of local triviality, for categorical bundles. We study categorical principal bundles that are product bundles in the categorical sense, developing the relationship between functorial sections of such bundles and trivializations. We construct functorial cocycles with values in categorical groups using a suitable family of locally defined functions on the object space of the base category. Categorical product bundles being too rigid to give a widely applicable model for local triviality, we introduce the notion of a twisted-product categorical bundle. We relate such bundles to decorated categorical bundles that contain more information, specifically parallel transport data.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Alternating Strategies Are Good For Low-Rank Matrix Reconstruction

This article focuses on the problem of reconstructing low-rank matrices from underdetermined measurements using alternating optimization strategies. We endeavour to combine an alternating least-squares based estimation strategy with ideas from the alternating direction method of multipliers (ADMM) to recover structured low-rank matrices, such as Hankel structure. We show that merging these two alternating strategies leads to a better performance than the existing alternating least squares (ALS) strategy. The performance is evaluated via numerical simulations.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Analysis of Democratic Voting Principles used in Distributed Greedy Algorithms

A key aspect for any greedy pursuit algorithm used in compressed sensing is a good support-set detection method. For distributed compressed sensing, we consider a setup where many sensors measure sparse signals that are correlated via the existence of a signals' intersection support-set. This intersection support-set is called the joint support-set. Estimation of the joint support-set has a high impact on the performance of a distributed greedy pursuit algorithm. This estimation can be achieved by exchanging local support-set estimates followed by a (consensus) voting method. In this paper we endeavor for a probabilistic analysis of two democratic voting principle that we call majority and consensus voting. In our analysis, we first model the input/output relation of a greedy algorithm (executed locally in a sensor) by a single parameter known as probability of miss. Based on this model, we analyze the voting principles and prove that the democratic voting principle has a merit to detect the joint support-set.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Analysis-by-Synthesis Quantization for Compressed Sensing Measurements

We consider a resource-limited scenario where a sensor that uses compressed sensing (CS) collects a low number of measurements in order to observe a sparse signal, and the measurements are subsequently quantized at a low bit-rate followed by transmission or storage. For such a scenario, we design new algorithms for source coding with the objective of achieving good reconstruction performance of the sparse signal. Our approach is based on an analysis-by-synthesis principle at the encoder, consisting of two main steps: (1) the synthesis step uses a sparse signal reconstruction technique for measuring the direct effect of quantization of CS measurements on the final sparse signal reconstruction quality, and (2) the analysis step decides appropriate quantized values to maximize the final sparse signal reconstruction quality. Through simulations, we compare the performance of the proposed quantization algorithms vis-a-vis existing quantization schemes.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Analysis-by-Synthesis-based Quantization of Compressed Sensing Measurements

We consider a resource-constrained scenario where a compressed sensing- (CS) based sensor has a low number of measurements which are quantized at a low rate followed by transmission or storage. Applying this scenario, we develop a new quantizer design which aims to attain a high-quality reconstruction performance of a sparse source signal based on analysis-by-synthesis framework. Through simulations, we compare the performance of the proposed quantization algorithm vis-a-vis existing quantization methods.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Channel-Optimized Vector Quantizer Design for Compressed Sensing Measurements

We consider vector-quantized (VQ) transmission of compressed sensing (CS) measurements over noisy channels. Adopting mean-square error (MSE) criterion to measure the distortion between a sparse vector and its reconstruction, we derive channel-optimized quantization principles for encoding CS measurement vector and reconstructing sparse source vector. The resulting necessary optimal conditions are used to develop an algorithm for training channel-optimized vector quantization (COVQ) of CS measurements by taking the end-to-end distortion measure into account.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Distributed Quantization for Compressed Sensing

We study distributed coding of compressed sensing (CS) measurements using vector quantizer (VQ). We develop a distributed framework for realizing optimized quantizer that enables encoding CS measurements of correlated sparse sources followed by joint decoding at a fusion center. The optimality of VQ encoder-decoder pairs is addressed by minimizing the sum of mean-square errors between the sparse sources and their reconstruction vectors at the fusion center. We derive a lower-bound on the end-to-end performance of the studied distributed system, and propose a practical encoder-decoder design through an iterative algorithm.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Estimation for the Linear Model with Uncertain Covariance Matrices

We derive a maximum a posteriori estimator for the linear observation model, where the signal and noise covariance matrices are both uncertain. The uncertainties are treated probabilistically by modeling the covariance matrices with prior inverse-Wishart distributions. The nonconvex problem of jointly estimating the signal of interest and the covariance matrices is tackled by a computationally efficient fixed-point iteration as well as an approximate variational Bayes solution. The statistical performance of estimators is compared numerically to state-of-the-art estimators from the literature and shown to perform favorably.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Joint Source-Channel Vector Quantization for Compressed Sensing

We study joint source-channel coding (JSCC) of compressed sensing (CS) measurements using vector quantizer (VQ). We develop a framework for realizing optimum JSCC schemes that enable encoding and transmitting CS measurements of a sparse source over discrete memoryless channels, and decoding the sparse source signal. For this purpose, the optimal design of encoder-decoder pair of a VQ is considered, where the optimality is addressed by minimizing end-to-end mean square error (MSE). We derive a theoretical lower-bound on the MSE performance, and propose a practical encoder-decoder design through an iterative algorithm. The resulting coding scheme is referred to as channel- optimized VQ for CS, coined COVQ-CS. In order to address the encoding complexity issue of the COVQ-CS, we propose to use a structured quantizer, namely low complexity multi-stage VQ (MSVQ). We derive new encoding and decoding conditions for the MSVQ, and then propose a practical encoder-decoder design algorithm referred to as channel-optimized MSVQ for CS, coined COMSVQ-CS. Through simulation studies, we compare the proposed schemes vis-a-vis relevant quantizers.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Performance Bounds for Vector Quantized Compressive Sensing

In this paper, we endeavor for predicting the performance of quantized compressive sensing under the use of sparse reconstruction estimators. We assume that a high rate vector quantizer is used to encode the noisy compressive sensing measurement vector. Exploiting a block sparse source model, we use Gaussian mixture density for modeling the distribution of the source. This allows us to formulate an optimal rate allocation problem for the vector quantizer. Considering noisy CS quantized measurements, we analyze upper- and lower-bounds on reconstruction error performance guarantee of two estimators - convex relaxation based basis pursuit de-noising estimator and an oracle-assisted least-squares estimator.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Piecewise Toeplitz Matrices-based Sensing for Rank Minimization

This paper proposes a set of piecewise Toeplitz matrices as the linear mapping/sensing operator $\mathcal{A}: \mathbf{R}^{n_1 \times n_2} \rightarrow \mathbf{R}^M$ for recovering low rank matrices from few measurements. We prove that such operators efficiently encode the information so there exists a unique reconstruction matrix under mild assumptions. This work provides a significant extension of the compressed sensing and rank minimization theory, and it achieves a tradeoff between reducing the memory required for storing the sampling operator from $\mathcal{O}(n_1n_2M)$ to $\mathcal{O}(\max(n_1,n_2)M)$ but at the expense of increasing the number of measurements by $r$. Simulation results show that the proposed operator can recover low rank matrices efficiently with a reconstruction performance close to the cases of using random unstructured operators.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Relevance Singular Vector Machine for low-rank matrix sensing

In this paper we develop a new Bayesian inference method for low rank matrix reconstruction. We call the new method the Relevance Singular Vector Machine (RSVM) where appropriate priors are defined on the singular vectors of the underlying matrix to promote low rank. To accelerate computations, a numerically efficient approximation is developed. The proposed algorithms are applied to matrix completion and matrix reconstruction problems and their performance is studied numerically.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

SEK: Sparsity exploiting $k$-mer-based estimation of bacterial community composition

Motivation: Estimation of bacterial community composition from a high-throughput sequenced sample is an important task in metagenomics applications. Since the sample sequence data typically harbors reads of variable lengths and different levels of biological and technical noise, accurate statistical analysis of such data is challenging. Currently popular estimation methods are typically very time consuming in a desktop computing environment. Results: Using sparsity enforcing methods from the general sparse signal processing field (such as compressed sensing), we derive a solution to the community composition estimation problem by a simultaneous assignment of all sample reads to a pre-processed reference database. A general statistical model based on kernel density estimation techniques is introduced for the assignment task and the model solution is obtained using convex optimization tools. Further, we design a greedy algorithm solution for a fast solution. Our approach offers a reasonably fast community composition estimation method which is shown to be more robust to input data variation than a recently introduced related method. Availability: A platform-independent Matlab implementation of the method is freely available at http://www.ee.kth.se/ctsoftware; source code that does not require access to Matlab is currently being tested and will be made available later through the above website.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Twisted actions of categorical groups

We develop a theory of twisted actions of categorical groups using a notion of semidirect product of categories. We work through numerous examples to demonstrate the power of these notions. Turning to representations, which are actions that respect vector space structures, we establish an analog of Schur's lemma in this context. Keeping new terminology to a minumum, we concentrate on examples exploring the essential new notions introduced.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

Distributed Greedy Pursuit Algorithms

For compressed sensing over arbitrarily connected networks, we consider the problem of estimating underlying sparse signals in a distributed manner. We introduce a new signal model that helps to describe inter-signal correlation among connected nodes. Based on this signal model along with a brief survey of existing greedy algorithms, we develop distributed greedy algorithms with low communication overhead. Incorporating appropriate modifications, we design two new distributed algorithms where the local algorithms are based on appropriately modified existing orthogonal matching pursuit and subspace pursuit. Further, by combining advantages of these two local algorithms, we design a new greedy algorithm that is well suited for a distributed scenario. By extensive simulations we demonstrate that the new algorithms in a sparsely connected network provide good performance, close to the performance of a centralized greedy solution.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

Line Spectrum Estimation with Probabilistic Priors

For line spectrum estimation, we derive the maximum a posteriori probability estimator where prior knowledge of frequencies is modeled probabilistically. Since the spectrum is periodic, an appropriate distribution is the circular von Mises distribution that can parameterize the entire range of prior certainty of the frequencies. An efficient alternating projections method is used to solve the resulting optimization problem. The estimator is evaluated numerically and compared with other estimators and the Cramér-Rao bound.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

Statistical Mechanics Approach to Sparse Noise Denoising

Reconstruction fidelity of sparse signals contaminated by sparse noise is considered. Statistical mechanics inspired tools are used to show that the l1-norm based convex optimization algorithm exhibits a phase transition between the possibility of perfect and imperfect reconstruction. Conditions characterizing this threshold are derived and the mean square error of the estimate is obtained for the case when perfect reconstruction is not possible. Detailed calculations are provided to expose the mathematical tools to a wide audience.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Analysis of MMSE Estimation for Compressive Sensing of Block Sparse Signals

Minimum mean square error (MMSE) estimation of block sparse signals from noisy linear measurements is considered. Unlike in the standard compressive sensing setup where the non-zero entries of the signal are independently and uniformly distributed across the vector of interest, the information bearing components appear here in large mutually dependent clusters. Using the replica method from statistical physics, we derive a simple closed-form solution for the MMSE obtained by the optimum estimator. We show that the MMSE is a version of the Tse-Hanly formula with system load and MSE scaled by parameters that depend on the sparsity pattern of the source. It turns out that this is equal to the MSE obtained by a genie-aided MMSE estimator which is informed in advance about the exact locations of the non-zero blocks. The asymptotic results obtained by the non-rigorous replica method are found to have an excellent agreement with finite sized numerical simulations.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Analysis of Sparse Representations Using Bi-Orthogonal Dictionaries

The sparse representation problem of recovering an N dimensional sparse vector x from M < N linear observations y = Dx given dictionary D is considered. The standard approach is to let the elements of the dictionary be independent and identically distributed (IID) zero-mean Gaussian and minimize the l1-norm of x under the constraint y = Dx. In this paper, the performance of l1-reconstruction is analyzed, when the dictionary is bi-orthogonal D = [O1 O2], where O1,O2 are independent and drawn uniformly according to the Haar measure on the group of orthogonal M x M matrices. By an application of the replica method, we obtain the critical conditions under which perfect l1-recovery is possible with bi-orthogonal dictionaries.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Dynamic Iterative Pursuit

For compressive sensing of dynamic sparse signals, we develop an iterative pursuit algorithm. A dynamic sparse signal process is characterized by varying sparsity patterns over time/space. For such signals, the developed algorithm is able to incorporate sequential predictions, thereby providing better compressive sensing recovery performance, but not at the cost of high complexity. Through experimental evaluations, we observe that the new algorithm exhibits a graceful degradation at deteriorating signal conditions while capable of yielding substantial performance gains as conditions improve.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Fusion of Greedy Pursuits for Compressed Sensing Signal Reconstruction

Greedy Pursuits are very popular in Compressed Sensing for sparse signal recovery. Though many of the Greedy Pursuits possess elegant theoretical guarantees for performance, it is well known that their performance depends on the statistical distribution of the non-zero elements in the sparse signal. In practice, the distribution of the sparse signal may not be known a priori. It is also observed that performance of Greedy Pursuits degrades as the number of available measurements decreases from a threshold value which is method dependent. To improve the performance in these situations, we introduce a novel fusion framework for Greedy Pursuits and also propose two algorithms for sparse recovery. Through Monte Carlo simulations we show that the proposed schemes improve sparse signal recovery in clean as well as noisy measurement cases.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

The Linear Model under Mixed Gaussian Inputs: Designing the Transfer Matrix

Suppose a linear model y = Hx + n, where inputs x, n are independent Gaussian mixtures. The problem is to design the transfer matrix H so as to minimize the mean square error (MSE) when estimating x from y. This problem has important applications, but faces at least three hurdles. Firstly, even for a fixed H, the minimum MSE (MMSE) has no analytical form. Secondly, the MMSE is generally not convex in H. Thirdly, derivatives of the MMSE w.r.t. H are hard to obtain. This paper casts the problem as a stochastic program and invokes gradient methods. The study is motivated by two applications in signal processing. One concerns the choice of error-reducing precoders; the other deals with selection of pilot matrices for channel estimation. In either setting, our numerical results indicate improved estimation accuracy - markedly better than those obtained by optimal design based on standard linear estimators. Some implications of the non-convexities of the MMSE are noteworthy, yet, to our knowledge, not well known. For example, there are cases in which more pilot power is detrimental for channel estimation. This paper explains why.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Typical $l_1$-recovery limit of sparse vectors represented by concatenations of random orthogonal matrices

We consider the problem of recovering an $N$-dimensional sparse vector $\vm{x}$ from its linear transformation $\vm{y}=\vm{D} \vm{x}$ of $M(< N)$ dimension. Minimizing the $l_{1}$-norm of $\vm{x}$ under the constraint $\vm{y} = \vm{D} \vm{x}$ is a standard approach for the recovery problem, and earlier studies report that the critical condition for typically successful $l_1$-recovery is universal over a variety of randomly constructed matrices $\vm{D}$. For examining the extent of the universality, we focus on the case in which $\vm{D}$ is provided by concatenating $\nb=N/M$ matrices $\vm{O}_{1}, \vm{O}_{2},..., \vm{O}_\nb$ drawn uniformly according to the Haar measure on the $M \times M$ orthogonal matrices. By using the replica method in conjunction with the development of an integral formula for handling the random orthogonal matrices, we show that the concatenated matrices can result in better recovery performance than what the universality predicts when the density of non-zero signals is not uniform among the $\nb$ matrix modules. The universal condition is reproduced for the special case of uniform non-zero signal densities. Extensive numerical experiments support the theoretical predictions.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Minimum Mean Square Error Estimation Under Gaussian Mixture Statistics

This paper investigates the minimum mean square error (MMSE) estimation of x, given the observation y = Hx+n, when x and n are independent and Gaussian Mixture (GM) distributed. The introduction of GM distributions, represents a generalization of the more familiar and simpler Gaussian signal and Gaussian noise instance. We present the necessary theoretical foundation and derive the MMSE estimator for x in a closed form. Furthermore, we provide upper and lower bounds for its mean square error (MSE). These bounds are validated through Monte Carlo simulations.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Projection-Based and Look Ahead Strategies for Atom Selection

In this paper, we improve iterative greedy search algorithms in which atoms are selected serially over iterations, i.e., one-by-one over iterations. For serial atom selection, we devise two new schemes to select an atom from a set of potential atoms in each iteration. The two new schemes lead to two new algorithms. For both the algorithms, in each iteration, the set of potential atoms is found using a standard matched filter. In case of the first scheme, we propose an orthogonal projection strategy that selects an atom from the set of potential atoms. Then, for the second scheme, we propose a look ahead strategy such that the selection of an atom in the current iteration has an effect on the future iterations. The use of look ahead strategy requires a higher computational resource. To achieve a trade-off between performance and complexity, we use the two new schemes in cascade and develop a third new algorithm. Through experimental evaluations, we compare the proposed algorithms with existing greedy search and convex relaxation algorithms.

0 citations0 reviewsNo code linkedOpen access