BZPEERPapers, people, evidence and research paths
Menu

Discover

SearchFind papers, people, topics, institutions and opportunities from one place.GraphSee how papers, authors, topics and evidence connect.

Research tools

HomeContinue from the papers, people and topics that matter now.CollectionsTurn saved papers into reading lists, evidence maps and shareable context.ResearchReconstruct the path that led you from search to paper to question.CollaborationMove from a paper, topic or expert request into a focused thread.PublishCreate full-text research articles with metadata, formulas and versions.

Network

ExpertsFind specialists whose work and reviews explain why they can help.CommunitiesJoin research groups organized around papers, topics and shared evidence.

Opportunities

ListingsBrowse roles, calls and collaborations connected to real research context.

Account

Log inReturn to your saved papers, graph views and active threads.Create accountStart saving papers, building a research trail and joining teams.
Log inCreate account

Source author record

Yoshiyuki Kabashima

Yoshiyuki Kabashima appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record
cond-mat.dis-nncond-mat.stat-mechInformation Theorymath.ITMachine LearningNeurons and Cognitionphysics.soc-phBiological Physicsmath-phmath.MPmath.OCNumerical AnalysisSocial and Information NetworksComputational Complexityeess.SPmath.NAnlin.AOphysics.data-anPopulations and Evolution

Catalog footprint

What is connected

56works
19topics
4close collaborators

Actions

Connect this record

Log in to claim
Open graphBrowse works

Research graph

See the researcher in context

Open full explorer

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

56 published item(s)

preprint2024arXiv

QCM-SGM+: Improved Quantized Compressed Sensing With Score-Based Generative Models

In practical compressed sensing (CS), the obtained measurements typically necessitate quantization to a limited number of bits prior to transmission or storage. This nonlinear quantization process poses significant recovery challenges, particularly with extreme coarse quantization such as 1-bit. Recently, an efficient algorithm called QCS-SGM was proposed for quantized CS (QCS) which utilizes score-based generative models (SGM) as an implicit prior. Due to the adeptness of SGM in capturing the intricate structures of natural signals, QCS-SGM substantially outperforms previous QCS methods. However, QCS-SGM is constrained to (approximately) row-orthogonal sensing matrices as the computation of the likelihood score becomes intractable otherwise. To address this limitation, we introduce an advanced variant of QCS-SGM, termed QCS-SGM+, capable of handling general matrices effectively. The key idea is a Bayesian inference perspective on the likelihood score computation, wherein expectation propagation is employed for its approximate computation. Extensive experiments are conducted, demonstrating the substantial superiority of QCS-SGM+ over QCS-SGM for general sensing matrices beyond mere row-orthogonality.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Assessing transfer entropy from biochemical data

We address the problem of evaluating the transfer entropy (TE) produced by biochemical reactions from experimentally measured data. Although these reactions are generally non-linear and non-stationary processes making it challenging to achieve accurate modeling, Gaussian approximation can facilitate the TE assessment only by estimating covariance matrices using multiple data obtained from simultaneously measured time series representing the activation levels of biomolecules such as proteins. Nevertheless, the non-stationary nature of biochemical signals makes it difficult to theoretically assess the sampling distributions of TE, which are necessary for evaluating the statistical confidence and significance of the data-driven estimates. We resolve this difficulty by computationally assessing the sampling distributions using techniques from computational statistics. The computational methods are tested by using them in analyzing data generated from a theoretically tractable time-varying signal model, which leads to the development of a method to screen only statistically significant estimates. The usefulness of the developed method is examined by applying it to real biological data experimentally measured from the ERBB-RAS-MAPK system that superintends diverse cell fate decisions. A comparison between cells containing wild-type and mutant proteins exhibits a distinct difference in the time evolution of TE while apparent difference is hardly found in average profiles of the raw signals. Such comparison may help in unveiling important pathways of biochemical reactions.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Statistical mechanics analysis of general multi-dimensional knapsack problems

Knapsack problem (KP) is a representative combinatorial optimization problem that aims to maximize the total profit by selecting a subset of items under given constraints on the total weights. In this study, we analyze a generalized version of KP, which is termed the generalized multidimensional knapsack problem (GMDKP). As opposed to the basic KP, GMDKP allows multiple choices per item type under multiple weight constraints. Although several efficient algorithms are known and the properties of their solutions have been examined to a significant extent for basic KPs, there is a paucity of known algorithms and studies on the solution properties of GMDKP. To gain insight into the problem, we assess the typical achievable limit of the total profit for a random ensemble of GMDKP using the replica method. Our findings are summarized as follows: (1) When the profits of item types are normally distributed, the total profit grows in the leading order with respect to the number of item types as the maximum number of choices per item type $x^{\rm max}$ increases while it depends on $x^{\rm max}$ only in a sub-leading order if the profits are constant among the item types. (2) A greedy-type heuristic can find a nearly optimal solution whose total profit is lower than the optimal value only by a sub-leading order with a low computational cost. (3) The sub-leading difference from the optimal total profit can be improved by a heuristic algorithm based on the cavity method. Extensive numerical experiments support these findings.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

Reconstructing Sparse Signals via Greedy Monte-Carlo Search

We propose a Monte-Carlo-based method for reconstructing sparse signals in the formulation of sparse linear regression in a high-dimensional setting. The basic idea of this algorithm is to explicitly select variables or covariates to represent a given data vector or responses and accept randomly generated updates of that selection if and only if the energy or cost function decreases. This algorithm is called the greedy Monte-Carlo (GMC) search algorithm. Its performance is examined via numerical experiments, which suggests that in the noiseless case, GMC can achieve perfect reconstruction in undersampling situations of a reasonable level: it can outperform the $\ell_1$ relaxation but does not reach the algorithmic limit of MC-based methods theoretically clarified by an earlier analysis. The necessary computational time is also examined and compared with that of an algorithm using simulated annealing. Additionally, experiments on the noisy case are conducted on synthetic datasets and on a real-world dataset, supporting the practicality of GMC.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Inferring neuronal couplings from spiking data using a systematic procedure with a statistical criterion

Recent remarkable advances in the experimental techniques have provided a background for inferring neuronal couplings from point process data that includes a great number of neurons. Here, we propose a systematic procedure for pre- and post-processing generic point process data in an objective manner, to handle data in the framework of a binary simple statistical model, the Ising or generalized McCulloch--Pitts model. The procedure involves two steps: (1) determining time-bin size for transforming the point-process data into discrete-time binary data and (2) screening relevant couplings from the estimated couplings. For the first step, we decide the optimal time-bin size by introducing the null hypothesis that all neurons would fire independently, then choosing a time-bin size so that the null hypothesis is rejected with the most strict criterion. The likelihood associated with the null hypothesis is analytically evaluated and used for the rejection process. For the second post-processing step, after a certain estimator of coupling is obtained based on the pre-processed dataset, the estimate is compared with many other estimates derived from datasets obtained by randomizing the original dataset in the time direction. We accept the original estimate as relevant only if its absolute value is sufficiently larger than them of randomized datasets. These manipulations suppress false positive couplings induced by statistical noise. We apply this inference procedure to spiking data from synthetic and in vitro neuronal networks. The results show that the proposed procedure identifies the presence/absence of synaptic couplings fairly well including their signs, for the synthetic and experimental data. In particular, the results support that we can infer the physical connections of underlying systems in favorable situations, even when using the simple statistical model.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Learning performance in inverse Ising problems with sparse teacher couplings

We investigate the learning performance of the pseudolikelihood maximization method for inverse Ising problems. In the teacher-student scenario under the assumption that the teacher's couplings are sparse and the student does not know the graphical structure, the learning curve and order parameters are assessed in the typical case using the replica and cavity methods from statistical mechanics. Our formulation is also applicable to a certain class of cost functions having locality; the standard likelihood does not belong to that class. The derived analytical formulas indicate that the perfect inference of the presence/absence of the teacher's couplings is possible in the thermodynamic limit taking the number of spins $N$ as infinity while keeping the dataset size $M$ proportional to $N$, as long as $α=M/N > 2$. Meanwhile, the formulas also show that the estimated coupling values corresponding to the truly existing ones in the teacher tend to be overestimated in the absolute value, manifesting the presence of estimation bias. These results are considered to be exact in the thermodynamic limit on locally tree-like networks, such as the regular random or Erdős--Rényi graphs. Numerical simulation results fully support the theoretical predictions. Additional biases in the estimators on loopy graphs are also discussed.

0 citations0 reviewsNo code linkedOpen access
preprint2019arXiv

Approximate matrix completion based on cavity method

In order to solve large matrix completion problems with practical computational cost, an approximate approach based on matrix factorization has been widely used. Alternating least squares (ALS) and stochastic gradient descent (SGD) are two major algorithms to this end. In this study, we propose a new algorithm, namely cavity-based matrix factorization (CBMF) and approximate cavity-based matrix factorization (ACBMF), which are developed based on the cavity method from statistical mechanics. ALS yields solutions with less iterations when compared to those of SGD. This is because its update rules are described in a closed form although it entails higher computational cost. CBMF can also write its update rules in a closed form, and its computational cost is lower than that of ALS. ACBMF is proposed to compensate a disadvantage of CBMF in terms of relatively high memory cost. We experimentally illustrate that the proposed methods outperform the two existing algorithms in terms of convergence speed per iteration, and it can work under the condition where observed entries are relatively fewer. Additionally, in contrast to SGD, (A)CBMF does not require scheduling of the learning rate.

0 citations0 reviewsNo code linkedOpen access
preprint2018arXiv

Objective and efficient inference for couplings in neuronal networks

Inferring directional couplings from the spike data of networks is desired in various scientific fields such as neuroscience. Here, we apply a recently proposed objective procedure to the spike data obtained from the Hodgkin--Huxley type models and in vitro neuronal networks cultured in a circular structure. As a result, we succeed in reconstructing synaptic connections accurately from the evoked activity as well as the spontaneous one. To obtain the results, we invent an analytic formula approximately implementing a method of screening relevant couplings. This significantly reduces the computational cost of the screening method employed in the proposed objective procedure, making it possible to treat large-size systems as in this study.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Analysis of Regularized LS Reconstruction and Random Matrix Ensembles in Compressed Sensing

Performance of regularized least-squares estimation in noisy compressed sensing is analyzed in the limit when the dimensions of the measurement matrix grow large. The sensing matrix is considered to be from a class of random ensembles that encloses as special cases standard Gaussian, row-orthogonal, geometric and so-called T-orthogonal constructions. Source vectors that have non-uniform sparsity are included in the system model. Regularization based on l1-norm and leading to LASSO estimation, or basis pursuit denoising, is given the main emphasis in the analysis. Extensions to l2-norm and "zero-norm" regularization are also briefly discussed. The analysis is carried out using the replica method in conjunction with some novel matrix integration results. Numerical experiments for LASSO are provided to verify the accuracy of the analytical results. The numerical experiments show that for noisy compressed sensing, the standard Gaussian ensemble is a suboptimal choice for the measurement matrix. Orthogonal constructions provide a superior performance in all considered scenarios and are easier to implement in practical applications. It is also discovered that for non-uniform sparsity patterns the T-orthogonal matrices can further improve the mean square error behavior of the reconstruction when the noise level is not too high. However, as the additive noise becomes more prominent in the system, the simple row-orthogonal measurement matrix appears to be the best choice out of the considered ensembles.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Approximate cross-validation formula for Bayesian linear regression

Cross-validation (CV) is a technique for evaluating the ability of statistical models/learning systems based on a given data set. Despite its wide applicability, the rather heavy computational cost can prevent its use as the system size grows. To resolve this difficulty in the case of Bayesian linear regression, we develop a formula for evaluating the leave-one-out CV error approximately without actually performing CV. The usefulness of the developed formula is tested by statistical mechanical analysis for a synthetic model. This is confirmed by application to a real-world supernova data set as well.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Cross validation in LASSO and its acceleration

We investigate leave-one-out cross validation (CV) as a determinator of the weight of the penalty term in the least absolute shrinkage and selection operator (LASSO). First, on the basis of the message passing algorithm and a perturbative discussion assuming that the number of observations is sufficiently large, we provide simple formulas for approximately assessing two types of CV errors, which enable us to significantly reduce the necessary cost of computation. These formulas also provide a simple connection of the CV errors to the residual sums of squares between the reconstructed and the given measurements. Second, on the basis of this finding, we analytically evaluate the CV errors when the design matrix is given as a simple random matrix in the large size limit by using the replica method. Finally, these results are compared with those of numerical simulations on finite-size systems and are confirmed to be correct. We also apply the simple formulas of the first type of CV error to an actual dataset of the supernovae.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Multiple peaks of species abundance distributions induced by sparse interactions

We investigate the replicator dynamics with "sparse" symmetric interactions which represent specialist-specialist interactions in ecological communities. By considering a large self interaction $u$, we conduct a perturbative expansion which manifests that the nature of the interactions has a direct impact on the species abundance distribution. The central results are all species coexistence in a realistic range of the model parameters and that a certain discrete nature of the interactions induces multiple peaks in the species abundance distribution, providing the possibility of theoretically explaining multiple peaks observed in various field studies. To get more quantitative information, we also construct a non-perturbative theory which becomes exact on tree-like networks if all the species coexist, providing exact critical values of $u$ below which extinct species emerge. Numerical simulations in various different situations are conducted and they clarify the robustness of the presented mechanism of all species coexistence and multiple peaks in the species abundance distributions.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Phase transitions and sample complexity in Bayes-optimal matrix factorization

We analyse the matrix factorization problem. Given a noisy measurement of a product of two matrices, the problem is to estimate back the original matrices. It arises in many applications such as dictionary learning, blind matrix calibration, sparse principal component analysis, blind source separation, low rank matrix completion, robust principal component analysis or factor analysis. It is also important in machine learning: unsupervised representation learning can often be studied through matrix factorization. We use the tools of statistical mechanics - the cavity and replica methods - to analyze the achievability and computational tractability of the inference problems in the setting of Bayes-optimal inference, which amounts to assuming that the two matrices have random independent elements generated from some known distribution, and this information is available to the inference algorithm. In this setting, we compute the minimal mean-squared-error achievable in principle in any computational time, and the error that can be achieved by an efficient approximate message passing algorithm. The computation is based on the asymptotic state-evolution analysis of the algorithm. The performance that our analysis predicts, both in terms of the achieved mean-squared-error, and in terms of sample complexity, is extremely promising and motivating for a further development of the algorithm.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Relative species abundance of replicator dynamics with sparse interactions

A theory of relative species abundance on sparsely-connected networks is presented by investigating the replicator dynamics with symmetric interactions. Sparseness of a network involves difficulty in analyzing the fixed points of the equation, and we avoid this problem by treating large self interaction $u$, which allows us to construct a perturbative expansion. Based on this perturbation, we find that the nature of the interactions is directly connected to the abundance distribution, and some characteristic behaviors, such as multiple peaks in the abundance distribution and all species coexistence at moderate values of $u$, are discovered in a wide class of the distribution of the interactions. The all species coexistence collapses at a critical value of $u$, $u_c$, and this collapsing is regarded as a phase transition. To get more quantitative information, we also construct a non-perturbative theory on random graphs based on techniques of statistical mechanics. The result shows those characteristic behaviors are sustained well even for not large $u$. For even smaller values of $u$, extinct species start to appear and the abundance distribution becomes rounded and closer to a standard functional form. Another interesting finding is the non-monotonic behavior of diversity, which quantifies the number of coexisting species, when changing the ratio of mutualistic relations $Δ$. These results are examined by numerical simulations, and the multiple peaks in the abundance distribution are confirmed to be robust against a certain level of modifications of the problem. The numerical results also show that our theory is exact for the case without extinct species, but becomes less and less precise as the proportion of extinct species grows.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Resilience of antagonistic networks with regard to the effects of initial failures and degree-degree correlations

In this study, we investigate the resilience of duplex networked layers ($α$ and $β$) coupled with antagonistic interlinks, each layer of which inhibits its counterpart at the microscopic level, changing the following factors: whether the influence of the initial failures in $α$ remains (quenched (Case Q)) or not (free (Case F)); the effect of intralayer degree-degree correlations in each layer and interlayer degree-degree correlations; and the type of the initial failures, such as random failures (RFs) or targeted attacks (TAs). We illustrate that the percolation processes repeat in both Cases Q and F, although only in Case F are nodes that initially failed reactivated. To analytically evaluate the resilience of each layer, we develop a methodology based on the cavity method for deriving the size of a giant component (GC). Strong hysteresis, which is ignored in the standard cavity analysis, is observed in the repetition of the percolation processes particularly in Case F. To handle this, we heuristically modify interlayer messages for macroscopic analysis, the utility of which is verified by numerical experiments. The percolation transition in each layer is continuous in both Cases Q and F. We also analyze the influences of degree-degree correlations on the robustness of layer $α$, in particular for the case of TAs. The analysis indicates that the critical fraction of initial failures that makes the GC size in layer $α$ vanish depends only on its intralayer degree-degree correlations. Although our model is defined in a somewhat abstract manner, it may have relevance to ecological systems that are composed of endangered species (layer $α$) and invaders (layer $β$), the former of which are damaged by the latter whereas the latter are exterminated in the areas where the former are active.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Sparse approximation based on a random overcomplete basis

We discuss a strategy of sparse approximation that is based on the use of an overcomplete basis, and evaluate its performance when a random matrix is used as this basis. A small combination of basis vectors is chosen from a given overcomplete basis, according to a given compression rate, such that they compactly represent the target data with as small a distortion as possible. As a selection method, we study the $\ell_0$- and $\ell_1$-based methods, which employ the exhaustive search and $\ell_1$-norm regularization techniques, respectively. The performance is assessed in terms of the trade-off relation between the representation distortion and the compression rate. First, we evaluate the performance analytically in the case that the methods are carried out ideally, using methods of statistical mechanics. Our result clarifies the fact that the $\ell_0$-based method greatly outperforms the $\ell_1$-based one. Second, we examine the practical performances of two well-known algorithms, orthogonal matching pursuit and approximate message passing, when they are used to execute the $\ell_0$- and $\ell_1$-based methods, respectively. Our examination shows that orthogonal matching pursuit achieves a much better performance than the exact execution of the $\ell_1$-based method, as well as approximate message passing. However, regarding the $\ell_0$-based method, there is still room to design more effective greedy algorithms than orthogonal matching pursuit. Finally, we evaluate the performances of the algorithms when they are applied to image data compression.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Sparse approximation problem: how rapid simulated annealing succeeds and fails

Information processing techniques based on sparseness have been actively studied in several disciplines. Among them, a mathematical framework to approximately express a given dataset by a combination of a small number of basis vectors of an overcomplete basis is termed the {\em sparse approximation}. In this paper, we apply simulated annealing, a metaheuristic algorithm for general optimization problems, to sparse approximation in the situation where the given data have a planted sparse representation and noise is present. The result in the noiseless case shows that our simulated annealing works well in a reasonable parameter region: the planted solution is found fairly rapidly. This is true even in the case where a common relaxation of the sparse approximation problem, the $\ell_1$-relaxation, is ineffective. On the other hand, when the dimensionality of the data is close to the number of non-zero components, another metastable state emerges, and our algorithm fails to find the planted solution. This phenomenon is associated with a first-order phase transition. In the case of very strong noise, it is no longer meaningful to search for the planted solution. In this situation, our algorithm determines a solution with close-to-minimum distortion fairly quickly.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Statistical mechanics analysis of thresholding 1-bit compressed sensing

The one-bit compressed sensing framework aims to reconstruct a sparse signal by only using the sign information of its linear measurements. To compensate for the loss of scale information, past studies in the area have proposed recovering the signal by imposing an additional constraint on the L2-norm of the signal. Recently, an alternative strategy that captures scale information by introducing a threshold parameter to the quantization process was advanced. In this paper, we analyze the typical behavior of the thresholding 1-bit compressed sensing utilizing the replica method of statistical mechanics, so as to gain an insight for properly setting the threshold value. Our result shows that, fixing the threshold at a constant value yields better performance than varying it randomly when the constant is optimally tuned, statistically. Unfortunately, the optimal threshold value depends on the statistical properties of the target signal, which may not be known in advance. In order to handle this inconvenience, we develop a heuristic that adaptively tunes the threshold parameter based on the frequency of positive (or negative) values in the binary outputs. Numerical experiments show that the heuristic exhibits satisfactory performance while incurring low computational cost.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Detectability of the spectral method for sparse graph partitioning

We show that modularity maximization with the resolution parameter offers a unifying framework of graph partitioning. In this framework, we demonstrate that the spectral method exhibits universal detectability, irrespective of the value of the resolution parameter, as long as the graph is partitioned. Furthermore, we show that when the resolution parameter is sufficiently small, a first-order phase transition occurs, resulting in the graph being unpartitioned.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Limitations in the spectral method for graph partitioning: detectability threshold and localization of eigenvectors

Investigating the performance of different methods is a fundamental problem in graph partitioning. In this paper, we estimate the so-called detectability threshold for the spectral method with both unnormalized and normalized Laplacians in sparse graphs. The detectability threshold is the critical point at which the result of the spectral method is completely uncorrelated to the planted partition. We also analyze whether the localization of eigenvectors affects the partitioning performance in the detectable region. We use the replica method, which is often used in the field of spin-glass theory, and focus on the case of bisection. We show that the gap between the estimated threshold for the spectral method and the threshold obtained from Bayesian inference is considerable in sparse graphs, even without eigenvector localization. This gap closes in a dense limit.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Online compressed sensing

In this paper, we explore the possibilities and limitations of recovering sparse signals in an online fashion. Employing a mean field approximation to the Bayes recursion formula yields an online signal recovery algorithm that can be performed with a computational cost that is linearly proportional to the signal length per update. Analysis of the resulting algorithm indicates that the online algorithm asymptotically saturates the optimal performance limit achieved by the offline method in the presence of Gaussian measurement noise, while differences in the allowable computational costs may result in fundamental gaps of the achievable performance in the absence of noise.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Replica analysis of Franz-Parisi potential for sparse systems

We propose a method for calculating the Franz-Parisi potential for spin glass models on sparse random graphs using the replica method under the replica symmetric ansatz. The resulting self-consistent equations have the solution with the characteristic structure of multi-body overlaps, and the self-consistent equations under this solution are equivalent to the one-step replica symmetry breaking (1RSB) cavity equation with Parisi parameter $x=1$. This method is useful for the evaluation of transition temperatures of the $p$-spin model on regular random graphs under a uniform magnetic field.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Replica Symmetric Bound for Restricted Isometry Constant

We develop a method for evaluating restricted isometry constants (RICs). This evaluation is reduced to the identification of the zero-points of entropy, which is defined for submatrices that are composed of columns selected from a given measurement matrix. Using the replica method developed in statistical mechanics, we assess RICs for Gaussian random matrices under the replica symmetric (RS) assumption. In order to numerically validate the adequacy of our analysis, we employ the exchange Monte Carlo (EMC) method, which has been empirically demonstrated to achieve much higher numerical accuracy than naive Monte Carlo methods. The EMC method suggests that our theoretical estimation of an RIC corresponds to an upper bound that is tighter than in preceding studies. Physical consideration indicates that our assessment of the RIC could be improved by taking into account the replica symmetry breaking.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Cavity-based robustness analysis of interdependent networks: Influences of intranetwork and internetwork degree-degree correlations

We develop a methodology for analyzing the percolation phenomena of two mutually coupled (interdependent) networks based on the cavity method of statistical mechanics. In particular, we take into account the influence of degree-degree correlations inside and between the networks on the network robustness against targeted attacks and random failures. We show that the developed methodology is reduced to the well-known generating function formalism in the absence of degree-degree correlations. The validity of the developed methodology is confirmed by a comparison with the results of numerical experiments. Our analytical results imply that the robustness of the interdependent networks depends considerably on both the intra- and internetwork degree-degree correlations in the case of targeted attacks, whereas the significance of the degree-degree correlations is relatively low for random failures.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Dynamics of asymmetric kinetic Ising systems revisited

The dynamics of an asymmetric kinetic Ising model is studied. Two schemes for improving the existing mean-field description are proposed. In the first scheme, we derive the formulas for instantaneous magnetization, equal-time correlation, and time-delayed correlation, considering the correlation between different local fields. To derive the time-delayed correlation, we emphasize that the small correlation assumption adopted in previous work [M. Mézard and J. Sakellariou, J. Stat. Mech., L07001 (2011)] is in fact not required. To confirm the inference efficiency of our method, we perform extensive simulations on single instances with either temporally constant external driving fields or sinusoidal external fields. In the second scheme, we develop an improved mean-field theory for instantaneous magnetization prediction utilizing the notion of the cavity system in conjunction with a perturbative expansion approach. Its efficiency is numerically confirmed by comparison with the existing mean-field theory when partially asymmetric couplings are present.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Origin of the computational hardness for learning with binary synapses

Supervised learning in a binary perceptron is able to classify an extensive number of random patterns by a proper assignment of binary synaptic weights. However, to find such assignments in practice, is quite a nontrivial task. The relation between the weight space structure and the algorithmic hardness has not yet been fully understood. To this end, we analytically derive the Franz-Parisi potential for the binary preceptron problem, by starting from an equilibrium solution of weights and exploring the weight space structure around it. Our result reveals the geometrical organization of the weight space\textemdash the weight space is composed of isolated solutions, rather than clusters of exponentially many close-by solutions. The point-like clusters far apart from each other in the weight space explain the previously observed glassy behavior of stochastic local search heuristics.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Sample Complexity of Bayesian Optimal Dictionary Learning

We consider a learning problem of identifying a dictionary matrix D (M times N dimension) from a sample set of M dimensional vectors Y = N^{-1/2} DX, where X is a sparse matrix (N times P dimension) in which the density of non-zero entries is 0<rho< 1. In particular, we focus on the minimum sample size P_c (sample complexity) necessary for perfectly identifying D of the optimal learning scheme when D and X are independently generated from certain distributions. By using the replica method of statistical mechanics, we show that P_c=O(N) holds as long as alpha = M/N >rho is satisfied in the limit of N to infinity. Our analysis also implies that the posterior distribution given Y is condensed only at the correct dictionary D when the compression rate alpha is greater than a certain critical value alpha_M(rho). This suggests that belief propagation may allow us to learn D with a low computational complexity using O(N) samples.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Signal recovery using expectation consistent approximation for linear observations

A signal recovery scheme is developed for linear observation systems based on expectation consistent (EC) mean field approximation. Approximate message passing (AMP) is known to be consistent with the results obtained using the replica theory, which is supposed to be exact in the large system limit, when each entry of the observation matrix is independently generated from an identical distribution. However, this is not necessarily the case for general matrices. We show that EC recovery exhibits consistency with the replica theory for a wider class of random observation matrices. This is numerically confirmed by experiments for the Bayesian optimal signal recovery of compressed sensing using random row-orthogonal matrices.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Statistical mechanics approach to 1-bit compressed sensing

Compressed sensing is a technique for recovering a high-dimensional signal from lower-dimensional data, whose components represent partial information about the signal, utilizing prior knowledge on the sparsity of the signal. For further reducing the data size of the compressed expression, a scheme to recover the original signal utilizing only the sign of each entry of the linearly transformed vector was recently proposed. This approach is often termed the 1-bit compressed sensing. Here we analyze the typical performance of an L1-norm based signal recovery scheme for the 1-bit compressed sensing using statistical mechanics methods. We show that the signal recovery performance predicted by the replica method under the replica symmetric ansatz, which turns out to be locally unstable for modes breaking the replica symmetry, is in a good consistency with experimental results of an approximate recovery algorithm developed earlier. This suggests that the L1-based recovery problem typically has many local optima of a similar recovery accuracy, which can be achieved by the approximate algorithm. We also develop another approximate recovery algorithm inspired by the cavity method. Numerical experiments show that when the density of nonzero entries in the original signal is relatively large the new algorithm offers better performance than the abovementioned scheme and does so with a lower computational cost.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

Adaptive Thouless-Anderson-Palmer approach to inverse Ising problems with quenched random fields

The adaptive Thouless-Anderson-Palmer equation is derived for inverse Ising problems in the presence of quenched random fields. We test the proposed scheme on Sherrington-Kirkpatrick, Hopfield, and random orthogonal models and find that the adaptive Thouless-Anderson-Palmer approach allows surprisingly accurate inference of quenched random fields whose distribution can be either Gaussian or bimodal, compared with other existing mean-field methods.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

Compressed sensing with sparse, structured matrices

In the context of the compressed sensing problem, we propose a new ensemble of sparse random matrices which allow one (i) to acquire and compress a ρ0-sparse signal of length N in a time linear in N and (ii) to perfectly recover the original signal, compressed at a rate α, by using a message passing algorithm (Expectation Maximization Belief Propagation) that runs in a time linear in N. In the large N limit, the scheme proposed here closely approaches the theoretical bound ρ0 = α, and so it is both optimal and efficient (linear time complexity). More generally, we show that several ensembles of dense random matrices can be converted into ensembles of sparse random matrices, having the same thresholds, but much lower computational complexity.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

Entropy landscape of solutions in the binary perceptron problem

The statistical picture of the solution space for a binary perceptron is studied. The binary perceptron learns a random classification of input random patterns by a set of binary synaptic weights. The learning of this network is difficult especially when the pattern (constraint) density is close to the capacity, which is supposed to be intimately related to the structure of the solution space. The geometrical organization is elucidated by the entropy landscape from a reference configuration and of solution-pairs separated by a given Hamming distance in the solution space. We evaluate the entropy at the annealed level as well as replica symmetric level and the mean field result is confirmed by the numerical simulations on single instances using the proposed message passing algorithms. From the first landscape (a random configuration as a reference), we see clearly how the solution space shrinks as more constraints are added. From the second landscape of solution-pairs, we deduce the coexistence of clustering and freezing in the solution space.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

Reconstruction algorithm in compressed sensing based on maximum a posteriori estimation

We propose a systematic method for constructing a sparse data reconstruction algorithm in compressed sensing at a relatively low computational cost for general observation matrix. It is known that the cost of l1-norm minimization using a standard linear programming algorithm is O(N^3). We show that this cost can be reduced to O(N^2) by applying the approach of posterior maximization. Furthermore, in principle, the algorithm from our approach is expected to achieve the widest successful reconstruction region, which is evaluated from theoretical argument. We also discuss the relation between the belief propagation-based reconstruction algorithm introduced in preceding works and our approach.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

Statistical Mechanics Approach to Sparse Noise Denoising

Reconstruction fidelity of sparse signals contaminated by sparse noise is considered. Statistical mechanics inspired tools are used to show that the l1-norm based convex optimization algorithm exhibits a phase transition between the possibility of perfect and imperfect reconstruction. Conditions characterizing this threshold are derived and the mean square error of the estimate is obtained for the case when perfect reconstruction is not possible. Detailed calculations are provided to expose the mathematical tools to a wide audience.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

Statistical Mechanics of Dictionary Learning

Finding a basis matrix (dictionary) by which objective signals are represented sparsely is of major relevance in various scientific and technological fields. We consider a problem to learn a dictionary from a set of training signals. We employ techniques of statistical mechanics of disordered systems to evaluate the size of the training set necessary to typically succeed in the dictionary learning. The results indicate that the necessary size is much smaller than previously estimated, which theoretically supports and/or encourages the use of dictionary learning in practical situations.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

A Mean-field Approach for an Intercarrier Interference Canceller for OFDM

The similarity of the mathematical description of random-field spin systems to orthogonal frequency-division multiplexing (OFDM) scheme for wireless communication is exploited in an intercarrier-interference (ICI) canceller used in the demodulation of OFDM. The translational symmetry in the Fourier domain generically concentrates the major contribution of ICI from each subcarrier in the subcarrier's neighborhood. This observation in conjunction with mean field approach leads to a development of an ICI canceller whose necessary cost of computation scales linearly with respect to the number of subcarriers. It is also shown that the dynamics of the mean-field canceller are well captured by a discrete map of a single macroscopic variable, without taking the spatial and time correlations of estimated variables into account.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

A study of the universal threshold in the L1 recovery by statistical mechanics

We discuss the universality of the L1 recovery threshold in compressed sensing. Previous studies in the fields of statistical mechanics and random matrix integration have shown that L1 recovery under a random matrix with orthogonal symmetry has a universal threshold. This indicates that the threshold of L1 recovery under a non-orthogonal random matrix differs from the universal one. Taking this into account, we use a simple random matrix without orthogonal symmetry, where the random entries are not independent, and show analytically that the threshold of L1 recovery for such a matrix does not coincide with the universal one. The results of an extensive numerical experiment are in good agreement with the analytical results, which validates our methodology. Though our analysis is based on replica heuristics in statistical mechanics and is not rigorous, the findings nevertheless support the fact that the universality of the threshold is strongly related to the symmetry of the random matrix.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Analysis of Sparse Representations Using Bi-Orthogonal Dictionaries

The sparse representation problem of recovering an N dimensional sparse vector x from M < N linear observations y = Dx given dictionary D is considered. The standard approach is to let the elements of the dictionary be independent and identically distributed (IID) zero-mean Gaussian and minimize the l1-norm of x under the constraint y = Dx. In this paper, the performance of l1-reconstruction is analyzed, when the dictionary is bi-orthogonal D = [O1 O2], where O1,O2 are independent and drawn uniformly according to the Haar measure on the group of orthogonal M x M matrices. By an application of the replica method, we obtain the critical conditions under which perfect l1-recovery is possible with bi-orthogonal dictionaries.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

First eigenvalue/eigenvector in sparse random symmetric matrices: influences of degree fluctuation

The properties of the first (largest) eigenvalue and its eigenvector (first eigenvector) are investigated for large sparse random symmetric matrices that are characterized by bimodal degree distributions. In principle, one should be able to accurately calculate them by solving a functional equation concerning auxiliary fields which come out in an analysis based on replica/cavity methods. However, the difficulty in analytically solving this equation makes an accurate calculation infeasible in practice. To overcome this problem, we develop approximation schemes on the basis of two exceptionally solvable examples. The schemes are reasonably consistent with numerical experiments when the statistical bias of positive matrix entries is sufficiently large, and they qualitatively explain why considerably large finite size effects of the first eigenvalue can be observed when the bias is relatively small.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Typical $l_1$-recovery limit of sparse vectors represented by concatenations of random orthogonal matrices

We consider the problem of recovering an $N$-dimensional sparse vector $\vm{x}$ from its linear transformation $\vm{y}=\vm{D} \vm{x}$ of $M(< N)$ dimension. Minimizing the $l_{1}$-norm of $\vm{x}$ under the constraint $\vm{y} = \vm{D} \vm{x}$ is a standard approach for the recovery problem, and earlier studies report that the critical condition for typically successful $l_1$-recovery is universal over a variety of randomly constructed matrices $\vm{D}$. For examining the extent of the universality, we focus on the case in which $\vm{D}$ is provided by concatenating $\nb=N/M$ matrices $\vm{O}_{1}, \vm{O}_{2},..., \vm{O}_\nb$ drawn uniformly according to the Haar measure on the $M \times M$ orthogonal matrices. By using the replica method in conjunction with the development of an integral formula for handling the random orthogonal matrices, we show that the concatenated matrices can result in better recovery performance than what the universality predicts when the density of non-zero signals is not uniform among the $\nb$ matrix modules. The universal condition is reproduced for the special case of uniform non-zero signal densities. Extensive numerical experiments support the theoretical predictions.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

A signal recovery algorithm for sparse matrix based compressed sensing

We have developed an approximate signal recovery algorithm with low computational cost for compressed sensing on the basis of randomly constructed sparse measurement matrices. The law of large numbers and the central limit theorem suggest that the developed algorithm saturates the Donoho-Tanner weak threshold for the perfect recovery when the matrix becomes as dense as the signal size $N$ and the number of measurements $M$ tends to infinity keep $α=M/N \sim O(1)$, which is supported by extensive numerical experiments. Even when the numbers of non-zero entries per column/row in the measurement matrices are limited to $O(1)$, numerical experiments indicate that the algorithm can still typically recover the original signal perfectly with an $O(N)$ computational cost per update as well if the density $ρ$ of non-zero entries of the signal is lower than a certain critical value $ρ_{\rm th}(α)$ as $N,M \to \infty$.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Statistical mechanical assessment of a reconstruction limit of compressed sensing: Toward theoretical analysis of correlated signals

We provide a scheme for exploring the reconstruction limit of compressed sensing by minimizing the general cost function under the random measurement constraints for generic correlated signal sources. Our scheme is based on the statistical mechanical replica method for dealing with random systems. As a simple but non-trivial example, we apply the scheme to a sparse autoregressive model, where the first differences in the input signals of the correlated time series are sparse, and evaluate the critical compression rate for a perfect reconstruction. The results are in good agreement with a numerical experiment for a signal reconstruction.

0 citations0 reviewsNo code linkedOpen access
preprint2010arXiv

Cavity analysis on the robustness of random networks against targeted attacks: Influences of degree-degree correlations

We developed a scheme for evaluating the size of the largest connected subnetwork (giant component) in random networks and the percolation threshold when sites (nodes) and/or bonds (edges) are removed from the networks based on the cavity method of statistical mechanics of disordered systems. An advantage of our scheme is the capability of handling targeted attacks on sites/bonds in the presence of degree correlations beyond naive analyses on random failures (crashes) in networks of no degree correlations. We apply our scheme particularly to random networks of bimodal degree distribution (two-peak networks), which have been proposed in earlier studies as robust networks against random failures of site and/or targeted attacks on sites, and show that the correlations among degrees affect a network's robustness against targeted attacks on sites or bonds non-trivially depending on details of network configurations.

0 citations0 reviewsNo code linkedOpen access
preprint2010arXiv

Cavity approach to the first eigenvalue problem in a family of symmetric random sparse matrices

A methodology to analyze the properties of the first (largest) eigenvalue and its eigenvector is developed for large symmetric random sparse matrices utilizing the cavity method of statistical mechanics. Under a tree approximation, which is plausible for infinitely large systems, in conjunction with the introduction of a Lagrange multiplier for constraining the length of the eigenvector, the eigenvalue problem is reduced to a bunch of optimization problems of a quadratic function of a single variable, and the coefficients of the first and the second order terms of the functions act as cavity fields that are handled in cavity analysis. We show that the first eigenvalue is determined in such a way that the distribution of the cavity fields has a finite value for the second order moment with respect to the cavity fields of the first order coefficient. The validity and utility of the developed methodology are examined by applying it to two analytically solvable and one simple but non-trivial examples in conjunction with numerical justification.

0 citations0 reviewsNo code linkedOpen access
preprint2010arXiv

Finite size scaling of the de Almeida-Thouless instability in random sparse networks

We study, in random sparse networks, finite size scaling of the spin glass susceptibility $χ_{\rm SG}$, which is a proper measure of the de Almeida-Thouless (AT) instability of spin glass systems. Using a phenomenological argument regarding the band edge behavior of the Hessian eigenvalue distribution, we discuss how $χ_{\rm SG}$ is evaluated in infinitely large random sparse networks, which are usually identified with Bethe trees, and how it should be corrected in finite systems. In the high temperature region, data of extensive numerical experiments are generally in good agreement with the theoretical values of $χ_{\rm SG}$ determined from the Bethe tree. In the absence of external fields, the data also show a scaling relation $χ_{\rm SG}=N^{1/3}F(N^{1/3}|T-T_c|/T_c)$, which has been conjectured in the literature, where $T_c$ is the critical temperature. In the presence of external fields, on the other hand, the numerical data are not consistent with this scaling relation. A numerical analysis of Hessian eigenvalues implies that strong finite size corrections of the lower band edge of the eigenvalue distribution, which seem relevant only in the presence of the fields, are a major source of inconsistency. This may be related to the known difficulty in using only numerical methods to detect the AT instability.

0 citations0 reviewsNo code linkedOpen access
preprint2010arXiv

Statistical Mechanical Analysis of Compressed Sensing Utilizing Correlated Compression Matrix

We investigate a reconstruction limit of compressed sensing for a reconstruction scheme based on the L1-norm minimization utilizing a correlated compression matrix with a statistical mechanics method. We focus on the compression matrix modeled as the Kronecker-type random matrix studied in research on multi-input multi-output wireless communication systems. We found that strong one-dimensional correlations between expansion bases of original information slightly degrade reconstruction performance.

0 citations0 reviewsNo code linkedOpen access
preprint2010arXiv

Zero-Temperature Complex Replica Zeros of the $\pm J$ Ising Spin Glass on Mean-Field Systems and Beyond

Zeros of the moment of the partition function $[Z^n]_{\bm{J}}$ with respect to complex $n$ are investigated in the zero temperature limit $β\to \infty$, $n\to 0$ keeping $y=βn \approx O(1)$. We numerically investigate the zeros of the $\pm J$ Ising spin glass models on several Cayley trees and hierarchical lattices and compare those results. In both lattices, the calculations are carried out with feasible computational costs by using recursion relations originated from the structures of those lattices. The results for Cayley trees show that a sequence of the zeros approaches the real axis of $y$ implying that a certain type of analyticity breaking actually occurs, although it is irrelevant for any known replica symmetry breaking. The result of hierarchical lattices also shows the presence of analyticity breaking, even in the two dimensional case in which there is no finite-temperature spin-glass transition, which implies the existence of the zero-temperature phase transition in the system. A notable tendency of hierarchical lattices is that the zeros spread in a wide region of the complex $y$ plane in comparison with the case of Cayley trees, which may reflect the difference between the mean-field and finite-dimensional systems.

0 citations0 reviewsNo code linkedOpen access
preprint2009arXiv

On analyticity with respect to the replica number in random energy models II: zeros on the complex plane

We characterize the breaking of analyticity with respect to the replica number which occurs in random energy models via the complex zeros of the moment of the partition function. We perturbatively evaluate the zeros in the vicinity of the transition point based on an exact expression of the moment of the partition function utilizing the steepest descent method, and examine an asymptotic form of the locus of the zeros as the system size tends to infinity. The incident angle of this locus indicates that analyticity breaking is analogous to a phase transition of the second order. We also evaluate the number of zeros utilizing the argument principle of complex analysis. The actual number of zeros calculated numerically for systems of finite size agrees fairly well with the analytical results.

0 citations0 reviewsNo code linkedOpen access
preprint2009arXiv

Statistical mechanical analysis of the Kronecker channel model for MIMO wireless communication

The Kronecker channel model of wireless communication is analyzed using statistical mechanics methods. In the model, spatial proximities among transmission/reception antennas are taken into account as certain correlation matrices, which generally yield non-trivial dependence among symbols to be estimated. This prevents accurate assessment of the communication performance by naively using a previously developed analytical scheme based on a matrix integration formula. In order to resolve this difficulty, we develop a formalism that can formally handle the correlations in Kronecker models based on the known scheme. Unfortunately, direct application of the developed scheme is, in general, practically difficult. However, the formalism is still useful, indicating that the effect of the correlations generally increase after the fourth order with respect to correlation strength. Therefore, the known analytical scheme offers a good approximation in performance evaluation when the correlation strength is sufficiently small. For a class of specific correlation, we show that the performance analysis can be mapped to the problem of one-dimensional spin systems in random fields, which can be investigated without approximation by the belief propagation algorithm.

0 citations0 reviewsNo code linkedOpen access
preprint2009arXiv

Weight space structure and analysis using a finite replica number in the Ising perceptron

The weight space of the Ising perceptron in which a set of random patterns is stored is examined using the generating function of the partition function $ϕ(n)=(1/N)\log [Z^n]$ as the dimension of the weight vector $N$ tends to infinity, where $Z$ is the partition function and $[ ... ]$ represents the configurational average. We utilize $ϕ(n)$ for two purposes, depending on the value of the ratio $α=M/N$, where $M$ is the number of random patterns. For $α< α_{\rm s}=0.833 ...$, we employ $ϕ(n)$, in conjunction with Parisi's one-step replica symmetry breaking scheme in the limit of $n \to 0$, to evaluate the complexity that characterizes the number of disjoint clusters of weights that are compatible with a given set of random patterns, which indicates that, in typical cases, the weight space is equally dominated by a single large cluster of exponentially many weights and exponentially many small clusters of a single weight. For $α> α_{\rm s}$, on the other hand, $ϕ(n)$ is used to assess the rate function of a small probability that a given set of random patterns is atypically separable by the Ising perceptrons. We show that the analyticity of the rate function changes at $α= α_{\rm GD}=1.245 ... $, which implies that the dominant configuration of the atypically separable patterns exhibits a phase transition at this critical ratio. Extensive numerical experiments are conducted to support the theoretical predictions.

0 citations0 reviewsNo code linkedOpen access
preprint2008arXiv

Learning of correlated patterns by simple perceptrons

Learning behavior of simple perceptrons is analyzed for a teacher-student scenario in which output labels are provided by a teacher network for a set of possibly correlated input patterns, and such that teacher and student networks are of the same type. Our main concern is the effect of statistical correlations among the input patterns on learning performance. For this purpose, we extend to the teacher-student scenario a methodology for analyzing randomly labeled patterns recently developed in {\em J. Phys. A: Math. Theor.} {\bf 41}, 324013 (2008). This methodology is used for analyzing situations in which orthogonality of the input patterns is enhanced in order to optimize the learning performance.

0 citations0 reviewsNo code linkedOpen access
preprint2007arXiv

Perceptron capacity revisited: classification ability for correlated patterns

In this paper, we address the problem of how many randomly labeled patterns can be correctly classified by a single-layer perceptron when the patterns are correlated with each other. In order to solve this problem, two analytical schemes are developed based on the replica method and Thouless-Anderson-Palmer (TAP) approach by utilizing an integral formula concerning random rectangular matrices. The validity and relevance of the developed methodologies are shown for one known result and two example problems. A message-passing algorithm to perform the TAP scheme is also presented.

0 citations0 reviewsNo code linkedOpen access
preprint2006arXiv

Numerical Study of TAP Metastable States in 3-body Ising Spin Glasses

The distribution of solutions of the Thouless-Anderson-Palmer equation is studied by extensive numerical experiments for fully connected 3-body interaction Ising spin glass models in a level of annealed calculation. A recent study predicted that when the equilibrium state of the system is characterized by one-step replica symmetry breaking, the distribution is described by a Becchi-Rouet-Stora-Tyutin (BRST) supersymmetric solution in the relatively low free energy region, whereas the BRST supersymmetry is broken for higher values of free energy (Crisanti et al., Phys. Rev. B 71 (2005) 094202). Our experiments qualitatively reproduce the discriminative behavior of macroscopic variables predicted by the theoretical assessment.

0 citations0 reviewsNo code linkedOpen access
preprint2005arXiv

An LDPCC decoding algorithm based on Bowman-Levin approximation --Comparison with BP and CCCP--

Belief propagation (BP) and the concave convex procedure (CCCP) are both methods that utilize the Bethe free energy as a cost function and solve information processing tasks. We have developed a new algorithm that also uses the Bethe free energy, but changes the roles of the master variables and the slave variables. This is called the Bowman-Levin (BL) approximation in the domain of statistical physics. When we applied the BL algorithm to decode the Gallager ensemble of short-length regular low-density parity check codes (LDPCC) over an additive white Gaussian noise (AWGN) channel, its average performance was somewhat better than that of either BP or CCCP. This implies that the BL algorithm can also be successfully applied to other problems to which BP or CCCP has already been applied.

0 citations0 reviewsNo code linkedOpen access
preprint2002arXiv

A statistical-mechanical approach to CDMA multiuser detection: propagating beliefs in a densely connected graph

The task of CDMA multiuser detection is to simultaneously estimate binary symbols of $K$ synchronous users from the received $N$ base-band CDMA signals. Mathematically, this can be formulated as an inference problem on a complete bipartite graph. In the research on graphically represented statistical models, it is known that the belief propagation (BP) can exactly perform the inference in a polynomial time scale of the system size when the graph is free from cycles in spite that the necessary computation for general graphs exponentially explodes in the worst case. In addition, recent several researches revealed that the BP can also serve as an excellent approximation algorithm even if the graph has cycles as far as they are relatively long. However, as there exit many short cycles in a complete bipartite graph, one might suspect that the BP would not provide a good performance when employed for the multiuser detection. The purpose of this paper is to make an objection to such suspicion. More specifically, we will show that appropriate employment of the central limit theorem and the law of large numbers to BP, which is one of the standard techniques in statistical mechanics, makes it possible to develop a novel multiuser detection algorithm the convergence property of which is considerably better than that of the conventional multistage detection without increasing the computational cost significantly. Furthermore, we will also provide a scheme to analyse the dynamics of the proposed algorithm, which can be naturally linked to the equilibrium analysis recently presented by Tanaka.

0 citations0 reviewsNo code linkedOpen access
preprint2000arXiv

Statistical Physics of Irregular Low-Density Parity-Check Codes

Low-density parity-check codes with irregular constructions have been recently shown to outperform the most advanced error-correcting codes to date. In this paper we apply methods of statistical physics to study the typical properties of simple irregular codes. We use the replica method to find a phase transition which coincides with Shannon's coding bound when appropriate parameters are chosen. The decoding by belief propagation is also studied using statistical physics arguments; the theoretical solutions obtained are in good agreement with simulations. We compare the performance of irregular with that of regular codes and discuss the factors that contribute to the improvement in performance.

0 citations0 reviewsNo code linkedOpen access