BZPEERPapers, people, evidence and research paths
Menu

Discover

SearchFind papers, people, topics, institutions and opportunities from one place.GraphSee how papers, authors, topics and evidence connect.

Research tools

HomeContinue from the papers, people and topics that matter now.CollectionsTurn saved papers into reading lists, evidence maps and shareable context.ResearchReconstruct the path that led you from search to paper to question.CollaborationMove from a paper, topic or expert request into a focused thread.PublishCreate full-text research articles with metadata, formulas and versions.

Network

ExpertsFind specialists whose work and reviews explain why they can help.CommunitiesJoin research groups organized around papers, topics and shared evidence.

Opportunities

ListingsBrowse roles, calls and collaborations connected to real research context.

Account

Log inReturn to your saved papers, graph views and active threads.Create accountStart saving papers, building a research trail and joining teams.
Log inCreate account

Source author record

Stefano Rini

Stefano Rini appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record
Information Theorymath.ITMachine Learningeess.SPComputationComputer VisionDistributed, Parallel, and Cluster ComputingInformation RetrievalNeurons and Cognition

Catalog footprint

What is connected

37works
9topics
4close collaborators

Actions

Connect this record

Log in to claim
Open graphBrowse works

Research graph

See the researcher in context

Open full explorer

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

37 published item(s)

preprint2022arXiv

Coded Demixing for Unsourced Random Access

Unsourced random access (URA) is a recently proposed multiple access paradigm tailored to the uplink channel of machine-type communication networks. By exploiting a strong connection between URA and compressed sensing, the massive multiple access problem may be cast as a compressed sensing (CS) problem, albeit one in exceedingly large dimensions. To efficiently handle the dimensionality of the problem, coded compressed sensing (CCS) has emerged as a pragmatic signal processing tool that, when applied to URA, offers good performance at low complexity. While CCS is effective at recovering a signal that is sparse with respect to a single basis, it is unable to jointly recover signals that are sparse with respect to separate bases. In this article, the CCS framework is extended to the demixing setting, yielding a novel technique called coded demixing. A generalized framework for coded demixing is presented and a low-complexity recovery algorithm based on approximate message passing (AMP) is developed. Coded demixing is applied to heterogeneous multi-class URA networks and traditional single-class networks. Its performance is analyzed and numerical simulations are presented to highlight the benefits of coded demixing.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Compressibility Measures for Affinely Singular Random Vectors

There are several ways to measure the compressibility of a random measure; they include general approaches such as using the rate-distortion curve, as well as more specific notions, such as the Renyi information dimension (RID). The RID parameter indicates the concentration of the measure around lower-dimensional subsets of the space. While the evaluation of such compressibility parameters is well-studied for continuous and discrete measures, the case of discrete-continuous measures is quite subtle. In this paper, we focus on a class of multi-dimensional random measures that have singularities on affine lower-dimensional subsets. This class of distributions naturally arises when considering linear transformation of component-wise independent discrete-continuous random variables. To measure the compressibility of such distributions, we introduce the new notion of dimensional-rate bias (DRB) which is closely related to the entropy and differential entropy in discrete and continuous cases, respectively. Similar to entropy and differential entropy, DRB is useful in evaluating the mutual information between distributions of the aforementioned type. Besides the DRB, we also evaluate the the RID of these distributions. We further provide an upper-bound for the RID of multi-dimensional random measures that are obtained by Lipschitz functions of component-wise independent discrete-continuous random variables ($\mathbf{X}$). The upper-bound is shown to be achievable when the Lipschitz function is $A \mathbf{X}$, where $A$ satisfies {\changed$\spark({A_{m\times n}}) = m+1$} (e.g., Vandermonde matrices). When considering discrete-domain moving-average processes with non-Gaussian excitation noise, the above results allow us to evaluate the block-average RID and DRB, as well as to determine a relationship between these parameters and other existing compressibility measures.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Convert, compress, correct: Three steps toward communication-efficient DNN training

In this paper, we introduce a novel algorithm, $\mathsf{CO}_3$, for communication-efficiency distributed Deep Neural Network (DNN) training. $\mathsf{CO}_3$ is a joint training/communication protocol, which encompasses three processing steps for the network gradients: (i) quantization through floating-point conversion, (ii) lossless compression, and (iii) error correction. These three components are crucial in the implementation of distributed DNN training over rate-constrained links. The interplay of these three steps in processing the DNN gradients is carefully balanced to yield a robust and high-performance scheme. The performance of the proposed scheme is investigated through numerical evaluations over CIFAR-10.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Lossy Gradient Compression: How Much Accuracy Can One Bit Buy?

In federated learning (FL), a global model is trained at a Parameter Server (PS) by aggregating model updates obtained from multiple remote learners. Generally, the communication between the remote users and the PS is rate-limited, while the transmission from the PS to the remote users are unconstrained. The FL setting gives rise to the distributed learning scenario in which the updates from the remote learners have to be compressed so as to meet communication rate constraints in the uplink transmission toward the PS. For this problem, one wishes to compress the model updates so as to minimize the loss in accuracy resulting from the compression error. In this paper, we take a rate-distortion approach to address the compressor design problem for the distributed training of deep neural networks (DNNs). In particular, we define a measure of the compression performance under communication-rate constraints -- the \emph{per-bit accuracy} -- which addresses the ultimate improvement of accuracy that a bit of communication brings to the centralized model. In order to maximize the per-bit accuracy, we consider modeling the DNN gradient updates at remote learners as a generalized normal distribution. Under this assumption on the DNN gradient distribution, we propose a class of distortion measures to aid the design of quantizers for the compression of the model updates. We argue that this family of distortion measures, which we refer to as "$M$-magnitude weighted $L_2$" norm, captures the practitioner's intuition in the choice of gradient compressor. Numerical simulations are provided to validate the proposed approach for the CIFAR-10 dataset.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Neural Capacity Estimators: How Reliable Are They?

Recently, several methods have been proposed for estimating the mutual information from sample data using deep neural networks and without the knowing closed form distribution of the data. This class of estimators is referred to as neural mutual information estimators. Although very promising, such techniques have yet to be rigorously bench-marked so as to establish their efficacy, ease of implementation, and stability for capacity estimation which is joint maximization frame-work. In this paper, we compare the different techniques proposed in the literature for estimating capacity and provide a practitioner perspective on their effectiveness. In particular, we study the performance of mutual information neural estimator (MINE), smoothed mutual information lower-bound estimator (SMILE), and directed information neural estimator (DINE) and provide insights on InfoNCE. We evaluated these algorithms in terms of their ability to learn the input distributions that are capacity approaching for the AWGN channel, the optical intensity channel, and peak power-constrained AWGN channel. For both scenarios, we provide insightful comments on various aspects of the training process, such as stability, sensitivity to initialization.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Sharp asymptotics on the compression of two-layer neural networks

In this paper, we study the compression of a target two-layer neural network with N nodes into a compressed network with M<N nodes. More precisely, we consider the setting in which the weights of the target network are i.i.d. sub-Gaussian, and we minimize the population L_2 loss between the outputs of the target and of the compressed network, under the assumption of Gaussian inputs. By using tools from high-dimensional probability, we show that this non-convex problem can be simplified when the target network is sufficiently over-parameterized, and provide the error rate of this approximation as a function of the input dimension and N. In this mean-field limit, the simplified objective, as well as the optimal weights of the compressed network, does not depend on the realization of the target network, but only on expected scaling factors. Furthermore, for networks with ReLU activation, we conjecture that the optimum of the simplified optimization problem is achieved by taking weights on the Equiangular Tight Frame (ETF), while the scaling of the weights and the orientation of the ETF depend on the parameters of the target network. Numerical evidence is provided to support this conjecture.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

Hierarchical Causal Bandit

Causal bandit is a nascent learning model where an agent sequentially experiments in a causal network of variables, in order to identify the reward-maximizing intervention. Despite the model's wide applicability, existing analytical results are largely restricted to a parallel bandit version where all variables are mutually independent. We introduce in this work the hierarchical causal bandit model as a viable path towards understanding general causal bandits with dependent variables. The core idea is to incorporate a contextual variable that captures the interaction among all variables with direct effects. Using this hierarchical framework, we derive sharp insights on algorithmic design in causal bandits with dependent arms and obtain nearly matching regret bounds in the case of a binary context.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

Multi-Class Unsourced Random Access via Coded Demixing

Unsourced random access (URA) is a recently proposed communication paradigm attuned to machine-driven data transfers. In the original URA formulation, all the active devices share the same number of bits per packet. The scenario where several classes of devices transmit concurrently has so far received little attention. An initial solution to this problem takes the form of group successive interference cancellation, where codewords from a class of devices with more resources are recovered first, followed by the decoding of the remaining messages. This article introduces a joint iterative decoding approach rooted in approximate message passing. This framework has a concatenated coding structure borrowed from the single-class coded compressed sensing and admits a solution that offers performance improvement at little added computational complexity. Our findings point to new connections between multi-class URA and compressive demixing. The performance of the envisioned algorithm is validated through numerical simulations.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

The Rate-Distortion Risk in Estimation from Compressed Data

Consider the problem of estimating a latent signal from a lossy compressed version of the data when the compressor is agnostic to the relation between the signal and the data. This situation arises in a host of modern applications when data is transmitted or stored prior to determining the downstream inference task. Given a bitrate constraint and a distortion measure between the data and its compressed version, let us consider the joint distribution achieving Shannon's rate-distortion (RD) function. Given an estimator and a loss function associated with the downstream inference task, define the rate-distortion risk as the expected loss under the RD-achieving distribution. We provide general conditions under which the operational risk in estimating from the compressed data is asymptotically equivalent to the RD risk. The main theoretical tools to prove this equivalence are transportation-cost inequalities in conjunction with properties of compression codes achieving Shannon's RD function. Whenever such equivalence holds, a recipe for designing estimators from datasets undergoing lossy compression without specifying the actual compression technique emerges: design the estimator to minimize the RD risk. Our conditions simplified in the special cases of discrete memoryless or multivariate normal data. For these scenarios, we derive explicit expressions for the RD risk of several estimators and compare them to the optimal source coding performance associated with full knowledge of the relation between the latent signal and the data.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Decentralized SGD with Over-the-Air Computation

We study the performance of decentralized stochastic gradient descent (DSGD) in a wireless network, where the nodes collaboratively optimize an objective function using their local datasets. Unlike the conventional setting, where the nodes communicate over error-free orthogonal communication links, we assume that transmissions are prone to additive noise and interference.We first consider a point-to-point (P2P) transmission strategy, termed the OAC-P2P scheme, in which the node pairs are scheduled in an orthogonal fashion to minimize interference. Since in the DSGD framework, each node requires a linear combination of the neighboring models at the consensus step, we then propose the OAC-MAC scheme, which utilizes the signal superposition property of the wireless medium to achieve over-the-air computation (OAC). For both schemes, we cast the scheduling problem as a graph coloring problem. We numerically evaluate the performance of these two schemes for the MNIST image classification task under various network conditions. We show that the OAC-MAC scheme attains better convergence performance with a fewer communication rounds.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

On the Capacity of the Oversampled Wiener Phase Noise Channel

In this paper, the capacity of the oversampled Wiener phase noise (OWPN) channel is investigated. The OWPN channel is a discrete-time point-to-point channel with a multi-sample receiver in which the channel output is affected by both additive and multiplicative noise. The additive noise is a white standard Gaussian process while the multiplicative noise is a Wiener phase noise process. This channel generalizes a number of channel models previously studied in the literature which investigate the effects of phase noise on the channel capacity, such as the Wiener phase noise channel and the non-coherent channel. We derive upper and inner bounds to the capacity of OWPN channel: (i) an upper bound is derived through the I-MMSE relationship by bounding the Fisher information when estimating a phase noise sample given the past channel outputs and phase noise realizations, then (ii) two inner bounds are shown: one relying on coherent combining of the oversampled channel outputs and one relying on non-coherent combining of the samples. After capacity, we study generalized degrees of freedom (GDoF) of the OWPN channel for the case in which the oversampling factor grows with the average transmit power $P$ as $P$? and the frequency noise variance as $P^α$?. Using our new capacity bounds, we derive the GDoF region in three regimes: regime (i) in which the GDoF region equals that of the classic additive white Gaussian noise (for $β\leq 1$), one (ii) in which GDoF region reduces to that of the non-coherent channel (for $β\geq \min \{α,1\}$) and, finally, one in which partially-coherent combining of the over-samples is asymptotically optimal (for $2 α-1\leq β\leq 1$). Overall, our results are the first to identify the regimes in which different oversampling strategies are asymptotically optimal.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

The Information & Mutual Information Ratio for Counting Image Features and Their Matches

Feature extraction and description is an important topic of computer vision, as it is the starting point of a number of tasks such as image reconstruction, stitching, registration, and recognition among many others. In this paper, two new image features are proposed: the Information Ratio (IR) and the Mutual Information Ratio (MIR). The IR is a feature of a single image, while the MIR describes features common across two or more images.We begin by introducing the IR and the MIR and motivate these features in an information theoretical context as the ratio of the self-information of an intensity level over the information contained over the pixels of the same intensity. Notably, the relationship of the IR and MIR with the image entropy and mutual information, classic information measures, are discussed. Finally, the effectiveness of these features is tested through feature extraction over INRIA Copydays datasets and feature matching over the Oxfords Affine Covariant Regions. These numerical evaluations validate the relevance of the IR and MIR in practical computer vision tasks

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

A unified graphical approach to random coding for multi-terminal networks

A unified graphical approach to random coding for any memoryless, single-hop, K-user channel with or without common information is defined through two steps. The first step is user virtualization: each user is divided into multiple virtual sub-users according to a chosen rate-splitting strategy. This results in an enhanced channel with a possibly larger number of users for which more coding possibilities are available and for which common messages to any subset of users can be encoded. Following user virtualization, the message of each user in the enhanced model is coded using a chosen combination of coded time-sharing, superposition coding and joint binning. A graph is used to represent the chosen coding strategies: nodes in the graph represent codewords while edges represent coding operations. This graph is used to construct a graphical Markov model which illustrates the statistical dependency among codewords that can be introduced by the superposition coding or joint binning. Using this statistical representation of the overall codebook distribution, the error probability of the code is shown to vanish via a unified analysis. The rate bounds that define the achievable rate region are obtained by linking the error analysis to the properties of the graphical Markov model. This proposed framework makes it possible to numerically obtain an achievable rate region by specifying a user virtualization strategy and describing a set of coding operations. The union of these rate regions defines the maximum achievable rate region of our unified coding strategy.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

On the Capacity of the Dirty Paper Channel with Fast Fading and Discrete Channel States

The "writing dirty paper" capacity result crucially dependents on the perfect channel knowledge at the transmitter as the presence of even a small uncertainty in the channel realization gravely hampers the ability of the transmitter to pre-code its transmission against the channel state. This is particularly disappointing as it implies that interference pre-coding in practical systems is effective only when the channel estimates at the users have very high precision, a condition which is generally unattainable in wireless environments. In this paper we show that substantial improvements are possible when the state sequence is drawn from a discrete distribution, such as a constrained input constellation, for which state decoding can be approximately optimal. We consider the "writing on dirty paper" channel in which the state sequence is multiplied by a fast fading process and derive conditions on the fading and state distributions for which state decoding closely approaches capacity. These conditions intuitively relate to the ability of the receiver to correctly identify both the input and the state realization despite of the uncertainty introduced by fading.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Optimal Rate Allocation in Mismatched Multiterminal Source Coding

We consider a multiterminal source coding problem in which a source is estimated at a central processing unit from lossy-compressed remote observations. Each lossy-encoded observation is produced by a remote sensor which obtains a noisy version of the source and compresses this observation minimizing a local distortion measure which depends only on the marginal distribution of its observation. The central node, on the other hand, has knowledge of the joint distribution of the source and all the observations and produces the source estimate which minimizes a different distortion measure between the source and its reconstruction. In this correspondence, we investigate the problem of optimally choosing the rate of each lossy-compressed remote estimate so as to minimize the distortion at the central processing unit subject to a bound on the overall communication rate between the remote sensors and the central unit. We focus, in particular, on two models of practical relevance: the case of a Gaussian source observed in additive Gaussian noise and reconstructed under quadratic distortion, and the case of a binary source observed in bit-flipping noise and reconstructed under Hamming distortion. In both scenarios we show that there exist regimes under which having more remote encoders does reduce the source distortion: in other words, having fewer, high-quality remote estimates provides a smaller distortion than having more, lower-quality estimates.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

The Carbon Copy onto Dirty Paper Channel with Statistically Equivalent States

Costa's "writing on dirty paper" capacity result establishes that full state pre-cancellation can be attained in Gelfand-Pinsker channel with additive state and additive Gaussian noise. The "carbon copy onto dirty paper" channel is the extension of Costa's model to the compound setting: M receivers each observe the sum of the channel input, Gaussian noise and one of M Gaussian state sequences and attempt to decode the same common message. The state sequences are all non-causally known at the transmitter which attempts to simultaneously pre-code its transmission against the channel state affecting each output. In this correspondence we derive the capacity to within 2.25 bits-per-channel-use of the carbon copying onto dirty paper channel in which the state sequences are statistically equivalent, having the same variance and the same pairwise correlation. For this channel capacity is approached by letting the channel input be the superposition of two codewords: a base codeword, simultaneously decoded at each user, and a top codeword which is pre-coded against the state realization at each user for a portion 1/M of the time. The outer bound relies on a recursive bounding in which incremental side information is provided at each receiver. This result represents a significant first step toward determining the capacity of the most general "carbon copy onto dirty paper" channel in which state sequences appearing in the different channel outputs have any jointly Gaussian distribution.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

A unified graphical approach to random coding for multi-terminal networks

A unified approach to the derivation of rate regions for single-hop memoryless networks is presented. A general transmission scheme for any memoryless, single-hop, k-user channel with or without common information, is defined through two steps. The first step is user virtualization: each user is divided into multiple virtual sub-users according to a chosen rate-splitting strategy which preserves the rates of the original messages. This results in an enhanced channel with a possibly larger number of users for which more coding possibilities are available. Moreover, user virtualization provides a simple mechanism to encode common messages to any subset of users. Following user virtualization, the message of each user in the enhanced model is coded using a chosen combination of coded time-sharing, superposition coding and joint binning. A graph is used to represent the chosen coding strategies: nodes in the graph represent codewords while edges represent coding operations. This graph is used to construct a graphical Markov model which illustrates the statistical dependency among codewords that can be introduced by the superposition coding or joint binning. Using this statistical representation of the overall codebook distribution, the error probability of the code is shown to vanish via a unified analysis. The rate bounds that define the achievable rate region are obtained by linking the error analysis to the properties of the graphical Markov model. This proposed framework makes it possible to numerically obtain an achievable rate region by specifying a user virtualization strategy and describing a set of coding operations. The largest achievable rate region can be obtained by considering all the possible rate-splitting strategies and taking the union over all the possible ways to superimpose or bin codewords.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Indirect Rate-Distortion Function of a Binary i.i.d Source

The indirect source-coding problem in which a Bernoulli process is compressed in a lossy manner from its noisy observations is considered. These noisy observations are obtained by passing the source sequence through a The indirect source-coding problem in which a Bernoulli process is compressed in a lossy manner from its noisy observations is considered. These noisy observations are obtained by passing the source sequence through a binary symmetric channel so that the channel crossover probability controls the amount of information available about the source realization at the encoder. We use classic results in rate-distortion theory to compute an expression of the rate-distortion function for this model, where the Bernoulli source is not necessarily symmetric. The indirect rate-distortion function is given in terms of a solution to a simple equation. In addition, we derive an upper bound on the indirect rate-distortion function which is given in a closed. These expressions capture precisely the expected behavior that the noisier the observations, the smaller the return from increasing bit-rate to reduce distortion.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

On the Dirty Paper Channel with Fast Fading Dirt

Costa`s "writing on dirty paper" result establishes that full state pre-cancellation can be attained in the Gel`fand-Pinsker problem with additive state and additive white Gaussian noise. This result holds under the assumptions that full channel knowledge is available at both the transmitter and the receiver. In this work we consider the scenario in which the state is multiplied by an ergodic fading process which is not known at the encoder. We study both the case in which the receiver has knowledge of the fading and the case in which it does not: for both models we derive inner and outer bounds to capacity and determine the distance between the two bounds when possible. For the channel without fading knowledge at either the transmitter or the receiver, the gap between inner and outer bounds is finite for a class of fading distributions which includes a number of canonical fading models. In the capacity approaching strategy for this class, the transmitter performs Costa`s pre-coding against the mean value of the fading times the state while the receiver treats the remaining signal as noise. For the case in which only the receiver has knowledge of the fading, we determine a finite gap between inner and outer bounds for two classes of discrete fading distribution. The first class of distributions is the one in which there exists a probability mass larger than one half while the second class is the one in which the fading is uniformly distributed over values that are exponentially spaced apart. Unfortunately, the capacity in the case of a continuous fading distribution remains very hard to characterize.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

On Capacity of the Dirty Paper Channel with Fading Dirt in the Strong Fading Regime

The classical writing on dirty paper capacity result establishes that full interference pre-cancellation can be attained in Gelfand-Pinsker problem with additive state and additive white Gaussian noise. This result holds under the idealized assumption that perfect channel knowledge is available at both transmitter and receiver. While channel knowledge at the receiver can be obtained through pilot tones, transmitter channel knowledge is harder to acquire. For this reason, we are interested in characterizing the capacity under the more realistic assumption that only partial channel knowledge is available at the transmitter. We study, more specifically, the dirty paper channel in which the interference sequence in multiplied by fading value unknown to the transmitter but known at the receiver. For this model, we establish an approximate characterization of capacity for the case in which fading values vary greatly in between channel realizations. In this regime, which we term the strong fading regime, the capacity pre-log factor is equal to the inverse of the number of possible fading realizations.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

On the Capacity Region of the Two-user Interference Channel with a Cognitive Relay

This paper considers a variation of the classical two-user interference channel where the communication of two interfering source-destination pairs is aided by an additional node that has a priori knowledge of the messages to be transmitted, which is referred to as the it cognitive relay. For this Interference Channel with a Cognitive Relay (ICCR) In particular, for the class of injective semi-deterministic ICCRs, a sum-rate upper bound is derived for the general memoryless ICCR and further tightened for the Linear Deterministic Approximation (LDA) of the Gaussian noise channel at high SNR, which disregards the noise and focuses on the interaction among the users' signals. The capacity region of the symmetric LDA is completely characterized except for the regime of moderately weak interference and weak links from the CR to the destinations. The insights gained from the analysis of the LDA are then translated back to the symmetric Gaussian noise channel (GICCR). For the symmetric GICCR, an approximate characterization (to within a constant gap) of the capacity region is provided for a parameter regime where capacity was previously unknown. The approximately optimal scheme suggests that message cognition at a relay is beneficial for interference management as it enables simultaneous over the air neutralization of the interference at both destinations.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

The Impact of Phase Fading on the Dirty Paper Channel

The impact of phase fading on the classical Costa dirty paper coding channel is studied. We consider a variation of this channel model in which the amplitude of the interference sequence is known at the transmitter while its phase is known at the receiver. Although the capacity of this channel has already been established, it is expressed using an auxiliary random variable and as the solution of a maximization problem. To circumvent the difficulty evaluating capacity, we derive alternative inner and outer bounds and show that the two expressions are to within a finite distance. This provide an approximate characterization of the capacity which depends only on the channel parameters. We consider, in particular, two distributions of the phase fading: circular binomial and circular uniform. The first distribution models the scenario in which the transmitter has a minimal uncertainty over the phase of the interference while the second distribution models complete uncertainty. For circular binomial fading, we show that binning with Gaussian signaling still approaches capacity, as in the channel without phase fading. In the case of circular uniform fading, instead, binning with Gaussian signaling is no longer effective and novel interference avoidance strategies are developed to approach capacity.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

Energy Efficient Cooperative Strategies for Relay-Assisted Downlink Cellular Systems, Part I: Theoretical Framework

The impact of cognition on the energy efficiency of a downlink cellular system in which multiple relays assist the transmission of the base station is considered. The problem is motivated by the practical importance of relay-assisted solutions in mobile networks, such as LTE-A, in which cooperation among relays holds the promise of greatly improving the energy efficiency of the system. We study the fundamental tradeoff between the power consumption at the base station and the level of cooperation and cognition at the relay nodes. By distributing the same message to multiple relays, the base station consumes more power but it enables cooperation among the relays, thus making the transmission between relays to destination a multiuser cognitive channel. Cooperation among the relays allows for a reduction of the power used to transmit from the relays to the end users due to interference management and the coherent combining gains. These gain are present even in the case of partial or unidirectional transmitter cooperation, which is the case in cognitive channels such as the cognitive interference channel and the interference channel with a cognitive relay. We therefore address the problem of determining the optimal level of cooperation at the relays which results in the smallest total power consumption when accounting for the power reduction due to cognition. A practical design examples and numerical simulation are presented in a companion paper (part II).

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

A New Random Coding Technique that Generalizes Superposition Coding and Binning

Proving capacity for networks without feedback or cooperation usually involves two fundamental random coding techniques: superposition coding and binning. Although conceptually very different, these two techniques often achieve the same performance, suggesting an underlying similarity. In this correspondence we propose a new random coding technique that generalizes superposition coding and binning and provides new insight on relationship among the two With this new theoretical tool, we derive new achievable regions for three classical information theoretical models: multi-access channel, broadcast channel, the interference channel, and show that, unfortunately, it does not improve over the largest known achievable regions for these cases.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Improving the Entropy Estimate of Neuronal Firings of Modeled Cochlear Nucleus Neurons

In this correspondence information theoretical tools are used to investigate the statistical properties of modeled cochlear nucleus globular bushy cell spike trains. The firing patterns are obtained from a simulation software that generates sample spike trains from any auditory input. Here we analyze for the first time the responses of globular bushy cells to voiced and unvoiced speech sounds. Classical entropy estimates, such as the direct method, are improved upon by considering a time-varying and time-dependent entropy estimate. With this method we investigated the relationship between the predictability of the neuronal response and the frequency content in the auditory signals. The analysis quantifies the temporal precision of the neuronal coding and the memory in the neuronal response.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

On the Capacity of a General Multiple-Access Channel and of a Cognitive Network in the Very Strong Interference Regime

The capacity of the multiple-access channel with any distribution of messages among the transmitting nodes was determined by Han in 1979 and the expression of the capacity region contains a number of rate bounds and that grows exponentially with the number of messages. We derive a more compact expression for the capacity region of this channel in which the number of rate bounds depends on the distribution of the messages at the encoders. Using this expression we prove capacity for a class of general cognitive network that we denote as "very strong interference" regime. In this regime there is no rate loss in having all the receivers decode all the messages and the capacity region reduces to the capacity of the compound multiple-access channel. This result generalizes the "very strong interference" capacity results for the interference channel, the cognitive interference channel, the interference channel with a cognitive relay and many others.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

On the Capacity of the Cognitive Interference Channel with a Common Cognitive Message

In this paper the cognitive interference channel with a common message, a variation of the classical cognitive interference channel in which the cognitive message is decoded at both receivers, is studied. For this channel model new outer and inner bounds are developed as well as new capacity results for both the discrete memoryless and the Gaussian case. The outer bounds are derived using bounding techniques originally developed by Sato for the classical interference channel and Nair and El Gamal for the broadcast channel. A general inner bound is obtained combining rate-splitting, superposition coding and binning. Inner and outer bounds are shown to coincide in the "very strong interference" and the "primary decodes cognitive" regimes. The first regime consists of channels in which there is no loss of optimality in having both receivers decode both messages while in the latter regime interference pre-cancellation at the cognitive receiver achieves capacity. Capacity for the Gaussian channel is shown to within a constant additive gap and a constant multiplicative factor.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Primary Rate-Splitting Achieves Capacity for the Gaussian Cognitive Interference Channel

The cognitive interference channel models cognitive overlay radio systems, where cognitive radios overhear the transmission of neighboring nodes. Capacity for this channel is not known in general. For the Gaussian case capacity is known in three regimes, usually denoted as the "weak interference", "very strong interference" and "primary decodes cognitive". This paper provides a new capacity result, based on rate-splitting of the primary user's message into a public and private part and that generalizes the capacity results in the "very strong interference" and "primary decodes cognitive" regimes. This result indicates that capacity of the cognitive interference channel not only depends on channel conditions but also the level of cooperation with the primary user.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

The Capacity of the Semi-Deterministic Cognitive Interference Channel with a Common Cognitive Message and Approximate Capacity for the Gaussian Case

In this paper the study of the cognitive interference channel with a common message, a variation of the classical cognitive interference channel in which the cognitive message is decoded at both receivers. We derive the capacity for the semideterministic channel in which the output at the cognitive decoder is a deterministic function of the channel inputs. We also show capacity to within a constant gap and a constant factor for the Gaussian channel in which the outputs are linear combinations of the channel inputs plus an additive Gaussian noise term. Most of these results are shown using an interesting transmission scheme in which the cognitive message, decoded at both receivers, is also pre-coded against the interference experienced at the cognitive decoder. The pre-coding of the cognitive message does not allow the primary decoder to reconstruct the interfering signal. The cognitive message acts instead as a side information at the primary receiver when decoding its intended message.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

A New Capacity Result for the Z-Gaussian Cognitive Interference Channel

This work proposes a novel outer bound for the Gaussian cognitive interference channel in strong interference at the primary receiver based on the capacity of a multi-antenna broadcast channel with degraded message set. It then shows that for the Z-channel, i.e., when the secondary receiver experiences no interference and the primary receiver experiences strong interference, the proposed outer bound not only is the tightest among known bounds but is actually achievable for sufficiently strong interference. The latter is a novel capacity result that from numerical evaluations appears to be generalizable to a larger (i.e., non-Z) class of Gaussian channels.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

On the Capacity of the Interference Channel with a Cognitive Relay

The InterFerence Channel with a Cognitive Relay (IFC-CR) consists of the classical interference channel with two independent source-destination pairs whose communication is aided by an additional node, referred to as the cognitive relay, that has a priori knowledge of both sources' messages. This a priori message knowledge is termed cognition and idealizes the relay learning the messages of the two sources from their transmissions over a wireless channel. This paper presents new inner and outer bounds for the capacity region of the general memoryless IFC-CR that are shown to be tight for a certain class of channels. The new outer bound follows from arguments originally devised for broadcast channels among which Sato's observation that the capacity region of channels with non-cooperative receivers only depends on the channel output conditional marginal distributions. The new inner bound is shown to include all previously proposed coding schemes and it is thus the largest known achievable rate region to date. The new inner and outer bounds coincide for a subset of channel satisfying a strong interference condition. For these channels there is no loss in optimality if both destinations decode both messages. This result parallels analogous results for the classical IFC and for the cognitive IFC and is the first known capacity result for the general IFC-CR. Numerical evaluations of the proposed inner and outer bounds are presented for the Gaussian noise case.

0 citations0 reviewsNo code linkedOpen access
preprint2010arXiv

Inner and Outer Bounds for the Gaussian Cognitive Interference Channel and New Capacity Results

The capacity of the Gaussian cognitive interference channel, a variation of the classical two-user interference channel where one of the transmitters (referred to as cognitive) has knowledge of both messages, is known in several parameter regimes but remains unknown in general. In this paper we provide a comparative overview of this channel model as we proceed through our contributions: we present a new outer bound based on the idea of a broadcast channel with degraded message sets, and another series of outer bounds obtained by transforming the cognitive channel into channels with known capacity. We specialize the largest known inner bound derived for the discrete memoryless channel to the Gaussian noise channel and present several simplified schemes evaluated for Gaussian inputs in closed form which we use to prove a number of results. These include a new set of capacity results for the a) "primary decodes cognitive" regime, a subset of the "strong interference" regime that is not included in the "very strong interference" regime for which capacity was known, and for the b) "S-channel" in which the primary transmitter does not interfere with the cognitive receiver. Next, for a general Gaussian cognitive interference channel, we determine the capacity to within one bit/s/Hz and to within a factor two regardless of channel parameters, thus establishing rate performance guarantees at high and low SNR, respectively. We also show how different simplified transmission schemes achieve a constant gap between inner and outer bound for specific channels. Finally, we numerically evaluate and compare the various simplified achievable rate regions and outer bounds in parameter regimes where capacity is unknown, leading to further insight on the capacity region of the Gaussian cognitive interference channel.

0 citations0 reviewsNo code linkedOpen access
preprint2010arXiv

New inner and outer bounds for the discrete memoryless cognitive interference channel and some capacity results

The cognitive interference channel is an interference channel in which one transmitter is non-causally provided with the message of the other transmitter. This channel model has been extensively studied in the past years and capacity results for certain classes of channels have been proved. In this paper we present new inner and outer bounds for the capacity region of the cognitive interference channel as well as new capacity results. Previously proposed outer bounds are expressed in terms of auxiliary random variables for which no cardinality constraint is known. Consequently it is not possible to evaluate such outer bounds explicitly for a given channel model. The outer bound we derive is based on an idea originally devised by Sato for the broadcast channel and does not contain auxiliary random variables, allowing it to be more easily evaluated. The inner bound we derive is the largest known to date and is explicitly shown to include all previously proposed achievable rate regions. This comparison highlights which features of the transmission scheme - which includes rate-splitting, superposition coding, a broadcast channel-like binning scheme, and Gel'fand Pinsker coding - are most effective in approaching capacity. We next present new capacity results for a class of discrete memoryless channels that we term the "better cognitive decoding regime" which includes all previously known regimes in which capacity results have been derived as special cases. Finally, we determine the capacity region of the semi-deterministic cognitive interference channel, in which the signal at the cognitive receiver is a deterministic function of the channel inputs.

0 citations0 reviewsNo code linkedOpen access
preprint2010arXiv

New Results on the Capacity of the Gaussian Cognitive Interference Channel

The capacity of the two-user Gaussian cognitive interference channel, a variation of the classical interference channel where one of the transmitters has knowledge of both messages, is known in several parameter regimes but remains unknown in general. In this paper, we consider the following achievable scheme: the cognitive transmitter pre-codes its message against the interference created at its intended receiver by the primary user, and the cognitive receiver only decodes its intended message, similar to the optimal scheme for "weak interference"; the primary decoder decodes both messages, similar to the optimal scheme for "very strong interference". Although the cognitive message is pre-coded against the primary message, by decoding it, the primary receiver obtains information about its own message, thereby improving its rate. We show: (1) that this proposed scheme achieves capacity in what we term the "primary decodes cognitive" regime, i.e., a subset of the "strong interference" regime that is not included in the "very strong interference" regime for which capacity was known; (2) that this scheme is within one bit/s/Hz, or a factor two, of capacity for a much larger set of parameters, thus improving the best known constant gap result; (3) we provide insights into the trade-off between interference pre-coding at the cognitive encoder and interference decoding at the primary receiver based on the analysis of the approximate capacity results.

0 citations0 reviewsNo code linkedOpen access
preprint2010arXiv

Outer Bounds for the Interference Channel with a Cognitive Relay

In this paper, we first present an outer bound for a general interference channel with a cognitive relay, i.e., a relay that has non-causal knowledge of both independent messages transmitted in the interference channel. This outer bound reduces to the capacity region of the deterministic broadcast channel and of the deterministic cognitive interference channel through nulling of certain channel inputs. It does not, however, reduce to that of certain deterministic interference channels for which capacity is known. As such, we subsequently tighten the bound for channels whose outputs satisfy an "invertibility" condition. This second outer bound now reduces to the capacity of this special class of deterministic interference channels. The second outer bound is further tightened for the high SNR deterministic approximation of the Gaussian interference channel with a cognitive relay by exploiting the special structure of the interference. We provide an example that suggests that this third bound is tight in at least some parameter regimes for the high SNR deterministic approximation of the Gaussian channel. Another example shows that the third bound is capacity in the special case where there are no direct links between the non-cognitive transmitters.

0 citations0 reviewsNo code linkedOpen access
preprint2010arXiv

State of the cognitive interference channel: a new unified inner bound

The capacity region of the interference channel in which one transmitter non-causally knows the message of the other, termed the cognitive interference channel, has remained open since its inception in 2005. A number of subtly differing achievable rate regions and outer bounds have been derived, some of which are tight under specific conditions. In this work we present a new unified inner bound for the discrete memoryless cognitive interference channel. We show explicitly how it encompasses all known discrete memoryless achievable rate regions as special cases. The presented achievable region was recently used in deriving the capacity region of the general deterministic cognitive interference channel, and thus also the linear high-SNR deterministic approximation of the Gaussian cognitive interference channel. The high-SNR deterministic approximation was then used to obtain the capacity of the Gaussian cognitive interference channel to within 1.87 bits.

0 citations0 reviewsNo code linkedOpen access
preprint2010arXiv

The Capacity of the Semi-Deterministic Cognitive Interference Channel and its Application to Constant Gap Results for the Gaussian Channel

The cognitive interference channel (C-IFC) consists of a classical two-user interference channel in which the message of one user (the "primary" user) is non-causally available at the transmitter of the other user (the "cognitive" user). We obtain the capacity of the semi-deterministic C-IFC: a discrete memoryless C-IFC in which the cognitive receiver output is a noise-less deterministic function of the channel inputs. We then use the insights obtained from the capacity-achieving scheme for the semi-deterministic model to derive new, unified and tighter constant gap results for the complex-valued Gaussian C-IFC. We prove: (1) a constant additive gap (difference between inner and outer bounds) of half a bit/sec/Hz per real dimension, of relevance at high SNRs, and (b) a constant multiplicative gap (ratio between outer and inner bounds) of a factor two, of relevance at low SNRs

0 citations0 reviewsNo code linkedOpen access