BZPEERPapers, people, evidence and research paths
Menu

Discover

SearchFind papers, people, topics, institutions and opportunities from one place.GraphSee how papers, authors, topics and evidence connect.

Research tools

HomeContinue from the papers, people and topics that matter now.CollectionsTurn saved papers into reading lists, evidence maps and shareable context.ResearchReconstruct the path that led you from search to paper to question.CollaborationMove from a paper, topic or expert request into a focused thread.PublishCreate full-text research articles with metadata, formulas and versions.

Network

ExpertsFind specialists whose work and reviews explain why they can help.CommunitiesJoin research groups organized around papers, topics and shared evidence.

Opportunities

ListingsBrowse roles, calls and collaborations connected to real research context.

Account

Log inReturn to your saved papers, graph views and active threads.Create accountStart saving papers, building a research trail and joining teams.
Log inCreate account

Source author record

Ahmad Beirami

Ahmad Beirami appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record
Information Theorymath.ITMachine LearningArtificial Intelligencemath.DSComputation and LanguageMultiagent SystemsNetworking and Internet ArchitectureCryptography and SecurityDistributed, Parallel, and Cluster ComputingHuman-Computer InteractionMolecular NetworksNeural and Evolutionary Computingnlin.AOnlin.CDphysics.opticsSocial and Information Networks

Catalog footprint

What is connected

29works
17topics
4close collaborators

Actions

Connect this record

Log in to claim
Open graphBrowse works

Research graph

See the researcher in context

Open full explorer

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

29 published item(s)

preprint2023arXiv

A Stochastic Optimization Framework for Fair Risk Minimization

Despite the success of large-scale empirical risk minimization (ERM) at achieving high accuracy across a variety of machine learning tasks, fair ERM is hindered by the incompatibility of fairness constraints with stochastic optimization. We consider the problem of fair classification with discrete sensitive attributes and potentially large models and data sets, requiring stochastic solvers. Existing in-processing fairness algorithms are either impractical in the large-scale setting because they require large batches of data at each iteration or they are not guaranteed to converge. In this paper, we develop the first stochastic in-processing fairness algorithm with guaranteed convergence. For demographic parity, equalized odds, and equal opportunity notions of fairness, we provide slight variations of our algorithm--called FERMI--and prove that each of these variations converges in stochastic optimization with any batch size. Empirically, we show that FERMI is amenable to stochastic solvers with multiple (non-binary) sensitive attributes and non-binary targets, performing well even with minibatch size as small as one. Extensive experiments show that FERMI achieves the most favorable tradeoffs between fairness violation and test accuracy across all tested setups compared with state-of-the-art baselines for demographic parity, equalized odds, equal opportunity. These benefits are especially significant with small batch sizes and for non-binary classification with large number of sensitive attributes, making FERMI a practical, scalable fairness algorithm. The code for all of the experiments in this paper is available at: https://github.com/optimization-for-data-driven-science/FERMI.

0 citations0 reviewsNo code linkedOpen access
preprint2023arXiv

Federated Learning with Heterogeneous Differential Privacy

Federated learning (FL) takes a first step towards privacy-preserving machine learning by training models while keeping client data local. Models trained using FL may still leak private client information through model updates during training. Differential privacy (DP) may be employed on model updates to provide privacy guarantees within FL, typically at the cost of degraded performance of the final trained model. Both non-private FL and DP-FL can be solved using variants of the federated averaging (FedAvg) algorithm. In this work, we consider a heterogeneous DP setup where clients require varying degrees of privacy guarantees. First, we analyze the optimal solution to the federated linear regression problem with heterogeneous DP in a Bayesian setup. We find that unlike the non-private setup, where the optimal solution for homogeneous data amounts to a single global solution for all clients learned through FedAvg, the optimal solution for each client in this setup would be a personalized one even for homogeneous data. We also analyze the privacy-utility trade-off for this setup, where we characterize the gain obtained from heterogeneous privacy where some clients opt for less strict privacy guarantees. We propose a new algorithm for FL with heterogeneous DP, named FedHDP, which employs personalization and weighted averaging at the server using the privacy choices of clients, to achieve better performance on clients' local models. Through numerical experiments, we show that FedHDP provides up to $9.27\%$ performance gain compared to the baseline DP-FL for the considered datasets where $5\%$ of clients opt out of DP. Additionally, we show a gap in the average performance of local models between non-private and private clients of up to $3.49\%$, empirically illustrating that the baseline DP-FL might incur a large utility cost when not all clients require the stricter privacy guarantees.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Annotation Inconsistency and Entity Bias in MultiWOZ

MultiWOZ is one of the most popular multi-domain task-oriented dialog datasets, containing 10K+ annotated dialogs covering eight domains. It has been widely accepted as a benchmark for various dialog tasks, e.g., dialog state tracking (DST), natural language generation (NLG), and end-to-end (E2E) dialog modeling. In this work, we identify an overlooked issue with dialog state annotation inconsistencies in the dataset, where a slot type is tagged inconsistently across similar dialogs leading to confusion for DST modeling. We propose an automated correction for this issue, which is present in a whopping 70% of the dialogs. Additionally, we notice that there is significant entity bias in the dataset (e.g., "cambridge" appears in 50% of the destination cities in the train domain). The entity bias can potentially lead to named entity memorization in generative models, which may go unnoticed as the test set suffers from a similar entity bias as well. We release a new test set with all entities replaced with unseen entities. Finally, we benchmark joint goal accuracy (JGA) of the state-of-the-art DST baselines on these modified versions of the data. Our experiments show that the annotation inconsistency corrections lead to 7-10% improvement in JGA. On the other hand, we observe a 29% drop in JGA when models are evaluated on the new test set with unseen entities.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Robust Conversational Agents against Imperceptible Toxicity Triggers

Warning: this paper contains content that maybe offensive or upsetting. Recent research in Natural Language Processing (NLP) has advanced the development of various toxicity detection models with the intention of identifying and mitigating toxic language from existing systems. Despite the abundance of research in this area, less attention has been given to adversarial attacks that force the system to generate toxic language and the defense against them. Existing work to generate such attacks is either based on human-generated attacks which is costly and not scalable or, in case of automatic attacks, the attack vector does not conform to human-like language, which can be detected using a language model loss. In this work, we propose attacks against conversational agents that are imperceptible, i.e., they fit the conversation in terms of coherency, relevancy, and fluency, while they are effective and scalable, i.e., they can automatically trigger the system into generating toxic language. We then propose a defense mechanism against such attacks which not only mitigates the attack but also attempts to maintain the conversational flow. Through automatic and human evaluations, we show that our defense is effective at avoiding toxic language generation even against imperceptible toxicity triggers while the generated language fits the conversation in terms of coherency and relevancy. Lastly, we establish the generalizability of such a defense mechanism on language generation models beyond conversational agents.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Centralized vs Decentralized Targeted Brute-Force Attacks: Guessing with Side-Information

According to recent empirical studies, a majority of users have the same, or very similar, passwords across multiple password-secured online services. This practice can have disastrous consequences, as one password being compromised puts all the other accounts at much higher risk. Generally, an adversary may use any side-information he/she possesses about the user, be it demographic information, password reuse on a previously compromised account, or any other relevant information to devise a better brute-force strategy (so called targeted attack). In this work, we consider a distributed brute-force attack scenario in which $m$ adversaries, each observing some side information, attempt breaching a password secured system. We compare two strategies: an uncoordinated attack in which the adversaries query the system based on their own side-information until they find the correct password, and a fully coordinated attack in which the adversaries pool their side-information and query the system together. For passwords $\mathbf{X}$ of length $n$, generated independently and identically from a distribution $P_X$, we establish an asymptotic closed-form expression for the uncoordinated and coordinated strategies when the side-information $\mathbf{Y}_{(m)}$ are generated independently from passing $\mathbf{X}$ through a memoryless channel $P_{Y|X}$, as the length of the password $n$ goes to infinity. We illustrate our results for binary symmetric channels and binary erasure channels, two families of side-information channels which model password reuse. We demonstrate that two coordinated agents perform asymptotically better than any finite number of uncoordinated agents for these channels, meaning that sharing side-information is very valuable in distributed attacks.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Competitive Balance in Team Sports Games

Competition is a primary driver of player satisfaction and engagement in multiplayer online games. Traditional matchmaking systems aim at creating matches involving teams of similar aggregated individual skill levels, such as Elo score or TrueSkill. However, team dynamics cannot be solely captured using such linear predictors. Recently, it has been shown that nonlinear predictors that target to learn probability of winning as a function of player and team features significantly outperforms these linear skill-based methods. In this paper, we show that using final score difference provides yet a better prediction metric for competitive balance. We also show that a linear model trained on a carefully selected set of team and individual features achieves almost the performance of the more powerful neural network model while offering two orders of magnitude inference speed improvement. This shows significant promise for implementation in online matchmaking systems.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Fair Resource Allocation in Federated Learning

Federated learning involves training statistical models in massive, heterogeneous networks. Naively minimizing an aggregate loss function in such a network may disproportionately advantage or disadvantage some of the devices. In this work, we propose q-Fair Federated Learning (q-FFL), a novel optimization objective inspired by fair resource allocation in wireless networks that encourages a more fair (specifically, a more uniform) accuracy distribution across devices in federated networks. To solve q-FFL, we devise a communication-efficient method, q-FedAvg, that is suited to federated networks. We validate both the effectiveness of q-FFL and the efficiency of q-FedAvg on a suite of federated datasets with both convex and non-convex models, and show that q-FFL (along with q-FedAvg) outperforms existing baselines in terms of the resulting fairness, flexibility, and efficiency.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Winning Isn't Everything: Enhancing Game Development with Intelligent Agents

Recently, there have been several high-profile achievements of agents learning to play games against humans and beat them. In this paper, we study the problem of training intelligent agents in service of game development. Unlike the agents built to "beat the game", our agents aim to produce human-like behavior to help with game evaluation and balancing. We discuss two fundamental metrics based on which we measure the human-likeness of agents, namely skill and style, which are multi-faceted concepts with practical implications outlined in this paper. We report four case studies in which the style and skill requirements inform the choice of algorithms and metrics used to train agents; ranging from A* search to state-of-the-art deep reinforcement learning. We, further, show that the learning potential of state-of-the-art deep RL models does not seamlessly transfer from the benchmark environments to target ones without heavily tuning their hyperparameters, leading to linear scaling of the engineering efforts and computational cost with the number of target domains.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Optimal Trade-offs in Multi-Processor Approximate Message Passing

We consider large-scale linear inverse problems in Bayesian settings. We follow a recent line of work that applies the approximate message passing (AMP) framework to multi-processor (MP) computational systems, where each processor node stores and processes a subset of rows of the measurement matrix along with corresponding measurements. In each MP-AMP iteration, nodes of the MP system and its fusion center exchange lossily compressed messages pertaining to their estimates of the input. In this setup, we derive the optimal per-iteration coding rates using dynamic programming. We analyze the excess mean squared error (EMSE) beyond the minimum mean squared error (MMSE), and prove that, in the limit of low EMSE, the optimal coding rates increase approximately linearly per iteration. Additionally, we obtain that the combined cost of computation and communication scales with the desired estimation quality according to $O(\log^2(1/\text{EMSE}))$. Finally, we study trade-offs between the physical costs of the estimation process including computation time, communication loads, and the estimation quality as a multi-objective optimization problem, and characterize the properties of the Pareto optimal surfaces.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Performance Trade-Offs in Multi-Processor Approximate Message Passing

We consider large-scale linear inverse problems in Bayesian settings. Our general approach follows a recent line of work that applies the approximate message passing (AMP) framework in multi-processor (MP) computational systems by storing and processing a subset of rows of the measurement matrix along with corresponding measurements at each MP node. In each MP-AMP iteration, nodes of the MP system and its fusion center exchange lossily compressed messages pertaining to their estimates of the input. There is a trade-off between the physical costs of the reconstruction process including computation time, communication loads, and the reconstruction quality, and it is impossible to simultaneously minimize all the costs. We pose this minimization as a multi-objective optimization problem (MOP), and study the properties of the best trade-offs (Pareto optimality) in this MOP. We prove that the achievable region of this MOP is convex, and conjecture how the combined cost of computation and communication scales with the desired mean squared error. These properties are verified numerically.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Mismatched Estimation in Large Linear Systems

We study the excess mean square error (EMSE) above the minimum mean square error (MMSE) in large linear systems where the posterior mean estimator (PME) is evaluated with a postulated prior that differs from the true prior of the input signal. We focus on large linear systems where the measurements are acquired via an independent and identically distributed random matrix, and are corrupted by additive white Gaussian noise (AWGN). The relationship between the EMSE in large linear systems and EMSE in scalar channels is derived, and closed form approximations are provided. Our analysis is based on the decoupling principle, which links scalar channels to large linear system analyses. Numerical examples demonstrate that our closed form approximations are accurate.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Packet-Level Network Compression: Realization and Scaling of the Network-Wide Benefits

The existence of considerable amount of redundancy in the Internet traffic at the packet level has stimulated the deployment of packet-level redundancy elimination techniques within the network by enabling network nodes to memorize data packets. Redundancy elimination results in traffic reduction which in turn improves the efficiency of network links. In this paper, the concept of network compression is introduced that aspires to exploit the statistical correlation beyond removing large duplicate strings from the flow to better suppress redundancy. In the first part of the paper, we introduce "memory-assisted compression", which utilizes the memorized content within the network to learn the statistics of the information source generating the packets which can then be used toward reducing the length of codewords describing the packets emitted by the source. Using simulations on data gathered from real network traces, we show that memory-assisted compression can result in significant traffic reduction. In the second part of the paper, we study the scaling of the average network-wide benefits of memory-assisted compression. We discuss routing and memory placement problems in network for the reduction of overall traffic. We derive a closed-form expression for the scaling of the gain in Erdos-Renyi random network graphs, where obtain a threshold value for the number of memories deployed in a random graph beyond which network-wide benefits start to shine. Finally, the network-wide benefits are studied on Internet-like scale-free networks. We show that non-vanishing network compression gain is obtained even when only a tiny fraction of the total number of nodes in the network are memory-enabled.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Universal Compression of a Mixture of Parametric Sources with Side Information

This paper investigates the benefits of the side information on the universal compression of sequences from a mixture of $K$ parametric sources. The output sequence of the mixture source is chosen from the source $i \in \{1,\ldots ,K\}$ with a $d_i$-dimensional parameter vector at random according to probability vector $\mathbf{w} = (w_1,\ldots,w_K)$. The average minimax redundancy of the universal compression of a new random sequence of length $n$ is derived when the encoder and the decoder have a common side information of $T$ sequences generated independently by the mixture source. Necessary and sufficient conditions on the distribution $\mathbf{w}$ and the mixture parameter dimensions $\mathbf{d} = (d_1,\ldots,d_K)$ are determined such that the side information provided by the previous sequences results in a reduction in the first-order term of the average codeword length compared with the universal compression without side information. Further, it is proved that the optimal compression with side information corresponds to the clustering of the side information sequences from the mixture source. Then, a clustering technique is presented to better utilize the side information by classifying the data sequences from a mixture source. Finally, the performance of the clustering on the universal compression with side information is validated using computer simulations on real network data traces.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

A Framework for Investigating the Performance of Chaotic-Map Truly Random Number Generators

In this paper, we approximate the hidden Markov model of chaotic-map truly random number generators (TRNGs) and describe its fundamental limits based on the approximate entropy-rate of the underlying bit-generation process. We demonstrate that entropy-rate plays a key role in the performance and robustness of chaotic-map TRNGs, which must be taken into account in the circuit design optimization. We further derive optimality conditions for post-processing units that extract truly random bits from a raw-RNG.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

Theoretical Analysis of the Characteristic Impedance in Metal-Insulator-Metal Plasmonic Transmission Lines

We propose a closed form formulation for the impedance of the Metal-Insulator-Metal (MIM) plasmonic transmission lines by solving the Maxwell's equations. We provide approximations for thin and thick insulator layers sandwiched between metallic layers. In the case of very thin dielectric layer, the surface waves on both interfaces are strongly coupled resulting in an almost linear dependence of the impedance of the plasmonic transmission line on the thickness of the insulator layer. On the other hand, for very thick insulator layer, the impedance does not vary with the insulator layer thickness due to the weak-coupling/decoupling of the surface waves on each metal-insulator interface. We demonstrate the effectiveness of our proposed formulation using two test scenarios, namely, almost zero reflection in Tee-junction and reflection from line discontinuity in the design of Bragg reflectors, where we compare our formulation against previously published results.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

A Performance Metric for Discrete-Time Chaos-Based Truly Random Number Generators

In this paper, we develop an information entropy based metric that represents the statistical quality of the generated binary sequence in Truly Random Number Generators (TRNG). The metric can be used for the design and optimization of the TRNG circuits as well as the development of efficient post-processing units for recovering the degraded statistical characteristics of the signal due to process variations.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

A Realizable Modified Tent Map for True Random Number Generation

Tent map is a discrete-time piecewise-affine I/O characteristic curve, which is used for chaos-based applications, such as true random number generation. However, tent map suffers from the inability to maintain the output state confined to the input range under noise and process variations. In this paper, we propose a modified tent map, which is interchangeable with the tent map for practical applications. In the proposed modified tent map, the confinement problem is solved while maintaining the functionality of the tent map. We also demonstrate simulation results for the circuit implementation of the presented modified tent map for true random number generation.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Data Gathering in Networks of Bacteria Colonies: Collective Sensing and Relaying Using Molecular Communication

The prospect of new biological and industrial applications that require communication in micro-scale, encourages research on the design of bio-compatible communication networks using networking primitives already available in nature. One of the most promising candidates for constructing such networks is to adapt and engineer specific types of bacteria that are capable of sensing, actuation, and above all, communication with each other. In this paper, we describe a new architecture for networks of bacteria to form a data collecting network, as in traditional sensor networks. The key to this architecture is the fact that the node in the network itself is a bacterial colony; as an individual bacterium (biological agent) is a tiny unreliable element with limited capabilities. We describe such a network under two different scenarios. We study the data gathering (sensing and multihop communication) scenario as in sensor networks followed by the consensus problem in a multi-node network. We will explain as to how the bacteria in the colony collectively orchestrate their actions as a node to perform sensing and relaying tasks that would not be possible (at least reliably) by an individual bacterium. Each single bacterium in the colony forms a belief by sensing external parameter (e.g., a molecular signal from another node) from the medium and shares its belief with other bacteria in the colony. Then, after some interactions, all the bacteria in the colony form a common belief and act as a single node. We will model the reception process of each individual bacteria and will study its impact on the overall functionality of a node. We will present results on the reliability of the multihop communication for data gathering scenario as well as the speed of convergence in the consensus scenario.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Discrete-Time Chaotic-Map Truly Random Number Generators: Design, Implementation, and Variability Analysis of the Zigzag Map

In this paper, we introduce a novel discrete chaotic map named zigzag map that demonstrates excellent chaotic behaviors and can be utilized in Truly Random Number Generators (TRNGs). We comprehensively investigate the map and explore its critical chaotic characteristics and parameters. We further present two circuit implementations for the zigzag map based on the switched current technique as well as the current-mode affine interpolation of the breakpoints. In practice, implementation variations can deteriorate the quality of the output sequence as a result of variation of the chaotic map parameters. In order to quantify the impact of variations on the map performance, we model the variations using a combination of theoretical analysis and Monte-Carlo simulations on the circuits. We demonstrate that even in the presence of the map variations, a TRNG based on the zigzag map passes all of the NIST 800-22 statistical randomness tests using simple post processing of the output data.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Memory-Assisted Universal Compression of Network Flows

Recently, the existence of considerable amount of redundancy in the Internet traffic has stimulated the deployment of several redundancy elimination techniques within the network. These techniques are often based on either packet-level Redundancy Elimination (RE) or Content-Centric Networking (CCN). However, these techniques cannot exploit sub-packet redundancies. Further, other alternative techniques such as the end-to-end universal compression solutions would not perform well either over the Internet traffic, as such techniques require infinite length traffic to effectively remove redundancy. This paper proposes a memory-assisted universal compression technique that holds a significant promise for reducing the amount of traffic in the networks. The proposed work is based on the observation that if a source is to be compressed and sent over a network, the associated universal code entails a substantial overhead in transmission due to finite length traffic. However, intermediate nodes can learn the source statistics and this can be used to reduce the cost of describing the source statistics, reducing the transmission overhead for such traffics. We present two algorithms (statistical and dictionary-based) for the memory-assisted universal lossless compression of information sources. These schemes are universal in the sense that they do not require any prior knowledge of the traffic's statistical distribution. We demonstrate the effectiveness of both algorithms and characterize the memorization gain using the real Internet traces. Furthermore, we apply these compression schemes to Internet-like power-law graphs and solve the routing problem for compressed flows.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Memory-Assisted Universal Source Coding

The problem of the universal compression of a sequence from a library of several small to moderate length sequences from similar context arises in many practical scenarios, such as the compression of the storage data and the Internet traffic. In such scenarios, it is often required to compress and decompress every sequence individually. However, the universal compression of the individual sequences suffers from significant redundancy overhead. In this paper, we aim at answering whether or not having a memory unit in the middle can result in a fundamental gain in the universal compression. We present the problem setup in the most basic scenario consisting of a server node $S$, a relay node $R$ (i.e., the memory unit), and a client node $C$. We assume that server $S$ wishes to send the sequence $x^n$ to the client $C$ who has never had any prior communication with the server, and hence, is not capable of memorization of the source context. However, $R$ has previously communicated with $S$ to forward previous sequences from $S$ to the clients other than $C$, and thus, $R$ has memorized a context $y^m$ shared with $S$. Note that if the relay node was absent the source could possibly apply universal compression to $x^n$ and transmit to $C$ whereas the presence of memorized context at $R$ can possibly reduce the communication overhead in $S$-$R$ link. In this paper, we investigate the fundamental gain of the context memorization in the memory-assisted universal compression of the sequence $x^n$ over conventional universal source coding by providing a lower bound on the gain of memory-assisted source coding.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Network Compression: Memory-Assisted Universal Coding of Sources with Correlated Parameters

In this paper, we propose {\em distributed network compression via memory}. We consider two spatially separated sources with correlated unknown source parameters. We wish to study the universal compression of a sequence of length $n$ from one of the sources provided that the decoder has access to (i.e., memorized) a sequence of length $m$ from the other source. In this setup, the correlation does not arise from symbol-by-symbol dependency of two outputs from the two sources (as in Slepian-Wolf setup). Instead, the two sequences are correlated because they are originated from the two sources with \emph{unknown} correlated parameters. The finite-length nature of the compression problem at hand requires considering a notion of almost lossless source coding, where coding incurs an error probability $p_e(n)$ that vanishes as sequence length $n$ grows to infinity. We obtain bounds on the redundancy of almost lossless codes when the decoder has access to a random memory of length $m$ as a function of the sequence length $n$ and the permissible error probability $p_e(n)$. Our results demonstrate that distributed network compression via memory has the potential to significantly improve over conventional end-to-end compression when sufficiently large memory from previous communications is available to the decoder.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

On Lossless Universal Compression of Distributed Identical Sources

Slepian-Wolf theorem is a well-known framework that targets almost lossless compression of (two) data streams with symbol-by-symbol correlation between the outputs of (two) distributed sources. However, this paper considers a different scenario which does not fit in the Slepian-Wolf framework. We consider two identical but spatially separated sources. We wish to study the universal compression of a sequence of length $n$ from one of the sources provided that the decoder has access to (i.e., memorized) a sequence of length $m$ from the other source. Such a scenario occurs, for example, in the universal compression of data from multiple mirrors of the same server. In this setup, the correlation does not arise from symbol-by-symbol dependency of two outputs from the two sources. Instead, the sequences are correlated through the information that they contain about the unknown source parameter. We show that the finite-length nature of the compression problem at hand requires considering a notion of almost lossless source coding, where coding incurs an error probability $p_e(n)$ that vanishes with sequence length $n$. We obtain a lower bound on the average minimax redundancy of almost lossless codes as a function of the sequence length $n$ and the permissible error probability $p_e$ when the decoder has a memory of length $m$ and the encoders do not communicate. Our results demonstrate that a strict performance loss is incurred when the two encoders do not communicate even when the decoder knows the unknown parameter vector (i.e., $m \to \infty$).

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Results on the Fundamental Gain of Memory-Assisted Universal Source Coding

Many applications require data processing to be performed on individual pieces of data which are of finite sizes, e.g., files in cloud storage units and packets in data networks. However, traditional universal compression solutions would not perform well over the finite-length sequences. Recently, we proposed a framework called memory-assisted universal compression that holds a significant promise for reducing the amount of redundant data from the finite-length sequences. The proposed compression scheme is based on the observation that it is possible to learn source statistics (by memorizing previous sequences from the source) at some intermediate entities and then leverage the memorized context to reduce redundancy of the universal compression of finite-length sequences. We first present the fundamental gain of the proposed memory-assisted universal source coding over conventional universal compression (without memorization) for a single parametric source. Then, we extend and investigate the benefits of the memory-assisted universal source coding when the data sequences are generated by a compound source which is a mixture of parametric sources. We further develop a clustering technique within the memory-assisted compression framework to better utilize the memory by classifying the observed data sequences from a mixture of parametric sources. Finally, we demonstrate through computer simulations that the proposed joint memorization and clustering technique can achieve up to 6-fold improvement over the traditional universal compression technique when a mixture of non-binary Markov sources is considered.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Capacity of Discrete Molecular Diffusion Channels

In diffusion-based molecular communications, messages can be conveyed via the variation in the concentration of molecules in the medium. In this paper, we intend to analyze the achievable capacity in transmission of information from one node to another in a diffusion channel. We observe that because of the molecular diffusion in the medium, the channel possesses memory. We then model the memory of the channel by a two-step Markov chain and obtain the equations describing the capacity of the diffusion channel. By performing a numerical analysis, we obtain the maximum achievable rate for different levels of the transmitter power, i.e., the molecule production rate.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Consensus Problem under Diffusion-based Molecular Communication

We investigate the consensus problem in a network where nodes communicate via diffusion-based molecular communication (DbMC). In DbMC, messages are conveyed via the variation in the concentration of molecules in the medium. Every node acquires sensory information about the environment. Communication enables the nodes to reach the best estimate for that measurement, e.g., the average of the initial estimates by all nodes. We consider an iterative method for communication among nodes that enables information spreading and averaging in the network. We show that the consensus can be attained after a finite number of iterations and variance of estimates of nodes can be made arbitrarily small via communication.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Exact Modeling of the Performance of Random Linear Network Coding in Finite-buffer Networks

In this paper, we present an exact model for the analysis of the performance of Random Linear Network Coding (RLNC) in wired erasure networks with finite buffers. In such networks, packets are delayed due to either random link erasures or blocking by full buffers. We assert that because of RLNC, the content of buffers have dependencies which cannot be captured directly using the classical queueing theoretical models. We model the performance of the network using Markov chains by a careful derivation of the buffer occupancy states and their transition rules. We verify by simulations that the proposed framework results in an accurate measure of the network throughput offered by RLNC. Further, we introduce a class of acyclic networks for which the number of state variables is significantly reduced.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

On the Network-Wide Gain of Memory-Assisted Source Coding

Several studies have identified a significant amount of redundancy in the network traffic. For example, it is demonstrated that there is a great amount of redundancy within the content of a server over time. This redundancy can be leveraged to reduce the network flow by the deployment of memory units in the network. The question that arises is whether or not the deployment of memory can result in a fundamental improvement in the performance of the network. In this paper, we answer this question affirmatively by first establishing the fundamental gains of memory-assisted source compression and then applying the technique to a network. Specifically, we investigate the gain of memory-assisted compression in random network graphs consisted of a single source and several randomly selected memory units. We find a threshold value for the number of memories deployed in a random graph and show that if the number of memories exceeds the threshold we observe network-wide reduction in the traffic.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Results on the Redundancy of Universal Compression for Finite-Length Sequences

In this paper, we investigate the redundancy of universal coding schemes on smooth parametric sources in the finite-length regime. We derive an upper bound on the probability of the event that a sequence of length $n$, chosen using Jeffreys' prior from the family of parametric sources with $d$ unknown parameters, is compressed with a redundancy smaller than $(1-ε)\frac{d}{2}\log n$ for any $ε>0$. Our results also confirm that for large enough $n$ and $d$, the average minimax redundancy provides a good estimate for the redundancy of most sources. Our result may be used to evaluate the performance of universal source coding schemes on finite-length sequences. Additionally, we precisely characterize the minimax redundancy for two--stage codes. We demonstrate that the two--stage assumption incurs a negligible redundancy especially when the number of source parameters is large. Finally, we show that the redundancy is significant in the compression of small sequences.

0 citations0 reviewsNo code linkedOpen access