BZPEERPapers, people, evidence and research paths
Menu

Discover

SearchFind papers, people, topics, institutions and opportunities from one place.GraphSee how papers, authors, topics and evidence connect.

Research tools

HomeContinue from the papers, people and topics that matter now.CollectionsTurn saved papers into reading lists, evidence maps and shareable context.ResearchReconstruct the path that led you from search to paper to question.CollaborationMove from a paper, topic or expert request into a focused thread.PublishCreate full-text research articles with metadata, formulas and versions.

Network

ExpertsFind specialists whose work and reviews explain why they can help.CommunitiesJoin research groups organized around papers, topics and shared evidence.

Opportunities

ListingsBrowse roles, calls and collaborations connected to real research context.

Account

Log inReturn to your saved papers, graph views and active threads.Create accountStart saving papers, building a research trail and joining teams.
Log inCreate account

Source author record

David Tse

David Tse appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record
Information Theorymath.ITCryptography and SecurityDistributed, Parallel, and Cluster ComputingGenomicsSystems and Controlmath.OCMachine LearningQuantitative MethodsApplicationsComputational Engineering, Finance, and ScienceComputer Science and Game TheoryData Structures and Algorithmsmath.STMethodologyNetworking and Internet ArchitecturePopulations and EvolutionSocial and Information NetworksStatistics Theory

Catalog footprint

What is connected

51works
19topics
4close collaborators

Actions

Connect this record

Log in to claim
Open graphBrowse works

Research graph

See the researcher in context

Open full explorer

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

51 published item(s)

preprint2023arXiv

Goldfish: No More Attacks on Ethereum?!

The LMD GHOST consensus protocol is a critical component of proof-of-stake Ethereum. In its current form, this protocol is brittle, as evidenced by recent attacks and patching attempts. We propose Goldfish, a new protocol that satisfies key properties required of a drop-in replacement for LMD GHOST: Goldfish is secure in the sleepy model, assuming a majority of the validators follows the protocol. Goldfish is reorg resilient so that honestly produced blocks are guaranteed inclusion in the ledger, and it supports fast confirmation with expected confirmation latency independent of the desired security level. Subsampling validators can improve the communication efficiency of Goldfish, and Goldfish is composable with finality/accountability gadgets. Crucially, Goldfish is structurally similar to LMD GHOST, providing a credible path to adoption in Ethereum. Attacks on LMD GHOST exploit lack of coordination among honest validators, typically provided by a locking mechanism in classical BFT protocols. However, locking requires votes from a quorum of all participants and is not compatible with fluctuating participation. Goldfish is powered by a novel coordination mechanism to synchronize the honest validators' actions. Experiments with our prototype implementation of Goldfish suggest practicality.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Approximate Function Evaluation via Multi-Armed Bandits

We study the problem of estimating the value of a known smooth function $f$ at an unknown point $\boldsymbolμ \in \mathbb{R}^n$, where each component $μ_i$ can be sampled via a noisy oracle. Sampling more frequently components of $\boldsymbolμ$ corresponding to directions of the function with larger directional derivatives is more sample-efficient. However, as $\boldsymbolμ$ is unknown, the optimal sampling frequencies are also unknown. We design an instance-adaptive algorithm that learns to sample according to the importance of each coordinate, and with probability at least $1-δ$ returns an $ε$ accurate estimate of $f(\boldsymbolμ)$. We generalize our algorithm to adapt to heteroskedastic noise, and prove asymptotic optimality when $f$ is linear. We corroborate our theoretical results with numerical experiments, showing the dramatic gains afforded by adaptivity.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Babylon: Reusing Bitcoin Mining to Enhance Proof-of-Stake Security

Bitcoin is the most secure blockchain in the world, supported by the immense hash power of its Proof-of-Work miners, but consumes huge amount of energy. Proof-of-Stake chains are energy-efficient, have fast finality and accountability, but face several fundamental security issues: susceptibility to non-slashable long-range safety attacks, non-slashable transaction censorship and stalling attacks and difficulty to bootstrap new PoS chains from low token valuation. We propose Babylon, a blockchain platform which combines the best of both worlds by reusing the immense Bitcoin hash power to enhance the security of PoS chains. Babylon provides a data-available timestamping service, securing PoS chains by allowing them to timestamp data-available block checkpoints, fraud proofs and censored transactions on Babylon. Babylon miners merge mine with Bitcoin and thus the platform has zero additional energy cost. The security of a Babylon-enhanced PoS protocol is formalized by a cryptoeconomic security theorem which shows slashable safety and liveness guarantees.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Information Dispersal with Provable Retrievability for Rollups

The ability to verifiably retrieve transaction or state data stored off-chain is crucial to blockchain scaling techniques such as rollups or sharding. We formalize the problem and design a storage- and communication-efficient protocol using linear erasure-correcting codes and homomorphic vector commitments. Motivated by application requirements for rollups, our solution Semi-AVID-PR departs from earlier Verifiable Information Dispersal schemes in that we do not require comprehensive termination properties. Compared to Data Availability Oracles, under no circumstance do we fall back to returning empty blocks. Distributing a file of 22 MB among 256 storage nodes, up to 85 of which may be adversarial, requires in total ~70 MB of communication and storage, and ~41 seconds of single-thread runtime (<3 seconds on 16 threads) on an AMD Opteron 6378 processor when using the BLS12-381 curve. Our solution requires no modification to on-chain contracts of Validium rollups such as StarkWare's StarkEx. Additionally, it provides privacy of the dispersed data against honest-but-curious storage nodes. Finally, we discuss an application of our Semi-AVID-PR scheme to data availability verification schemes based on random sampling.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Longest Chain Consensus Under Bandwidth Constraint

Spamming attacks are a serious concern for consensus protocols, as witnessed by recent outages of a major blockchain, Solana. They cause congestion and excessive message delays in a real network due to its bandwidth constraints. In contrast, longest chain (LC), an important family of consensus protocols, has previously only been proven secure assuming an idealized network model in which all messages are delivered within bounded delay. This model-reality mismatch is further aggravated for Proof-of-Stake (PoS) LC where the adversary can spam the network with equivocating blocks. Hence, we extend the network model to capture bandwidth constraints, under which nodes now need to choose carefully which blocks to spend their limited download budget on. To illustrate this point, we show that 'download along the longest header chain', a natural download rule for Proof-of-Work (PoW) LC, is insecure for PoS LC. We propose a simple rule 'download towards the freshest block', formalize two common heuristics 'not downloading equivocations' and 'blocklisting', and prove in a unified framework that PoS LC with any one of these download rules is secure in bandwidth-constrained networks. In experiments, we validate our claims and showcase the behavior of these download rules under attack. By composing multiple instances of a PoS LC protocol with a suitable download rule in parallel, we obtain a PoS consensus protocol that achieves a constant fraction of the network's throughput limit even under worst-case adversarial strategies.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

The Availability-Accountability Dilemma and its Resolution via Accountability Gadgets

For applications of Byzantine fault tolerant (BFT) consensus protocols where the participants are economic agents, recent works highlighted the importance of accountability: the ability to identify participants who provably violate the protocol. At the same time, being able to reach consensus under dynamic levels of participation is desirable for censorship resistance. We identify an availability-accountability dilemma: in an environment with dynamic participation, no protocol can simultaneously be accountably-safe and live. We provide a resolution to this dilemma by constructing a provably secure optimally-resilient accountability gadget to checkpoint a longest chain protocol, such that the full ledger is live under dynamic participation and the checkpointed prefix ledger is accountable. Our accountability gadget construction is black-box and can use any BFT protocol which is accountable under static participation. Using HotStuff as the black box, we implemented our construction as a protocol for the Ethereum 2.0 beacon chain, and our Internet-scale experiments with more than 4000 nodes show that the protocol achieves the required scalability and has better latency than the current solution Gasper, which was shown insecure by recent attacks.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Two Attacks On Proof-of-Stake GHOST/Ethereum

We present two attacks targeting the Proof-of-Stake (PoS) Ethereum consensus protocol. The first attack suggests a fundamental conceptual incompatibility between PoS and the Greedy Heaviest-Observed Sub-Tree (GHOST) fork choice paradigm employed by PoS Ethereum. In a nutshell, PoS allows an adversary with a vanishing amount of stake to produce an unlimited number of equivocating blocks. While most equivocating blocks will be orphaned, such orphaned `uncle blocks' still influence fork choice under the GHOST paradigm, bestowing upon the adversary devastating control over the canonical chain. While the Latest Message Driven (LMD) aspect of current PoS Ethereum prevents a straightforward application of this attack, our second attack shows how LMD specifically can be exploited to obtain a new variant of the balancing attack that overcomes a recent protocol addition that was intended to mitigate balancing-type attacks. Thus, in its current form, PoS Ethereum without and with LMD is vulnerable to our first and second attack, respectively.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

Ebb-and-Flow Protocols: A Resolution of the Availability-Finality Dilemma

The CAP theorem says that no blockchain can be live under dynamic participation and safe under temporary network partitions. To resolve this availability-finality dilemma, we formulate a new class of flexible consensus protocols, ebb-and-flow protocols, which support a full dynamically available ledger in conjunction with a finalized prefix ledger. The finalized ledger falls behind the full ledger when the network partitions but catches up when the network heals. Gasper, the current candidate protocol for Ethereum 2.0's beacon chain, combines the finality gadget Casper FFG with the LMD GHOST fork choice rule and aims to achieve this property. However, we discovered an attack in the standard synchronous network model, highlighting a general difficulty with existing finality-gadget-based designs. We present a construction of provably secure ebb-and-flow protocols with optimal resilience. Nodes run an off-the-shelf dynamically available protocol, take snapshots of the growing available ledger, and input them into a separate off-the-shelf BFT protocol to finalize a prefix. We explore connections with flexible BFT and improve upon the state-of-the-art for that problem.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

PoSAT: Proof-of-Work Availability and Unpredictability, without the Work

An important feature of Proof-of-Work (PoW) blockchains is full dynamic availability, allowing miners to go online and offline while requiring only 50% of the online miners to be honest. Existing Proof-of-stake (PoS), Proof-of-Space and related protocols are able to achieve this property only partially, either putting the additional assumption that adversary nodes to be online from the beginning and no new adversary nodes come online afterwards, or use additional trust assumptions for newly joining nodes.We propose a new PoS protocol PoSAT which can provably achieve dynamic availability fully without any additional assumptions. The protocol is based on the longest chain and uses a Verifiable Delay Function for the block proposal lottery to provide an arrow of time. The security analysis of the protocol draws on the recently proposed technique of Nakamoto blocks as well as the theory of branching random walks. An additional feature of PoSAT is the complete unpredictability of who will get to propose a block next, even by the winner itself. This unpredictability is at the same level of PoW protocols, and is stronger than that of existing PoS protocols using Verifiable Random Functions.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Boomerang: Redundancy Improves Latency and Throughput in Payment-Channel Networks

In multi-path routing schemes for payment-channel networks, Alice transfers funds to Bob by splitting them into partial payments and routing them along multiple paths. Undisclosed channel balances and mismatched transaction fees cause delays and failures on some payment paths. For atomic transfer schemes, these straggling paths stall the whole transfer. We show that the latency of transfers reduces when redundant payment paths are added. This frees up liquidity in payment channels and hence increases the throughput of the network. We devise Boomerang, a generic technique to be used on top of multi-path routing schemes to construct redundant payment paths free of counterparty risk. In our experiments, applying Boomerang to a baseline routing scheme leads to 40% latency reduction and 2x throughput increase. We build on ideas from publicly verifiable secret sharing, such that Alice learns a secret of Bob iff Bob overdraws funds from the redundant paths. Funds are forwarded using Boomerang contracts, which allow Alice to revert the transfer iff she has learned Bob's secret. We implement the Boomerang contract in Bitcoin Script.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Everything is a Race and Nakamoto Always Wins

Nakamoto invented the longest chain protocol, and claimed its security by analyzing the private double-spend attack, a race between the adversary and the honest nodes to grow a longer chain. But is it the worst attack? We answer the question in the affirmative for three classes of longest chain protocols, designed for different consensus models: 1) Nakamoto's original Proof-of-Work protocol; 2) Ouroboros and SnowWhite Proof-of-Stake protocols; 3) Chia Proof-of-Space protocol. As a consequence, exact characterization of the maximum tolerable adversary power is obtained for each protocol as a function of the average block time normalized by the network delay. The security analysis of these protocols is performed in a unified manner by a novel method of reducing all attacks to a race between the adversary and the honest nodes.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Free2Shard: Adaptive-adversary-resistant sharding via Dynamic Self Allocation

Propelled by the growth of large-scale blockchain deployments, much recent progress has been made in designing sharding protocols that achieve throughput scaling linearly in the number of nodes. However, existing protocols are not robust to an adversary adaptively corrupting a fixed fraction of nodes. In this paper, we propose Free2Shard -- a new architecture that achieves near-linear scaling while being secure against a fully adaptive adversary. The focal point of this architecture is a dynamic self-allocation algorithm that lets users allocate themselves to shards in response to adversarial action, without requiring a central or cryptographic proof. This architecture has several attractive features unusual for sharding protocols, including: (a) the ability to handle the regime of large number of shards (relative to the number of nodes); (b) heterogeneous shard demands; (c) requiring only a small minority to follow the self-allocation; (d) asynchronous shard rotation; (e) operation in a purely identity-free proof-of-work setting. The key technical contribution is a deep mathematical connection to the classical work of Blackwell in dynamic game theory.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Prism Removes Consensus Bottleneck for Smart Contracts

The performance of existing permissionless smart contract platforms such as Ethereum is limited by the consensus layer. Prism is a new proof-of-work consensus protocol that provably achieves throughput and latency up to physical limits while retaining the strong guarantees of the longest chain protocol. This paper reports experimental results from implementations of two smart contract virtual machines, EVM and MoveVM, on top of Prism and demonstrates that the consensus bottleneck has been removed. Code can be found at https://github.com/wgr523/prism-smart-contracts.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Proof-of-Stake Longest Chain Protocols: Security vs Predictability

The Nakamoto longest chain protocol is remarkably simple and has been proven to provide security against any adversary with less than 50% of the total hashing power. Proof-of-stake (PoS) protocols are an energy efficient alternative; however existing protocols adopting Nakamoto's longest chain design achieve provable security only by allowing long-term predictability (which have serious security implications). In this paper, we prove that a natural longest chain PoS protocol with similar predictability as Nakamoto's PoW protocol can achieve security against any adversary with less than 1/(1+e) fraction of the total stake. Moreover we propose a new family of longest chain PoS protocols that achieve security against a 50% adversary, while only requiring short-term predictability. Our proofs present a new approach to analyzing the formal security of blockchains, based on a notion of adversary-proof convergence.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Community Recovery in Graphs with Locality

Motivated by applications in domains such as social networks and computational biology, we study the problem of community recovery in graphs with locality. In this problem, pairwise noisy measurements of whether two nodes are in the same community or different communities come mainly or exclusively from nearby nodes rather than uniformly sampled between all nodes pairs, as in most existing models. We present an algorithm that runs nearly linearly in the number of measurements and which achieves the information theoretic limit for exact recovery.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Novel probabilistic models of spatial genetic ancestry with applications to stratification correction in genome-wide association studies

Genetic variation in human populations is influenced by geographic ancestry due to spatial locality in historical mating and migration patterns. Spatial population structure in genetic datasets has been traditionally analyzed using either model-free algorithms, such as principal components analysis (PCA) and multidimensional scaling, or using explicit spatial probabilistic models of allele frequency evolution. We develop a general probabilistic model and an associated inference algorithm that unify the model-based and data-driven approaches to visualizing and inferring population structure. Our algorithm, Geographic Ancestry Positioning (GAP), relates local genetic distances between samples to their spatial distances, and can be used for visually discerning population structure as well as accurately inferring the spatial origin of individuals on a two-dimensional continuum. On both simulated and several real datasets from diverse human populations, GAP exhibits substantially lower error in reconstructing spatial ancestry coordinates compared to PCA. Our spatial inference algorithm can also be effectively applied to the problem of population stratification in genome-wide association studies (GWAS), where hidden population structure can create fictitious associations when population ancestry is correlated with both the genotype and the trait. We develop an association test that uses the ancestry coordinates inferred by GAP to accurately account for ancestry-induced correlations in GWAS. Based on simulations and analysis of a dataset of 10 metabolic traits measured in a Northern Finland cohort, which is known to exhibit significant population structure, we find that our method has superior power to current approaches.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

An Optimal and Distributed Method for Voltage Regulation in Power Distribution Systems

This paper addresses the problem of voltage regulation in power distribution networks with deep-penetration of distributed energy resources, e.g., renewable-based generation, and storage-capable loads such as plug-in hybrid electric vehicles. We cast the problem as an optimization program, where the objective is to minimize the losses in the network subject to constraints on bus voltage magnitudes, limits on active and reactive power injections, transmission line thermal limits and losses. We provide sufficient conditions under which the optimization problem can be solved via its convex relaxation. Using data from existing networks, we show that these sufficient conditions are expected to be satisfied by most networks. We also provide an efficient distributed algorithm to solve the problem. The algorithm adheres to a communication topology described by a graph that is the same as the graph that describes the electrical network topology. We illustrate the operation of the algorithm, including its robustness against communication link failures, through several case studies involving 5-, 34-, and 123-bus power distribution systems.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Capacity-Achieving Rateless Polar Codes

A rateless coding scheme transmits incrementally more and more coded bits over an unknown channel until all the information bits are decoded reliably by the receiver. We propose a new rateless coding scheme based on polar codes, and we show that this scheme is capacity-achieving, i.e. its information rate is as good as the best code specifically designed for the unknown channel. Previous rateless coding schemes are designed for specific classes of channels such as AWGN channels, binary erasure channels, etc. but the proposed rateless coding scheme is capacity-achieving for broad classes of channels as long as they are ordered via degradation. Moreover, it inherits the conceptual and computational simplicity of polar codes.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Discrete Rényi Classifiers

Consider the binary classification problem of predicting a target variable $Y$ from a discrete feature vector $X = (X_1,...,X_d)$. When the probability distribution $\mathbb{P}(X,Y)$ is known, the optimal classifier, leading to the minimum misclassification rate, is given by the Maximum A-posteriori Probability decision rule. However, estimating the complete joint distribution $\mathbb{P}(X,Y)$ is computationally and statistically impossible for large values of $d$. An alternative approach is to first estimate some low order marginals of $\mathbb{P}(X,Y)$ and then design the classifier based on the estimated low order marginals. This approach is also helpful when the complete training data instances are not available due to privacy concerns. In this work, we consider the problem of finding the optimum classifier based on some estimated low order marginals of $(X,Y)$. We prove that for a given set of marginals, the minimum Hirschfeld-Gebelein-Renyi (HGR) correlation principle introduced in [1] leads to a randomized classification rule which is shown to have a misclassification rate no larger than twice the misclassification rate of the optimal classifier. Then, under a separability condition, we show that the proposed algorithm is equivalent to a randomized linear regression approach. In addition, this method naturally results in a robust feature selection method selecting a subset of features having the maximum worst case HGR correlation with the target variable. Our theoretical upper-bound is similar to the recent Discrete Chebyshev Classifier (DCC) approach [2], while the proposed algorithm has significant computational advantages since it only requires solving a least square optimization problem. Finally, we numerically compare our proposed algorithm with the DCC classifier and show that the proposed algorithm results in better misclassification rate over various datasets.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Do Read Errors Matter for Genome Assembly?

While most current high-throughput DNA sequencing technologies generate short reads with low error rates, emerging sequencing technologies generate long reads with high error rates. A basic question of interest is the tradeoff between read length and error rate in terms of the information needed for the perfect assembly of the genome. Using an adversarial erasure error model, we make progress on this problem by establishing a critical read length, as a function of the genome and the error rate, above which perfect assembly is guaranteed. For several real genomes, including those from the GAGE dataset, we verify that this critical read length is not significantly greater than the read length required for perfect assembly from reads without errors.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Minimum HGR Correlation Principle: From Marginals to Joint Distribution

Given low order moment information over the random variables $\mathbf{X} = (X_1,X_2,\ldots,X_p)$ and $Y$, what distribution minimizes the Hirschfeld-Gebelein-Rényi (HGR) maximal correlation coefficient between $\mathbf{X}$ and $Y$, while remains faithful to the given moments? The answer to this question is important especially in order to fit models over $(\mathbf{X},Y)$ with minimum dependence among the random variables $\mathbf{X}$ and $Y$. In this paper, we investigate this question first in the continuous setting by showing that the jointly Gaussian distribution achieves the minimum HGR correlation coefficient among distributions with the given first and second order moments. Then, we pose a similar question in the discrete scenario by fixing the pairwise marginals of the random variables $\mathbf{X}$ and $Y$. To answer this question in the discrete setting, we first derive a lower bound for the HGR correlation coefficient over the class of distributions with fixed pairwise marginals. Then we show that this lower bound is tight if there exists a distribution with certain {\it additive} structure satisfying the given pairwise marginals. Moreover, the distribution with the additive structure achieves the minimum HGR correlation coefficient. Finally, we conclude by showing that the event of obtaining pairwise marginals containing an additive structured distribution has a positive Lebesgue measure over the probability simplex.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Optimal Haplotype Assembly from High-Throughput Mate-Pair Reads

Humans have $23$ pairs of homologous chromosomes. The homologous pairs are almost identical pairs of chromosomes. For the most part, differences in homologous chromosome occur at certain documented positions called single nucleotide polymorphisms (SNPs). A haplotype of an individual is the pair of sequences of SNPs on the two homologous chromosomes. In this paper, we study the problem of inferring haplotypes of individuals from mate-pair reads of their genome. We give a simple formula for the coverage needed for haplotype assembly, under a generative model. The analysis here leverages connections of this problem with decoding convolutional codes.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Reduce the Complexity of List Decoding of Polar Codes by Tree-Pruning

Polar codes under cyclic redundancy check aided successive cancellation list (CA-SCL) decoding can outperform the turbo codes and the LDPC codes when code lengths are configured to be several kilobits. In order to reduce the decoding complexity, a novel tree-pruning scheme for the \mbox{SCL/CA-SCL} decoding algorithms is proposed in this paper. In each step of the decoding procedure, the candidate paths with metrics less than a threshold are dropped directly to avoid the unnecessary computations for the path searching on the descendant branches of them. Given a candidate path, an upper bound of the path metric of its descendants is proposed to determined whether the pruning of this candidate path would affect frame error rate (FER) performance. By utilizing this upper bounding technique and introducing a dynamic threshold, the proposed scheme deletes the redundant candidate paths as many as possible while keeping the performance deterioration in a tolerant region, thus it is much more efficient than the existing pruning scheme. With only a negligible loss of FER performance, the computational complexity of the proposed pruned decoding scheme is only about $40\%$ of the standard algorithm in the low signal-to-noise ratio (SNR) region (where the FER under CA-SCL decoding is about $0.1 \sim 0.001$), and it can be very close to that of the successive cancellation (SC) decoder in the moderate and high SNR regions.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

The two-unicast problem

We consider the communication capacity of wireline networks for a two-unicast traffic pattern. The network has two sources and two destinations with each source communicating a message to its own destination, subject to the capacity constraints on the directed edges of the network. We propose a simple outer bound for the problem that we call the Generalized Network Sharing (GNS) bound. We show this bound is the tightest edge-cut bound for two-unicast networks and is tight in several bottleneck cases, though it is not tight in general. We also show that the problem of computing the GNS bound is NP-complete. Finally, we show that despite its seeming simplicity, the two-unicast problem is as hard as the most general network coding problem. As a consequence, linear coding is insufficient to achieve capacity for general two-unicast networks, and non-Shannon inequalities are necessary for characterizing capacity of general two-unicast networks.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Near-optimal Assembly for Shotgun Sequencing with Noisy Reads

Recent work identified the fundamental limits on the information requirements in terms of read length and coverage depth required for successful de novo genome reconstruction from shotgun sequencing data, based on the idealistic assumption of no errors in the reads (noiseless reads). In this work, we show that even when there is noise in the reads, one can successfully reconstruct with information requirements close to the noiseless fundamental limit. A new assembly algorithm, X-phased Multibridging, is designed based on a probabilistic model of the genome. It is shown through analysis to perform well on the model, and through simulations to perform well on real genomes.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Network Risk Limiting Dispatch: Optimal Control and Price of Uncertainty

Increased uncertainty due to high penetration of renewables imposes significant costs to the system operators. The added costs depend on several factors including market design, performance of renewable generation forecasting and the specific dispatch procedure. Quantifying these costs has been limited to small sample Monte Carlo approaches applied specific dispatch algorithms. The computational complexity and accuracy of these approaches has limited the understanding of tradeoffs between different factors. {In this work we consider a two-stage stochastic economic dispatch problem. Our goal is to provide an analytical quantification and an intuitive understanding of the effects of uncertainties and network congestion on the dispatch procedure and the optimal cost.} We first consider an uncongested network and calculate the risk limiting dispatch. In addition, we derive the price of uncertainty, a number that characterizes the intrinsic impact of uncertainty on the integration cost of renewables. Then we extend the results to a network where one link can become congested. Under mild conditions, we calculate price of uncertainty even in this case. We show that risk limiting dispatch is given by a set of deterministic equilibrium equations. The dispatch solution yields an important insight: congested links do not create isolated nodes, even in a two-node network. In fact, the network can support backflows in congested links, that are useful to reduce the uncertainty by averaging supply across the network. We demonstrate the performance of our approach in standard IEEE benchmark networks.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

A Local Control Approach to Voltage Regulation in Distribution Networks

This paper address the problem of voltage regulation in power distribution networks with deep penetration of distributed energy resources (DERs) without any explicit communication between the buses in the network. We cast the problem as an optimization problem with the objective of minimizing the distance between the bus voltage magnitudes and some reference voltage profile. We present an iterative algorithm where each bus updates the reactive power injection provided by their DER. The update at a bus only depends on the voltage magnitude at that bus, and for this reason, we call the algorithm a local control algorithm. We provide sufficient conditions that guarantee the convergence of the algorithm and these conditions can be checked a priori for a set of feasible power injections. We also provide necessary conditions establishing that longer and more heavily loaded networks are inherently more difficult to control. We illustrate the operation of the algorithm through case studies involving 8-,34- and 123-bus test distribution systems.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

Geometry of Power Flows and Optimization in Distribution Networks

We investigate the geometry of injection regions and its relationship to optimization of power flows in tree networks. The injection region is the set of all vectors of bus power injections that satisfy the network and operation constraints. The geometrical object of interest is the set of Pareto-optimal points of the injection region. If the voltage magnitudes are fixed, the injection region of a tree network can be written as a linear transformation of the product of two-bus injection regions, one for each line in the network. Using this decomposition, we show that under the practical condition that the angle difference across each line is not too large, the set of Pareto-optimal points of the injection region remains unchanged by taking the convex hull. Moreover, the resulting convexified optimal power flow problem can be efficiently solved via }{ semi-definite programming or second order cone relaxations. These results improve upon earlier works by removing the assumptions on active power lower bounds. It is also shown that our practical angle assumption guarantees two other properties: (i) the uniqueness of the solution of the power flow problem, and (ii) the non-negativity of the locational marginal prices. Partial results are presented for the case when the voltage magnitudes are not fixed but can lie within certain bounds.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

Information Theory of DNA Shotgun Sequencing

DNA sequencing is the basic workhorse of modern day biology and medicine. Shotgun sequencing is the dominant technique used: many randomly located short fragments called reads are extracted from the DNA sequence, and these reads are assembled to reconstruct the original sequence. A basic question is: given a sequencing technology and the statistics of the DNA sequence, what is the minimum number of reads required for reliable reconstruction? This number provides a fundamental limit to the performance of {\em any} assembly algorithm. For a simple statistical model of the DNA sequence and the read process, we show that the answer admits a critical phenomena in the asymptotic limit of long DNA sequences: if the read length is below a threshold, reconstruction is impossible no matter how many reads are observed, and if the read length is above the threshold, having enough reads to cover the DNA sequence is sufficient to reconstruct. The threshold is computed in terms of the Renyi entropy rate of the DNA sequence. We also study the impact of noise in the read process on the performance.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

Optimal Assembly for High Throughput Shotgun Sequencing

We present a framework for the design of optimal assembly algorithms for shotgun sequencing under the criterion of complete reconstruction. We derive a lower bound on the read length and the coverage depth required for reconstruction in terms of the repeat statistics of the genome. Building on earlier works, we design a de Brujin graph based assembly algorithm which can achieve very close to the lower bound for repeat statistics of a wide range of sequenced genomes, including the GAGE datasets. The results are based on a set of necessary and sufficient conditions on the DNA sequence and the reads for reconstruction. The conditions can be viewed as the shotgun sequencing analogue of Ukkonen-Pevzner's necessary and sufficient conditions for Sequencing by Hybridization.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

Optimal DNA shotgun sequencing: Noisy reads are as good as noiseless reads

We establish the fundamental limits of DNA shotgun sequencing under noisy reads. We show a surprising result: for the i.i.d. DNA model, noisy reads are as good as noiseless reads, provided that the noise level is below a certain threshold which can be surprisingly high. As an example, for a uniformly distributed DNA sequence and a symmetric substitution noisy read channel, the threshold is as high as 19%.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

Parallel Decoders of Polar Codes

In this letter, we propose parallel SC (Successive Cancellation) decoder and parallel SC-List decoder for polar codes. The parallel decoder is composed of M=2^m(m>=1) component decoders working in parallel and each component decoder decodes a Polar code of a block size of 1/M of the original Polar code. Therefore the parallel decoder has M times faster decoding speed. Our simulation results show that the parallel decoder has almost the same error-rate performance as the conventional non-parallel decoder.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

A Compression Algorithm Using Mis-aligned Side-information

We study the problem of compressing a source sequence in the presence of side-information that is related to the source via insertions, deletions and substitutions. We propose a simple algorithm to compress the source sequence when the side-information is present at both the encoder and decoder. A key attribute of the algorithm is that it encodes the edits contained in runs of different extents separately. For small insertion and deletion probabilities, the compression rate of the algorithm is shown to be asymptotically optimal.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

An Adaptive Successive Cancellation List Decoder for Polar Codes with Cyclic Redundancy Check

In this letter, we propose an adaptive SC (Successive Cancellation)-List decoder for polar codes with CRC. This adaptive SC-List decoder iteratively increases the list size until the decoder outputs contain at least one survival path which can pass CRC. Simulation shows that the adaptive SC-List decoder provides significant complexity reduction. We also demonstrate that polar code (2048, 1024) with 24-bit CRC decoded by our proposed adaptive SC-List decoder with very large list size can achieve a frame error rate FER=0.001 at Eb/No=1.1dB, which is about 0.2dB from the information theoretic limit at this block length.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Asynchronous Capacity per Unit Cost

The capacity per unit cost, or equivalently minimum cost to transmit one bit, is a well-studied quantity. It has been studied under the assumption of full synchrony between the transmitter and the receiver. In many applications, such as sensor networks, transmissions are very bursty, with small amounts of bits arriving infrequently at random times. In such scenarios, the cost of acquiring synchronization is significant and one is interested in the fundamental limits on communication without assuming a priori synchronization. In this paper, we show that the minimum cost to transmit B bits of information asynchronously is (B + \bar{H})k_sync, where k_sync is the synchronous minimum cost per bit and \bar{H} is a measure of timing uncertainty equal to the entropy for most reasonable arrival time distributions. This result holds when the transmitter can stay idle at no cost and is a particular case of a general result which holds for arbitrary cost functions.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Coding and System Design for Quantize-Map-and-Forward Relaying

In this paper we develop a low-complexity coding scheme and system design framework for the half duplex relay channel based on the Quantize-Map-and-Forward (QMF) relay- ing scheme. The proposed framework allows linear complexity operations at all network terminals. We propose the use of binary LDPC codes for encoding at the source and LDGM codes for mapping at the relay. We express joint decoding at the destination as a belief propagation algorithm over a factor graph. This graph has the LDPC and LDGM codes as subgraphs connected via probabilistic constraints that model the QMF relay operations. We show that this coding framework extends naturally to the high SNR regime using bit interleaved coded modulation (BICM). We develop density evolution analysis tools for this factor graph and demonstrate the design of practical codes for the half-duplex relay channel that perform within 1dB of information theoretic QMF threshold.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Completely Stale Transmitter Channel State Information is Still Very Useful

Transmitter channel state information (CSIT) is crucial for the multiplexing gains offered by advanced interference management techniques such as multiuser MIMO and interference alignment. Such CSIT is usually obtained by feedback from the receivers, but the feedback is subject to delays. The usual approach is to use the fed back information to predict the current channel state and then apply a scheme designed assuming perfect CSIT. When the feedback delay is large compared to the channel coherence time, such a prediction approach completely fails to achieve any multiplexing gain. In this paper, we show that even in this case, the completely stale CSI is still very useful. More concretely, we show that in a MIMO broadcast channel with $K$ transmit antennas and $K$ receivers each with 1 receive antenna, $\frac{K}{1+1/2+ ...+ \frac{1}{K}} (> 1) $ degrees of freedom is achievable even when the fed back channel state is completely independent of the current channel state. Moreover, we establish that if all receivers have independent and identically distributed channels, then this is the optimal number of degrees of freedom achievable. In the optimal scheme, the transmitter uses the fed back CSI to learn the side information that the receivers receive from previous transmissions rather than to predict the current channel state. Our result can be viewed as the first example of feedback providing a degree-of-freedom gain in memoryless channels.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Geometry of Injection Regions of Power Networks

We investigate the constraints on power flow in networks and its implications to the optimal power flow problem. The constraints are described by the injection region of a network; this is the set of all vectors of power injections, one at each bus, that can be achieved while satisfying the network and operation constraints. If there are no operation constraints, we show the injection region of a network is the set of all injections satisfying the conservation of energy. If the network has a tree topology, e.g., a distribution network, we show that under voltage magnitude, line loss constraints, line flow constraints and certain bus real and reactive power constraints, the injection region and its convex hull have the same Pareto-front. The Pareto-front is of interest since these are the the optimal solutions to the minimization of increasing functions over the injection region. For non-tree networks, we obtain a weaker result by characterize the convex hull of the voltage constraint injection region for lossless cycles and certain combinations of cycles and trees.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Two-way Interference Channels

We consider two-way interference channels (ICs) where forward and backward channels are ICs but not necessarily the same. We first consider a scenario where there are only two forward messages and feedback is offered through the backward IC for aiding forward-message transmission. For a linear deterministic model of this channel, we develop inner and outer bounds that match for a wide range of channel parameters. We find that the backward IC can be more efficiently used for feedback rather than if it were used for sending its own independent backward messages. As a consequence, we show that feedback can provide a net increase in capacity even if feedback cost is taken into consideration. Moreover we extend this to a more general scenario with two additional independent backward messages, from which we find that interaction can provide an arbitrarily large gain in capacity.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Distributed Algorithms for Optimal Power Flow Problem

Optimal power flow (OPF) is an important problem for power generation and it is in general non-convex. With the employment of renewable energy, it will be desirable if OPF can be solved very efficiently so its solution can be used in real time. With some special network structure, e.g. trees, the problem has been shown to have a zero duality gap and the convex dual problem yields the optimal solution. In this paper, we propose a primal and a dual algorithm to coordinate the smaller subproblems decomposed from the convexified OPF. We can arrange the subproblems to be solved sequentially and cumulatively in a central node or solved in parallel in distributed nodes. We test the algorithms on IEEE radial distribution test feeders, some random tree-structured networks, and the IEEE transmission system benchmarks. Simulation results show that the computation time can be improved dramatically with our algorithms over the centralized approach of solving the problem without decomposition, especially in tree-structured problems. The computation time grows linearly with the problem size with the cumulative approach while the distributed one can have size-independent computation time.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Distributed Storage for Intermittent Energy Sources: Control Design and Performance Limits

One of the most important challenges in the integration of renewable energy sources into the power grid lies in their `intermittent' nature. The power output of sources like wind and solar varies with time and location due to factors that cannot be controlled by the provider. Two strategies have been proposed to hedge against this variability: 1) use energy storage systems to effectively average the produced power over time; 2) exploit distributed generation to effectively average production over location. We introduce a network model to study the optimal use of storage and transmission resources in the presence of random energy sources. We propose a Linear-Quadratic based methodology to design control strategies, and we show that these strategies are asymptotically optimal for some simple network topologies. For these topologies, the dependence of optimal performance on storage and transmission capacity is explicitly quantified.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Efficient File Synchronization: a Distributed Source Coding Approach

The problem of reconstructing a source sequence with the presence of decoder side-information that is mis-synchronized to the source due to deletions is studied in a distributed source coding framework. Motivated by practical applications, the deletion process is assumed to be bursty and is modeled by a Markov chain. The minimum rate needed to reconstruct the source sequence with high probability is characterized in terms of an information theoretic expression, which is interpreted as the amount of information of the deleted content and the locations of deletions, subtracting "nature's secret", that is, the uncertainty of the locations given the source and side-information. For small bursty deletion probability, the asymptotic expansion of the minimum rate is computed.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Geometry of the 3-user MIMO interference channel

This paper studies vector space interference alignment for the three-user MIMO interference channel with no time or frequency diversity. The main result is a characterization of the feasibility of interference alignment in the symmetric case where all transmitters have M antennas and all receivers have N antennas. If N >= M and all users desire d transmit dimensions, then alignment is feasible if and only if (2r+1)d <= max(rN,(r+1)M) for all nonnegative integers r. The analogous result holds with M and N switched if M >= N. It turns out that, just as for the 3-user parallel interference channel \cite{BT09}, the length of alignment paths captures the essence of the problem. In fact, for each feasible value of M and N the maximum alignment path length dictates both the converse and achievability arguments. One of the implications of our feasibility criterion is that simply counting equations and comparing to the number of variables does not predict feasibility. Instead, a more careful investigation of the geometry of the alignment problem is required. The necessary condition obtained by counting equations is implied by our new feasibility criterion.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Polytope Codes Against Adversaries in Networks

Network coding is studied when an adversary controls a subset of nodes in the network of limited quantity but unknown location. This problem is shown to be more difficult than when the adversary controls a given number of edges in the network, in that linear codes are insufficient. To solve the node problem, the class of Polytope Codes is introduced. Polytope Codes are constant composition codes operating over bounded polytopes in integer vector fields. The polytope structure creates additional complexity, but it induces properties on marginal distributions of code vectors so that validities of codewords can be checked by internal nodes of the network. It is shown that Polytope Codes achieve a cut-set bound for a class of planar networks. It is also shown that this cut-set bound is not always tight, and a tighter bound is given for an example network.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Settling the feasibility of interference alignment for the MIMO interference channel: the symmetric square case

Determining the feasibility conditions for vector space interference alignment in the K-user MIMO interference channel with constant channel coefficients has attracted much recent attention yet remains unsolved. The main result of this paper is restricted to the symmetric square case where all transmitters and receivers have N antennas, and each user desires d transmit dimensions. We prove that alignment is possible if and only if the number of antennas satisfies N>= d(K+1)/2. We also show a necessary condition for feasibility of alignment with arbitrary system parameters. An algebraic geometry approach is central to the results.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Wireless Network Information Flow: A Deterministic Approach

In a wireless network with a single source and a single destination and an arbitrary number of relay nodes, what is the maximum rate of information flow achievable? We make progress on this long standing problem through a two-step approach. First we propose a deterministic channel model which captures the key wireless properties of signal strength, broadcast and superposition. We obtain an exact characterization of the capacity of a network with nodes connected by such deterministic channels. This result is a natural generalization of the celebrated max-flow min-cut theorem for wired networks. Second, we use the insights obtained from the deterministic analysis to design a new quantize-map-and-forward scheme for Gaussian networks. In this scheme, each relay quantizes the received signal at the noise level and maps it to a random Gaussian codeword for forwarding, and the final destination decodes the source's message based on the received signal. We show that, in contrast to existing schemes, this scheme can achieve the cut-set upper bound to within a gap which is independent of the channel parameters. In the case of the relay channel with a single relay as well as the two-relay Gaussian diamond network, the gap is 1 bit/s/Hz. Moreover, the scheme is universal in the sense that the relays need no knowledge of the values of the channel parameters to (approximately) achieve the rate supportable by the network. We also present extensions of the results to multicast networks, half-duplex networks and ergodic networks.

0 citations0 reviewsNo code linkedOpen access
preprint2010arXiv

Downlink Interference Alignment

We develop an interference alignment (IA) technique for a downlink cellular system. In the uplink, IA schemes need channel-state-information exchange across base-stations of different cells, but our downlink IA technique requires feedback only within a cell. As a result, the proposed scheme can be implemented with a few changes to an existing cellular system where the feedback mechanism (within a cell) is already being considered for supporting multi-user MIMO. Not only is our proposed scheme implementable with little effort, it can in fact provide substantial gain especially when interference from a dominant interferer is significantly stronger than the remaining interference: it is shown that in the two-isolated cell layout, our scheme provides four-fold gain in throughput performance over a standard multi-user MIMO technique. We show through simulations that our technique provides respectable gain under a more realistic scenario: it gives approximately 20% gain for a 19 hexagonal wrap-around-cell layout. Furthermore, we show that our scheme has the potential to provide substantial gain for macro-pico cellular networks where pico-users can be significantly interfered with by the nearby macro-BS.

0 citations0 reviewsNo code linkedOpen access
preprint2010arXiv

Feedback Capacity of the Gaussian Interference Channel to within 2 Bits

We characterize the capacity region to within 2 bits/s/Hz and the symmetric capacity to within 1 bit/s/Hz for the two-user Gaussian interference channel (IC) with feedback. We develop achievable schemes and derive a new outer bound to arrive at this conclusion. One consequence of the result is that feedback provides multiplicative gain, i.e., the gain becomes arbitrarily large for certain channel parameters. It is a surprising result because feedback has been so far known to provide no gain in memoryless point-to-point channels and only bounded additive gain in multiple access channels. The gain comes from using feedback to maximize resource utilization, thereby enabling more efficient resource sharing between the interfering users. The result makes use of a deterministic model to provide insights into the Gaussian channel. This deterministic model is a special case of El Gamal-Costa deterministic model and as a side-generalization, we establish the exact feedback capacity region of this general class of deterministic ICs.

0 citations0 reviewsNo code linkedOpen access
preprint2010arXiv

Linear Capacity Scaling in Wireless Networks: Beyond Physical Limits?

We investigate the role of cooperation in wireless networks subject to a spatial degrees of freedom limitation. To address the worst case scenario, we consider a free-space line-of-sight type environment with no scattering and no fading. We identify three qualitatively different operating regimes that are determined by how the area of the network A, normalized with respect to the wavelength lambda, compares to the number of users n. In networks with sqrt{A}/lambda < sqrt{n}, the limitation in spatial degrees of freedom does not allow to achieve a capacity scaling better than sqrt{n} and this performance can be readily achieved by multi-hopping. This result has been recently shown by Franceschetti et al. However, for networks with sqrt{A}/lambda > sqrt{n}, the number of available degrees of freedom is min(n, sqrt{A}/lambda), larger that what can be achieved by multi-hopping. We show that the optimal capacity scaling in this regime is achieved by hierarchical cooperation. In particular, in networks with sqrt{A}/lambda> n, hierarchical cooperation can achieve linear scaling.

0 citations0 reviewsNo code linkedOpen access
preprint2008arXiv

The Approximate Capacity of the Many-to-One and One-to-Many Gaussian Interference Channels

Recently, Etkin, Tse, and Wang found the capacity region of the two-user Gaussian interference channel to within one bit/s/Hz. A natural goal is to apply this approach to the Gaussian interference channel with an arbitrary number of users. We make progress towards this goal by finding the capacity region of the many-to-one and one-to-many Gaussian interference channels to within a constant number of bits. The result makes use of a deterministic model to provide insight into the Gaussian channel. The deterministic model makes explicit the dimension of signal scale. A central theme emerges: the use of lattice codes for alignment of interfering signals on the signal scale.

0 citations0 reviewsNo code linkedOpen access