BZPEERPapers, people, evidence and research paths
Menu

Discover

SearchFind papers, people, topics, institutions and opportunities from one place.GraphSee how papers, authors, topics and evidence connect.

Research tools

HomeContinue from the papers, people and topics that matter now.CollectionsTurn saved papers into reading lists, evidence maps and shareable context.ResearchReconstruct the path that led you from search to paper to question.CollaborationMove from a paper, topic or expert request into a focused thread.PublishCreate full-text research articles with metadata, formulas and versions.

Network

ExpertsFind specialists whose work and reviews explain why they can help.CommunitiesJoin research groups organized around papers, topics and shared evidence.

Opportunities

ListingsBrowse roles, calls and collaborations connected to real research context.

Account

Log inReturn to your saved papers, graph views and active threads.Create accountStart saving papers, building a research trail and joining teams.
Log inCreate account

Source author record

Bhaskar Krishnamachari

Bhaskar Krishnamachari appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record
Networking and Internet ArchitectureMachine Learningmath.OCmath.PRSystems and ControlInformation Theorymath.ITRoboticsComputer Science and Game TheoryCryptography and SecurityDiscrete MathematicsComputational ComplexityData Structures and AlgorithmsDistributed, Parallel, and Cluster Computingeess.SPmath.COMultiagent Systemsphysics.soc-phPopulations and Evolutionq-fin.GNq-fin.TR

Catalog footprint

What is connected

34works
21topics
4close collaborators

Actions

Connect this record

Log in to claim
Open graphBrowse works

Research graph

See the researcher in context

Open full explorer

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

34 published item(s)

preprint2022arXiv

Blizzard: a Distributed Consensus Protocol for Mobile Devices

We present Blizzard, a Byzantine Fault Tolerant (BFT) distributed ledger protocol that is aimed at making mobile devices first-class citizens in the consensus process. Blizzard introduces a novel two-tier architecture by having the mobile nodes communicate through online brokers, and includes a decentralized matching scheme to ensure each node connects to a certain number of random brokers. Through mathematical analysis, we derive a guaranteed safety region (i.e. the set of ratios of malicious nodes and malicious brokers for which the safety is assured) for the Blizzard protocol. Liveness is shown as well. We analyze the performance of Blizzard in terms of its throughput, latency and message complexity. Through experiments based on a software implementation, we show that Blizzard is capable of throughput on the order of several thousand transactions per second per shard, and sub-second confirmation latency.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

DEFER: Distributed Edge Inference for Deep Neural Networks

Modern machine learning tools such as deep neural networks (DNNs) are playing a revolutionary role in many fields such as natural language processing, computer vision, and the internet of things. Once they are trained, deep learning models can be deployed on edge computers to perform classification and prediction on real-time data for these applications. Particularly for large models, the limited computational and memory resources on a single edge device can become the throughput bottleneck for an inference pipeline. To increase throughput and decrease per-device compute load, we present DEFER (Distributed Edge inFERence), a framework for distributed edge inference, which partitions deep neural networks into layers that can be spread across multiple compute nodes. The architecture consists of a single "dispatcher" node to distribute DNN partitions and inference data to respective compute nodes. The compute nodes are connected in a series pattern where each node's computed result is relayed to the subsequent node. The result is then returned to the Dispatcher. We quantify the throughput, energy consumption, network payload, and overhead for our framework under realistic network conditions using the CORE network emulator. We find that for the ResNet50 model, the inference throughput of DEFER with 8 compute nodes is 53% higher and per node energy consumption is 63% lower than single device inference. We further reduce network communication demands and energy consumption using the ZFP serialization and LZ4 compression algorithms. We have implemented DEFER in Python using the TensorFlow and Keras ML libraries, and have released DEFER as an open-source framework to benefit the research community.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Federated Learning for Internet of Things: Applications, Challenges, and Opportunities

Billions of IoT devices will be deployed in the near future, taking advantage of faster Internet speed and the possibility of orders of magnitude more endpoints brought by 5G/6G. With the growth of IoT devices, vast quantities of data that may contain users' private information will be generated. The high communication and storage costs, mixed with privacy concerns, will increasingly challenge the traditional ecosystem of centralized over-the-cloud learning and processing for IoT platforms. Federated Learning (FL) has emerged as the most promising alternative approach to this problem. In FL, training data-driven machine learning models is an act of collaboration between multiple clients without requiring the data to be brought to a central point, hence alleviating communication and storage costs and providing a great degree of user-level privacy. However, there are still some challenges existing in the real FL system implementation on IoT networks. In this paper, we will discuss the opportunities and challenges of FL in IoT platforms, as well as how it can enable diverse IoT applications. In particular, we identify and discuss seven critical challenges of FL in IoT platforms and highlight some recent promising approaches towards addressing them.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

Dynamic Curves for Decentralized Autonomous Cryptocurrency Exchanges

One of the exciting recent developments in decentralized finance (DeFi) has been the development of decentralized cryptocurrency exchanges that can autonomously handle conversion between different cryptocurrencies. Decentralized exchange protocols such as Uniswap, Curve and other types of Automated Market Makers (AMMs) maintain a liquidity pool (LP) of two or more assets constrained to maintain at all times a mathematical relation to each other, defined by a given function or curve. Examples of such functions are the constant-sum and constant-product AMMs. Existing systems however suffer from several challenges. They require external arbitrageurs to restore the price of tokens in the pool to match the market price. Such activities can potentially drain resources from the liquidity pool. In particular, dramatic market price changes can result in low liquidity with respect to one or more of the assets and reduce the total value of the LP. We propose in this work a new approach to constructing the AMM by proposing the idea of dynamic curves. It utilizes input from a market price oracle to modify the mathematical relationship between the assets so that the pool price continuously and automatically adjusts to be identical to the market price. This approach eliminates arbitrage opportunities and, as we show through simulations, maintains liquidity in the LP for all assets and the total value of the LP over a wide range of market prices.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

Reducing the Volatility of Cryptocurrencies -- A Survey of Stablecoins

In the wake of financial crises, stablecoins are gaining adoption among digital currencies. We discuss how stablecoins help reduce the volatility of cryptocurrencies by surveying different types of stablecoins and their stability mechanisms. We classify different approaches to stablecoins in three main categories i) fiat or asset backed, ii) crypto-collateralized and iii) algorithmic stablecoins, giving examples of concrete projects in each class. We assess the relative tradeoffs between the different approaches. We also discuss challenges associated with the future of stablecoins and their adoption, their adoption and point out future research directions.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

CONTAIN: Privacy-oriented Contact Tracing Protocols for Epidemics

Pandemic and epidemic diseases such as CoVID-19, SARS-CoV2, and Ebola have spread to multiple countries and infected thousands of people. Such diseases spread mainly through person-to-person contacts. Health care authorities recommend contact tracing procedures to prevent the spread to a vast population. Although several mobile applications have been developed to trace contacts, they typically require collection of privacy-intrusive information such as GPS locations, and the logging of privacy-sensitive data on a third party server, or require additional infrastructure such as WiFi APs with known locations. In this paper, we introduce CONTAIN, a privacy-oriented mobile contact tracing application that does not rely on GPS or any other form of infrastructure-based location sensing, nor the continuous logging of any other personally identifiable information on a server. The goal of CONTAIN is to allow users to determine with complete privacy if they have been within a short distance, specifically, Bluetooth wireless range, of someone that is infected, and potentially also when. We identify and prove the privacy guarantees provided by our approach. Our simulation study utilizing an empirical trace dataset (Asturies) involving 100 mobile devices and around 60000 records shows that users can maximize their possibility of identifying if they were near an infected user by turning on the app during active times.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

COVID-19 Risk Estimation using a Time-varying SIR-model

Policy-makers require data-driven tools to assess the spread of COVID-19 and inform the public of their risk of infection on an ongoing basis. We propose a rigorous hybrid model-and-data-driven approach to risk scoring based on a time-varying SIR epidemic model that ultimately yields a simplified color-coded risk level for each community. The risk score $Γ_t$ that we propose is proportional to the probability of someone currently healthy getting infected in the next 24 hours. We show how this risk score can be estimated using another useful metric of infection spread, $R_t$, the time-varying average reproduction number which indicates the average number of individuals an infected person would infect in turn. The proposed approach also allows for quantification of uncertainty in the estimates of $R_t$ and $Γ_t$ in the form of confidence intervals. Code and data from our effort have been open-sourced and are being applied to assess and communicate the risk of infection in the City and County of Los Angeles.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Inter-Mobile-Device Distance Estimation using Network Localization Algorithms for Digital Contact Logging Applications

Mobile applications are being developed for automated logging of contacts via Bluetooth to help scale up digital contact tracing efforts in the context of the ongoing COVID-19 pandemic. A useful component of such applications is inter-device distance estimation, which can be formulated as a network localization problem. We survey several approaches and evaluate the performance of each on real and simulated Bluetooth Low Energy (BLE) measurement datasets with respect to both distance estimate accuracy and the proximity detection problem. We investigate the effects of obstructions like pockets, differences between device models, and the environment (i.e. indoors or outdoors) on performance. We conclude that while direct estimation can provide the best proximity detection when Received Signal Strength Indicator (RSSI) measurements are available, network localization algorithms like Isomap, Local Linear Embedding, and the spring model outperform direct estimation in the presence of missing or very noisy measurements. The spring model consistently achieves the best distance estimation accuracy.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Interference Power Bound Analysis of a Network of Wireless Robots

We consider a fundamental problem concerning the deployment of a wireless robotic network: to fulfill various end-to-end performance requirements, a "sufficient" number of robotic relays must be deployed to ensure that links are of acceptable quality. Prior work has not addressed how to find this number. We use the properties of Carrier Sense Multiple Access (CSMA) based wireless communication to derive an upper bound on the spacing between any transmitter-receiver pair, which directly translates to a lower bound on the number of robots to deploy. We focus on SINR-based performance requirements due to their wide applicability. Next, we show that the bound can be improved by exploiting the geometrical structure of a network, such as linearity in the case of flow-based robotic router networks. Furthermore, we also use the bound on robot count to formulate a lower bound on the number of orthogonal codes required for a high probability of interference free communication. We demonstrate and validate our proposed bounds through simulations.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Online Learning for Wireless Distributed Computing

There has been a growing interest for Wireless Distributed Computing (WDC), which leverages collaborative computing over multiple wireless devices. WDC enables complex applications that a single device cannot support individually. However, the problem of assigning tasks over multiple devices becomes challenging in the dynamic environments encountered in real-world settings, considering that the resource availability and channel conditions change over time in unpredictable ways due to mobility and other factors. In this paper, we formulate a task assignment problem as an online learning problem using an adversarial multi-armed bandit framework. We propose MABSTA, a novel online learning algorithm that learns the performance of unknown devices and channel qualities continually through exploratory probing and makes task assignment decisions by exploiting the gained knowledge. For maximal adaptability, MABSTA is designed to make no stochastic assumption about the environment. We analyze it mathematically and provide a worst-case performance guarantee for any dynamic environment. We also compare it with the optimal offline policy as well as other baselines via emulations on trace-data obtained from a wireless IoT testbed, and show that it offers competitive and robust performance in all cases. To the best of our knowledge, MABSTA is the first online algorithm in this domain of task assignment problems and provides provable performance guarantee.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Online Learning Schemes for Power Allocation in Energy Harvesting Communications

We consider the problem of power allocation over a time-varying channel with unknown distribution in energy harvesting communication systems. In this problem, the transmitter has to choose the transmit power based on the amount of stored energy in its battery with the goal of maximizing the average rate obtained over time. We model this problem as a Markov decision process (MDP) with the transmitter as the agent, the battery status as the state, the transmit power as the action and the rate obtained as the reward. The average reward maximization problem over the MDP can be solved by a linear program (LP) that uses the transition probabilities for the state-action pairs and their reward values to choose a power allocation policy. Since the rewards associated the state-action pairs are unknown, we propose two online learning algorithms: UCLP and Epoch-UCLP that learn these rewards and adapt their policies along the way. The UCLP algorithm solves the LP at each step to decide its current policy using the upper confidence bounds on the rewards, while the Epoch-UCLP algorithm divides the time into epochs, solves the LP only at the beginning of the epochs and follows the obtained policy in that epoch. We prove that the reward losses or regrets incurred by both these algorithms are upper bounded by constants. Epoch-UCLP incurs a higher regret compared to UCLP, but reduces the computational requirements substantially. We also show that the presented algorithms work for online learning in cost minimization problems like the packet scheduling with power-delay tradeoff with minor changes.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Stochastic Contextual Bandits with Known Reward Functions

Many sequential decision-making problems in communication networks can be modeled as contextual bandit problems, which are natural extensions of the well-known multi-armed bandit problem. In contextual bandit problems, at each time, an agent observes some side information or context, pulls one arm and receives the reward for that arm. We consider a stochastic formulation where the context-reward tuples are independently drawn from an unknown distribution in each trial. Motivated by networking applications, we analyze a setting where the reward is a known non-linear function of the context and the chosen arm's current state. We first consider the case of discrete and finite context-spaces and propose DCB($ε$), an algorithm that we prove, through a careful analysis, yields regret (cumulative reward gap compared to a distribution-aware genie) scaling logarithmically in time and linearly in the number of arms that are not optimal for any context, improving over existing algorithms where the regret scales linearly in the total number of arms. We then study continuous context-spaces with Lipschitz reward functions and propose CCB($ε, δ$), an algorithm that uses DCB($ε$) as a subroutine. CCB($ε, δ$) reveals a novel regret-storage trade-off that is parametrized by $δ$. Tuning $δ$ to the time horizon allows us to obtain sub-linear regret bounds, while requiring sub-linear storage. By exploiting joint learning for all contexts we get regret bounds for CCB($ε, δ$) that are unachievable by any existing contextual bandit algorithm for continuous context-spaces. We also show similar performance bounds for the unknown horizon case.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Towards Controllability of Wireless Network Quality using Mobile Robotic Routers

We consider a problem of robotic router placement and mobility control with the objective of formation and maintenance of an optimal communication network between a set of transmitter-receiver pairs. In this scenario, the communication path between any transmitter-receiver pair contains a predetermined set of mobile robotic routers nodes. The goal of this work is to design an algorithm to optimize the positions of the robotic nodes to improve the overall performance of the network. We define the optimization metric to be the minimum of the Signal to Interference plus Noise Ratios (SINR) over all the links. In this manuscript, we propose two optimization algorithms to solve this problem in a centralized and a decentralized manner, respectively.We also demonstrate the performances of both algorithms based on a set of simulation experiments.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

On the Meeting Time for Two Random Walks on a Regular Graph

We provide an analysis of the expected meeting time of two independent random walks on a regular graph. For 1-D circle and 2-D torus graphs, we show that the expected meeting time can be expressed as the sum of the inverse of non-zero eigenvalues of a suitably defined Laplacian matrix. We also conjecture based on empirical evidence that this result holds more generally for simple random walks on arbitrary regular graphs. Further, we show that the expected meeting time for the 1-D circle of size $N$ is $Θ(N^2)$, and for a 2-D $N \times N$ torus it is $Θ(N^2 log N)$.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Route Swarm: Wireless Network Optimization through Mobility

In this paper, we demonstrate a novel hybrid architecture for coordinating networked robots in sensing and information routing applications. The proposed INformation and Sensing driven PhysIcally REconfigurable robotic network (INSPIRE), consists of a Physical Control Plane (PCP) which commands agent position, and an Information Control Plane (ICP) which regulates information flow towards communication/sensing objectives. We describe an instantiation where a mobile robotic network is dynamically reconfigured to ensure high quality routes between static wireless nodes, which act as source/destination pairs for information flow. The ICP commands the robots towards evenly distributed inter-flow allocations, with intra-flow configurations that maximize route quality. The PCP then guides the robots via potential-based control to reconfigure according to ICP commands. This formulation, deemed Route Swarm, decouples information flow and physical control, generating a feedback between routing and sensing needs and robotic configuration. We demonstrate our propositions through simulation under a realistic wireless network regime.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

Optimal Power Allocation over Multiple Identical Gilbert-Elliott Channels

We study the fundamental problem of power allocation over multiple Gilbert-Elliott communication channels. In a communication system with time varying channel qualities, it is important to allocate the limited transmission power to channels that will be in good state. However, it is very challenging to do so because channel states are usually unknown when the power allocation decision is made. In this paper, we derive an optimal power allocation policy that can maximize the expected discounted number of bits transmitted over an infinite time span by allocating the transmission power only to those channels that are believed to be good in the coming time slot. We use the concept belief to represent the probability that a channel will be good and derive an optimal power allocation policy that establishes a mapping from the channel belief to an allocation decision. Specifically, we first model this problem as a partially observable Markov decision processes (POMDP), and analytically investigate the structure of the optimal policy. Then a simple threshold-based policy is derived for a three-channel communication system. By formulating and solving a linear programming formulation of this power allocation problem, we further verified the derived structure of the optimal policy.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

Robotic Message Ferrying for Wireless Networks using Coarse-Grained Backpressure Control

We formulate the problem of robots ferrying messages between statically-placed source and sink pairs that they can communicate with wirelessly. We first analyze the capacity region for this problem under both ideal (arbitrarily high velocity, long scheduling periods) and realistic conditions. We indicate how robots could be scheduled optimally to satisfy any arrival rate in the capacity region, given prior knowledge about arrival rates. We find that if the number of robots allocated grows proportionally with the number of source-sink pairs, then the capacity of the network scales as $Θ(1)$, similar to what was shown previously by Grossglauser and Tse for uncontrolled mobility; however, in contrast to that prior result, we also find that with controlled mobility this constant capacity scaling can be obtained while ensuring finite delay. We then consider the setting where the arrival rates are unknown and present a coarse-grained backpressure message ferrying algorithm (CBMF) for it. In CBMF, the robots are matched to sources and sinks once every epoch to maximize a queue-differential-based weight. The matching controls both motion and transmission for each robot: if a robot is matched to a source, it moves towards that source and collects data from it; and if it is matched to a sink, it moves towards that sink and transmits data to it. We show through analysis and simulations the conditions under which CBMF can stabilize the network. We show that the maximum achievable stable throughput with this policy tends to the ideal capacity as the schedule duration and robot velocity increase.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Funding Games: the Truth but not the Whole Truth

We introduce the Funding Game, in which $m$ identical resources are to be allocated among $n$ selfish agents. Each agent requests a number of resources $x_i$ and reports a valuation $\tilde{v}_i(x_i)$, which verifiably {\em lower}-bounds $i$'s true value for receiving $x_i$ items. The pairs $(x_i, \tilde{v}_i(x_i))$ can be thought of as size-value pairs defining a knapsack problem with capacity $m$. A publicly-known algorithm is used to solve this knapsack problem, deciding which requests to satisfy in order to maximize the social welfare. We show that a simple mechanism based on the knapsack {\it highest ratio greedy} algorithm provides a Bayesian Price of Anarchy of 2, and for the complete information version of the game we give an algorithm that computes a Nash equilibrium strategy profile in $O(n^2 \log^2 m)$ time. Our primary algorithmic result shows that an extension of the mechanism to $k$ rounds has a Price of Anarchy of $1 + \frac{1}{k}$, yielding a graceful tradeoff between communication complexity and the social welfare.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Multiflow Transmission in Delay Constrained Cooperative Wireless Networks

This paper considers the problem of energy-efficient transmission in multi-flow multihop cooperative wireless networks. Although the performance gains of cooperative approaches are well known, the combinatorial nature of these schemes makes it difficult to design efficient polynomial-time algorithms for joint routing, scheduling and power control. This becomes more so when there is more than one flow in the network. It has been conjectured by many authors, in the literature, that the multiflow problem in cooperative networks is an NP-hard problem. In this paper, we formulate the problem, as a combinatorial optimization problem, for a general setting of $k$-flows, and formally prove that the problem is not only NP-hard but it is $o(n^{1/7-ε})$ inapproxmiable. To our knowledge*, these results provide the first such inapproxmiablity proof in the context of multiflow cooperative wireless networks. We further prove that for a special case of k = 1 the solution is a simple path, and devise a polynomial time algorithm for jointly optimizing routing, scheduling and power control. We then use this algorithm to establish analytical upper and lower bounds for the optimal performance for the general case of $k$ flows. Furthermore, we propose a polynomial time heuristic for calculating the solution for the general case and evaluate the performance of this heuristic under different channel conditions and against the analytical upper and lower bounds.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Optimal Power Allocation Policy over Two Identical Gilbert-Elliott Channels

We study the fundamental problem of optimal power allocation over two identical Gilbert-Elliott (Binary Markov) communication channels. Our goal is to maximize the expected discounted number of bits transmitted over an infinite time span by judiciously choosing one of the four actions for each time slot: 1) allocating power equally to both channels, 2) allocating all the power to channel 1, 3) allocating all the power to channel 2, and 4) allocating no power to any of the channels. As the channel state is unknown when power allocation decision is made, we model this problem as a partially observable Markov decision process(POMDP), and derive the optimal policy which gives the optimal action to take under different possible channel states. Two different structures of the optimal policy are derived analytically and verified by linear programming simulation. We also illustrate how to construct the optimal policy by the combination of threshold calculation and linear programming simulation once system parameters are known.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Power Allocation over Two Identical Gilbert-Elliott Channels

We study the problem of power allocation over two identical Gilbert-Elliot communication channels. Our goal is to maximize the expected discounted number of bits transmitted over an infinite time horizon. This is achieved by choosing among three possible strategies: (1) betting on channel 1 by allocating all the power to this channel, which results in high data rate if channel 1 happens to be in good state, and zero bits transmitted if channel 1 is in bad state (even if channel 2 is in good state) (2) betting on channel 2 by allocating all the power to the second channel, and (3) a balanced strategy whereby each channel is allocated half the total power, with the effect that each channel can transmit a low data rate if it is in good state. We assume that each channel's state is only revealed upon transmission of data on that channel. We model this problem as a partially observable Markov decision processes (MDP), and derive key threshold properties of the optimal policy. Further, we show that by formulating and solving a relevant linear program the thresholds can be determined numerically when system parameters are known.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

A Packet Dropping Mechanism for Efficient Operation of M/M/1 Queues with Selfish Users

We consider a fundamental game theoretic problem concerning selfish users contributing packets to an M/M/1 queue. In this game, each user controls its own input rate so as to optimize a desired tradeoff between throughput and delay. We first show that the original game has an inefficient Nash Equilibrium (NE), with a Price of Anarchy (PoA) that scales linearly or worse in the number of users. In order to improve the outcome efficiency, we propose an easily implementable mechanism design whereby the server randomly drops packets with a probability that is a function of the total arrival rate. We show that this results in a modified M/M/1 queueing game that is an ordinal potential game with at least one NE. In particular, for a linear packet dropping function, which is similar to the Random Early Detection (RED) algorithm used in Internet Congestion Control, we prove that there is a unique NE. We also show that the simple best response dynamic converges to this unique equilibrium. Finally, for this scheme, we prove that the social welfare (expressed either as the summation of utilities of all players, or as the summation of the logarithm of utilities of all players) at the equilibrium point can be arbitrarily close to the social welfare at the global optimal point, i.e. the PoA can be made arbitrarily close to 1. We also study the impact of arrival rate estimation error on the PoA through simulations.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Algorithmic Aspects of Energy-Delay Tradeoff in Multihop Cooperative Wireless Networks

We consider the problem of energy-efficient transmission in delay constrained cooperative multihop wireless networks. The combinatorial nature of cooperative multihop schemes makes it difficult to design efficient polynomial-time algorithms for deciding which nodes should take part in cooperation, and when and with what power they should transmit. In this work, we tackle this problem in memoryless networks with or without delay constraints, i.e., quality of service guarantee. We analyze a wide class of setups, including unicast, multicast, and broadcast, and two main cooperative approaches, namely: energy accumulation (EA) and mutual information accumulation (MIA). We provide a generalized algorithmic formulation of the problem that encompasses all those cases. We investigate the similarities and differences of EA and MIA in our generalized formulation. We prove that the broadcast and multicast problems are, in general, not only NP hard but also o(log(n)) inapproximable. We break these problems into three parts: ordering, scheduling and power control, and propose a novel algorithm that, given an ordering, can optimally solve the joint power allocation and scheduling problems simultaneously in polynomial time. We further show empirically that this algorithm used in conjunction with an ordering derived heuristically using the Dijkstra's shortest path algorithm yields near-optimal performance in typical settings. For the unicast case, we prove that although the problem remains NP hard with MIA, it can be solved optimally and in polynomial time when EA is used. We further use our algorithm to study numerically the trade-off between delay and power-efficiency in cooperative broadcast and compare the performance of EA vs MIA as well as the performance of our cooperative algorithm with a smart noncooperative algorithm in a broadcast setting.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Backpressure with Adaptive Redundancy (BWAR)

Backpressure scheduling and routing, in which packets are preferentially transmitted over links with high queue differentials, offers the promise of throughput-optimal operation for a wide range of communication networks. However, when the traffic load is low, due to the corresponding low queue occupancy, backpressure scheduling/routing experiences long delays. This is particularly of concern in intermittent encounter-based mobile networks which are already delay-limited due to the sparse and highly dynamic network connectivity. While state of the art mechanisms for such networks have proposed the use of redundant transmissions to improve delay, they do not work well when the traffic load is high. We propose in this paper a novel hybrid approach that we refer to as backpressure with adaptive redundancy (BWAR), which provides the best of both worlds. This approach is highly robust and distributed and does not require any prior knowledge of network load conditions. We evaluate BWAR through both mathematical analysis and simulations based on cell-partitioned model. We prove theoretically that BWAR does not perform worse than traditional backpressure in terms of the maximum throughput, while yielding a better delay bound. The simulations confirm that BWAR outperforms traditional backpressure at low load, while outperforming a state of the art encounter-routing scheme (Spray and Wait) at high load.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Decentralized Online Learning Algorithms for Opportunistic Spectrum Access

The fundamental problem of multiple secondary users contending for opportunistic spectrum access over multiple channels in cognitive radio networks has been formulated recently as a decentralized multi-armed bandit (D-MAB) problem. In a D-MAB problem there are $M$ users and $N$ arms (channels) that each offer i.i.d. stochastic rewards with unknown means so long as they are accessed without collision. The goal is to design a decentralized online learning policy that incurs minimal regret, defined as the difference between the total expected rewards accumulated by a model-aware genie, and that obtained by all users applying the policy. We make two contributions in this paper. First, we consider the setting where the users have a prioritized ranking, such that it is desired for the $K$-th-ranked user to learn to access the arm offering the $K$-th highest mean reward. For this problem, we present the first distributed policy that yields regret that is uniformly logarithmic over time without requiring any prior assumption about the mean rewards. Second, we consider the case when a fair access policy is required, i.e., it is desired for all users to experience the same mean reward. For this problem, we present a distributed policy that yields order-optimal regret scaling with respect to the number of users and arms, better than previously proposed policies in the literature. Both of our distributed policies make use of an innovative modification of the well known UCB1 policy for the classic multi-armed bandit problem that allows a single user to learn how to play the arm that yields the $K$-th largest mean reward.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Efficient Online Learning for Opportunistic Spectrum Access

The problem of opportunistic spectrum access in cognitive radio networks has been recently formulated as a non-Bayesian restless multi-armed bandit problem. In this problem, there are N arms (corresponding to channels) and one player (corresponding to a secondary user). The state of each arm evolves as a finite-state Markov chain with unknown parameters. At each time slot, the player can select K < N arms to play and receives state-dependent rewards (corresponding to the throughput obtained given the activity of primary users). The objective is to maximize the expected total rewards (i.e., total throughput) obtained over multiple plays. The performance of an algorithm for such a multi-armed bandit problem is measured in terms of regret, defined as the difference in expected reward compared to a model-aware genie who always plays the best K arms. In this paper, we propose a new continuous exploration and exploitation (CEE) algorithm for this problem. When no information is available about the dynamics of the arms, CEE is the first algorithm to guarantee near-logarithmic regret uniformly over time. When some bounds corresponding to the stationary state distributions and the state-dependent rewards are known, we show that CEE can be easily modified to achieve logarithmic regret over time. In contrast, prior algorithms require additional information concerning bounds on the second eigenvalues of the transition matrices in order to guarantee logarithmic regret. Finally, we show through numerical simulations that CEE is more efficient than prior algorithms.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

LIFO-Backpressure Achieves Near Optimal Utility-Delay Tradeoff

There has been considerable recent work developing a new stochastic network utility maximization framework using Backpressure algorithms, also known as MaxWeight. A key open problem has been the development of utility-optimal algorithms that are also delay efficient. In this paper, we show that the Backpressure algorithm, when combined with the LIFO queueing discipline (called LIFO-Backpressure), is able to achieve a utility that is within $O(1/V)$ of the optimal value, while maintaining an average delay of $O([\log(V)]^2)$ for all but a tiny fraction of the network traffic. This result holds for general stochastic network optimization problems and general Markovian dynamics. Remarkably, the performance of LIFO-Backpressure can be achieved by simply changing the queueing discipline; it requires no other modifications of the original Backpressure algorithm. We validate the results through empirical measurements from a sensor network testbed, which show good match between theory and practice.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

On the Combinatorial Multi-Armed Bandit Problem with Markovian Rewards

We consider a combinatorial generalization of the classical multi-armed bandit problem that is defined as follows. There is a given bipartite graph of $M$ users and $N \geq M$ resources. For each user-resource pair $(i,j)$, there is an associated state that evolves as an aperiodic irreducible finite-state Markov chain with unknown parameters, with transitions occurring each time the particular user $i$ is allocated resource $j$. The user $i$ receives a reward that depends on the corresponding state each time it is allocated the resource $j$. The system objective is to learn the best matching of users to resources so that the long-term sum of the rewards received by all users is maximized. This corresponds to minimizing regret, defined here as the gap between the expected total reward that can be obtained by the best-possible static matching and the expected total reward that can be achieved by a given algorithm. We present a polynomial-storage and polynomial-complexity-per-step matching-learning algorithm for this problem. We show that this algorithm can achieve a regret that is uniformly arbitrarily close to logarithmic in time and polynomial in the number of users and resources. This formulation is broadly applicable to scheduling and switching problems in networks and significantly extends prior results in the area.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Online Learning Algorithms for Stochastic Water-Filling

Water-filling is the term for the classic solution to the problem of allocating constrained power to a set of parallel channels to maximize the total data-rate. It is used widely in practice, for example, for power allocation to sub-carriers in multi-user OFDM systems such as WiMax. The classic water-filling algorithm is deterministic and requires perfect knowledge of the channel gain to noise ratios. In this paper we consider how to do power allocation over stochastically time-varying (i.i.d.) channels with unknown gain to noise ratio distributions. We adopt an online learning framework based on stochastic multi-armed bandits. We consider two variations of the problem, one in which the goal is to find a power allocation to maximize $\sum\limits_i \mathbb{E}[\log(1 + SNR_i)]$, and another in which the goal is to find a power allocation to maximize $\sum\limits_i \log(1 + \mathbb{E}[SNR_i])$. For the first problem, we propose a \emph{cognitive water-filling} algorithm that we call CWF1. We show that CWF1 obtains a regret (defined as the cumulative gap over time between the sum-rate obtained by a distribution-aware genie and this policy) that grows polynomially in the number of channels and logarithmically in time, implying that it asymptotically achieves the optimal time-averaged rate that can be obtained when the gain distributions are known. For the second problem, we present an algorithm called CWF2, which is, to our knowledge, the first algorithm in the literature on stochastic multi-armed bandits to exploit non-linear dependencies between the arms. We prove that the number of times CWF2 picks the incorrect power allocation is bounded by a function that is polynomial in the number of channels and logarithmic in time, implying that its frequency of incorrect allocation tends to zero.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Online Learning for Combinatorial Network Optimization with Restless Markovian Rewards

Combinatorial network optimization algorithms that compute optimal structures taking into account edge weights form the foundation for many network protocols. Examples include shortest path routing, minimal spanning tree computation, maximum weighted matching on bipartite graphs, etc. We present CLRMR, the first online learning algorithm that efficiently solves the stochastic version of these problems where the underlying edge weights vary as independent Markov chains with unknown dynamics. The performance of an online learning algorithm is characterized in terms of regret, defined as the cumulative difference in rewards between a suitably-defined genie, and that obtained by the given algorithm. We prove that, compared to a genie that knows the Markov transition matrices and uses the single-best structure at all times, CLRMR yields regret that is polynomial in the number of edges and nearly-logarithmic in time.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

The Non-Bayesian Restless Multi-Armed Bandit: A Case of Near-Logarithmic Strict Regret

In the classic Bayesian restless multi-armed bandit (RMAB) problem, there are $N$ arms, with rewards on all arms evolving at each time as Markov chains with known parameters. A player seeks to activate $K \geq 1$ arms at each time in order to maximize the expected total reward obtained over multiple plays. RMAB is a challenging problem that is known to be PSPACE-hard in general. We consider in this work the even harder non-Bayesian RMAB, in which the parameters of the Markov chain are assumed to be unknown \emph{a priori}. We develop an original approach to this problem that is applicable when the corresponding Bayesian problem has the structure that, depending on the known parameter values, the optimal solution is one of a prescribed finite set of policies. In such settings, we propose to learn the optimal policy for the non-Bayesian RMAB by employing a suitable meta-policy which treats each policy from this finite set as an arm in a different non-Bayesian multi-armed bandit problem for which a single-arm selection policy is optimal. We demonstrate this approach by developing a novel sensing policy for opportunistic spectrum access over unknown dynamic channels. We prove that our policy achieves near-logarithmic regret (the difference in expected reward compared to a model-aware genie), which leads to the same average reward that can be achieved by the optimal policy under a known model. This is the first such result in the literature for a non-Bayesian RMAB. For our proof, we also develop a novel generalization of the Chernoff-Hoeffding bound.

0 citations0 reviewsNo code linkedOpen access
preprint2010arXiv

Combinatorial Network Optimization with Unknown Variables: Multi-Armed Bandits with Linear Rewards

In the classic multi-armed bandits problem, the goal is to have a policy for dynamically operating arms that each yield stochastic rewards with unknown means. The key metric of interest is regret, defined as the gap between the expected total reward accumulated by an omniscient player that knows the reward means for each arm, and the expected total reward accumulated by the given policy. The policies presented in prior work have storage, computation and regret all growing linearly with the number of arms, which is not scalable when the number of arms is large. We consider in this work a broad class of multi-armed bandits with dependent arms that yield rewards as a linear combination of a set of unknown parameters. For this general framework, we present efficient policies that are shown to achieve regret that grows logarithmically with time, and polynomially in the number of unknown parameters (even though the number of dependent arms may grow exponentially). Furthermore, these policies only require storage that grows linearly in the number of unknown parameters. We show that this generalization is broadly applicable and useful for many interesting tasks in networks that can be formulated as tractable combinatorial optimization problems with linear objective functions, such as maximum weight matching, shortest path, and minimum spanning tree computations.

0 citations0 reviewsNo code linkedOpen access
preprint2010arXiv

The Non-Bayesian Restless Multi-Armed Bandit: a Case of Near-Logarithmic Regret

In the classic Bayesian restless multi-armed bandit (RMAB) problem, there are $N$ arms, with rewards on all arms evolving at each time as Markov chains with known parameters. A player seeks to activate $K \geq 1$ arms at each time in order to maximize the expected total reward obtained over multiple plays. RMAB is a challenging problem that is known to be PSPACE-hard in general. We consider in this work the even harder non-Bayesian RMAB, in which the parameters of the Markov chain are assumed to be unknown \emph{a priori}. We develop an original approach to this problem that is applicable when the corresponding Bayesian problem has the structure that, depending on the known parameter values, the optimal solution is one of a prescribed finite set of policies. In such settings, we propose to learn the optimal policy for the non-Bayesian RMAB by employing a suitable meta-policy which treats each policy from this finite set as an arm in a different non-Bayesian multi-armed bandit problem for which a single-arm selection policy is optimal. We demonstrate this approach by developing a novel sensing policy for opportunistic spectrum access over unknown dynamic channels. We prove that our policy achieves near-logarithmic regret (the difference in expected reward compared to a model-aware genie), which leads to the same average reward that can be achieved by the optimal policy under a known model. This is the first such result in the literature for a non-Bayesian RMAB.

0 citations0 reviewsNo code linkedOpen access
preprint2008arXiv

On Myopic Sensing for Multi-Channel Opportunistic Access: Structure, Optimality, and Performance

We consider a multi-channel opportunistic communication system where the states of these channels evolve as independent and statistically identical Markov chains (the Gilbert-Elliot channel model). A user chooses one channel to sense and access in each slot and collects a reward determined by the state of the chosen channel. The problem is to design a sensing policy for channel selection to maximize the average reward, which can be formulated as a multi-arm restless bandit process. In this paper, we study the structure, optimality, and performance of the myopic sensing policy. We show that the myopic sensing policy has a simple robust structure that reduces channel selection to a round-robin procedure and obviates the need for knowing the channel transition probabilities. The optimality of this simple policy is established for the two-channel case and conjectured for the general case based on numerical results. The performance of the myopic sensing policy is analyzed, which, based on the optimality of myopic sensing, characterizes the maximum throughput of a multi-channel opportunistic communication system and its scaling behavior with respect to the number of channels. These results apply to cognitive radio networks, opportunistic transmission in fading environments, and resource-constrained jamming and anti-jamming.

0 citations0 reviewsNo code linkedOpen access