Source author record

Vivek Borkar

Vivek Borkar appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record

Machine Learning math.OC Systems and Control Computer Science and Game Theory Information Retrieval math.PR Networking and Internet Architecture

Catalog footprint

What is connected

5works

7topics

4close collaborators

Actions

Connect this record

Open graph Browse works

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

preprint2016arXiv

Approachability in Stackelberg Stochastic Games with Vector Costs

The notion of approachability was introduced by Blackwell [1] in the context of vector-valued repeated games. The famous Blackwell's approachability theorem prescribes a strategy for approachability, i.e., for `steering' the average cost of a given agent towards a given target set, irrespective of the strategies of the other agents. In this paper, motivated by the multi-objective optimization/decision making problems in dynamically changing environments, we address the approachability problem in Stackelberg stochastic games with vector valued cost functions. We make two main contributions. Firstly, we give a simple and computationally tractable strategy for approachability for Stackelberg stochastic games along the lines of Blackwell's. Secondly, we give a reinforcement learning algorithm for learning the approachable strategy when the transition kernel is unknown. We also recover as a by-product Blackwell's necessary and sufficient condition for approachability for convex sets in this set up and thus a complete characterization. We also give sufficient conditions for non-convex sets.

preprint2016arXiv

On the fastest finite Markov processes

Consider a finite irreducible Markov chain with invariant probability $π$. Define its inverse communication speed as the expectation to go from x to y, when x, y are sampled independently according to $π$. In the discrete time setting and when $π$ is the uniform distribution $\upsilon$, Litvak and Ejov have shown that the permutation matrices associated to Hamiltonian cycles are the fastest Markov chains. Here we prove (A) that the above optimality is with respect to all processes compatible with a fixed graph of permitted transitions (assuming that it does contain a Hamiltonian cycle), not only the Markov chains, and, (B) that this result admits a natural extension in both discrete and continuous time when $π$ is close to $\upsilon$: the fastest Markov chains/processes are those moving successively on the points of a Hamiltonian cycle, with transition probabilities/jump rates dictated by $π$. Nevertheless, the claim is no longer true when $π$ is significantly different from $\upsilon$.

preprint2015arXiv

Parallel and Distributed Approaches for Graph Based Semi-supervised Learning

Two approaches for graph based semi-supervised learning are proposed. The firstapproach is based on iteration of an affine map. A key element of the affine map iteration is sparsematrix-vector multiplication, which has several very efficient parallel implementations. The secondapproach belongs to the class of Markov Chain Monte Carlo (MCMC) algorithms. It is based onsampling of nodes by performing a random walk on the graph. The latter approach is distributedby its nature and can be easily implemented on several processors or over the network. Boththeoretical and practical evaluations are provided. It is found that the nodes are classified intotheir class with very small error. The sampling algorithm's ability to track new incoming nodesand to classify them is also demonstrated.

preprint2015arXiv

Whittle Index Policy for Crawling Ephemeral Content

We consider a task of scheduling a crawler to retrieve content from several sites with ephemeral content. A user typically loses interest in ephemeral content, like news or posts at social network groups, after several days or hours. Thus, development of timely crawling policy for such ephemeral information sources is very important. We first formulate this problem as an optimal control problem with average reward. The reward can be measured in the number of clicks or relevant search requests. The problem in its initial formulation suffers from the curse of dimensionality and quickly becomes intractable even with moderate number of information sources. Fortunately, this problem admits a Whittle index, which leads to problem decomposition and to a very simple and efficient crawling policy. We derive the Whittle index and provide its theoretical justification.

preprint2013arXiv

Model-based clock synchronization protocol for wireless sensor networks

In a wireless sensor network, nodes communicate time-stamped packets in order to synchronize their clocks, i.e., estimate each other's time display. We introduce and analyze a parametrized stochastic model for clocks and use it to calculate, at any given time, the Maximum-Likelihood (ML) estimate of the relative skew/offset between two communicating nodes. For network synchronization, event-based Kalman-Bucy filtering gives rise to a centralized scheme, and we propose an efficient distributed suboptimal algorithm. We study the performance both analytically and experimentally and provide provable guarantees. We summarize our findings into defining a new distributed model-based clock synchronization protocol (MBCSP), and present a comparative simulation study of its accuracy versus prior art to showcase improvements.