BZPEERPapers, people, evidence and research paths
Menu

Discover

SearchFind papers, people, topics, institutions and opportunities from one place.GraphSee how papers, authors, topics and evidence connect.

Research tools

HomeContinue from the papers, people and topics that matter now.CollectionsTurn saved papers into reading lists, evidence maps and shareable context.ResearchReconstruct the path that led you from search to paper to question.CollaborationMove from a paper, topic or expert request into a focused thread.PublishCreate full-text research articles with metadata, formulas and versions.

Network

ExpertsFind specialists whose work and reviews explain why they can help.CommunitiesJoin research groups organized around papers, topics and shared evidence.

Opportunities

ListingsBrowse roles, calls and collaborations connected to real research context.

Account

Log inReturn to your saved papers, graph views and active threads.Create accountStart saving papers, building a research trail and joining teams.
Log inCreate account

Source author record

Ashish Goel

Ashish Goel appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record
Data Structures and AlgorithmsComputer Science and Game TheorySocial and Information Networkscs.CYDiscrete MathematicsMultiagent SystemsDistributed, Parallel, and Cluster ComputingInformation RetrievalArtificial IntelligenceDatabasesq-fin.TRastro-ph.COastro-ph.IMCryptography and Securityeess.SPMachine Learningphysics.soc-phRobotics

Catalog footprint

What is connected

33works
18topics
4close collaborators

Actions

Connect this record

Log in to claim
Open graphBrowse works

Research graph

See the researcher in context

Open full explorer

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

33 published item(s)

preprint2022arXiv

A Lightweight Space-based Solar Power Generation and Transmission Satellite

We propose a novel design for a lightweight, high-performance space-based solar power array combined with power beaming capability for operation in geosynchronous orbit and transmission of power to Earth. We use a modular configuration of small, repeatable unit cells, called tiles, that each individually perform power collection, conversion, and transmission. Sunlight is collected via lightweight parabolic concentrators and converted to DC electric power with high efficiency III-V photovoltaics. Several CMOS integrated circuits within each tile generates and controls the phase of multiple independently-controlled microwave sources using the DC power. These sources are coupled to multiple radiating antennas which act as elements of a large phased array to beam the RF power to Earth. The power is sent to Earth at a frequency chosen in the range of 1-10 GHz and collected with ground-based rectennas at a local intensity no larger than ambient sunlight. We achieve significantly reduced mass compared to previous designs by taking advantage of solar concentration, current CMOS integrated circuit technology, and ultralight structural elements. Of note, the resulting satellite has no movable parts once it is fully deployed and all beam steering is done electronically. Our design is safe, scalable, and able to be deployed and tested with progressively larger configurations starting with a single unit cell that could fit on a cube satellite. The design reported on here has an areal mass density of 160 g/m2 and an end-to-end efficiency of 7-14%. We believe this is a significant step forward to the realization of space-based solar power, a concept once of science fiction.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Probing the Cosmic Dark Ages with the Lunar Crater Radio Telescope

The Cosmic Dark Ages represent the period in the early evolution of the Universe, starting immediately after the decoupling of CMB photons from matter, and ending with the formation of the first stars and galaxies. The HI signal from the neutral hydrogen atoms is the only mechanism for us to understand this crucial phase in the cosmological history of the Universe and answer fundamental questions about the validity of the standard cosmological model, dark matter physics, and inflation. Due to cosmological redshift, this signal is now only observable in the 3-30~MHz frequency band, which is blocked from reaching the surface of the Earth by the ionosphere. In this paper, we present the design of the Lunar Crater Radio Telescope that intends to carry out unprecedented measurements of this signal by deploying a kilometer-sized parabolic reflector mesh inside a lunar crater on the far side of the Moon and suspending a receiver at its focus.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Advertising for Demographically Fair Outcomes

Online advertising on platforms such as Google or Facebook has become an indispensable outreach tool, including for applications where it is desirable to engage different demographics in an equitable fashion, such as hiring, housing, civic processes, and public health outreach efforts. Somewhat surprisingly, the existing online advertising ecosystem provides very little support for advertising to (and recruiting) a demographically representative cohort. We study the problem of advertising for demographic representativeness from both an empirical and algorithmic perspective. In essence, we seek fairness in the outcome or conversions generated by the advertising campaigns. We first present detailed empirical findings from real-world experiments for recruiting for civic processes, using which we show that methods using Facebook-inferred features are too inaccurate for achieving equity in outcomes, while targeting via custom audiences based on a list of registered voters segmented on known attributes has much superior accuracy. This motivates us to consider the algorithmic question of optimally segmenting the list of individuals with known attributes into a few custom campaigns and allocating budgets to them so that we cost-effectively achieve outcome parity with the population on the maximum possible number of demographics. Under the assumption that a platform can reasonably enforce proportionality in spend across demographics, we present efficient exact and approximation algorithms for this problem. We present simulation results on our datasets to show the efficacy of these algorithms in achieving demographic parity.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Continuous Credit Networks and Layer 2 Blockchains: Monotonicity and Sampling

To improve transaction rates, many cryptocurrencies have implemented so-called ''Layer-2'' transaction protocols, where payments are routed across networks of private payment channels. However, for a given transaction, not every network state provides a feasible route to perform the payment; in this case, the transaction must be put on the public ledger. The payment channel network thus multiplies the transaction rate of the overall system; the less frequently it fails, the higher the multiplier. We build on earlier work on credit networks and show that this network liquidity problem is connected to the combinatorics of graphical matroids. Earlier work could only analyze the (unnatural) scenario where transactions had discrete sizes. Superficially, it might seem like the continuous case would be harder to examine. However, removing this assumption lets us make progress in two important directions. First, we give a partial answer to the ``monotonicity conjecture'' that previous work left open. This conjecture asks that the network's performance not degrade as capacity on any edge increases. And second, we construct here a network state sampling procedure with much faster asymptotic performance than off-the-shelf Markov chains ($O(\vert E\vert β(\vert E\vert))$, where $β(x)$ is the complexity of solving a linear program on $x$ constraints.) We obtain our results by mapping the underlying graphs to convex bodies and then showing that the liquidity and sampling problems reduce to bounding and computing the volumes of these bodies. The transformation relies crucially on the combinatorial properties of the underlying graphic matroid, as do the proofs of monotonicity and fast sampling.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Counteracting Inequality in Markets via Convex Pricing

We study market mechanisms for allocating divisible goods to competing agents with quasilinear utilities. For \emph{linear} pricing (i.e., the cost of a good is proportional to the quantity purchased), the First Welfare Theorem states that Walrasian equilibria maximize the sum of agent valuations. This ensures efficiency, but can lead to extreme inequality across individuals. Many real-world markets -- especially for water -- use \emph{convex} pricing instead, often known as increasing block tariffs (IBTs). IBTs are thought to promote equality, but there is a dearth of theoretical support for this claim. In this paper, we study a simple convex pricing rule and show that the resulting equilibria are guaranteed to maximize a CES welfare function. Furthermore, a parameter of the pricing rule directly determines which CES welfare function is implemented; by tweaking this parameter, the social planner can precisely control the tradeoff between equality and efficiency. Our result holds for any valuations that are homogeneous, differentiable, and concave. We also give an iterative algorithm for computing these pricing rules, derive a truthful mechanism for the case of a single good, and discuss Sybil attacks.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Knapsack Voting for Participatory Budgeting

We address the question of aggregating the preferences of voters in the context of participatory budgeting. We scrutinize the voting method currently used in practice, underline its drawbacks, and introduce a novel scheme tailored to this setting, which we call "Knapsack Voting". We study its strategic properties - we show that it is strategy-proof under a natural model of utility (a dis-utility given by the $\ell_1$ distance between the outcome and the true preference of the voter), and "partially" strategy-proof under general additive utilities. We extend Knapsack Voting to more general settings with revenues, deficits or surpluses, and prove a similar strategy-proofness result. To further demonstrate the applicability of our scheme, we discuss its implementation on the digital voting platform that we have deployed in partnership with the local government bodies in many cities across the nation. From voting data thus collected, we present empirical evidence that Knapsack Voting works well in practice.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Liquidity in Credit Networks with Constrained Agents

In order to scale transaction rates for deployment across the global web, many cryptocurrencies have deployed so-called "Layer-2" networks of private payment channels. An idealized payment network behaves like a Credit Network, a model for transactions across a network of bilateral trust relationships. Credit Networks capture many aspects of traditional currencies as well as new virtual currencies and payment mechanisms. In the traditional credit network model, if an agent defaults, every other node that trusted it is vulnerable to loss. In a cryptocurrency context, trust is manufactured by capital deposits, and thus there arises a natural tradeoff between network liquidity (i.e. the fraction of transactions that succeed) and the cost of capital deposits. In this paper, we introduce constraints that bound the total amount of loss that the rest of the network can suffer if an agent (or a set of agents) were to default - equivalently, how the network changes if agents can support limited solvency guarantees. We show that these constraints preserve the analytical structure of a credit network. Furthermore, we show that aggregate borrowing constraints greatly simplify the network structure and in the payment network context achieve the optimal tradeoff between liquidity and amount of escrowed capital.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Random Dictators with a Random Referee: Constant Sample Complexity Mechanisms for Social Choice

We study social choice mechanisms in an implicit utilitarian framework with a metric constraint, where the goal is to minimize \textit{Distortion}, the worst case social cost of an ordinal mechanism relative to underlying cardinal utilities. We consider two additional desiderata: Constant sample complexity and Squared Distortion. Constant sample complexity means that the mechanism (potentially randomized) only uses a constant number of ordinal queries regardless of the number of voters and alternatives. Squared Distortion is a measure of variance of the Distortion of a randomized mechanism. Our primary contribution is the first social choice mechanism with constant sample complexity \textit{and} constant Squared Distortion (which also implies constant Distortion). We call the mechanism Random Referee, because it uses a random agent to compare two alternatives that are the favorites of two other random agents. We prove that the use of a comparison query is necessary: no mechanism that only elicits the top-k preferred alternatives of voters (for constant k) can have Squared Distortion that is sublinear in the number of alternatives. We also prove that unlike any top-k only mechanism, the Distortion of Random Referee meaningfully improves on benign metric spaces, using the Euclidean plane as a canonical example. Finally, among top-1 only mechanisms, we introduce Random Oligarchy. The mechanism asks just 3 queries and is essentially optimal among the class of such mechanisms with respect to Distortion. In summary, we demonstrate the surprising power of constant sample complexity mechanisms generally, and just three random voters in particular, to provide some of the best known results in the implicit utilitarian framework.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Satellite Navigation for the Age of Autonomy

Global Navigation Satellite Systems (GNSS) brought navigation to the masses. Coupled with smartphones, the blue dot in the palm of our hands has forever changed the way we interact with the world. Looking forward, cyber-physical systems such as self-driving cars and aerial mobility are pushing the limits of what localization technologies including GNSS can provide. This autonomous revolution requires a solution that supports safety-critical operation, centimeter positioning, and cyber-security for millions of users. To meet these demands, we propose a navigation service from Low Earth Orbiting (LEO) satellites which deliver precision in-part through faster motion, higher power signals for added robustness to interference, constellation autonomous integrity monitoring for integrity, and encryption / authentication for resistance to spoofing attacks. This paradigm is enabled by the 'New Space' movement, where highly capable satellites and components are now built on assembly lines and launch costs have decreased by more than tenfold. Such a ubiquitous positioning service enables a consistent and secure standard where trustworthy information can be validated and shared, extending the electronic horizon from sensor line of sight to an entire city. This enables the situational awareness needed for true safe operation to support autonomy at scale.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

The impossibility of low rank representations for triangle-rich complex networks

The study of complex networks is a significant development in modern science, and has enriched the social sciences, biology, physics, and computer science. Models and algorithms for such networks are pervasive in our society, and impact human behavior via social networks, search engines, and recommender systems to name a few. A widely used algorithmic technique for modeling such complex networks is to construct a low-dimensional Euclidean embedding of the vertices of the network, where proximity of vertices is interpreted as the likelihood of an edge. Contrary to the common view, we argue that such graph embeddings do not}capture salient properties of complex networks. The two properties we focus on are low degree and large clustering coefficients, which have been widely established to be empirically true for real-world networks. We mathematically prove that any embedding (that uses dot products to measure similarity) that can successfully create these two properties must have rank nearly linear in the number of vertices. Among other implications, this establishes that popular embedding techniques such as Singular Value Decomposition and node2vec fail to capture significant structural aspects of real-world complex networks. Furthermore, we empirically study a number of different embedding techniques based on dot product, and show that they all fail to capture the triangle structure.

0 citations0 reviewsNo code linkedOpen access
preprint2017arXiv

Metric Distortion of Social Choice Rules: Lower Bounds and Fairness Properties

We study social choice rules under the utilitarian distortion framework, with an additional metric assumption on the agents' costs over the alternatives. In this approach, these costs are given by an underlying metric on the set of all agents plus alternatives. Social choice rules have access to only the ordinal preferences of agents but not the latent cardinal costs that induce them. Distortion is then defined as the ratio between the social cost (typically the sum of agent costs) of the alternative chosen by the mechanism at hand, and that of the optimal alternative chosen by an omniscient algorithm. The worst-case distortion of a social choice rule is, therefore, a measure of how close it always gets to the optimal alternative without any knowledge of the underlying costs. Under this model, it has been conjectured that Ranked Pairs, the well-known weighted-tournament rule, achieves a distortion of at most 3 [Anshelevich et al. 2015]. We disprove this conjecture by constructing a sequence of instances which shows that the worst-case distortion of Ranked Pairs is at least 5. Our lower bound on the worst case distortion of Ranked Pairs matches a previously known upper bound for the Copeland rule, proving that in the worst case, the simpler Copeland rule is at least as good as Ranked Pairs. And as long as we are limited to (weighted or unweighted) tournament rules, we demonstrate that randomization cannot help achieve an expected worst-case distortion of less than 3. Using the concept of approximate majorization within the distortion framework, we prove that Copeland and Randomized Dictatorship achieve low constant factor fairness-ratios (5 and 3 respectively), which is a considerable generalization of similar results for the sum of costs and single largest cost objectives. In addition to all of the above, we outline several interesting directions for further research in this space.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Re-incentivizing Discovery: Mechanisms for Partial-Progress Sharing in Research

An essential primitive for an efficient research ecosystem is \emph{partial-progress sharing} (PPS) -- whereby a researcher shares information immediately upon making a breakthrough. This helps prevent duplication of work; however there is evidence that existing reward structures in research discourage partial-progress sharing. Ensuring PPS is especially important for new online collaborative-research platforms, which involve many researchers working on large, multi-stage problems. We study the problem of incentivizing information-sharing in research, under a stylized model: non-identical agents work independently on subtasks of a large project, with dependencies between subtasks captured via an acyclic subtask-network. Each subtask carries a reward, given to the first agent who publicly shares its solution. Agents can choose which subtasks to work on, and more importantly, when to reveal solutions to completed subtasks. Under this model, we uncover the strategic rationale behind certain anecdotal phenomena. Moreover, for any acyclic subtask-network, and under a general model of agent-subtask completion times, we give sufficient conditions that ensure PPS is incentive-compatible for all agents. One surprising finding is that rewards which are approximately proportional to perceived task-difficulties are sufficient to ensure PPS in all acyclic subtask-networks. The fact that there is no tension between local fairness and global information-sharing in multi-stage projects is encouraging, as it suggests practical mechanisms for real-world settings. Finally, we show that PPS is necessary, and in many cases, sufficient, to ensure a high rate of progress in research.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

The Core of the Participatory Budgeting Problem

In participatory budgeting, communities collectively decide on the allocation of public tax dollars for local public projects. In this work, we consider the question of fairly aggregating the preferences of community members to determine an allocation of funds to projects. This problem is different from standard fair resource allocation because of public goods: The allocated goods benefit all users simultaneously. Fairness is crucial in participatory decision making, since generating equitable outcomes is an important goal of these processes. We argue that the classic game theoretic notion of core captures fairness in the setting. To compute the core, we first develop a novel characterization of a public goods market equilibrium called the Lindahl equilibrium, which is always a core solution. We then provide the first (to our knowledge) polynomial time algorithm for computing such an equilibrium for a broad set of utility functions; our algorithm also generalizes (in a non-trivial way) the well-known concept of proportional fairness. We use our theoretical insights to perform experiments on real participatory budgeting voting data. We empirically show that the core can be efficiently computed for utility functions that naturally model our practical setting, and examine the relation of the core with the familiar welfare objective. Finally, we address concerns of incentives and mechanism design by developing a randomized approximately dominant-strategy truthful mechanism building on the exponential mechanism from differential privacy.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

The Ratio Index for Budgeted Learning, with Applications

In the budgeted learning problem, we are allowed to experiment on a set of alternatives (given a fixed experimentation budget) with the goal of picking a single alternative with the largest possible expected payoff. Approximation algorithms for this problem were developed by Guha and Munagala by rounding a linear program that couples the various alternatives together. In this paper we present an index for this problem, which we call the ratio index, which also guarantees a constant factor approximation. Index-based policies have the advantage that a single number (i.e. the index) can be computed for each alternative irrespective of all other alternatives, and the alternative with the highest index is experimented upon. This is analogous to the famous Gittins index for the discounted multi-armed bandit problem. The ratio index has several interesting structural properties. First, we show that it can be computed in strongly polynomial time. Second, we show that with the appropriate discount factor, the Gittins index and our ratio index are constant factor approximations of each other, and hence the Gittins index also gives a constant factor approximation to the budgeted learning problem. Finally, we show that the ratio index can be used to create an index-based policy that achieves an O(1)-approximation for the finite horizon version of the multi-armed bandit problem. Moreover, the policy does not require any knowledge of the horizon (whereas we compare its performance against an optimal strategy that is aware of the horizon). This yields the following surprising result: there is an index-based policy that achieves an O(1)-approximation for the multi-armed bandit problem, oblivious to the underlying discount factor.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Towards large-scale deliberative decision-making: small groups and the importance of triads

Though deliberation is a critical component of democratic decision-making, existing deliberative processes do not scale to large groups of people. Motivated by this, we propose a model in which large-scale decision-making takes place through a sequence of small group interactions. Our model considers a group of participants, each having an opinion which together form a graph. We show that for median graphs, a class of graphs including grids and trees, it is possible to use a small number of three-person interactions to tightly approximate the wisdom of the crowd, defined here to be the generalized median of participant opinions, even when agents are strategic. Interestingly, we also show that this sharply contrasts with small groups of size two, for which we prove an impossibility result. Specifically, we show that it is impossible to use sequences of two-person interactions satisfying natural axioms to find a tight approximation of the generalized median, even when agents are non-strategic. Our results demonstrate the potential of small group interactions for reaching global decision-making properties.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Bidirectional PageRank Estimation: From Average-Case to Worst-Case

We present a new algorithm for estimating the Personalized PageRank (PPR) between a source and target node on undirected graphs, with sublinear running-time guarantees over the worst-case choice of source and target nodes. Our work builds on a recent line of work on bidirectional estimators for PPR, which obtained sublinear running-time guarantees but in an average-case sense, for a uniformly random choice of target node. Crucially, we show how the reversibility of random walks on undirected networks can be exploited to convert average-case to worst-case guarantees. While past bidirectional methods combine forward random walks with reverse local pushes, our algorithm combines forward local pushes with reverse random walks. We also discuss how to modify our methods to estimate random-walk probabilities for any length distribution, thereby obtaining fast algorithms for estimating general graph diffusions, including the heat kernel, on undirected networks.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Personalized PageRank Estimation and Search: A Bidirectional Approach

We present new algorithms for Personalized PageRank estimation and Personalized PageRank search. First, for the problem of estimating Personalized PageRank (PPR) from a source distribution to a target node, we present a new bidirectional estimator with simple yet strong guarantees on correctness and performance, and 3x to 8x speedup over existing estimators in experiments on a diverse set of networks. Moreover, it has a clean algebraic structure which enables it to be used as a primitive for the Personalized PageRank Search problem: Given a network like Facebook, a query like "people named John", and a searching user, return the top nodes in the network ranked by PPR from the perspective of the searching user. Previous solutions either score all nodes or score candidate nodes one at a time, which is prohibitively slow for large candidate sets. We develop a new algorithm based on our bidirectional PPR estimator which identifies the most relevant results by sampling candidates based on their PPR; this is the first solution to PPR search that can find the best results without iterating through the set of all candidate results. Finally, by combining PPR sampling with sequential PPR estimation and Monte Carlo, we develop practical algorithms for PPR search, and we show via experiments that our algorithms are efficient on networks with billions of edges.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Crowdsourcing for Participatory Democracies: Efficient Elicitation of Social Choice Functions

We present theoretical and empirical results demonstrating the usefulness of voting rules for participatory democracies. We first give algorithms which efficiently elicit ε-approximations to two prominent voting rules: the Borda rule and the Condorcet winner. This result circumvents previous prohibitive lower bounds and is surprisingly strong: even if the number of ideas is as large as the number of participants, each participant will only have to make a logarithmic number of comparisons, an exponential improvement over the linear number of comparisons previously needed. We demonstrate the approach in an experiment in Finland's recent off-road traffic law reform, observing that the total number of comparisons needed to achieve a fixed εapproximation is linear in the number of ideas and that the constant is not large. Finally, we note a few other experimental observations which support the use of voting rules for aggregation. First, we observe that rating, one of the common alternatives to ranking, manifested effects of bias in our data. Second, we show that very few of the topics lacked a Condorcet winner, one of the prominent negative results in voting. Finally, we show data hinting at a potential future direction: the use of partial rankings as opposed to pairwise comparisons to further decrease the elicitation time.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

FAST-PPR: Scaling Personalized PageRank Estimation for Large Graphs

We propose a new algorithm, FAST-PPR, for estimating personalized PageRank: given start node $s$ and target node $t$ in a directed graph, and given a threshold $δ$, FAST-PPR estimates the Personalized PageRank $π_s(t)$ from $s$ to $t$, guaranteeing a small relative error as long $π_s(t)>δ$. Existing algorithms for this problem have a running-time of $Ω(1/δ)$; in comparison, FAST-PPR has a provable average running-time guarantee of ${O}(\sqrt{d/δ})$ (where $d$ is the average in-degree of the graph). This is a significant improvement, since $δ$ is often $O(1/n)$ (where $n$ is the number of nodes) for applications. We also complement the algorithm with an $Ω(1/\sqrtδ)$ lower bound for PageRank estimation, showing that the dependence on $δ$ cannot be improved. We perform a detailed empirical study on numerous massive graphs, showing that FAST-PPR dramatically outperforms existing algorithms. For example, on the 2010 Twitter graph with 1.5 billion edges, for target nodes sampled by popularity, FAST-PPR has a $20$ factor speedup over the state of the art. Furthermore, an enhanced version of FAST-PPR has a $160$ factor speedup on the Twitter graph, and is at least $20$ times faster on all our candidate graphs.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Personalized PageRank to a Target Node

Personalalized PageRank uses random walks to determine the importance or authority of nodes in a graph from the point of view of a given source node. Much past work has considered how to compute personalized PageRank from a given source node to other nodes. In this work we consider the problem of computing personalized PageRanks to a given target node from all source nodes. This problem can be interpreted as finding who supports the target or who is interested in the target. We present an efficient algorithm for computing personalized PageRank to a given target up to any given accuracy. We give a simple analysis of our algorithm's running time in both the average case and the parameterized worst-case. We show that for any graph with $n$ nodes and $m$ edges, if the target node is randomly chosen and the teleport probability $α$ is given, the algorithm will compute a result with $ε$ error in time $O\left(\frac{1}{αε} \left(\frac{m}{n} + \log(n)\right)\right)$. This is much faster than the previously proposed method of computing personalized PageRank separately from every source node, and it is comparable to the cost of computing personalized PageRank from a single source. We present results from experiments on the Twitter graph which show that the constant factors in our running time analysis are small and our algorithm is efficient in practice.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

Dimension Independent Similarity Computation

We present a suite of algorithms for Dimension Independent Similarity Computation (DISCO) to compute all pairwise similarities between very high dimensional sparse vectors. All of our results are provably independent of dimension, meaning apart from the initial cost of trivially reading in the data, all subsequent operations are independent of the dimension, thus the dimension can be very large. We study Cosine, Dice, Overlap, and the Jaccard similarity measures. For Jaccard similiarity we include an improved version of MinHash. Our results are geared toward the MapReduce framework. We empirically validate our theorems at large scale using data from the social networking site Twitter. At time of writing, our algorithms are live in production at twitter.com.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Biased Assimilation, Homophily and the Dynamics of Polarization

Are we as a society getting more polarized, and if so, why? We try to answer this question through a model of opinion formation. Empirical studies have shown that homophily results in polarization. However, we show that DeGroot's well-known model of opinion formation based on repeated averaging can never be polarizing, even if individuals are arbitrarily homophilous. We generalize DeGroot's model to account for a phenomenon well-known in social psychology as biased assimilation: when presented with mixed or inconclusive evidence on a complex issue, individuals draw undue support for their initial position thereby arriving at a more extreme opinion. We show that in a simple model of homophilous networks, our biased opinion formation process results in either polarization, persistent disagreement or consensus depending on how biased individuals are. In other words, homophily alone, without biased assimilation, is not sufficient to polarize society. Quite interestingly, biased assimilation also provides insight into the following related question: do internet based recommender algorithms that show us personalized content contribute to polarization? We make a connection between biased assimilation and the polarizing effects of some random-walk based recommender algorithms that are similar in spirit to some commonly used recommender algorithms.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Complexity Measures for Map-Reduce, and Comparison to Parallel Computing

The programming paradigm Map-Reduce and its main open-source implementation, Hadoop, have had an enormous impact on large scale data processing. Our goal in this expository writeup is two-fold: first, we want to present some complexity measures that allow us to talk about Map-Reduce algorithms formally, and second, we want to point out why this model is actually different from other models of parallel programming, most notably the PRAM (Parallel Random Access Memory) model. We are looking for complexity measures that are detailed enough to make fine-grained distinction between different algorithms, but which also abstract away many of the implementation details.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Efficient Distributed Locality Sensitive Hashing

Distributed frameworks are gaining increasingly widespread use in applications that process large amounts of data. One important example application is large scale similarity search, for which Locality Sensitive Hashing (LSH) has emerged as the method of choice, specially when the data is high-dimensional. At its core, LSH is based on hashing the data points to a number of buckets such that similar points are more likely to map to the same buckets. To guarantee high search quality, the LSH scheme needs a rather large number of hash tables. This entails a large space requirement, and in the distributed setting, with each query requiring a network call per hash bucket look up, this also entails a big network load. The Entropy LSH scheme proposed by Panigrahy significantly reduces the number of required hash tables by looking up a number of query offsets in addition to the query itself. While this improves the LSH space requirement, it does not help with (and in fact worsens) the search network efficiency, as now each query offset requires a network call. In this paper, focusing on the Euclidian space under $l_2$ norm and building up on Entropy LSH, we propose the distributed Layered LSH scheme, and prove that it exponentially decreases the network cost, while maintaining a good load balance between different machines. Our experiments also verify that our scheme results in a significant network traffic reduction that brings about large runtime improvement in real world applications.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Liquidity in Credit Networks: A Little Trust Goes a Long Way

Credit networks represent a way of modeling trust between entities in a network. Nodes in the network print their own currency and trust each other for a certain amount of each other's currency. This allows the network to serve as a decentralized payment infrastructure---arbitrary payments can be routed through the network by passing IOUs between trusting nodes in their respective currencies---and obviates the need for a common currency. Nodes can repeatedly transact with each other and pay for the transaction using trusted currency. A natural question to ask in this setting is: how long can the network sustain liquidity, i.e., how long can the network support the routing of payments before credit dries up? We answer this question in terms of the long term failure probability of transactions for various network topologies and credit values. We prove that the transaction failure probability is independent of the path along which transactions are routed. We show that under symmetric transaction rates, the transaction failure probability in a number of well-known graph families goes to zero as the size, density or credit capacity of the network increases. We also show via simulations that even networks of small size and credit capacity can route transactions with high probability if they are well-connected. Further, we characterize a centralized currency system as a special type of a star network (one where edges to the root have infinite credit capacity, and transactions occur only between leaf nodes) and compute the steady-state transaction failure probability in a centralized system. We show that liquidity in star networks, complete graphs and Erdös-Rényi networks is comparable to that in equivalent centralized currency systems; thus we do not lose much liquidity in return for their robustness and decentralized properties.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Single pass sparsification in the streaming model with edge deletions

In this paper we give a construction of cut sparsifiers of Benczur and Karger in the {\em dynamic} streaming setting in a single pass over the data stream. Previous constructions either required multiple passes or were unable to handle edge deletions. We use $\tilde{O}(1/\e^2)$ time for each stream update and $\tilde{O}(n/\e^2)$ time to construct a sparsifier. Our $\e$-sparsifiers have $O(n\log^3 n/\e^2)$ edges. The main tools behind our result are an application of sketching techniques of Ahn et al.[SODA'12] to estimate edge connectivity together with a novel application of sampling with limited independence and sparse recovery to produce the edges of the sparsifier.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Triadic Consensus: A Randomized Algorithm for Voting in a Crowd

Typical voting rules do not work well in settings with many candidates. If there are just several hundred candidates, then even a simple task such as choosing a top candidate becomes impractical. Motivated by the hope of developing group consensus mechanisms over the internet, where the numbers of candidates could easily number in the thousands, we study an urn-based voting rule where each participant acts as a voter and a candidate. We prove that when participants lie in a one-dimensional space, this voting protocol finds a $(1-ε/sqrt{n})$ approximation of the Condorcet winner with high probability while only requiring an expected $O(\frac{1}{ε^2}\log^2 \frac{n}{ε^2})$ comparisons on average per voter. Moreover, this voting protocol is shown to have a quasi-truthful Nash equilibrium: namely, a Nash equilibrium exists which may not be truthful, but produces a winner with the same probability distribution as that of the truthful strategy.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Widescope - A social platform for serious conversations on the Web

There are several web platforms that people use to interact and exchange ideas, such as social networks like Facebook, Twitter, and Google+; Q&A sites like Quora and Yahoo! Answers; and myriad independent fora. However, there is a scarcity of platforms that facilitate discussion of complex subjects where people with divergent views can easily rationalize their points of view using a shared knowledge base, and leverage it towards shared objectives, e.g. to arrive at a mutually acceptable compromise. In this paper, as a first step, we present Widescope, a novel collaborative web platform for catalyzing shared understanding of the US Federal and State budget debates in order to help users reach data-driven consensus about the complex issues involved. It aggregates disparate sources of financial data from different budgets (i.e. from past, present, and proposed) and presents a unified interface using interactive visualizations. It leverages distributed collaboration to encourage exploration of ideas and debate. Users can propose budgets ab-initio, support existing proposals, compare between different budgets, and collaborate with others in real time. We hypothesize that such a platform can be useful in bringing people's thoughts and opinions closer. Toward this, we present preliminary evidence from a simple pilot experiment, using triadic voting (which we also formally analyze to show that is better than hot-or-not voting), that 5 out of 6 groups of users with divergent views (conservatives vs liberals) come to a consensus while aiming to halve the deficit using Widescope. We believe that tools like Widescope could have a positive impact on other complex, data-driven social issues.

0 citations0 reviewsNo code linkedOpen access
preprint2010arXiv

Fast Incremental and Personalized PageRank

In this paper, we analyze the efficiency of Monte Carlo methods for incremental computation of PageRank, personalized PageRank, and similar random walk based methods (with focus on SALSA), on large-scale dynamically evolving social networks. We assume that the graph of friendships is stored in distributed shared memory, as is the case for large social networks such as Twitter. For global PageRank, we assume that the social network has $n$ nodes, and $m$ adversarially chosen edges arrive in a random order. We show that with a reset probability of $ε$, the total work needed to maintain an accurate estimate (using the Monte Carlo method) of the PageRank of every node at all times is $O(\frac{n\ln m}{ε^{2}})$. This is significantly better than all known bounds for incremental PageRank. For instance, if we naively recompute the PageRanks as each edge arrives, the simple power iteration method needs $Ω(\frac{m^2}{\ln(1/(1-ε))})$ total time and the Monte Carlo method needs $O(mn/ε)$ total time; both are prohibitively expensive. Furthermore, we also show that we can handle deletions equally efficiently. We then study the computation of the top $k$ personalized PageRanks starting from a seed node, assuming that personalized PageRanks follow a power-law with exponent $α< 1$. We show that if we store $R>q\ln n$ random walks starting from every node for large enough constant $q$ (using the approach outlined for global PageRank), then the expected number of calls made to the distributed social network database is $O(k/(R^{(1-α)/α}))$. We also present experimental results from the social networking site, Twitter, verifying our assumptions and analyses. The overall result is that this algorithm is fast enough for real-time queries over a dynamic social network.

0 citations0 reviewsNo code linkedOpen access
preprint2010arXiv

Graph Sparsification via Refinement Sampling

A graph G'(V,E') is an \eps-sparsification of G for some \eps>0, if every (weighted) cut in G' is within (1\pm \eps) of the corresponding cut in G. A celebrated result of Benczur and Karger shows that for every undirected graph G, an \eps-sparsification with O(n\log n/\e^2) edges can be constructed in O(m\log^2n) time. Applications to modern massive data sets often constrain algorithms to use computation models that restrict random access to the input. The semi-streaming model, in which the algorithm is constrained to use \tilde O(n) space, has been shown to be a good abstraction for analyzing graph algorithms in applications to large data sets. Recently, a semi-streaming algorithm for graph sparsification was presented by Anh and Guha; the total running time of their implementation is Ω(mn), too large for applications where both space and time are important. In this paper, we introduce a new technique for graph sparsification, namely refinement sampling, that gives an \tilde{O}(m) time semi-streaming algorithm for graph sparsification. Specifically, we show that refinement sampling can be used to design a one-pass streaming algorithm for sparsification that takes O(\log\log n) time per edge, uses O(\log^2 n) space per node, and outputs an \eps-sparsifier with O(n\log^3 n/\eps^2) edges. At a slightly increased space and time complexity, we can reduce the sparsifier size to O(n \log n/\e^2) edges matching the Benczur-Karger result, while improving upon the Benczur-Karger runtime for m=ω(n\log^3 n). Finally, we show that an \eps-sparsifier with O(n \log n/\eps^2) edges can be constructed in two passes over the data and O(m) time whenever m =Ω(n^{1+δ}) for some constant δ>0. As a by-product of our approach, we also obtain an O(m\log\log n+n \log n) time streaming algorithm to compute a sparse k-connectivity certificate of a graph.

0 citations0 reviewsNo code linkedOpen access
preprint2010arXiv

One Tree Suffices: A Simultaneous O(1)-Approximation for Single-Sink Buy-at-Bulk

We study the single-sink buy-at-bulk problem with an unknown cost function. We wish to route flow from a set of demand nodes to a root node, where the cost of routing x total flow along an edge is proportional to f(x) for some concave, non-decreasing function f satisfying f(0)=0. We present a simple, fast, combinatorial algorithm that takes a set of demands and constructs a single tree T such that for all f the cost f(T) is a 47.45-approximation of the optimal cost for that f. This is within a factor of 2.33 of the best approximation ratio currently achievable when the tree can be optimized for a specific function. Trees achieving simultaneous O(1)-approximations for all concave functions were previously not known to exist regardless of computation time.

0 citations0 reviewsNo code linkedOpen access
preprint2010arXiv

Perfect Matchings in O(n \log n) Time in Regular Bipartite Graphs

In this paper we consider the well-studied problem of finding a perfect matching in a d-regular bipartite graph on 2n nodes with m=nd edges. The best-known algorithm for general bipartite graphs (due to Hopcroft and Karp) takes time O(m\sqrt{n}). In regular bipartite graphs, however, a matching is known to be computable in O(m) time (due to Cole, Ost and Schirra). In a recent line of work by Goel, Kapralov and Khanna the O(m) time algorithm was improved first to \tilde O(min{m, n^{2.5}/d}) and then to \tilde O(min{m, n^2/d}). It was also shown that the latter algorithm is optimal up to polylogarithmic factors among all algorithms that use non-adaptive uniform sampling to reduce the size of the graph as a first step. In this paper, we give a randomized algorithm that finds a perfect matching in a d-regular graph and runs in O(n\log n) time (both in expectation and with high probability). The algorithm performs an appropriately truncated random walk on a modified graph to successively find augmenting paths. Our algorithm may be viewed as using adaptive uniform sampling, and is thus able to bypass the limitations of (non-adaptive) uniform sampling established in earlier work. We also show that randomization is crucial for obtaining o(nd) time algorithms by establishing an Ω(nd) lower bound for any deterministic algorithm. Our techniques also give an algorithm that successively finds a matching in the support of a doubly stochastic matrix in expected time O(n\log^2 n) time, with O(m) pre-processing time; this gives a simple O(m+mn\log^2 n) time algorithm for finding the Birkhoff-von Neumann decomposition of a doubly stochastic matrix.

0 citations0 reviewsNo code linkedOpen access
preprint2010arXiv

Similarity Search and Locality Sensitive Hashing using TCAMs

Similarity search methods are widely used as kernels in various machine learning applications. Nearest neighbor search (NNS) algorithms are often used to retrieve similar entries, given a query. While there exist efficient techniques for exact query lookup using hashing, similarity search using exact nearest neighbors is known to be a hard problem and in high dimensions, best known solutions offer little improvement over a linear scan. Fast solutions to the approximate NNS problem include Locality Sensitive Hashing (LSH) based techniques, which need storage polynomial in $n$ with exponent greater than $1$, and query time sublinear, but still polynomial in $n$, where $n$ is the size of the database. In this work we present a new technique of solving the approximate NNS problem in Euclidean space using a Ternary Content Addressable Memory (TCAM), which needs near linear space and has O(1) query time. In fact, this method also works around the best known lower bounds in the cell probe model for the query time using a data structure near linear in the size of the data base. TCAMs are high performance associative memories widely used in networking applications such as access control lists. A TCAM can query for a bit vector within a database of ternary vectors, where every bit position represents $0$, $1$ or $*$. The $*$ is a wild card representing either a $0$ or a $1$. We leverage TCAMs to design a variant of LSH, called Ternary Locality Sensitive Hashing (TLSH) wherein we hash database entries represented by vectors in the Euclidean space into $\{0,1,*\}$. By using the added functionality of a TLSH scheme with respect to the $*$ character, we solve an instance of the approximate nearest neighbor problem with 1 TCAM access and storage nearly linear in the size of the database. We believe that this work can open new avenues in very high speed data mining.

0 citations0 reviewsNo code linkedOpen access