BZPEERPapers, people, evidence and research paths
Menu

Discover

SearchFind papers, people, topics, institutions and opportunities from one place.GraphSee how papers, authors, topics and evidence connect.

Research tools

HomeContinue from the papers, people and topics that matter now.CollectionsTurn saved papers into reading lists, evidence maps and shareable context.ResearchReconstruct the path that led you from search to paper to question.CollaborationMove from a paper, topic or expert request into a focused thread.PublishCreate full-text research articles with metadata, formulas and versions.

Network

ExpertsFind specialists whose work and reviews explain why they can help.CommunitiesJoin research groups organized around papers, topics and shared evidence.

Opportunities

ListingsBrowse roles, calls and collaborations connected to real research context.

Account

Log inReturn to your saved papers, graph views and active threads.Create accountStart saving papers, building a research trail and joining teams.
Log inCreate account

Source author record

Amanda Bower

Amanda Bower appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record
Machine Learningmath.NTArtificial Intelligencecs.CYInformation Retrievalmath.COMethodologySocial and Information Networks

Catalog footprint

What is connected

8works
8topics
4close collaborators

Actions

Connect this record

Log in to claim
Open graphBrowse works

Research graph

See the researcher in context

Open full explorer

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

8 published item(s)

preprint2022arXiv

De-biasing "bias" measurement

When a model's performance differs across socially or culturally relevant groups--like race, gender, or the intersections of many such groups--it is often called "biased." While much of the work in algorithmic fairness over the last several years has focused on developing various definitions of model fairness (the absence of group-wise model performance disparities) and eliminating such "bias," much less work has gone into rigorously measuring it. In practice, it important to have high quality, human digestible measures of model performance disparities and associated uncertainty quantification about them that can serve as inputs into multi-faceted decision-making processes. In this paper, we show both mathematically and through simulation that many of the metrics used to measure group-wise model performance disparities are themselves statistically biased estimators of the underlying quantities they purport to represent. We argue that this can cause misleading conclusions about the relative group-wise model performance disparities along different dimensions, especially in cases where some sensitive variables consist of categories with few members. We propose the "double-corrected" variance estimator, which provides unbiased estimates and uncertainty quantification of the variance of model performance across groups. It is conceptually simple and easily implementable without statistical software package or numerical optimization. We demonstrate the utility of this approach through simulation and show on a real dataset that while statistically biased estimators of group-wise model performance disparities indicate statistically significant differences, when accounting for statistical bias in the estimator, the estimated between-group disparities are no longer statistically significant.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Measuring Disparate Outcomes of Content Recommendation Algorithms with Distributional Inequality Metrics

The harmful impacts of algorithmic decision systems have recently come into focus, with many examples of systems such as machine learning (ML) models amplifying existing societal biases. Most metrics attempting to quantify disparities resulting from ML algorithms focus on differences between groups, dividing users based on demographic identities and comparing model performance or overall outcomes between these groups. However, in industry settings, such information is often not available, and inferring these characteristics carries its own risks and biases. Moreover, typical metrics that focus on a single classifier's output ignore the complex network of systems that produce outcomes in real-world settings. In this paper, we evaluate a set of metrics originating from economics, distributional inequality metrics, and their ability to measure disparities in content exposure in a production recommendation system, the Twitter algorithmic timeline. We define desirable criteria for metrics to be used in an operational setting, specifically by ML practitioners. We characterize different types of engagement with content on Twitter using these metrics, and use these results to evaluate the metrics with respect to the desired criteria. We show that we can use these metrics to identify content suggestion algorithms that contribute more strongly to skewed outcomes between users. Overall, we conclude that these metrics can be useful tools for understanding disparate outcomes in online social networks.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Random Isn't Always Fair: Candidate Set Imbalance and Exposure Inequality in Recommender Systems

Traditionally, recommender systems operate by returning a user a set of items, ranked in order of estimated relevance to that user. In recent years, methods relying on stochastic ordering have been developed to create "fairer" rankings that reduce inequality in who or what is shown to users. Complete randomization -- ordering candidate items randomly, independent of estimated relevance -- is largely considered a baseline procedure that results in the most equal distribution of exposure. In industry settings, recommender systems often operate via a two-step process in which candidate items are first produced using computationally inexpensive methods and then a full ranking model is applied only to those candidates. In this paper, we consider the effects of inequality at the first step and show that, paradoxically, complete randomization at the second step can result in a higher degree of inequality relative to deterministic ordering of items by estimated relevance scores. In light of this observation, we then propose a simple post-processing algorithm in pursuit of reducing exposure inequality that works both when candidate sets have a high level of imbalance and when they do not. The efficacy of our method is illustrated on both simulated data and a common benchmark data set used in studying fairness in recommender systems.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Preference Modeling with Context-Dependent Salient Features

We consider the problem of estimating a ranking on a set of items from noisy pairwise comparisons given item features. We address the fact that pairwise comparison data often reflects irrational choice, e.g. intransitivity. Our key observation is that two items compared in isolation from other items may be compared based on only a salient subset of features. Formalizing this framework, we propose the salient feature preference model and prove a finite sample complexity result for learning the parameters of our model and the underlying ranking with maximum likelihood estimation. We also provide empirical results that support our theoretical bounds and illustrate how our model explains systematic intransitivity. Finally we demonstrate strong performance of maximum likelihood estimation of our model on both synthetic data and two real data sets: the UT Zappos50K data set and comparison data about the compactness of legislative districts in the US.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Training individually fair ML models with Sensitive Subspace Robustness

We consider training machine learning models that are fair in the sense that their performance is invariant under certain sensitive perturbations to the inputs. For example, the performance of a resume screening system should be invariant under changes to the gender and/or ethnicity of the applicant. We formalize this notion of algorithmic fairness as a variant of individual fairness and develop a distributionally robust optimization approach to enforce it during training. We also demonstrate the effectiveness of the approach on two ML tasks that are susceptible to gender and racial biases.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

The Distribution of Gaps between Summands in Generalized Zeckendorf Decompositions

Zeckendorf proved that any integer can be decomposed uniquely as a sum of non-adjacent Fibonacci numbers, $F_n$. Using continued fractions, Lekkerkerker proved the average number of summands of an $m \in [F_n, F_{n+1})$ is essentially $n/(φ^2 +1)$, with $φ$ the golden ratio. Miller-Wang generalized this by adopting a combinatorial perspective, proving that for any positive linear recurrence the number of summands in decompositions for integers in $[G_n, G_{n+1})$ converges to a Gaussian distribution. We prove the probability of a gap larger than the recurrence length converges to decaying geometrically, and that the distribution of the smaller gaps depends in a computable way on the coefficients of the recurrence. These results hold both for the average over all $m \in [G_n, G_{n+1})$, as well as holding almost surely for the gap measure associated to individual $m$. The techniques can also be used to determine the distribution of the longest gap between summands, which we prove is similar to the distribution of the longest gap between heads in tosses of a biased coin. It is a double exponential strongly concentrated about the mean, and is on the order of $\log n$ with computable constants depending on the recurrence.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Coordinate sum and difference sets of $d$-dimensional modular hyperbolas

Many problems in additive number theory, such as Fermat's last theorem and the twin prime conjecture, can be understood by examining sums or differences of a set with itself. A finite set $A \subset \mathbb{Z}$ is considered sum-dominant if $|A+A|>|A-A|$. If we consider all subsets of ${0, 1, ..., n-1}$, as $n\to\infty$ it is natural to expect that almost all subsets should be difference-dominant, as addition is commutative but subtraction is not; however, Martin and O'Bryant in 2007 proved that a positive percentage are sum-dominant as $n\to\infty$. This motivates the study of "coordinate sum dominance". Given $V \subset (\Z/n\Z)^2$, we call $S:={x+y: (x,y) \in V}$ a coordinate sumset and $D:=\{x-y: (x,y) \in V\}$ a coordinate difference set, and we say $V$ is coordinate sum dominant if $|S|>|D|$. An arithmetically interesting choice of $V$ is $\bar{H}_2(a;n)$, which is the reduction modulo $n$ of the modular hyperbola $H_2(a;n) := {(x,y): xy \equiv a \bmod n, 1 \le x,y < n}$. In 2009, Eichhorn, Khan, Stein, and Yankov determined the sizes of $S$ and $D$ for $V=\bar{H}_2(1;n)$ and investigated conditions for coordinate sum dominance. We extend their results to reduced $d$-dimensional modular hyperbolas $\bar{H}_d(a;n)$ with $a$ coprime to $n$.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

The Average Gap Distribution for Generalized Zeckendorf Decompositions

An interesting characterization of the Fibonacci numbers is that, if we write them as $F_1 = 1$, $F_2 = 2$, $F_3 = 3$, $F_4 = 5, ...$, then every positive integer can be written uniquely as a sum of non-adjacent Fibonacci numbers. This is now known as Zeckendorf's theorem [21], and similar decompositions exist for many other sequences ${G_{n+1} = c_1 G_{n} + ... + c_L G_{n+1-L}}$ arising from recurrence relations. Much more is known. Using continued fraction approaches, Lekkerkerker [15] proved the average number of summands needed for integers in $[G_n, G_{n+1})$ is on the order of $C_{\rm Lek} n$ for a non-zero constant; this was improved by others to show the number of summands has Gaussian fluctuations about this mean. Kolo$\breve{\rm g}$lu, Kopp, Miller and Wang [17, 18] recently recast the problem combinatorially, reproving and generalizing these results. We use this new perspective to investigate the distribution of gaps between summands. We explore the average behavior over all $m \in [G_n, G_{n+1})$ for special choices of the $c_i$'s. Specifically, we study the case where each $c_i \in {0,1}$ and there is a $g$ such that there are always exactly $g-1$ zeros between two non-zero $c_i$'s; note this includes the Fibonacci, Tribonacci and many other important special cases. We prove there are no gaps of length less than $g$, and the probability of a gap of length $j > g$ decays geometrically, with the decay ratio equal to the largest root of the recurrence relation. These methods are combinatorial and apply to related problems; we end with a discussion of similar results for far-difference (i.e., signed) decompositions.

0 citations0 reviewsNo code linkedOpen access