BZPEERPapers, people, evidence and research paths
Menu

Discover

SearchFind papers, people, topics, institutions and opportunities from one place.GraphSee how papers, authors, topics and evidence connect.

Research tools

HomeContinue from the papers, people and topics that matter now.CollectionsTurn saved papers into reading lists, evidence maps and shareable context.ResearchReconstruct the path that led you from search to paper to question.CollaborationMove from a paper, topic or expert request into a focused thread.PublishCreate full-text research articles with metadata, formulas and versions.

Network

ExpertsFind specialists whose work and reviews explain why they can help.CommunitiesJoin research groups organized around papers, topics and shared evidence.

Opportunities

ListingsBrowse roles, calls and collaborations connected to real research context.

Account

Log inReturn to your saved papers, graph views and active threads.Create accountStart saving papers, building a research trail and joining teams.
Log inCreate account

Source author record

Adam Kapelner

Adam Kapelner appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record
MethodologyApplicationsMachine LearningHuman-Computer Interactionstat.OT

Catalog footprint

What is connected

13works
5topics
4close collaborators

Actions

Connect this record

Log in to claim
Open graphBrowse works

Research graph

See the researcher in context

Open full explorer

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

13 published item(s)

preprint2022arXiv

The Role of Pairwise Matching in Experimental Design for an Incidence Outcome

We consider the problem of evaluating designs for a two-arm randomized experiment with an incidence (binary) outcome under a nonparametric general response model. Our two main results are that the priori pair matching design of Greevy et al. (2004) is (1) the optimal design as measured by mean squared error among all block designs which includes complete randomization. And (2), this pair-matching design is minimax, i.e. it provides the lowest mean squared error under an adversarial response model. Theoretical results are supported by simulations and clinical trial data.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

Better Experimental Design by Hybridizing Binary Matching with Imbalance Optimization

We present a new experimental design procedure that divides a set of experimental units into two groups in order to minimize error in estimating an additive treatment effect. One concern is minimizing error at the experimental design stage is large covariate imbalance between the two groups. Another concern is robustness of design to misspecification in response models. We address both concerns in our proposed design: we first place subjects into pairs using optimal nonbipartite matching, making our estimator robust to complicated non-linear response models. Our innovation is to keep the matched pairs extant, take differences of the covariate values within each matched pair and then we use the greedy switching heuristic of Krieger et al. (2019) or rerandomization on these differences. This latter step greatly reduce covariate imbalance to the rate $O_p(n^{-4})$ in the case of one covariate that are uniformly distributed. This rate benefits from the greedy switching heuristic which is $O_p(n^{-3})$ and the rate of matching which is $O_p(n^{-1})$. Further, our resultant designs are shown to be as random as matching which is robust to unobserved covariates. When compared to previous designs, our approach exhibits significant improvement in the mean squared error of the treatment effect estimator when the response model is nonlinear and performs at least as well when it the response model is linear. Our design procedure is found as a method in the open source R package available on CRAN called GreedyExperimentalDesign.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

Optimal Rerandomization via a Criterion that Provides Insurance Against Failed Experiments

We present an optimized rerandomization design procedure for a non-sequential treatment-control experiment. Randomized experiments are the gold standard for finding causal effects in nature. But sometimes random assignments result in unequal partitions of the treatment and control group visibly seen as imbalance in observed covariates. There can additionally be imbalance on unobserved covariates. Imbalance in either observed or unobserved covariates increases treatment effect estimator error inflating the width of confidence regions and reducing experimental power. "Rerandomization" is a strategy that omits poor imbalance assignments by limiting imbalance in the observed covariates to a prespecified threshold. However, limiting this threshold too much can increase the risk of contracting error from unobserved covariates. We introduce a criterion that combines observed imbalance while factoring in the risk of inadvertently imbalancing unobserved covariates. We then use this criterion to locate the optimal rerandomization threshold based on the practitioner's level of desired insurance against high estimator error. We demonstrate the gains of our designs in simulation and in a dataset from a large randomized experiment in education. We provide an open source R package available on CRAN named OptimalRerandExpDesigns which generates designs according to our algorithm.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Improving the Power of the Randomization Test

We consider the problem of evaluating designs for a two-arm randomized experiment with the criterion being the power of the randomization test for the one-sided null hypothesis. Our evaluation assumes a response that is linear in one observed covariate, an unobserved component and an additive treatment effect where the only randomness comes from the treatment allocations. It is well-known that the power depends on the allocations' imbalance in the observed covariate and this is the reason for the classic restricted designs such as rerandomization. We show that power is also affected by two other design choices: the number of allocations in the design and the degree of linear dependence among the allocations. We prove that the more allocations, the higher the power and the lower the variability in the power. Designs that feature greater independence of allocations are also shown to have higher performance. Our theoretical findings and extensive simulation studies imply that the designs with the highest power provide thousands of highly independent allocations that each provide nominal imbalance in the observed covariates. These high powered designs exhibit less randomization than complete randomization and more randomization than recently proposed designs based on numerical optimization. Model choices for a practicing experimenter are rerandomization and greedy pair switching, where both outperform complete randomization and numerical optimization. The tradeoff we find also provides a means to specify the imbalance threshold parameter when rerandomizing.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Optimal Experimental Designs for Estimating Henry's Law Constants via the Method of Phase Ratio Variation

When measuring Henry's Law constants ($k_H$) using the phase ratio method via headspace gas chromatography (GC), the value of $k_H$ of the compound under investigation is calculated from the ratio of the slope to the intercept of a linear regression of the the inverse GC response versus the ratio of gas to liquid volumes of a series of vials drawn from the same parent solution. Thus, an experimenter will collect measurements consisting of the independent variable (the gas/liquid volume ratio) and dependent variable (the inverse GC peak area). There is a choice of values of the independent variable during measurement. A review of the literature found that the common approach is a simple uniformly spacing of the liquid volumes. We present an optimal experimental design which estimates $k_H$ with minimum error and provides multiple means for building confidence intervals for such estimates. We illustrate efficiency improvements of our new design with an example measuring the $k_H$ for napthalene in aqueous solution as well as simulations on previous studies. The designs can be easily computed using our open source software optDesignSlopeInt, an R package on CRAN. We also discuss applicability of this method to other fields.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

bartMachine: Machine Learning with Bayesian Additive Regression Trees

We present a new package in R implementing Bayesian additive regression trees (BART). The package introduces many new features for data analysis using BART such as variable selection, interaction detection, model diagnostic plots, incorporation of missing data and the ability to save trees for future prediction. It is significantly faster than the current R implementation, parallelized, and capable of handling both large sample sizes and high-dimensional data.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Bayesian Additive Regression Trees With Parametric Models of Heteroskedasticity

We incorporate heteroskedasticity into Bayesian Additive Regression Trees (BART) by modeling the log of the error variance parameter as a linear function of prespecified covariates. Under this scheme, the Gibbs sampling procedure for the original sum-of- trees model is easily modified, and the parameters for the variance model are updated via a Metropolis-Hastings step. We demonstrate the promise of our approach by providing more appropriate posterior predictive intervals than homoskedastic BART in heteroskedastic settings and demonstrating the model's resistance to overfitting. Our implementation will be offered in an upcoming release of the R package bartMachine.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Peeking Inside the Black Box: Visualizing Statistical Learning with Plots of Individual Conditional Expectation

This article presents Individual Conditional Expectation (ICE) plots, a tool for visualizing the model estimated by any supervised learning algorithm. Classical partial dependence plots (PDPs) help visualize the average partial relationship between the predicted response and one or more features. In the presence of substantial interaction effects, the partial response relationship can be heterogeneous. Thus, an average curve, such as the PDP, can obfuscate the complexity of the modeled relationship. Accordingly, ICE plots refine the partial dependence plot by graphing the functional relationship between the predicted response and the feature for individual observations. Specifically, ICE plots highlight the variation in the fitted values across the range of a covariate, suggesting where and to what extent heterogeneities might exist. In addition to providing a plotting suite for exploratory analysis, we include a visual test for additive structure in the data generating model. Through simulated examples and real data sets, we demonstrate how ICE plots can shed light on estimated models in ways PDPs cannot. Procedures outlined are available in the R package ICEbox.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Prediction with Missing Data via Bayesian Additive Regression Trees

We present a method for incorporating missing data in non-parametric statistical learning without the need for imputation. We focus on a tree-based method, Bayesian Additive Regression Trees (BART), enhanced with "Missingness Incorporated in Attributes," an approach recently proposed incorporating missingness into decision trees (Twala, 2008). This procedure takes advantage of the partitioning mechanisms found in tree-based models. Simulations on generated models and real data indicate that our proposed method can forecast well on complicated missing-at-random and not-missing-at-random models as well as models where missingness itself influences the response. Our procedure has higher predictive performance and is more stable than competitors in many cases. We also illustrate BART's abilities to incorporate missingness into uncertainty intervals and to detect the influence of missingness on the model fit.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Using Regression Kernels to Forecast A Failure to Appear in Court

Forecasts of prospective criminal behavior have long been an important feature of many criminal justice decisions. There is now substantial evidence that machine learning procedures will classify and forecast at least as well, and typically better, than logistic regression, which has to date dominated conventional practice. However, machine learning procedures are adaptive. They "learn" inductively from training data. As a result, they typically perform best with very large datasets. There is a need, therefore, for forecasting procedures with the promise of machine learning that will perform well with small to moderately-sized datasets. Kernel methods provide precisely that promise. In this paper, we offer an overview of kernel methods in regression settings and compare such a method, regularized with principle components, to stepwise logistic regression. We apply both to a timely and important criminal justice concern: a failure to appear (FTA) at court proceedings following an arraignment. A forecast of an FTA can be an important factor is a judge's decision to release a defendant while awaiting trial and can influence the conditions imposed on that release. Forecasting accuracy matters, and our kernel approach forecasts far more accurately than stepwise logistic regression. The methods developed here are implemented in the R package kernReg currently available on CRAN.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Variable selection for BART: An application to gene regulation

We consider the task of discovering gene regulatory networks, which are defined as sets of genes and the corresponding transcription factors which regulate their expression levels. This can be viewed as a variable selection problem, potentially with high dimensionality. Variable selection is especially challenging in high-dimensional settings, where it is difficult to detect subtle individual effects and interactions between predictors. Bayesian Additive Regression Trees [BART, Ann. Appl. Stat. 4 (2010) 266-298] provides a novel nonparametric alternative to parametric regression approaches, such as the lasso or stepwise regression, especially when the number of relevant predictors is sparse relative to the total number of available predictors and the fundamental relationships are nonlinear. We develop a principled permutation-based inferential approach for determining when the effect of a selected predictor is likely to be real. Going further, we adapt the BART procedure to incorporate informed prior information about variable importance. We present simulations demonstrating that our method compares favorably to existing parametric and nonparametric procedures in a variety of data settings. To demonstrate the potential of our approach in a biological context, we apply it to the task of inferring the gene regulatory network in yeast (Saccharomyces cerevisiae). We find that our BART-based procedure is best able to recover the subset of covariates with the largest signal compared to other variable selection methods. The methods developed in this work are readily available in the R package bartMachine.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

Matching on-the-fly in Sequential Experiments for Higher Power and Efficiency

We propose a dynamic allocation procedure that increases power and efficiency when measuring an average treatment effect in sequential randomized trials. Subjects arrive iteratively and are either randomized or paired via a matching criterion to a previously randomized subject and administered the alternate treatment. We develop estimators for the average treatment effect that combine information from both the matched pairs and unmatched subjects as well as an exact test. Simulations illustrate the method's higher efficiency and power over competing allocation procedures in both controlled scenarios and historical experimental data.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Breaking Monotony with Meaning: Motivation in Crowdsourcing Markets

We conduct the first natural field experiment to explore the relationship between the "meaningfulness" of a task and worker effort. We employed about 2,500 workers from Amazon's Mechanical Turk (MTurk), an online labor market, to label medical images. Although given an identical task, we experimentally manipulated how the task was framed. Subjects in the meaningful treatment were told that they were labeling tumor cells in order to assist medical researchers, subjects in the zero-context condition (the control group) were not told the purpose of the task, and, in stark contrast, subjects in the shredded treatment were not given context and were additionally told that their work would be discarded. We found that when a task was framed more meaningfully, workers were more likely to participate. We also found that the meaningful treatment increased the quantity of output (with an insignificant change in quality) while the shredded treatment decreased the quality of output (with no change in quantity). We believe these results will generalize to other short-term labor markets. Our study also discusses MTurk as an exciting platform for running natural field experiments in economics.

0 citations0 reviewsNo code linkedOpen access