Source author record

Paul Gustafson

Paul Gustafson appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record

Methodology Applications math.CT math.DS math.ST Robotics Statistics Theory

Catalog footprint

What is connected

7works

7topics

4close collaborators

Actions

Connect this record

Open graph Browse works

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

preprint2022arXiv

re:Linde et al. (2021): The Bayes factor, HDI-ROPE and frequentist equivalence tests can all be reverse engineered -- almost exactly -- from one another

Following an extensive simulation study comparing the operating characteristics of three different procedures used for establishing equivalence (the frequentist "TOST", the Bayesian "HDI-ROPE", and the Bayes factor interval null procedure), Linde et al. (2021) conclude with the recommendation that "researchers rely more on the Bayes factor interval null approach for quantifying evidence for equivalence." We redo the simulation study of Linde et al. (2021) in its entirety but with the different procedures calibrated to have the same predetermined maximum type 1 error rate. Our results suggest that, when calibrated in this way, the Bayes Factor, HDI-ROPE, and frequentist equivalence tests all have similar -- almost exactly -- type 2 error rates. In general any advocating for frequentist testing as better or worse than Bayesian testing in terms of empirical findings seems dubious at best. If one decides on which underlying principle to subscribe to in tackling a given problem, then the method follows naturally. Bearing in mind that each procedure can be reverse-engineered from the others (at least approximately), trying to use empirical performance to argue for one approach over another seems like tilting at windmills.

preprint2021arXiv

Uncertainty and Value of Information in Risk Prediction Modeling

Background: Due to the finite size of the development sample, predicted probabilities from a risk prediction model are inevitably uncertain. We apply Value of Information methodology to evaluate the decision-theoretic implications of prediction uncertainty. Methods: Adopting a Bayesian perspective, we extend the definition of the Expected Value of Perfect Information (EVPI) from decision analysis to net benefit calculations in risk prediction. In the context of model development, EVPI is the expected gain in net benefit by using the correct predictions as opposed to predictions from a proposed model. We suggest bootstrap methods for sampling from the posterior distribution of predictions for EVPI calculation using Monte Carlo simulations. In a case study, we used subsets of data of various sizes from a clinical trial for predicting mortality after myocardial infarction to show how EVPI changes with sample size. Results: With a sample size of 1,000 and at the pre-specified threshold of 2% on predicted risks, the gain in net benefit by using the proposed and the correct models were 0.0006 and 0.0011, respectively, resulting in an EVPI of 0.0005 and a relative EVPI of 87%. EVPI was zero only at unrealistically high thresholds (>85%). As expected, EVPI declined with larger samples. We summarize an algorithm for incorporating EVPI calculations into the commonly used bootstrap method for optimism correction. Conclusion: Value of Information methods can be applied to explore decision-theoretic consequences of uncertainty in risk prediction and can complement inferential methods when developing risk prediction models. R code for implementing this method is provided.

preprint2021arXiv

What to make of non-inferiority and equivalence testing with a post-specified margin?

In order to determine whether or not an effect is absent based on a statistical test, the recommended frequentist tool is the equivalence test. Typically, it is expected that an appropriate equivalence margin has been specified before any data are observed. Unfortunately, this can be a difficult task. If the margin is too small, then the test's power will be substantially reduced. If the margin is too large, any claims of equivalence will be meaningless. Moreover, it remains unclear how defining the margin afterwards will bias one's results. In this short article, we consider a series of hypothetical scenarios in which the margin is defined post-hoc or is otherwise considered controversial. We also review a number of relevant, potentially problematic actual studies from clinical trials research, with the aim of motivating a critical discussion as to what is acceptable and desirable in the reporting and interpretation of equivalence tests.

preprint2020arXiv

Formal composition of hybrid systems

We develop a compositional framework for formal synthesis of hybrid systems using the language of category theory. More specifically, we provide mutually compatible tools for hierarchical, sequential, and independent parallel composition. In our framework, hierarchies of hybrid systems correspond to template-anchor pairs, which we model as spans of subdividing and embedding semiconjugacies. Hierarchical composition of template-anchor pairs corresponds to the composition of spans via pullback. To model sequential composition, we introduce "directed hybrid systems," each of which flows from an initial subsystem to a final subsystem in a Conley-theoretic sense. Sequential composition of directed systems is given by a pushout of graph embeddings, rewriting the continuous dynamics of the overlapping subsystem to prioritize the second directed system. Independent parallel composition corresponds to a categorical product with respect to semiconjugacy. To formalize the compatibility of these three types of composition, we construct a vertically cartesian double category of hybrid systems where the vertical morphisms are semiconjugacies, and the horizontal morphisms are directed hybrid systems.

preprint2019arXiv

A threshold-free summary index for quantifying the capacity of covariates to yield efficient treatment rules

The focus of this paper is on quantifying the capacity of covariates in devising efficient treatment rules when data from a randomized trial are available. Conventional one-variable-at-a-time subgroup analysis based on statistical hypothesis testing of covariate-by-treatment interaction is ill-suited for this purpose. The application of decision theory results in treatment rules that compare the expected benefit of treatment given the patient's covariates against a treatment threshold. However, determining treatment threshold is often context-specific, and any given threshold might seem arbitrary at the reporting stages of a clinical trial. We propose a threshold-free metric that quantifies the capacity of a set of covariates towards finding individuals who will benefit the most from treatment. The construct of the proposed metric is comparing the expected outcomes with and without knowledge of covariates when one of a two randomly selected patients are to be treated. We show that the resulting index can also be expressed in terms of integrated treatment benefit as a function of covariates over the entire range of treatment thresholds. We also propose a semi-parametric estimation method suitable for out-of-sample validation and adjustment for optimism. We use data from a clinical trial of preventive antibiotic therapy for reducing exacerbation rate in Chronic Obstructive Pulmonary Disease to demonstrate the calculations in a step-by-step fashion. The proposed index has intuitive and theoretically sound interpretation and can be estimated with relative ease for a wide class of regression models. Beyond the conceptual developments presented in this work, various aspects of estimation and inference for such metrics need to be pursued in future research.

preprint2016arXiv

The Constrained Maximum Likelihood Estimation For Parameters Arising From Partially Identified Models

We extend the constrained maximum likelihood estimation theory for parameters of a completely identified model, proposed by Aitchison and Silvey (1958), to parameters arising from a partially identified model. With a partially identified model, some parameters of the model may only be identified through constraints imposed by additional assumptions. We show that, under certain conditions, the constrained maximum likelihood estimator exists and locally maximize the likelihood function subject to constraints. We then study the asymptotic distribution of the estimator and propose a numerical algorithm for estimating parameters. We also discuss a special situation where exploiting additional assumptions does not improve estimation efficiency.

preprint2010arXiv

Interval Estimation for Messy Observational Data

We review some aspects of Bayesian and frequentist interval estimation, focusing first on their relative strengths and weaknesses when used in "clean" or "textbook" contexts. We then turn attention to observational-data situations which are "messy," where modeling that acknowledges the limitations of study design and data collection leads to nonidentifiability. We argue, via a series of examples, that Bayesian interval estimation is an attractive way to proceed in this context even for frequentists, because it can be supplied with a diagnostic in the form of a calibration-sensitivity simulation analysis. We illustrate the basis for this approach in a series of theoretical considerations, simulations and an application to a study of silica exposure and lung cancer.