BZPEERPapers, people, evidence and research paths
Menu

Discover

SearchFind papers, people, topics, institutions and opportunities from one place.GraphSee how papers, authors, topics and evidence connect.

Research tools

HomeContinue from the papers, people and topics that matter now.CollectionsTurn saved papers into reading lists, evidence maps and shareable context.ResearchReconstruct the path that led you from search to paper to question.CollaborationMove from a paper, topic or expert request into a focused thread.PublishCreate full-text research articles with metadata, formulas and versions.

Network

ExpertsFind specialists whose work and reviews explain why they can help.CommunitiesJoin research groups organized around papers, topics and shared evidence.

Opportunities

ListingsBrowse roles, calls and collaborations connected to real research context.

Account

Log inReturn to your saved papers, graph views and active threads.Create accountStart saving papers, building a research trail and joining teams.
Log inCreate account

Source author record

David R. Bickel

David R. Bickel appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record
math.STMethodologyStatistics TheoryInformation Theorymath.ITQuantitative MethodsApplicationsmath.PRBiomoleculesGenomics

Catalog footprint

What is connected

13works
10topics
3close collaborators

Actions

Connect this record

Log in to claim
Open graphBrowse works

Research graph

See the researcher in context

Open full explorer

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

13 published item(s)

preprint2012arXiv

Empirical Bayes methods corrected for small numbers of tests

Histogram-based empirical Bayes methods developed for analyzing data for large numbers of genes, SNPs, or other biological features tend to have large biases when applied to data with a smaller number of features such as genes with expression measured conventionally, proteins, and metabolites. To analyze such small-scale and medium-scale data in an empirical Bayes framework, we introduce corrections of maximum likelihood estimators (MLE) of the local false discovery rate (LFDR). In this context, the MLE estimates the LFDR, which is a posterior probability of null hypothesis truth, by estimating the prior distribution. The corrections lie in excluding each feature when estimating one or more parameters on which the prior depends. An application of the new estimators and previous estimators to protein abundance data illustrates how different estimators lead to very different conclusions about which proteins are affected by cancer. The estimators are compared using simulated data of two different numbers of features, two different detectability levels, and all possible numbers of affected features. The simulations show that some of the corrected MLEs substantially reduce a negative bias of the MLE. (The best-performing corrected MLE was derived from the minimum description length principle.) However, even the corrected MLEs have strong negative biases when the proportion of features that are unaffected is greater than 90%. Therefore, since the number of affected features is unknown in the case of real data, we recommend an optimally weighted combination of the best of the corrected MLEs with a conservative estimator that has weaker parametric assumptions.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Blending Bayesian and frequentist methods according to the precision of prior information with an application to hypothesis testing

The following zero-sum game between nature and a statistician blends Bayesian methods with frequentist methods such as p-values and confidence intervals. Nature chooses a posterior distribution consistent with a set of possible priors. At the same time, the statistician selects a parameter distribution for inference with the goal of maximizing the minimum Kullback-Leibler information gained over a confidence distribution or other benchmark distribution. An application to testing a simple null hypothesis leads the statistician to report a posterior probability of the hypothesis that is informed by both Bayesian and frequentist methodology, each weighted according how well the prior is known. Since neither the Bayesian approach nor the frequentist approach is entirely satisfactory in situations involving partial knowledge of the prior distribution, the proposed procedure reduces to a Bayesian method given complete knowledge of the prior, to a frequentist method given complete ignorance about the prior, and to a blend between the two methods given partial knowledge of the prior. The blended approach resembles the Bayesian method rather than the frequentist method to the precise extent that the prior is known. The problem of testing a point null hypothesis illustrates the proposed framework. The blended probability that the null hypothesis is true is equal to the p-value or a lower bound of an unknown Bayesian posterior probability, whichever is greater. Thus, given total ignorance represented by a lower bound of 0, the p-value is used instead of any Bayesian posterior probability. At the opposite extreme of a known prior, the p-value is ignored. In the intermediate case, the possible Bayesian posterior probability that is closest to the p-value is used for inference. Thus, both the Bayesian method and the frequentist method influence the inferences made.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Controlling the degree of caution in statistical inference with the Bayesian and frequentist approaches as opposite extremes

In statistical practice, whether a Bayesian or frequentist approach is used in inference depends not only on the availability of prior information but also on the attitude taken toward partial prior information, with frequentists tending to be more cautious than Bayesians. The proposed framework defines that attitude in terms of a specified amount of caution, thereby enabling data analysis at the level of caution desired and on the basis of any prior information. The caution parameter represents the attitude toward partial prior information in much the same way as a loss function represents the attitude toward risk. When there is very little prior information and nonzero caution, the resulting inferences correspond to those of the candidate confidence intervals and p-values that are most similar to the credible intervals and hypothesis probabilities of the specified Bayesian posterior. On the other hand, in the presence of a known physical distribution of the parameter, inferences are based only on the corresponding physical posterior. In those extremes of either negligible prior information or complete prior information, inferences do not depend on the degree of caution. Partial prior information between those two extremes leads to intermediate inferences that are more frequentistic to the extent that the caution is high and more Bayesian to the extent that the caution is low.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Empirical Bayes estimation of posterior probabilities of enrichment

To interpret differentially expressed genes or other discovered features, researchers conduct hypothesis tests to determine which biological categories such as those of the Gene Ontology (GO) are enriched in the sense of having differential representation among the discovered features. We study application of better estimators of the local false discovery rate (LFDR), a probability that the biological category has equivalent representation among the preselected features. We identified three promising estimators of the LFDR for detecting differential representation: a semiparametric estimator (SPE), a normalized maximum likelihood estimator (NMLE), and a maximum likelihood estimator (MLE). We found that the MLE performs at least as well as the SPE for on the order of 100 of GO categories even when the ideal number of components in its underlying mixture model is unknown. However, the MLE is unreliable when the number of GO categories is small compared to the number of PMM components. Thus, if the number of categories is on the order of 10, the SPE is a more reliable LFDR estimator. The NMLE depends not only on the data but also on a specified value of the prior probability of differential representation. It is therefore an appropriate LFDR estimator only when the number of GO categories is too small for application of the other methods. For enrichment detection, we recommend estimating the LFDR by the MLE given at least a medium number (~100) of GO categories, by the SPE given a small number of GO categories (~10), and by the NMLE given a very small number (~1) of GO categories.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Measuring support for a hypothesis about a random parameter without estimating its unknown prior

For frequentist settings in which parameter randomness represents variability rather than uncertainty, the ideal measure of the support for one hypothesis over another is the difference in the posterior and prior log odds. For situations in which the prior distribution cannot be accurately estimated, that ideal support may be replaced by another measure of support, which may be any predictor of the ideal support that, on a per-observation basis, is asymptotically unbiased. Two qualifying measures of support are defined. The first is minimax optimal with respect to the population and is equivalent to a particular Bayes factor. The second is worst-sample minimax optimal and is equivalent to the normalized maximum likelihood. It has been extended by likelihood weights for compatibility with more general models. One such model is that of two independent normal samples, the standard setting for gene expression microarray data analysis. Applying that model to proteomics data indicates that support computed from data for a single protein can closely approximate the estimated difference in posterior and prior odds that would be available with the data for 20 proteins. This suggests the applicability of random-parameter models to other situations in which the parameter distribution cannot be reliably estimated.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Resolving conflicts between statistical methods by probability combination: Application to empirical Bayes analyses of genomic data

In the typical analysis of a data set, a single method is selected for statistical reporting even when equally applicable methods yield very different results. Examples of equally applicable methods can correspond to those of different ancillary statistics in frequentist inference and of different prior distributions in Bayesian inference. More broadly, choices are made between parametric and nonparametric methods and between frequentist and Bayesian methods. Rather than choosing a single method, it can be safer, in a game-theoretic sense, to combine those that are equally appropriate in light of the available information. Since methods of combining subjectively assessed probability distributions are not objective enough for that purpose, this paper introduces a method of distribution combination that does not require any assignment of distribution weights. It does so by formalizing a hedging strategy in terms of a game between three players: nature, a statistician combining distributions, and a statistician refusing to combine distributions. The optimal move of the first statistician reduces to the solution of a simpler problem of selecting an estimating distribution that minimizes the Kullback-Leibler loss maximized over the plausible distributions to be combined. The resulting combined distribution is a linear combination of the most extreme of the distributions to be combined that are scientifically plausible. The optimal weights are close enough to each other that no extreme distribution dominates the others. The new methodology is illustrated by combining conflicting empirical Bayes methodologies in the context of gene expression data analysis.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Simple estimators of false discovery rates given as few as one or two p-values without strong parametric assumptions

Multiple comparison procedures that control a family-wise error rate or false discovery rate provide an achieved error rate as the adjusted p-value for each hypothesis tested. However, since such p-values are not probabilities that the null hypotheses are true, empirical Bayes methods have been devised to estimate such posterior probabilities, called local false discovery rates (LFDRs) to emphasize the frequency interpretation of their priors. The main approaches to LFDR estimation, relying either on numerical algorithms to maximize likelihood or on the selection of smoothing parameters for nonparametric density estimation, lack the automatic nature of the methods of error rate control. To begin filling the gap, this paper introduces automatic methods of LFDR estimation with proven asymptotic conservatism under the independence of p-values but without strong parametric assumptions. Simulations indicate that they remain conservative even for very small numbers of hypotheses. One of the proposed procedures enables interpreting the original FDR control rule in terms of LFDR estimation, thereby facilitating practical interpretation. The most conservative of the new procedures is applied to measured abundance levels of 20 proteins.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Small-scale inference: Empirical Bayes and confidence methods for as few as a single comparison

By restricting the possible values of the proportion of null hypotheses that are true, the local false discovery rate (LFDR) can be estimated using as few as one comparison. The proportion of proteins with equivalent abundance was estimated to be about 20% for patient group I and about 90% for group II. The simultaneously-estimated LFDRs give approximately the same inferences as individual-protein confidence levels for group I but are much closer to individual-protein LFDR estimates for group II. Simulations confirm that confidence-based inference or LFDR-based inference performs markedly better for low or high proportions of true null hypotheses, respectively.

0 citations0 reviewsNo code linkedOpen access
preprint2010arXiv

Large-scale interval and point estimates from an empirical Bayes extension of confidence posteriors

The proposed approach extends the confidence posterior distribution to the semi-parametric empirical Bayes setting. Whereas the Bayesian posterior is defined in terms of a prior distribution conditional on the observed data, the confidence posterior is defined such that the probability that the parameter value lies in any fixed subset of parameter space, given the observed data, is equal to the coverage rate of the corresponding confidence interval. A confidence posterior that has correct frequentist coverage at each fixed parameter value is combined with the estimated local false discovery rate to yield a parameter distribution from which interval and point estimates are derived within the framework of minimizing expected loss. The point estimates exhibit suitable shrinkage toward the null hypothesis value, making them practical for automatically ranking features in order of priority. The corresponding confidence intervals are also shrunken and tend to be much shorter than their fixed-parameter counterparts, as illustrated with gene expression data. Further, simulations confirm a theoretical argument that the shrunken confidence intervals cover the parameter at a higher-than-nominal frequency.

0 citations0 reviewsNo code linkedOpen access
preprint2010arXiv

Statistical inference optimized with respect to the observed sample for single or multiple comparisons

The normalized maximum likelihood (NML) is a recent penalized likelihood that has properties that justify defining the amount of discrimination information (DI) in the data supporting an alternative hypothesis over a null hypothesis as the logarithm of an NML ratio, namely, the alternative hypothesis NML divided by the null hypothesis NML. The resulting DI, like the Bayes factor but unlike the p-value, measures the strength of evidence for an alternative hypothesis over a null hypothesis such that the probability of misleading evidence vanishes asymptotically under weak regularity conditions and such that evidence can support a simple null hypothesis. Unlike the Bayes factor, the DI does not require a prior distribution and is minimax optimal in a sense that does not involve averaging over outcomes that did not occur. Replacing a (possibly pseudo-) likelihood function with its weighted counterpart extends the scope of the DI to models for which the unweighted NML is undefined. The likelihood weights leverage side information, either in data associated with comparisons other than the comparison at hand or in the parameter value of a simple null hypothesis. Two case studies, one involving multiple populations and the other involving multiple biological features, indicate that the DI is robust to the type of side information used when that information is assigned the weight of a single observation. Such robustness suggests that very little adjustment for multiple comparisons is warranted if the sample size is at least moderate.

0 citations0 reviewsNo code linkedOpen access
preprint2009arXiv

A frequentist framework of inductive reasoning

Reacting against the limitation of statistics to decision procedures, R. A. Fisher proposed for inductive reasoning the use of the fiducial distribution, a parameter-space distribution of epistemological probability transferred directly from limiting relative frequencies rather than computed according to the Bayes update rule. The proposal is developed as follows using the confidence measure of a scalar parameter of interest. (With the restriction to one-dimensional parameter space, a confidence measure is essentially a fiducial probability distribution free of complications involving ancillary statistics.) A betting game establishes a sense in which confidence measures are the only reliable inferential probability distributions. The equality between the probabilities encoded in a confidence measure and the coverage rates of the corresponding confidence intervals ensures that the measure's rule for assigning confidence levels to hypotheses is uniquely minimax in the game. Although a confidence measure can be computed without any prior distribution, previous knowledge can be incorporated into confidence-based reasoning. To adjust a p-value or confidence interval for prior information, the confidence measure from the observed data can be combined with one or more independent confidence measures representing previous agent opinion. (The former confidence measure may correspond to a posterior distribution with frequentist matching of coverage probabilities.) The representation of subjective knowledge in terms of confidence measures rather than prior probability distributions preserves approximate frequentist validity.

0 citations0 reviewsNo code linkedOpen access
preprint2009arXiv

Coherent frequentism

By representing the range of fair betting odds according to a pair of confidence set estimators, dual probability measures on parameter space called frequentist posteriors secure the coherence of subjective inference without any prior distribution. The closure of the set of expected losses corresponding to the dual frequentist posteriors constrains decisions without arbitrarily forcing optimization under all circumstances. This decision theory reduces to those that maximize expected utility when the pair of frequentist posteriors is induced by an exact or approximate confidence set estimator or when an automatic reduction rule is applied to the pair. In such cases, the resulting frequentist posterior is coherent in the sense that, as a probability distribution of the parameter of interest, it satisfies the axioms of the decision-theoretic and logic-theoretic systems typically cited in support of the Bayesian posterior. Unlike the p-value, the confidence level of an interval hypothesis derived from such a measure is suitable as an estimator of the indicator of hypothesis truth since it converges in sample-space probability to 1 if the hypothesis is true or to 0 otherwise under general conditions.

0 citations0 reviewsNo code linkedOpen access
preprint2009arXiv

Estimating the null distribution for conditional inference and genome-scale screening

In a novel approach to the multiple testing problem, Efron (2004; 2007) formulated estimators of the distribution of test statistics or nominal p-values under a null distribution suitable for modeling the data of thousands of unaffected genes, non-associated single-nucleotide polymorphisms, or other biological features. Estimators of the null distribution can improve not only the empirical Bayes procedure for which it was originally intended, but also many other multiple comparison procedures. Such estimators serve as the groundwork for the proposed multiple comparison procedure based on a recent frequentist method of minimizing posterior expected loss, exemplified with a non-additive loss function designed for genomic screening rather than for validation. The merit of estimating the null distribution is examined from the vantage point of conditional inference in the remainder of the paper. In a simulation study of genome-scale multiple testing, conditioning the observed confidence level on the estimated null distribution as an approximate ancillary statistic markedly improved conditional inference. To enable researchers to determine whether to rely on a particular estimated null distribution for inference or decision making, an information-theoretic score is provided that quantifies the benefit of conditioning. As the sum of the degree of ancillarity and the degree of inferential relevance, the score reflects the balance conditioning would strike between the two conflicting terms. Applications to gene expression microarray data illustrate the methods introduced.

0 citations0 reviewsNo code linkedOpen access