Paper detail

Negatively Biased Relevant Subsets Induced by the Most-Powerful One-Sided Upper Confidence Limits for a Bounded Physical Parameter

Suppose an observable x is the measured value (negative or non-negative) of a true mean mu (physically non-negative) in an experiment with a Gaussian resolution function with known fixed rms deviation s. The most powerful one-sided upper confidence limit at 95% C.L. is UL = x+1.64s, which I refer to as the "original diagonal line". Perceived problems in HEP with small or non-physical upper limits for x<0 historically led, for example, to substitution of max(0,x) for x, and eventually to abandonment in the Particle Data Group's Review of Particle Physics of this diagonal line relationship between UL and x. Recently Cowan, Cranmer, Gross, and Vitells (CCGV) have advocated a concept of "power constraint" that when applied to this problem yields variants of diagonal line, including UL = max(-1,x)+1.64s. Thus it is timely to consider again what is problematic about the original diagonal line, and whether or not modifications cure these defects. In a 2002 Comment, statistician Leon Jay Gleser pointed to the literature on recognizable and relevant subsets. For upper limits given by the original diagonal line, the sample space for x has recognizable relevant subsets in which the quoted 95% C.L. is known to be negatively biased (anti-conservative) by a finite amount for all values of mu. This issue is at the heart of a dispute between Jerzy Neyman and Sir Ronald Fisher over fifty years ago, the crux of which is the relevance of pre-data coverage probabilities when making post-data inferences. The literature describes illuminating connections to Bayesian statistics as well. Methods such as that advocated by CCGV have 100% unconditional coverage for certain values of mu and hence formally evade the traditional criteria for negatively biased relevant subsets; I argue that concerns remain. Comparison with frequentist intervals advocated by Feldman and Cousins also sheds light on the issues.