Mini-Minimax Uncertainty Quantification for Emulators
Consider approximating a "black box" function $f$ by an emulator $\hat{f}$ based on $n$ noiseless observations of $f$. Let $w$ be a point in the domain of $f$. How big might the error $|\hat{f}(w) - f(w)|$ be? If $f$ could be arbitrarily rough, this error could be arbitrarily large: we need some constraint on $f$ besides the data. Suppose $f$ is Lipschitz with known constant. We find a lower bound on the number of observations required to ensure that for the best emulator $\hat{f}$ based on the $n$ data, $|\hat{f}(w) - f(w)| \le ε$. But in general, we will not know whether $f$ is Lipschitz, much less know its Lipschitz constant. Assume optimistically that $f$ is Lipschitz-continuous with the smallest constant consistent with the $n$ data. We find the maximum (over such regular $f$) of $|\hat{f}(w) - f(w)|$ for the best possible emulator $\hat{f}$; we call this the "mini-minimax uncertainty" at $w$. In reality, $f$ might not be Lipschitz or---if it is---it might not attain its Lipschitz constant on the data. Hence, the mini-minimax uncertainty at $w$ could be much smaller than $|\hat{f}(w) - f(w)|$. But if the mini-minimax uncertainty is large, then---even if $f$ satisfies the optimistic regularity assumption---$|\hat{f}(w) - f(w)|$ could be large, no matter how cleverly we choose $\hat{f}$. For the Community Atmosphere Model, the maximum (over $w$) of the mini-minimax uncertainty based on a set of 1154~observations of $f$ is no smaller than it would be for a single observation of $f$ at the centroid of the 21-dimensional parameter space. We also find lower confidence bounds for quantiles of the mini-minimax uncertainty and its mean over the domain of $f$. For the Community Atmosphere Model, these lower confidence bounds are an appreciable fraction of the maximum.