BZPEERPapers, people, evidence and research pathsDiscover
Research tools
Network
Opportunities
Account
Source author record
Pascal Pernot appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.
Catalog footprint
Research graph
Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.
BZPEER is loading the nearby papers, people, topics and institutions for this page.
Published work
We review the alternative proposals introduced recently in the literature to update the standard formula to estimate the uncertainty on the mean of repeated measurements, and we compare their performances on synthetic examples with normal and non-normal error distributions.
Uncertainty quantification (UQ) in computational chemistry (CC) is still in its infancy. Very few CC methods are designed to provide a confidence level on their predictions, and most users still rely improperly on the mean absolute error as an accuracy metric. The development of reliable uncertainty quantification methods is essential, notably for computational chemistry to be used confidently in industrial processes. A review of the CC-UQ literature shows that there is no common standard procedure to report nor validate prediction uncertainty. I consider here analysis tools using concepts (calibration and sharpness) developed in meteorology and machine learning for the validation of probabilistic forecasters. These tools are adapted to CC-UQ and applied to datasets of prediction uncertainties provided by composite methods, Bayesian Ensembles methods, machine learning and a posteriori statistical methods.
Computational chemistry has become an important complement to experimental measurements. In order to choose among the multitude of the existing approximations, it is common to use benchmark data sets, and to issue recommendations based on numbers such as mean absolute errors. We argue, using as an example band gaps calculated with density functional approximations, that a more careful study of the benchmark data is needed, stressing that the user's requirements play a role in the choice of an appropriate method. We also appeal to those who measure data capable of being used as a reference, to publish error estimates. We show how the latter can affect the judgment of approximations used in computational chemistry.
With the upcoming launch of space telescopes dedicated to the study of exoplanets, the \textit{Atmospheric Remote-Sensing Infrared Exoplanet Large-survey} (ARIEL) and the \textit{James Webb Space Telescope} (JWST), a new era is opening in the exoplanetary atmospheric explorations. However, especially in relatively cold planets around later-type stars, photochemical hazes and clouds may mask the composition of the lower part of the atmosphere, making it difficult to detect any chemical species in the troposphere or to understand whether there is a surface or not. This issue is particularly exacerbated if the goal is to study the habitability of said exoplanets and to search for biosignatures.\par This work combines innovative laboratory experiments, chemical modeling and simulated observations at ARIEL and JWST resolutions. We focus on the signatures of molecular ions that can be found in upper atmospheres above cloud decks. Our results suggest that H$_3^+$ along with H$_3$O$^+$ could be detected in the observational spectra of sub-Neptunes based on realistic mixing ratio assumption. This new parametric set may help to distinguish super-Earths with a thin atmosphere from H$_2$-dominated sub-Neptunes, to address the critical question whether a low-gravity planet around a low-mass active star is able to retain its volatile components. These ions may also constitute potential tracers to certain molecules of interest such as H$_2$O or O$_2$ to probe the habitability of exoplanets. Their detection will be an enthralling challenge for the future JWST and ARIEL telescopes.
Thanks to the \textit{Cassini} spacecraft onboard instruments, it has been known that Titan's ionospheric chemistry is complex and the molecular growth is initiated through the photolysis of the most abundant species directly in the upper atmosphere. Among the pool of chemical compounds formed by the photolysis, N-bearing species are involved in the haze formation but the chemical incorporation pathways need to be better constrained.}\\ In this work, we performed low-pressure EUV photochemical laboratory experiments. The APSIS reactor was filled with a N$_2$/CH$_4$ (90/10\%) gas mixture relevant to the upper atmosphere of Titan. The cell was irradiated by using a EUV photon source at 73.6 nm {which has been difficult to produce in the laboratory for previous studies.}. The photoproducts (both neutral and ionic species) were monitored \textit{in situ} with a quadrupole mass spectrometer. The chemical pathways are explained by confronting experimental observations and numerical predictions of the photoproducts. \\ The most interesting result in this work is that methanimine was the only stable N-bearing neutral molecule detected during the experiments and it relies on N$_2^+$ production. This experimental result is in agreement with the relatively high abundance predicted by 1D-photochemical models of Titan's atmosphere and comforts methanimine as an intermediate towards the formation of complex N-bearing organic molecules.\\ {This experiment is only testing one part of the overall chemical scheme for Titan's upper atmosphere due to the selective wavelength but demonstrates the capability to probe the chemical pathways occurring in Titan's atmosphere by minimizing bias coming from wall surface reactions.}
In the first part of this study (Paper I), we introduced the systematic improvement probability (SIP) as a tool to assess the level of improvement on absolute errors to be expected when switching between two computational chemistry methods. We developed also two indicators based on robust statistics to address the uncertainty of ranking in computational chemistry benchmarks: Pinv , the inversion probability between two values of a statistic, and Pr , the ranking probability matrix. In this second part, these indicators are applied to nine data sets extracted from the recent benchmarking literature. We illustrate also how the correlation between the error sets might contain useful information on the benchmark dataset quality, notably when experimental data are used as reference.
The distribution of errors is a central object in the assesment and benchmarking of computational chemistry methods. The popular and often blind use of the mean unsigned error as a benchmarking statistic leads to ignore distributions features that impact the reliability of the tested methods. We explore how the Gini coefficient offers a global representation of the errors distribution, but, except for extreme values, does not enable an unambiguous diagnostic. We propose to relieve the ambiguity by applying the Gini coefficient to mode-centered error distributions. This version can usefully complement benchmarking statistics and alert on error sets with potentially problematic shapes.
Bayesian Model Calibration is used to revisit the problem of scaling factor calibration for semi-empirical correction of ab initio harmonic properties (e.g. vibrational frequencies and zero-point energies). A particular attention is devoted to the evaluation of scaling factor uncertainty, and to its effect on the accuracy of scaled properties. We argue that in most cases of interest the standard calibration model is not statistically valid, in the sense that it is not able to fit experimental calibration data within their uncertainty limits. This impairs any attempt to use the results of the standard model for uncertainty analysis and/or uncertainty propagation. We propose to include a stochastic term in the calibration model to account for model inadequacy. This new model is validated in the Bayesian Model Calibration framework. We provide explicit formulae for prediction uncertainty in typical limit cases: large and small calibration sets of data with negligible measurement uncertainty, and datasets with large measurement uncertainties.