Veritas: Answering Causal Queries from Video Streaming Traces
In this paper, we seek to answer what-if questions - i.e., given recorded data of an existing deployed networked system, what would be the performance impact if we changed the design of the system (a task also known as causal inference). We make three contributions. First, we expose the complexity of causal inference in the context of adaptive bit rate video streaming, a challenging domain where the network conditions during the session act as a sequence of latent and confounding variables, and a change at any point in the session has a cascading impact on the rest of the session. Second, we present Veritas, a novel framework that tackles causal reasoning for video streaming without resorting to randomised trials. Integral to Veritas is an easy to interpret domain-specific ML model (an embedded Hidden Markov Model) that relates the latent stochastic process (intrinsic bandwidth that the video session can achieve) to actual observations (download times) while exploiting control variables such as the TCP state (e.g., congestion window) observed at the start of the download of video chunks. We show through experiments on an emulation testbed that Veritas can answer both counterfactual queries (e.g., the performance of a completed video session had it used a different buffer size) and interventional queries (e.g., estimating the download time for every possible video quality choice for the next chunk in a session in progress). In doing so, Veritas achieves accuracy close to an ideal oracle, while significantly outperforming both a commonly used baseline approach, and Fugu (an off-the-shelf neural network) neither of which account for causal effects.