Paper detail

Batch learning equals online learning in Bayesian supervised learning

In this paper we study Bayesian supervised learning models proposed by Lê in \cite{Le2025}. We show the existence of Bayesian inversions on universal Bayesian supervised learning models $(\Pp (\Yy)^\Xx, μ, \Id_{\Pp (\Yy) ^\Xx}, \Pp (\Yy)^\Xx)$ for arbitrary input space $\Xx$, Souslin label space $\Yy$, and prior probability measure $μ\in \Pp (\Pp (\Yy) ^\Xx)$. Using functoriality of probabilistic morphisms, we prove that sequential and batch Bayesian inversions coincide in supervised learning models with conditionally independent (possibly non-i.i.d.) data \cite{Le2025}. This equivalence holds without domination or discreteness assumptions on sampling operators. We derive a recursive formula for posterior predictive distributions, which reduces to the Kalman filter in Gaussian process regression. For Polish label spaces $\mathcal{Y}$ and arbitrary input sets $\mathcal{X}$, we characterize probability measures on $\mathcal{P}(\mathcal{Y})^{\mathcal{X}}$ via projective systems, generalizing Orbanz \cite{Orbanz2011}. We revisit MacEachern's Dependent Dirichlet Processes (DDP) \cite{MacEachern2000} using copula-based constructions \cite{BJQ2012} and show how to compute posterior predictive distributions in universal Bayesian supervised models with DDP priors.