Bayesian Covariance Modelling of Large Tensor-Variate Data Sets $\&$ Inverse Non-parametric Learning of the Unknown Model Parameter Vector
Tensor-valued data are being encountered increasingly more commonly, in the biological, natural as well as the social sciences. The learning of the unknown model parameter vector given such data, involves covariance modelling of such data, though this can be difficult owing to the high-dimensional nature of the data, where the numerical challenge of such modelling can only be compounded by the largeness of the available data set. Assuming such data to be modelled using a correspondingly high-dimensional Gaussian Process (${\cal GP}$), the joint density of a finite set of such data sets is then a tensor normal distribution, with density parametrised by a mean tensor $\boldsymbol{M}$ (that is of the same dimensionality as the $k$-tensor valued observable), and the $k$ covariance matrices $\boldsymbolΣ_1,...,\boldsymbolΣ_k$. When aiming to model the covariance structure of the data, we need to estimate/learn $\{\boldsymbolΣ_1,...,\boldsymbolΣ_k \}$ and $\boldsymbol{M}$, given tha data. We present a new method in which we perform such covariance modelling by first expressing the probability density of the available data sets as tensor-normal. We then invoke appropriate priors on these unknown parameters and express the posterior of the unknowns given the data. We sample from this posterior using an appropriate variant of Metropolis Hastings. Since the classical MCMC is time and resource intensive in high-dimensional state spaces, we use an efficient variant of the Metropolis-Hastings algorithm--Transformation based MCMC--employed to perform efficient sampling from a high-dimensional state space. Once we perform the covariance modelling of such a data set, we will learn the unknown model parameter vector at which a measured (or test) data set has been obtained, given the already modelled data (training data), augmented by the test data.
