Is there a way to perform Gaussian Process Regression on multidimensional output (possibly correlated) using GPML?

In the demo script I could only find a 1D example.

A similar question on CV that tackles case of multidimensional input.

I went through their book to see if I could find anything. In the 9th chapter of this book (section 9.1), they have mentioned this case of multiple outputs. They have mentioned a couple of ways to deal with this, One – using a correlated noise process and Two – Cokriging (Correlated prior).

I still don’t know, how I can incorporate any of these ideas into the GPML framework.

Also, are there any other GP libraries/frameworks that support multi-dimensional output?

**Answer**

I believe Twin Gaussian Processes is exactly what you are looking for.

I can’t describe the model better than the abstract of the paper itself, so I’m just gonna copy paste it:

We describe twin Gaussian processes (TGP) 1, a generic structured prediction method that uses Gaussian process (GP) priors [2] on both covariates and responses, both multivariate, and estimates outputs by minimizing the Kullback-Leibler divergence between two GP modeled as normal distributions over finite index sets of training and testing examples, emphasizing the goal that similar inputs should produce similar percepts and this should hold, on average, between their marginal distributions. TGP captures not only the interdependencies between covariates, as in a typical GP, but also those between responses, so correlations among both inputs and outputs are accounted for. TGP is exemplified, with promising results, for the reconstruction of 3d human poses from monocular and multicamera video sequences in the recently introduced HumanEva benchmark, where we achieve 5 cm error on average per 3d marker for models trained jointly, using data from multiple people and multiple activities. The method is fast and automatic: it requires no hand-crafting of the initial pose, camera calibration parameters, or the availability of a 3d body model associated with human subjects used for training or testing.

The authors have generously provided code and sample datasets for getting started.

**Attribution***Source : Link , Question Author : steadyfish , Answer Author : Yanshuai Cao*