Additive Gaussian Processes
David Duvenaud
Department of Engineering
\And Hannes Nickisch
%MPI Biological Cybernetics
MPI for Intelligent Systems
T\"{u}bingen
Germany
\And Carl Edward Rasmussen
We introduce a Gaussian process model of functions which are $\textit additive$. An additive function is one which decomposes into a sum of low-dimensional functions, each depending on only a subset of the input variables. Additive GPs generalize both Generalized Additive Models, and the standard GP models which use squared-exponential kernels. Hyperparameter learning in this model can be seen as Bayesian Hierarchical Kernel Learning (HKL). We introduce an expressive but tractable parameterization of the kernel function, which allows efficient evaluation of all input interaction terms, whose number is exponential in the input dimension. The additional structure discoverable by this model results in increased interpretability, as well as state-of-the-art predictive power in regression tasks.
main.tex
\documentclass{article} % For LaTeX2e
\usepackage{nips11submit_e,times}
\usepackage{cite}
\usepackage[pdftex]{graphicx}
\usepackage{rotating}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{array}
\usepackage{nth}
\makeatletter
\newcommand{\rmnum}[1]{\romannumeral #1}
\newcommand{\Rmnum}[1]{\expandafter\@slowromancap\romannumeral #1@}
\makeatother

\title{Additive Gaussian Processes}


\author{
David Duvenaud \\
Department of Engineering\\
Cambridge University\\
\texttt{dkd23@cam.ac.uk} \\
\And
Hannes Nickisch \\
%MPI Biological Cybernetics \\
MPI for Intelligent Systems \\
T\"{u}bingen, Germany \\
\texttt{hn@tue.mpg.de} \\
\And
Carl Edward Rasmussen \\
Department of Engineering \\
Cambridge University \\
\texttt{cer54@cam.ac.uk} \\
}

\newcommand{\fix}{\marginpar{FIX}}
\newcommand{\new}{\marginpar{NEW}}
\nipsfinalcopy % Uncomment for camera-ready version
\begin{document}
\maketitle

\begin{abstract}
We introduce a Gaussian process model of functions which are $\textit additive$.  An additive function is one which decomposes into a sum of low-dimensional functions, each depending on only a subset of the input variables. Additive GPs generalize both Generalized Additive Models, and the standard GP models which use squared-exponential kernels.  Hyperparameter learning in this model can be seen as Bayesian Hierarchical Kernel Learning (HKL).  We introduce an expressive but tractable parameterization of the kernel function, which allows efficient evaluation of all input interaction terms, whose number is exponential in the input dimension.  The additional structure discoverable by this model results in increased interpretability, as well as state-of-the-art predictive power in regression tasks.
\end{abstract}


\section{Introduction}
%$f: {\bf x} \mapsto y$
Most statistical regression models in use today are of the form: $g(y) = f(x_1) + f(x_2) + \dots + f(x_D)$.  Popular examples include logistic regression, linear regression, and Generalized Linear Models\cite{nelder1972generalized}.  This family of functions, known as Generalized Additive Models (GAM)\cite{hastie1990generalized}, are typically easy to fit and interpret.   Some extensions of this family, such as smoothing-splines ANOVA \cite{wahba1990spline}, add terms depending on more than one variable.  However, such models generally become intractable and difficult to fit as the number of terms increases.

At the other end of the spectrum are kernel-based models, which typically allow the response to depend on all input variables simultaneously.  These have the form: $y = f(x_1, x_2, \dots, x_D)$.  A popular example would be a Gaussian process model using a squared-exponential (or Gaussian) kernel.  We denote this model as SE-GP.  This model is much more flexible than the GAM, but its flexibility makes it difficult to generalize to new combinations of input variables. %To the extent that a set of variables is relevant to the output, this model must be highly uncertain about any novel combination of those variables.

In this paper, we introduce a Gaussian process model that generalizes both GAMs and the SE-GP.  This is achieved through a kernel which allow additive interactions of all orders, ranging from first order interactions (as in a GAM) all the way to $D$th-order interactions (as in a SE-GP).  Although this kernel amounts to a sum over an exponential number of terms, we show how to compute this kernel efficiently, and introduce a parameterization which limits the number of hyperparameters to $O(D)$.  A Gaussian process with this kernel function (an additive GP) constitutes a powerful model that allows one to automatically determine which orders of interaction are important.  We show that this model can significantly improve modeling efficacy, and has major advantages for model interpretability.  This model is also extremely simple to implement, and we provide example code.

We note that a similar breakthrough has recently been made, called Hierarchical Kernel Learning (HKL)\cite{DBLP:journals/corr/abs-0909-0844}.  HKL explores a similar class of models, and sidesteps the possibly exponential number of interaction terms by cleverly selecting only a tractable subset.  However, this method suffers considerably from the fact that cross-validation must be used to set hyperparameters.  In addition, the machinery necessary to train these models is immense.  Finally, on real datasets, HKL is outperformed by the standard SE-GP \cite{DBLP:journals/corr/abs-0909-0844}.

\section{Gaussian Process Models}

Gaussian processes are a flexible and tractable prior over functions, useful for solving regression and classification tasks\cite{rasmussen38gaussian}.  The kind of structure which can be captured by a GP model is mainly determined by its \emph{kernel}: the covariance function.  One of the main difficulties in specifying a Gaussian process model is in choosing a kernel which can represent the structure present in the data.  For small to medium-sized datasets, the kernel has a large impact on modeling efficacy.
  

%\subsection{Additivity in a GP}
\begin{figure}
\centering
\begin{tabular}{ccccc|c}
\hspace{-0.2cm}\includegraphics[width=0.2\textwidth]{figures/additive_kernel_sum_p2.pdf} & \hspace{-0.4cm} + \hspace{-0.4cm} & 
\includegraphics[width=0.2\textwidth]{figures/additive_kernel_sum_p1.pdf} & \hspace{-0.4cm} = \hspace{-0.4cm} & 
\includegraphics[width=0.2\textwidth]{figures/additive_kernel.pdf} &
\includegraphics[width=0.2\textwidth]{figures/sqexp_kernel.pdf} \\
$k_1(x_1, x_1')$ & & $k_2(x_2, x_2')$ & & $k_1(x_1,x_1') + k_2(x_2,x_2')$ &$k_1(x_1,x_1')k_2(x_2,x_2')$ \\
1D kernel & & 1D kernel & & 1st order kernel & 2nd order kernel \\ 
%& & & & & \\
%(Second Order) & & & & & Additive Kernel \\
$\downarrow$ & & $\downarrow$ & & $\downarrow$ & $\downarrow$  \\
\hspace{-0.2cm}\includegraphics[width=0.2\textwidth]{figures/additive_kernel_draw_sum_p1.pdf}& \hspace{-0.4cm} + \hspace{-0.4cm}& 
\includegraphics[width=0.2\textwidth]{figures/additive_kernel_draw_sum_p2.pdf}& \hspace{-0.4cm} = \hspace{-0.4cm}&
\includegraphics[width=0.2\textwidth]{figures/additive_kernel_draw_sum.pdf} &
\includegraphics[width=0.2\textwidth]{figures/sqexp_draw.pdf} \\
$f_1(x_1)$ & & $f_2(x_2)$ & & $f_1(x_1) + f_2(x_2)$ & $f(x_1, x_2)$ \\
draw from & & draw from & & draw from & draw from\\
1D GP prior & & 1D GP prior & & 1st order GP prior & 2nd order GP prior\\
%Draw from product kernel GP prior & Draw from additive kernel GP prior\\
\end{tabular}
\caption{A first-order additive kernel, and a product kernel.  Left: a draw from a first-order additive kernel corresponds to a sum of draws from one-dimensional kernels.  Right: functions drawn from a product kernel prior have weaker long-range dependencies, and less long-range structure.
%In this example, both kernels are composed of one dimensional squared-exponential kernels, but this need not be the case in general.
}
\label{fig:kernels}
\end{figure}

Figure \ref{fig:kernels} compares, for two-dimensional functions, a first-order additive kernel with a second-order kernel. We can see that a GP with a first-order additive kernel is an example of a GAM:  Each function drawn from this model is a sum of orthogonal one-dimensional functions.  Compared to functions drawn from the higher-order GP, draws from the first-order GP have more long-range structure.

  %The only weaknesses with this method are due to over-fitting the kernel hyperparameters, the kernel not being rich enough to capture the structure exhibited by the data, or due to the optimization process not finding the global optima of the hyperparameters.

%\subsection{Additivity in the wild}

We can expect many natural functions to depend only on sums of low-order interactions.  For example, the price of a house or car will presumably be well approximated by a sum of prices of individual features, such as a sun-roof.  
%Other features of the object may depend on the size of the object to jointly determine the price.  
Other parts of the price may depend jointly on a small set of features, such as the size and building materials of a house.
Capturing these regularities will mean that a model can confidently extrapolate to unseen combinations of features.


%\subsection{Additive Covariance}

%Figure \ref{fig:draws} shows that functions drawn from sqexp prior in high dimensions can have tons of zero crossings, and thus will take alot of data to learn.

%\subsection{Why additive functions can be expected to be a good model}

\section{Additive Kernels}
%\subsection{Definition}


We now give a precise definition of additive kernels.  We first assign each dimension $i \in \{1 \dots D\}$ a one-dimensional \emph{base kernel} $k_i(x_i, x'_i)$.  We then define the first order, second order and $n$th order additive kernel as:
\begin{eqnarray}
k_{add_1}({\bf x, x'}) & = & \sigma_1^2 \sum_{i=1}^D k_i(x_i, x_i') \\
k_{add_2}({\bf x, x'}) & = & \sigma_2^2 \sum_{i=1}^D \sum_{j = i + 1}^D k_i(x_i, x_i') k_j(x_j, x_j') \\
k_{add_n}({\bf x, x'}) & = & \sigma_n^2 \sum_{1 \leq i_1 < i_2 < ... < i_n \leq D} \left[ \prod_{d=1}^n k_{i_d}(x_{i_d}, x_{i_d}') \right]
%k_{add_D}({\bf x, x'}) & = & \sigma_D^2 \prod_{d=1}^D k_{d}(x_{d}, x_{d}')
\end{eqnarray}
where $D$ is the dimension of our input space, and $\sigma_n^2$ is the variance assigned to all $n$th order interactions. The $n$th covariance function is a sum of ${D \choose n}$ terms.  In particular, the $D$th order additive covariance function has ${D \choose D} = 1$ term, a product of each dimension's covariance function:
\begin{equation}
k_{add_D}({\bf x, x'}) = \sigma_D^2 \prod_{d=1}^D k_{d}(x_{d}, x_{d}')
\end{equation}
In the case where each base kernel is a one-dimensional squared-exponential kernel, the $D$th-order term corresponds to the multivariate squared-exponential kernel:
\begin{equation}
%\resizebox{\textwidth}{!}{$
k_{add_D}({\bf x, x'}) = \sigma_D^2 \prod_{d=1}^D k_{d}(x_{d}, x_{d}') = \sigma_D^2 \prod_{d=1}^D \exp \Big( -\frac{ ( x_{d} - x_{d}')^2}{2l^2_d} \Big) = \sigma_D^2  \exp \Big( -\sum_{d=1}^D \frac{ ( x_{d} - x_{d}')^2}{2l^2_d} \Big)
%$}
\end{equation}
also commonly known as the Gaussian kernel.  The full additive kernel is a sum of the additive kernels of all orders.
%
\subsection{Parameterization}

The only design choice necessary in specifying an additive kernel is the selection of a one-dimensional base kernel for each input dimension.  Any parameters (such as length-scales) of the base kernels can be learned as usual by maximizing the marginal likelihood of the training data.  

In addition to the hyperparameters of each dimension-wise kernel, additive kernels are equipped with a set of $D$ hyperparameters $\sigma_1^2 \dots \sigma_D^2$ controlling how much variance we assign to each order of interaction.  These ``order variance'' hyperparameters have a useful interpretation:  The $d$th order variance hyperparameter controls how much of the target function's variance comes from interactions of the $d$th order.
%
%\input{tables/r_concrete_500_hypers_table.tex}
%Table \ref{tbl:concrete} gives an example of hyperparameters learnt by both a SE-GP, and by those learnt by an additive GP whose base kernels are one-dimensional squared exponential kernels.  In this case, the 1st, 2nd and 3rd-order interactions contribute most to the final function.
%
%
Table \ref{tbl:all_orders} shows examples of normalized order variance hyperparameters learned on real datasets.
\input{tables/all_orders_table_frozen.tex}

On different datasets, the dominant order of interaction estimated by the additive model varies widely.  An additive GP with all of its variance coming from the 1st order is equivalent to a GAM; an additive GP with all its variance coming from the $D$th order is equivalent to a SE-GP.
%
% We can see that the length-scales learnt are not necessarily the same.  This is because, in a normal ARD-style kernel, the length-scale of a dimension is conflated with the variance of the function along that dimension.  The additive kernel separates these two hyperparameters.
%
%This table lets us examine the relative contribution of each order of interaction to the total function variance.  Again, the ARD kernel is equivalent to an additive kernel in which all the variance is constrained to arise only from the highest-order interactions.

%\subsubsection{Model Selection}

Because the hyperparameters can specify which degrees of interaction are important, the additive GP is an extremely general model.  
%By optimizing these hyperparameters, we are approximately performing weighted model selection over a set of $D$ models.
  If the function we are modeling is decomposable into a sum of low-dimensional functions, our model can discover this fact and exploit it (see Figure \ref{fig:synth2d}) .  If this is not the case, the hyperparameters can specify a suitably flexible model.
%In the limit of infinite data, a product kernel and its all-subsets expansion will converge to the same mean function (proof?).  However, in the medium-data regime, the additive kernel can discover structure in the functions it is modeling that the product kernel cannot.

%The expected number of zero crossings is $O(k^d)$ for product kernels (sqexp), $O(kd)$ for first-order additive, and $O\left(k  {d \choose r}\right)$ for $r$th order additive functions. [Conjecture]

%To push the analogy, using a product kernel is analogous to performing polynomial regression when we are forced to use a high-degree polynomial, even when the data do not support it.  One could also view the Gaussian kernel as one which makes the pessimistic assumption that the function we are model can vary independently for each new combination of variables observed.
\subsection{Interpretability}

As noted by Plate\cite{plate1999accuracy}, one of the chief advantages of additive models such as GAM is their interpretability.
% since a high-dimensional function is decomposed into a series of one- or two-dimensional functions.  
Plate also notes that by allowing high-order interactions as well as low-order interactions, one can trade off interpretability with predictive accuracy.  In the case where the hyperparameters indicate that most of the variance in a function can be explained by low-order interactions, it is useful and easy to plot the corresponding low-order functions, as in Figure \ref{fig:interpretable functions}. 

\begin{figure}[h]
\centering
\begin{tabular}{ccc}
\includegraphics[width=0.3\textwidth]{figures/interpretable_1st_order1.pdf} &
\includegraphics[width=0.3\textwidth]{figures/interpretable_1st_order2.pdf}& 
\includegraphics[width=0.3\textwidth]{figures/interpretable_2nd_order1.pdf}\\
\end{tabular}
\caption{Low-order functions on the concrete dataset.  Left, Centre:  By considering only first-order terms of the additive kernel, we recover a form of Generalized Additive Model, and can plot the corresponding 1-dimensional functions.  Green points indicate the original data, blue points are data after the mean contribution from the other dimensions' first-order terms has been subtracted.  The black line is the posterior mean of a GP with only one term in its kernel.  Right:  The posterior mean of a GP with only one second-order term in its kernel.}
\label{fig:interpretable functions}
\end{figure}

%\subsection{Allowable Base Kernels}
%Any one-dimensional base kernel is allowable, and each dimension can use a different base kernel.  One could even specify a mixture of base kernels for each dimension, allowing the hyperparameters to specify the mixture weights.

\subsection{Efficient Evaluation of Additive Kernels}
An additive kernel over $D$ inputs with interactions up to order $n$ has $O(2^n)$ terms.  Na\"{i}vely summing over these terms quickly becomes intractable.  In this section, we show how one can evaluate the sum over all terms in $O(D^2)$.

%\subsection{ Elementary Symmetric Polynomials}
The $n$th order additive kernel corresponds to the $n$th \textit{elementary symmetric polynomial}\cite{macdonald1998symmetric} \cite{stanley2001enumerative}, which we denote $e_n$.  For example:  if $\bf x$ has 4 input dimensions ($D = 4$), and if we let $z_i = k_i(x_i,x_i')$, then
\begin{align*}
%k_{a_1}({\bf x, x'}) & = e_0( z_1, z_2, z_3, z_4 ) & = & 1 \\
k_{add_1}({\bf x, x'}) & = e_1( z_1, z_2, z_3, z_4 ) = z_1 + z_2 + z_3 + z_4 \\
k_{add_2}({\bf x, x'}) & = e_2( z_1, z_2, z_3, z_4 ) = z_1 z_2 + z_1 z_3 + z_1z_4 + z_2 z_3 + z_2 z_4 + z_3 z_4 \\
k_{add_3}({\bf x, x'}) & = e_3( z_1, z_2, z_3, z_4 ) = z_1 z_2 z_3 + z_1 z_2 z_4 + z_1 z_3 z_4 + z_2 z_3 z_4 \\
k_{add_4}({\bf x, x'}) & = e_4( z_1, z_2, z_3, z_4 ) = z_1 z_2 z_3 z_4
\end{align*}
The Newton-Girard formulae give an efficient recursive form for computing these polynomials.  If we define $s_k$ to be the $k$th power sum:  $s_k(z_1,z_2,\dots,z_D) = \sum_{i=1}^Dz_i^k$, then
\begin{equation}
k_{add_n}({\bf x, x'}) = e_n(z_1,\dots,z_D) = \frac{1}{n} \sum_{k=1}^n (-1)^{(k-1)} e_{n-k}(z_1,\dots,z_D)s_k(z_1,\dots,z_D)
\end{equation}
%\begin{equation}
%$s_k(z_1,z_2,\dots,z_n) = z_1^k + z_2^k + \dots + z_D^k = \sum_{i=1}^Dz_i^k$.
%\end{equation}
Where $e_0 \triangleq 1$.  The Newton-Girard formulae have time complexity $O( D^2 )$, while computing a sum over an exponential number of terms.
%\subsection{Evaluation of Derivatives}

Conveniently, we can use the same trick to efficiently compute all of the necessary derivatives of the additive kernel with respect to the base kernels.  We merely need to remove the kernel of interest from each term of the polynomials:
\begin{align}
\frac{\partial k_{add_n}}{\partial z_j} & = e_{n-1}(z_1,\dots,z_{j-1},z_{j+1}, \dots z_D)
% & = \frac{1}{n-1} \sum_{k=1}^{n-1} (-1)^{(k-1)} e_{n-k-1}(z_1,\dots,z_{j-1},z_{j+1}, \dots z_D)s_k(z_1,\dots,z_{j-1},z_{j+1}, \dots z_D)
\end{align}
This trick allows us to optimize the base kernel hyperparameters with respect to the marginal likelihood.

%  The final derivative is a sum of multilinear terms, so if only one term depends on the hyperparameter under consideration, we can factorise it out and compute the sum with one degree less.
\subsection{Computation}
The computational cost of evaluating the Gram matrix of a product kernel (such as the SE kernel) is $O(N^2D)$, while the cost of evaluating the Gram matrix of the additive kernel is $O(N^2DR)$, where R is the maximum degree of interaction allowed (up to D).  In higher dimensions, this can be a significant cost, even relative to the fixed $O(N^3)$ cost of inverting the Gram matrix.
%
However, as our experiments show, typically only the first few orders of interaction are important for modeling a given function; hence if one is computationally limited, one can simply limit the maximum degree of interaction without losing much accuracy.

%\subsection{Code}
Additive Gaussian processes are particularly appealing in practice because their use requires only the specification of the base kernel.  All other aspects of GP inference remain the same.  All of the experiments in this paper were performed using the standard GPML toolbox\footnote{Available at \texttt{http://www.gaussianprocess.org/gpml/code/}}; code to perform all experiments is available at the author's website.\footnote{Example code available at: \texttt{http://mlg.eng.cam.ac.uk/duvenaud/}}


\section{Related Work}

Plate\cite{plate1999accuracy} constructs a form of additive GP, but using only the first-order and $D$th order terms.  This model is motivated by the desire to trade off the interpretability of first-order models, with the flexibility of full-order models.  Our experiments show that often, the intermediate degrees of interaction contribute most of the variance.

A related functional ANOVA GP model\cite{kaufman2010bayesian} decomposes the \emph{mean} function into a weighted sum of GPs. However, the effect of a particular degree of interaction cannot be quantified by that approach. Also, computationally, the Gibbs sampling approach used in \cite{kaufman2010bayesian} is disadvantageous.
%[Todo:  read and mention  and \cite{friedman1981projection} from Hannes]

%\subsection{Multiple Kernel Learning}

%Our current framework relates to Multiple Kernel Learning in that we are learning the weights in a certain mixture of kernels.  
Christoudias et al.\cite{christoudias2009bayesian} previously showed how mixtures of kernels can be learnt by gradient descent in the Gaussian process framework.  They call this \emph{Bayesian localized multiple kernel learning}.
However, their approach learns a mixture over a small, fixed set of kernels, while our method learns a mixture over all possible products of those kernels.

\subsection{Hierarchical Kernel Learning}

\begin{figure}
\centering
\begin{tabular}{c|c|c|c}
\hspace{-0.06in} \includegraphics[width=0.23\textwidth]{figures/compare_models/hkl.pdf} \hspace{-0.07in} &
\hspace{-0.06in} \includegraphics[width=0.23\textwidth]{figures/compare_models/gam.pdf} \hspace{-0.07in} &
\hspace{-0.06in} \includegraphics[width=0.23\textwidth]{figures/compare_models/ard.pdf} \hspace{-0.07in} &
\hspace{-0.06in} \includegraphics[height=0.19\textwidth]{figures/compare_models/additive.pdf} \\
HKL kernel & GP-GAM kernel & Squared-exp GP & Additive GP kernel\\
 & & kernel & \\
\end{tabular}
\caption{
%A comparison of model flexibility between HKL, additive GPs, the GP-Generalized Additive Model, and the squared-exponential GP model.  
A comparison of different models.  Nodes represent different interaction terms, ranging from first-order to fourth-order interactions.  Far left:  HKL can select a hull of interaction terms, but must use a pre-determined weighting over those terms.  Far right: the additive GP model can weight each order of interaction seperately.  Neither the HKL nor the additive model dominate one another in terms of flexibility, however the GP-GAM and the SE-GP are special cases of additive GPs. }
\label{hulls-figure}
\end{figure}

Bach\cite{DBLP:journals/corr/abs-0909-0844} uses a regularized optimization framework to learn a weighted sum over an exponential number of kernels which can be computed in polynomial time.  The subsets of kernels considered by this method are restricted to be a \textit{hull} of kernels.\footnote{In the setting we are considering in this paper, a hull can be defined as a subset of all terms such that if term $\prod_{j \in J} k_j(\bf x, x')$ is included in the subset, then so are all terms $\prod_{j \in J / i} k_j(\bf x, x')$, for all $i \in J$.  For details, see \cite{DBLP:journals/corr/abs-0909-0844}.}
Given each dimension's kernel, and a pre-defined weighting over all terms, HKL performs model selection by searching over hulls of interaction terms.
% \subsubsection{All-subsets kernel with uniform weightings}
In \cite{DBLP:journals/corr/abs-0909-0844}, Bach also fixes the relative weighting between orders of interaction with a single term $\alpha$, computing the sum over all orders by:
\begin{equation}
\label{eqn:uniform}
k_{a}({\bf x, x'}) = v_D^2 \prod_{d=1}^D \left(1 + \alpha k_{d}(x_{d}, x_{d}') \right)
\end{equation}
which has computational complexity $O(D)$.  However, this formulation forces the weight of all $n$th order terms to be weighted by $\alpha^n$.

Figure \ref{hulls-figure} contrasts the HKL hull-selection method with the Additive GP hyperparameter-learning method. Neither method dominates the other in flexibility.  The main difficulty with the approach of \cite{DBLP:journals/corr/abs-0909-0844} is that hyperparameters are hard to set other than by cross-validation.  In contrast, our method optimizes the hyperparameters of each dimension's base kernel, as well as the relative weighting of each order of interaction. 


\subsection{ANOVA Procedures}

Vapnik \cite{vapnik1998statistical} introduces the support vector ANOVA decomposition, which has the same form as our additive kernel.  However, they recommend approximating the sum over all $D$ orders with only one term ``of appropriate order'', presumably because of the difficulty of setting the hyperparameters of an SVM. Stitson et al.\cite{stitson1999support} performed experiments which favourably compared the support vector ANOVA decomposition to polynomial and spline kernels.  They too allowed only one order to be active, and set hyperparameters by cross-validation.
%
%  The order of the kernel, kernel hyperparameters, the allowable regression error $\epsilon$, and the cost hyperparameter $c$ were all set by a lengthy cross-validation process.

%\subsection{Smoothing Splines ANOVA}
A closely related procedure from the statistics literature is smoothing-splines ANOVA (SS-ANOVA)\cite{wahba1990spline}. An SS-ANOVA model is estimated as a weighted sum of splines along each dimension, plus a sum of splines over all pairs of dimensions, all triplets, etc, with each individual interaction term having a separate weighting parameter.  Because the number of terms to consider grows exponentially in the order, in practice, only terms of first and second order are usually considered.  Learning in SS-ANOVA is usually done via penalized-maximum likelihood with a fixed sparsity hyperparameter.
% estimated by generalized cross-validation\cite{craven1978smoothing}. 
%MARS\cite{friedman1991multivariate} is another spline-based regression method.

In contrast to these procedures, our method can easily include all $D$ orders of interaction, each weighted by a separate hyperparameter. As well, we can learn kernel hyperparameters individually per input dimension, allowing automatic relevance determination to operate.

\subsection{Non-local Interactions}

%In this section we give one motivation for why low-order interactions might help our predictive accuracy.
%
%The SE-GP model relies on local smoothing to make predictions at novel locations.  Recent work by Bengio et. al. discusses the limitations
By far the most popular kernels for regression and classification tasks on continuous data are the squared exponential (Gaussian) kernel, and the Mat\'{e}rn kernels.  These kernels depend only on the scaled Euclidean distance between two points, both having the form: $ k({\bf x, x'}) = f(\sum_{d=1}^D \left( x_{d} - x_{d}' \right)^2 / l_d^2)$.
%\begin{eqnarray}
%k_{se}({\bf x, x'}) & = & v_D^2  \exp \left( -\frac{r^2}{2} \right) \\
%k_{\nu}({\bf x, x'}) & = & \frac{2^{1-\nu}}{\Gamma(\nu)}\left(\sqrt{2\nu}r\right) K_\nu \left(\sqrt{2\nu}r\right)
%\end{eqnarray}
%Where
%\begin{equation}
%$r = \sqrt{\sum_{d=1}^D \left( x_{d} - x_{d}' \right)^2 / l_d^2 }$.
%\end{equation}
Bengio et al.\cite{bengio2006curse} argue that models based on squared-exponential kernels are particularily susceptible to the \textit{curse of dimensionality}.  They emphasize that the locality of the kernels means that these models cannot capture non-local structure.  They argue that many functions that we care about have such structure.  Methods based solely on local kernels will require training examples at all combinations of relevant inputs.

\begin{figure}[h]
\centering
\begin{tabular}{cccc}
\hspace{-0.25in} \includegraphics[width=0.27\textwidth]{figures/3d_add_kernel_1.pdf} &
\hspace{-0.25in} \includegraphics[width=0.27\textwidth]{figures/3d_add_kernel_2.pdf} &
\hspace{-0.25in} \includegraphics[width=0.27\textwidth]{figures/3d_add_kernel_3.pdf} & 
\hspace{-0.25in} \includegraphics[width=0.27\textwidth]{figures/3d_add_kernel_321.pdf}\\
1st order interactions & 2nd order interactions & 3rd order interactions & All interactions \\
$k_1 + k_2 + k_3$ & $k_1k_2 + k_2k_3 + k_1k_3$ & $k_1k_2k_3$ & \\
& & (Squared-exp kernel) & (Additive kernel)\\
\end{tabular}
\caption{Isocontours of additive kernels in 3 dimensions.  The third-order kernel only considers nearby points relevant, while the lower-order kernels allow the output to depend on distant points, as long as they share one or more input value.}
\label{fig:kernels3d}
\end{figure}

Additive kernels have a much more complex structure, and allow extrapolation based on distant parts of the input space, without spreading the mass of the kernel over the whole space.  For example, additive kernels of the second order allow strong non-local interactions between any points which are similar in any two input dimensions.
Figure \ref{fig:kernels3d} provides a geometric comparison between squared-exponential kernels and additive kernels in 3 dimensions.


\section{Experiments}

\subsection{Synthetic Data}

Because additive kernels can discover non-local structure in data, they are exceptionally well-suited to problems where local interpolation fails.  
%[Would this make them good for nonstationary noise?]
\begin{figure}[h]
\centering
\begin{tabular}{cccc}
\hspace{-0.1in}\includegraphics[width=0.24\textwidth]{figures/1st_order_censored_truth.pdf} &
\hspace{-0.1in}\includegraphics[width=0.24\textwidth]{figures/1st_order_censored_ard.pdf}&
\hspace{-0.1in}\includegraphics[width=0.24\textwidth]{figures/1st_order_censored_add.pdf}& 
\hspace{-0.1in}\includegraphics[width=0.24\textwidth]{figures/1st_order_censored_d1d2.pdf}\\ 
True Function & Squared-exp GP & Additive GP & Additive GP \\
 \& data locations & posterior mean & posterior mean & 1st-order functions\\
%\includegraphics[width=0.3\textwidth]{figures/1st_order_censored_data.pdf} &
%\includegraphics[width=0.3\textwidth]{figures/1st_order_censored_d1.pdf}&
%\includegraphics[width=0.3\textwidth]{figures/1st_order_censored_d2.pdf}\\
%Training Data Locations & Estimated 1D $f(x_1)$ & Estimated 1D $f(x_2)$ \\
\end{tabular}
\caption{Long-range inference in functions with additive structure.%  The additive GP is able to discover the additive pattern, and use it to fill in a distant mode.  The ARD kernel can only interpolate, and thus predicts the mean in locations missing data.
}
\label{fig:synth2d}
\end{figure}
%Figure \ref{fig:synth2d} shows a simple experiment demonstrating the ability of additive kernels to discover low-order structure, and to exploit that structure to make predictions about unseen combinations of inputs.
Figure \ref{fig:synth2d} shows a dataset which demonstrates this feature of additive GPs, consisting of data drawn from a sum of two axis-aligned sine functions.  The training set is restricted to a small, L-shaped area; the test set contains a peak far from the training set locations.  The additive GP recovered both of the original sine functions (shown in green), and inferred correctly that most of the variance in the function comes from first-order interactions.  The ability of additive GPs to discover long-range structure suggests that this model may be well-suited to deal with covariate-shift problems.

\subsection{Experimental Setup}
%\subsubsection{Learning}

%\subsubsection{Overfitting}
%A multivariate squared exponential kernel has $D + 1$ hyperparameters, while an additive function with one hyperparameter per dimension (e.g., a scale parameter) on each dimension's covariance function will have $D \times 2$ hyperparameters.  Because each additional hyperparameter increases the tendency to overfit, we recommend allowing only one hyperparameter per input dimension. 

%
%Since there always exists a parameterization of the additive function corresponding to the product kernel, we initialize the hyperparameter search for the additive kernel by first learning hyperparameters for the equivalent product kernel.  This ensures that we start our search in a reasonable area of hyperparameter space.
%
%  Results were averaged over 10 cross-validation folds.  To limit the possibility of overfitting, we parametrized each of our one-dimensional kernel functions with only the lengthscale.  
%Note that this parameterization is still more flexible than previous methods (HKL, SS-ANOVA).  % This needs to be double-checked.

%\subsubsection{Initialization}
%We tried 2 different hyperparameterizations, and 3 initialization strategies: \texttt{vo} indicates learning only the variance parameter of each one-dimensional kernel, and \texttt{lo} indicates learning only the lengthscale.  \texttt{grow} indicates that the model was initialized with only 1st-order interactions, and higher interactions were added in stages.  \texttt{ard init} indicates that lengthscales were initialized by first optimizing the hyperparameters of a squared-exp kernel, then copying the lengthscales for use in an additive kernel.
%\subsection{Models}

On a diverse collection of datasets, we compared five different models.  In the results tables below, GP Additive refers to a GP using the additive kernel with squared-exp base kernels.  For speed, we limited the maximum order of interaction in the additive kernels to 10.  GP-GAM denotes an additive GP model with only first-order interactions.  GP Squared-Exp is a GP model with a squared-exponential ARD kernel.  HKL\footnote{Code for HKL available at \texttt{http://www.di.ens.fr/\textasciitilde fbach/hkl/}} was run using the all-subsets kernel, which corresponds to the same set of kernels as considered by the additive GP with a squared-exp base kernel.     

%For all 3 GP models, we allowed 5 multiple restarts for all of our hyperparameter maximization steps, choosing the set of hyperparameters with the highest training-set marginal likelihood.  We ran each hyperparameter search for 500 function evaluations by the usual method of maximizing marginal likelihood, using L-BFGS \cite{nocedal1980updating}.  In addition to kernel hyperparameters, we fit a constant mean function to the data.
For all GP models, we fit hyperparameters by the standard method of maximizing training-set marginal likelihood, using L-BFGS \cite{nocedal1980updating} for 500 iterations, allowing five random restarts.  In addition to learning kernel hyperparameters, we fit a constant mean function to the data.
%
In the classification experiments, GP inference was done using Expectation Propagation \cite{minka2001expectation}.
 
\subsection{Results}

Tables \ref{tbl:Regression Mean Squared Error}, \ref{tbl:Regression Negative Log Likelihood}, \ref{tbl:Classification Percent Error} and \ref{tbl:Classification Negative Log Likelihood} show mean performance across 10 train-test splits.  Because HKL does not specify a noise model, it could not be included in the likelihood comparisons.

\input{tables/reg_table_mse2.tex}
\input{tables/reg_table_ll2.tex}
\input{tables/class_table2.tex}
\input{tables/class_table_ll2.tex}

The model with best performance on each dataset is in bold, along with all other models that were not significantly different under a paired t-test. The additive model never performs significantly worse than any other model, and sometimes performs significantly better than all other models.  The difference between all methods is larger in the case of regression experiments. The performance of HKL is consistent with the results in \cite{DBLP:journals/corr/abs-0909-0844}, performing competitively but slightly worse than SE-GP.%  This is especially clear in the case of the regression tasks.

The additive GP performed best on datasets well-explained by low orders of interaction, and approximately as well as the SE-GP model on datasets which were well explained by high orders of interaction (see table \ref{tbl:all_orders}).
Because the additive GP is a superset of both the GP-GAM model and the SE-GP model, instances where the additive GP performs slightly worse are presumably due to over-fitting, or due to the hyperparameter optimization becoming stuck in a local maximum. % Absence of over-fitting may explain the relatively strong performance of GP-GAM on classification tasks.  
Additive GP performance can be expected to benefit from integrating out the kernel hyperparameters.



\section{Conclusion}

%We present a simple model which generalizes two widely-used classes of models.  A large increase in modeling power is gained at only modest computational cost.
We present additive Gaussian processes: a simple family of models which generalizes two widely-used classes of models.  Additive GPs also introduce a tractable new type of structure into the GP framework.   Our experiments indicate that such additive structure is present in real datasets, allowing our model to perform better than standard GP models.  In the case where no such structure exists, our model can recover arbitrarily flexible models, as well.

In addition to improving modeling efficacy, the additive GP also improves model interpretability:  the order variance hyperparameters indicate which sorts of structure are present in our model.

Compared to HKL, which is the only other tractable procedure able to capture the same types of structure, our method benefits from being able to learn individual kernel hyperparameters, as well as the weightings of different orders of interaction.  Our experiments show that additive GPs are a state-of-the-art regression model.




%The related HKL framework

\subsubsection*{Acknowledgments}
The authors would like to thank John J. Chew and Guillaume Obozonksi for their helpful comments.
%\Rmnum{3}
%\pagebreak
%\nocite{*}
%\bibliographystyle{alpha}
%\bibliographystyle{apalike}
\bibliographystyle{unsrt}
\bibliography{additive}

\end{document}

% ==============================================================================================================
% ==============================================================================================================
% ==============================================================================================================

\pagebreak
\section{Stuff to Cut}

\begin{figure}
\centering
\begin{tabular}{cc}
\hspace{-0.1in}\includegraphics[width=0.475\textwidth]{figures/class_graph.pdf} &
\hspace{-0.1in}\includegraphics[width=0.475\textwidth]{figures/class_graph_ll.pdf}\\
\hspace{-0.1in}\includegraphics[width=0.475\textwidth]{figures/reg_graph.pdf}& 
\hspace{-0.1in}\includegraphics[width=0.475\textwidth]{figures/reg_graph_ll.pdf}\\ 
\end{tabular}
\label{fig:results}
\end{figure}


%\section{Discussion}
%\input{tables/housing_hypers_table.tex}
%\input{tables/pima_hypers_table.tex}


%As a special case, additive kernels with only one degree of interaction can be computed in something like linear time [cite Yunus?]
 % E = elsympol(Kd(:,:,[1:j-1,j+1:D]),max(R)-1);


\subsection{Bach Synthetic Dataset}
In addition to standard UCI repository datasets, we generated a synthetic dataset following the same recipe as \cite{DBLP:journals/corr/abs-0909-0844}: From a covariance matrix drawn from a Wishart distribution with 1024 degrees of freedom, we select 8 variables.  We then construct the non-linear function $f(X) = \sum_{i=1}^4 \sum_{j=1+1}^4 X_i X_j + \epsilon$, which sums all 2-way products of the first 4 variables, and adds Gaussian noise $\epsilon$.  This dataset is one which can be predicted well by a kernel which is a sum of two-way interactions over the first 4 variables, ignoring the extra 4 noisy copies.

This dataset was designed by \cite{DBLP:journals/corr/abs-0909-0844} to demonstrate the advantages of HKL over GP-ARD. 

 If the dataset is large enough, HKL can construct a hull around only those subsets of cross terms that are optimal for predicting the output.  GP-ARD, in contrast, can only learn to ignore the noisy copy variables, but cannot learn to ignore the higher-term interactions between the predictive variables.  However, a GP with an additive kernel can learn both to ignore irrelevant variables, and to ignore certain orders of interaction.  In this example, the additive GP is able to recover the relevant structure.

\input{tables/synth_c_hypers_table.tex}

Table \ref{tbl:bach synth c 200} shows the hyperparameters learnt.  Note that while GP-ARD increases the lengthscale of the irrelevant variables, GP-AS reduces the signal variance.
 
 
\subsection{Polynomial Kernels}

A simple variation on the all-subsets kernel would be one which includes repeated terms, of the form
\begin{equation}
k_{h_n}({\bf x, x'}) = v_n^2 \sum_{i_1, \dots, i_n} \prod_{d=1}^n k_{i_d}(x_{i_d}, x_{i_d}') = v_n^2 \prod_{d=1}^n \sum_{i_1, \dots, i_n}^n k_{i_d}(x_{i_d}, x_{i_d}')
\end{equation}
This formulation allows repeated terms, such as $x_i^2$ and is equivalent to the polynomial kernel\cite{shawe2004kernel}.  However, this formulation cases the length scale of each dimension to vary with the order of interaction considered.


\subsection{Future Work}
Since the non-local structure capturable by additive kernels is necessarily axis-aligned, we can naturally consider that combining the hyperparameter optimization with an initial rotation of the input space might allow us to recover non-axis aligned additivity in functions.


A Gaussian process with an additive kernel corresponds to a prior over functions which are additive.
If a function $\bf{f} = \bf{a} + \bf{b}$ is a sum of two functions $\bf a$ and $\bf b$ drawn from a Gaussian process prior with covariances $\bf K_a$ and $\bf K_b$, then equivalently, f is drawn from a Gaussian process prior with covariance $\bf K_f = \bf K_a + \bf K_b$.  The converse also holds.  If $\bf K_a$ and $\bf K_b$ are positive definite, then so is $\bf K_f$.

%[Equation for derivatives]

Note that we are free to choose a different covariance function along each dimension.

[ Explain how squared exp somehow puts too much mass on complicated hypotheses, or is unable to represent simple hypotheses ].

[ Show differences in marginal likelihoods]

[Should we try learning a convex hull of features as well?]

[ To discuss: Shawe-Taylor says in Remark 9.3 that we can control the relative weights of the degree monomials by just computing $\prod(k(x,x') + R)$.  If $R$ is high then the high-degree monomials don't matter as much.  Seems like we end up with a huge constant at the end, though...]

\cite{shawe2004kernel} define ANOVA kernels thusly: "The ANOVA kernel of degree d is like the all-subsets kernel except that it is restricted to subsets of the given cardinality $d$."

\subsection{Graph Kernels}

\cite{shawe2004kernel} introduce a very flexible way of specifying sums of products of kernels, called \textit{graph kernels}. [Should this be in future work?]

\begin{figure}[h]
\centering
\begin{tabular}{cc}
\includegraphics[width=0.5\textwidth]{figures/housing_order_mse.pdf}& 
\includegraphics[width=0.5\textwidth]{figures/housing_order_ll.pdf} \\
Mean Squared Error & Log Likelihood\\
\end{tabular}
\caption{ Likelihood versus order of interaction }
\label{fig:housing}
\end{figure}



\begin{figure}[h]
\centering
\begin{tabular}{cc}
\includegraphics[width=0.5\textwidth]{figures/synth_plot_1}\\
\end{tabular}
\caption{ ARD vs AS in an irrelevant variables contest. }
\label{fig:housing}
\end{figure}



%Especially good for trying to find the effects of interventions?
%The fact that our model is additive does not mean that it ignores multiplicative, or higher-order interactions


all_orders_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 03-Jun-2011 00:22:23
\begin{table}[h]
\caption{{\small
Relative variance contribution of each order in the additive model, on different datasets. Here, the maximum order of interaction is set to 10, or smaller if the input dimension less than 10.  Values are normalized to sum to 100.
}}
\label{tbl:all_orders}
\begin{center}
\begin{tabular}{r | r r r r r r r r r r}
Order of interaction & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} & \nth{9} & \nth{10} \\ \hline
pima  & $0.1 $ & $0.1 $ & $0.1 $ & $0.3 $ & $1.5 $ & $96.4 $ & $1.4 $ & $0.0 $ & & \\
liver  & $0.0 $ & $0.2 $ & $99.7 $ & $0.1 $ & $0.0 $ & $0.0 $ & & & & \\
heart  & $77.6 $ & $0.0 $ & $0.0 $ & $0.0 $ & $0.1 $ & $0.1 $ & $0.1 $ & $0.1 $ & $0.1 $ & $22.0 $ \\
concrete  & $70.6 $ & $13.3 $ & $13.8 $ & $2.3 $ & $0.0 $ & $0.0 $ & $0.0 $ & $0.0 $ & & \\
pumadyn-8nh  & $0.0 $ & $0.1 $ & $0.1 $ & $0.1 $ & $0.1 $ & $0.1 $ & $0.1 $ & $99.5 $ & & \\
servo  & $58.7 $ & $27.4 $ & $0.0 $ & $13.9 $ & & & & & & \\
housing  & $0.1 $ & $0.6 $ & $80.6 $ & $1.4 $ & $1.8 $ & $0.8 $ & $0.7 $ & $0.8 $ & $0.6 $ & $12.7 $ \\
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

r_heart_hypers_table_grow.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 20-May-2011 20:59:40
\begin{table}[h]
\caption{{\small
Hyperparameters for r heart grow dataset
}}
\label{tbl:r heart grow}
\begin{center}
\begin{tabular}{l | r r r r r r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  & $x_9$  & $x_10$  & $x_11$  & $x_12$  & $x_13$  \\ \hline
Sq-exp Lengthscale & $0.11$  & $0.13$  & $0.08$  & $2.20$  & $357.27$  & $0.21$  & $2.85$  & $54.17$  & $0.12$  & $1.99$  & $241.64$  & $272.17$  & $240.19$  \\ 
\hline
Additive Lengthscale & $0.22$  & $0.62$  & $2.15$  & $3.79$  & $0.39$  & $7.08$  & $7.06$  & $2.63$  & $2.61$  & $2.98$  & $2.65$  & $6.25$  & $0.89$  \\
\hline
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} & \nth{9} & \nth{10} \\
Additive Order Variance & $0.1$\% & $0.1$\% & $0.1$\% & $0.1$\% & $0.1$\% & $0.1$\% & $0.1$\% & $0.1$\% & $9.1$\% & $90.1$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

r_heart_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 03-Jun-2011 00:23:43
\begin{table}[h]
\caption{{\small
Hyperparameters for heart dataset.
}}
\label{tbl:heart}
\begin{center}
\begin{tabular}{r | r r r r r r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  & $x_9$  & $x_10$  & $x_11$  & $x_12$  & $x_13$  \\ \hline
Sq-exp lengthscale & $2.74$  & $1.11$  & $1.65$  & $4.07$  & $121.56$  & $4.91$  & $12.92$  & $7.53$  & $5.26$  & $3.61$  & $172.40$  & $1.95$  & $1.83$  \\ 
\hline
Additive lengthscale & $0.34$  & $0.42$  & $0.05$  & $4.72$  & $2.53$  & $56.99$  & $3.98$  & $2.64$  & $2.79$  & $2.42$  & $2.33$  & $0.07$  & $1.43$  \\
\hline
Order of interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} & \nth{9} & \nth{10} \\
Additive order variance & $77.6$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.1$\% & $0.1$\% & $0.1$\% & $0.1$\% & $0.1$\% & $22.0$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

pumadyn512_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 12-May-2011 16:56:21
\begin{table}[h]
\caption{{\small
Hyperparameters for pumadyn512 dataset
}}
\label{tbl:pumadyn512}
\begin{center}
\begin{tabular}{l | r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  \\ \hline
ARD Lengthscale & $137.74$  & $1.55$  & $1.42$  & $2543.62$  & $2727.28$  & $2113.23$  & $2493.23$  & $2745.56$  \\ 
\hline
ADD Lengthscale & $119.00$  & $1.55$  & $1.42$  & $1897.37$  & $2084.78$  & $1594.39$  & $1852.53$  & $2063.86$  \\
ADD Variance & $0.88$ & $1.08$ & $0.94$ & $0.90$ & $0.90$ & $0.90$ & $0.90$ & $0.90$ \\ \hline
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} \\
ADD Order Variance & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $100.0$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

abalone_500_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 12-May-2011 16:56:21
\begin{table}[h]
\caption{{\small
Hyperparameters for abalone 500 dataset
}}
\label{tbl:abalone 500}
\begin{center}
\begin{tabular}{l | r r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  & $x_9$  \\ \hline
ARD Lengthscale & $0.37$  & $0.02$  & $0.21$  & $0.06$  & $0.03$  & $5.55$  & $0.72$  & $1.49$  & $0.17$  \\ 
\hline
ADD Lengthscale & $0.37$  & $0.02$  & $0.21$  & $0.06$  & $0.03$  & $2.83$  & $0.73$  & $1.37$  & $0.17$  \\
ADD Variance & $0.65$ & $0.59$ & $0.95$ & $1.00$ & $1.73$ & $0.73$ & $0.75$ & $0.70$ & $2.63$ \\ \hline
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} & \nth{9} \\
ADD Order Variance & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.2$\% & $0.1$\% & $99.6$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

ionosphere_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 12-May-2011 16:56:20
\begin{table}[h]
\caption{{\small
Hyperparameters for ionosphere dataset
}}
\label{tbl:ionosphere}
\begin{center}
\begin{tabular}{l | r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  & $x_9$  & $x_10$  & $x_11$  & $x_12$  & $x_13$  & $x_14$  & $x_15$  & $x_16$  & $x_17$  & $x_18$  & $x_19$  & $x_20$  & $x_21$  & $x_22$  & $x_23$  & $x_24$  & $x_25$  & $x_26$  & $x_27$  & $x_28$  & $x_29$  & $x_30$  & $x_31$  & $x_32$  \\ \hline
ARD Lengthscale & $418.00$  & $61384.65$  & $1.12$  & $600.24$  & $2023.45$  & $41752.37$  & $764.96$  & $605.18$  & $588.51$  & $3962.02$  & $580.67$  & $3226.23$  & $1811.45$  & $1136.30$  & $1599.51$  & $2307.05$  & $586.22$  & $2440.60$  & $1522.15$  & $804.51$  & $506.52$  & $0.93$  & $1953.48$  & $1386.07$  & $2174.96$  & $0.68$  & $7030.63$  & $9761.00$  & $2224.68$  & $79535.55$  & $4149.84$  & $306363.78$  \\ 
\hline
ADD Lengthscale & $342.74$  & $61380.83$  & $1.16$  & $552.25$  & $1703.40$  & $41663.14$  & $473.03$  & $506.46$  & $419.03$  & $3785.82$  & $348.09$  & $3130.99$  & $1294.73$  & $933.35$  & $1504.78$  & $1765.24$  & $468.96$  & $2503.94$  & $1227.99$  & $729.17$  & $291.96$  & $4.30$  & $1957.94$  & $1217.86$  & $2158.30$  & $0.18$  & $6934.52$  & $9722.82$  & $1611.85$  & $79537.41$  & $4197.43$  & $306360.44$  \\
ADD Variance & $0.53$ & $0.31$ & $0.33$ & $0.19$ & $0.39$ & $0.31$ & $0.95$ & $0.10$ & $0.38$ & $0.33$ & $0.42$ & $0.32$ & $0.70$ & $0.45$ & $0.34$ & $0.59$ & $0.31$ & $0.28$ & $0.46$ & $0.38$ & $0.05$ & $0.00$ & $0.29$ & $0.36$ & $0.30$ & $0.02$ & $0.32$ & $0.31$ & $0.72$ & $0.31$ & $0.30$ & $0.31$ \\ \hline
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} & \nth{9} & \nth{10} \\
ADD Order Variance & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $100.0$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

sonar_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 12-May-2011 16:56:20
\begin{table}[h]
\caption{{\small
Hyperparameters for sonar dataset
}}
\label{tbl:sonar}
\begin{center}
\begin{tabular}{l | r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  & $x_9$  & $x_10$  & $x_11$  & $x_12$  & $x_13$  & $x_14$  & $x_15$  & $x_16$  & $x_17$  & $x_18$  & $x_19$  & $x_20$  & $x_21$  & $x_22$  & $x_23$  & $x_24$  & $x_25$  & $x_26$  & $x_27$  & $x_28$  & $x_29$  & $x_30$  & $x_31$  & $x_32$  & $x_33$  & $x_34$  & $x_35$  & $x_36$  & $x_37$  & $x_38$  & $x_39$  & $x_40$  & $x_41$  & $x_42$  & $x_43$  & $x_44$  & $x_45$  & $x_46$  & $x_47$  & $x_48$  & $x_49$  & $x_50$  & $x_51$  & $x_52$  & $x_53$  & $x_54$  & $x_55$  & $x_56$  & $x_57$  & $x_58$  & $x_59$  & $x_60$  \\ \hline
ARD Lengthscale & $0.37$  & $0.02$  & $0.77$  & $0.93$  & $0.10$  & $2.19$  & $2.19$  & $0.64$  & $0.97$  & $0.82$  & $0.53$  & $1.45$  & $0.29$  & $0.56$  & $0.76$  & $1.97$  & $0.72$  & $0.59$  & $0.19$  & $0.94$  & $0.06$  & $1.44$  & $3.07$  & $0.25$  & $1.64$  & $2.30$  & $0.05$  & $1.25$  & $0.39$  & $1.40$  & $1.58$  & $1.43$  & $2.37$  & $1.38$  & $2.19$  & $0.09$  & $0.65$  & $0.55$  & $0.03$  & $0.90$  & $0.12$  & $2.62$  & $0.20$  & $1.20$  & $0.86$  & $0.16$  & $0.62$  & $0.14$  & $1.30$  & $1.17$  & $3.22$  & $1.28$  & $0.27$  & $1.03$  & $0.14$  & $0.65$  & $0.63$  & $0.67$  & $0.44$  & $2.02$  \\ 
\hline
ADD Lengthscale & $0.46$  & $0.02$  & $0.85$  & $1.05$  & $0.11$  & $2.30$  & $2.32$  & $0.75$  & $1.00$  & $0.85$  & $0.60$  & $1.42$  & $0.36$  & $0.66$  & $0.85$  & $2.00$  & $0.78$  & $0.67$  & $0.22$  & $1.04$  & $0.07$  & $1.56$  & $3.14$  & $0.29$  & $1.74$  & $2.38$  & $0.05$  & $1.31$  & $0.47$  & $1.51$  & $1.71$  & $1.57$  & $2.49$  & $1.51$  & $2.28$  & $0.10$  & $0.74$  & $0.62$  & $0.03$  & $1.04$  & $0.14$  & $2.72$  & $0.23$  & $1.30$  & $0.97$  & $0.19$  & $0.72$  & $0.16$  & $1.37$  & $1.32$  & $3.29$  & $1.38$  & $0.31$  & $1.21$  & $0.15$  & $0.80$  & $0.73$  & $0.77$  & $0.53$  & $2.13$  \\
ADD Variance & $0.07$ & $0.06$ & $0.07$ & $0.07$ & $0.07$ & $0.07$ & $0.07$ & $0.07$ & $0.06$ & $0.06$ & $0.07$ & $0.06$ & $0.07$ & $0.07$ & $0.07$ & $0.06$ & $0.07$ & $0.07$ & $0.07$ & $0.07$ & $0.07$ & $0.07$ & $0.07$ & $0.07$ & $0.07$ & $0.07$ & $0.07$ & $0.07$ & $0.07$ & $0.07$ & $0.07$ & $0.07$ & $0.07$ & $0.07$ & $0.07$ & $0.07$ & $0.07$ & $0.07$ & $0.06$ & $0.07$ & $0.07$ & $0.07$ & $0.07$ & $0.07$ & $0.07$ & $0.07$ & $0.07$ & $0.07$ & $0.07$ & $0.07$ & $0.06$ & $0.07$ & $0.07$ & $0.07$ & $0.07$ & $0.07$ & $0.07$ & $0.07$ & $0.07$ & $0.07$ \\ \hline
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} & \nth{9} & \nth{10} \\
ADD Order Variance & $2.6$\% & $96.5$\% & $0.8$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

r_ionosphere_hypers_table_grow.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 20-May-2011 20:59:40
\begin{table}[h]
\caption{{\small
Hyperparameters for r ionosphere grow dataset
}}
\label{tbl:r ionosphere grow}
\begin{center}
\begin{tabular}{l | r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  & $x_9$  & $x_10$  & $x_11$  & $x_12$  & $x_13$  & $x_14$  & $x_15$  & $x_16$  & $x_17$  & $x_18$  & $x_19$  & $x_20$  & $x_21$  & $x_22$  & $x_23$  & $x_24$  & $x_25$  & $x_26$  & $x_27$  & $x_28$  & $x_29$  & $x_30$  & $x_31$  & $x_32$  \\ \hline
Sq-exp Lengthscale & $361.78$  & $1.07$  & $171.47$  & $313.51$  & $477.85$  & $1.17$  & $390.48$  & $0.84$  & $320.42$  & $267.95$  & $340.24$  & $26049.93$  & $289.53$  & $308.86$  & $335.96$  & $259.96$  & $312.91$  & $34769.47$  & $399.52$  & $532.71$  & $1.10$  & $153.52$  & $442.52$  & $291.68$  & $418.55$  & $0.57$  & $314.45$  & $236.04$  & $382.56$  & $1.06$  & $488.69$  & $281.76$  \\ 
\hline
Additive Lengthscale & $517.34$  & $1.62$  & $0.83$  & $718.01$  & $716.10$  & $718.25$  & $556.21$  & $1.58$  & $708.35$  & $665.93$  & $509.27$  & $656.33$  & $609.31$  & $0.45$  & $697.25$  & $784.49$  & $655.27$  & $782.47$  & $616.51$  & $704.36$  & $656.55$  & $693.36$  & $751.11$  & $946.12$  & $860.08$  & $803.15$  & $771.20$  & $932.21$  & $719.30$  & $887.17$  & $684.94$  & $653.89$  \\
\hline
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} & \nth{9} & \nth{10} \\
Additive Order Variance & $100.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

r_solar_500_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 01-Jun-2011 11:50:30
\begin{table}[h]
\caption{{\small
Hyperparameters for solar dataset.
}}
\label{tbl:solar}
\begin{center}
\begin{tabular}{r | r r r r r r r r r r r r r r r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  & $x_9$  & $x_10$  & $x_11$  & $x_12$  & $x_13$  & $x_14$  & $x_15$  & $x_16$  & $x_17$  & $x_18$  & $x_19$  & $x_20$  & $x_21$  & $x_22$  \\ \hline
Sq-exp lengthscale & $386.76$  & $394.02$  & $21.98$  & $0.46$  & $1.09$  & $376.04$  & $387.76$  & $380.85$  & $369.40$  & $0.10$  & $5.82$  & $20.43$  & $376.05$  & $355.20$  & $0.15$  & $0.37$  & $0.05$  & $340.06$  & $0.64$  & $2.60$  & $401.57$  & $0.74$  \\ 
\hline
Additive lengthscale & $389.82$  & $398.12$  & $22.09$  & $0.31$  & $1.01$  & $333.85$  & $386.96$  & $375.53$  & $360.26$  & $320.38$  & $375.28$  & $0.37$  & $337.76$  & $332.38$  & $7.11$  & $0.01$  & $6.56$  & $253.62$  & $0.20$  & $305.44$  & $344.27$  & $0.34$  \\
\hline
Order of interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} & \nth{9} & \nth{10} \\
Additive order variance & $0.0$\% & $0.1$\% & $0.2$\% & $0.4$\% & $0.7$\% & $1.1$\% & $1.8$\% & $2.8$\% & $5.0$\% & $87.8$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

bach_synth_c_100_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 18-May-2011 13:44:56
\begin{table}[h]
\caption{{\small
Hyperparameters for bach synth c 100 dataset
}}
\label{tbl:bach synth c 100}
\begin{center}
\begin{tabular}{l | r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  \\ \hline
ARD Lengthscale & $241.41$  & $0.20$  & $2.15$  & $179.79$  & $0.13$  & $4.99$  & $340.24$  & $317.47$  \\ 
\hline
ADD Lengthscale & $161.90$  & $0.19$  & $2.22$  & $92.58$  & $0.12$  & $42.47$  & $366.37$  & $432.12$  \\
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} \\
ADD Order Variance & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $100.0$\% & $0.0$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

add_256_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 12-May-2011 16:56:21
\begin{table}[h]
\caption{{\small
Hyperparameters for add 256 dataset
}}
\label{tbl:add 256}
\begin{center}
\begin{tabular}{l | r r r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  & $x_9$  & $x_10$  \\ \hline
ARD Lengthscale & $2.31$  & $2.40$  & $3.42$  & $4546.50$  & $3490.22$  & $4185.86$  & $7877.10$  & $2915.66$  & $1643.61$  & $1830.64$  \\ 
\hline
ADD Lengthscale & $2.30$  & $2.40$  & $3.41$  & $2520.25$  & $2741.79$  & $2566.25$  & $2276.74$  & $2135.77$  & $2406.35$  & $2213.08$  \\
ADD Variance & $1.03$ & $1.02$ & $1.07$ & $1.03$ & $1.03$ & $1.03$ & $1.03$ & $1.03$ & $1.03$ & $1.03$ \\ \hline
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} & \nth{9} & \nth{10} \\
ADD Order Variance & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $100.0$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

r_pima_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 03-Jun-2011 00:23:43
\begin{table}[h]
\caption{{\small
Hyperparameters for pima dataset.
}}
\label{tbl:pima}
\begin{center}
\begin{tabular}{r | r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  \\ \hline
Sq-exp lengthscale & $9.10$  & $3.01$  & $9.60$  & $264.78$  & $6.66$  & $5.14$  & $6.57$  & $2.36$  \\ 
\hline
Additive lengthscale & $9.31$  & $2.77$  & $8.73$  & $249.16$  & $5.93$  & $4.41$  & $5.70$  & $2.17$  \\
\hline
Order of interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} \\
Additive order variance & $0.1$\% & $0.1$\% & $0.1$\% & $0.3$\% & $1.5$\% & $96.4$\% & $1.4$\% & $0.0$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

reg_table_ll2.tex
% --- Automatically generated by resultsToLatex2.m ---
% Exported at 07-Dec-2011 14:58:35
\begin{table}[h!]
\caption{{\small
Regression Negative Log Likelihood
}}
\label{tbl:Regression Negative Log Likelihood}
\begin{center}
\begin{tabular}{l | r r r r r}
Method & \rotatebox{0}{ bach  }  & \rotatebox{0}{ concrete  }  & \rotatebox{0}{ pumadyn-8nh }  & \rotatebox{0}{ servo }  & \rotatebox{0}{ housing }  \\ \hline
Linear Regression & $2.430$ & $1.403$ & $1.881$ & $1.678$ & $1.052$ \\
GP GAM & $1.708$ & $0.467$ & $1.195$ & $0.800$ & $0.457$ \\
GP Squared-exp & $\mathbf{-0.131}$ & $0.398$ & $\mathbf{0.843}$ & $0.429$ & $\mathbf{0.207}$ \\
GP Additive & $\mathbf{-0.131}$ & $\mathbf{0.114}$ & $\mathbf{0.841}$ & $\mathbf{0.309}$ & $\mathbf{0.194}$ \\
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

reg_table_mse2.tex
% --- Automatically generated by resultsToLatex2.m ---
% Exported at 07-Dec-2011 14:58:35
\begin{table}[h!]
\caption{{\small
Regression Mean Squared Error
}}
\label{tbl:Regression Mean Squared Error}
\begin{center}
\begin{tabular}{l | r r r r r}
Method & \rotatebox{0}{ bach  }  & \rotatebox{0}{ concrete  }  & \rotatebox{0}{ pumadyn-8nh }  & \rotatebox{0}{ servo }  & \rotatebox{0}{ housing }  \\ \hline
Linear Regression & $1.031$ & $0.404$ & $0.641$ & $0.523$ & $0.289$ \\
GP GAM & $1.259$ & $0.149$ & $0.598$ & $0.281$ & $0.161$ \\
HKL & $\mathbf{0.199}$ & $0.147$ & $0.346$ & $0.199$ & $0.151$ \\
GP Squared-exp & $\mathbf{0.045}$ & $0.157$ & $\mathbf{0.317}$ & $\mathbf{0.126}$ & $\mathbf{0.092}$ \\
GP Additive & $\mathbf{0.045}$ & $\mathbf{0.089}$ & $\mathbf{0.316}$ & $\mathbf{0.110}$ & $\mathbf{0.102}$ \\
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

mse_table_ll2.tex
% --- Automatically generated by resultsToLatex2.m ---
% Exported at 20-May-2011 23:25:35
\begin{table}[h!]
\caption{{\small
Regression Negative Log Likelihood
}}
\label{tbl:Regression Negative Log Likelihood}
\begin{center}
\begin{tabular}{l | r r r r r r r}
Method & \rotatebox{0}{ sanity easy }  & \rotatebox{0}{ sanity hard }  & \rotatebox{0}{ bach 200 }  & \rotatebox{0}{ concrete 500 }  & \rotatebox{0}{ pumadyn512 }  & \rotatebox{0}{ servo }  & \rotatebox{0}{ housing }  \\ \hline
Linear Regression & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ \\
GP Sqared-exp & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ \\
GP GAM & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ \\
GP Additive & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ \\
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

mse_table2.tex
% --- Automatically generated by resultsToLatex2.m ---
% Exported at 20-May-2011 23:25:35
\begin{table}[h!]
\caption{{\small
Regression Mean Squared Error
}}
\label{tbl:Regression Mean Squared Error}
\begin{center}
\begin{tabular}{l | r r r r r r r}
Method & \rotatebox{0}{ sanity easy }  & \rotatebox{0}{ sanity hard }  & \rotatebox{0}{ bach 200 }  & \rotatebox{0}{ concrete 500 }  & \rotatebox{0}{ pumadyn512 }  & \rotatebox{0}{ servo }  & \rotatebox{0}{ housing }  \\ \hline
Linear Regression & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ \\
GP Sqared-exp & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ \\
GP GAM & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ \\
GP Additive & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ & $  NaN \pm NaN$ \\
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

class_table_ll2.tex
% --- Automatically generated by resultsToLatex2.m ---
% Exported at 03-Jun-2011 00:23:28
\begin{table}[h!]
\caption{{\small
Classification Negative Log Likelihood
}}
\label{tbl:Classification Negative Log Likelihood}
\begin{center}
\begin{tabular}{l | r r r r r r}
Method & \rotatebox{0}{ breast }  & \rotatebox{0}{ pima }  & \rotatebox{0}{ sonar }  & \rotatebox{0}{ ionosphere }  & \rotatebox{0}{ liver }  & \rotatebox{0}{ heart }  \\ \hline
Logistic Regression & $0.247$ & $0.560$ & $4.609$ & $0.878$ & $0.864$ & $0.575$ \\
GP GAM & $\mathbf{0.163}$ & $\mathbf{0.461}$ & $\mathbf{0.377}$ & $\mathbf{0.312}$ & $\mathbf{0.569}$ & $\mathbf{0.393}$ \\
GP Squared-exp & $\mathbf{0.146}$ & $0.478$ & $\mathbf{0.425}$ & $\mathbf{0.236}$ & $\mathbf{0.601}$ & $0.480$ \\
GP Additive & $\mathbf{0.150}$ & $\mathbf{0.466}$ & $\mathbf{0.409}$ & $\mathbf{0.295}$ & $\mathbf{0.588}$ & $\mathbf{0.415}$ \\
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

class_table2.tex
% --- Automatically generated by resultsToLatex2.m ---
% Exported at 03-Jun-2011 00:23:25
\begin{table}[h!]
\caption{{\small
Classification Percent Error
}}
\label{tbl:Classification Percent Error}
\begin{center}
\begin{tabular}{l | r r r r r r}
Method & \rotatebox{0}{ breast }  & \rotatebox{0}{ pima }  & \rotatebox{0}{ sonar }  & \rotatebox{0}{ ionosphere }  & \rotatebox{0}{ liver }  & \rotatebox{0}{ heart }  \\ \hline
Logistic Regression & $7.611$ & $24.392$ & $26.786$ & $16.810$ & $45.060$ & $\mathbf{16.082}$ \\
GP GAM & $\mathbf{5.189}$ & $\mathbf{22.419}$ & $\mathbf{15.786}$ & $\mathbf{8.524}$ & $\mathbf{29.842}$ & $\mathbf{16.839}$ \\
HKL & $\mathbf{5.377}$ & $24.261$ & $\mathbf{21.000}$ & $9.119$ & $\mathbf{27.270}$ & $\mathbf{18.975}$ \\
GP Squared-exp & $\mathbf{4.734}$ & $\mathbf{23.722}$ & $\mathbf{16.357}$ & $\mathbf{6.833}$ & $\mathbf{31.237}$ & $\mathbf{20.642}$ \\
GP Additive & $\mathbf{5.566}$ & $\mathbf{23.076}$ & $\mathbf{15.714}$ & $\mathbf{7.976}$ & $\mathbf{30.060}$ & $\mathbf{18.496}$ \\
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

r_housing_hypers_table_grow.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 20-May-2011 20:59:40
\begin{table}[h]
\caption{{\small
Hyperparameters for r housing grow dataset
}}
\label{tbl:r housing grow}
\begin{center}
\begin{tabular}{l | r r r r r r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  & $x_9$  & $x_10$  & $x_11$  & $x_12$  & $x_13$  \\ \hline
Sq-exp Lengthscale & $7.38$  & $134.51$  & $100.28$  & $238.89$  & $0.75$  & $2.32$  & $3.87$  & $0.44$  & $21.61$  & $0.55$  & $6.08$  & $321.24$  & $1.34$  \\ 
\hline
Additive Lengthscale & $2.02$  & $134.10$  & $51.04$  & $190.53$  & $0.01$  & $1.75$  & $51.01$  & $0.28$  & $0.89$  & $1.34$  & $12.51$  & $224.25$  & $1.39$  \\
\hline
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} & \nth{9} & \nth{10} \\
Additive Order Variance & $0.1$\% & $0.1$\% & $69.5$\% & $0.1$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $28.0$\% & $2.3$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

r_housing_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 03-Jun-2011 00:23:44
\begin{table}[h]
\caption{{\small
Hyperparameters for housing dataset.
}}
\label{tbl:housing}
\begin{center}
\begin{tabular}{r | r r r r r r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  & $x_9$  & $x_10$  & $x_11$  & $x_12$  & $x_13$  \\ \hline
Sq-exp lengthscale & $5.89$  & $233.67$  & $6.78$  & $46.83$  & $1.24$  & $3.12$  & $3.95$  & $1.09$  & $2.31$  & $0.75$  & $7.27$  & $6.60$  & $1.10$  \\ 
\hline
Additive lengthscale & $2.28$  & $261.27$  & $244.46$  & $14.08$  & $0.57$  & $2.15$  & $3.11$  & $0.45$  & $1.47$  & $0.49$  & $5.04$  & $8.97$  & $1.22$  \\
\hline
Order of interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} & \nth{9} & \nth{10} \\
Additive order variance & $0.1$\% & $0.6$\% & $80.6$\% & $1.4$\% & $1.8$\% & $0.8$\% & $0.7$\% & $0.8$\% & $0.6$\% & $12.7$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

r_servo_hypers_table_grow.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 20-May-2011 20:59:40
\begin{table}[h]
\caption{{\small
Hyperparameters for r servo grow dataset
}}
\label{tbl:r servo grow}
\begin{center}
\begin{tabular}{l | r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  \\ \hline
Sq-exp Lengthscale & $0.61$  & $0.88$  & $0.27$  & $3.33$  \\ 
\hline
Additive Lengthscale & $0.49$  & $0.69$  & $0.16$  & $2.46$  \\
\hline
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} \\
Additive Order Variance & $0.0$\% & $51.8$\% & $16.3$\% & $32.0$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

r_servo_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 03-Jun-2011 00:23:44
\begin{table}[h]
\caption{{\small
Hyperparameters for servo dataset.
}}
\label{tbl:servo}
\begin{center}
\begin{tabular}{r | r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  \\ \hline
Sq-exp lengthscale & $1.46$  & $2.70$  & $1.13$  & $6.86$  \\ 
\hline
Additive lengthscale & $0.58$  & $0.80$  & $0.07$  & $2.57$  \\
\hline
Order of interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} \\
Additive order variance & $58.7$\% & $27.4$\% & $0.0$\% & $13.9$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

r_pumadyn512_hypers_table_grow.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 20-May-2011 20:59:40
\begin{table}[h]
\caption{{\small
Hyperparameters for r pumadyn512 grow dataset
}}
\label{tbl:r pumadyn512 grow}
\begin{center}
\begin{tabular}{l | r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  \\ \hline
Sq-exp Lengthscale & $202.80$  & $1.55$  & $1.42$  & $10062.54$  & $10815.41$  & $8446.90$  & $9691.28$  & $10811.76$  \\ 
\hline
Additive Lengthscale & $411.07$  & $1.23$  & $347.27$  & $403.13$  & $422.59$  & $430.96$  & $431.13$  & $442.19$  \\
\hline
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} \\
Additive Order Variance & $0.4$\% & $0.7$\% & $1.2$\% & $3.0$\% & $0.9$\% & $2.9$\% & $14.3$\% & $76.6$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

r_pumadyn512_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 03-Jun-2011 00:23:44
\begin{table}[h]
\caption{{\small
Hyperparameters for pumadyn dataset.
}}
\label{tbl:pumadyn}
\begin{center}
\begin{tabular}{r | r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  \\ \hline
Sq-exp lengthscale & $332.71$  & $1.53$  & $1.36$  & $419.15$  & $323.71$  & $60.12$  & $164.47$  & $399.32$  \\ 
\hline
Additive lengthscale & $328.36$  & $1.54$  & $1.36$  & $409.44$  & $315.04$  & $60.31$  & $155.55$  & $392.64$  \\
\hline
Order of interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} \\
Additive order variance & $0.0$\% & $0.1$\% & $0.1$\% & $0.1$\% & $0.1$\% & $0.1$\% & $0.1$\% & $99.5$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

r_concrete_500_hypers_table_grow.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 20-May-2011 20:59:40
\begin{table}[h]
\caption{{\small
Hyperparameters for r concrete 500 grow dataset
}}
\label{tbl:r concrete 500 grow}
\begin{center}
\begin{tabular}{l | r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  \\ \hline
Sq-exp Lengthscale & $2.45$  & $3.07$  & $846.60$  & $1.12$  & $2.65$  & $27.40$  & $1.68$  & $0.24$  \\ 
\hline
Additive Lengthscale & $1.25$  & $1.62$  & $94.20$  & $0.07$  & $0.78$  & $102.76$  & $1.36$  & $0.09$  \\
\hline
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} \\
Additive Order Variance & $0.0$\% & $78.9$\% & $2.6$\% & $17.4$\% & $0.2$\% & $0.4$\% & $0.3$\% & $0.3$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

r_concrete_500_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 03-Jun-2011 00:23:44
\begin{table}[h]
\caption{{\small
Hyperparameters for concrete dataset.
}}
\label{tbl:concrete}
\begin{center}
\begin{tabular}{r | r r r r r r r r}
Variable: & cement  & slag  & ash  & water  & plast.  & coarse  & fine  & age  \\ \hline
Sq-exp lengthscale & $3.53$  & $3.90$  & $7.13$  & $1.17$  & $3.94$  & $13.61$  & $2.82$  & $0.54$  \\ 
\hline
Additive lengthscale & $1.42$  & $1.57$  & $3.00$  & $0.08$  & $0.52$  & $298.00$  & $1.46$  & $0.11$  \\
\hline
Order of interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} \\
Additive order variance & $70.6$\% & $13.3$\% & $13.8$\% & $2.3$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

r_concrete_100_hypers_table_grow.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 20-May-2011 20:59:40
\begin{table}[h]
\caption{{\small
Hyperparameters for r concrete 100 grow dataset
}}
\label{tbl:r concrete 100 grow}
\begin{center}
\begin{tabular}{l | r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  \\ \hline
Sq-exp Lengthscale & $0.76$  & $428.07$  & $2.91$  & $1.72$  & $1.62$  & $3.48$  & $3.13$  & $0.00$  \\ 
\hline
Additive Lengthscale & $179.04$  & $196.26$  & $171.19$  & $1.33$  & $51.75$  & $187.33$  & $139.30$  & $0.00$  \\
\hline
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} \\
Additive Order Variance & $0.0$\% & $0.0$\% & $0.1$\% & $1.1$\% & $0.3$\% & $8.9$\% & $5.6$\% & $84.1$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

r_concrete_100_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 20-May-2011 20:59:40
\begin{table}[h]
\caption{{\small
Hyperparameters for r concrete 100 dataset
}}
\label{tbl:r concrete 100}
\begin{center}
\begin{tabular}{l | r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  \\ \hline
Sq-exp Lengthscale & $0.76$  & $428.07$  & $2.91$  & $1.72$  & $1.62$  & $3.48$  & $3.13$  & $0.00$  \\ 
\hline
Additive Lengthscale & $0.76$  & $246.87$  & $2.90$  & $1.72$  & $1.61$  & $3.48$  & $3.13$  & $0.00$  \\
\hline
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} \\
Additive Order Variance & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $100.0$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

r_liver_hypers_table_grow.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 20-May-2011 20:59:40
\begin{table}[h]
\caption{{\small
Hyperparameters for r liver grow dataset
}}
\label{tbl:r liver grow}
\begin{center}
\begin{tabular}{l | r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  \\ \hline
Sq-exp Lengthscale & $0.84$  & $2.74$  & $0.85$  & $143.08$  & $0.49$  & $2.56$  \\ 
\hline
Additive Lengthscale & $0.78$  & $558.24$  & $0.71$  & $169.62$  & $0.44$  & $415.27$  \\
\hline
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} \\
Additive Order Variance & $0.0$\% & $0.0$\% & $0.0$\% & $1.3$\% & $98.2$\% & $0.5$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

r_liver_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 03-Jun-2011 00:23:43
\begin{table}[h]
\caption{{\small
Hyperparameters for liver dataset.
}}
\label{tbl:liver}
\begin{center}
\begin{tabular}{r | r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  \\ \hline
Sq-exp lengthscale & $1.53$  & $6.42$  & $1.46$  & $3.39$  & $1.31$  & $6.06$  \\ 
\hline
Additive lengthscale & $0.75$  & $4.17$  & $0.74$  & $1.96$  & $0.55$  & $4.86$  \\
\hline
Order of interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} \\
Additive order variance & $0.0$\% & $0.2$\% & $99.7$\% & $0.1$\% & $0.0$\% & $0.0$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

r_ionosphere_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 03-Jun-2011 00:23:43
\begin{table}[h]
\caption{{\small
Hyperparameters for ionosphere dataset.
}}
\label{tbl:ionosphere}
\begin{center}
\begin{tabular}{r | r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  & $x_9$  & $x_10$  & $x_11$  & $x_12$  & $x_13$  & $x_14$  & $x_15$  & $x_16$  & $x_17$  & $x_18$  & $x_19$  & $x_20$  & $x_21$  & $x_22$  & $x_23$  & $x_24$  & $x_25$  & $x_26$  & $x_27$  & $x_28$  & $x_29$  & $x_30$  & $x_31$  & $x_32$  \\ \hline
Sq-exp lengthscale & $4.45$  & $177.21$  & $1.65$  & $111.51$  & $3.38$  & $4.51$  & $2.93$  & $4.25$  & $3.30$  & $195.17$  & $122.22$  & $3.18$  & $51.85$  & $275.31$  & $119.34$  & $3.67$  & $153.15$  & $99.95$  & $137.44$  & $6.59$  & $142.85$  & $2.40$  & $174.16$  & $7.65$  & $255.85$  & $132.60$  & $31.18$  & $180.54$  & $140.89$  & $148.72$  & $182.35$  & $238.70$  \\ 
\hline
Additive lengthscale & $2.94$  & $206.17$  & $1.03$  & $80.47$  & $2.71$  & $2.30$  & $0.49$  & $2.34$  & $2.29$  & $164.16$  & $117.20$  & $143.31$  & $5.02$  & $169.80$  & $130.45$  & $124.14$  & $128.44$  & $168.23$  & $50.49$  & $4.26$  & $122.52$  & $0.67$  & $136.19$  & $3.05$  & $108.61$  & $131.04$  & $138.26$  & $170.03$  & $140.90$  & $2.33$  & $130.20$  & $166.60$  \\
\hline
Order of interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} & \nth{9} & \nth{10} \\
Additive order variance & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $99.7$\% & $0.1$\% & $0.2$\% & $0.0$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

r_sonar_hypers_table_grow.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 20-May-2011 20:59:40
\begin{table}[h]
\caption{{\small
Hyperparameters for r sonar grow dataset
}}
\label{tbl:r sonar grow}
\begin{center}
\begin{tabular}{l | r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  & $x_9$  & $x_10$  & $x_11$  & $x_12$  & $x_13$  & $x_14$  & $x_15$  & $x_16$  & $x_17$  & $x_18$  & $x_19$  & $x_20$  & $x_21$  & $x_22$  & $x_23$  & $x_24$  & $x_25$  & $x_26$  & $x_27$  & $x_28$  & $x_29$  & $x_30$  & $x_31$  & $x_32$  & $x_33$  & $x_34$  & $x_35$  & $x_36$  & $x_37$  & $x_38$  & $x_39$  & $x_40$  & $x_41$  & $x_42$  & $x_43$  & $x_44$  & $x_45$  & $x_46$  & $x_47$  & $x_48$  & $x_49$  & $x_50$  & $x_51$  & $x_52$  & $x_53$  & $x_54$  & $x_55$  & $x_56$  & $x_57$  & $x_58$  & $x_59$  & $x_60$  \\ \hline
Sq-exp Lengthscale & $2.42$  & $0.76$  & $3.71$  & $4.19$  & $1.37$  & $7.47$  & $7.48$  & $3.33$  & $4.29$  & $3.85$  & $2.90$  & $5.59$  & $2.08$  & $3.04$  & $3.65$  & $6.92$  & $3.49$  & $3.09$  & $1.64$  & $4.15$  & $0.93$  & $5.54$  & $9.54$  & $1.90$  & $6.09$  & $7.73$  & $0.72$  & $0.92$  & $5.04$  & $2.43$  & $5.46$  & $5.95$  & $5.55$  & $7.91$  & $5.39$  & $7.46$  & $1.12$  & $3.29$  & $2.97$  & $0.77$  & $4.04$  & $1.29$  & $8.50$  & $1.70$  & $4.92$  & $3.95$  & $1.50$  & $0.43$  & $3.22$  & $1.43$  & $5.22$  & $4.86$  & $9.91$  & $5.15$  & $1.96$  & $4.48$  & $1.51$  & $3.30$  & $3.30$  & $3.49$  \\ 
\hline
Additive Lengthscale & $2.34$  & $0.62$  & $3.67$  & $4.13$  & $1.16$  & $7.45$  & $7.44$  & $3.23$  & $4.25$  & $3.80$  & $2.87$  & $5.58$  & $2.03$  & $2.99$  & $3.63$  & $6.91$  & $3.49$  & $3.09$  & $1.61$  & $4.15$  & $0.94$  & $5.54$  & $9.53$  & $1.85$  & $6.08$  & $7.73$  & $0.68$  & $0.82$  & $5.04$  & $2.40$  & $5.45$  & $5.91$  & $5.53$  & $7.90$  & $5.37$  & $7.45$  & $1.09$  & $3.28$  & $2.94$  & $0.67$  & $4.04$  & $1.28$  & $8.48$  & $1.66$  & $4.89$  & $3.93$  & $1.47$  & $0.28$  & $3.18$  & $1.35$  & $5.18$  & $4.82$  & $9.89$  & $5.10$  & $1.93$  & $4.41$  & $1.35$  & $3.28$  & $3.21$  & $3.33$  \\
\hline
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} & \nth{9} & \nth{10} \\
Additive Order Variance & $53.1$\% & $46.6$\% & $0.0$\% & $0.3$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

r_sonar_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 03-Jun-2011 00:23:43
\begin{table}[h]
\caption{{\small
Hyperparameters for sonar dataset.
}}
\label{tbl:sonar}
\begin{center}
\begin{tabular}{r | r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  & $x_9$  & $x_10$  & $x_11$  & $x_12$  & $x_13$  & $x_14$  & $x_15$  & $x_16$  & $x_17$  & $x_18$  & $x_19$  & $x_20$  & $x_21$  & $x_22$  & $x_23$  & $x_24$  & $x_25$  & $x_26$  & $x_27$  & $x_28$  & $x_29$  & $x_30$  & $x_31$  & $x_32$  & $x_33$  & $x_34$  & $x_35$  & $x_36$  & $x_37$  & $x_38$  & $x_39$  & $x_40$  & $x_41$  & $x_42$  & $x_43$  & $x_44$  & $x_45$  & $x_46$  & $x_47$  & $x_48$  & $x_49$  & $x_50$  & $x_51$  & $x_52$  & $x_53$  & $x_54$  & $x_55$  & $x_56$  & $x_57$  & $x_58$  & $x_59$  & $x_60$  \\ \hline
Sq-exp lengthscale & $9.20$  & $169.05$  & $241.70$  & $226.93$  & $277.59$  & $193.88$  & $200.15$  & $96.82$  & $2.43$  & $168.65$  & $169.94$  & $9.48$  & $208.19$  & $243.81$  & $214.63$  & $180.36$  & $1.96$  & $154.73$  & $166.96$  & $4.76$  & $195.59$  & $14.25$  & $8.24$  & $209.55$  & $246.85$  & $190.62$  & $2.79$  & $200.86$  & $259.59$  & $251.15$  & $234.58$  & $246.09$  & $276.05$  & $250.38$  & $204.41$  & $2.61$  & $177.80$  & $208.40$  & $229.45$  & $246.73$  & $182.31$  & $194.81$  & $7.27$  & $175.63$  & $159.60$  & $8.15$  & $140.43$  & $5.63$  & $150.87$  & $209.74$  & $11.20$  & $140.54$  & $207.62$  & $236.61$  & $197.56$  & $249.87$  & $220.91$  & $15.08$  & $21.47$  & $175.19$  \\ 
\hline
Additive lengthscale & $19.81$  & $82.86$  & $142.43$  & $9.89$  & $141.69$  & $152.84$  & $6.80$  & $10.91$  & $124.49$  & $124.27$  & $1.75$  & $21.53$  & $124.12$  & $141.85$  & $137.64$  & $134.36$  & $139.19$  & $141.07$  & $113.38$  & $3.83$  & $78.53$  & $92.81$  & $0.66$  & $120.21$  & $125.13$  & $91.17$  & $1.65$  & $1.53$  & $116.73$  & $157.89$  & $121.73$  & $148.87$  & $159.05$  & $142.36$  & $108.97$  & $0.48$  & $6.40$  & $134.32$  & $138.80$  & $130.88$  & $105.76$  & $1.64$  & $80.56$  & $79.41$  & $1.51$  & $86.84$  & $116.51$  & $123.33$  & $2.65$  & $131.16$  & $7.83$  & $3.65$  & $135.42$  & $3.05$  & $1.69$  & $129.34$  & $132.29$  & $9.52$  & $6.01$  & $111.40$  \\
\hline
Order of interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} & \nth{9} & \nth{10} \\
Additive order variance & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $100.0$\% & $0.0$\% & $0.0$\% & $0.0$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

r_breast_hypers_table_grow.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 20-May-2011 20:59:40
\begin{table}[h]
\caption{{\small
Hyperparameters for r breast grow dataset
}}
\label{tbl:r breast grow}
\begin{center}
\begin{tabular}{l | r r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  & $x_9$  \\ \hline
Sq-exp Lengthscale & $1.50$  & $0.20$  & $3.33$  & $3.24$  & $1.25$  & $7.03$  & $7.39$  & $2.43$  & $4.22$  \\ 
\hline
Additive Lengthscale & $0.68$  & $0.15$  & $3.39$  & $3.28$  & $1.38$  & $6.79$  & $7.54$  & $1.16$  & $4.30$  \\
\hline
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} & \nth{9} \\
Additive Order Variance & $1.0$\% & $1.0$\% & $1.2$\% & $1.0$\% & $3.1$\% & $2.8$\% & $4.2$\% & $14.8$\% & $70.8$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

r_breast_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 03-Jun-2011 00:23:43
\begin{table}[h]
\caption{{\small
Hyperparameters for breast dataset.
}}
\label{tbl:breast}
\begin{center}
\begin{tabular}{r | r r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  & $x_9$  \\ \hline
Sq-exp lengthscale & $4.17$  & $187.14$  & $23.11$  & $5.15$  & $14.09$  & $4.44$  & $6.11$  & $10.56$  & $9.24$  \\ 
\hline
Additive lengthscale & $4.18$  & $189.58$  & $23.08$  & $5.15$  & $14.11$  & $4.44$  & $6.12$  & $10.57$  & $9.24$  \\
\hline
Order of interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} & \nth{9} \\
Additive order variance & $0.0$\% & $0.0$\% & $0.1$\% & $0.1$\% & $0.1$\% & $0.1$\% & $0.1$\% & $0.1$\% & $99.3$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

r_sanity_hard_hypers_table_grow.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 18-May-2011 13:44:57
\begin{table}[h]
\caption{{\small
Hyperparameters for r sanity hard dataset
}}
\label{tbl:r sanity hard}
\begin{center}
\begin{tabular}{l | r r}
Variable: & $x_1$  & $x_2$  \\ \hline
ARD Lengthscale & $1538.91$  & $0.21$  \\ 
\hline
ADD Lengthscale & $119.83$  & $0.21$  \\
Order of Interaction: & \nth{1} & \nth{2} \\
ADD Order Variance & $0.0$\% & $100.0$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

r_sanity_hard_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 03-Jun-2011 00:23:44
\begin{table}[h]
\caption{{\small
Hyperparameters for sanity hard dataset.
}}
\label{tbl:sanity hard}
\begin{center}
\begin{tabular}{r | r r}
Variable: & $x_1$  & $x_2$  \\ \hline
Sq-exp lengthscale & $1.43$  & $0.19$  \\ 
\hline
Additive lengthscale & $1.43$  & $0.19$  \\
\hline
Order of interaction: & \nth{1} & \nth{2} \\
Additive order variance & $0.1$\% & $99.9$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

r_sanity_easy_hypers_table_grow.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 18-May-2011 13:44:56
\begin{table}[h]
\caption{{\small
Hyperparameters for r sanity easy dataset
}}
\label{tbl:r sanity easy}
\begin{center}
\begin{tabular}{l | r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  \\ \hline
ARD Lengthscale & $1.20$  & $0.18$  & $4.43$  \\ 
\hline
ADD Lengthscale & $3.65$  & $0.23$  & $6.09$  \\
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} \\
ADD Order Variance & $6.6$\% & $39.5$\% & $53.9$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

r_sanity_easy_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 03-Jun-2011 00:23:44
\begin{table}[h]
\caption{{\small
Hyperparameters for sanity easy dataset.
}}
\label{tbl:sanity easy}
\begin{center}
\begin{tabular}{r | r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  \\ \hline
Sq-exp lengthscale & $26253.73$  & $25456.47$  & $0.47$  \\ 
\hline
Additive lengthscale & $26254.84$  & $25470.23$  & $0.47$  \\
\hline
Order of interaction: & \nth{1} & \nth{2} & \nth{3} \\
Additive order variance & $0.0$\% & $0.1$\% & $99.9$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

ionosphere_hypers_table_grow.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 12-May-2011 16:56:20
\begin{table}[h]
\caption{{\small
Hyperparameters for ionosphere dataset
}}
\label{tbl:ionosphere}
\begin{center}
\begin{tabular}{l | r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  & $x_9$  & $x_10$  & $x_11$  & $x_12$  & $x_13$  & $x_14$  & $x_15$  & $x_16$  & $x_17$  & $x_18$  & $x_19$  & $x_20$  & $x_21$  & $x_22$  & $x_23$  & $x_24$  & $x_25$  & $x_26$  & $x_27$  & $x_28$  & $x_29$  & $x_30$  & $x_31$  & $x_32$  \\ \hline
ARD Lengthscale & $418.00$  & $61384.65$  & $1.12$  & $600.24$  & $2023.45$  & $41752.37$  & $764.96$  & $605.18$  & $588.51$  & $3962.02$  & $580.67$  & $3226.23$  & $1811.45$  & $1136.30$  & $1599.51$  & $2307.05$  & $586.22$  & $2440.60$  & $1522.15$  & $804.51$  & $506.52$  & $0.93$  & $1953.48$  & $1386.07$  & $2174.96$  & $0.68$  & $7030.63$  & $9761.00$  & $2224.68$  & $79535.55$  & $4149.84$  & $306363.78$  \\ 
\hline
ADD Lengthscale & $0.32$  & $0.01$  & $0.99$  & $0.93$  & $0.13$  & $1.97$  & $2.19$  & $0.97$  & $0.97$  & $1.37$  & $0.53$  & $2.71$  & $0.35$  & $0.54$  & $0.93$  & $2.00$  & $2.03$  & $0.26$  & $0.14$  & $4.70$  & $0.06$  & $1.14$  & $3.08$  & $0.28$  & $1.76$  & $2.46$  & $0.03$  & $2.89$  & $0.39$  & $2.58$  & $3.62$  & $4.26$  \\
ADD Variance & $0.00$ & $0.00$ & $6077061.97$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $704441.37$ & $0.00$ & $0.01$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.01$ \\ \hline
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} & \nth{9} & \nth{10} \\
ADD Order Variance & $NaN$\% & $NaN$\% & $NaN$\% & $NaN$\% & $NaN$\% & $NaN$\% & $NaN$\% & $NaN$\% & $NaN$\% & $NaN$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

pumadyn512_hypers_table_grow.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 12-May-2011 16:56:21
\begin{table}[h]
\caption{{\small
Hyperparameters for pumadyn512 dataset
}}
\label{tbl:pumadyn512}
\begin{center}
\begin{tabular}{l | r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  \\ \hline
ARD Lengthscale & $137.74$  & $1.55$  & $1.42$  & $2543.62$  & $2727.28$  & $2113.23$  & $2493.23$  & $2745.56$  \\ 
\hline
ADD Lengthscale & $1.71$  & $14.85$  & $13.43$  & $52.36$  & $0.32$  & $54.70$  & $60.82$  & $86.14$  \\
ADD Variance & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ \\ \hline
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} \\
ADD Order Variance & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $100.0$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

abalone_500_hypers_table_grow.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 12-May-2011 16:56:21
\begin{table}[h]
\caption{{\small
Hyperparameters for abalone 500 dataset
}}
\label{tbl:abalone 500}
\begin{center}
\begin{tabular}{l | r r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  & $x_9$  \\ \hline
ARD Lengthscale & $0.37$  & $0.02$  & $0.21$  & $0.06$  & $0.03$  & $5.55$  & $0.72$  & $1.49$  & $0.17$  \\ 
\hline
ADD Lengthscale & $0.37$  & $0.02$  & $37.12$  & $37.40$  & $117.41$  & $30.81$  & $50.42$  & $34.28$  & $160.40$  \\
ADD Variance & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ \\ \hline
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} \\
ADD Order Variance & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $100.0$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

add_256_hypers_table_grow.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 12-May-2011 16:56:21
\begin{table}[h]
\caption{{\small
Hyperparameters for add 256 dataset
}}
\label{tbl:add 256}
\begin{center}
\begin{tabular}{l | r r r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  & $x_9$  & $x_10$  \\ \hline
ARD Lengthscale & $2.31$  & $2.40$  & $3.42$  & $4546.50$  & $3490.22$  & $4185.86$  & $7877.10$  & $2915.66$  & $1643.61$  & $1830.64$  \\ 
\hline
ADD Lengthscale & $1.26$  & $2.62$  & $2.62$  & $0.50$  & $0.09$  & $2.90$  & $2.39$  & $0.54$  & $0.93$  & $0.72$  \\
ADD Variance & $7889.94$ & $102156.41$ & $157768.07$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ \\ \hline
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} \\
ADD Order Variance & $100.0$\% & $0.0$\% & $0.0$\% & $0.0$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

bach_synth_r_200_hypers_table_grow.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 12-May-2011 16:56:21
\begin{table}[h]
\caption{{\small
Hyperparameters for bach synth r 200 dataset
}}
\label{tbl:bach synth r 200}
\begin{center}
\begin{tabular}{l | r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  \\ \hline
ARD Lengthscale & $8.45$  & $9.82$  & $9.78$  & $9.90$  & $22177.79$  & $21217.89$  & $21805.70$  & $22842.27$  \\ 
\hline
ADD Lengthscale & $8.75$  & $9.89$  & $10.08$  & $10.10$  & $0.03$  & $110.97$  & $190.65$  & $0.21$  \\
ADD Variance & $2.10$ & $0.04$ & $3.09$ & $2.43$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ \\ \hline
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} \\
ADD Order Variance & $0.0$\% & $0.6$\% & $0.0$\% & $99.3$\% & $0.0$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

pima_hypers_table_grow.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 12-May-2011 16:56:20
\begin{table}[h]
\caption{{\small
Hyperparameters for pima dataset
}}
\label{tbl:pima}
\begin{center}
\begin{tabular}{l | r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  \\ \hline
ARD Lengthscale & $0.30$  & $0.05$  & $4.15$  & $3.62$  & $0.26$  & $1.78$  & $326.52$  & $0.76$  \\ 
\hline
ADD Lengthscale & $0.06$  & $0.46$  & $18.24$  & $2.06$  & $0.08$  & $0.57$  & $2.03$  & $0.54$  \\
ADD Variance & $0.00$ & $0.00$ & $0.00$ & $0.02$ & $0.00$ & $0.00$ & $0.00$ & $0.08$ \\ \hline
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} \\
ADD Order Variance & $0.0$\% & $0.0$\% & $0.0$\% & $100.0$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

breast_hypers_table_grow.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 12-May-2011 16:56:20
\begin{table}[h]
\caption{{\small
Hyperparameters for breast dataset
}}
\label{tbl:breast}
\begin{center}
\begin{tabular}{l | r r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  & $x_9$  \\ \hline
ARD Lengthscale & $12597.35$  & $0.02$  & $1948.32$  & $0.00$  & $15320796065.96$  & $0.83$  & $446.51$  & $24312.96$  & $1420.93$  \\ 
\hline
ADD Lengthscale & $2.73$  & $0.02$  & $0.66$  & $0.02$  & $0.10$  & $0.83$  & $1.85$  & $126.56$  & $0.93$  \\
ADD Variance & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ \\ \hline
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} \\
ADD Order Variance & $0.0$\% & $0.0$\% & $0.0$\% & $100.0$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

housing_hypers_table_grow.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 12-May-2011 16:56:21
\begin{table}[h]
\caption{{\small
Hyperparameters for housing dataset
}}
\label{tbl:housing}
\begin{center}
\begin{tabular}{l | r r r r r r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  & $x_9$  & $x_10$  & $x_11$  & $x_12$  & $x_13$  \\ \hline
ARD Lengthscale & $1508.26$  & $504.64$  & $296.62$  & $3.91$  & $0.08$  & $2.21$  & $5.77$  & $2.77$  & $424.77$  & $0.74$  & $233.78$  & $605.23$  & $3.75$  \\ 
\hline
ADD Lengthscale & $7.27$  & $0.02$  & $7.23$  & $0.93$  & $0.12$  & $9518.28$  & $197.06$  & $7131.57$  & $3.19$  & $0.95$  & $13.03$  & $57.66$  & $45269.51$  \\
ADD Variance & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ \\ \hline
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} \\
ADD Order Variance & $100.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $-0.0$\% & $-0.0$\% & $0.0$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

servo_hypers_table_grow.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 12-May-2011 16:56:21
\begin{table}[h]
\caption{{\small
Hyperparameters for servo dataset
}}
\label{tbl:servo}
\begin{center}
\begin{tabular}{l | r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  \\ \hline
ARD Lengthscale & $0.49$  & $0.02$  & $0.12$  & $1.47$  \\ 
\hline
ADD Lengthscale & $0.38$  & $0.02$  & $0.13$  & $0.11$  \\
ADD Variance & $0.00$ & $0.00$ & $0.17$ & $0.00$ \\ \hline
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} \\
ADD Order Variance & $21.0$\% & $0.0$\% & $0.0$\% & $78.9$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

bach_synth_r_100_hypers_table_grow.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 20-May-2011 20:59:40
\begin{table}[h]
\caption{{\small
Hyperparameters for bach synth r 100 grow dataset
}}
\label{tbl:bach synth r 100 grow}
\begin{center}
\begin{tabular}{l | r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  \\ \hline
Sq-exp Lengthscale & $5.40$  & $5.83$  & $5.27$  & $4.89$  & $8620.49$  & $8278.75$  & $7444.24$  & $7866.09$  \\ 
\hline
Additive Lengthscale & $7.00$  & $6.86$  & $6.51$  & $6.18$  & $5558.41$  & $5431.29$  & $5412.15$  & $5594.23$  \\
\hline
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} \\
Additive Order Variance & $0.0$\% & $0.0$\% & $0.0$\% & $18.8$\% & $0.0$\% & $0.0$\% & $0.0$\% & $81.2$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

sonar_hypers_table_grow.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 12-May-2011 16:56:20
\begin{table}[h]
\caption{{\small
Hyperparameters for sonar dataset
}}
\label{tbl:sonar}
\begin{center}
\begin{tabular}{l | r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  & $x_9$  & $x_10$  & $x_11$  & $x_12$  & $x_13$  & $x_14$  & $x_15$  & $x_16$  & $x_17$  & $x_18$  & $x_19$  & $x_20$  & $x_21$  & $x_22$  & $x_23$  & $x_24$  & $x_25$  & $x_26$  & $x_27$  & $x_28$  & $x_29$  & $x_30$  & $x_31$  & $x_32$  & $x_33$  & $x_34$  & $x_35$  & $x_36$  & $x_37$  & $x_38$  & $x_39$  & $x_40$  & $x_41$  & $x_42$  & $x_43$  & $x_44$  & $x_45$  & $x_46$  & $x_47$  & $x_48$  & $x_49$  & $x_50$  & $x_51$  & $x_52$  & $x_53$  & $x_54$  & $x_55$  & $x_56$  & $x_57$  & $x_58$  & $x_59$  & $x_60$  \\ \hline
ARD Lengthscale & $0.37$  & $0.02$  & $0.77$  & $0.93$  & $0.10$  & $2.19$  & $2.19$  & $0.64$  & $0.97$  & $0.82$  & $0.53$  & $1.45$  & $0.29$  & $0.56$  & $0.76$  & $1.97$  & $0.72$  & $0.59$  & $0.19$  & $0.94$  & $0.06$  & $1.44$  & $3.07$  & $0.25$  & $1.64$  & $2.30$  & $0.05$  & $1.25$  & $0.39$  & $1.40$  & $1.58$  & $1.43$  & $2.37$  & $1.38$  & $2.19$  & $0.09$  & $0.65$  & $0.55$  & $0.03$  & $0.90$  & $0.12$  & $2.62$  & $0.20$  & $1.20$  & $0.86$  & $0.16$  & $0.62$  & $0.14$  & $1.30$  & $1.17$  & $3.22$  & $1.28$  & $0.27$  & $1.03$  & $0.14$  & $0.65$  & $0.63$  & $0.67$  & $0.44$  & $2.02$  \\ 
\hline
ADD Lengthscale & $0.41$  & $0.03$  & $11.30$  & $0.57$  & $0.11$  & $30.71$  & $7.71$  & $7.64$  & $24.84$  & $24.02$  & $4.55$  & $0.26$  & $4.09$  & $6.93$  & $13.27$  & $2.46$  & $64.70$  & $7.37$  & $0.85$  & $23.99$  & $0.11$  & $20.06$  & $1.33$  & $0.63$  & $11.03$  & $9.04$  & $0.12$  & $5.32$  & $2.89$  & $1.35$  & $14.78$  & $16.12$  & $11.51$  & $13.21$  & $22.44$  & $0.52$  & $4.23$  & $1.59$  & $0.03$  & $9.74$  & $0.23$  & $1.40$  & $0.97$  & $1.99$  & $2.24$  & $2.43$  & $7.83$  & $0.26$  & $2.75$  & $16.66$  & $19.29$  & $78.15$  & $0.78$  & $6.20$  & $0.37$  & $8.15$  & $6.69$  & $5.06$  & $1.43$  & $18.50$  \\
ADD Variance & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.01$ & $0.00$ & $0.00$ & $0.00$ & $7.60$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $7.22$ & $0.00$ & $0.00$ & $0.00$ & $9.19$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $8.22$ & $0.00$ & $8.57$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.01$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.01$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.02$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ & $0.00$ \\ \hline
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} & \nth{9} & \nth{10} \\
ADD Order Variance & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $100.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

bach_synth_c_200_hypers_table_grow.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 12-May-2011 16:56:20
\begin{table}[h]
\caption{{\small
Hyperparameters for bach synth c 200 dataset
}}
\label{tbl:bach synth c 200}
\begin{center}
\begin{tabular}{l | r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  \\ \hline
ARD Lengthscale & $598.53$  & $0.17$  & $3.41$  & $2.84$  & $0.19$  & $0.88$  & $208.99$  & $350.09$  \\ 
\hline
ADD Lengthscale & $0.25$  & $0.00$  & $0.45$  & $5.17$  & $0.72$  & $0.73$  & $0.48$  & $0.11$  \\
ADD Variance & $0.00$ & $0.00$ & $0.00$ & $0.12$ & $0.12$ & $0.17$ & $0.00$ & $0.00$ \\ \hline
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} \\
ADD Order Variance & $0.0$\% & $0.0$\% & $0.0$\% & $100.0$\% & $0.0$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

bach_synth_c_100_hypers_table_grow.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 18-May-2011 13:44:56
\begin{table}[h]
\caption{{\small
Hyperparameters for bach synth c 100 dataset
}}
\label{tbl:bach synth c 100}
\begin{center}
\begin{tabular}{l | r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  \\ \hline
ARD Lengthscale & $241.41$  & $0.20$  & $2.15$  & $179.79$  & $0.13$  & $4.99$  & $340.24$  & $317.47$  \\ 
\hline
ADD Lengthscale & $117.30$  & $0.21$  & $2.08$  & $1.64$  & $0.11$  & $95.55$  & $255.84$  & $329.96$  \\
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} \\
ADD Order Variance & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $99.9$\% & $0.0$\% & $0.1$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

sanity_hard_hypers_table_grow.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 12-May-2011 16:56:21
\begin{table}[h]
\caption{{\small
Hyperparameters for sanity hard dataset
}}
\label{tbl:sanity hard}
\begin{center}
\begin{tabular}{l | r r}
Variable: & $x_1$  & $x_2$  \\ \hline
ARD Lengthscale & $0.47$  & $0.06$  \\ 
\hline
ADD Lengthscale & $0.12$  & $0.02$  \\
ADD Variance & $0.00$ & $0.04$ \\ \hline
Order of Interaction: & \nth{1} & \nth{2} \\
ADD Order Variance & $100.0$\% & $0.0$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

sanity_easy_hypers_table_grow.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 12-May-2011 16:56:21
\begin{table}[h]
\caption{{\small
Hyperparameters for sanity easy dataset
}}
\label{tbl:sanity easy}
\begin{center}
\begin{tabular}{l | r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  \\ \hline
ARD Lengthscale & $2514.48$  & $12980.31$  & $0.35$  \\ 
\hline
ADD Lengthscale & $17.14$  & $0.05$  & $0.47$  \\
ADD Variance & $0.00$ & $0.00$ & $0.00$ \\ \hline
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} \\
ADD Order Variance & $0.1$\% & $0.0$\% & $99.9$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

sanity_hard_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 12-May-2011 16:56:21
\begin{table}[h]
\caption{{\small
Hyperparameters for sanity hard dataset
}}
\label{tbl:sanity hard}
\begin{center}
\begin{tabular}{l | r r}
Variable: & $x_1$  & $x_2$  \\ \hline
ARD Lengthscale & $0.47$  & $0.06$  \\ 
\hline
ADD Lengthscale & $0.47$  & $0.06$  \\
ADD Variance & $0.11$ & $1.06$ \\ \hline
Order of Interaction: & \nth{1} & \nth{2} \\
ADD Order Variance & $0.0$\% & $100.0$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

sanity_easy_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 12-May-2011 16:56:21
\begin{table}[h]
\caption{{\small
Hyperparameters for sanity easy dataset
}}
\label{tbl:sanity easy}
\begin{center}
\begin{tabular}{l | r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  \\ \hline
ARD Lengthscale & $2514.48$  & $12980.31$  & $0.35$  \\ 
\hline
ADD Lengthscale & $871.23$  & $700.46$  & $0.45$  \\
ADD Variance & $0.87$ & $0.87$ & $1.08$ \\ \hline
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} \\
ADD Order Variance & $2.9$\% & $0.0$\% & $97.1$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

rmse_table_all.tex
% --- Automatically generated by resultsToLatex3.m ---
% Exported at 04-May-2011 12:57:02
\begin{table}[h!]
\caption{{\small
Negative Log Likelihood
}}
\label{tbl:Negative Log Likelihood}
\begin{center}
\begin{tabular}{l | r r r r r r r r r r r r r r r r}
Method & \multicolumn{2}{c|}{ \rotatebox{0}{ lin model } } & \multicolumn{2}{c|}{ \rotatebox{0}{ gp ard } } & \multicolumn{2}{c|}{ \rotatebox{0}{ gp ard laplace } } & \multicolumn{2}{c|}{ \rotatebox{0}{ gp add 4 } } & \multicolumn{2}{c|}{ \rotatebox{0}{ gp add } } & \multicolumn{2}{c|}{ \rotatebox{0}{ gp add grow } } & \multicolumn{2}{c|}{ \rotatebox{0}{ gp add laplace 4 } } & \multicolumn{2}{c|}{ \rotatebox{0}{ gp add laplace } } & \multicolumn{2}{c|}{ \rotatebox{0}{ gp add laplace grow } }
class_table_all.tex
% --- Automatically generated by resultsToLatex3.m ---
% Exported at 04-May-2011 12:56:53
\begin{table}[h!]
\caption{{\small
Negative Log Likelihood
}}
\label{tbl:Negative Log Likelihood}
\begin{center}
\begin{tabular}{l | r r r r r r r r r r r r r r r r}
Method & \multicolumn{2}{c|}{ \rotatebox{0}{ lin model } } & \multicolumn{2}{c|}{ \rotatebox{0}{ gp ard class } } & \multicolumn{2}{c|}{ \rotatebox{0}{ gp add class 4 } } & \multicolumn{2}{c|}{ \rotatebox{0}{ gp add class } } & \multicolumn{2}{c|}{ \rotatebox{0}{ gp add class grow } } \\ \hline
 & \multicolumn{1}{c|}{ \rotatebox{0}{ Err\% } } & \multicolumn{1}{c|}{ \rotatebox{0}{ NLL } } & \multicolumn{1}{c|}{ \rotatebox{0}{ Err\% } } & \multicolumn{1}{c|}{ \rotatebox{0}{ NLL } } & \multicolumn{1}{c|}{ \rotatebox{0}{ Err\% } } & \multicolumn{1}{c|}{ \rotatebox{0}{ NLL } } & \multicolumn{1}{c|}{ \rotatebox{0}{ Err\% } } & \multicolumn{1}{c|}{ \rotatebox{0}{ NLL } } & \multicolumn{1}{c|}{ \rotatebox{0}{ Err\% } } & \multicolumn{1}{c|}{ \rotatebox{0}{ NLL } } & \multicolumn{1}{c|}{ \rotatebox{0}{ Err\% } } & \multicolumn{1}{c|}{ \rotatebox{0}{ NLL } } & \multicolumn{1}{c|}{ \rotatebox{0}{ Err\% } } & \multicolumn{1}{c|}{ \rotatebox{0}{ NLL } } & \multicolumn{1}{c|}{ \rotatebox{0}{ Err\% } } & \multicolumn{1}{c|}{ \rotatebox{0}{ NLL } } \\ \hline
sanity easy & \multicolumn{1}{l|}{ $\mathbf{0.00}$ } & \multicolumn{1}{l|}{ $\mathbf{-0.79}$ } & \multicolumn{1}{l|}{0.00} & \multicolumn{1}{l|}{0.00} & \multicolumn{1}{l|}{0.00} & \multicolumn{1}{l|}{0.00} & \multicolumn{1}{l|}{0.00} & \multicolumn{1}{l|}{0.00} & \multicolumn{1}{l|}{0.00} & \multicolumn{1}{l|}{0.00} \\
sanity hard & \multicolumn{1}{l|}{ $\mathbf{45.00}$ } & \multicolumn{1}{l|}{2.44} & \multicolumn{1}{l|}{65.00} & \multicolumn{1}{l|}{1.37} & \multicolumn{1}{l|}{65.00} & \multicolumn{1}{l|}{1.37} & \multicolumn{1}{l|}{65.00} & \multicolumn{1}{l|}{1.37} & \multicolumn{1}{l|}{65.00} & \multicolumn{1}{l|}{ $\mathbf{1.37}$ } \\
bach synth c 100 & \multicolumn{1}{l|}{55.00} & \multicolumn{1}{l|}{ $\mathbf{ NaN}$ } & \multicolumn{1}{l|}{ $\mathbf{43.33}$ } & \multicolumn{1}{l|}{ NaN} & \multicolumn{1}{l|}{46.67} & \multicolumn{1}{l|}{ NaN} & \multicolumn{1}{l|}{43.33} & \multicolumn{1}{l|}{ NaN} & \multicolumn{1}{l|}{43.33} & \multicolumn{1}{l|}{ NaN} \\
bach synth c 200 & \multicolumn{1}{l|}{37.50} & \multicolumn{1}{l|}{ $\mathbf{ NaN}$ } & \multicolumn{1}{l|}{ $\mathbf{18.75}$ } & \multicolumn{1}{l|}{ NaN} & \multicolumn{1}{l|}{20.00} & \multicolumn{1}{l|}{ NaN} & \multicolumn{1}{l|}{18.75} & \multicolumn{1}{l|}{ NaN} & \multicolumn{1}{l|}{41.25} & \multicolumn{1}{l|}{ NaN} \\
breast & \multicolumn{1}{l|}{8.99} & \multicolumn{1}{l|}{ $\mathbf{ NaN}$ } & \multicolumn{1}{l|}{ $\mathbf{7.87}$ } & \multicolumn{1}{l|}{ NaN} & \multicolumn{1}{l|}{7.87} & \multicolumn{1}{l|}{ NaN} & \multicolumn{1}{l|}{7.87} & \multicolumn{1}{l|}{ NaN} & \multicolumn{1}{l|}{56.18} & \multicolumn{1}{l|}{ NaN} \\
pima & \multicolumn{1}{l|}{ $\mathbf{22.55}$ } & \multicolumn{1}{l|}{ $\mathbf{ NaN}$ } & \multicolumn{1}{l|}{26.14} & \multicolumn{1}{l|}{ NaN} & \multicolumn{1}{l|}{27.45} & \multicolumn{1}{l|}{ NaN} & \multicolumn{1}{l|}{26.47} & \multicolumn{1}{l|}{ NaN} & \multicolumn{1}{l|}{41.50} & \multicolumn{1}{l|}{ NaN} \\
sonar & \multicolumn{1}{l|}{25.61} & \multicolumn{1}{l|}{1.96} & \multicolumn{1}{l|}{26.83} & \multicolumn{1}{l|}{157.29} & \multicolumn{1}{l|}{25.61} & \multicolumn{1}{l|}{172.47} & \multicolumn{1}{l|}{25.61} & \multicolumn{1}{l|}{167.32} & \multicolumn{1}{l|}{ $\mathbf{9.76}$ } & \multicolumn{1}{l|}{ $\mathbf{0.36}$ } \\
ionosphere & \multicolumn{1}{l|}{17.14} & \multicolumn{1}{l|}{ $\mathbf{ NaN}$ } & \multicolumn{1}{l|}{12.86} & \multicolumn{1}{l|}{ NaN} & \multicolumn{1}{l|}{12.86} & \multicolumn{1}{l|}{ NaN} & \multicolumn{1}{l|}{ $\mathbf{8.57}$ } & \multicolumn{1}{l|}{ NaN} & \multicolumn{1}{l|}{65.71} & \multicolumn{1}{l|}{ NaN} \\
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

pumadyn256_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 02-May-2011 13:41:46
\begin{table}[h!]
\caption{{\small
Hyperparameters for pumadyn256 dataset
}}
\label{tbl:pumadyn256}
\begin{center}
\begin{tabular}{l | r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  \\ \hline
ARD Lengthscale & $3563.54$  & $1.64$  & $1.26$  & $3416.66$  & $3266.26$  & $2424.32$  & $3140.52$  & $3856.31$  \\ 
ARD Linear & $-0.02$  & $-0.24$  & $-0.16$  & $0.01$  & $0.06$  & $0.01$  & $0.02$  & $-0.06$  \\ 
\hline
ADD Lengthscale & $835.92$  & $1.64$  & $1.26$  & $807.57$  & $731.11$  & $610.88$  & $805.50$  & $906.10$  \\
ADD Linear & $-0.02$  & $-0.24$  & $-0.16$  & $0.01$  & $0.06$  & $0.01$  & $0.02$  & $-0.06$  \\ 
ADD Variance & $0.61$ & $1.43$ & $0.77$ & $0.61$ & $0.61$ & $0.61$ & $0.61$ & $0.61$ \\ \hline
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} \\
ADD Order Variance & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $99.9$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

prostate_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 02-May-2011 13:41:46
\begin{table}[h!]
\caption{{\small
Hyperparameters for prostate dataset
}}
\label{tbl:prostate}
\begin{center}
\begin{tabular}{l | r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  \\ \hline
ARD Lengthscale & $122.35$  & $0.07$  & $0.31$  & $2.96$  & $0.50$  & $8.58$  & $122.22$  & $223.95$  \\ 
ARD Linear & $0.63$  & $0.16$  & $-0.16$  & $0.17$  & $0.25$  & $-0.12$  & $0.04$  & $0.11$  \\ 
\hline
ADD Lengthscale & $90.40$  & $0.07$  & $0.31$  & $2.95$  & $0.52$  & $10.32$  & $90.66$  & $165.50$  \\
ADD Linear & $0.62$  & $0.16$  & $-0.16$  & $0.17$  & $0.25$  & $-0.12$  & $0.04$  & $0.11$  \\ 
ADD Variance & $0.31$ & $5.37$ & $1.76$ & $0.31$ & $0.26$ & $0.32$ & $0.31$ & $0.31$ \\ \hline
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} \\
ADD Order Variance & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $99.9$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

bach_synth_r_200_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 03-Jun-2011 00:23:44
\begin{table}[h]
\caption{{\small
Hyperparameters for bach dataset.
}}
\label{tbl:bach}
\begin{center}
\begin{tabular}{r | r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  \\ \hline
Sq-exp lengthscale & $9.43$  & $8.98$  & $9.81$  & $9.29$  & $2585.28$  & $1706.85$  & $1883.74$  & $2779.83$  \\ 
\hline
Additive lengthscale & $9.43$  & $8.98$  & $9.81$  & $9.29$  & $2588.94$  & $1709.18$  & $1885.96$  & $2783.38$  \\
\hline
Order of interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} \\
Additive order variance & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $100.0$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

bach_synth_r_100_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 20-May-2011 20:59:40
\begin{table}[h]
\caption{{\small
Hyperparameters for bach synth r 100 dataset
}}
\label{tbl:bach synth r 100}
\begin{center}
\begin{tabular}{l | r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  \\ \hline
Sq-exp Lengthscale & $5.40$  & $5.83$  & $5.27$  & $4.89$  & $8620.49$  & $8278.75$  & $7444.24$  & $7866.09$  \\ 
\hline
Additive Lengthscale & $5.42$  & $5.82$  & $5.25$  & $4.88$  & $11819.22$  & $11900.45$  & $10785.42$  & $15172.07$  \\
\hline
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} \\
Additive Order Variance & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $99.9$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

breast_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 12-May-2011 16:56:20
\begin{table}[h]
\caption{{\small
Hyperparameters for breast dataset
}}
\label{tbl:breast}
\begin{center}
\begin{tabular}{l | r r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  & $x_9$  \\ \hline
ARD Lengthscale & $12597.35$  & $0.02$  & $1948.32$  & $0.00$  & $15320796065.96$  & $0.83$  & $446.51$  & $24312.96$  & $1420.93$  \\ 
\hline
ADD Lengthscale & $12597.73$  & $0.02$  & $1948.63$  & $0.00$  & $15320796065.96$  & $0.83$  & $448.01$  & $24313.12$  & $1423.47$  \\
ADD Variance & $1.00$ & $1.01$ & $1.00$ & $1.01$ & $1.00$ & $1.00$ & $1.00$ & $1.00$ & $1.00$ \\ \hline
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} & \nth{9} \\
ADD Order Variance & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $100.0$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

crabs_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 02-May-2011 13:41:45
\begin{table}[h!]
\caption{{\small
Hyperparameters for crabs dataset
}}
\label{tbl:crabs}
\begin{center}
\begin{tabular}{l | r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  \\ \hline
ARD Lengthscale & $1.60$  & $0.98$  & $3.50$  & $3.35$  & $0.29$  & $7.42$  & $7.38$  \\ 
ARD Linear & $45.97$  & $-70.04$  & $58.23$  & $-626.17$  & $709.52$  & $-121.25$  & $57.40$  \\ 
\hline
ADD Lengthscale & $1.60$  & $0.98$  & $3.50$  & $3.35$  & $0.29$  & $7.42$  & $7.38$  \\
ADD Linear & $110.90$  & $-208.15$  & $143.59$  & $-1620.79$  & $1864.38$  & $-247.14$  & $73.17$  \\ 
ADD Variance & $1.00$ & $1.00$ & $1.00$ & $1.00$ & $1.00$ & $1.00$ & $1.00$ \\ \hline
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} \\
ADD Order Variance & $29.7$\% & $17.0$\% & $8.2$\% & $4.6$\% & $28.9$\% & $11.5$\% & $0.0$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

bach_synth_c_200_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 12-May-2011 16:56:20
\begin{table}[h]
\caption{{\small
Hyperparameters for bach synth c 200 dataset
}}
\label{tbl:bach synth c 200}
\begin{center}
\begin{tabular}{l | r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  \\ \hline
ARD Lengthscale & $598.53$  & $0.17$  & $3.41$  & $2.84$  & $0.19$  & $0.88$  & $208.99$  & $350.09$  \\ 
\hline
ADD Lengthscale & $391.45$  & $0.17$  & $3.42$  & $2.85$  & $0.19$  & $0.88$  & $143.25$  & $240.48$  \\
ADD Variance & $0.98$ & $1.08$ & $0.98$ & $0.98$ & $1.06$ & $1.00$ & $0.98$ & $0.98$ \\ \hline
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} \\
ADD Order Variance & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $100.0$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

synth_c_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 29-Apr-2011 15:03:44
\begin{table}[h!]
\caption{{\small
Hyperparameters for bach synth c 200 dataset
}}
\label{tbl:bach synth c 200}
\begin{center}
\begin{tabular}{l | r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  \\ \hline
ARD Lengthscale & $1.09$  & $0.32$  & $0.09$  & $0.74$  & $760.60$  & $522.32$  & $667.74$  & $886.37$  \\ \hline
ADD Lengthscale & $0.93$  & $0.83$  & $0.04$  & $0.62$  & $473.04$  & $325.51$  & $415.76$  & $551.96$  \\
ADD Variance & $15.48$ & $21.02$ & $0.01$ & $60.93$ & $0.67$ & $0.67$ & $0.67$ & $0.67$ \\ \hline
Order of Interaction: & $1$  & $2$  & $3$  & $4$  & $5$  & $6$  & $7$  & $8$  \\
ADD Order Variance & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $49.0$\% & $0.1$\% & $50.8$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

servo_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 12-May-2011 16:56:21
\begin{table}[h]
\caption{{\small
Hyperparameters for servo dataset
}}
\label{tbl:servo}
\begin{center}
\begin{tabular}{l | r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  \\ \hline
ARD Lengthscale & $0.49$  & $0.02$  & $0.12$  & $1.47$  \\ 
\hline
ADD Lengthscale & $0.41$  & $0.02$  & $0.12$  & $1.38$  \\
ADD Variance & $0.02$ & $0.01$ & $10.07$ & $95.16$ \\ \hline
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} \\
ADD Order Variance & $0.0$\% & $17.0$\% & $40.9$\% & $42.1$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

pima_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 12-May-2011 16:56:20
\begin{table}[h]
\caption{{\small
Hyperparameters for pima dataset
}}
\label{tbl:pima}
\begin{center}
\begin{tabular}{l | r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  \\ \hline
ARD Lengthscale & $0.30$  & $0.05$  & $4.15$  & $3.62$  & $0.26$  & $1.78$  & $326.52$  & $0.76$  \\ 
\hline
ADD Lengthscale & $0.23$  & $0.05$  & $3.75$  & $6.37$  & $0.27$  & $1.81$  & $107.98$  & $0.58$  \\
ADD Variance & $2.47$ & $0.03$ & $0.86$ & $0.75$ & $0.35$ & $1.56$ & $0.71$ & $60.36$ \\ \hline
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} \\
ADD Order Variance & $0.0$\% & $0.1$\% & $0.1$\% & $0.3$\% & $0.0$\% & $29.0$\% & $0.1$\% & $70.4$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

nd_table.tex
% --- Automatically generated by resultsToLatex2.m ---
% Exported at 06-Apr-2011 14:10:51
\begin{table}[h!]
\caption{{\small
Dataset Statistics
}}
\label{tbl:Dataset Statistics}
\begin{center}
\begin{tabular}{l | r r}
Method & \rotatebox{90}{ N }  & \rotatebox{90}{ D }  \\ \hline
sanity easy
mse_table.tex
% --- Automatically generated by resultsToLatex2.m ---
% Exported at 07-Dec-2011 14:58:35
\begin{table}[h!]
\caption{{\small
Regression Mean Squared Error
}}
\label{tbl:Regression Mean Squared Error}
\begin{center}
\begin{tabular}{l | r r r r r}
Method & \rotatebox{0}{ bach  }  & \rotatebox{0}{ concrete  }  & \rotatebox{0}{ pumadyn-8nh }  & \rotatebox{0}{ servo }  & \rotatebox{0}{ housing }  \\ \hline
Linear Regression & 1.031 & 0.404 & 0.641 & 0.523 & 0.289 \\
GP GAM & 1.259 & 0.149 & 0.598 & 0.281 & 0.161 \\
HKL & 0.199 & 0.147 & 0.346 & 0.199 & 0.151 \\
GP Squared-exp & 0.045 & 0.157 & 0.317 & 0.126 & $\mathbf{0.092}$ \\
GP Additive & $\mathbf{0.045}$ & $\mathbf{0.089}$ & $\mathbf{0.316}$ & $\mathbf{0.110}$ & 0.102 \\
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

ll_table.tex
% --- Automatically generated by resultsToLatex2.m ---
% Exported at 07-Dec-2011 14:58:35
\begin{table}[h!]
\caption{{\small
Regression Negative Log Likelihood
}}
\label{tbl:Regression Negative Log Likelihood}
\begin{center}
\begin{tabular}{l | r r r r r}
Method & \rotatebox{0}{ bach  }  & \rotatebox{0}{ concrete  }  & \rotatebox{0}{ pumadyn-8nh }  & \rotatebox{0}{ servo }  & \rotatebox{0}{ housing }  \\ \hline
Linear Regression & 2.430 & 1.403 & 1.881 & 1.678 & 1.052 \\
GP GAM & 1.708 & 0.467 & 1.195 & 0.800 & 0.457 \\
HKL &   NaN &   NaN &   NaN &   NaN &   NaN \\
GP Squared-exp & -0.131 & 0.398 & 0.843 & 0.429 & 0.207 \\
GP Additive & $\mathbf{-0.131}$ & $\mathbf{0.114}$ & $\mathbf{0.841}$ & $\mathbf{0.309}$ & $\mathbf{0.194}$ \\
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

housing_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 12-May-2011 16:56:21
\begin{table}[h]
\caption{{\small
Hyperparameters for housing dataset
}}
\label{tbl:housing}
\begin{center}
\begin{tabular}{l | r r r r r r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  & $x_9$  & $x_10$  & $x_11$  & $x_12$  & $x_13$  \\ \hline
ARD Lengthscale & $1508.26$  & $504.64$  & $296.62$  & $3.91$  & $0.08$  & $2.21$  & $5.77$  & $2.77$  & $424.77$  & $0.74$  & $233.78$  & $605.23$  & $3.75$  \\ 
\hline
ADD Lengthscale & $392.41$  & $147.62$  & $183.48$  & $12.16$  & $0.04$  & $2.33$  & $5.71$  & $0.49$  & $402.78$  & $0.68$  & $322.01$  & $1027.36$  & $1.48$  \\
ADD Variance & $0.29$ & $0.29$ & $0.29$ & $0.44$ & $0.02$ & $19.14$ & $0.25$ & $0.97$ & $0.29$ & $10.77$ & $0.29$ & $0.29$ & $0.37$ \\ \hline
Order of Interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} & \nth{9} & \nth{10} \\
ADD Order Variance & $0.1$\% & $0.1$\% & $0.1$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $99.8$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

class_table_ll.tex
% --- Automatically generated by resultsToLatex2.m ---
% Exported at 03-Jun-2011 00:23:24
\begin{table}[h!]
\caption{{\small
Classification Negative Log Likelihood
}}
\label{tbl:Classification Negative Log Likelihood}
\begin{center}
\begin{tabular}{l | r r r r r r}
Method & \rotatebox{0}{ breast }  & \rotatebox{0}{ pima }  & \rotatebox{0}{ sonar }  & \rotatebox{0}{ ionosphere }  & \rotatebox{0}{ liver }  & \rotatebox{0}{ heart }  \\ \hline
Logistic Regression & 0.247 & 0.560 & 4.609 & 0.878 & 0.864 & 0.575 \\
GP GAM & 0.163 & $\mathbf{0.461}$ & $\mathbf{0.377}$ & 0.312 & $\mathbf{0.569}$ & $\mathbf{0.393}$ \\
HKL &   NaN &   NaN &   NaN &   NaN &   NaN &   NaN \\
GP Squared-exp & $\mathbf{0.146}$ & 0.478 & 0.425 & $\mathbf{0.236}$ & 0.601 & 0.480 \\
GP Additive & 0.150 & 0.466 & 0.409 & 0.295 & 0.588 & 0.415 \\
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

class_table.tex
% --- Automatically generated by resultsToLatex2.m ---
% Exported at 03-Jun-2011 00:23:24
\begin{table}[h!]
\caption{{\small
Classification Percent Error
}}
\label{tbl:Classification Percent Error}
\begin{center}
\begin{tabular}{l | r r r r r r}
Method & \rotatebox{0}{ breast }  & \rotatebox{0}{ pima }  & \rotatebox{0}{ sonar }  & \rotatebox{0}{ ionosphere }  & \rotatebox{0}{ liver }  & \rotatebox{0}{ heart }  \\ \hline
Logistic Regression & 7.611 & 24.392 & 26.786 & 16.810 & 45.060 & $\mathbf{16.082}$ \\
GP GAM & 5.189 & $\mathbf{22.419}$ & 15.786 & 8.524 & 29.842 & 16.839 \\
HKL & 5.377 & 24.261 & 21.000 & 9.119 & $\mathbf{27.270}$ & 18.975 \\
GP Squared-exp & $\mathbf{4.734}$ & 23.722 & 16.357 & $\mathbf{6.833}$ & 31.237 & 20.642 \\
GP Additive & 5.566 & 23.076 & $\mathbf{15.714}$ & 7.976 & 30.060 & 18.496 \\
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

unicycle_y_origin_scc_400_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 06-Dec-2011 10:51:52
\begin{table}[h]
\caption{{\small
Hyperparameters for unicycle y origin sc dataset.
}}
\label{tbl:unicycle y origin sc}
\begin{center}
\begin{tabular}{r | r r r r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  & $x_9$  & $x_10$  & $x_11$  \\ \hline
Sq-exp lengthscale & $2409.88$  & $628.42$  & $48.98$  & $178.77$  & $7.42$  & $10.62$  & $50.78$  & $425.33$  & $96.22$  & $27.55$  & $555.86$  \\ 
\hline
Additive lengthscale & $3825.99$  & $7.97$  & $96.29$  & $366.68$  & $1195.94$  & $16.63$  & $75.59$  & $1405.59$  & $3634.49$  & $25.10$  & $2199.98$  \\
\hline
Order of interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} & \nth{9} & \nth{10} \\
Additive order variance & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $99.9$\% & $0.0$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

unicycle_yaw_ang_vel_400_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 06-Dec-2011 10:51:52
\begin{table}[h]
\caption{{\small
Hyperparameters for unicycle yaw ang vel dataset.
}}
\label{tbl:unicycle yaw ang vel}
\begin{center}
\begin{tabular}{r | r r r r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  & $x_9$  & $x_10$  & $x_11$  \\ \hline
Sq-exp lengthscale & $6.11$  & $52.47$  & $50.02$  & $28.94$  & $102.74$  & $6349.74$  & $8588.54$  & $2675.68$  & $5.64$  & $17.99$  & $25.37$  \\ 
\hline
Additive lengthscale & $5.51$  & $41.70$  & $37.02$  & $19.67$  & $89.15$  & $5726.56$  & $7975.25$  & $2818.12$  & $4.75$  & $17.23$  & $20.06$  \\
\hline
Order of interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} & \nth{9} & \nth{10} \\
Additive order variance & $1.2$\% & $0.0$\% & $0.5$\% & $17.0$\% & $1.0$\% & $62.4$\% & $11.8$\% & $5.7$\% & $0.3$\% & $0.0$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

unicycle_x_origin_scc_400_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 06-Dec-2011 10:51:52
\begin{table}[h]
\caption{{\small
Hyperparameters for unicycle x origin sc dataset.
}}
\label{tbl:unicycle x origin sc}
\begin{center}
\begin{tabular}{r | r r r r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  & $x_9$  & $x_10$  & $x_11$  \\ \hline
Sq-exp lengthscale & $2739.32$  & $227.19$  & $37.42$  & $124.37$  & $10.72$  & $150.69$  & $16.58$  & $777.15$  & $36.06$  & $34.73$  & $17.10$  \\ 
\hline
Additive lengthscale & $2684.49$  & $213.70$  & $35.61$  & $118.49$  & $10.11$  & $134.39$  & $15.40$  & $723.36$  & $34.50$  & $32.62$  & $16.33$  \\
\hline
Order of interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} & \nth{9} & \nth{10} \\
Additive order variance & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $99.9$\% & $0.0$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

unicycle_wheel_ang_vel_400_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 06-Dec-2011 10:51:52
\begin{table}[h]
\caption{{\small
Hyperparameters for unicycle wheel ang vel dataset.
}}
\label{tbl:unicycle wheel ang vel}
\begin{center}
\begin{tabular}{r | r r r r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  & $x_9$  & $x_10$  & $x_11$  \\ \hline
Sq-exp lengthscale & $10.28$  & $3654.69$  & $3007.84$  & $9.94$  & $3719.83$  & $4153.73$  & $5198.35$  & $3132.43$  & $5.07$  & $3440.06$  & $7.97$  \\ 
\hline
Additive lengthscale & $8.18$  & $3299.12$  & $2887.37$  & $3.99$  & $3364.71$  & $4347.59$  & $4100.68$  & $2989.40$  & $5.41$  & $3383.42$  & $6.86$  \\
\hline
Order of interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} & \nth{9} & \nth{10} \\
Additive order variance & $98.3$\% & $0.0$\% & $0.0$\% & $0.2$\% & $0.0$\% & $1.0$\% & $0.3$\% & $0.1$\% & $0.1$\% & $0.0$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

unicycle_turn_table_ang_vel_400_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 06-Dec-2011 10:51:52
\begin{table}[h]
\caption{{\small
Hyperparameters for unicycle turn table ang vel dataset.
}}
\label{tbl:unicycle turn table ang vel}
\begin{center}
\begin{tabular}{r | r r r r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  & $x_9$  & $x_10$  & $x_11$  \\ \hline
Sq-exp lengthscale & $5279.35$  & $1167.19$  & $21.80$  & $4025.89$  & $50.62$  & $15156.94$  & $15831.66$  & $58.52$  & $5.19$  & $12.47$  & $91.87$  \\ 
\hline
Additive lengthscale & $5095.86$  & $45.87$  & $21.31$  & $3890.08$  & $880.51$  & $15086.49$  & $15631.53$  & $54.71$  & $5.02$  & $11.70$  & $81.60$  \\
\hline
Order of interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} & \nth{9} & \nth{10} \\
Additive order variance & $0.1$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $99.8$\% & $0.0$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

unicycle_roll_angular_velocity_400_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 06-Dec-2011 10:51:52
\begin{table}[h]
\caption{{\small
Hyperparameters for unicycle roll angulavelocity dataset.
}}
\label{tbl:unicycle roll angulavelocity}
\begin{center}
\begin{tabular}{r | r r r r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  & $x_9$  & $x_10$  & $x_11$  \\ \hline
Sq-exp lengthscale & $151.84$  & $826.95$  & $14.47$  & $11.52$  & $6.33$  & $6141.33$  & $2625.43$  & $33.94$  & $8.94$  & $20.67$  & $21.83$  \\ 
\hline
Additive lengthscale & $146.49$  & $750.54$  & $13.73$  & $11.02$  & $5.88$  & $6090.44$  & $2497.55$  & $32.38$  & $8.41$  & $19.02$  & $20.69$  \\
\hline
Order of interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} & \nth{9} & \nth{10} \\
Additive order variance & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $99.9$\% & $0.0$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

unicycle_roll_angle_400_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 06-Dec-2011 10:51:52
\begin{table}[h]
\caption{{\small
Hyperparameters for unicycle roll angle dataset.
}}
\label{tbl:unicycle roll angle}
\begin{center}
\begin{tabular}{r | r r r r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  & $x_9$  & $x_10$  & $x_11$  \\ \hline
Sq-exp lengthscale & $17.95$  & $481.46$  & $50.53$  & $26.20$  & $25.22$  & $3514.87$  & $1839.92$  & $12.64$  & $26.19$  & $2220.49$  & $49.30$  \\ 
\hline
Additive lengthscale & $11.34$  & $2247.48$  & $25.43$  & $14.03$  & $2255.29$  & $2532.24$  & $2665.32$  & $20.04$  & $17.91$  & $2401.82$  & $7.55$  \\
\hline
Order of interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} & \nth{9} & \nth{10} \\
Additive order variance & $78.8$\% & $0.0$\% & $0.1$\% & $16.7$\% & $0.5$\% & $3.8$\% & $0.1$\% & $0.0$\% & $0.0$\% & $0.0$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

unicycle_pitch_ang_vel_400_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 06-Dec-2011 10:51:52
\begin{table}[h]
\caption{{\small
Hyperparameters for unicycle pitch ang vel dataset.
}}
\label{tbl:unicycle pitch ang vel}
\begin{center}
\begin{tabular}{r | r r r r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  & $x_9$  & $x_10$  & $x_11$  \\ \hline
Sq-exp lengthscale & $6.91$  & $61.31$  & $132.38$  & $16.04$  & $464.74$  & $3856.66$  & $4686.85$  & $164.32$  & $6.13$  & $2827.18$  & $8.55$  \\ 
\hline
Additive lengthscale & $6.59$  & $53.23$  & $127.77$  & $15.61$  & $476.21$  & $3939.83$  & $4456.74$  & $148.91$  & $6.15$  & $2663.27$  & $8.36$  \\
\hline
Order of interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} & \nth{9} & \nth{10} \\
Additive order variance & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $99.9$\% & $0.0$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

unicycle_pitch_angle_400_hypers_table.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 06-Dec-2011 10:51:52
\begin{table}[h]
\caption{{\small
Hyperparameters for unicycle pitch angle dataset.
}}
\label{tbl:unicycle pitch angle}
\begin{center}
\begin{tabular}{r | r r r r r r r r r r r}
Variable: & $x_1$  & $x_2$  & $x_3$  & $x_4$  & $x_5$  & $x_6$  & $x_7$  & $x_8$  & $x_9$  & $x_10$  & $x_11$  \\ \hline
Sq-exp lengthscale & $223.14$  & $880.20$  & $3356.16$  & $9.47$  & $84.89$  & $5733.92$  & $6092.69$  & $4467.49$  & $6.77$  & $53.37$  & $15.37$  \\ 
\hline
Additive lengthscale & $15.22$  & $1187.06$  & $2168.08$  & $7.77$  & $549.71$  & $3792.06$  & $4028.69$  & $3055.90$  & $5.87$  & $7.12$  & $7.32$  \\
\hline
Order of interaction: & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} & \nth{9} & \nth{10} \\
Additive order variance & $99.7$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.0$\% & $0.2$\% \\ \hline
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

all_orders_table_frozen.tex
% --- Automatically generated by hypers_to_latex.m ---
% Exported at 03-Jun-2011 00:22:23
\begin{table}[h]
\caption{{\small
Relative variance contribution of each order in the additive model, on different datasets. Here, the maximum order of interaction is set to 10, or smaller if the input dimension less than 10.  Values are normalized to sum to 100.
}}
\label{tbl:all_orders}
\begin{center}
\begin{tabular}{r | r r r r r r r r r r}
Order of interaction & \nth{1} & \nth{2} & \nth{3} & \nth{4} & \nth{5} & \nth{6} & \nth{7} & \nth{8} & \nth{9} & \nth{10} \\ \hline
pima  & $0.1 $ & $0.1 $ & $0.1 $ & $0.3 $ & $1.5 $ & ${\bf 96.4}$ & $1.4 $ & $0.0 $ & & \\
liver  & $0.0 $ & $0.2 $ & ${\bf 99.7 } $ & $0.1 $ & $0.0 $ & $0.0 $ & & & & \\
heart  & ${\bf 77.6} $ & $0.0 $ & $0.0 $ & $0.0 $ & $0.1 $ & $0.1 $ & $0.1 $ & $0.1 $ & $0.1 $ & $22.0 $ \\
concrete  & ${\bf 70.6 } $ & $13.3 $ & $13.8 $ & $2.3 $ & $0.0 $ & $0.0 $ & $0.0 $ & $0.0 $ & & \\
pumadyn-8nh  & $0.0 $ & $0.1 $ & $0.1 $ & $0.1 $ & $0.1 $ & $0.1 $ & $0.1 $ & ${\bf 99.5 } $ & & \\
servo  & ${\bf 58.7 }$ & $27.4 $ & $0.0 $ & $13.9 $ & & & & & & \\
housing  & $0.1 $ & $0.6 $ & ${\bf 80.6 }$ & $1.4 $ & $1.8 $ & $0.8 $ & $0.7 $ & $0.8 $ & $0.6 $ & $12.7 $ \\
\end{tabular}
\end{center}
\end{table}
% End automatically generated LaTeX

main.bbl
\begin{thebibliography}{10}

\bibitem{nelder1972generalized}
J.A. Nelder and R.W.M. Wedderburn.
\newblock Generalized linear models.
\newblock {\em Journal of the Royal Statistical Society. Series A (General)},
  135(3):370--384, 1972.

\bibitem{hastie1990generalized}
T.J. Hastie and R.J. Tibshirani.
\newblock {\em Generalized additive models}.
\newblock Chapman \& Hall/CRC, 1990.

\bibitem{wahba1990spline}
G.~Wahba.
\newblock {\em {Spline models for observational data}}.
\newblock Society for Industrial Mathematics, 1990.

\bibitem{DBLP:journals/corr/abs-0909-0844}
Francis Bach.
\newblock High-dimensional non-linear variable selection through hierarchical
  kernel learning.
\newblock {\em CoRR}, abs/0909.0844, 2009.

\bibitem{rasmussen38gaussian}
C.E. Rasmussen and CKI Williams.
\newblock {Gaussian Processes for Machine Learning}.
\newblock {\em The MIT Press, Cambridge, MA, USA}, 2006.

\bibitem{plate1999accuracy}
T.A. Plate.
\newblock {Accuracy versus interpretability in flexible modeling: Implementing
  a tradeoff using Gaussian process models}.
\newblock {\em Behaviormetrika}, 26:29--50, 1999.

\bibitem{macdonald1998symmetric}
I.G. Macdonald.
\newblock {\em {Symmetric functions and Hall polynomials}}.
\newblock Oxford University Press, USA, 1998.

\bibitem{stanley2001enumerative}
R.P. Stanley.
\newblock {\em {Enumerative combinatorics}}.
\newblock Cambridge University Press, 2001.

\bibitem{kaufman2010bayesian}
C.G. Kaufman and S.R. Sain.
\newblock Bayesian functional anova modeling using {G}aussian process prior
  distributions.
\newblock {\em Bayesian Analysis}, 5(1):123--150, 2010.

\bibitem{christoudias2009bayesian}
M.~Christoudias, R.~Urtasun, and T.~Darrell.
\newblock {Bayesian localized multiple kernel learning}.
\newblock {\em Technical report}, 2009.

\bibitem{vapnik1998statistical}
V.N. Vapnik.
\newblock {\em {Statistical learning theory}}, volume~2.
\newblock Wiley New York, 1998.

\bibitem{stitson1999support}
M.~Stitson, A.~Gammerman, V.~Vapnik, V.~Vovk, C.~Watkins, and J.~Weston.
\newblock {Support vector regression with ANOVA decomposition kernels}.
\newblock {\em Advances in kernel methods: Support vector learning}, pages
  285--292, 1999.

\bibitem{bengio2006curse}
Y.~Bengio, O.~Delalleau, and N.~Le~Roux.
\newblock {The curse of highly variable functions for local kernel machines}.
\newblock {\em Advances in neural information processing systems}, 18, 2006.

\bibitem{nocedal1980updating}
J.~Nocedal.
\newblock Updating quasi-newton matrices with limited storage.
\newblock {\em Mathematics of computation}, 35(151):773--782, 1980.

\bibitem{minka2001expectation}
T.P. Minka.
\newblock Expectation propagation for approximate {B}ayesian inference.
\newblock In {\em Uncertainty in Artificial Intelligence}, volume~17, pages
  362--369, 2001.

\end{thebibliography}
