\vspace{-0.8in
Trisha Maitra and Sourabh Bhattacharya\thanks{ Trisha Maitra is a PhD student and Sourabh Bhattacharya is an Associate Professor in Interdisciplinary Statistical Research Unit
Indian Statistical Institute
203
B. T. Road
Kolkata 700108. Corresponding e-mail: sourabh@isical.ac.in.
The problem of model selection in the context of a system of stochastic differential equations ($SDE$'s) has not been touched upon in the literature. Indeed, properties of Bayes factors have not been studied even in single $SDE$ based model comparison problems. In this article, we first develop an asymptotic theory of Bayes factors when two $SDE$'s are compared, assuming the time domain expands. Using this we then develop an asymptotic theory of Bayes factors when systems of $SDE$'s are compared, assuming that the number of equations in each system, as well as the time domain, increase indefinitely. Our asymptotic theory covers situations when the observed processes associated with the $SDE$'s are independently and identically distributed ($iid$), as well as when they are independently but not identically distributed (non-$iid$). Quite importantly, we allow inclusion of available time-dependent covariate information into each $SDE$ through a multiplicative factor of the drift function in a random effects set-up; different initial values for the $SDE$'s are also permitted. Thus, our general model-selection framework includes simultaneously the variable selection problem associated with time-varying covariates, as well as choice of the part of the drift function free of covariates. It is to be noted that given that the underlying process is wholly observed, the diffusion coefficient becomes known, and hence is not involved in the model selection problem. For both $iid$ and non-$iid$ set-ups we establish almost sure exponential convergence of the Bayes factor. As we show, the Bayes factor is inconsistent for comparing individual $SDE$'s, in the sense that the log-Bayes factor converges only in expectation, while the relevant variance does not converge to zero. Nevertheless, %in spite of the seemingly negative outcome, it has been possible to exploit this result to establish almost sure exponential convergence of the Bayes factor when, in addition, the number of individuals are also allowed to increase indefinitely. We carry out simulated and real data analyses to demonstrate that Bayes factor is a suitable candidate for covariate selection in our $SDE$ models even in non-asymptotic situations. \\[2mm] {Keywords: Bayes factor consistency; Kullback-Leibler divergence; Martingale; Stochastic differential equations; Time-dependent covariates and random effects; Variable selection.
application.tex

\section{Simulation studies}
\label{sec:simulated_data}

\subsection{Covariate selection when $n=15$, $T=1$}
\label{subsec:N15T1}
We first demonstrate with simulation study the finite sample analogue of Bayes factor analysis
as $n\rightarrow\infty$ and $T$ is fixed. In this regard, we consider $n=15$ individuals, where the $i$-th one is modeled by
\begin{equation}
dX_i(t)=(\xi_1+\xi_2z_1(t)+\xi_3z_2(t)+\xi_4z_3(t))(\xi_5+\xi_6X_i(t))dt+\sigma_i dW_i(t),
\label{eq:sde2_appl}
\end{equation}
for $i=1,\cdots,15$. We fix our diffusion coefficients as $\sigma_{i+1}=\sigma_i +5$ for $i=1\cdots,14$ 
where $\sigma_1=10$. We consider the initial value $X(0)=0$ and the time interval $[0, T]$ with $T=1$. 
%Thus, compared to Section \ref{subsec:N1T5}, here there are more individuals but smaller time domain.
%As before, we consider $500$ discretized and equispaced time steps, using which we generated the data.

We obtain the true values of $\xi_i$; $i=1,\ldots,6$ as follows:
$\xi_{i}\stackrel{iid}{\sim} N(\mu_i,0.001^2)$, where 
$\mu_i\stackrel{iid}{\sim} N(0,1)$. This is not to be interpreted as the prior; this
is just a means to set the true vaues of the parameters of the data-generating model.
%Indeed, see for our purpose, we consider a multivariate normal prior for $\btheta$ with mean
%being the maximum likelihood estimator ($MLE$) and appropriate variance.
\\[2mm]
We assume that the time dependent covariates $z_i(t)$ satisfy the following $SDE$s 
\begin{align}
dz_1(t)=&(\tilde\theta_{1}+\tilde\theta_{2}z_1(t))dt+ dW_1(t)\notag\\
dz_2(t)=&\tilde\theta_{3}dt+ dW_2(t)\notag\\
dz_3(t)=&\tilde\theta_{4}z_3(t))dt+ dW_3(t),
\label{eq:covariate_appl}
\end{align}
where $W_i(\cdot)$; $i=1,2,3$, are independent Wiener processes, and 
$\tilde\theta_{i}\stackrel{iid}{\sim} N(0,0.01^2)$ for $i=1,\cdots,4$. 
%Here the diffusion coefficient $\sigma=1$. 

We obtain the covariates by first simulating $\tilde\theta_{i}\stackrel{iid}{\sim} N(0,0.01^2)$ for $i=1,\cdots,4$,
fixing the values, and then by simulating the covariates using the $SDE$s (\ref{eq:covariate_appl}) 
by discretizing the time interval $[0,5]$ into $500$ equispaced time points. 
In all our applications we have standardized the covariates over time so that they have zero means
and unit variances.

Once the covariates are thus obtained, we assume that the data are generated from the (true) model where all the covariates
are present. For the true values of the parameters, we simulated $(\xi_1,\ldots,\xi_6)$ from
the prior and treated the obtained values as the true set of parameters $\btheta_0$.
We then generated the data using (\ref{eq:sde2_appl}) by discretizing the time interval $[0,5]$ 
into $500$ equispaced time points.



As we have three covariates so we will have $2^3=8$ different models. Denoting
a model by the presence and absence of the respective covariates, it then is the case that 
$(1,1,1)$ is the true, data-generating model, while $(0,0,0)$, $(0,0,1)$, $(0,1,0)$, $(0,1,1)$, $(1,0,0)$,
$(1,0,1)$, and $(1,1,0)$ are the other $7$ possible models. 
%Now, we have to check values of the Bayes factor corresponding to $7$ different models. 

\subsubsection{Case 1: the true parameter set $\btheta_0$ is fixed}
\label{subsubsec:case1_appl}
{\bf Prior on $\btheta$}
\\[2mm]
%\label{subsubsec:case1_prior_BF}
For the prior $\pi$ on $\btheta$, we first 
obtain the maximum likelihood estimator ($MLE$) of $\btheta$ using simulated annealing
(see, for example, \ctn{Liu01}, \ctn{Robert04}), and
consider a normal prior where the mean is the $MLE$ of $\xi_i$ for $i=1,\ldots,6$ 
and the variance is $0.8^2\mathbb I_6$, $\mathbb I_6$ being the $6$-dimensional identity matrix. 
As will be seen, this results in consistent model selection using Bayes factor.
\\[2mm]
{\bf Form of the Bayes factor}
%\label{subsec:case1_bf}
\\[2mm]
In this case the related Bayes factor has the form
\begin{equation*}
I_n=\int \prod_{i=1}^n\frac {f_{i,\btheta_1}(X_i)}{f_{i,\btheta_0}(X_i)}\pi (d\btheta_1),
\label{eq:bf_T_appl}
\end{equation*}
where $\btheta_0=(\xi_{0,1},\xi_{0,2},\xi_{0,3},\xi_{0,4},\xi_{0,5},\xi_{0,6})$ is the true parameter set 
and $\btheta_1=(\xi_1,\xi_2,\xi_3,\xi_4,\xi_5,\xi_6)$ is the unknown set of parameters corresponding to 
any other model. 
%The prior distribution on the parameter set is denoted by $\pi$. The true likelihood is 
%$f_{\btheta_0,T}(X_{0,T})$. To check whether (\ref{eq:bf_appl}) holds we write $7$ 
%different wrong likelihoods in the place of $f_{\btheta_1,T}(X_{0,T})$ sequentially and obtain $7$ 
%different values of Bayes factor. 
%In this regard we use the $\btheta$ from a normal prior where 
%the mean is the $MLE$ of $\xi_i$ for $i=1,\cdots,6$ obtained by simulated annealing and variance is $0.8^2$. 
Table \ref{table:values} describes the results of our Bayes factor analyses. 
%where we denote the presence and absence of covariates by 1 and 0 respectively.
\begin{table}[h]
\centering
\caption{Bayes factor results}
\label{table:values}
\begin{tabular}{|c||c|}
\hline
Model & $\frac{1}{15}\log I_{15}$\\
\hline
$(0,0,0)$ & -3.25214\\ 
$(0,0,1)$ & -1.39209\\
$(0,1,0)$ & -3.31954\\
$(0,1,1)$ & -1.11729\\
$(1,0,0)$ & -3.40378\\
$(1,0,1)$ & -1.22529\\
$(1,1,0)$ & -3.46790\\
\hline
\end{tabular}
\end{table}
It is clear from the 7 values of the table that the correct model $(1,1,1)$ is always preferred.

\subsubsection{Case 2: the parameter set $\btheta_0$ is random and has the prior distribution $\pi$}
\label{subsubsec:case2_appl}
%{\bf Prior on $\btheta$}
%\\[2mm]
%\label{subsubsec:case2_prior_BF}
We consider the same form of the prior $\pi$ as in %Section \ref{subsubsec:case1_prior_BF}, 
Section \ref{subsubsec:case1_appl}, but with
variance $0.1^2\mathbb I_6$. The smaller variance compared to that in Case 1 attempts to somewhat 
compensate, in essence, for the lack of precise information about the true parameter values. 

In this case we calculate the marginal log-likelihood of the $8$ possible models as
$$\ell_i=\frac{1}{15}\log\int \prod_{i=1}^nf_{i,\btheta_1}(X_i)\pi (d\btheta_1);~i=1,\ldots,8,$$
with $\ell_8$ corresponding to the true model.
Table \ref{table:deltavalues} shows that $\ell_8$ 
is the highest. This clearly implies that 
the Bayes factor consistently selects the correct set of covariates even though the parameters of the true model are not fixed.
%As then 
%$$\frac{1}{5}E_{\btheta_0}\left[\left(\log\int f_{i,\btheta_1,5}(X_{0,5})\pi (d\btheta_1)\right)
%-\left(\log\int f_{\btheta_0,5}(X_{0,5})\pi (d\btheta_0)\right)\right]=\alpha_i-\alpha_0<0$$ for $i=1,\ldots,7$.
\begin{table}[h]
\centering
\caption{Values of $\frac{1}{15}\times$ marginal log-likelihoods}
\label{table:deltavalues}
\begin{tabular}{|c||c|}
\hline
Model & $\ell_i$\\
\hline
$(0,0,0)$ & 2.42430\\ 
$(0,0,1)$ & 4.29608\\
$(0,1,0)$ & 1.75213\\
$(0,1,1)$ & 4.84717\\
$(1,0,0)$ & 1.56242\\
$(1,0,1)$ & 4.92628\\
$(1,1,0)$ & 0.47111\\
$(1,1,1)$ & 5.84665 ($\mbox{true model}$)\\
\hline
\end{tabular}
\end{table}

\subsection{Covariate selection when $n=1$, $T=5$}
\label{subsec:N1T5}

We now assume the same set-up as Section \ref{subsec:N15T1}, but set $n=1$, $T=5$, and 
for the diffusion coefficient, set $\sigma_1=\sigma=20$.
As per our theory, for a single individual, the Bayes factor is not consistent for increasing time domain.
However, we have shown that 
\begin{equation*}
\frac{1}{T}E_{\btheta_0}(\log I_T)\rightarrow -\delta
\label{eq:bf_appl}
\end{equation*}
as $T\rightarrow \infty$. Thus, the Bayes factor is consistent with respect to the expectation. 
Our simulation results show that this holds even for the time domain $[0,5]$,
where we approximate the expectation with the average of %$500$ 
$1000$ realizations of $I_T$
associated with as many simulated data sets. %case we have simulated the data 500 times to verify the result (\ref{eq:bf_appl}). 


\subsubsection{Case 1: the true parameter set $\btheta_0$ is fixed}
\label{subsubsec:case1_appl2}
{\bf Prior on $\btheta$}
\\[2mm]
As Section \ref{subsubsec:case1_appl}, 
we obtain the maximum likelihood estimator ($MLE$) of $\btheta$ using simulated annealing
and consider a normal prior with the $MLE$ as the mean and variance $0.8^2\mathbb I_6$.
\\[2mm]
{\bf Form of the Bayes factor}
%\label{subsec:case1_bf}
\\[2mm]
In this case the related Bayes factor has the form
\begin{equation}
I_T=\int \frac {f_{\btheta_1,T}(X_{0,T})}{f_{\btheta_0,T}(X_{0,T})}\pi (d\btheta_1),
\label{eq:bf_T_appl2}
\end{equation}
where $\btheta_0=(\xi_{0,1},\xi_{0,2},\xi_{0,3},\xi_{0,4},\xi_{0,5},\xi_{0,6})$ is the true parameter set 
and $\btheta_1=(\xi_1,\xi_2,\xi_3,\xi_4,\xi_5,\xi_6)$ is the unknown set of parameters corresponding to 
any other model. 
%The prior distribution on the parameter set is denoted by $\pi$. The true likelihood is 
%$f_{\btheta_0,T}(X_{0,T})$. To check whether (\ref{eq:bf_appl}) holds we write $7$ 
%different wrong likelihoods in the place of $f_{\btheta_1,T}(X_{0,T})$ sequentially and obtain $7$ 
%different values of Bayes factor. 
%In this regard we use the $\btheta$ from a normal prior where 
%the mean is the $MLE$ of $\xi_i$ for $i=1,\cdots,6$ obtained by simulated annealing and variance is $0.8^2$. 
Table \ref{table:values2} describes the results of our Bayes factor analyses. 
%where we denote the presence and absence of covariates by 1 and 0 respectively.
\begin{table}[h]
\centering
\caption{Bayes factor results}
\label{table:values2}
\begin{tabular}{|c||c|}
\hline
Model & Averaged $\frac{1}{5}\log I_5$\\
\hline
$(0,0,0)$ & -2.5756029\\ 
$(0,0,1)$ & -0.913546\\
$(0,1,1)$ & -0.5454860\\
$(0,1,0)$ & -0.763952\\
$(1,0,0)$ & -2.5774163\\
$(1,0,1)$ & -0.9312218\\
$(1,1,0)$ & -0.7628154\\
\hline
\end{tabular}
\end{table}
It is clear from the 7 values of the table that the correct model $(1,1,1)$ is always preferred.

\subsubsection{Case 2: the parameter set $\btheta_0$ is random and has the prior distribution $\pi$}
\label{subsubsec:case2_appl2}
%{\bf Prior on $\btheta$}
%\\[2mm]
%\label{subsubsec:case2_prior_BF}
As before, we consider the same form of the prior as in Section \ref{subsubsec:case1_appl2},
but with variance $0.1^2\mathbb I_6$. 
In this case we calculate marginal likelihood of the 8 possible models, and approximate
$$\frac{1}{5}E_{\btheta_0}\left(\log\int f_{i,\btheta_1,5}(X_{0,5})\pi (d\btheta_1)\right)$$ for $i=1,\ldots,8$ by 
averaging over $1000$ replications of the data obtained from the true model. 
Denoting its values by $\ell_i$,  Table \ref{table:deltavalues2} shows that $\ell_8$ 
is the highest, implying consistency of the averaged Bayes factor.
\begin{table}[h]
\centering
\caption{Averages of $\frac{1}{5}\times$ marginal log-likelihood}
\label{table:deltavalues2}
\begin{tabular}{|c||c|}
\hline
Model & $\ell_i$\\
\hline
$(0,0,0)$ & -1.21923\\ 
$(0,0,1)$ & -0.21428\\
$(0,1,0)$ & 1.47992\\
$(0,1,1)$ & 2.102966\\
$(1,0,0)$ & -1.222362\\
$(1,0,1)$ & -0.21898\\
$(1,1,0)$ & 1.459921\\
$(1,1,1)$ & 2.121237 ($\mbox{true model}$)\\
\hline
\end{tabular}
\end{table}

\begin{comment}
\subsubsection{Posterior predictive inference for the chosen model}
\label{subsubsec:conv_parameter}

We now evaluate the performance of the selected model by means of posterior predictive inference. Specifically,
we attempt to predict the last $50$ observations, given the first $450$. 
For this, we first sample from the posterior of $\btheta_1=(\xi_1,\xi_2,\xi_3,\xi_4,\xi_5,\xi_6)$
given the first $450$ observations, and given each realized value of $\btheta_1$ from this posterior,
we obtain the discretized time series for the last $50$ observations, after sampling the Wiener process
for the last $50$ time points conditional on the Wiener process for the first $450$ time points. 
For posterior predictive inference we assume the following prior: for $i=1,\ldots,6$, 
$\xi_i\stackrel{iid}{\sim}N(0,0.0006^2)$.

We sampled from the posterior of $\btheta_1$ using Transformation based Markov Chain Monte Carlo (TMCMC)
introduced by \ctn{SS2012}, considering the additive transformation in particular. 
The importance of TMCMC is that it updates high-dimensional parameters in a single
block using suitable deterministic transformations of arbitrary low-dimensional/one-dimensional
random variables defined on some relevant support. So this method drastically reduces dimension, which,
in turn, significantly reduces computational burden and improves acceptance rate. 

In a nutshell, first we fix an initial (old) value $\xi_{i,old}$; $i=1,\ldots,6$, which we simulate from the prior. 
At each iteration of additive TMCMC we then propose an update of the form
$\xi_{i,new}=\xi_{i,old}+b_i a |\epsilon|$ for $i=1,\cdots,6$, , where 
$b_i$ such that $b_i=\pm 1$ with probability $1/2$, $\epsilon \sim N(0,1)$ and $a=0.01$ is a tuning parameter.
Let $\ell_{new}-\ell_{old}$ denote the difference between the log-likelihood evaluated at the
new and old values of the parameters. If $\log u\leq \ell_{new}-\ell_{old}$,
we update the old values of the parameters, otherwise, we retain 
the old values.
We have thus simulated $150000$ TMCMC realizations from the posterior. 

We found that the TMCMC convergence of the parameters are good under considered prior distribution. 
Here we have shown the TMCMC convergence of the parameters under the prior $N(0,0.006^2)$ by Figure \ref{fig:posteriors}.

\begin{figure}
\begin{subfigure}
\centering
\includegraphics[height=6cm,width=5cm]{Application/xi1.pdf}
\label{fig:sfig1}
\end{subfigure}
\begin{subfigure}
\centering
\includegraphics[height=6cm,width=5cm]{Application/xi2.pdf}
\label{fig:sfig2}
\end{subfigure}
\begin{subfigure}
\centering
\includegraphics[height=6cm,width=5cm]{Application/xi3.pdf}
\label{fig:sfig3}
\end{subfigure}
\begin{subfigure}
\centering
\includegraphics[height=6cm,width=5cm]{Application/xi4.pdf}
\label{fig:sfig4}
\end{subfigure}
\begin{subfigure}
\centering
\includegraphics[height=6cm,width=5cm]{Application/xi5.pdf}
\label{fig:sfig5}
\end{subfigure}
\begin{subfigure}
\centering
\includegraphics[height=6cm,width=5cm]{Application/xi6.pdf}
\label{fig:sfig6}
\end{subfigure}
\caption{TMCMC convergence of the posterior distribution of the parameters}
\label{fig:posteriors}
\end{figure}

\subsubsection{Posterior Predictives}
\label{subsubsec:pp}

Conditioning on the Brownian motions for the first 450 time poins we obtain the Brownian motion for the last 50 time points which will be useful for the evaluation of posterior predictives of the last 50 data points. By thinning from 150000 posterior values of the parameter set we have chosen 1000 values. Corresponding to 1000 values of the parameter set we will have 1000 posterior predictives which we denote by $Yp_j$  for $j=1,\cdots,1000$. 

From (\ctn{Sabya11}) we get the idea of obtaining the modes and any desired highest posterior density credible regions associated with the posterior distribution. With the help of that idea we define posterior mode of $Yp_j$  for $j=1,2,\cdots,N$ as
$$P(\{Yp_j:d(Yp^*,Yp_j)< \epsilon\})=\underset{Yp_i; 1\leq i\leq N}{\sup} P(\{Yp_j:d(Yp_i,Yp_j)< \epsilon\})$$
where the metric $d$ is chosen as supnorm metric and $N=1000$.
 For $N=1000, Yp^*$ is caculated emperically by
$$Yp^*=\underset{1\leq i\leq N }{\arg \max}\frac{1}{N}\#\{Yp_j; 1\leq j\leq N:d(Yp_i,Yp_j)< \epsilon\}.$$
 Given $Yp^*,$ $95\%$ posterior density credible region is obtained as the set 

$\{Yp_j; 1\leq j\leq N:d(Yp^*,Yp_j)< \epsilon^*\}$ where $\epsilon^*$ is such that
$$\frac{1}{N}\#\{Yp_j; 1\leq j\leq N:d(Yp^*,Yp_j)< \epsilon^*\}\approx 0.95$$

We also find the posterior predictive from which the distance of the true data is minimum (call it as minimum posterior predictive) and calculate $95\%$ posterior density credible region around it as the same way as mentioned in the case of posterior mode. 

Now, to conclude which posterior predictive we will chose, for that we note the distance of posterior mode from true data (denoted by PMD) and the radius of $95\%$ posterior density credible region around it (denoted by PMDR). Besides, we also note the minimum distance of posterior predictive from the true data (denoted by MD) and the radius of $95\%$ posterior density credible region around that particular posterior predictive (denoted by MDR). Now by comparing the sum of PMD and PMDR (denoted by $T_{PM}$) and the sum of MD and MDR (denoted by $T_{M}$) whichever is minimum we consider that particular posterior predictive as our required posterior predictive. In our case we find $T_M$ is minimum. The figure \ref{fig:figpp} shows the posterior predictives along with true data in which by "minimum" we mean that posterior predictive from which the distance of the true data is minimum.

\begin{figure}
\centering
\includegraphics[height=10cm,width=9cm]{Application/data1time5.pdf}
\caption{Posterior Predictive for N=1, T=5}
\label{fig:figpp}
\end{figure}

\end{comment}

\begin{comment}
\subsection{Comparison between different models for more than one individual}
\label{subsec:N15T1}

We now consider $15$ individuals, where the $i$-th one is modeled by
\begin{equation}
dX_i(t)=(\xi_1+\xi_2z_1(t)+\xi_3z_2(t)+\xi_4z_3(t))(\xi_5+\xi_6X_i(t))dt+\sigma_i dW_i(t),
\label{eq:sde2_appl}
\end{equation}
for $i=1,\cdots,15$. We fix our diffusion coefficients as $\sigma_{i+1}=\sigma_i +5$ for $i=1\cdots,14$ 
where $\sigma_1=10$. We consider the initial value $X(0)=0$ and the time interval $[0, T]$ with $T=1$. 
Thus, compared to Section \ref{subsec:N1T5}, here there are more individuals but smaller time domain.
As before, we consider $500$ discretized and equispaced time steps, using which we generated the data.
%We break the time domain in 500 time points i.e we break the time interval with the step size 0.002. 

Here our covariates and the true parameter set $\btheta_0=(\xi_{0,1,}\xi_{0,2},\xi_{0,3},\xi_{0,4},\xi_{0,5},\xi_{0,6})$  
are simulated in a similar way as described in Section  \ref{subsec:N1T5}. 

For each of the $15$ individuals, we selected the covariate combination randomly. Thus, 
Also we compare this combination with other $100$ different covariate combinations for these $15$ 
individuals to check whether Bayes Factor can select the correct combination.

In this case our related Bayes Factor corresponding to j-th covariate combination
\begin{align}
I_N^j&=\int  R_N^j(\btheta_1)\pi(d\btheta_1),
\label{eq:I_n_appl}
\end{align}
 where
\[
R_N^j(\btheta_1)=\prod_{i=1}^N\frac{f^j_{i,\btheta_1}(X_i)}{f_{i,\btheta_0}(X_i)}.
\]
for $j=1,\cdots,100$ where $\btheta_0$ is the true parameter set. Note  that in our set-up, N=15.

Firstly, by simulated annealing we obtain the MLE of the parameter set. Then we calculate the Bayes factor by taking the normal prior with mean as the respective MLEs and variance 1. In all the cases corresponding to 100 covariate combinations we obtain
$\frac{1}{N}\log I^j_{N}$ i.e. $$\frac{1}{15}\log I^j_{15}=-\delta_j$$ for $j=1,\cdots,100$. This shows that Bayes Factor indicates the correct covariate combination. As in section 
\ref{subsec:bf_appl} we also consider the case when the true covariate combination is unknown to us. We denote the joint likelihood corresponding to j-th covariate combination by $\prod_{i=1}^{15}f^j_{i,\btheta_1}(X_{i})$ and that corresponding to true covariate combination by $\prod_{i=1}^{15}f_{i,\btheta_0}(X_{i})$.  With respect to the normal prior with mean as obtained from simulated annealing and variance $0.1^2$  we find

$$\frac{1}{15}\left[\left(\log\int \prod_{i=1}^{15}f^j_{i,\btheta_1}(X_{i})\pi (d\btheta_1)\right)-\left(\log\int \prod_{i=1}^{15}f_{i,\btheta_0}(X_{i})\pi (d\btheta_0)\right)\right]=\alpha_j-\alpha_0<0$$ for $j=1,\cdots,100,$ as maximum value of $\alpha_j$  is -0.118787 among 100 values of $\alpha_j$ and $\alpha_0=1.31028$. This result indicates that though the correct covariate combination is unknown to us we are able to find that combination.

 After getting the true covariate combination for 15 individuals we want to predict the last 50 time points for each individual by conditioning on first 450 time points as mentioned in section \ref{subsubsec:pp}. The process of obtaining posterior predictives is same as described in section \ref{subsubsec:pp}. Here also after obtaining the posterior mode and the posterior predictive from which the distance of true data is minimum we calculate $T_{PM}$ and $T_M$.  In the figure \ref{fig:pp2} in case of 1st and 6th data we find $T_M$ is minimum. For the 7th data the posterior mode and the posterior predictive from which the distance of the true data is minimum (which is denoted in other two figures by "minimum") coincide.


\begin{figure}
\begin{subfigure}
\centering
\includegraphics[height=6cm,width=5cm]{Application/data15time11st.pdf}
\label{fig:sfig8}
\end{subfigure}
\begin{subfigure}
\centering
\includegraphics[height=6cm,width=5cm]{Application/data15time16th.pdf}
\label{fig:sfig9}
\end{subfigure}
\begin{subfigure}
\centering
\includegraphics[height=6cm,width=5cm]{Application/data15time17th.pdf}
\label{fig:sfig10}
\end{subfigure}
\caption{Posterior Predictives of 3 Data}
\label{fig:pp2}
\end{figure}

\end{comment}

\subsection{Bayes factor analysis for $n=15$ and $T=5$}
\label{subsec:N15T5}

In this case we allow our parameter and the covariate sets to vary from individual to individual. 
We consider $15$ individuals modeled by
\begin{equation}
dX_i(t)=(\xi_1^i+\xi_2^iz_1(t)+\xi_3^iz_2(t)+\xi_4^iz_3(t))(\xi_5^i+\xi_6^iX_i(t))dt+\sigma_i dW_i(t)
\label{eq:sde3_appl3}
\end{equation}
for $i=1,\cdots,15$. We fix our diffusion coefficients as $\sigma_{i+1}=\sigma_i +5$ for $i=1\cdots,14$ 
where $\sigma_1=10$. We consider the initial value $X(0)=0$ and the interval $[0, T]$, with $T=5$. 
As before, we generated the observed data after discretizing the time interval into $500$ equispaced time points. 
%We break the time domain in 500 time points i.e we break the time interval with the step size 0.01. 
Here our covariates and the parameter set 
$\btheta_0^i=(\xi_{0,1}^i,\xi_{0,2}^i,\xi_{0,3}^i,\xi_{0,4}^i,\xi_{0,5}^i,\xi_{0,6}^i)$; $i=1,\ldots,15$, 
are simulated in a similar way as mentioned in Section \ref{subsec:N1T5}.

For each of the $15$ individuals, the true set of covariate combination is randomly selected. 
Thus, for a given model, there are $15$ sets of covariate combinations to be compared with other models
consisting of $15$ different sets of covariate combinations.
To decrease computational burden we compare the true model with $100$ other models consisting of 
different sets of covariate combinations. 

The Bayes factor corresponding to the $j$-th covariate combination is given by
\begin{equation}
I_{nT}^j=\prod_{i=1}^n\int \frac{f^j_{i,\btheta^{(i)}_1}(X_{i,0,T})}
{f_{i,\btheta^{(i)}_0}(X_{i,0,T})}\pi\left(\btheta^{(i)}_{1}\right)d\btheta^{(i)}_{1}
\label{eq:I_iT_appl}
\end{equation}
for $j=1,\cdots,1000$, where $n=15$, $T=5$ and $\btheta_0^{(i)}$ is the true parameter set corresponding to 
the $i$-th individual.

We obtain the $MLE$ of the $15$ parameter sets by simulated annealing. Then we calculate the Bayes factor 
with the prior such that the parameter components are independent normal with means as the respective $MLE$s and variances $1$. 
In all the cases corresponding to $100$ covariate combinations we obtain 
$\frac{1}{nT}\log I^j_{nT}<0$ for $j=1,\cdots,100$. 
Thus, Bayes factor indicated the correct covariate combination in all the cases considered. 
%As in section \ref{subsec:bf_appl} 
We also considered the case when a normal prior is considered for the parameters of the true model. 
In this case with respect to the component-wise independent normal prior with individual mean as obtained 
from simulated annealing and component-wise variance $0.1^2$, we obtain 
\begin{equation}
\frac{1}{15\times 5}\left[\log\left(\prod_{i=1}^{15}\int f^j_{i,\btheta_1^{(i)}}(X_{i,0,T})
\pi(\btheta^{(i)}_1)d\btheta_1^{(i)}\right)-\log\left(\prod_{i=1}^{15}\int f_{i,\btheta_0^{(i)}}(X_{i,0,T})
\pi(\btheta^{(i)}_0)d\btheta_0^{(i)}\right)\right]<0, %=\ell_j-\ell_0<0
\label{eq:INT_appl}
\end{equation}
for $j=1,\cdots,100$. Indeed, it turned out that 
$\frac{1}{15\times 5}\log\left(\prod_{i=1}^{15}\int f_{i,\btheta_0^{(i)}}(X_{i,0,T})
\pi(\btheta^{(i)}_0)d\btheta_0^{(i)}\right)=0.4865$ 
and the maximum of 
$\frac{1}{15\times 5}\log\left(\prod_{i=1}^{15}\int f_{i,\btheta^{(i)}}(X_{i,0,T})
\pi(\btheta^{(i)})d\btheta^{(i)}\right)$ is $0.4127$.
In other words, the Bayes factor consistently selects the correct model even in this situation.


\section{Company-wise data from national stock exchange}
\label{sec:truedata} 

To deal with real data we collect the stock market data 
($467$ observations during the time range August $5$, 2013, to June $30$, 2015) 
for $15$ companies which is available on 
{\it www.nseindia.com}. 
Each company-wise data is modeled by various availabe standard financial $SDE$ 
models with the available ``fitsde" package in $R$. After obtaining the BIC (Bayesian Information Criterion) 
for each company corresponding to each available financial model, we find that the minimum value of BIC corresponds to 
the $CKLS$ model, given, for process $X(t)$, by
$$dX(t)=(\theta_1+\theta_2X(t))dt+\theta_3X(t)^{\theta_4}dW(t).$$
As per our theory we treat the diffusion coefficient as a fixed quantity. 
So, after obtaining the estimated value of the coefficients by the ``fitsde" function, 
we fix the values of $\theta_3$ and $\theta_4$, so that the diffusion coefficient becomes fixed.
We let $\theta_3=A$, $\theta_4=B$.


In this $CKLS$ model, we now wish to include time varying covariates. 
In our work we consider the ``close price" of each company. The stock market data is assumed to 
be dependent on IIP general index, bank interest rate, US dollar exchange rate and on various other quantities. 
But we assume only these three quantities as possibly the most important time dependent covariates. 

Briefly, IIP, that is, index of industrial production, is a measurement which represents 
the status of production in the industrial sector for a given period of time compared to a reference period of time. 
It is one of the best statistical data, which helps us measure the level of industrial activity in Indian economy. 
Its importance lies in the fact that low industrial production  will result in lower corporate sales and profits, 
which will directly affect stock prices. So a direct impact of weak IIP data is a sudden fall in stock prices. 

As the IIP data is purely industrial data, banking sector is not included in it. So, 
we also consider the bank interest rate as another covariate. Note that, higher the bank interest rate, 
fixed deposits become more attractive and one will preferably deposit money in bank rather than invest in stock market. 
Besides, companies with a high amount of loans in their balance sheets would be affected very seriously. 
Interest cost on existing debt would go up affecting their EPS (Earning per Share) and ultimately the stock prices. 
But during low interest rate these companies would stand to gain. Banking sector is likely to benefit most due 
to high interest rates. The Net Interest Margins (it is the difference between the interest they 
earn on the money they lend and the interest they pay to the depositors) for banks is likely to 
increase leading to growth in profits and the stock prices. Hence, it is clear that, the interest rates 
and stock markets are inversely related. As the interest rates go up, stock market 
activities tend to come down. 

Finally, exchange rates directly affect the realized return on 
an investment portfolio with overseas holdings. If one own stock in a foreign company and the 
local currency goes up, the value of the investment also goes up. 
Foreign investment is also related very much to US dollar exchange rate.

Hence, we collect the values of the aforementioned time varying covariates 
during the time range August $5$, 2013, to June $30$, 2015. 
We denote these three quantities by $c_1,c_2,c_3$ respectively. Now, our considered $SDE$ models
for national stock exchange data associated with the $15$ companies are the
following:
\begin{equation}
dX_i(t)=(\theta_1^i+\theta_2^ic_1(t)+\theta_3^ic_2(t)+\theta_4^ic_3(t))(\theta_5^i+\theta_6^iX_i(t))dt+A^iX_i(t)^{B^i}dW_i(t),
\label{eq:sde4_appl}
\end{equation}
for $i=1,\cdots,15$. 
%It is important to mention that we are able to find out $467$ 
%data points which implies our data is observed in [0, 467] i.e. T=467.

\subsection{Selection of covariates by Bayes factor}
\label{subsec:covariate_bf}

Among the considered three time varying covariates we now select the best set of covariate combinations
for the $15$ companies from among $100$ such sets through Bayes factor, 
computing the log-marginal-likelihoods with respect to the normal prior
on the parameter set, assuming {\it a priori} independence of the parameter components 
with individual means being the corresponding $MLE$ (based on simulated annealing)
and unit variances.
Table \ref{table:companies} provides the sets of covariates for the $15$ companies obtained
by our Bayes factor analysis. Also observe that each of the three covariates occurs about $50\%$ times
among the companies, demonstrating that overall impact of these on national stock exchange is
undeniable.
\begin{table}[h]
\centering
\caption{Company-wise covariates obtained by Bayes factor analysis}
\label{table:companies}
\begin{tabular}{|c||c|}
\hline
Company & Covariates\\
\hline
$1$ & Bank rate\\ 
$2$ & US dollar exchange rate\\
$3$ & None\\
$4$ & None\\
$5$ & Bank rate and US dollar exchange rate\\
$6$ & Bank rate and US dollar exchange rate\\
$7$ & IIP general index and US dollar exchange rate\\
$8$ & Bank rate\\
$9$ & IIP general index and Bank rate\\
$10$ & IIP general index\\
$11$ & IIP general index, Bank rate and US dollar exchange rate\\
$12$ & IIP general index and Bank rate\\
$13$ & US dollar exchange rate\\
$14$ & IIP general index, Bank rate and US dollar exchange rate\\
$15$ & IIP general index\\
\hline
\end{tabular}
\end{table}


%For that we have done simulated annealing for the 15 parameter sets. 
%Considering $100$ different covariate combinations for 15 companies we find a certain combination appropriate for 15 companies by Bayes Factor. Note that, our set up is similar to the setup considered in section \ref{subsec:N15T5} except T=467. Hence, according to equation \ref{eq:INT_appl}
%for $j=1,\cdots,100$ the highest value of $\alpha_j$ corresponds to the appropriate combination among 100 combinations. We have shown here three companies among which 2nd company is dependent on IIP and Bank rate. IIP and USD exchange rate are important for 13th company and for 15th company Bank rate and USD exchange rate is important.

\begin{comment}
Here also after obtaining posterior distribution of parameters by TMCMC method we find the posterior predictives of last 47 time point by conditioning on first 420 time points exactly in the same way as described in section \ref{subsec:N1T5} . Figure \ref{fig:pp4} describes the posterior predictives of three companies. In all the cases we calculate $T_{PM}$ and $T_M$ and find that in each case $T_M$ is minimum. Hence ``minimum" will be our required posterior predictives.

\begin{figure}
\begin{subfigure}
\centering
\includegraphics[height=6cm,width=5cm]{Application/C14.pdf}
\label{fig:sfig8_3}
\end{subfigure}
\begin{subfigure}
\centering
\includegraphics[height=6cm,width=5cm]{Application/C50.pdf}
\label{fig:sfig9_3}
\end{subfigure}
\begin{subfigure}
\centering
\includegraphics[height=6cm,width=5cm]{Application/C63.pdf}
\label{fig:sfig10_3}
\end{subfigure}
\caption{Posterior Predictives of 3 Company}
\label{fig:pp4}
\end{figure}

\end{comment}

bf_T.tex
\documentclass[11pt, DIV10,a4paper]{article}
\usepackage{natbib}
\newcommand {\ctn}{\citet} % change to \citet if using natbib
\newcommand {\ctp}{\cite}       % change to \citep if using natbib
\usepackage{graphicx,subfigure,amsmath,latexsym,amssymb}
\usepackage{float,epsfig,multirow,rotating,times}
\usepackage{upgreek,wrapfig}
\usepackage{comment}
\newcommand{\abs}[1]{|#1|}
\newcommand{\norm}[1]{\Vert#1\Vert}
\newcommand{\bell}{\boldsymbol{\ell}}
\newcommand{\ba}{\boldsymbol{a}}
\newcommand{\bb}{\boldsymbol{b}}
\newcommand{\bd}{\boldsymbol{d}}
\newcommand{\bg}{\boldsymbol{g}}
\newcommand{\bof}{\boldsymbol{f}}
\newcommand{\boeta}{\boldsymbol{\eta}}
\newcommand{\blambda}{\boldsymbol{\lambda}}
\newcommand{\balpha}{\boldsymbol{\alpha}}
\newcommand{\btheta}{\boldsymbol{\theta}}
\newcommand{\bLambda}{\boldsymbol{\Lambda}}
\newcommand{\bDelta}{\boldsymbol{\Delta}}
\newcommand{\bkappa}{\boldsymbol{\kappa}}
\newcommand{\bbeta}{\boldsymbol{\beta}}
\newcommand{\bdelta}{\boldsymbol{\delta}}
\newcommand{\bphi}{\boldsymbol{\phi}}
\newcommand{\bPhi}{\boldsymbol{\Phi}}
\newcommand{\bPi}{\boldsymbol{\Pi}}
\newcommand{\bChii}{\boldsymbol{\Chi}}
\newcommand{\bXi}{\boldsymbol{\Xi}}
\newcommand{\bxi}{\boldsymbol{\xi}}
\newcommand{\bGamma}{\boldsymbol{\Gamma}}
\newcommand{\bTheta}{\boldsymbol{\Theta}}
\newcommand{\bgamma}{\boldsymbol{\gamma}}
\newcommand{\bSigma}{\boldsymbol{\Sigma}}
\newcommand{\bPsi}{\boldsymbol{\Psi}}
\newcommand{\bpsi}{\boldsymbol{\psi}}
\newcommand{\bpi}{\boldsymbol{\pi}}
\newcommand{\bepsilon}{\boldsymbol{\epsilon}}
\newcommand{\bUpsilon}{\boldsymbol{\Upsilon}}
\newcommand{\bmu}{\boldsymbol{\mu}}
\newcommand{\bnu}{\boldsymbol{\nu}}
\newcommand{\btau}{\boldsymbol{\tau}}
\newcommand{\bzeta}{\boldsymbol{\zeta}}
\newcommand{\bvarrho}{\boldsymbol{\varrho}}
\newcommand{\bvarphi}{\boldsymbol{\varphi}}
\newcommand{\bvartheta}{\boldsymbol{\vartheta}}
\newcommand{\bOmega}{\boldsymbol{\Omega}}
\newcommand{\bomega}{\boldsymbol{\omega}}
\newcommand{\prm}{^{\prime}}
\newcommand{\bB}{\boldsymbol{B}}
\newcommand{\bC}{\boldsymbol{C}}
\newcommand{\bc}{\boldsymbol{c}}
\newcommand{\bD}{\boldsymbol{D}}
\newcommand{\bff}{\boldsymbol{f}}
\newcommand {\bF}{\textbf{F}}
\newcommand{\bV}{\boldsymbol{V}}
\newcommand{\bG}{\boldsymbol{G}}
\newcommand{\bh}{\boldsymbol{h}}
\newcommand{\bH}{\boldsymbol{H}}
\newcommand{\bmm}{\boldsymbol{m}}
%\newcommand{\bM}{\boldsymbol{M}}
\newcommand{\bI}{\boldsymbol{I}}
\newcommand{\bJ}{\boldsymbol{J}}
\newcommand{\bK}{\boldsymbol{K}}
\newcommand{\bk}{\boldsymbol{k}}
\newcommand{\bA}{\boldsymbol{A}}
\newcommand{\bS}{\boldsymbol{S}}
\newcommand{\bt}{\boldsymbol{t}}
\newcommand{\bT}{\boldsymbol{T}}
\newcommand{\bE}{\boldsymbol{E}}
\newcommand{\bp}{\boldsymbol{p}}
\newcommand{\bP}{\boldsymbol{P}}
\newcommand{\bq}{\boldsymbol{q}}
\newcommand{\bQ}{\boldsymbol{Q}}
\newcommand{\bR}{\boldsymbol{R}}
\newcommand{\bW}{\boldsymbol{W}}
\newcommand{\bu}{\boldsymbol{u}}
\newcommand{\bv}{\boldsymbol{v}}
\newcommand{\bs}{\boldsymbol{s}}
\newcommand{\bU}{\boldsymbol{U}}
\newcommand{\bx}{\bm{x}}
\newcommand{\bX}{\boldsymbol{X}}
\newcommand{\by}{\boldsymbol{y}}
\newcommand{\bY}{\boldsymbol{Y}}
\newcommand{\bZ}{\boldsymbol{Z}}
\newcommand{\bz}{\boldsymbol{z}}
\newcommand{\blda}{\boldsymbol{a}}
\newcommand{\bOO}{\boldsymbol{O}}
\newcommand{\bw}{\boldsymbol{w}}
\newcommand{\bee}{\boldsymbol{e}}
\newcommand{\bzero}{\boldsymbol{0}}
\newcommand{\lsb}{\bigl\{}
\newcommand{\rsb}{\bigr\}}
\newcommand{\bj}{{\boldsymbol{j}}}
\newcommand{\bone}{\boldsymbol{1}}
\newcommand{\one}{$\phantom{1}$}
\newcommand{\oo}{$\phantom{00}$}
\newcommand{\ooo}{$\phantom{000}$}
\newcommand{\oc}{$\phantom{0,}$}
\newcommand{\ooc}{$\phantom{00,}$}
\newcommand{\oooc}{$\phantom{000,}$}
\newcommand{\Ncand}{N_{\rm cand}}
\newcommand{\tover}{t_{\rm over}}
\newcommand{\tstd}{t_{\rm std,1}}
\newcommand{\ttwostage}{t_{\rm 2S,1}}
\newcommand{\psp}{P(S')}
\newcommand{\pep}{P(E')}
\newcommand{\pacc}{P_{\rm acc}}
\newcommand{\prob}{\mbox{I}\!\mbox{Pr}}
\DeclareMathOperator*{\argmin}{argmin}

\newtheorem{theorem}{Theorem}
\newtheorem{acknowledgement}[theorem]{Acknowledgement}
\newtheorem{algorithm}[theorem]{Algorithm}
\newtheorem{axiom}[theorem]{Axiom}
\newtheorem{case}[theorem]{Case}
\newtheorem{claim}[theorem]{Claim}
\newtheorem{conclusion}[theorem]{Conclusion}
\newtheorem{condition}[theorem]{Condition}
\newtheorem{conjecture}[theorem]{Conjecture}
\newtheorem{corollary}[theorem]{Corollary}
\newtheorem{criterion}[theorem]{Criterion}
\newtheorem{definition}[theorem]{Definition}
\newtheorem{example}[theorem]{Example}
\newtheorem{examples}[theorem]{Examples}
\newtheorem{exercise}[theorem]{Exercise}
\newtheorem{lemma}[theorem]{Lemma}
\newtheorem{notation}[theorem]{Notation}
\newtheorem{problem}[theorem]{Problem}
\newtheorem{proposition}[theorem]{Proposition}
\newtheorem{remark}[theorem]{Remark}
\newtheorem{solution}[theorem]{Solution}
\newtheorem{summary}[theorem]{Summary}

\newtheorem{thm}{\bf Theorem}[section]
\newtheorem{lem}{\bf Lemma}[section]
\newtheorem{rmk}{\bf Remark}[section]
\newtheorem{cor}{\bf Corollary}[section]
\newtheorem{exa}{\bf Example}[section]
\newtheorem{dfn}{\bf Definition}[section]

\newenvironment{proof}[1][Proof]{\textbf{#1.} }{\ \rule{0.5em}{0.5em}}

\newcommand{\C}{\mbox{\boldmath{$\cal C$}}}
\newcommand{\boldcal}[1]{\mbox{\boldmath{$\mathcal #1 $}}}
\newcommand{\statesp}{\ensuremath{\mathcal X}}
\newcommand{\Y}{\ensuremath{\mathcal Y}}
\newcommand{\D}{\ensuremath{\mathcal D}}
\newcommand{\union}{\cup}
\newcommand{\intersect}{\cap}
\newcommand{\imply}{\Longrightarrow}
\newcommand{\R}{\ensuremath{\mathbb R}}
\newcommand{\e}{\ensuremath{\epsilon}}

\newcommand{\supr}[2]{{#1}^{(#2)}}
\newcommand{\bM}{\mathbf M}
\newcommand{\be}{\boldsymbol\e}
\newcommand{\bm}{\mathbf}
\renewcommand{\t}{\ensuremath{\theta}}

\newcommand{\topline}{\hrule height 1pt width \textwidth \vspace*{2pt}}
\newcommand{\botline}{\vspace*{2pt}\hrule height 1pt width \textwidth \vspace*{4pt}}
\newtheorem{algo}{Algorithm} 

\numberwithin{equation}{section}
\numberwithin{algo}{section}
\numberwithin{table}{section}
\numberwithin{figure}{section}
\newcommand{\boldm}[1]{\mbox{\boldmath{$ #1 $}}}
\newcommand{\bi}[1]{\mbox{\boldmath{$ #1 $}}}

\usepackage{setspace}
\usepackage[mathscr]{euscript}
\usepackage[margin=1in]{geometry}
\singlespacing


\begin{document}
\normalsize

\title{\vspace{-0.8in}
On Convergence of Bayes Factor in Stochastic Differential Equations: Part II}
\author{Trisha Maitra and Sourabh Bhattacharya\thanks{
Trisha Maitra is a PhD student and Sourabh Bhattacharya 
is an Associate Professor in
Interdisciplinary Statistical Research Unit, Indian Statistical
Institute, 203, B. T. Road, Kolkata 700108.
Corresponding e-mail: sourabh@isical.ac.in.}}
\date{\vspace{-0.5in}}
\maketitle%


\begin{abstract}

The problem of model selection in the context of a system of stochastic differential equations ($SDE$'s) 
has not been touched upon in the literature. Indeed, properties of Bayes factors have not been studied even 
in single $SDE$ based model comparison problems.

In this article, we first develop an asymptotic theory of Bayes factors when two $SDE$'s are compared, 
assuming the time domain expands. Using this we then develop an asymptotic theory of Bayes factors 
when systems of $SDE$'s are compared, 
assuming that the number of equations in each system, as
well as the time domain, increase indefinitely. 
Our asymptotic theory covers situations when the observed processes associated with the
$SDE$'s are independently and identically distributed ($iid$), as well as when they are independently 
but not identically distributed (non-$iid$). Quite importantly, we allow inclusion of available time-dependent 
covariate information into each $SDE$ through a multiplicative factor of the drift function in a random effects set-up; 
different initial values for the $SDE$'s are also permitted.

Thus, our general model-selection framework includes simultaneously the variable selection problem associated with
time-varying covariates, as well as choice of the part of the drift function free of covariates.
It is to be noted that given that the underlying process is wholly observed, the diffusion coefficient becomes known,
and hence is not involved in the model selection problem.

For both $iid$ and non-$iid$ set-ups we establish almost sure exponential convergence
of the Bayes factor. As we show, the Bayes factor is inconsistent for comparing individual $SDE$'s,
in the sense that the log-Bayes factor converges only in expectation, while
the relevant variance does not converge to zero.
Nevertheless, %in spite of the seemingly negative outcome, 
it has been possible to exploit this result to establish
almost sure exponential convergence of the Bayes factor when, in addition, the number of individuals 
are also allowed to increase
indefinitely.

We carry out simulated and real data analyses to demonstrate that Bayes factor is a suitable candidate
for covariate selection in our $SDE$ models even in non-asymptotic situations.
\\[2mm]
{\it {\bf Keywords:} Bayes factor consistency; Kullback-Leibler divergence; Martingale; 
Stochastic differential equations; Time-dependent covariates and random effects; Variable selection.}
 
\end{abstract}


\section{Introduction}
\label{sec:intro}

Stochastic differential equations ($SDE$'s) have important standing in statistical applications 
where ``within" subject variability is caused by some random component
varying continuously in time. It also seems worthwhile to incorporate available time-dependent covariate
information into the subject-wise $SDE$'s.
Apart from the covariates there may also be random effects
associated with the individuals, which may be useful in modeling variabilities between the individuals.
%Such collections of $SDE$'s, consisting of the covariates, which are associated with the fixed effects, as well as the
%random effects, may be termed as mixed effects $SDE$ models.

$SDE$-based models with time-dependent covariates are considered in
\ctn{Zita11}, \ctn{Overgaard05}, \ctn{Leander15}; moreover, \ctn{Zita11} analyse their covariate-based $SDE$ model in 
the hierarchical Bayesian paradigm. In the literature, random effects $SDE$ models without covariates seem to be 
more popular than those based on covariates. 
%These are often good candidates for modeling continuous time biological processes  
%with repeated measurements on individuals. 
%In this regard, the pharmacokinetic/pharmaco-dynamic field
%is a fertile area for application of models of the above type;
%see, for example, 
%\ctn{Dit05}, \ctn{Overgaard05}, \ctn{Picchini10}, \ctn{Picchini11}, \ctn{Yan14}, \ctn{Leander15}
%A comprehensive overview of various techniques of parameter estimation in these contexts is provided by \ctn{Donnet13}. 
A brief overview of random effects $SDE$ models is provided in
\ctn{Maud12} who undertake theoretical and classical asymptotic 
investigation of a class of
random effects models based on $SDE$'s. Specifically, they model the $i$-th individual
by
\begin{equation}
d X_i(t)=b(X_i(t),\phi_i)dt+\sigma(X_i(t))dW_i(t),%\quad\mbox{with}\quad X_i(0)=x^i,~i=1,\ldots,n,
\label{eq:sde_basic1}
\end{equation}
where, for $i=1,\ldots,n$, $X_i(0)=x^i$ is the initial value of the stochastic process $X_i(t)$, which 
is assumed to be continuously observed on the time interval $[0,T_i]$; $T_i>0$ assumed to be known.
The function $b(x,\varphi)$, which is the drift function, is a known, real-valued function on $\mathbb R\times\mathbb R^d$ 
($\mathbb R$ is the real line and $d$ is the dimension), and the function $\sigma:\mathbb R\mapsto\mathbb R$ is the known 
diffusion coefficient.
The $SDE$'s given by (\ref{eq:sde_basic1}) are driven by independent standard Wiener processes $\{W_i(\cdot);~i=1,\ldots,n\}$, 
and $\{\phi_i;~i=1,\ldots,n\}$, which are to be interpreted as the random effect parameters associated
with the $n$ individuals, which are assumed by \ctn{Maud12} to be independent of the Brownian motions and
independently and identically distributed ($iid$) random variables with some common distribution.
For the sake of convenience 
\ctn{Maud12} (see also \ctn{Maitra14a} and \ctn{Maitra14b}) assume $b(x,\phi_i)=\phi_ib(x)$. 
Thus, the random effect is a multiplicative factor of the drift function. In this work,
we generalize this to a random effects $SDE$ set-up consisting of time-dependent covariates. 
%Assuming $\phi_i$ to be 
%Gaussian random variables with mean $\mu$ and variance $\omega^2$, \ctn{Maud12} obtained
%a closed form of the likelihood of $(\mu,\omega^2)$ and proved weak convergence in probability
%and asymptotic normality of the maximum likelihood estimator of $(\mu,\omega^2)$ under the $iid$ set-up, that is, when 
%$x^i=x$ and $T_i=T$, for $i=1,\ldots,n$. \ctn{Maitra14a} proved strong consistency
%and asymptotic normality of the maximum likelihood estimator under weaker assumptions in the
%$iid$ set-up. They also proved weak convergence and asymptotic normality under the independent but
%non-identical set-up, where $(x_i,T_i)$ are allowed to differ for each $i$. 
%\ctn{Maitra14a} refer to this as the non-$iid$ setup. 
%\ctn{Maitra14b} investigated Bayesian asymptotics in the $iid$ and non-$iid$ set-ups, as a follow-up of
%\ctn{Maitra14a}.



Note that model selection constitutes an important part of research in both Bayesian and 
classical paradigms; see, for example, 
\ctn{Dey00}, \ctn{Jiang07}, \ctn{Claeskens08}, \ctn{Muller13}.
In the case of $SDE$-based mixed effects models as well, model selection constitutes an important issue
involving the choice of the drift function and selection of the appropriate subset of (time-dependent) covariates.
Here Bayes factors are expected to play the central role as their effectiveness in model selection
in complex problems is well-established (see, for example, \ctn{Kass95} for a good account of Bayes factors).
Unavailability of closed form expressions in the traditional $SDE$ set-ups usually
prompt usage of numerical approximations based on Markov chain Monte Carlo or 
related criteria such as the Akaike Information Criterion (\ctn{Akaike73}) 
and Bayes Information Criterion (\ctn{Schwarz78}). For details, see, for example, \ctn{Fuchs13}, \ctn{Iacus08}.
But we are not aware of any research existing in the literature that attempts to address covariate selection in $SDE$'s.


We are also not aware of any existing literature on asymptotic investigation of Bayes factors in the $SDE$ context 
%except \ctn{Maitra15a}, 
although \ctn{Siva02} present some asymptotic investigation of intrinsic
and fractional Bayes factors in the context of three specific diffusion models.
The only investigation available in this context seems to be 
that of \ctn{Maitra15a}, who model a multiplicative part of the drift function using time-varying covariates, and
address Bayes factor asymptotics in a general set-up consisting of the 
covariate selection problem as well as selection of 
the part of the drift function independent of the covariates. Different initial values
and domains of observations pertaining to different individuals, are also considered in their set-up.
Assuming that only the number of individuals increase without bound, \ctn{Maitra15a} establish
almost sure exponential convergence of Bayes factor in both $iid$ and non-$iid$ situations.
Here we recall that the $iid$ set-up is the case when there is no covariate associated with 
the model and when the initial values and the domains of observations
are the same for every individual. 
The non-$iid$ set-up, on the other hand, consists of time-varying covariates, different  
initial values and domains of observations; in this work we also consider random effects. 
Thus, unlike the $iid$ case, here the model selection problem also deals with covariate selection 
apart from selection of the part of the drift functions free of the covariates.


In this article, we prove almost sure exponential convergence of the relevant Bayes 
factors in both $iid$ and non-$iid$ cases, assuming that the number
of individuals, as well as the domains of observations, increase without bound.
Hence, for our current purpose, the asymptotic theory developed by \ctn{Maitra15a} when only the number of individuals
tends to infinity, is clearly inapplicable.
Indeed, incorporation of random effects is asymptotically feasible only in our current asymptotic framework;
\ctn{Maitra15a} elucidate that inclusion of random effects does not make sense asymptotically unless the domains 
of observations are also increased indefinitely.
Also, only our current asymptotic framework allows different sets of time-dependent covariates for different individuals.

It is important to remark that the diffusion coefficient becomes known once the continuous process is completely
observed; see \ctn{Robert01}. Hence, following \ctn{Maitra15a} we assume that the diffusion coefficient is known,
and is not involved in the model selection problem.

We begin by establishing an asymptotic theory of Bayes factor for two
competing individual $SDE$'s, and then extend the theory to systems of $SDE$'s. In this context it 
is important to draw attention
to the fact that even this relatively simple problem of comparing any two individual $SDE$'s using Bayes factors
has not yet been considered in the literature.
Our investigation in this simpler case, however, faced with an apparently negative result; 
the associated Bayes factor failed to be consistent in the sense that the relevant variance failed
to converge to zero, even though convergence of the log-Bayes factor in expectation is ensured.
Despite this, we have been able to utilise this result to establish almost sure exponential convergence of the
Bayes factor when the number of individuals are also allowed to increase indefinitely.


The rest of our article is structured as follows. 
We begin with formalization of our set-up in Section \ref{sec:bf_sde}, while
we provide the necessary assumptions and results in Section \ref{sec:case2}. 
In Section \ref{sec:result_BF_consistency}
we investigate the asymptotics of Bayes factor for comparing two individual $SDE$'s.
We illustrate our results with a special case in Section \ref{sec:illustration}.
In Section \ref{sec:asymp_BF_nT} we exploit the asymptotic theory of Bayes factors developed for
comparing individual $SDE$'s to construct a convergence theory of Bayes factors comparing
systems of $SDE$'s in both $iid$ and non-$iid$ cases. 
In Section \ref{sec:simulated_data} we carry out two simulation studies to demonstrate
that Bayes factor yields the correct set of covariates in our $SDE$ models even in non-asymptotic
cases, and in Section \ref{sec:truedata}, we model a real, company-wise national stock exchange data set,
using a system of $SDE$'s, each consisting of a plausible set of covariates,
and obtain the best possible sets of covariate combinations
for the companies, using Bayes factor.
We summarize our contributions and provide concluding remarks in Section \ref{sec:conclusion}.



\section{Formalization of the model selection problem in the $SDE$ set-up when
$n\rightarrow\infty$ and $T_i\rightarrow\infty$ for every $i$}
\label{sec:bf_sde}

That the systems considered by us are well-defined and the exact likelihoods are computable, are guaranteed by
assumption (H2$^{\prime\prime}$) in Section \ref{sec:case2}. 
For our purpose we consider the filtration ($\mathcal F_t^W,t\geq 0$),
where $\mathcal F_t^W=\sigma(W_i(s),s\leq t)$. Each process $W_i$ is a $(\mathcal F_t^W , t\geq 0)$-adapted Brownian
motion.


Here we consider the set-up where, for $i=1,2,\ldots,n$,
\begin{equation}
d X_i(t)=\phi_{i,\bxi^{(i)}_0}(t)b_{\bbeta^{(i)}_{0}}(t,X_i(t))dt+\sigma(t,X_i(t))dW_i(t)
\label{eq:sde3}
\end{equation}
and
\begin{equation}
d X_i(t)=\phi_{i,\bxi^{(i)}_1}(t)b_{\bbeta^{(i)}_{1}}(t,X_i(t))dt+\sigma(t,X_i(t))dW_i(t),
\label{eq:sde4}
\end{equation}
where, $X_i(0)=x^i$ is the initial value of the stochastic process $X_i(t)$, 
which is assumed to be continuously observed on the time interval $[0,T_i]$; $T_i>0$.
We consider (\ref{eq:sde3}) as representing the true model and (\ref{eq:sde4}) is any other model.

It is useful to remark that we must analyze the same data set with respect to two different models for the purpose
of model selection. Hence, even though the distribution of the underlying stochastic process under the two models are different, 
for notational convenience we denote the process by $X_i(t)$ under both the models, relying on the context 
and the model-specific parameters to naturally clarify the distinction.


%Our main aim is to study the consistency of Bayes factor related to models corresponding to two 
%different distributions when $T_i\rightarrow\infty$ and $n\rightarrow\infty$ .


\subsection{Inclusion of time-dependent covariates}
\label{subsec:covariates}
We model $\phi_{i,\bxi^{(i)}_j}(t)$ for $j=0,1$, and $i=1,\ldots,n$, as  
\begin{equation}
\phi_{i,\bxi^{(i)}_j}(t)=\phi_{i,\bxi^{(i)}_j}(\bz_i(t))
=\xi^{(i)}_{0j}+\xi^{(i)}_{1j}g_1(z_{i1}(t))+\xi^{(i)}_{2j}g_2(z_{i2}(t))+\cdots+\xi^{(i)}_{pj}g_p(z_{ip}(t)),
\label{eq:phi_model}
\end{equation}
where %$\bxi_j=(\xi_{0j},\xi_{1j},\ldots,\xi_{pj})$ is a set of real constants for $j=0,1$, 
$\bz_i(t)=(z_{i1}(t),z_{i2}(t),\ldots,z_{ip}(t))$ is the set of available covariate information 
corresponding to the $i$-th individual, depending upon time $t$. Following \ctn{Maitra15a} 
we assume $\bz_i(t)$ is continuous in $t$,
$z_{il}(t)\in \bZ_l$  where $\bZ_l$ is  compact and 
$g_l:\bZ_l\rightarrow \mathbb R$ is  continuous, for $l=1,\ldots,p$.
We let $\boldcal Z=\bZ_1\times\cdots\times\bZ_p$, and 
$\mathfrak Z=\left\{\bz(t)\in\boldcal Z:t\in[0,\infty)~\mbox{such that}~\bz(t)~\mbox{is continuous in}~t\right\}$.
Hence, $\bz_i\in\boldcal Z$ for all $i$.
%The functions $b_{\bbeta_j}$ are multiplicative parts of the drift functions free of the covariates.

\subsection{The random effects set-up}
\label{subsec:random_effects_2}

In (\ref{eq:sde3}), $\btheta^{(i)}_{0}=\left(\bbeta^{(i)}_{0},\xi^{(i)}_{00},\xi^{(i)}_{10},\ldots,\xi^{(i)}_{p0}
\right)=\left(\bbeta^{(i)}_{0},\bxi_0^{(i)}\right)$ stands for 
the true parameters, and  
$\btheta^{(i)}_{1}=\left(\bbeta^{(i)}_{1},\xi^{(i)}_{01},\xi^{(i)}_{11},\ldots,\xi^{(i)}_{p1}\right)
=\left(\bbeta^{(i)}_{1},\bxi_1^{(i)}\right)$ are
the parameters associated with (\ref{eq:sde4}). 
Let $\btheta^{(i)}_j\in\bTheta=\mathfrak B\times\bGamma$ for all $i$,
where both $\mathfrak B$ and $\bGamma$ are compact spaces. 
We also assume that for $i=1,2,\ldots,n$,
$$\btheta^{(i)}_{1}\stackrel{iid}{\sim}\pi,$$
where $\pi$ is some specified distribution on $\bTheta$.

Hence, the above describes a random effects set-up.
Observe that if $\xi^{(i)}_{lj}=0$ for $l=1,\ldots,p$, and for $i=1,\ldots,n$, then it reduces
to the random effects model of \ctn{Maud12}, showing that the latter is a special case of our model.

As is well-known, even though the term ``prior" is not appropiate
for the random effects coefficients, operationally there is no difference between
a prior and a distribution for random effects in the Bayesian paradigm. Somewhat abusing the terminology, we continue
to refer to the distribution of the $iid$ random effects coeffcients, $\pi$, as the relevant prior. 


\subsection{Covariate and drift function selection}
\label{subsec:model_selection_drift_covariates}

The key difference between our current model selection idea and that of \ctn{Maitra15a} is that
here, for every individual, there is an independent model selection problem. In other words, for each $i$,
one needs to choose between $\btheta^{(i)}_0$ and $\btheta^{(i)}_1$. This involves selection of
perhaps different sets of covariates for different $i$ with respect to the coefficients
$\bxi^{(i)}_j$, and different drift functions $b_{\bbeta^{(i)}_j}$. 
Obviously, the dimensions of $\bxi^{(i)}_0$ and $\bxi^{(i)}_1$ are allowed to differ for each $i$;
likewise, for every $i$, the dimensions of $\bbeta^{(i)}_0$ and $\bbeta^{(i)}_1$ may be different as well.
Thus, from this perspective, our current model selection framework appears to be more general compared to that 
of \ctn{Maitra15a}, who consider
the same set of parameters $\bxi_j$ and $\bbeta_j$ for all the individuals, allowing only a fixed set of covariates
for every subject.


%\subsection{Incorporation of random effects}
%\label{subsec:random_effects_2}
%Unlike \ctn{Maitra15a}, here we can assume that some of the covariates, say, 
%$\left\{z_{il_1},z_{il_2},\ldots,z_{il_r}\right\}$ are associated
%with random effects $\left\{\xi^{(i)}_{l_1j},\xi^{(i)}_{l_2j},\ldots,\xi^{(i)}_{l_rj}\right\}$,
%where $1\leq l_1,l_2,\ldots,l_r\leq p$. 
%In any case, observe that if $\xi^{(i)}_{lj}=0$ for $l=1,\ldots,p$, and for $i=1,\ldots,n$, then it reduces
%to the random effects model of \ctn{Maud12}, showing that the latter is a special case of our model.
%
%As is well-known, even though the term ``prior" is not appropiate
%for the random effects coefficients, operationally there is no difference between
%random and non-random effects in our Bayesian framework. Somewhat abusing the terminology, we continue
%to refer to the distribution of the $iid$ random effects coeffcients as the relevant prior. 

\subsection{Form of the Bayes factor in our set-up}
\label{subsec:bf_form}
For $j=0,1$, we
first define the following quantities:
\begin{equation}
U_{i,\btheta^{(i)}_j} =\int_0^{T_i}\frac{\phi_{i,\bxi^{(i)}_j}(s)b_{\bbeta^{(i)}_j}(s,X_i(s))}{\sigma^2(s,X_i(s))}dX_i(s),
\quad\quad V_{i,\btheta^{(i)}_j} =\int_0^{T_i}\frac{\phi^2_{i,\bxi^{(i)}_j}(s)b_{\bbeta^{(i)}_j}^2(s,X_i(s))}
{\sigma^2(s,X_i(s))}ds
\label{eq:u_v}
\end{equation}
for $j=0,1$ and $i=1,\ldots,n$. 

Let $\bC_{T_i}$ denote the space of real continuous functions $(x(t), t \in [0,T_i ])$ defined on $[0,T_i ]$, 
endowed with the $\sigma$-field $\mathcal C_{T_i}$ associated with the topology of uniform convergence 
on $[0,T_i ]$. We consider the distribution $P^{x_i,T_i,\bz_i}_j$ on $(C_{T_i} ,\mathcal C_{T_i})$ 
of $(X_i (t), t\in [0,T_i])$ given by (\ref{eq:sde3}) and (\ref{eq:sde4}) for $j=0,1$. 
We choose the dominating measure $P_i $ as the distribution of (\ref{eq:sde3}) and (\ref{eq:sde4}) with null
drift. So, for $j=0,1$, 
\begin{align}
\frac{dP^{x_i,T_i,\bz_i}_j}{dP_i}=f_{i,\btheta^{(i)}_j}(X_i)
&=\exp\left(U_{i,\btheta^{(i)}_j}-\frac{V_{i,\btheta^{(i)}_j}}{2}\right), 
\label{eq:densities}
\end{align}
where $f_{i,\btheta^{(i)}_0}(X_i)$ denotes the true density and $f_{i,\btheta^{(i)}_1}(X_i)$ stands for the other density
associated with the modeled $SDE$.


For each $i=1,\ldots,n$, letting $X_{i,a,b}$ denote the $i$-th process 
observed on $[a,b]$ for any $0\leq a<b<\infty$,
\begin{equation}
I_{x^i,T_i,\bz_i}=\int_{\bTheta} \frac{f_{i,\btheta^{(i)}_1}(X_{i,0,T_i})}
{f_{i,\btheta^{(i)}_0}(X_{i,0,T_i})}\pi\left(\btheta^{(i)}_{1}\right)d\btheta^{(i)}_{1}
\label{eq:I_iT}
\end{equation}
denotes the Bayes factor associated with the $i$-th equation of the above two systems of equations. 
Assuming that the $SDE$'s (\ref{eq:sde3}) and (\ref{eq:sde4}) are independent for $i=1,\ldots,n$, 
\begin{equation*}
I_{n,T_1,\ldots,T_n}=\prod_{i=1}^nI_{x^i,T_i,\bz_i}
\label{eq:I_nT}
\end{equation*}
is the Bayes factor comparing the entire systems of $SDE$'s (\ref{eq:sde3}) and (\ref{eq:sde4}). 

Comparisons between a collection of different models using Bayes factor, none of which may be the true model, is 
expected to favour that model which minimizes the Kullback-Leibler
divergence from the true model. 


\subsection{The $iid$ and the non-$iid$ cases}
\label{subsec:iid_non_iid_2}
We are interested in studying the properties of $I_{n,T_1,\ldots,T_n}$ in both $iid$ and non-$iid$ cases
when $n\rightarrow\infty$ and $T_i\rightarrow\infty$. 
In the $iid$ set-up, we assume that $x^i=x$, $T_i=T$ and $\btheta^{(i)}_j=\left(\bbeta_j^{(i)},\xi^{(i)}_{0j}\right)$, 
for $i=1,\ldots,n$ and $j=0,1$. In the non-$iid$ case we relax these assumptions.
However, for simplicity, we assume
$T_i=T$ for each $i$, even in the non-$iid$ set-up, so that in our asymptotic framework we study convergence of 
\begin{equation}
\tilde I_{n,T}=\prod_{i=1}^nI_{i,T},
\label{eq:tilde_I_nT}
\end{equation}
as $n\rightarrow\infty$ and $T\rightarrow\infty$, where $I_{i,T}=I_{x^i,T,\bz_i}$.



\subsection{A key relation between $U_{i,\btheta^{(i)}_j}$ and $V_{i,\btheta^{(i)}_j}$ in the context
of model selection using Bayes factors}
\label{subsec:key_relation_U_V}
An useful relation between $U_{i,\btheta^{(i)}_j}$ and $V_{i,\btheta^{(i)}_j}$ which we will often make use of
in this paper is as follows.
\begin{align}
U_{i,\btheta^{(i)}_j}&=\int_0^{T_i}\frac{\phi_{i,\bxi^{(i)}_j}(s)b_{\bbeta^{(i)}_j}\left(X_i(s)\right)}
{\sigma^2\left(X_i(s)\right)}dX_i(s)\notag\\
&=\int_0^{T_i}\frac{\phi_{i,\bxi^{(i)}_j}(s)b_{\bbeta^{(i)}_j}\left(X_i(s)\right)}{\sigma^2\left(X_i(s)\right)}
\left[\phi_{i,\bxi^{(i)}_0}(s)b_{\bbeta^{(i)}_0}\left(X_i(s)\right)ds+\sigma\left(X_i(s)\right)dW_i(s)\right]\notag\\
&=\int_0^{T_i}\frac{\phi_{i,\bxi^{(i)}_j}(s)\phi_{i,\bxi^{(i)}_0}(s)b_{\bbeta^{(i)}_j}
\left(X_i(s)\right)b_{\bbeta^{(i)}_0}\left(X_i(s)\right)}
{\sigma^2\left(X_i(s)\right)}ds
+\int_0^{T_i}\frac{\phi_{i,\bxi^{(i)}_j}(s)b_{\bbeta^{(i)}_j}\left(X_i(s)\right)}
{\sigma\left(X_i(s)\right)}dW_i(s)\notag\\
&=V_{i,\btheta^{(i)}_0,\btheta^{(i)}_j}
+\int_0^{T_i}\frac{\phi_{i,\bxi^{(i)}_j}(s)b_{\bbeta^{(i)}_j}\left(X_i(s)\right)}
{\sigma\left(X_i(s)\right)}dW_i(s),
\label{eq:u_v_relation}
\end{align}
with
\begin{equation}
V_{i,\btheta^{(i)}_0,\btheta^{(i)}_j}=
\int_0^{T_i}\frac{\phi_{i,\bxi^{(i)}_j}(s)\phi_{i,\bxi^{(i)}_0}(s)b_{\bbeta^{(i)}_j}
\left(X_i(s)\right)b_{\bbeta^{(i)}_0}
\left(X_i(s)\right)}{\sigma^2\left(X_i(s)\right)}ds.
\label{eq:V_0_j}
\end{equation}
Note that $V_{i,\btheta^{(i)}_0}=V_{i,\btheta^{(i)}_0,\btheta^{(i)}_0}$ and 
$V_{i,\btheta^{(i)}_1}=V_{i,\btheta^{(i)}_1,\btheta^{(i)}_1}$.
Also note that, for $j=0,1$, for each $i$,
\begin{equation}
E_{\btheta^{(i)}_0}\left[\int_0^{T_i}\frac{\phi_{i,\bxi^{(i)}_j}(s)b_{\bbeta^{(i)}_j}\left(X_i(s)\right)}
{\sigma\left(X_i(s)\right)}dW_i(s)\right]=0,
\label{eq:zero_mean}
\end{equation}
so that $E_{\btheta^{(i)}_0}\left(U_{i,\btheta^{(i)}_j}\right)
=E_{\btheta^{(i)}_0}\left(V_{i,\btheta^{(i)}_0,\btheta^{(i)}_j}\right)$.


%We now turn towards the case where the number of individuals as well as the time domains
%increase indefinitely. Recall from Section \ref{subsec:case2} that we assume $T_i=T$ for every $i$,
%where $T\rightarrow\infty$. In other words, we are interested in the asymptotic framework where
%$n\rightarrow\infty$ and $T\rightarrow\infty$.

\section{Requisite assumptions and results for the asymptotic theory of Bayes factor 
when $n\rightarrow\infty$ and $T\rightarrow\infty$}
\label{sec:case2}

%Although some of the assumptions we make here are similar to that assumed in \ctn{Maitra15a}
%%Section \ref{sec:case1} 
%(and the compactness assumption regarding the parameter space remains intact),
%they are different in details. For instance, unlike \ctn{Maitra15a}, here we allow
%$b_{\bbeta_j}$ and $\sigma$ to depend on time, and this creates differences in the details
%of the assumptions. Moreover, some extra assumptions are also needed here, for instance, with respect
%to the time-varying covariates. 
%
%All our assumptions and results are true for each $i$, in particular true for each 
%$\bbeta_j^{(i)},\bxi_j^{(i)}$ and consequently for $U_{i,\btheta_j}, 
%V_{i,\btheta_j},V_{i,\btheta_0,\btheta_j}$. For the sake of notational simplicity 
%we provide all the assumptions and results without mentioning $i$ at every stage. 

%Below, we provide our
%assumptions in this asymptotic framework afresh.
All our following assumptions and results are true for each $i$, in particular true for each 
$\bbeta_j^{(i)},\bxi_j^{(i)}$ and consequently for $U_{i,\btheta_j^{(i)}}, 
V_{i,\btheta_j^{(i)}},V_{i,\btheta_0^{(i)},\btheta_j^{(i)}}$. For the sake of notational simplicity 
we provide all the assumptions and results without mentioning $i$ at every stage. 
We make the following assumptions:
\begin{itemize}
\item[(H1$^{\prime\prime}$)] The parameter space $\bTheta=\mathfrak B\times\bGamma$ such that 
$\bGamma$ and $\mathfrak B$ are compact.
\end{itemize}

\begin{itemize}
\item[(H2$^{\prime\prime}$)] 
For $j=0,1$, given any $s$, $\bbeta_j$,
$b_{\bbeta_j}(s,\cdot)$,  $\sigma(s,\cdot)$ are $C^1$ on $\mathbb R$;  
we also assume that $b^2_{\bbeta_j}(s,x)\leq K_1(1+x^2+\|\bbeta_j\|^2)$  
and $\sigma^2(x)\leq K_2(1+x^2)$
for all $s\in [0,T]$, $x\in\mathbb R$, for some $K_1,K_2>0$. 
%Now, due to (H1$^\prime$), the latter
%reduces to the assumption that
%for $j=0,1$, given any $s$, $\bbeta_j$ and $\bgamma_j$, 
%$b_{\bbeta_j}(s,\cdot)$ and
%$\sigma_{\bgamma_j}(s,\cdot)$ are $C^1$ on $\mathbb R$ %having linear growth.
%satisfying 
By (H1$^{\prime\prime}$) it follows as before that for $s\in[0,T]$, 
$b^2_{\bbeta_j}(s,x)\leq K(1+x^2)$  and $\sigma^2(s,x)\leq K(1+x^2)$
for all $x\in\mathbb R$, for some $K>0$. %Assuming that  $\phi_i\stackrel{iid}{\sim}N(\mu,\omega^2)$, 


\end{itemize}
%respectively, for $i=0,1$.
%\item[(H5$^\prime$)] 
%Note that due to continuity of $\zeta(\varphi_j)$ in $\varphi_j$ for $j=1,2$, and due to compactness of the parameter space, 
%for $k\geq 1$, and $T>0$, $\zeta(\varphi_j)$ satisfies the following:
%\begin{equation*}
%E\left\vert\zeta(\varphi_j)\right\vert^{2k}<\infty.
%\end{equation*}
%\end{itemize}
%Due to (H5$^\prime$), %(see \ctn{Maitra14a}, \ctn{Maitra14b}, \ctn{Maitra15a}) 
Because of (H2$^{\prime\prime}$) it follows from Theorem 4.4 of \ctn{Mao11}, page 61, that  
for all $T>0$, and any $k\geq 2$, 
\begin{equation}
E\left(\underset{s\in [0,T]}{\sup}~|X_i(s)|^k\right)\leq\left(1+3^{k-1}E|X_i(0)|^k\right)\exp\left(\tilde\vartheta T\right),
\label{eq:moment1}
\end{equation}
where
$$\tilde\vartheta=\frac{1}{6}\left(18K\right)^{\frac{k}{2}}T^{\frac{k-2}{2}}\left[T^{\frac{k}{2}}
+\left(\frac{k^3}{2(k-1)}\right)^{\frac{k}{2}}\right].$$

Specifically, for any $k\geq 2$, we can write, as $T\rightarrow\infty$,
%for all $T>0$, and any $k\geq 2$, 
\begin{equation}
E\left(\underset{s\in [0,T]}{\sup}~|X(s)|^k\right)=o\left(\exp\left\{T^{k+1}\right\}\right).
\label{eq:moment1_T}
\end{equation}
%The $2k$-th order moments of the other parameters are also finite because of 
%
%
%\begin{equation}
%\underset{t\in[0,T]}{\sup}~E\left\vert X(t)\right\vert^{2k}<\infty. 
%\label{eq:finite_moments}
%\end{equation}
We further assume the following conditions.
\begin{itemize}
%\item[(H6$^\prime$)] There exists $m\geq 1$ such that
%\begin{align}
%&\frac{\underset{s\in[0,T]}{\sup}~\left[X(s)\right]^2}{T^m}\rightarrow 0,~\mbox{almost surely, as}~T\rightarrow\infty,~\mbox{and}
%\label{eq:H6_1}\\
%&\frac{\underset{s\in[0,T]}{\sup}~E\left[X(s)\right]^4}{T^{2m}}=O(1),
%%&\frac{\underset{s\in[0,T]}{\sup}~E\left[X(s)\right]^2}{T^{m}}=o(1).
%\label{eq:H6_2}
%\end{align}
%almost surely as $T\rightarrow\infty$.
\item[(H3$^{\prime\prime}$)] $b_{\bbeta_j}(s,x)$ is continuous 
in $(x,\bbeta_j)$.


\item[(H4$^{\prime\prime}$)] %With $m$ satisfying (\ref{eq:H6_1}) and (\ref{eq:H6_2}), 
For $s\in[0,T]$ and $j=0,1$, 
$\frac{b_{\bbeta_j}^2(s,x)}{\sigma^2(s,x)}$ and $\frac{b_{\bbeta_1}(s,x)b_{\bbeta_0}(s,x)}{\sigma^2(s,x)}$ 
satisfy the following:
\begin{align}
&\kappa_j(\bbeta_j) + \frac{K_{\bbeta_j}\left(1+x^2+\|\bbeta_j\|^2\right)}{c_j+\exp\left(T^5\right)}
<\frac{b_{\bbeta_j}^2(s,x)}{\sigma^2(s,x)}\notag\\
&\quad\quad\quad<\kappa_j(\bbeta_j)+
\frac{M_{\bbeta_j}\left(1+x^2+\|\bbeta_j\|^2\right)}{d_j+\exp\left(T^5\right)},
\label{eq:H4_prime_prime_1}
\end{align}
and
\begin{align}
&\bar{\kappa}(\bbeta_0,\bbeta_1) + \frac{K_{\bbeta_0,\bbeta_1}
\left(1+x^2+\|\bbeta_0\|^2+\|\bbeta_1\|^2\right)}{\bar c+\exp\left(T^5\right)}< 
\frac{b_{\bbeta_1}(s,x)b_{\bbeta_0}(s,x)}{\sigma^2(s,x)}\notag\\
&\quad\quad\quad<\bar{\kappa}(\bbeta_0,\bbeta_1)+\frac{M_{\bbeta_0,\bbeta_1}
\left(1+x^2+\|\bbeta_0\|^2+\|\bbeta_1\|^2\right)}{\bar d+\exp\left(T^5\right)},
\label{eq:H4_prime_prime_2}
\end{align}
where $0<c_j,d_j,\bar c,\bar d<\infty$, are some constants;
$\kappa_j(\bbeta_j)$ are positive, continuous functions of $\bbeta_j$; $\bar{\kappa}(\bbeta_0,\bbeta_1)$
is a continuous function of $(\bbeta_0,\bbeta_1)$;
$K_{\bbeta_j}$, $M_{\bbeta_j}$ are continuous in $\bbeta_j$, for $j=0,1$,
and $K_{\bbeta_0,\bbeta_1}$, $M_{\bbeta_0,\bbeta_1}$ are continuous in $(\bbeta_0,\bbeta_1)$.

\item[(H5$^{\prime\prime}$)]

(i) We assume that $\boldcal Z= \bZ_1\times\bZ_2\times\cdots\times\bZ_p$ is the space of the covariates
where $\bZ_l$ is compact for $l=1,\ldots,p$, and for every $t\geq 0$, 
$\bz_i(t)=(z_{i1}(t),z_{i2}(t),\ldots,z_{ip}(t))\in\boldcal Z$ for $i=1,\ldots,n$.
Also, we assume that $\bz_i(t)$ are continuous in $t$ for every $i$, so that $\bz_i\in\mathfrak Z$, for every $i$.

(ii) For $j=0,1$, and for $t\geq 0$, we assume that the vector of covariates $\bz_i(t)$ is related to the $i$-th $SDE$
of the $j$-th model via
$$\phi_{i,\bxi^{(i)}_j}(t)=\phi_{i,\bxi^{(i)}_j}(\bz_i(t))=\xi^{(i)}_{0j}+\sum_{l=1}^p\xi^{(i)}_{lj}g_l(\bz_i(t)),$$
where, for $l=1,\ldots,p$, $g_l:\bZ_l\rightarrow \mathbb R$ is 
continuous. Notationally, when reference to the $i$-th individual is self-explanatory, 
we shall denote the function $\xi_{0j}+\sum_{l=1}^p\xi_{lj}g_l$
by $\phi_{\bxi_j}$.

(iii) For $l=1,2,\ldots,p$, for $i=1,\ldots,n$, and for $t\geq 0$,
\begin{equation}
\frac{1}{n}\sum_{i=1}^n g_l(z_{il}(t))\rightarrow c_{l}(t),
\label{eq:H5_prime_1_T}
\end{equation}
and
\begin{equation}
\frac{1}{n}\sum_{i=1}^n g_l(z_{il}(t))g_m(z_{im}(t))\rightarrow c_l(t)c_m(t),
\label{eq:H5_prime_2_T}
\end{equation}
as $n\rightarrow\infty$, where $\left\{c_l(t):t\geq 0\right\}$ are real constants for $l=1,\ldots,p$.

(iv) For $l=1,2,\ldots,p$, and for $i=1,\ldots,n$,
\begin{equation}
\underset{T\rightarrow\infty}{\lim}~\frac{1}{T}\int_0^T g_l(z_{il}(s))ds =\bar c^{(1)}_{il}\label{eq::H5_prime_3_T}
\end{equation}
and
\begin{equation}
\underset{T\rightarrow\infty}{\lim}~\frac{1}{T}\int_0^T g_l(z_{il}(s))g_m(z_{im}(s))ds
=\bar c^{(2)}_{ilm},\label{eq::H5_prime_4_T}
\end{equation}
where $\bar c^{(1)}_{il}$ and $\bar c^{(2)}_{ilm}$ are real constants.
\end{itemize}
\begin{remark}
\label{remark:time_dependent_covariates2}
Observe that although (H4$^{\prime\prime}$) is seemingly restrictive in the sense that the ratios
$\frac{b_{\bbeta_j}^2(s,x)}{\sigma^2(s,x)}$ and $\frac{b_{\bbeta_1}(s,x)b_{\bbeta_0}(s,x)}{\sigma^2(s,x)}$
are approximately independent of the underlying stochastic process, assumption (H5$^{\prime\prime}$)
attempts to compensate for the restrictions by providing a rich structure to $\phi_{\bxi_j}$ consisting of 
covariate information varying continuously with time. 
Hence, assumption (H4$^{\prime\prime}$) need not be viewed as restrictive. 
\end{remark}

%Note that, given $l$ and $t$, had $z_{il}(t)$ been random and $iid$ with respect to $i$, then 
%(\ref{eq:H5_prime_1}) would hold almost surely by the strong law of large numbers. Additionally,
%if $z_{il}(t)$ and $z_{im}(t)$ were independent, then (\ref{eq:H5_prime_2}) would hold almost surely
%as well. Hence, in this paper, 
\ctn{Maitra15a} argue that (\ref{eq:H5_prime_1_T}) and (\ref{eq:H5_prime_2_T}) hold
if one assumes that for $i=1,\ldots,n$, and  
$l=1,\ldots,p$, the covariates $z_{il}$ are observed realizations
of stochastic processes that are $iid$ for $i=1,\ldots,n$, for all $l=1,\ldots,p$, and that 
for $l\neq m$, the processes generating $z_{il}$ and $z_{im}$ are independent. 
In other words, although we assume the covariates to be non-random, in
essence, it may be assumed $g_l(z_{il}(t))$ and $g_m(z_{im}(t))$ are  
uncorrelated for $l\neq m$. 

In order that (H5$^{\prime\prime}$) (iv) holds, one needs to further assume that
the relevant stochastic processes converge to appropriate stationary distributions.
%Additionally, (H5$^{\prime\prime}$) (iv) holds by the ergodic theorem
%if $z_{il}(t)$ are realizations converging to appropriate stationary distributions.
For example, $z_{il}(t)$ may be realizations of Markov processes which are
irreducible (with respect to some appropriate measure), aperiodic, positive recurrent and possses invariant
distributions; see, for example, \ctn{Konto03}. 
%Thus, (H5$^{\prime\prime}$) (iii)
%and (H5$^{\prime\prime}$) (iv) hold 
%In this work, these covariates should not be misconstrued as random.
%Indeed, we assume that we observe only the realizations, which we treat as non-random covariates.




It follows from (H5$^{\prime\prime}$) (iv), that, 
\begin{align}
&\underset{T\rightarrow\infty}{\lim}~ \frac{1}{T}\int_0^T\phi_{i,\bxi^{(i)}_j}(\bz_i(s))ds\notag\\
&= \xi^{(i)}_{0j}+\sum_{l=1}^p\xi^{(i)}_{lj}\bar c^{(1)}_{il}\notag\\
&=\bar\phi^{(1)}_{i,\bxi^{(i)}_j}~\mbox{(say)},
\label{eq:phi_conv}
\end{align}
\begin{align}
&\underset{T\rightarrow\infty}{\lim}~\frac{1}{T}\int_0^T\phi^2_{i,\bxi^{(i)}_j}(\bz_i(s))ds\notag\\
&= \left\{\xi^{(i)}_{0j}\right\}^2+2\xi^{(i)}_{0j}\sum_{l=1}^p\xi^{(i)}_{lj}\bar c^{(1)}_{il}
+\sum_{l=1}^p\sum_{m=1}^p\xi^{(i)}_{lj}\xi^{(i)}_{mj}\bar c^{(2)}_{ilm}\notag\\
&=\bar\phi^{(2)}_{i,\bxi^{(i)}_j},~\mbox{(say)},
\label{eq:phisq_conv}
\end{align}
and
\begin{align}
&\underset{T\rightarrow\infty}{\lim}~\frac{1}{T}\int_0^T\phi_{i,\bxi^{(i)}_0}(\bz_i(s))\phi_{i,\bxi^{(i)}_1}(\bz_i(s))ds\notag\\
&=\xi^{(i)}_{00}\xi^{(i)}_{01}+\xi^{(i)}_{00}\sum_{l=1}^p\xi^{(i)}_{l1}\bar c^{(1)}_{il}
+\xi^{(i)}_{01}\sum_{l=1}^p\xi^{(i)}_{l0}\bar c^{(1)}_{il}+
\sum_{l=1}^p\sum_{m=1}^p\xi^{(i)}_{l0}\xi^{(i)}_{m1}\bar c^{(2)}_{ilm}\notag\\
&=\bar\phi^{(2)}_{i,\bxi^{(i)}_0,\bxi^{(i)}_1},~\mbox{(say)}.
\label{eq:phiphi_conv}
\end{align}
When $i$ is clear from the context, we shall often use the notations 
$\bar\phi^{(1)}_{\bxi_j}$, $\bar\phi^{(2)}_{\bxi_j}$ and $\bar\phi^{(2)}_{\bxi_0,\bxi_1}$.

Note that, (\ref{eq:phi_conv}), (\ref{eq:phisq_conv}) and (\ref{eq:phiphi_conv}) are limits of expectations
with respect to the uniform distribution on $[0,T]$. Hence, by the Cauchy-Schwartz inequality it follows that
\begin{equation}
\bar\phi^{(2)}_{\bxi_0,\bxi_1}
\leq \sqrt{\bar\phi^{(2)}_{\bxi_0}\bar\phi^{(2)}_{\bxi_1}}.
\label{eq:phi_cs}
\end{equation}



The following lemmas will be useful in our proceedings. The proofs of these lemmas are provided in sections S-1, S-2 and S-3 of the supplement. 
\begin{lemma}
\label{lemma:uniform_convergence}
The limits
$\bar\phi^{(1)}_{\bxi_1}$, $\bar\phi^{(2)}_{\bxi_1}$ and $\bar\phi^{(2)}_{\bxi_0,\bxi_1}$
are continuous in $\bxi_1$.
\end{lemma}

\begin{comment}
\begin{proof}
Due to compactness of $\bGamma$ it follows, using the form of $\phi_{\bxi_j}$ provided in
(H5$^{\prime\prime}$), that the convergences (\ref{eq:phi_conv}), (\ref{eq:phisq_conv})
and (\ref{eq:phiphi_conv}) are uniform over $\bGamma$. The same form shows that the above integrals  
are continuous in $\bxi_1$, for every $T>0$. Hence, due to uniform convergence, the limits 
$\bar\phi^{(1)}_{\bxi_1}$, $\bar\phi^{(2)}_{\bxi_1}$ and $\bar\phi^{(2)}_{\bxi_0,\bxi_1}$
are continuous in $\bxi_1$.
\end{proof}
\end{comment}
%It is important to mention that the covariates may be allowed to vary continuously
%with time, with the assumption that (\ref{eq:H5_prime_1_T}) and (\ref{eq:H5_prime_2_T}) hold for 
%every time point $t$ (the relevant notations must be modified to indicate dependence on the time point $t$).
%Then all our results continue to hold with minor adjustments. In this work we do not further pursue time-dependent
%covariates due to increased notational complexity and lack of space, but note that when time-dependent covariates
%are available, 

%We then have the following lemmas:
%\begin{lemma}
%\label{lemma:moment_existence}
%For any $t>0$, %for all $\btheta_1\in\bTheta$, 
%and any $k\geq 1$, 
%\begin{align}
%E_{\btheta_0}\left[U_{\btheta_1,t}\right]^k<\infty,\label{eq:finite_moment_u}\\
%E_{\btheta_0}\left[V_{\btheta_1,t}\right]^k<\infty.\label{eq:finite_moment_v}
%\end{align}
%Moreover, the above expectations are continuous in $\btheta_1$.
%\end{lemma}
%\begin{proof}
%The proof follows in the same way as Lemma 1 and Lemma 9 of \ctn{Maitra15a}.
%\end{proof}
%Due to assumption (H6$^\prime$) and (\ref{eq:finite_moments}), (\ref{eq:finite_moment_v}) is easily seen to hold.
%That (\ref{eq:finite_moment_u}) holds follows from Assumption (H6$^\prime$), (\ref{eq:u_v_relation}), (\ref{eq:zero_mean})
%and It\^{o} isometry, given by
%\begin{equation}
%E\left[\int_0^{T_i}\frac{b_{\bbeta_j}\left(X(s)\right)}{\sigma_{\bgamma_j}\left(X(s)\right)}dW_i(s)\right]^2
%=\int_0^{T_i}E\left[\frac{b_{\bbeta_j}\left(X(s)\right)}{\sigma_{\bgamma_j}\left(X(s)\right)}\right]^2ds,
%\label{eq:ito_isometry}
%\end{equation}
%for $j=0,1$.
%Since due to (H5$^\prime$) $V_{it}$ is continuous in $\btheta_1$, so is $U_{it}$, due to the relation (\ref{eq:u_v_relation}). 
%From assumptions (H4$^\prime$) and (H6$^\prime$) it follows that the expectations 
%(\ref{eq:finite_moment_u}) and (\ref{eq:finite_moment_v}) are continuous in $\btheta_1$.


\begin{lemma}
\label{lemma:convergence1}
Assume (H1$^{\prime\prime}$) -- (H5$^{\prime\prime}$). Then, the following hold:
\begin{align}
%(i)\quad &\frac{V_{\btheta_j,T}}{T}\stackrel{a.s.}{\rightarrow} \kappa_j(\btheta_j), \label{eq:V_convergence}\\
%(ii)\quad &\int_0^T\frac{b(s,X(s))}{\sigma(s,X(s))}dW(s)\stackrel{a.s.}{\rightarrow} 0,\label{eq:integral_convergence}\\
%(iii)\quad &\frac{U_{\btheta_j,T}}{T}\stackrel{a.s.}{\rightarrow} \zeta\left(\varphi_j\right)\kappa_j(\btheta_j),
%\label{eq:U_convergence}\\
(i)\quad &E_{\btheta_0}\left(\frac{V_{\btheta_j,T}}{T}\right)\rightarrow 
\bar \phi^{(2)}_{\bxi_j}\kappa_j(\bbeta_j);j=0,1.\label{eq:E_V_convergence}\\
(ii)\quad &E_{\btheta_0}\left(\frac{V_{\btheta_0,\btheta_1,T}}{T}\right)
\rightarrow  \bar \phi^{(2)}_{\bxi_0,\bxi_1}\bar{\kappa}(\bbeta_0,\bbeta_1).
\label{eq:E_V_convergence2}\\
(iii)\quad &E_{\btheta_0}\left(\frac{U_{\btheta_0,T}}{T}\right)\rightarrow 
\bar\phi^{(2)}_{\bxi_0}\kappa_0(\bbeta_0).\label{eq:E_U_convergence_0}\\
(iv)\quad &E_{\btheta_0}\left(\frac{U_{\btheta_1,T}}{T}\right)\rightarrow 
\bar\phi^{(2)}_{\bxi_0,\bxi_1}\bar{\kappa}(\bbeta_0,\bbeta_1).\label{eq:E_U_convergence}\\
(v)\quad &\frac{1}{T}\int_0^T\frac{\phi_{\bxi_j}(s)b_{\bbeta_j}(s,X(s))}{\sigma(s,X(s))}dW(s)
\stackrel{a.s.}{\longrightarrow} 0; j=0,1,
\label{eq:integral_convergence}\\
(vi)\quad &\frac{V_{\btheta_j,T}}{T}\stackrel{a.s.}{\longrightarrow} 
 \bar \phi^{(2)}_{\bxi_j}\kappa_j(\bbeta_j); j=0,1, \label{eq:V_convergence}\\
(vii)\quad &\frac{V_{\btheta_0,\btheta_1,T}}{T}\stackrel{a.s.}{\longrightarrow} 
\bar \phi^{(2)}_{\bxi_0,\bxi_1}\bar{\kappa}(\bbeta_0,\bbeta_1), 
\label{eq:V_convergence2}\\
(viii)\quad &\frac{U_{\btheta_0,T}}{T}\stackrel{a.s.}{\longrightarrow} 
\bar\phi^{(2)}_{\bxi_0}\kappa_0(\bbeta_0),
\label{eq:U_convergence_0}\\
(ix)\quad &\frac{U_{\btheta_1,T}}{T}\stackrel{a.s.}{\longrightarrow} 
\bar\phi^{(2)}_{\bxi_0,\bxi_1}\bar{\kappa}(\bbeta_0,\bbeta_1),
\label{eq:U_convergence}
\end{align}
In the above, $``\stackrel{a.s.}{\longrightarrow}"$ denotes convergence ``almost surely" as $T\rightarrow\infty$
with respect to $X$ (under $\btheta_0$), and the expectations are also with respect to $X$ (under $\btheta_0$).
\end{lemma}

\begin{comment}
\begin{proof}
%The proofs of (\ref{eq:V_convergence}) and (\ref{eq:E_V_convergence}) 
%easily follow from (\ref{eq:H6_1}) and (\ref{eq:H6_2}), respectively, of assumption (H6$^\prime$), and (H7$^\prime$).
The proofs of (i) -- (iv) follow from (H1$^{\prime\prime}$), (H4$^{\prime\prime}$), the results
(\ref{eq:phi_conv}), (\ref{eq:phisq_conv}) and (\ref{eq:phiphi_conv}) following from (H5$^{\prime\prime}$), 
(\ref{eq:moment1}) and its asymptotic form (\ref{eq:moment1_T}) (with $k=2$), 
using the relation (\ref{eq:u_v_relation}). 
%\begin{equation}
%U_{\btheta_0,T}=\zeta(x,\xi_0)V_{\btheta_0,T}+\int_0^T\frac{b_{\bbeta_0}(s,X(s))}{\sigma_{\bgamma_j}(s,X(s))}dW(s),
%\label{eq:U_V_relation}
%\end{equation}
%and noting that for $j=0,1$,
%$$E\left(\int_0^T\frac{b_{\bbeta_j}(s,X(s))}{\sigma_{\bgamma_j}(s,X(s))}dW(s)\right)=0.$$
%
To prove (v), %(\ref{eq:integral_convergence}), 
%note that
%$$\frac{1}{T}E\left(\int_0^T\frac{b_{\bbeta_j}(s,X(s))}{\sigma_{\bgamma_j}(s,X(s))}dW(s)\right)=0.$$
%and
%\begin{align}
%&\frac{1}{T^2}E\left(\int_0^T\frac{b_{\bbeta_j}(s,X(s))}{\sigma_{\bgamma_j}(s,X(s))}dW(s)\right)^2\notag\\
%&=\frac{1}{T^2}\int_0^TE\left(\frac{b_{\bgamma_j}(s,X(s))}{\sigma_{\bgamma_j}(s,X(s))}\right)^2ds\notag\\
%&\leq\frac{\kappa_j(\btheta_j)}{T}+\frac{M_{\bbeta_j,\bgamma_j}(1+\|\bbeta_j\|^2+\|\bgamma_j\|^2)}
%{T\left(d_j+\exp\left(T^4\right)\right)}
%+\frac{E\left[\underset{s\in[0,T]}{\sup}~|X(s)|^2\right]}{T\left(d_j+\exp\left(T^4\right)\right)},
%\label{eq:upper_bound1}
%\end{align}
%where the last inequality follows from (H6$^\prime$). Since $\btheta_j\in\bTheta$, where $\bTheta$ is compact
%by (H3$^\prime$), and since $\kappa_j(\btheta_j)$, $K_{\bbeta_j,\bgamma_j}$ and $M_{\bbeta_j,\bgamma_j}$
%are continuous in $(\bbeta_j,\bgamma_j)$ by (H6$^\prime$), it follows that the first two terms in
%(\ref{eq:upper_bound1}) converge to zero as $T$ tends to infinity. The last term converges to zero
%due to (\ref{eq:moment1_T}).
%%$\frac{\underset{s\in[0,T]}{\sup}~E\left[X(s)\right]^2}{T^{m+1}}\rightarrow 0$ due to (\ref{eq:H6_2}).
%This implies that
%$$\int_0^T\frac{b_{\bbeta_j}(s,X(s))}{\sigma_{\bgamma_j}(s,X(s))}dW(s)\rightarrow 0$$
%in probability.
%
note that, since for any $k\geq 1$, 
it holds, due to (H4$^{\prime\prime}$), (\ref{eq:moment1}) and boundedness of $\phi_{\bxi_j}$ on $[0,T]$, that for $j=0,1$,
$$E\int_0^T\left|\frac{\phi_{\bxi_j}(s)b_{\bbeta_j}(s,X(s))}{\sigma(s,X(s))}\right|^{2k}ds<\infty,$$
it follows from Theorem 7.1 of \ctn{Mao11}, page 39, that
\begin{align}
E\left|\int_0^T\frac{\phi_{\bxi_j}(s)b_{\bbeta_j}(s,X(s))}{\sigma(s,X(s))}dW(s)\right|^{2k}
\leq\left(k(2k-1)\right)^{k}T^{k-1}E\int_0^T
\left|\frac{\phi_{\bxi_j}(s)b_{\bbeta_j}(s,X(s))}{\sigma(s,X(s))}\right|^{2k}ds.
\end{align}
Hence, using Chebychev's inequality, it follows that for any $\epsilon>0$,
\begin{align}
&P\left(\left|\frac{1}{T}\int_0^T\frac{\phi_{\bxi_j}(s)b_{\bbeta_j}(s,X(s))}{\sigma(s,X(s))}dW(s)\right|
>\epsilon\right)\notag\\
&\quad\quad\quad<\epsilon^{-2k}\left(k(2k-1)\right)^{k}T^{-(k+1)}E\int_0^T
\left|\frac{\phi_{\bxi_j}(s)b_{\bbeta_j}(s,X(s))}{\sigma(s,X(s))}\right|^{2k}ds.
\end{align}
In particular, if $k=2$ is chosen, then it 
follows from the above inequality, (H4$^{\prime\prime}$) , and (\ref{eq:moment1_T}), and boundedness of
$\phi_{\bxi_j}$ on $[0,T]$, that
\begin{equation*}
\sum_{T=1}^{\infty}P\left(\left|\frac{1}{T}
\int_0^T\frac{\phi_{\bxi_j}(s)b_{\bbeta_j}(s,X(s))}{\sigma(s,X(s))}dW(s)\right|>\epsilon\right)
<\infty,
\end{equation*}
proving that 
$$\frac{1}{T}\int_0^T
\frac{\phi_{\bxi_j}(s)b_{\bbeta_j}(s,X(s))}{\sigma(s,X(s))}dW(s)\stackrel{a.s.}{\longrightarrow} 0.$$

To prove (vi), first note that 
\begin{align}
%&E\left|\frac{1}{T}\int_0^T\left\{\frac{\phi^2_{\bxi_j}(s)b^2_{\bbeta_j}(s,X(s))}{\sigma^2(s,X(s))}
%-\phi^2_{\bxi_j}(s)\kappa_j(\bbeta_j)\right\}ds\right|^{2k}\notag\\
&E\left|\frac{1}{T}\int_0^T\left[\frac{\phi^2_{\bxi_j}(s)b^2_{\bbeta_j}(s,X(s))}{\sigma^2(s,X(s))}
-\phi^2_{\bxi_j}(s)\kappa_j(\bbeta_j)\right]ds\right|^{2k}\notag\\
&\leq T^{-1}E\int_0^T\phi^{4k}_{\bxi_j}(s)\left|\frac{b^2_{\bbeta_j}(s,X(s))}{\sigma^2(s,X(s))}
-\kappa_j(\bbeta_j)\right|^{2k}ds\notag\\
&\leq K_4T^{-1}E\int_0^T\left|\frac{b^2_{\bbeta_j}(s,X(s))}{\sigma^2(s,X(s))}
-\kappa_j(\bbeta_j)\right|^{2k}ds,
\end{align}
for some finite constant $K_4>0$.
The second last inequality is by H\"{o}lder's inequality, and the last inequality
holds because 
$\phi_{\bxi_j}(t)$ is uniformly bounded on $[0,\infty]$ thanks to compactness of $\boldcal Z$ and continuity of 
the functions $g_l;l=1,\ldots,p$. Hence, for any $\epsilon>0$,
\begin{align}
&P\left(\left|\frac{1}{T}\int_0^T\frac{\phi^2_{\bxi_j}(s)b^2_{\bbeta_j}(s,X(s))}{\sigma^2(s,X(s))}ds
-\kappa_j(\bbeta_j)\frac{1}{T}\int_0^T\phi^2_{\bxi_j}(s)ds\right|>\epsilon\right)\notag\\
&\qquad\qquad
<K_4\epsilon^{-2k}T^{-1}E\int_0^T\left|\frac{b^2_{\bbeta_j}(s,X(s))}{\sigma^2(s,X(s))}-\kappa_j(\bbeta_j)\right|^{2k}ds.
\notag
\end{align}
In the same way as the proof of (v), it follows, using the above inequality, (\ref{eq:H4_prime_prime_1})
and (\ref{eq:moment1_T}), that
\begin{equation*}
\sum_{T=1}^{\infty}P\left(\left|\frac{1}{T}\int_0^T\frac{\phi^2_{\bxi_j}(s)b^2_{\bbeta_j}(s,X(s))}{\sigma^2(s,X(s))}ds
-\kappa_j(\bbeta_j)\frac{1}{T}\int_0^T\phi^2_{\bxi_j}(s)ds\right|>\epsilon\right)
<\infty.
\end{equation*}
That is, 
$$\frac{1}{T}\int_0^T\frac{\phi^2_{\bxi_j}(s)b^2_{\bbeta_j}(s,X(s))}{\sigma^2(s,X(s))}ds
-\kappa_j(\bbeta_j)\frac{1}{T}\int_0^T\phi^2_{\bxi_j}(s)ds\rightarrow 0,$$
almost surely, as $T\rightarrow\infty$. Since, as $T\rightarrow\infty$, 
$\frac{1}{T}\int_0^T\phi^2_{\bxi_j}(s)ds\rightarrow\bar\phi^{(2)}_{\bxi_j}$
by (\ref{eq:phisq_conv}), the result follows.
%
Using (\ref{eq:H4_prime_prime_2}) instead of (\ref{eq:H4_prime_prime_1}), (vii) can be proved in the same way as (vi). 
%
The proofs of %(\ref{eq:U_convergence}) follows from (\ref{eq:V_convergence}) and (\ref{eq:integral_convergence}), 
(viii) and (ix) follow from (v), (vi) and (vii), using the relation (\ref{eq:u_v_relation}).
\end{proof}
\end{comment}

\begin{lemma}
\label{lemma:phi_kappa_relations}
Assume (H1$^{\prime\prime}$) -- (H5$^{\prime\prime}$). Then, the following holds:
\begin{equation}
\bar\phi^{(2)}_{\bxi_0,\bxi_1}\bar\kappa(\bbeta_0,\bbeta_1)
\leq \sqrt{\bar\phi^{(2)}_{\bxi_0}\bar\phi^{(2)}_{\bxi_1}}
\times\sqrt{\kappa_0(\bbeta_0)\kappa_1(\bbeta_1)}.
\label{eq:phi_kappa_relations}
\end{equation}
\end{lemma}

\begin{comment}
\begin{proof}
Using the Cauchy-Schwartz inequality twice we obtain
\begin{align}
&\frac{1}{T}\int_0^T
E\left(\frac{\phi_{\bxi_0}(s)\phi_{\bxi_1}(s)b_{\bbeta_0}(s,X(s))b_{\bbeta_1}(s,X(s))}{\sigma^2(s,X(s))}\right)ds
\notag\\
&\leq\frac{1}{T}\int_0^T
\sqrt{E\left(\frac{\phi^2_{\bxi_0}(s)b^2_{\bbeta_0}(s,X(s))}{\sigma^2(s,X(s))}\right)}
\times \sqrt{E\left(\frac{\phi^2_{\bxi_1}(s)b^2_{\bbeta_1}(s,X(s))}{\sigma^2(s,X(s))}\right)}ds\notag\\
&=\sqrt{\kappa_0(\bbeta_0)\kappa_1(\bbeta_1)+O\left(\exp\left(T^3-T^5\right)\right)}\times
\frac{1}{T}\int_0^T\left|\phi_{\bxi_0}(s)\right|\times\left|\phi_{\bxi_1}(s)\right|ds\notag\\
&\leq \sqrt{\kappa_0(\bbeta_0)\kappa_1(\bbeta_1)+O\left(\exp\left(T^3-T^5\right)\right)}\times
\left(\sqrt{\frac{1}{T}\int_0^T\phi^2_{\bxi_0}(s)ds}\right)\times
\left(\sqrt{\frac{1}{T}\int_0^T\phi^2_{\bxi_1}(s)ds}\right).
\label{eq:phi_kappa_2}
\end{align}
Taking the limit of both sides of (\ref{eq:phi_kappa_2}) as $T\rightarrow\infty$, using (ii) of 
Lemma \ref{lemma:convergence1} and the limits (\ref{eq:phisq_conv}), the result (\ref{eq:phi_kappa_relations})
follows.
\end{proof}
\end{comment}

\section{Convergence of Bayes factor with respect to time when two individual $SDE$'s are compared}
\label{sec:result_BF_consistency}

From the system of $SDE$'s defined by (\ref{eq:sde3}) and (\ref{eq:sde4}) we now consider the $i$-th individual only. 
To avoid notational complexity we denote $X_i$ simply by $X$. Consequently, $\phi_{i,\bxi_j}(t)$ and $T_i$ 
will be denoted by $\phi_{\bxi_j}(t)$ and $T$, respectively. In connection with the $i$-th individual 
we consider the following two $SDE$'s:
\begin{equation}
d X(t)=\phi_{\bxi_0}(t)b_{\bbeta_0}(t,X(t))dt+\sigma(t,X(t))dW(t)
\label{eq:sde1_T}
\end{equation}
and
\begin{equation}
d X(t)=\phi_{\bxi_1}(t)b_{\bbeta_1}(t,X(t))dt+\sigma(t,X(t))dW(t).
\label{eq:sde2_T}
\end{equation}

For any $t\in[0,T]$, for $j=0,1$, let
\begin{align}
U_{\btheta_j,t} &=\int_0^t\frac{\phi_{\bxi_j}(s)b_{\bbeta_j}(s,X(s))}{\sigma^2(s,X(s))}dX(s),\quad 
V_{\btheta_j,t} =\int_0^t\frac{\phi^2_{\bxi_j}(s)b^2_{\bbeta_j}(s,X(s))}{\sigma^2(s,X(s))}ds,\notag\\
V_{\btheta_0,\btheta_j,t} 
&=\int_0^t\frac{\phi_{\bxi_j}(s)b_{\bbeta_j}(s,X(s))\phi_{\bxi_0}(s)b_{\bbeta_0}(s,X(s))}{\sigma^2(s,X(s))}ds.
\label{eq:u_v_t}
\end{align}
Note that $V_{\btheta_0,t}= V_{\btheta_0,\btheta_0,t}$ and 
$V_{\btheta_1,t}= V_{\btheta_1,\btheta_1,t}$.
We also let
\begin{equation}
f_{\btheta_j,t}(X_{0,t})=\exp\left(U_{\btheta_j,t}-\frac{V_{\btheta_j,t}}{2}\right).
\label{eq:f_t}
\end{equation}
%
Here we are interested in asymptotic properties of the Bayes factor, given by
\begin{equation}
I_T=\int \frac {f_{\btheta_1,T}(X_{0,T})}{f_{\btheta_0,T}(X_{0,T})}\pi (d\btheta_1),
\label{eq:bf_T}
\end{equation}
as $T\rightarrow\infty$.

For our purpose, let us define, for any $h>0$,
\begin{align}
U_{\btheta_j,t,t+h} &=\int_t^{t+h}\frac{\phi_{\bxi_j}(s)b_{\bbeta_j}(s,X(s))}{\sigma^2(s,X(s))}dX(s),\quad 
V_{\btheta_j,t,t+h} =\int_t^{t+h}\frac{\phi^2_{\bxi_j}(s)b^2_{\bbeta_j}(s,X(s))}{\sigma^2(s,X(s))}ds,\notag\\
V_{\btheta_0,\btheta_j,t,t+h} 
&=\int_t^{t+h}\frac{\phi_{\bxi_j}(s)b_{\bbeta_j}(s,X(s))\phi_{\bxi_0}(s)b_{\bbeta_0}(s,X(s))}{\sigma^2(s,X(s))}ds.
\label{eq:u_v_t2}
\end{align}
Observe, as before, that $V_{\btheta_0,t,t+h}= V_{\btheta_0,\btheta_0,t,t+h}$ 
and $V_{\btheta_1,t,t+h}= V_{\btheta_1,\btheta_1,t,t+h}$.
We let
\begin{equation}
f_{\btheta_j,t,t+h}(X_{t,t+h})=\exp\left(U_{\btheta_j,t,t+h}-\frac{V_{\btheta_j,t,t+h}}{2}\right),
\label{eq:f_t2}
\end{equation}
where, for any $0\leq a<b<\infty$, $X_{a,b}$ denotes a path of the process $X$ from $a$ to $b$.
For any $t>0$ and $h>0$, we define %a new divergence measure between $f_{\btheta_0,t,t+h}$ and $f_{\btheta_1,t,t+h}$  by 
\begin{equation}
\tilde{\mathcal K}(f_{\btheta_0,t,t+h},f_{\btheta_1,t,t+h})
=E_{\btheta_0}\left[\log\frac{f_{\btheta_0,t,t+h}}{f_{\btheta_1,t,t+h}}\right],
%=\int \log\frac{f_{\btheta_0,t,t+h}(X_t)}{f_{\btheta_1,t,t+h}(X_t)}f_{\btheta_0,t}(X_t)
\label{eq:kl1_T}
\end{equation}
where $E_{\btheta_0}\equiv E_{f_{\btheta_0,t}}$.
Note that this is not the familiar Kullback-Leibler divergence measure, since $f_{\btheta_0,t}$, with respect
to which the expectation is taken, is not the same as $f_{\btheta_0,t,t+h}$.
In fact, since in our case, for $j=0,1$,
\begin{align}
f_{\btheta_j,t,t+h}\left(X_{t,t+h}\right)&=\exp\left(U_{\btheta_j,t,t+h}-\frac{V_{\btheta_j,t,t+h}}{2}\right)\notag\\
&=\exp\left((U_{\btheta_j,t+h}-U_{\btheta_j,t})
-\frac{(V_{\btheta_j,t+h}-V_{\btheta_j,t})}{2}\right),
\label{eq:f_t3}
\end{align}
it follows that
\begin{equation}
\tilde{\mathcal K}(f_{\btheta_0,t,t+h},f_{\btheta_1,t,t+h})
=\mathcal K(f_{\btheta_0,t+h},f_{\btheta_1,t+h})-\mathcal K(f_{\btheta_0,t},f_{\btheta_1,t}),
\label{eq:kl2}
\end{equation}
where $\mathcal K(f_{\btheta_0,t+h},f_{\btheta_1,t+h})$ and $\mathcal K(f_{\btheta_0,t},f_{\btheta_1,t})$
are proper Kullback-Leibler divergences between $f_{\btheta_0,t+h}$, $f_{\btheta_1,t+h}$,
and $f_{\btheta_0,t}$, $f_{\btheta_1,t}$,  respectively.
We now define 
\begin{align}
\tilde{\mathcal K}'_t(f_{\btheta_0},f_{\btheta_1})&=\lim_{h\rightarrow 0} 
\frac{\tilde {\mathcal K}(f_{\btheta_0,t,t+h},f_{\btheta_1,t,t+h})}{h}\notag\\
&=\frac{1}{2}\frac{d}{dt}E_{\theta_0}(V_{\btheta_0,t})
-\frac{d}{dt}E_{\theta_0}(V_{\btheta_0,\btheta_1,t})
+\frac{1}{2}\frac{d}{dt}E_{\theta_0}(V_{\btheta_1,t}).
\label{eq:kl3}
\end{align}
%where $E_{\btheta_0,t}\equiv E_{f_{\btheta_0,t}}$ and $\tilde{\mathcal K}(f_{\btheta_0,t,t+h},f_{\btheta_1,t,t+h})$ 
%is our known Kullback-Leibler (KL) divergence measure between $(f_{\btheta_0,t,t+h}$ and $f_{\btheta_1,t,t+h}$, given by
The expression (\ref{eq:kl3}) easily follows using (\ref{eq:f_t3}), 
the relation (\ref{eq:u_v_relation}) and (\ref{eq:zero_mean}).

\subsection{Pseudo  Kullback-Leibler $(\delta)$ property}
\label{subsec:pseudo_kl}
We make the following assumption: 
\begin{itemize}
\item[(H6$^{\prime\prime}$)]
For a fixed $\delta\geq 0$, the prior $\pi$ satisfies
\begin{equation}
\pi\left(\btheta_1\in\bTheta:\underset{t}{\inf}~\tilde{\mathcal K}'_t(f_{\btheta_0},f_{\btheta_1})\geq\delta\right)=1.
\label{eq:pi_prob_1}
\end{equation}
\end{itemize}

Let us define
\begin{equation}
\bar{\mathcal K}^{\infty}(f_{\btheta_0},f_{\btheta_1})
=\underset{T\rightarrow\infty}{\lim}~\frac{1}{T}\int_0^T\tilde{\mathcal K}'_t(f_{\btheta_0},f_{\btheta_1})dt.
\label{eq:kl_average_infinite}
\end{equation}
We assume the following:
\begin{itemize}
\item[(H7$^{\prime\prime}$)]
Given $\delta$ associated with (H6$^{\prime\prime}$), for any $c\geq 0$, the prior $\pi$ satisfies
\begin{equation}
\pi\left(\btheta_1\in\bTheta:\delta\leq\bar{\mathcal K}^{\infty}(f_{\btheta_0},f_{\btheta_1})\leq\delta+c\right)>0.
\label{eq:kl_property_T}
\end{equation}
\end{itemize}
We refer to property (H7$^{\prime\prime}$) as the pseudo Kullback-Leibler ($\delta$) property of the prior $\pi$.
Note that, (\ref{eq:kl3}), (\ref{eq:E_V_convergence}) and (\ref{eq:E_V_convergence2}) imply
\begin{align}
\bar{\mathcal K}^{\infty}(f_{\btheta_0},f_{\btheta_1})
&=\frac{\bar\phi^{(2)}_{\bxi_0}}{2}\kappa_0(\bbeta_0)
-\bar\phi^{(2)}_{\bxi_0,\bxi_1}\bar{\kappa}(\bbeta_0,\bbeta_1)
+\frac{\bar\phi^{(2)}_{\bxi_1}}{2}\kappa_1(\bbeta_1)\label{eq:kl_average_infinite2}\\
&\geq\frac{1}{2}\left(\sqrt{\bar\phi^{(2)}_{\bxi_0}\kappa_0(\bbeta_0)}
-\sqrt{\bar\phi^{(2)}_{\bxi_1}\kappa_1(\bbeta_1)}\right)^2\notag\\
&\geq 0,\notag
%\label{eq:kl_average_infinite2}
\end{align}
by Lemma \ref{lemma:phi_kappa_relations}.
Provided that (\ref{eq:pi_prob_1}) holds and the prior $\pi$ is dominated by the Lebesgue measure, 
the pseudo Kullback-Leibler ($\delta$)
property holds because of continuity of (\ref{eq:kl_average_infinite2}) 
%$\kappa_1(\bbeta_1)$, $\bar\phi^{(2)}_{\bxi_1}$ 
%and $\bar{\kappa}(\bbeta_0,\bbeta_1)$ 
in $\btheta_1=(\bbeta_1,\bxi_1)$ ensured by Lemma \ref{lemma:uniform_convergence}.
%For a fixed $\delta>0$ and any $c>0$, let us define
%\begin{align}
%B_{\delta}(c)&=\left\{f:\delta\leq \underset{t}{\inf}~\tilde{\mathcal K}(f_{0,t,t+h},f_{1,t,t+h})<\delta+c\right\}
%\label{eq:B_delta}\\
%B^{\infty}_{\delta}(c)&=\left\{f:\delta\leq\lim_{T\rightarrow\infty}\frac{1}{T}\left(\frac{\phi_0^2}{2}\int_0^T\frac{d}{dt}E(V_{0t})dt-\frac{\phi_1^2}{2}\int_0^T\frac{d}{dt}E(V_{1t})dt\right)<\delta+c\right\}.
%\label{eq:B_delta_infinite}
%\end{align}
%Then a prior $\pi$ has the Kullback-Leibler $(\delta)$ property if, for any $c_1,c_2>0$, 
%\begin{equation}
%\pi(B_{\delta}(c_1)\cap B_{\delta}(c_2))>0.  
%\label{eq:kl_property_t}
%\end{equation}

\subsection{$Q^*$ property}
\label{subsec:Q_star_T}
%Let us denote the posterior distribution with respect to time $t$ by $\pi_t$. 
%Let us define the predictive density by
%\[
%\hat f_{t-h,t}=\int f_{\btheta_1,t-h,t}\pi_{t}(d\btheta_1)
%\]
%and
%\[
%\hat f_{t-h,t,A}=\int f_{\btheta_1,t-h,t}\pi_{tA}(d\btheta_1)
%\]
%to be the predictive density with posterior restricted to the set $A$, that is, for $\pi(A)>0$,
%\[
%\pi_{tA}(d\btheta_1)=\frac{{\bI}_A(\btheta_1)\pi_t(d\btheta_1)}{\int_A \pi_t(d\btheta_1)}.
%\]
%
%
%Let $I_0\equiv 1$ and for $t> 0$, let us define, analogous to (\ref{eq:bf_T}),
%\begin{equation}
%I_t=\int \frac {f_1(U_{1t},V_{1t})}{f_0(U_{0t},V_{0t})}\Pi (df)
%I_t=\int \frac {f_{\btheta_1,t}(X_t)}{f_{\btheta_0,t}(X_t)}\pi (d\btheta_1).
%\label{eq:bf_t}
%\end{equation}
%which is relevant for the study of the Bayes factors. 
For $t\geq 0$, let $\mathcal F_{t}$ be the $\sigma$-algebra generated by $X(0)$ and the history of the process upto 
(and including) time $t$, and let $\pi_t(\btheta_1) =\pi(\btheta_1|\mathcal F_t)$ be the posterior
of $\btheta_1$ given $\mathcal F_t$. Also, let
\begin{align}
&\hat f_{t-h,t}\left(X_{t-h,t}\right)\notag\\
&=\int_{\btheta_1\in\bTheta}
\exp\left(\int_{t-h}^t\frac{\phi_{\bxi_1}(s)b_{\bbeta_1}(s,X(s))}{\sigma^2(s,X(s))}dX(s)
-\frac{1}{2}\int_{t-h}^t\frac{\phi_{\bxi_1}^2(s)b_{\bbeta_1}^2(s,X(s))}{\sigma^2(s,X(s))}ds\right)
\pi_{t-h}(d\btheta_1)\notag\\
&=E\left[\exp\left(\int_{t-h}^t\frac{\phi_{\bxi_1}(s)b_{\bbeta_1}(s,X(s))}{\sigma^2(s,X(s))}dX(s)
-\frac{1}{2}\int_{t-h}^t\frac{\phi_{\bxi_1}^2(s)b_{\bbeta_1}^2(s,X(s))}{\sigma^2(s,X(s))}ds\right)
\bigg |\mathcal F_{t-h}\right]
\label{eq:postpred}
\end{align}
be the posterior predictive density.
Further, for any Borel set $A$ such that $\pi(A)>0$, let
\begin{align}
&\hat f_{t-h,t,A}\left(X_{t-h,t}\right)\notag\\
&=\int_{\btheta_1\in A}
\exp\left(\int_{t-h}^t\frac{\phi_{\bxi_1}(s)b_{\bbeta_1}(s,X(s))}{\sigma^2(s,X(s))}dX(s)
-\frac{1}{2}\int_{t-h}^t\frac{\phi_{\bxi_1}^2(s)b_{\bbeta_1}^2(s,X(s))}{\sigma^2(s,X(s))}ds\right)
\pi_{t-h,A}(d\btheta_1)\notag\\
&=E\left[\exp\left(\int_{t-h}^t\frac{\phi_{\bxi_1}(s)b_{\bbeta_1}(s,X(s))}{\sigma^2(s,X(s))}dX(s)
-\frac{1}{2}\int_{t-h}^t\frac{\phi_{\bxi_1}^2(s)b_{\bbeta_1}^2(s,X(s))}{\sigma^2(s,X(s))}ds\right)
\bigg |\mathcal F_{t-h},A\right],
\end{align}
where 
\[
\pi_{t,A}(d\btheta_1)=\frac{{\bI}_A(\btheta_1)\pi_t(d\btheta_1)}{\int_A \pi_t(d\btheta_1)}
\]
is the posterior restricted to the set $A$.
%Here, $h_{n^*}$ is such that $n^*>n_0$ is a term of a sequence 
%$\{h_n\}_{n=1}^\infty$ where $h_n\rightarrow 0$ as $n\rightarrow\infty$, that is, 
%$|h_n|<\epsilon$ for any $\epsilon>0$ whenever $n>n_0$, $n,n_0\in\mathbb N$.
We assume the following:
\begin{itemize}
\item[(H8$^{\prime\prime}$)]
\begin{equation}
%\underset{t}{\lim\inf}~\frac{\mathcal K(f_{\btheta_0,t-h,t},\hat f_{\btheta_1,t-h,t,A_t(\delta)})}{h}\geq\delta,
\underset{t}{\lim\inf}~E_{\btheta_0}\left[\tilde{\mathcal K}'_t\left(f_{\btheta_0},\hat f_{A_t(\delta)}\right)\right]\geq\delta,
\label{eq:Q_star_t}
\end{equation}
whenever 
\begin{equation}
%A_t(\delta)=\left\{\btheta_1:\frac{\mathcal K(f_{\btheta_0,t-h,t}, f_{\btheta_1,t-h,t})}{h}\geq\delta\right\}
A_t(\delta)=\left\{\btheta_1\in\bTheta:\tilde{\mathcal K}'_t(f_{\btheta_0}, f_{\btheta_1})\geq\delta\right\}.
\label{eq:A_t}
\end{equation}
\end{itemize}
We refer to (H8$^{\prime\prime}$) as the $Q^*$ property.


\subsection{Main result on convergence of Bayes factor when two individual $SDE$'s are compared}
\label{subsec:main_result}

Let $I_0\equiv 1$ and for $t> 0$, let us define, analogous to (\ref{eq:bf_T}),
\begin{equation}
%I_t=\int \frac {f_1(U_{1t},V_{1t})}{f_0(U_{0t},V_{0t})}\Pi (df)
I_t=\int \frac {f_{\btheta_1,t}(X_{0,t})}{f_{\btheta_0,t}(X_{0,t})}\pi (d\btheta_1).
\label{eq:bf_t}
\end{equation}
%which is relevant for the study of the Bayes factors. 
%Let $\mathcal F_{t-h}$ is the $\sigma$-algebra generated by $X(0)$ and the history of the process upto 
%(and including) time $t-h$, and let 
%\begin{equation}
%\hat f_{t-h,t}\left(X_{t-h,t}\right)
%=E\left[\exp\left(\zeta\left(\varphi_1\right)\int_{t-h}^t\frac{b_{\bbeta_1}(s,X(s))}{\sigma_{\bgamma_1}^2(s,X(s))}dX(s)
%-\frac{\zeta\left(\varphi_1\right)^2}{2}\int_{t-h}^t\frac{b_{\bbeta_1}^2(s,X(s))}{\sigma_{\bgamma_1}^2(s,X(s))}ds\right)
%\bigg |\mathcal F_{t-h}\right],
%\label{eq:postpred}
%\end{equation}
%where the above expectation is with respect to the posterior distribution of $\btheta_1$ given $\mathcal F_{t-h}$.
The following lemma, proved in section S-4 of the supplement, will prove useful in proving our main theorem on convergence of Bayes factor.
\begin{lemma}
\label{lemma:towards_martingale}
\begin{align}
E_{\btheta_0}\left[\log \frac{I_t}{I_{t-h}}\bigg |\mathcal F_{t-h}\right]
&=E_{\btheta_0}\left[\log \frac{\hat f_{t-h,t}(X_{t-h,t})}{f_{\btheta_0,t-h,t}(X_{t-h,t})}\bigg |\mathcal F_{t-h}\right]
= -\tilde{\mathcal K}(f_{\btheta_0,t-h,t},\hat f_{t-h,t}).
\label{eq:bf_ratio2}
\end{align}
\end{lemma}

\begin{comment}
\begin{proof}
For any $h\in (0,t)$,
\begin{align}
\frac{I_t}{I_{t-h}} 
&=\frac{\int_{\bTheta} \frac{f_{\btheta_1,t}(X_{0,t})}{f_{\btheta_0,t}(X_{0,t})}\pi (d\btheta_1)}
{\int_{\bTheta} \frac{f_{\btheta_1,t}(X_{0,t-h})}{f_{\btheta_0,t}(X_{0,t-h})}\pi (d\btheta_1)}\notag\\
&=\frac{\int_{\bTheta} \exp\left (\left(U_{\btheta_1,t} 
- U_{\btheta_0,t}\right)-\frac{\left(V_{\btheta_1,t}
-V_{\btheta_0,t}\right)}{2}\right) \pi(d\btheta_1)}
{\int_{\bTheta} \exp \left(\left(U_{\btheta_1,t-h} 
- U_{\btheta_0,t-h}\right)-\frac{\left(V_{\btheta_1,t-h}
-V_{\btheta_0,t-h}\right)}{2}\right) \pi(d\btheta_1)}\notag\\
&=\int_{\bTheta} \exp\left(\left(U_{\btheta_1,t-h} 
- U_{\btheta_0,t-h}\right)-\frac{\left(V_{\btheta_1,t-h}
-V_{\btheta_0,t-h}\right)}{2}\right)\notag\\
&\quad\times\frac{\exp\left(U_{\btheta_1,t-h,t}-U_{\btheta_0,t-h,t}
-\frac{\left(V_{\btheta_1,t-h,t}
-V_{\btheta_0,t-h,t}\right)}{2}\right)\pi(d\btheta_1)}
{\int_{\bTheta}\exp\left(\left(U_{\btheta_1,t-h} - U_{\btheta_0,t-h}\right)
-\frac{(V_{\btheta_1,t-h}-V_{\btheta_0,t-h})}{2}\right)\pi(d\btheta_1)}\notag\\
&=E\left[\exp\left(\int_{t-h}^t\frac{\phi_{\bxi_1}(s)b_{\bbeta_1}(s,X(s))}{\sigma^2(s,X(s))}dX(s)
-\int_{t-h}^t\frac{\phi_{\bxi_0}(s)b_{\bbeta_0}(s,X(s))}{\sigma^2(s,X(s))}dX(s)\right.\right.\notag\\
&\left.\left.\quad-\left(\frac{1}{2}
\int_{t-h}^t\frac{\phi_{\bxi_1}^2(s)b_{\bbeta_1}^2(s,X(s))}{\sigma^2(s,X(s))}ds
-\frac{1}{2}\int_{t-h}^t\frac{\phi_{\bxi_0}^2(s)b_{\bbeta_0}^2(s,X(s))}{\sigma^2(s,X(s))}ds\right)\right)
\bigg\vert\mathcal F_{t-h}\right]\notag\\
&=\frac{E\left[\exp\left(\int_{t-h}^t\frac{\phi_{\bxi_1}(s)b_{\bbeta_1}(s,X(s))}{\sigma^2(s,X(s))}dX(s)
-\frac{1}{2}\int_{t-h}^t\frac{\phi_{\bxi_1}^2(s)b_{\bbeta_1}^2(s,X(s))}{\sigma^2(s,X(s))}ds\right)
\bigg |\mathcal F_{t-h}\right]}{\exp\left(\int_{t-h}^t\frac{\phi_{\bxi_0}(s)b_{\bbeta_0}(s,X(s))}{\sigma^2(s,X(s))}dX(s)
-\frac{1}{2}\int_{t-h}^t\frac{\phi_{\bxi_0}^2(s)b_{\bbeta_0}^2(s,X(s))}{\sigma^2(s,X(s))}ds\right)}\notag\\
&=\frac{\hat f_{t-h,t}(X_{t-h,t})}{f_{\btheta_0,t-h,t}(X_{t-h,t})}.
\label{eq:ration_I_t}
\end{align}
Hence, (\ref{eq:bf_ratio2}) holds.
\end{proof}
\end{comment}
%
%We now assume mean square differentiability of $\log I_t$; we follow \ctn{Stein99} for the definition.
%\begin{itemize}
%\item[(H11$^\prime$)] We assume that %$\log I_t$ is mean square differentiable, that is,
%$J_{h_n}(t)=\frac{\log I_{t+h_n}-\log I_t}{h_n}$ converges in expectation  
%%\begin{equation*}
%%\underset{n\rightarrow\infty}{\lim}E\left(\frac{\log I_{t+h_n}-\log I_t}{_nh}\right)^2
%%\label{eq:msd}
%%\end{equation*}
%for all sequences $\{h_n\}$ converging to zero as $n\rightarrow\infty$, with limit independent
%of $\{h_n\}$. We refer to the limiting process as $\frac{d\log I_t}{dt}=J'_t$. In other words, 
%for any $t\geq 0$, 
%\begin{equation}
%\underset{n\rightarrow\infty}{\lim}~E\left(J_{h_n}(t)\right)=E\left(J'_t\right).
%\label{eq:msd}
%\end{equation}
%\end{itemize}

We make the following further assumption:
\begin{itemize}
\item[(H9$^{\prime\prime}$)]
For any $t\geq 0$, $\tilde{\mathcal K}\left(f_{\btheta_0,t,t+h_n},\hat f_{t,t+h_n}\right)$
converges in expectation 
for all sequences $\{h_n\}$ converging to zero as $n\rightarrow\infty$, with limit independent
of $\{h_n\}$. We refer to the limiting process as $\tilde{\mathcal K}'_t$. In other words, 
\begin{equation}
\underset{n\rightarrow\infty}{\lim}~
E\left(\frac{\tilde{\mathcal K}\left(f_{\btheta_0,t,t+h_n},\hat f_{t,t+h_n}\right)}{h_n}\right)
=E\left(\tilde{\mathcal K}'_t\left(f_{\btheta_0},\hat f\right)\right),
\label{eq:msd2}
\end{equation}
for any sequence $\{h_n\}$ such that $h_n\rightarrow 0$ as $n\rightarrow\infty$.
\end{itemize}
Because of Lemma \ref{lemma:towards_martingale} it follows from (H9$^{\prime\prime}$), using uniform integrability 
(which is easily seen to hold because of (H1$^{\prime\prime}$) -- (H4$^{\prime\prime}$) and (\ref{eq:moment1_T})), that
$J_{h_n}(t)=\frac{\log I_{t+h_n}-\log I_t}{h_n}$ converges in expectation  
%\begin{equation*}
%\underset{n\rightarrow\infty}{\lim}E\left(\frac{\log I_{t+h_n}-\log I_t}{_nh}\right)^2
%\label{eq:msd}
%\end{equation*}
for all sequences $\{h_n\}$ converging to zero as $n\rightarrow\infty$, with limit independent
of $\{h_n\}$. We refer to the limiting process as $J'_t$. That is, 
for any $t\geq 0$, 
\begin{equation}
\underset{n\rightarrow\infty}{\lim}~E\left(J_{h_n}(t)\right)=E\left(J'_t\right)=\frac{d}{dt}E(\log I_t).
\label{eq:msd}
\end{equation}
Now,
\begin{equation}
\hat f_{t,t+h}
=E\left[\exp\left(\int_{t}^{t+h}\frac{\phi_{\bxi_1}(s)b_{\bbeta_1}(s,X(s))}{\sigma^2(s,X(s))}dX(s)
-\frac{1}{2}\int_{t}^{t+h}\frac{\phi_{\bxi_1}^2(s)b_{\bbeta_1}^2(s,X(s))}{\sigma^2(s,X(s))}ds\right)
\bigg |\mathcal F_{t}\right],
\label{eq:postpred2}
\end{equation}
so that Lemma \ref{lemma:towards_martingale} implies 
\begin{align}
E\left[\log \frac{I_{t+h}}{I_{t}}\bigg |\mathcal F_{t}\right]
&=E\left[\log \frac{\hat f_{t,t+h}(X_{t,t+h})}{f_{\btheta_0,t,t+h}(X_{t,t+h})}\bigg |\mathcal F_{t}\right]
= -\tilde{\mathcal K}(f_{\btheta_0,t,t+h},\hat f_{t,t+h}).
\label{eq:bf_ratio3}
\end{align}
It follows, using (H9$^{\prime\prime}$), %that and (H12$^\prime$), 
that
\begin{equation}
E\left(J'_t|\mathcal F_{t}\right)=-\tilde{\mathcal K}'_t\left(f_{\btheta_0},\hat f\right).
\label{eq:towards_martingale2}
\end{equation}
Note that for all sequences $\{h_n\}$ such that $h_n\rightarrow 0$ as $n\rightarrow\infty$, 
$J_{h_n}(t)=\frac{\log I_{t+h_n}-\log I_t}{h_n}$ is measurable with respect to 
$\mathcal F_{t^*}=\sigma\left(X(s):0\leq s\leq t^*\right)$, for all $t^*>t\geq 0$.
Hence, $E\left(J'_t|\mathcal F_{t}\right)\neq J'_t$.

Regarding convergence of $I_T$, we are now ready to present our main theorem whose proof is provided in section S-5 of the supplement.
\begin{theorem}
\label{theorem:bf_convergence}
Assume the $SDE$ set-up and conditions (H1$^{\prime\prime}$) -- (H9$^{\prime\prime}$).
%Let $\Pi$ be a prior distribution such that it has Kullback-Leibler$(\delta)$ property and $Q^*$ proerty. 
%Also assume 
%\begin{equation}
%%\int_0^\infty t^{-2} Var\left[\frac{\log\frac{I_{t+h_{n^*}}}{I_t}}{h_{n^*}}\right]dt<\infty
%\int_0^\infty t^{-2} Var\left(J'_t\right)dt<\infty.
%\label{eq:convergent_series1}
%\end{equation}
%where $h_{n^*}$ such that $n^*>n_0$ is a term of a sequence $\{h_n\}_{n=1}^\infty$ 
%where $h_n\rightarrow 0$ as $n\rightarrow\infty$ i.e $|h_n|<\epsilon$ 
%for any $\epsilon>0$ whenever $n>n_0$, $n,n_0\in\mathbb N$. Define $I_0=1$.
Then 
\begin{equation}
\frac{1}{T}E_{\btheta_0}\left(\log I_T\right)\rightarrow -\delta, 
\label{eq:bf_convergence}
\end{equation}
but
\begin{equation}
\frac{1}{T^2}Var_{\btheta_0}\left(\log I_T\right)=O(1), 
\label{eq:bf_var}
\end{equation}
as $T\rightarrow\infty$.
%In other words, $\frac{1}{T}\log I_T$ is inconsistent.
\end{theorem}

\begin{comment}
\begin{proof}
% Let us consider the sequence
%\[
%%S_{T}=\int_0^{T}\frac{d}{dt}\left(\log I_t\right)dt +\int_0^T\frac{d}{dt}\mathcal K(f_{0,t-h,t},\hat f_{1,t-h,t})dt
%S_{T}=\int_0^{T}J'_tdt +\int_0^T\tilde{\mathcal K}'_t(f_{\btheta_0},\hat f)dt
%\]
%By Riemann-sum approximation $S_T$ is given by 
Let us consider
\begin{equation}
S_{Tq_n}=Tq_n\sum_{r=0}^{n(T)-1}\left(J'_{rTq_n}+\tilde{\mathcal K}'_{rTq_n}\right), 
\label{eq:Rsum}
\end{equation}
where $q_n=\frac{1}{n(T)}$, where, given $T>0$, $n(T)$ is the 
number of intervals partitioning $[0,T]$ each of length $\frac{T}{n(T)}$. We assume that as $T\rightarrow\infty$,
$\frac{T}{n(T)}\rightarrow 0$.
%\[
%f(rTq_n)=\lim_{h_n\rightarrow 0}\frac{\log I_{rTq_n+h_n}-\log I_{rTq_n}}{h_n}
%\]
%and
%\[
%g(rTq_n)=\lim_{h_n\rightarrow 0}\frac{\mathcal K(f_{0,rTq_n,rTq_n+h_n},\hat f_{1,rTq_n,rTq_n+h_n})}{h_n}
%\]
%Define,
%\[
%f^*(rTq_n)=\frac{\log I_{rTq_n+h_{n^*}}-\log I_{rTq_n}}{h_{n^*}}
%\]
%and
%\[
%g^*(rTq_n)=\frac{\mathcal K(f_{0,rTq_n,rTq_n+h_{n^*}},\hat f_{1,rTq_n,rTq_n+h_{n^*}})}{h_{n^*}}
%\]
%where $h_{n^*}$ is chosen as mentioned in the statement of the theorem.\\[2mm]
%Consider,
%\[
%S^*_{T,q_n}=Tq_n\sum_{r=0}^{n-1}[f^*(rTq_n)+g^*(rTq_n)]
%\]
%
%
%which implies
%\begin{equation}
%E\left(\tilde S_{Tq_n}\right)=0
%\end{equation}

%where $\mathcal F_{rTq_n}$ is the $\sigma$-algebra generated by $X(0)$ and history of the 
%process upto (and including) time $rTq_n$.
%Now consider
%\[
%%\tilde S_{T,q_n}=Tq_n \sum_{r=0}^{n-1}\frac{f^*(rTq_n)+g^*(rTq_n)}{rTq_n}
%\tilde S_{Tq_n}=Tq_n\sum_{r=0}^{n(T)-1}\frac{\left(J'_{rTq_n}+\tilde{\mathcal K}'_{rTq_n}\right)}{rTq_n}. 
%\]


%Then, this is also a martingale because $S^*_{Tq_n}$ is so.
%Note that, 
%\begin{align}
%E|\tilde S_{T,q_n}|&\leq \sqrt{E[\tilde S_{T,q_n}]^2}\notag\\
%&=\sqrt{E\left[T^2q^2_n\left\{\sum_{r=0}^{n(T)-1}
%\frac{\left(J'_{rTq_n}+\tilde{\mathcal K}'_{rTq_n}\right)}{rTq_n}\right\}^2\right]}
%\label{eq:martingale_upper_bound}
%\end{align}
%Now, $E[J'_{rTq_n}|\mathcal F_{rTq_n}]=-\tilde{\mathcal K}'_{rTq_n}$ implies 
%%$E[f^*(rTq_n)+g^*(rTq_n)|\mathcal F_{rTq_n}]=0$.\\[2mm]
%%Hence, 
%$E\left[J'_{rTq_n}+\tilde{\mathcal K}'_{rTq_n}\right]
%=E\left[E\left\{J'_{rTq_n}+\tilde{\mathcal K}'_{rTq_n}|\mathcal F_{rTq_n}\right\}\right]=0$.
%Then observe that
%\begin{align}
%E\left[J'_{rTq_n}+\tilde{\mathcal K}'_{rTq_n}\right]^2&=Var\left[J'_{rTq_n}+\tilde{\mathcal K}'_{rTq_n}\right]\notag\\
%&=Var\left[E\left\{J'_{rTq_n}+\tilde{\mathcal K}'_{rTq_n}|\mathcal F_{rTq_n}\right\}\right]
%+E\left[Var\left\{J'_{rTq_n}+\tilde{\mathcal K}'_{rTq_n}|\mathcal F_{rTq_n}\right\}\right]\notag\\
%&=0+E\left[Var\left\{J'_{rTq_n}+\tilde{\mathcal K}'_{rTq_n}|\mathcal F_{rTq_n}\right\}\right]\notag\\
%&=E\left[Var\left\{J'_{rTq_n}|\mathcal F_{rTq_n}\right\}\right]\notag\\
%&\leq Var\left(J'_{rTq_n}\right).
%\label{eq:variance}
%\end{align}
%The second last equality follows because $\tilde{\mathcal K}'_{rTq_n}$ is measurable with respect to
%$\mathcal F_{rTq_n}$ (as $\tilde{\mathcal K}'_{rTq_n}=-E[J'_{rTq_n}|\mathcal F_{rTq_n}]$), 
%and so, it is constant, given $\mathcal F_{rTq_n}$.
%%
%Now consider $r_1,r_2\in\{0,1,2,\ldots\}$ such that $r_1<r_2$.
%For $j=1,2$, let $L_{r_i}=J'_{r_iTq_n}+\tilde{\mathcal K}'_{r_iTq_n}$.
%In (H12$^\prime$) and (H13$^\prime$), we consider all sequences $\{h_n\}$ converging to zero 
%only in an $\epsilon_n$-neighborhood of zero, where $\{\epsilon_n\}$ is a non-negative sequence
%converging to zero as $n\rightarrow\infty$ such that
%$\mathcal F_{r_1Tq_n+\epsilon_n}\subseteq\mathcal F_{r_2Tq_n}$, for all $n$, $T>0$, and for all integers $r_1<r_2$.
%In that case, since $L_{r_1}$ is measurable with respect to $\mathcal F_{r_1Tq_n+\epsilon_n}$, it is
%measurable with respect to $F_{r_2Tq_n}$.
%Then,
%\begin{align}
%Cov\left(L_{r_1},L_{r_2}\right)&=E\left[L_{r_1}L_{r_2}\right]-E\left[L_{r_1}\right]E\left[L_{r_2}\right]\notag\\
%&=E\left[L_{r_1}L_{r_2}\right]-E\left[L_{r_1}\right]E\left[E\left\{L_{r_2}|\mathcal F_{r_2Tq_n}\right\}\right]\notag\\
%&=E\left[L_{r_1}L_{r_2}\right]-E\left[L_{r_1}\right]\times 0\notag\\
%&=E\left[E\left\{L_{r_1}L_{r_2}|\mathcal F_{r_2Tq_n}\right\}\right]\notag\\
%&=E\left[L_{r_1}E\left\{L_{r_2}|\mathcal F_{r_2Tq_n}\right\}\right]\quad\mbox{(since $L_{r_1}$ is $F_{r_2Tq_n}$-measurable)}
%\notag\\
%&=E\left[L_{r_1}\times 0\right]\notag\\
%&=0.
%\label{eq:covariance}
%\end{align}
%Hence, from (\ref{eq:martingale_upper_bound}), (\ref{eq:variance}), and (\ref{eq:covariance}), it follows that
%\begin{align}
%E|\tilde S_{Tq_n}|&\leq \sqrt{T^2q^2_n\sum_{r=0}^{n(T)-1}\frac{ Var\left(J'_{rTq_n}\right)}{r^2T^2q^2_n}}\notag\\
%&=\sqrt{Tq_n\times Tq_n\sum_{r=0}^{n(T)-1}\frac{ Var\left(J'_{rTq_n}\right)}{r^2T^2q^2_n}}
%\label{eq:martingale_upper_bound2}
%\end{align}
%Now, as $n\rightarrow\infty$, 
%\begin{equation}
%Tq_n\sum_{r=0}^{n(T)-1}\frac{ Var\left(J'_{rTq_n}\right)}{r^2T^2q^2_n}
%\sim\int_0^{\infty} \frac{Var(J'_t)}{t^2}dt,
%\label{eq:Rsum2}
%\end{equation}
%where, for any two sequences $\{a_n\}$ and $\{b_n\}$, $a_n\sim b_n$ denotes $a_n/b_n\rightarrow 1$, as
%$n\rightarrow\infty$.
%Because $n(T)$ is the number of intervals partitioning $[0,T]$ each of length $\frac{T}{n(T)}$, 
%we have $Tq_n=\frac{T}{n(T)}<1$, it follows from (\ref{eq:martingale_upper_bound2}) and (\ref{eq:Rsum2})
%that for any $\epsilon>0$, for any $T>0$, there exists $n_0(T,\epsilon)$ such that for $n(T)>n_0(T,\epsilon)$,
%\begin{equation}
%E|\tilde S_{Tq_n}|\leq \sqrt{\int_0^{\infty} \frac{Var(J'_t)}{t^2}dt}
%\label{eq:martingale_upper_bound3}
%\end{equation}
%Hence, by the martingale convergence theorem, 
%if $\int_0^{\infty} \frac{Var(J'_t)}{t^2}dt<\infty$ then $\tilde S_{Tq_n}$ 
%converges almost surely as $T\rightarrow \infty$.
%We denote the limit of $\tilde S_{Tq_n}$ by $\tilde S_{\infty}$, that is,
%\[
%Tq_n \sum_{r=0}^{n(T)-1}\frac{\left(J'_{rTq_n}+\tilde{\mathcal K}'_{rTq_n}\right)}{rTq_n}\rightarrow\tilde S_{\infty},
%\]
%almost surely, as $T\rightarrow \infty$.
%Hence, by Kronecker's Lemma,
%\[
% Tq_n \sum_{r=0}^{n(T)-1}\frac{\left(J'_{rTq_n}+\tilde{\mathcal K}'_{rTq_n}\right)}{(n(T)-1)Tq_n}\rightarrow 0,
%\]
%almost surely, as $T\rightarrow \infty$.
%But, $(n(T)-1)q_n=\frac{n(T)-1}{n(T)}\sim 1$ as $T\rightarrow\infty$.
%Now, again using Riemann sum approximation
%\begin{align}
%S^*_{T,q_n}&=Tq_n\sum_{r=0}^{n(T)-1}\left(J'_{rTq_n}+\tilde{\mathcal K}'_{rTq_n}\right)\notag\\
%&\sim \int_0^T\left(J'_t+\tilde{\mathcal K}'_t\right)dt\notag\\
%\end{align}
%For $n>n_0$, $S^*_{T,q_n}\sim S_T=\log I_T+\int_0^T g^*(t)dt$.\\
%From the result $\frac{S^*_{T,q_n}}{T}\stackrel{a.s.}{\rightarrow}0$, we have for $n>n_0$, $\frac{S_T}{T}\stackrel{a.s.}{\rightarrow}0$.\\
%i.e. $\frac{\log I_T}{T}+\frac{\int_0^T g^*(t)}{T}dt\stackrel{a.s.}{\rightarrow}0$ as $T\rightarrow\infty$.\\


It follows, using (H9$^{\prime\prime}$), %and (H12$^\prime$), 
that for any $T>0$, 
\begin{align}
E\left(\frac{S_{Tq_n}}{T}\right)&\rightarrow \frac{1}{T}\int_0^T\frac{d}{dt}E_{\btheta_0}\left(\log I_t\right)dt
+\frac{1}{T}\int_0^TE_{\btheta_0}\left[\tilde{\mathcal K}'_t(f_{\btheta_0},\hat f)\right]dt\notag\\
&=E_{\btheta_0}\left(\frac{1}{T}\log I_T\right)
+\frac{1}{T}\int_0^TE_{\btheta_0}\left[\tilde{\mathcal K}'_t(f_{\btheta_0},\hat f)\right]dt,
\label{eq:conv1}
\end{align}
as $n(T)\rightarrow\infty$, for any given $T>0$.
Also, since due to (\ref{eq:towards_martingale2}),  
%$E[J'_{rTq_n}|\mathcal F_{rTq_n}]=-\tilde{\mathcal K}'_{rTq_n}\left(f_{\btheta_0},\hat f\right)$, 
$E\left(J'_{rTq_n}+\tilde{\mathcal K}'_{rTq_n}|\mathcal F_{rTq_n}\right)=0$,
we must have $E\left(J'_{rTq_n}+\tilde{\mathcal K}'_{rTq_n}\right)
=E\left[E\left(J'_{rTq_n}+\tilde{\mathcal K}'_{rTq_n}|\mathcal F_{rTq_n}\right)\right]=0$, for any $r,T,n(T)$.
Hence, $E\left(\frac{S_{Tq_n}}{T}\right)=0$ for any $T,n(T)$. 
Thus, it follows from (\ref{eq:conv1}), that
\begin{equation}
E_{\btheta_0}\left(\frac{1}{T}\log I_T\right)
+\frac{1}{T}\int_0^TE_{\btheta_0}\left[\tilde{\mathcal K}'_t(f_{\btheta_0},\hat f)\right]dt
\rightarrow 0,\quad\mbox{as}~T\rightarrow\infty.
\label{eq:conv2}
\end{equation}
We now deal with the second term of the left hand side of (\ref{eq:conv2}).
Since, by (H5$^{\prime\prime}$), %the Kullback-Leibler property (\ref{eq:kl_property_t}) that
$$\pi\left(\btheta_1: \underset{t}{\inf}~\tilde{\mathcal K}'_t(f_{\btheta_0},f_{\btheta_1})\geq \delta\right)=1,$$
it holds that $\tilde{\mathcal K}'_t(f_{\btheta_0},f_{\btheta_1})\geq \delta$ for all $t$ with probability 1,
so that 
\[
%g^*(t)=\frac{\mathcal K(f_{0,t,t+h_{n^*}},\hat f_{1,t,t+h_{n^*}})}{h_{n^*}}
%=\frac{\mathcal K(f_{0,t,t+h_{n^*}},\hat f_{1,t,t+h_{n^*},A_t(\delta)})}{h_{n^*}}
\tilde{\mathcal K}'_t(f_{\btheta_0},\hat f)=\tilde{\mathcal K}'_t(f_{\btheta_0},\hat f_{A_t(\delta)}),
\]
where $A_t(\delta)$ is given by (\ref{eq:A_t}). 
The $Q^*$ property (\ref{eq:Q_star_t}) implies that  
\begin{equation}
\underset{T}{\lim\inf}~ \frac{1}{T}\int_0^TE_{\btheta_0}\left[\tilde{\mathcal K}'_t(f_{\btheta_0},\hat f)\right]dt\geq \delta.
\label{eq:liminf_K_average_T}
\end{equation}
The results (\ref{eq:conv2}) and (\ref{eq:liminf_K_average_T}) imply that
\begin{align}
\underset{T}{\limsup}~ E_{\btheta_0}\left(\frac{1}{T}\log I_T\right)\leq -\delta.
\label{eq:limsup_T}
\end{align}
%
%Consider
%\begin{align}
%I_T=&\int \exp\left(\zeta\left(\varphi_1\right)\int_0^T\frac{b_{\bbeta_1}(s,X(s))}{\sigma_{\bgamma_1}^2(s,X(s))}dX_s
%-\zeta\left(\varphi_0\right)\int_0^T\frac{b_{\bbeta_0}(s,X(s))}{\sigma_{\bgamma_0}^2(s,X(s))}dX_s\right)\notag\\
%&\times\exp\left(-\frac{\zeta\left(\varphi_1\right)^2}{2}\int_0^T\frac{b_{\bbeta_1}^2(s,X(s))}{\sigma_{\bgamma_1}^2(s,X(s))}ds
%+\frac{\zeta\left(\varphi_0\right)^2}{2}\int_0^T\frac{b_{\bbeta_0}^2(s,X(s))}{\sigma_{\bgamma_0}^2(s,X(s))}ds\right)
%\pi(\btheta_1)d\btheta_1\notag\\
%=&\int \exp\left(\varphi\left(\varphi_1\right)\int_0^T\frac{b_{\bbeta_1}(s,X(s))}{\sigma_{\bgamma_1}(s,X(s))}dW_s
%-\zeta\left(\varphi_0\right)\int_0^T\frac{b_{\bbeta_0}(s,X(s))}{\sigma_{\bgamma_0}(s,X(s))}dW(s)\right)\notag\\
%&\times\exp\left(\frac{\zeta\left(\varphi_1\right)^2}{2}\int_0^T\frac{b_{\bbeta_1}^2(s,X(s))}{\sigma_{\bgamma_1}^2(s,X(s))}ds
%-\frac{\zeta\left(\varphi_0\right)^2}{2}\int_0^T\frac{b_{\bbeta_0}^2(s,X(s))}{\sigma_{\bgamma_0}^2(s,X(s))}ds\right)
%\pi(\btheta_1)d\btheta_1\notag\\
%\geq&\int_{\mathcal N_0(c_T)} \exp\left(\varphi\left(\varphi_1\right)
%\int_0^T\frac{b_{\btheta_1}(s,X(s))}{\sigma_{\bgamma_1}(s,X(s))}dW(s)-\zeta\left(\varphi_0\right)
%\int_0^T\frac{b_{\bbeta_0}(s,X(s))}{\sigma_{\bgamma_0}(s,X(s))}dW(s)\right)\times\notag\\
%&\exp\left(\frac{\zeta\left(\varphi_1\right)^2}{2}\int_0^T\frac{b_{\bbeta_1}^2(s,X(s))}{\sigma_{\bgamma_1}^2(s,X(s))}ds
%-\frac{\zeta\left(\varphi_0\right)^2}{2}\int_0^T\frac{b_{\bbeta_0}^2(s,X(s))}{\sigma_{\bgamma_0}^2(s,X(s))}ds\right)
%\pi(\btheta_1)d\btheta_1,
%\label{eq:lower_bound1}
%\end{align}
Now observe that
\begin{align}
I_T&=\int_{\bTheta} \exp\left(U_{\btheta_1,T}
-U_{\btheta_0,T}\right)
\times\exp\left\{-\frac{1}{2}\left(V_{\btheta_1,T}
-V_{\btheta_0,T}\right)\right\}
\pi(d\btheta_1)\notag\\
&\geq 
\int_{\mathcal N_0(c)}\exp\left\{T\left(\frac{U_{\btheta_1,T}}{T}
-\frac{U_{\btheta_0,T}}{T}\right)\right\}\notag\\
&\quad\quad\times\exp\left\{-\frac{T}{2}\left(\frac{V_{\btheta_1,T}}{T}
-\frac{V_{\btheta_0,T}}{T}\right)\right\}
\pi(d\btheta_1),
\label{eq:lower_bound1}
\end{align}
%
%=&\int \exp\left(\varphi\left(\varphi_1\right)\int_0^T\frac{b_{\bbeta_1}(s,X(s))}{\sigma_{\bgamma_1}(s,X(s))}dW_s
%-\zeta\left(\varphi_0\right)\int_0^T\frac{b_{\bbeta_0}(s,X(s))}{\sigma_{\bgamma_0}(s,X(s))}dW(s)\right)\notag\\
%&\times\exp\left(\frac{\zeta\left(\varphi_1\right)^2}{2}\int_0^T\frac{b_{\bbeta_1}^2(s,X(s))}{\sigma_{\bgamma_1}^2(s,X(s))}ds
%-\frac{\zeta\left(\varphi_0\right)^2}{2}\int_0^T\frac{b_{\bbeta_0}^2(s,X(s))}{\sigma_{\bgamma_0}^2(s,X(s))}ds\right)
%\pi(\btheta_1)d\btheta_1\notag\\
%\geq&\int_{\mathcal N_0(c_T)} \exp\left(\varphi\left(\varphi_1\right)
%\int_0^T\frac{b_{\btheta_1}(s,X(s))}{\sigma_{\bgamma_1}(s,X(s))}dW(s)-\zeta\left(\varphi_0\right)
%\int_0^T\frac{b_{\bbeta_0}(s,X(s))}{\sigma_{\bgamma_0}(s,X(s))}dW(s)\right)\times\notag\\
%&\exp\left(\frac{\zeta\left(\varphi_1\right)^2}{2}\int_0^T\frac{b_{\bbeta_1}^2(s,X(s))}{\sigma_{\bgamma_1}^2(s,X(s))}ds
%-\frac{\zeta\left(\varphi_0\right)^2}{2}\int_0^T\frac{b_{\bbeta_0}^2(s,X(s))}{\sigma_{\bgamma_0}^2(s,X(s))}ds\right)
%\pi(\btheta_1)d\btheta_1,
%\label{eq:lower_bound1}
%\end{align}
where $c>0$, and
\begin{align}
\mathcal N_0(c)&=\left\{\btheta_1\in\bTheta:\delta\leq\bar{K}^{\infty}\left(f_{\btheta_0},f_{\btheta_1}\right)
\leq\delta+c\right\}\notag\\
&=\left\{\btheta_1\in\bTheta:\delta\leq\frac{\bar\phi^{(2)}_{\bxi_0}}{2}\kappa_0(\bbeta_0)
-\bar\phi^{(2)}_{\bxi_0,\bxi_1}\bar{\kappa}(\bbeta_0,\bbeta_1)
+\frac{\bar\phi^{(2)}_{\bxi_1}}{2}\kappa_1(\bbeta_1)
\leq\delta+c\right\},\notag
\end{align}
the second line following from (\ref{eq:kl_average_infinite2}).
Using Jensen's inequality, we obtain
\begin{align}
\frac{1}{T}\log\left(I_T\right)&\geq 
\int_{\mathcal N_0(c)}\left[\left(\frac{U_{\btheta_1,T}}{T}
-\frac{U_{\btheta_0,T}}{T}\right)\right.\notag\\
&\quad\quad\left.-\frac{1}{2}\left(\frac{V_{\btheta_1,T}}{T}
-\frac{V_{\btheta_0,T}}{T}\right)\right]
\pi(d\btheta_1).
\label{eq:lower_bound2}
\end{align}
By (vi) -- (ix) of Lemma \ref{lemma:convergence1}, the integrand of the right hand side of the above
inequality, which we denote by $g_{X_T}(\btheta_1)$, converges to 
$g(\btheta_1)=-\left[\frac{\bar\phi^{(2)}_{\bxi_1}}{2}\kappa_1(\bbeta_1)
-\bar\phi^{(2)}_{\bxi_0,\bxi_1}\bar{\kappa}(\bbeta_0,\bbeta_1)
+\frac{\bar\phi^{(2)}_{\bxi_0}}{2}\kappa_0(\bbeta_0)\right]$, pointwise for every $\btheta_1$, given any 
path of the process $X$ in the complement of the null set. 
Due to (H1$^{\prime\prime}$), (H4$^{\prime\prime}$) and (\ref{eq:moment1_T}), 
$\underset{T}{\sup}~E_{\btheta_1}\left[g_{X_T}(\btheta_1)\right]^2<\infty$,
so that $\left\{g_{X_T}(\btheta_1)\right\}_{T=1}^{\infty}$ is uniformly integrable. Hence,
$$\int_{\mathcal N_0(c)}g_{X_T}(\btheta_1)\pi(d\btheta_1)\rightarrow 
\int_{\mathcal N_0(c)}g(\btheta_1)\pi(d\btheta_1),$$
given any path of the process $X$ in the complement of the null set.
Let us denote the left hand side of the above by $H_{X_T}$ let $H$ denote the right hand side.
We just proved that $H_{X_T}$ converges to $H$ almost surely. Now observe that
\begin{align}
\underset{T}{\sup}~E_{\btheta_0}\left[H_{X_T}\right]^2
&=\underset{T}{\sup}~E_{\btheta_0}\left[\int_{\mathcal N_0(c)}g_{X_T}(\btheta_1)\pi(d\btheta_1)\right]^2\notag\\
&\leq \int_{\mathcal N_0(c)}\underset{T}{\sup}~E_{\btheta_0}\left[g_{X_T}(\btheta_1)\right]^2\pi(d\btheta_1).\notag
%&\leq \int_{\mathcal N_0(c)}E_{\btheta_0}\left[\underset{T}{\sup}~g_{X_T}(\btheta_1)\right]^2\pi(d\btheta_1).\notag
\end{align}
Again, due to (H1$^{\prime\prime}$), (H4$^{\prime\prime}$) and (\ref{eq:moment1_T}), the last expression is finite, 
proving uniform integrability
of $\left\{H_{X_T}\right\}_{T=1}^{\infty}$. Hence, 
$$\underset{T\rightarrow\infty}{\lim}~E_{\btheta_0}\left(H_{X_T}\right)= E_{\btheta_0}\left(H\right).$$
It follows that
\begin{align}
&\underset{T\rightarrow\infty}{\liminf}~E_{\btheta_0}\left[\frac{1}{T}\log\left(I_T\right)\right]
\geq \underset{T\rightarrow\infty}{\liminf}~E_{\btheta_0}\left(H_{X_T}\right)
=E_{\btheta_0}\left(H\right)\notag\\
&=\int_{\mathcal N_0(c)}g(\btheta_1)\pi(\btheta_1)d\btheta_1\notag\\
&\geq-\left(\delta+c\right)\pi\left(\mathcal N_0(c)\right)\notag\\
&\geq-\left(\delta+c\right).
\end{align}
Since the above holds for arbitrary $c>0$,
it holds that
\begin{equation}
\underset{T\rightarrow\infty}{\lim\inf}~E_{\btheta_0}\left(\frac{1}{T}\log I_T\right)\geq -\delta.
\label{eq:liminf_T}
\end{equation}
Thus (\ref{eq:limsup_T}) and (\ref{eq:liminf_T}) together help us conclude that 
$$E_{\btheta_0}\left(\frac{1}{T}\log I_T\right)\rightarrow-\delta,$$ as $T\rightarrow\infty$.



We now show that the variance of $\frac{1}{T}\log I_T$ is $O(1)$, as $T\rightarrow\infty$.
First note, due to compactness of $\bTheta$, the mean value theorem for integrals
ensure existence of $\acute\btheta_1=(\acute\bbeta_1,\acute\bxi_1)=\acute\btheta_1(W)\in\bTheta$, depending 
on the Wiener process $W$ %as the system has strong solution, 
such that 
\begin{equation}
\log I_T=\left(U_{\acute\btheta_1,T}
-U_{\btheta_0,T}\right)
-\frac{1}{2}\left(V_{\acute\btheta_1,T}
-V_{\btheta_0,T}\right).
\label{eq:log_I_T}
\end{equation}

Now note that the results presented in Lemma 16 of MB continue to hold even when 
$\btheta_1$ is replaced with $\acute\btheta_1$. Specifically, the following hold 
in addition to the results of Lemma 16:
\begin{align}
\quad &\frac{V_{\acute\btheta_1,T}}{T}-\bar \phi^{(2)}_{\acute\bxi_1}\kappa_1(\acute\bbeta_1)
\stackrel{a.s.}{\longrightarrow} 0; \notag\\ %\label{eq:V_convergence}\\
%\quad &\frac{V_{\btheta_0,\btheta_1,T}}{T}\stackrel{a.s.}{\longrightarrow} 
%\bar \phi^{(2)}_{\bxi_0,\bxi_1}\bar{\kappa}(\bbeta_0,\bbeta_1), 
%\label{eq:V_convergence2}\\
%\quad &\frac{U_{\btheta_0,T}}{T}\stackrel{a.s.}{\longrightarrow} 
%\bar\phi^{(2)}_{\bxi_0}\kappa_0(\bbeta_0),
%\label{eq:U_convergence_0}\\
\quad &\frac{U_{\acute\btheta_1,T}}{T}-\bar\phi^{(2)}_{\bxi_0,\acute\bxi_1}\bar{\kappa}(\bbeta_0,\acute\bbeta_1)
\stackrel{a.s.}{\longrightarrow}0.\notag 
\end{align}
It follows that, as $T\rightarrow\infty$,
$$\ell_T=\frac{1}{T}\log I_T-\bar\phi^{(2)}_{\bxi_0,\acute\bxi_1}\bar{\kappa}(\bbeta_0,\acute\bbeta_1)
+\frac{1}{2}\bar \phi^{(2)}_{\acute\bxi_1}\kappa_1(\acute\bbeta_1)
+\frac{1}{2}\bar \phi^{(2)}_{\bxi_0}\kappa_0(\bbeta_0)
\stackrel{a.s.}{\longrightarrow} 0.$$
By uniform integrability arguments, which follow in similar lines as the proof of Lemma 10 of MB using
(H4$^\prime\prime$), compactness, and Cauchy-Schwartz, it holds that
\begin{equation}
Var_{\btheta_0}\left(\ell_T\right)\rightarrow 0,~\mbox{as}~T\rightarrow\infty.
\label{eq:var_ell}
\end{equation}
Hence,
\begin{align}
Var_{\btheta_0}\left(\frac{1}{T}\log I_T\right)
&=Var_{\btheta_0}\left(\ell_T+\bar\phi^{(2)}_{\bxi_0,\acute\bxi_1}\bar{\kappa}(\bbeta_0,\acute\bbeta_1)
-\frac{1}{2}\bar \phi^{(2)}_{\acute\bxi_1}\kappa_1(\acute\bbeta_1)
-\frac{1}{2}\bar \phi^{(2)}_{\bxi_0}\kappa_0(\bbeta_0)\right)\notag\\
&= Var_{\btheta_0}\left(\ell_T\right)
+Var_{\btheta_0}\left(\bar\phi^{(2)}_{\bxi_0,\acute\bxi_1}\bar{\kappa}(\bbeta_0,\acute\bbeta_1)
-\frac{1}{2}\bar \phi^{(2)}_{\acute\bxi_1}\kappa_1(\acute\bbeta_1)
-\frac{1}{2}\bar \phi^{(2)}_{\bxi_0}\kappa_0(\bbeta_0)\right)\notag\\
&\qquad-2cov_{\btheta_0}\left(\ell_T,\bar\phi^{(2)}_{\bxi_0,\acute\bxi_1}\bar{\kappa}(\bbeta_0,\acute\bbeta_1)
-\frac{1}{2}\bar \phi^{(2)}_{\acute\bxi_1}\kappa_1(\acute\bbeta_1)
-\frac{1}{2}\bar \phi^{(2)}_{\bxi_0}\kappa_0(\bbeta_0)\right).
\label{eq:var2}
\end{align}
By (\ref{eq:var_ell}), the first term of (\ref{eq:var2}) goes to zero as $T\rightarrow\infty$, and
the third, covariance term tends to zero by Cauchy-Schwartz and (\ref{eq:var_ell}). In other words,
as $T\rightarrow\infty$,
\begin{equation}
\left|Var_{\btheta_0}\left(\frac{1}{T}\log I_T\right)
-Var_{\btheta_0}\left(\bar\phi^{(2)}_{\bxi_0,\acute\bxi_1}\bar{\kappa}(\bbeta_0,\acute\bbeta_1)
-\frac{1}{2}\bar \phi^{(2)}_{\acute\bxi_1}\kappa_1(\acute\bbeta_1)\right)\right|
%-\frac{1}{2}\bar \phi^{(2)}_{\bxi_0}\kappa_0(\bbeta_0)\right)\right|
\rightarrow 0.
\label{eq:var3}
\end{equation}
However, $$Var_{\btheta_0}\left(\bar\phi^{(2)}_{\bxi_0,\acute\bxi_1}\bar{\kappa}(\bbeta_0,\acute\bbeta_1)
-\frac{1}{2}\bar \phi^{(2)}_{\acute\bxi_1}\kappa_1(\acute\bbeta_1)\right)\nrightarrow 0,$$
unless $\bar\phi^{(2)}_{\bxi_0,\acute\bxi_1}\bar{\kappa}(\bbeta_0,\acute\bbeta_1)
-\frac{1}{2}\bar \phi^{(2)}_{\acute\bxi_1}\kappa_1(\acute\bbeta_1)$ is constant almost surely.
It then follows from (\ref{eq:var3}) that
\begin{equation}
Var_{\btheta_0}\left(\frac{1}{T}\log I_T\right)=O(1),~\mbox{as}~T\rightarrow\infty.
\label{eq:var4}
\end{equation}

\end{proof}
\end{comment}


%\begin{remark}
%The conditions 
%(\ref{eq:convergent_series1}) hold if along with the assumption that near zero 
%$Var\left[\frac{\log\frac{I_{t+h_{n^*}}}{I_t}}{h_{n^*}}\right]=O(t^{2})$, the condition 
%$\sup_t\frac{1}{h_{n^*}^2} Var\left[\log\frac{I_{t+h_{n^*}}}{I_{t}}\right]<\infty$ is also satisfied.
%\end{remark}

\begin{corollary}
\label{corollary:bf_t}
For $j=1,2$, let $R_{jT}(\btheta_j)=\frac{f_{\btheta_j,T}(X_t)}{f_{\btheta_0,T}(X_T)}$, where
$\btheta_1$ and $\btheta_2$ are two different finite sets of parameters, perhaps with different dimensionalities,
associated with the two models
to be compared. For $j=1,2$, let
\[
I_{jT}=\int R_{jT}(\btheta_j)\pi_j(d\btheta_j),
\]
where $\pi_j$ is the prior on $\btheta_j$.
Let $B_T=I_{1T}/I_{2T}$ denote
the Bayes factor for comparing the two models associated with $\pi_1$ and $\pi_2$.   
Assume %the non-$iid$ case of the $SDE$ based random effects set-up and suppose 
that both
the models satisfy (H1$^{\prime\prime}$) -- (H9$^{\prime\prime}$), and
have the pseudo Kullback-Leibler property %(\ref{eq:kl_property}) 
with $\delta=\delta_1$ and $\delta=\delta_2$ respectively.
Then
\begin{equation}
E_{\btheta_0}\left(\frac{1}{T}\log B_T\right)\rightarrow \delta_2-\delta_1,
\label{eq:bf_convergence2}
\end{equation}
as $T\rightarrow\infty$.
\end{corollary}

\section{Illustration of our asymptotic result for comparing two individual $SDE$'s with a special case}
\label{sec:illustration}

Let the parameter space $\bTheta$ be compact, so that (H1$^{\prime\prime}$) holds.
Let $b_{\bbeta_j}$ and $\sigma$ satisfy
(H2$^{\prime\prime}$) such that
\begin{align}
\frac{b_{\bbeta_j}(s,x)}{\sigma(s,x)}&\equiv\eta_j(\bbeta_j);~j=0,1,
\label{eq:b_sigma_ratio}
%\frac{b_{\bbeta_1}(s,x)b_{\bbeta_0}(s,x)}{\sigma^2(s,x)}&\equiv\bar{\eta}(\btheta_0,\btheta_1),
%\frac{b_{\bbeta_0}(s,x)}{\sigma(s,x)}&\equiv\frac{\bar{\eta}(\bbeta_0,\bbeta_1)}{\eta_1(\bbeta_1)},
%\label{eq:b_sigma_ratio2}
\end{align}
so that
\begin{equation}
\frac{b_{\bbeta_1}(s,x)b_{\bbeta_0}(s,x)}{\sigma^2(s,x)}\equiv\eta_0(\bbeta_0)\eta_1(\bbeta_1).%\bar{\eta}(\bbeta_0,\bbeta_1).
\label{eq:b_sigma_ratio3}
\end{equation}
In the above, $\eta_1(\bbeta_1)$ %and $\bar{\eta}(\bbeta_0,\bbeta_1)$ 
is continuous in $\bbeta_1$.
Hence, (H3$^{\prime\prime}$) and (H4$^{\prime\prime}$) are satisfied. 
We assume that the relevant covariates and the functions $g_l$ are such that (H5$^{\prime\prime}$) holds.
%For our illustrative purpose we make the further extra assumption that 
%\begin{itemize}
%\item[(A)] Every $\left\{\bz(t)\in\boldcal Z:t\geq 0\right\}$
%is continuous in $t$. 
%\end{itemize}
%This assumption implies that $\phi_{\bxi_j}(t)$ are continuous in $t$, for $j=0,1$.

Letting $\kappa_j(\bbeta_j)=\left\{\eta_j(\bbeta_j)\right\}^2$ and
$\bar{\kappa}(\bbeta_0,\bbeta_1)=\eta(\bbeta_0)\eta(\bbeta_1)$, equations 
(\ref{eq:b_sigma_ratio}) -- (\ref{eq:b_sigma_ratio3}) entail
\begin{align}
V_{\btheta_j,t}&=\kappa_j(\bbeta_j)\int_0^t\phi^2_{\bxi_j}(s)ds;\label{eq:V_example}\\
V_{\btheta_0,\btheta_1,t}&=\bar{\kappa}(\bbeta_0,\bbeta_1)\int_0^t\phi_{\bxi_0}(s)\phi_{\bxi_1}(s)ds;
\label{eq:V_example2}\\
U_{\btheta_0,t}&=\kappa_0(\bbeta_0)\int_0^t\phi^2_{\bxi_0}(s)ds
+\eta_0(\bbeta_0)\int_0^t\phi_{\bxi_0}(s)dW(s);\label{eq:U_example}\\
U_{\btheta_0,\btheta_1,t}&=\bar{\kappa}(\bbeta_0,\bbeta_1)\int_0^t\phi_{\bxi_0}(s)\phi_{\bxi_1}(s)ds
+\eta_1(\bbeta_1)\int_0^t\phi_{\bxi_1}(s)dW(s);\label{eq:U_example2}\\
V_{\btheta_j,t,t+h}&=\kappa_j(\bbeta_j)\int_t^{t+h}\phi^2_{\bxi_j}(s)ds;\label{eq:V_example_h}\\
V_{\btheta_0,\btheta_1,t,t+h}&=\bar{\kappa}(\bbeta_0,\bbeta_1)\int_t^{t+h}\phi_{\bxi_0}(s)\phi_{\bxi_1}(s)ds;
\label{eq:V_example_h2}\\
U_{\btheta_0,t,t+h}&=\kappa_0(\bbeta_0)\int_t^{t+h}\phi^2_{\bxi_0}(s)ds
+\eta_0(\bbeta_0)\int_t^{t+h}\phi_{\bxi_0}(s)dW(s);
\label{eq:U_example_h}\\
U_{\btheta_0,\btheta_1,t,t+h}&=\bar{\kappa}(\bbeta_0,\bbeta_1)\int_t^{t+h}\phi_{\bxi_0}(s)\phi_{\bxi_1}(s)ds
+\eta_1(\bbeta_1)\int_t^{t+h}\phi_{\bxi_1}(s)dW(s).
\label{eq:U_example_h2}
\end{align}
%Let $\zeta(x,\varphi_j)$ be a linear function of $\varphi_j$,
%and let $\varphi_j$ have a normal distribution, truncated on a compact interval, under the prior $\pi$.
%Then (H5$^\prime$) holds.
Due to (\ref{eq:V_example}) and (\ref{eq:V_example2}), we obtain
\begin{align}
\tilde{\mathcal K}'_t(f_{\btheta_0},f_{\btheta_1})&=\lim_{h\rightarrow 0} 
\frac{\tilde {\mathcal K}(f_{\btheta_0,t,t+h},f_{\btheta_1,t,t+h})}{h}\notag\\
&=\frac{1}{2}\frac{d}{dt}E_{\theta_0}(V_{\btheta_0,t})
-\frac{d}{dt}E_{\theta_0}(V_{\btheta_0,\btheta_1,t})
+\frac{1}{2}\frac{d}{dt}E_{\theta_0}(V_{\btheta_1,t})\notag\\
&=\frac{\phi_{\bxi_0}^2(t)}{2}\kappa_0(\bbeta_0)
-\phi_{\bxi_1}(t)\phi_{\bxi_0}(t)\bar{\kappa}(\bbeta_0,\bbeta_1)
+\frac{\phi_{\bxi_1}^2(t)}{2}\kappa_1(\bbeta_1)\notag\\
&=\frac{1}{2}\left(\phi_{\bxi_0}(t)\eta_0(\bbeta_0)-\phi_{\bxi_1}(t)\eta_1(\bbeta_1)\right)^2.
%&=\bar{\mathcal K}^{\infty}(f_{\btheta_0},f_{\btheta_1}).
\label{eq:kl4}
\end{align}
Note that 
\begin{align}
\underset{t}{\inf}~\tilde{\mathcal K}'_t(f_{\btheta_0},f_{\btheta_1})
&\geq \frac{1}{2}~\underset{\bbeta_1\in\mathfrak B,\bxi_1\in\bGamma,\bz\in\mathfrak Z}{\inf}
~\left(\phi_{\bxi_0}(\bz)\eta_0(\bbeta_0)
-\phi_{\bxi_1}(\bz)\eta_1(\bbeta_1)\right)^2\notag\\
&=\delta.
%&=\frac{1}{2}\left(\phi_{\bxi_0}(\tilde\bz_{\bbeta_0,\bxi_0,\bbeta_1,\bxi_1})\eta_0(\bbeta_0)
%-\phi_{\bxi_1}(\tilde\bz_{\bbeta_0,\bxi_0,\bbeta_1,\bxi_1})\eta_1(\bbeta_1)\right)^2,
\label{eq:kl_inf2}
\end{align}
%where $\tilde\bz_{\bbeta_0,\bxi_0,\bbeta_1,\bxi_1}\in\boldcal Z$, perhaps depending upon 
%$(\bbeta_0,\bxi_0,\bbeta_1,\bxi_1)$, is such that (\ref{eq:kl_inf2}) holds. 
%From Berge's maximum theorem, it again follows as in Section \ref{subsec:kl_property_non_iid}
%that (\ref{eq:kl_inf2}) is coninuous in $(\bbeta_1,\bxi_1)$.
%Now, letting $\delta$  
%$$\delta=\frac{1}{2}~\underset{\bbeta_1\in\mathfrak B,\bxi_1\in\bGamma}{\inf}~
%\left(\phi_{\bxi_0}(\tilde\bz_{\bbeta_0,\bxi_0,\bbeta_1,\bxi_1})\eta_0(\bbeta_0)
%-\phi_{\bxi_1}(\tilde\bz_{\bbeta_0,\bxi_0,\bbeta_1,\bxi_1})\eta_1(\bbeta_1)\right)^2.$$
It follows from (\ref{eq:kl_inf2}) that (H6$^{\prime\prime}$) holds for any prior on $\btheta_1$. 

Also, it follows directly from (\ref{eq:kl4}), that
\begin{equation}
\bar{\mathcal K}^{\infty}(f_{\btheta_0},f_{\btheta_1})
=\frac{\bar\phi^{(2)}_{\bxi_0}}{2}\kappa_0(\bbeta_0)-\bar\phi^{(2)}_{\bxi_0,\bxi_1}\bar{\kappa}(\bbeta_0,\bbeta_1)
+\frac{\bar\phi^{(2)}_{\bxi_1}}{2}\kappa_1(\bbeta_1),
\label{eq:kl_inf3}
\end{equation}
which is continuous in $(\bbeta_1,\bxi_1)$, due to the continuity assumption of $\eta_1(\bbeta_1)$ in $\bbeta_1$ and 
Lemma \ref{lemma:uniform_convergence}, which guarantees continuity of $\bar\phi^{(2)}_{\bxi_0,\bxi_1}$
and $\bar\phi^{(2)}_{\bxi_1}$ in $\bxi_1$.
%agrees with (\ref{eq:kl_average_infinite2}).
%
%Hence, $$\underset{t}{\inf}~\tilde{\mathcal K}'_t(f_{\btheta_0},f_{\btheta_1})
%=\bar{\mathcal K}^{\infty}(f_{\btheta_0},f_{\btheta_1})
%=\frac{\bar\phi^{(2)}_{\bxi_0}}{2}\kappa_0(\bbeta_0)-\bar\phi^{(2)}_{\bxi_0,\bxi_1}\bar{\kappa}(\bbeta_0,\bbeta_1)
%+\frac{\phi^{(2)}_{\bxi_1}}{2}\kappa_1(\bbeta_1),$$
%
Since the right-most side of (\ref{eq:kl_inf3}) is a continuous function of $\btheta_1$, 
it follows that (H7$^{\prime\prime}$) is clearly satisfied if the prior $\pi$ is dominated by the Lebesgue measure.

We now verify the $Q^*$ property (H8$^{\prime\prime}$). 
Recall that $\hat f_{A_t(\delta)}=\hat f_{t,t+h}$, since $\pi\left(A_t(\delta)\right)=1$. Since
\begin{equation}
\hat f_{t,t+h}(X_{t,t+h})\leq \underset{\btheta_1\in A_t(\delta)}{\sup}~f_{\btheta_1,t,t+h}(X_{t,t+h})
=f_{\hat\btheta_1(X_{t,t+h}),t,t+h}(X_{t,t+h}),
\label{eq:q_star_example}
\end{equation}
where $\hat\btheta_1(X_{t,t+h})\in A_t(\delta)$, is the maximizer of $f_{\btheta_1,t,t+h}$ in the compact set $A_t(\delta)$.
Hence,
\begin{align}
&\tilde{\mathcal K}\left(f_{\btheta_0,t,t+h},\hat f_{A_t(\delta)}\right)\notag\\
&=E_{\btheta_0}\left(\log f_{\btheta_0,t,t+h}(X_{t,t+h})\right)
-E_{\btheta_0}\left(\log \hat f_{A_t(\delta)}(X_{t,t+h})|\mathcal F_t\right)\notag\\
&\geq E_{\btheta_0}\left(\log f_{\btheta_0,t,t+h}(X_{t,t+h})\right)
-E_{\btheta_0}\left(\log f_{\hat\btheta_1(X_{t,t+h}),t,t+h}(X_{t,t+h})|\mathcal F_t\right)\notag\\
&=E_{\btheta_0}\left(\log\frac{f_{\btheta_0,t,t+h}(X_{t,t+h})}{f_{\hat\btheta_1(X_{t,t+h}),t,t+h}(X_{t,t+h})}
\bigg |\mathcal F_t\right)\notag\\
&=E_{\hat\btheta_1(X_{t,t+h})|\btheta_0}E_{X_{t,t+h}|\hat\btheta_1(X_{t,t+h})=\tau,\btheta_0}
\left(\log\frac{f_{\btheta_0,t,t+h}(X_{t,t+h})}
{f_{\left\{\hat\btheta_1(X_{t,t+h})=\tau\right\},t,t+h}(X_{t,t+h})}\bigg |\mathcal F_t\right)\notag\\
&\geq E_{\hat\btheta_1(X_{t,t+h})|\btheta_0}\underset{\tau\in A_t(\delta)}{\inf}~
\tilde{\mathcal K}\left(f_{\btheta_0,t,t+h},f_{\tau,t,t+h}\right)\notag\\
&=E_{\hat\btheta_1(X_{t,t+h})|\btheta_0}\tilde {\mathcal K}\left(f_{\btheta_0,t,t+h},f_{\tau^*_t,t,t+h}\right)\notag\\
&=\tilde {\mathcal K}\left(f_{\btheta_0,t,t+h},f_{\tau^*_t,t,t+h}\right)\notag\\
&\geq\delta,
\label{eq:example_K}
\end{align}
where $\tau^*_t=\underset{\tau\in A_t(\delta)}{\arg\min}
~\tilde {\mathcal K}\left(f_{\btheta_0,t,t+h},f_{\tau,t,t+h}\right)\in A_t(\delta)$.
Hence, the $Q^*$ property is satisfied.

%We now verify (H11$^\prime$). %and (H12$^\prime$). 
%Note that $\tilde{\mathcal K}\left(f_{\btheta_0,t,t+h},f_{\btheta_j,t,t+h}\right)$ is a linear function
%of $E_{\btheta_0}\left(V_{\btheta_0,t,t+h}\right)$ and $E_{\btheta_0}\left(U_{\btheta_0,t,t+h}\right)$, given by 
%(\ref{eq:V_example_h}) and (\ref{eq:U_example_h}), respectively. Since $E_{\btheta_0}\left\{W(t+h)-W(t)\right\}=0$,
%the expectation of $\tilde{\mathcal K}\left(f_{\btheta_0,t,t+h},f_{\btheta_j,t,t+h}\right)$ is a linear
%function of $h$, from which (H12$^\prime$) follows.


To see that (H9$^{\prime\prime}$) holds, first observe that it follows from the proof of Lemma \ref{lemma:towards_martingale}
that 
$\frac{I_{t+h}}{I_{t}}=\frac{\hat f_{t,t+h}}{f_{\btheta_0,t,t+h}}$, which implies
%(\ref{eq:ration_I_t}) implies
\begin{align}
%E_{\btheta_0}\left[J_h(t)\right]&=
\frac{\tilde{\mathcal K}\left(f_{\btheta_0,t,t+h},\hat f_{t,t+h}\right)}{h}
%&=E_{\btheta_0}\left(\frac{\log I_{t+h}-\log I_t}{h}|\mathcal F_t\right)\notag\\
%&=E_{\btheta_0}\left(\frac{\log \hat f_{t,t+h}-\log f_{\btheta_0,t,t+h}}{h}\right)\notag\\
&=\frac{E_{\btheta_0}\left(\log f_{\btheta_0,t,t+h}(X_{t,t+h})-\log \hat f_{t,t+h}(X_{t,t+h})|\mathcal F_t\right)}{h}
\label{eq:verify_H11}
\end{align}
Now,
\begin{equation}
E_{\btheta_0}\left(\log f_{\btheta_0,t,t+h}(X_{t,t+h})|\mathcal F_t\right)
=\frac{\kappa_0(\bbeta_0)}{2}\int_t^{t+h}\phi^2_{\bxi_0}(s)ds=\frac{\kappa_0(\bbeta_0)}{2}h\phi^2_{\bxi_0}(s^*(h)),
\label{eq:inner0}
\end{equation}
by the mean value theorem for integrals, where $s^*(h)\rightarrow t$, as $h\rightarrow 0$.
Hence, using continuity of $\phi_{\bxi_0}(t)$ in $t$, we obtain 
\begin{equation}
\underset{h\rightarrow 0}{\lim}~\frac{1}{h}~E_{\btheta_0}\left(\log f_{\btheta_0,t,t+h}(X_{t,t+h})|\mathcal F_t\right)
=\frac{\kappa_0(\bbeta_0)}{2}~ \underset{h\rightarrow 0}{\lim}~\phi^2_{\bxi_0}(s^*(h))
=\frac{\kappa_0(\bbeta_0)}{2}\phi^2_{\bxi_0}(t).
\label{eq:verify_H11_1}
\end{equation}
%=E_{\btheta_0}\left(\log f_{\btheta_0,t,t+h}(X_{t,t+h})\right)$ is linear in $h$, since 
%$E_{\btheta_0}\left\{W(t+h)-W(t)\right\}=0$. 
To deal with $E_{\btheta_0}\left(\log \hat f_{t,t+h}(X_{t,t+h})|\mathcal F_t\right)$, 
note that
for any $X_{t,t+h}$, by the mean value theorem for integrals, 
$$\hat f_{t,t+h}(X_{t,t+h})=f_{\breve\btheta(X_{0,t},X_{t,t+h}),t,t+h}(X_{t,t+h}),$$ where 
$\breve\btheta(X_{0,t},X_{t,t+h})\in\bTheta$. 
%is a continuous
%function of $X_{0,t}$ and $X_{t,t+h}$. 
%Since $f_{\btheta^*(x_t,X_{t,t+h}),t,t+h}(X_{t,t+h})$ can
%also be viewed as a function of $r_{t,t+h}=W(t+h)-W(t)$, %\int_t^{t+h}\frac{b_{\bbeta_1}(s,X(s))}{\sigma(s,X(s))}dW(s)$, 
%$\btheta^*(x_t,X_{t,t+h})$ is also a continuous function of the latter. 
%Here note that if $\breve\btheta(X_{0,t},X_{t,t+h})$ is not continuous, then, because
%of compactness of $\bTheta$, it is possible to construct a continuous function $\Breve{\Breve\btheta}(X_{0,t},X_{t,t+h})$ which agrees 
%with $\breve\btheta(X_{0,t},X_{t,t+h})$ at any given point $(X_{0,t},X_{t,t+h})$.
%Since $X_{t,t+h}\rightarrow X_t$ almost surely as $h\rightarrow 0$ because of sample path continuity
%of the solution process $X$, 
%the above continuity of $\breve\btheta(X_{0,t},X_{t,t+h})$ implies that 
It is clear that
$\breve\btheta(X_{0,t},X_{t,t+h})\rightarrow \breve\btheta(X_{0,t},X_{t})=\breve\btheta(X_{0,t})$ almost surely, 
as $h\rightarrow 0$. 
%(equivalently, $(x_t,r_{t,t+h}))$.
Hence,
\begin{align}
&E_{\btheta_0}\left(\log \hat f_{t,t+h}(X_{t,t+h})|\mathcal F_t\right)\notag\\
&=E_{\breve\btheta(X_{0,t},X_{t,t+h})|\btheta_0}E_{X_{t,t+h}|\breve\btheta(X_{0,t},X_{t,t+h})=\alpha,\btheta_0}
\left(\log f_{\{\breve\btheta(X_{0,t},X_{t,t+h})=\alpha\},t,t+h}(X_{t,t+h})|\mathcal F_t\right),\notag
\end{align}
where
\begin{align}
%&E_{X_{t,t+h}|\btheta^*(x_t,X_{t,t+h}),\btheta_0}
&E_{\btheta_0}\left(\log \hat f_{t,t+h}(X_{t,t+h})|\mathcal F_t\right)\notag\\
&=E_{\btheta_0}\left(\log f_{\{\breve\btheta(X_{0,t},X_{t,t+h})\},t,t+h}(X_{t,t+h})|\mathcal F_t\right)\notag\\
%=h\frac{\zeta\left(x,\breve\xi(x_t,X_{t,t+h})\right)^2}{2}\kappa_1(\breve\btheta(x_t,X_{t,t+h})).
&=E_{\btheta_0}\left(\bar\kappa(\bbeta_0,\breve\bbeta_1(X_{0,t},X_{t,t+h}))
\int_t^{t+h}\phi_{\breve\bxi_1(X_{0,t},X_{t,t+h})}(s)\phi_{\bxi_0}(s)ds\right.\notag\\
&\quad\quad\quad\left.-\frac{\kappa_1(\breve\bbeta_1(X_{0,t},X_{t,t+h}))}{2}\int_t^{t+h}
\phi^2_{\breve\bxi_1(X_{0,t},X_{t,t+h})}(s)ds\right)
\notag\\
&=E_{\btheta_0}\left(\bar\kappa(\bbeta_0,\breve\bbeta_1(X_{0,t},X_{t,t+h}))
h\phi_{\breve\bxi_1(X_{0,t},X_{t,t+h})}(s_1(h))\phi_{\bxi_0}(s_1(h))\right.\notag\\
&\quad\quad\quad\left.-\frac{\kappa_1(\breve\bbeta_1(X_{0,t},X_{t,t+h}))}{2}h\phi^2_{\breve\bxi_1(X_{0,t},X_{t,t+h})}(s_2(h))\right),
\label{eq:inner1}
\end{align}
where $s_1(h),s_2(h)\in[t,t+h]$, associated with the mean value theorem for integrals. Hence,
$s_1(h)\rightarrow t$ and $s_2(h)\rightarrow t$, almost surely as $h\rightarrow 0$.
%$r_t=\int_0^t\frac{b_{\bbeta_1}(s,X(s))}{\sigma(s,X(s))}dW(s)$.

Continuity of $\bar\kappa(\cdot),\kappa_1(\cdot)$ and the results
$\breve\btheta(X_{0,t},X_{t,t+h})\rightarrow \breve\btheta(X_{0,t})$, $s_1(h)\rightarrow t$, $s_2(h)\rightarrow t$, almost surely, 
as $h\rightarrow 0$, in conjunction with the dominated convergence theorem exploiting boundedness of the functions 
$\phi_{\bxi_0}$, $\phi_{\breve\bxi_1}$, $\bar\kappa$ and $\kappa_j$, imply, using continuity of 
$\phi_{\breve\bxi_1}(t)$ in $t$, that
\begin{align}
&\underset{h\rightarrow 0}{\lim}~\frac{1}{h}~E_{\btheta_0}\left(\log \hat f_{t,t+h}(X_{t,t+h})|\mathcal F_t\right)\notag\\ 
%\frac{\zeta\left(x,\tilde\xi(x_t)\right)^2}{2}\kappa_1(\tilde\btheta(x_t)),
&\quad\quad=\phi_{\bxi_0}(t)\phi_{\breve\bxi_1(X_{0,t})}(t)\bar\kappa(\bbeta_0,\breve\bbeta(X_{0,t}))
-\frac{\phi_{\breve\bxi_1(X_{0,t})}^2(t)}{2}\kappa_1(\breve\bbeta(X_{0,t})).\notag
\end{align}
In other words, the limit of (\ref{eq:verify_H11}) exists and is unique as $h\rightarrow 0$.
Now, equations (\ref{eq:inner0}) and (\ref{eq:inner1}) along with dominated convergence theorem imply that  (H9$^{\prime\prime}$) holds. %Condition (H12$^\prime$) follows using the same techniques.

Thus, all the assumptions required for Theorem \ref{theorem:bf_convergence} and Corollary \ref{corollary:bf_t}
are satisfied.
Hence, both (\ref{eq:bf_convergence}) and (\ref{eq:bf_convergence2}) hold. 




%\subsection{Verification of the Kullback-Leibler property}
%\label{subsec:kl_property_t}
%
%Firstly, note that in our case, 
%\begin{align}
%\lim_{T\rightarrow\infty}\frac{1}{T}\left(\frac{\phi_0^2}{2}\int_0^T\frac{d}{dt}E(V_{0t})dt-\frac{\phi_1^2}{2}\int_0^T\frac{d}{dt}E(V_{1t})dt\right)&=\lim_{T\rightarrow\infty}\frac{1}{T}\left(\frac{\phi_0^2}{2}E(V_{0T})-\frac{\phi_1^2}{2}E(V_{1T})\right)\notag\\
%&=\frac{\phi_0^2}{2}k_0-\frac{\phi_1^2}{2}k_1~~~~~\mbox{by lemma (\ref{lemma:convergence1})}
%\end{align}
%exists almost surely with respect to $\btheta_1$ and is also continuous with respect to $\btheta_1$. We denote the above limit by 
%$\tilde{\mathcal K}^{\infty}\left(f_{\btheta_0},f_{\btheta_1}\right)$
%
%The continuity of the function $h$ and $\tilde{\mathcal K}(f_{\btheta_0,t,t+h},f_{\btheta_1,t,t+h})$ 
%with respect to $x$ and $t$ implies
%\begin{align}
%\psi(\btheta_1)&=\underset{t}{\inf}~\tilde{\mathcal K}(f_{\btheta_0,t,t+h},f_{\btheta_1,t,t+h})\notag\\
%&=\underset{t}{\inf}~\left\{\frac{h^2\left(x,\varphi_0\right)}{2}
%\frac{d}{dt}E_{\btheta_0}(V_{t,\btheta_0})-\frac{h^2\left(x,\varphi_1\right)}{2}
%\frac{d}{dt}E_{\btheta_0}(V_{t,\btheta_1})\right\}
%\label{eq:kl_non_iid2}
%\end{align}
%
%\section{One Important Observation}
%\label{sec:observation}
%
%Consider two arbitrary model,
%\begin{equation}
%d X(t)=b_1(X(t),\phi_1)dt+\sigma_1(X(t))dW(t)
%\label{eq:sde2}
%\end{equation}
%\begin{equation}
%d X(t)=b_2(X(t),\phi_2)dt+\sigma_2(X(t))dW(t)
%\label{eq:compare_sde3}
%\end{equation}
%Our aim is to determine among these two models which is closest to the true model.
%\[
%d X(t)=b_0(X(t),\phi_0)dt+\sigma_0(X(t))dW(t)
%\label{eq:sde1}
%\]
%Suppose according to theory we have got
%\[
%T^{-1}logI_{1T} \stackrel{a.s.}{\rightarrow}-\delta_1
%\]
%and
%\[
%T^{-1}logI_{2T} \stackrel{a.s.}{\rightarrow}-\delta_2
%\]
%as $T\rightarrow\infty$ and $\delta_1 > \delta_2$.\\
%Then, $T^{-1}\log\frac{I_{1T}}{I_{2T}}{\rightarrow}-(\delta_1-\delta_2)=-\delta$ (say).\\
%By the theorem in this case Bayes Factor is consistent implying (\ref{eq:compare_sde3}) is preferred model.
%
%It is clear that the model whose Kullback-Leibler divergence measure from the true model is minimum will closely approximate the data and the above observation says that consistent Bayes factor also implies to prefer that model only. 
%
%In other words, suppose we have a set of models describing the same dataset and the true model is unknown. Then comparing the set of models, the model with minimum $\delta$ will be our preferred model. Moreover, if our set of models include the true model then we will get for that model $\delta$ as zero and obviously true model will be then preferred.

\section{Asymptotic convergence of Bayes factor in the $SDE$ set-up with respect to  
number of individuals and time}
\label{sec:asymp_BF_nT}


%Recall that (see Section \ref{subsec:case2})
%in this asymptotic framework we are interested in studying the properties of 
%$I_{n,T_1,\ldots,T_n}$ in both the $iid$ and non-$iid$ set-ups
%when both $n\rightarrow\infty$ and $T_i\rightarrow\infty$. However, for simplicity, we assume
%$T_i=T$ for each $i$, even in the non-$iid$ set-up, so that in our asymptotic framework we study convergence of 
%of the Bayes factor of the form (\ref{eq:tilde_I_nT})
%%\begin{equation}
%%\tilde I_{n,T}=\prod_{i=1}^nI_{x^i,T},
%%\label{eq:tilde_I_nT}
%%\end{equation}
%as $n\rightarrow\infty$ and $T\rightarrow\infty$.



\subsection{Convergence of Bayes factor in the $iid$ set-up}
\label{subsec:iid}


%\begin{equation}
%E_{\btheta_0}\left(\frac{1}{nT}\log\tilde I_{n,T}\right)
%=\frac{1}{n}\sum_{i=1}^nE_{\btheta_0}\left(\frac{1}{T}\log I_{x^i,T}\right)=
%E_{\btheta_0}\left(\frac{1}{T}\log I_{x,T}\right),
%\label{eq:bf_convergence_iid}
%\end{equation}
%where $x_i=x$ for $i=1,\ldots,n$.
%Under conditions (H1$^{\prime\prime}$) -- (H8$^{\prime\prime}$), by Theorem \ref{theorem:bf_convergence},
%$E_{\btheta_0}\left(\frac{1}{T}\log I_{x,T}\right)\rightarrow -\delta$, as $T\rightarrow\infty$. 
%%for each $i$. 
%Hence, for any $n$, the right hand side of (\ref{eq:bf_convergence_iid}) converges
%to $-\delta$.
%Also, quite importantly, for $r\geq 2$, it is easily seen that
%\begin{equation}
%E_{\btheta_0}\left(\frac{1}{nT}\log\tilde I_{n,T}+\delta\right)^r=O\left(n^{-r}\right).
%\label{eq:bf_convergence_moments}
%\end{equation}
%Since (\ref{eq:bf_convergence_moments}) is summable, it follows that 
%$\frac{1}{nT}\log\tilde I_{n,T}$ converges to $-\delta$ almost surely as $n\rightarrow\infty$, $T\rightarrow\infty$.

Although Theorem \ref{theorem:bf_convergence} fails to ensure consistency of the Bayes factor as $T\rightarrow\infty$
in the sense that the relevant variance is asymptotically positive,
the theorem is useful to prove almost sure consistency when $T\rightarrow\infty$ as well as $n\rightarrow\infty$, for
both $iid$ and non-$iid$ situations. Theorem \ref{theorem:bf_convergence_iid} formalizes this for the $iid$ set-up,
while Theorem \ref{theorem:bf_convergence_non_iid} establishes almost sure consistency of the Bayes factor
in the non-$iid$ situation. Proofs of these theorems are contained in section S-6 and S-9 respectively in the supplement.
\begin{theorem}
\label{theorem:bf_convergence_iid}
Assume the $iid$ set-up;
%that is, assume that $\btheta^{(i)}_j=\xi^{(i)}_{0,j}$ (that is, $\xi^{(i)}_{1,j}=\xi^{(i)}_{2,j}=\xi^{(i)}_{3,j}=\cdots=\xi^{(i)}_{p,j}=0$), 
%for $i=1,\ldots,n$, $j=0,1$, and that $\btheta^{(i)}_1\stackrel{iid}{\sim}\pi;~i=1,2,\ldots$.
also assume that conditions (H1$^{\prime\prime}$) -- (H9$^{\prime\prime}$) hold for each $SDE$ 
in the systems (\ref{eq:sde3}) and (\ref{eq:sde4}). Then
\begin{equation}
\frac{1}{nT}\log\tilde I_{n,T}
\rightarrow -\delta,
\label{eq:bf_convergence_iid2}
\end{equation}
almost surely, as $n\rightarrow\infty$ and $T\rightarrow\infty$.
\end{theorem}

\begin{comment}
\begin{proof}
%Assuming the set-up described in Section \ref{subsec:case2}, 
In our set-up it follows from (\ref{eq:tilde_I_nT}) that
\begin{equation}
\frac{1}{nT}\log\tilde I_{n,T}
=\frac{1}{n}\sum_{i=1}^n\frac{1}{T}\log I_{i,T}.
\label{eq:bf_convergence_iid}
\end{equation}
In the $iid$ case, given $T>0$, 
using the above form, it follows by the strong law of large numbers, that 
\begin{equation}
\underset{n\rightarrow\infty}{\lim}\frac{1}{nT}\log\tilde I_{n,T}= E_{\btheta_0}\left(\frac{1}{T}\log I_{i,T}\right),
\label{eq:convergence_given_T}
\end{equation}
almost surely. Now, in the $iid$ situation, for each $i$,
$E_{\btheta_0}\left(\frac{1}{T}\log I_{i,T}\right)\rightarrow -\delta$, as $T\rightarrow\infty$.
Hence, taking limit as $T\rightarrow\infty$ on both sides of (\ref{eq:convergence_given_T}) yields
\begin{equation}
\underset{T\rightarrow\infty}{\lim}\underset{n\rightarrow\infty}{\lim}\frac{1}{nT}\log\tilde I_{n,T}
= \underset{T\rightarrow\infty}{\lim}E_{\btheta_0}\left(\frac{1}{T}\log I_{i,T}\right)=-\delta,
\label{eq:convergence_T_n}
\end{equation}
almost surely, proving the theorem.
\end{proof}
\end{comment}

The following corollary is obvious.
\begin{corollary}
\label{corollary:bf_t_iid}
For $j=1,2$, and $i=1,\ldots,n$, 
let $R_{j,i,T}(\btheta^{(i)}_{j})=\frac{f_{\btheta^{(i)}_j,i,T}(X_{i,0,T})}{f_{\btheta^{(i)}_0,i,T}(X_{i,0,T})}$, where, for each $i$,
$\btheta^{(i)}_1$ and $\btheta^{(i)}_2$ are two different finite sets of parameters, perhaps with different dimensionalities,
associated with the two systems (\ref{eq:sde3}) and (\ref{eq:sde4})
to be compared. For $j=1,2$, let
\[
\tilde I_{j,n,T}=\prod_{i=1}^n\int R_{j,i,T}(\btheta^{(i)}_j)\pi_j(d\btheta^{(i)}_j),
\]
where $\pi_j$ is the prior on $\btheta^{(i)}_j$, for $i=1,2,\ldots$.
Let $B_{n,T}=\tilde I_{1,n,T}/\tilde I_{2,n,T}$ denote
the Bayes factor for comparing the two models associated with $\pi_1$ and $\pi_2$.   
Assume the $iid$ case and suppose 
that both
the systems satisfy (H1$^{\prime\prime}$) -- (H9$^{\prime\prime}$), and
have the pseudo Kullback-Leibler property %(\ref{eq:kl_property}) 
with $\delta=\delta_1$ and $\delta=\delta_2$ respectively.
Then
\begin{equation*}
\frac{1}{nT}\log B_{n,T}\rightarrow \delta_2-\delta_1,
\label{eq:bf_convergence2_iid}
\end{equation*}
almost surely, as $n\rightarrow\infty$ and $T\rightarrow\infty$.
\end{corollary}

\subsection{Convergence of Bayes factor in the non-$iid$ set-up}
\label{subsec:bf_convergence_non_iid}

%Here we assume a non-$iid$ set-up similar to that in Section \ref{sec:sde_non_iid}, given as follows:
We now relax the assumptions $x^i=x$ and $\xi^{(i)}_{1j}=\xi^{(i)}_{2j}=\xi^{(i)}_{3j}=\cdots=\xi^{(i)}_{pj}=0$ for $j=0,1$. 
Thus, we are now in a non-$iid$ situation
where the processes $X_i(\cdot);~i=1,\ldots,n$, are independently,
but not identically distributed.
As mentioned in Section \ref{subsec:covariates} we assume that $\btheta^{(i)}_1\stackrel{iid}{\sim}\pi$.
In this set-up, for each $\bz\in\mathfrak Z=\left\{\bz(t)\in\boldcal Z:t\in[0,\infty)\right\}$,
it holds, due to Theorem \ref{theorem:bf_convergence}, that
\begin{equation}
\frac{1}{T}E_{\btheta_0}\left(\log I_{x,T,\bz}\right)\rightarrow -\delta(x,\bz), %\left(x^i\right),
\label{eq:bf_convergence_non_iid}
\end{equation}
as $T\rightarrow\infty$, where $\delta(x,\bz)$ depends upon the initial value $x\in\mathfrak X$ 
and the set of time-dependent covariates 
$\bz\in\mathfrak Z$. 
The following lemma shows that $\delta(x,\bz)$ is continuous in $(x,\bz)\in\mathfrak X\times\mathfrak Z$.
%where ``$\otimes$" denotes cartesian poduct.
\begin{lemma}
\label{lemma:delta_continuity}
Assume the conditions of Theorem \ref{theorem:bf_convergence}. Then,
$\delta(x,\bz)$ is continuous in $(x,\bz)\in\mathfrak X\times\mathfrak Z$.
\end{lemma}

\begin{comment}
\begin{proof}
Note that, due to compactness of $\mathfrak X$, $\boldcal Z$ and continuity of the covariates
in time $t$, there exists $x^*\in\mathfrak X$ and $\bz^*\in\mathfrak Z$, such that
\begin{equation}
\underset{x\in\mathfrak X,\bz\in\mathfrak Z}{\sup}~
\left|\frac{1}{T}E_{\btheta_0}\left(\log I_{x,T,\bz}\right) +\delta(x,\bz)\right|
=\left|\frac{1}{T}E_{\btheta_0}\left(\log I_{x^*,T,\bz^*}\right) +\delta(x^*,\bz^*)\right|
\rightarrow 0,
\label{eq:uniform_convergence}
\end{equation}
as $T\rightarrow\infty$, where the convergence is due to (\ref{eq:bf_convergence_non_iid}). 
Also, $\frac{1}{T}E_{\btheta_0}\left(\log I_{x,T,\bz}\right)$ is clearly
continuous in $(x,\bz)$ for every $T>0$ (this can be easily proved in the same way as Theorem 5
of \ctn{Maitra14a}). 
Combining this with the uniform convergence (\ref{eq:uniform_convergence})
it follows that $\delta(x,\bz)$ is also continuous in $(x,\bz)$.
\end{proof}
\end{comment}

Now consider the following limit:
\begin{equation}
\delta^{\infty}=\underset{n\rightarrow\infty}{\lim}~\frac{1}{n}\sum_{i=1}^n\delta(x^i,\bz_i).
\label{eq:delta_convergence}
\end{equation}
The following lemma shows that the above limit exists for all sequences 
$\left\{(x^i,\bz_i)\right\}_{i=1}^{\infty}\in\mathfrak X\times\mathfrak Z$.
\begin{lemma}
\label{lemma:delta_limit_existence}
The limit (\ref{eq:delta_convergence}) exists for all sequences $\left\{(x^i,\bz_i)\right\}_{i=1}^{\infty}
\in\mathfrak X\times\mathfrak Z$.
\end{lemma}

\begin{comment}
\begin{proof}
Note that the limit (\ref{eq:delta_convergence}) can be represented as
\begin{equation}
\underset{n\rightarrow\infty}{\lim}~\frac{1}{n}\sum_{r=0}^{n-1}
\varrho_{\delta}\left(\frac{r}{n}\right),
\label{eq:Riemann_delta}
\end{equation}
where $\varrho_{\delta}:[0,1]\mapsto\mathbb R^+$ is some continuous function satisfying 
$\varrho_{\delta}\left(\frac{r}{n}\right)=\delta\left(x^{r+1},\bz_{r+1}\right)$ for $r=0,\ldots,n-1$.
For the remaining points $y\in[0,1]$, we set $\varrho_{\delta}(y)=\delta(x,\bz)$,
where $(x,\bz)\in\mathfrak X\times\mathfrak Z$ are such that $\varrho_{\delta}(y)$ is continuous in $y\in[0,1]$. Since 
$\delta(x,\bz)$ is continuous in $(x,\bz)$, $\varrho_{\delta}(y)$ can be thus constructed.
Note that, it is possible to relate $y\in[0,1]$ to $(x,\bz)\in\mathfrak X\times\mathfrak Z$ by some continuous mapping
$G:\mathfrak X\times\mathfrak Z\mapsto[0,1]$, taking $(x,\bz)$ to $y$.
Thus, $\delta^{\infty}$ in
(\ref{eq:delta_convergence}) is the limit of the Riemann sum (\ref{eq:Riemann_delta}) associated with the continuous
function $\varrho_{\delta}$, given by $\int_0^1\varrho_{\delta}(y)dy$. 
Since the domain of integration is $[0,1]$, it follows, using continuity of $\varrho_{\delta}$, 
that the integral is finite. Observe that for any given sequence $\left\{(x^i,\bz_i)\right\}_{i=1}^{\infty}$, one
can construct a continuous function $\varrho_{\delta}$ such that $\delta^{\infty}=\int_0^1\varrho_{\delta}(y)dy$.
In other words,
$\delta^{\infty}$ in (\ref{eq:delta_convergence})
exists for all sequences $\left\{(x^i,\bz_i)\right\}_{i=1}^{\infty}$.
\end{proof}
\end{comment}
%More generally, for any $x\in\mathfrak X$, it holds that
%\begin{equation}
%\frac{1}{T}E_{\btheta_0}\left(\log I_{x,T}\right)\rightarrow -\delta(x), %\left(x^i\right),
%\label{eq:bf_convergence_non_iid2}
%\end{equation}
%where $\delta(\cdot)$ can be thought of as a continuous function
%of $x$, with $\delta\left(x^i\right)=\delta_i$. Since $x^i\rightarrow x^{\infty}$, it follows that
%\begin{equation*}
%\underset{n\rightarrow\infty}{\lim}~
%\frac{1}{n}\sum_{i=1}^n\delta\left(x^i\right)=\delta\left(x^{\infty}\right).
%\label{eq:delta_convergence}
%\end{equation*}
%It also follows, noting that $\underset{x\in\mathfrak X}{\sup}~
%Var_{\btheta_0}\left(\frac{1}{T}\log I_{x,T}\right)<\infty$, 
%that $E_{\btheta_0}\left(\log \tilde I_{n,T}+\delta(x^{\infty})\right)^r$
%is summable for $r\geq 2$.

Proof of these two lemmas are provided in section S-7 and S-8 respectively in the supplement.
Now, we  have the following theorem.
\begin{theorem}
\label{theorem:bf_convergence_non_iid}
Assume the non-$iid$ set-up, and conditions (H1$^{\prime\prime}$) -- (H9$^{\prime\prime}$), for each $SDE$ 
in the systems (\ref{eq:sde3}) and (\ref{eq:sde4}). Then
\begin{equation}
\frac{1}{nT}\log\tilde I_{n,T}
\rightarrow -\delta^{\infty},
\label{eq:bf_convergence_non_iid3}
\end{equation}
almost surely, as $T\rightarrow\infty$ and $n\rightarrow\infty$.
\end{theorem}

\begin{comment}
\begin{proof}
For given $T>0$, it follows from (\ref{eq:var4}), the subsequent expressions
compactness of $\bTheta$, $\mathfrak X$, $\boldcal Z$,
and continuity of the relevant functions $\phi_{\bxi_j}$,
$b_{\bbeta_j}$, $g_1,\ldots,g_p$, $\kappa_0$, $\kappa_1$ and $\bar\kappa$, that
\begin{equation}
\underset{x\in\mathfrak X,\bz\in\mathfrak Z}{\sup}~Var_{\btheta_0}\left(\frac{1}{T}\log I_{x,T,\bz}\right)<\infty.
\label{eq:var_sup_Z}
\end{equation}
Hence, given $T>0$,
\begin{equation}
\sum_{i=1}^{\infty}\frac{Var_{\btheta_0}\left(\frac{1}{T}\log I_{i,T}\right)}{i^2}<\infty.
\end{equation}
It then follows due to Kolmogorov's strong law of large numbers for independent random variables that
\begin{equation}
\underset{n\rightarrow\infty}{\lim}~\frac{1}{n}\sum_{i=1}^n\left(\frac{1}{T}\log I_{i,T}\right)
=\underset{n\rightarrow\infty}{\lim}~\frac{1}{n}\sum_{i=1}^nE_{\btheta_0}\left(\frac{1}{T}\log I_{i,T}\right),
\label{eq:non_iid_convergence1}
\end{equation}
almost surely.

Now observe that the right hand side of (\ref{eq:non_iid_convergence1}) admits the following representation
\begin{equation}
\underset{n\rightarrow\infty}{\lim}~\frac{1}{n}\sum_{r=0}^{n-1}\breve\varrho\left(\frac{r}{n},T\right),
\label{eq:Riemann1}
\end{equation}
where $\breve\varrho(\cdot,T):[0,1]\mapsto\mathbb R$ is some continuous function depending upon $T$ with 
$\breve\varrho(\frac{r}{n},T)=E_{\btheta_0}\left(\frac{1}{T}\log I_{r+1,T}\right)$.% for $r=0,\ldots,n-1$. 
%For the remaining points $y\in[0,1]$, we assume that $\varrho(y,T)$ is appropriately chosen, depending on $T>0$
%so that $\varrho(y,T)$ is continuous, and that the pointwise limits of $\varrho(y,T)$ exist as
%$T\rightarrow\infty$. Note that, as $T\rightarrow\infty$, 
%$\varrho(\frac{r}{n},T)\rightarrow\delta\left(\bz^{r+1}\right)$, for $r=0,\ldots,n-1$. 
%For the remaining points $y\in[0,1]$, we set $\varrho(y,T)=E_{\btheta_0}\left(\frac{1}{T}\log I_{\bz,T}\right)$,
%where $\bz\in\boldcal Z$ are such that $\varrho(y,T)$ is continuous in $y\in[0,1]$. 
 
Since $E_{\btheta_0}\left(\frac{1}{T}\log I_{x,T,\bz}\right)$ is continuous in $\bz$, $\breve\varrho(y,T)$ can be constructed 
as in Lemma \ref{lemma:delta_limit_existence}.
%Note that, it is possible to relate $y\in[0,1]$ to $\bz\in\boldcal Z$ by some continuous mapping
%$G:\boldcal Z\mapsto[0,1]$, taking $\bz$ to $y$.
Then, for almost all $y\in [0,1]$, $\breve\varrho(y,T)\rightarrow -\delta(x,\bz)$ as $T\rightarrow\infty$, for appropriate
$(x,\bz)$ associated with $y$ via $y=G(x,\bz)$ as in Lemma \ref{lemma:delta_limit_existence}. 
Also, it follows from (\ref{eq:log_I_T}) that 
$\breve\varrho(\cdot,T)$ so constructed is uniformly bounded in $T>0$. 
%Since $\varrho(y,T)$ is continuous in $y$ for any given $T>0$, 
%it is clearly bounded by some positive constant, for all, which is integrable on $[0,1]$. 
Thus, the conditions
of the dominated convergence theorem are satisfied.
%and $\varphi:[0,1]\mapsto\mathbb R$ is integrable. 
%To see that such $\varrho$ exists, first note that, due to Lemma \ref{lemma:delta_limit_existence}, guaranteeing
%continuity of $\delta(\bz)$,
%for each $y\in[0,1]$ there exists $\bz\in\boldcal Z$
%such that $\varphi(y)=\delta(\bz)$. 
%Hence 
%\begin{equation}
%\int_0^1\varphi(y)dy=\underset{n\rightarrow\infty}{\lim}~\frac{1}{n}\sum_{i=1}^n\varphi(y_i)
%=\underset{n\rightarrow\infty}{\lim}~\frac{1}{n}\sum_{i=1}^n\delta(\bz_i).
%\label{eq:existence_varphi}
%\end{equation}
%Since the rightmost side of (\ref{eq:existence_varphi}) exists by Lemma \ref{lemma:delta_limit_existence}, 
%it follows that $\varphi$, and hence $\varrho$, exist.

Since (\ref{eq:Riemann1}) is nothing but the Riemann sum associated with $\breve\varrho(\cdot)$, it follows that
\begin{equation}
\underset{n\rightarrow\infty}{\lim}~\frac{1}{n}\sum_{i=1}^nE_{\btheta_0}\left(\frac{1}{T}\log I_{i,T}\right)
=\int_0^1\breve\varrho(y,T)dy.
\label{eq:towards_DCT}
\end{equation}
By construction of $\breve\varrho(y,T)$, the dominated convergence theorem holds for the right hand side
of (\ref{eq:towards_DCT}). Hence, 
\begin{align}
&\underset{T\rightarrow\infty}{\lim}\underset{n\rightarrow\infty}{\lim}
~\frac{1}{n}\sum_{i=1}^nE_{\btheta_0}\left(\frac{1}{T}\log I_{i,T}\right)
=\underset{T\rightarrow\infty}{\lim}~\int_0^1\breve\varrho(y,T)dy
=\int_0^1\underset{T\rightarrow\infty}{\lim}~\breve\varrho(y,T)dy\notag\\
&\quad\quad=\underset{n\rightarrow\infty}{\lim}~\frac{1}{n}\sum_{i=1}^n
\underset{T\rightarrow\infty}{\lim}~E_{\btheta_0}\left(\frac{1}{T}\log I_{i,T}\right)
=-\underset{n\rightarrow\infty}{\lim}~\frac{1}{n}\sum_{i=1}^n\delta(x^i,\bz_i)=-\delta^{\infty}.
\label{eq:DCT}
\end{align}
Combining (\ref{eq:DCT}) with (\ref{eq:non_iid_convergence1}) it follows that
\begin{equation}
\underset{T\rightarrow\infty}{\lim}\underset{n\rightarrow\infty}{\lim}
~\frac{1}{n}\sum_{i=1}^n\left(\frac{1}{T}\log I_{i,T}\right)
=\underset{T\rightarrow\infty}{\lim}\underset{n\rightarrow\infty}{\lim}
~\frac{1}{n}\sum_{i=1}^nE_{\btheta_0}\left(\frac{1}{T}\log I_{i,T}\right)
=-\delta^{\infty},
\label{eq:non_iid_convergence2}
\end{equation}
almost surely.
\end{proof}
\end{comment}

We then have the following corollary for the non-$iid$ case.
\begin{corollary}
\label{corollary:bf_t_non_iid}
%Assume the non-$iid$ set-up.
For $j=1,2$, and $i=1,\ldots,n$, 
let $R_{j,i,T}(\btheta^{(i)}_{j})=\frac{f_{\btheta^{(i)}_j,i,T}(X_{i,0,T})}{f_{\btheta^{(i)}_0,i,T}(X_{i,0,T})}$, 
where, for each $i$,
$\btheta^{(i)}_1$ and $\btheta^{(i)}_2$ are two different finite sets of parameters, perhaps with different dimensionalities,
associated with the two systems (\ref{eq:sde3}) and (\ref{eq:sde4})
to be compared. For $j=1,2$, let
\[
\tilde I_{j,n,T}=\prod_{i=1}^n\int R_{j,i,T}(\btheta^{(i)}_j)\pi_j(d\btheta^{(i)}_j),
\]
where $\pi_j$ is the prior on $\btheta^{(i)}_j;~i=1,2,\ldots$.
Let $B_{n,T}=\tilde I_{1,n,T}/\tilde I_{2,n,T}$ denote
the Bayes factor for comparing the two models associated with $\pi_1$ and $\pi_2$.   
Assume the non-$iid$ case and suppose 
that both
the systems satisfy (H1$^{\prime\prime}$) -- (H9$^{\prime\prime}$), and
have the pseudo Kullback-Leibler property %(\ref{eq:kl_property}) 
with $\delta_i=\delta_{1i}$ and $\delta_i=\delta_{2i}$ respectively.
Let, for $j=1,2$,
\begin{equation*}
\delta^{\infty}_j=\underset{n\rightarrow\infty}{\lim}~\frac{1}{n}\sum_{i=1}^n\delta_{ji}.
\label{eq:delta_convergence2}
\end{equation*}
Then
\begin{equation*}
\frac{1}{nT}\log B_{n,T}\rightarrow \delta^{\infty}_2-\delta^{\infty}_1,
\label{eq:bf_convergence2_non_iid}
\end{equation*}
almost surely, as $n\rightarrow\infty$ and $T\rightarrow\infty$. 
\end{corollary}

\section{Simulation studies}
\label{sec:simulated_data}


\subsection{Covariate selection when $n=1$, $T=5$}
\label{subsec:N1T5}

We first demonstrate with simulation study the finite sample analogue of Bayes factor analysis
associated with a single individual, when $T\rightarrow\infty$ . In this regard, 
we consider modeling a single individual by
\begin{equation}
dX(t)=(\xi_1+\xi_2z_1(t)+\xi_3z_2(t)+\xi_4z_3(t))(\xi_5+\xi_6X(t))dt+\sigma dW(t),
\label{eq:sde2_appl}
\end{equation}
where we fix our diffusion coefficient as $\sigma=20$. 
We consider the initial value $X(0)=0$ and the time interval $[0, T]$ with $T=5$. 

To achieve numerical stability of the marginal likelihood corresponding to data we choose 
the true values of $\xi_i$; $i=1,\ldots,6$ as follows:
$\xi_{i}\stackrel{iid}{\sim} N(\mu_i,0.001^2)$, where 
$\mu_i\stackrel{iid}{\sim} N(0,1)$. This is not to be interpreted as the prior; this
is just a means to set the true values of the parameters of the data-generating model.
%Indeed, see for our purpose, we consider a multivariate normal prior for $\btheta$ with mean
%being the maximum likelihood estimator ($MLE$) and appropriate variance.
\\[2mm]
We assume that the time dependent covariates $z_i(t)$ satisfy the following $SDE$s 
\begin{align}
dz_1(t)=&(\tilde\theta_{1}+\tilde\theta_{2}z_1(t))dt+ dW_1(t)\notag\\
dz_2(t)=&\tilde\theta_{3}dt+ dW_2(t)\notag\\
dz_3(t)=&\tilde\theta_{4}z_3(t))dt+ dW_3(t),
\label{eq:covariate_appl}
\end{align}
where $W_i(\cdot)$; $i=1,2,3$, are independent Wiener processes, and 
$\tilde\theta_{i}\stackrel{iid}{\sim} N(0,0.01^2)$ for $i=1,\cdots,4$. 
%Here the diffusion coefficient $\sigma=1$. 

We obtain the covariates by first simulating $\tilde\theta_{i}\stackrel{iid}{\sim} N(0,0.01^2)$ for $i=1,\cdots,4$,
fixing the values, and then by simulating the covariates using the $SDE$s (\ref{eq:covariate_appl}) 
by discretizing the time interval $[0,5]$ into $500$ equispaced time points. 
In all our applications we have standardized the covariates over time so that they have zero means
and unit variances.

Once the covariates are thus obtained, we assume that the data are generated from the (true) model where all the covariates
are present. For the true values of the parameters, we simulated $(\xi_1,\ldots,\xi_6)$ from
the prior and treated the obtained values as the true set of parameters $\btheta_0$.
We then generated the data using (\ref{eq:sde2_appl}) by discretizing the time interval $[0,5]$ 
into $500$ equispaced time points.
%The simulated data is shown in figure \ref{fig:simdata}.
%
%\begin{figure}
%\centering
%\includegraphics[height=9cm,width=8cm]{graph1.pdf}
%\label{fig:sfig1}
%\caption{Simulated Data}
%\label{fig:simdata}
%\end{figure}




As we have three covariates so we will have $2^3=8$ different models. Denoting
a model by the presence and absence of the respective covariates, it then is the case that 
$(1,1,1)$ is the true, data-generating model, while $(0,0,0)$, $(0,0,1)$, $(0,1,0)$, $(0,1,1)$, $(1,0,0)$,
$(1,0,1)$, and $(1,1,0)$ are the other $7$ possible models. 

As per our theory, for a single individual, the Bayes factor is not consistent for increasing time domain.
However, we have shown that 
\begin{equation*}
\frac{1}{T}E_{\btheta_0}(\log I_T)\rightarrow -\delta
\label{eq:bf_appl}
\end{equation*}
as $T\rightarrow \infty$. Thus, the Bayes factor is consistent with respect to the expectation. 
Our simulation results show that this holds even for the time domain $[0,5]$,
where we approximate the expectation with the average of %$500$ 
$1000$ realizations of $I_T$
associated with as many simulated data sets. %case we have simulated the data 500 times to verify the result (\ref{eq:bf_appl}). 


\subsubsection{Case 1: the true parameter set $\btheta_0$ is fixed}
\label{subsubsec:case1_appl2}
{\bf Prior on $\btheta$}
\\[2mm]
We first obtain the maximum likelihood estimator ($MLE$) of $\btheta$ using simulated annealing
and then consider a normal prior with the $MLE$ as the mean and variance $0.8^2\mathbb I_6$, where
$\mathbb I_6$ is the identity matrix of order $6$.
\\[2mm]
{\bf Form of the Bayes factor}
%\label{subsec:case1_bf}
\\[2mm]
In this case the related Bayes factor has the form
\begin{equation}
I_T=\int \frac {f_{\btheta_1,T}(X_{0,T})}{f_{\btheta_0,T}(X_{0,T})}\pi (d\btheta_1),
\label{eq:bf_T_appl2}
\end{equation}
where $\btheta_0=(\xi_{0,1},\xi_{0,2},\xi_{0,3},\xi_{0,4},\xi_{0,5},\xi_{0,6})$ is the true parameter set 
and $\btheta_1=(\xi_1,\xi_2,\xi_3,\xi_4,\xi_5,\xi_6)$ is the unknown set of parameters corresponding to 
any other model. 
%The prior distribution on the parameter set is denoted by $\pi$. The true likelihood is 
%$f_{\btheta_0,T}(X_{0,T})$. To check whether (\ref{eq:bf_appl}) holds we write $7$ 
%different wrong likelihoods in the place of $f_{\btheta_1,T}(X_{0,T})$ sequentially and obtain $7$ 
%different values of Bayes factor. 
%In this regard we use the $\btheta$ from a normal prior where 
%the mean is the $MLE$ of $\xi_i$ for $i=1,\cdots,6$ obtained by simulated annealing and variance is $0.8^2$. 
Table \ref{table:values2} describes the results of our Bayes factor analyses. 
%where we denote the presence and absence of covariates by 1 and 0 respectively.
\begin{table}[h]
\centering
\caption{Bayes factor results}
\label{table:values2}
\begin{tabular}{|c||c|}
\hline
Model & Averaged $\frac{1}{5}\log I_5$\\
\hline
$(0,0,0)$ & -2.5756029\\ 
$(0,0,1)$ & -0.913546\\
$(0,1,1)$ & -0.5454860\\
$(0,1,0)$ & -0.763952\\
$(1,0,0)$ & -2.5774163\\
$(1,0,1)$ & -0.9312218\\
$(1,1,0)$ & -0.7628154\\
\hline
\end{tabular}
\end{table}
It is clear from the 7 values of the table that the correct model $(1,1,1)$ is always preferred.

\subsubsection{Case 2: the parameter set $\btheta_0$ is random and has the prior distribution $\pi$}
\label{subsubsec:case2_appl2}
%{\bf Prior on $\btheta$}
%\\[2mm]
%\label{subsubsec:case2_prior_BF}
As before, we consider the same form of the prior as in Section \ref{subsubsec:case1_appl2},
but with variance $0.1^2\mathbb I_6$. 
In this case we calculate marginal likelihood of the 8 possible models, and approximate
$$\frac{1}{5}E_{\btheta_0}\left(\log\int f_{i,\btheta_1,5}(X_{0,5})\pi (d\btheta_1)\right)$$ for $i=1,\ldots,8$ by 
averaging over $1000$ replications of the data obtained from the true model. 
Denoting its values by $\ell_i$,  Table \ref{table:deltavalues2} shows that $\ell_8$ 
is the highest, implying consistency of the averaged Bayes factor.
\begin{table}[h]
\centering
\caption{Averages of $\frac{1}{5}\times$ marginal log-likelihood}
\label{table:deltavalues2}
\begin{tabular}{|c||c|}
\hline
Model & $\ell_i$\\
\hline
$(0,0,0)$ & -1.21923\\ 
$(0,0,1)$ & -0.21428\\
$(0,1,0)$ & 1.47992\\
$(0,1,1)$ & 2.102966\\
$(1,0,0)$ & -1.222362\\
$(1,0,1)$ & -0.21898\\
$(1,1,0)$ & 1.459921\\
$(1,1,1)$ & 2.121237 ($\mbox{true model}$)\\
\hline
\end{tabular}
\end{table}



\subsection{Bayes factor analysis for $n=15$ and $T=5$}
\label{subsec:N15T5}

In this case we allow our parameter and the covariate sets to vary from individual to individual. 
We consider $15$ individuals modeled by
\begin{equation}
dX_i(t)=(\xi_1^i+\xi_2^iz_1(t)+\xi_3^iz_2(t)+\xi_4^iz_3(t))(\xi_5^i+\xi_6^iX_i(t))dt+\sigma_i dW_i(t)
\label{eq:sde3_appl3}
\end{equation}
for $i=1,\cdots,15$. We fix our diffusion coefficients as $\sigma_{i+1}=\sigma_i +5$ for $i=1\cdots,14$ 
where $\sigma_1=10$. We consider the initial value $X(0)=0$ and the interval $[0, T]$, with $T=5$. 
As before, we generated the observed data after discretizing the time interval into $500$ equispaced time points. 
%We break the time domain in 500 time points i.e we break the time interval with the step size 0.01. 
Here our covariates and the parameter set 
$\btheta_0^i=(\xi_{0,1}^i,\xi_{0,2}^i,\xi_{0,3}^i,\xi_{0,4}^i,\xi_{0,5}^i,\xi_{0,6}^i)$; $i=1,\ldots,15$, 
are simulated in a similar way as mentioned in Section \ref{subsec:N1T5}.

For each of the $15$ individuals, the true set of covariate combination is randomly selected. 
Thus, for a given model, there are $15$ sets of covariate combinations to be compared with other models
consisting of $15$ different sets of covariate combinations.
To decrease computational burden we compare the true model with $100$ other models consisting of 
different sets of covariate combinations.  
%For the true covariate combination corresponding to 15 simulated data set fig \ref{fig:sim15} shows some of the simulated data type.
%
%\begin{figure}
%\begin{subfigure}
%\centering
%\includegraphics[height=6cm,width=5cm]{1stdatatime5.pdf}
%\label{fig:sfig2}
%\end{subfigure}
%\begin{subfigure}
%\centering
%\includegraphics[height=6cm,width=5cm]{8thdatatime5.pdf}
%\label{fig:sfig3}
%\end{subfigure}
%\begin{subfigure}
%\centering
%\includegraphics[height=6cm,width=5cm]{10thdatatime5.pdf}
%\label{fig:sfig4}
%\end{subfigure}
%\caption{Some Simulated Data Type}
%\label{fig:sim15}
%\end{figure}


The Bayes factor corresponding to the $j$-th covariate combination is given by
\begin{equation}
I_{nT}^j=\prod_{i=1}^n\int \frac{f^j_{i,\btheta^{(i)}_1}(X_{i,0,T})}
{f_{i,\btheta^{(i)}_0}(X_{i,0,T})}\pi\left(\btheta^{(i)}_{1}\right)d\btheta^{(i)}_{1}
\label{eq:I_iT_appl}
\end{equation}
for $j=1,\cdots,100$, where $n=15$, $T=5$ and $\btheta_0^{(i)}$ is the true parameter set corresponding to 
the $i$-th individual.

We obtain the $MLE$ of the $15$ parameter sets by simulated annealing. Then we calculate the Bayes factor 
with the prior such that the parameter components are independent normal with means as the respective $MLE$s and variances $1$. 
In all the cases corresponding to $100$ covariate combinations we obtain 
$\frac{1}{nT}\log I^j_{nT}<0$ for $j=1,\cdots,100$. 
Thus, Bayes factor indicated the correct covariate combination in all the cases considered. 
%As in section \ref{subsec:bf_appl} 
We also considered the case when a normal prior is considered for the parameters of the true model. 
In this case with respect to the component-wise independent normal prior with individual mean as obtained 
from simulated annealing and component-wise variance $0.1^2$, we obtain 
\begin{equation}
\frac{1}{15\times 5}\left[\log\left(\prod_{i=1}^{15}\int f^j_{i,\btheta_1^{(i)}}(X_{i,0,T})
\pi(\btheta^{(i)}_1)d\btheta_1^{(i)}\right)-\log\left(\prod_{i=1}^{15}\int f_{i,\btheta_0^{(i)}}(X_{i,0,T})
\pi(\btheta^{(i)}_0)d\btheta_0^{(i)}\right)\right]<0, %=\ell_j-\ell_0<0
\label{eq:INT_appl}
\end{equation}
for $j=1,\cdots,100$. Indeed, it turned out that 
$\frac{1}{15\times 5}\log\left(\prod_{i=1}^{15}\int f_{i,\btheta_0^{(i)}}(X_{i,0,T})
\pi(\btheta^{(i)}_0)d\btheta_0^{(i)}\right)=0.4865$ 
and the maximum of 
$\frac{1}{15\times 5}\log\left(\prod_{i=1}^{15}\int f ^j_{i,\btheta_1^{(i)}}(X_{i,0,T})
\pi(\btheta^{(i)})d\btheta^{(i)}\right)$ is $0.4127$.
In other words, the Bayes factor consistently selects the correct model even in this situation.


\section{Company-wise data from national stock exchange}
\label{sec:truedata} 

To deal with real data we collect the stock market data 
($467$ observations during the time range August $5$, 2013, to June $30$, 2015) 
for $15$ companies which is available on 
{\it www.nseindia.com}. The nature of some company-wise data are shown in Figure \ref{fig:true15}.

\begin{figure}
\begin{subfigure}
\centering
\includegraphics[height=6cm,width=5cm]{realdata1.pdf}
\label{fig:sfig2}
\end{subfigure}
\begin{subfigure}
\centering
\includegraphics[height=6cm,width=5cm]{realdata11.pdf}
\label{fig:sfig3}
\end{subfigure}
\begin{subfigure}
\centering
\includegraphics[height=6cm,width=5cm]{realdata12.pdf}
\label{fig:sfig4}
\end{subfigure}
\caption{Some company-wise time-series data.}
\label{fig:true15}
\end{figure}


Each company-wise data is modeled by various availabe standard financial $SDE$ 
models with the available ``fitsde" package in $R$. After obtaining the BIC (Bayesian Information Criterion) 
for each company corresponding to each available financial model, we find that the minimum value of BIC corresponds to 
the $CKLS$ model, given, for process $X(t)$, by
$$dX(t)=(\theta_1+\theta_2X(t))dt+\theta_3X(t)^{\theta_4}dW(t).$$
As per our theory we treat the diffusion coefficient as a fixed quantity. 
So, after obtaining the estimated value of the coefficients by the ``fitsde" function, 
we fix the values of $\theta_3$ and $\theta_4$, so that the diffusion coefficient becomes fixed.
We let $\theta_3=A$, $\theta_4=B$.


In this $CKLS$ model, we now wish to include time varying covariates. 
In our work we consider the ``close price" of each company. The stock market data is assumed to 
be dependent on IIP general index, bank interest rate, US dollar exchange rate and on various other quantities. 
But we assume only these three quantities as possibly the most important time dependent covariates. 

Briefly, IIP, that is, index of industrial production, is a measurement which represents 
the status of production in the industrial sector for a given period of time compared to a reference period of time. 
It is one of the best statistical data, which helps us measure the level of industrial activity in Indian economy. 
Its importance lies in the fact that low industrial production  will result in lower corporate sales and profits, 
which will directly affect stock prices. So a direct impact of weak IIP data is a sudden fall in stock prices. 

As the IIP data is purely industrial data, banking sector is not included in it. So, 
we also consider the bank interest rate as another covariate. Note that, higher the bank interest rate, 
fixed deposits become more attractive and one will preferably deposit money in bank rather than invest in stock market. 
Besides, companies with a high amount of loans in their balance sheets would be affected very seriously. 
Interest cost on existing debt would go up affecting their EPS (Earning per Share) and ultimately the stock prices. 
But during low interest rate these companies would stand to gain. Banking sector is likely to benefit most due 
to high interest rates. The Net Interest Margins (it is the difference between the interest they 
earn on the money they lend and the interest they pay to the depositors) for banks is likely to 
increase leading to growth in profits and the stock prices. Hence, it is clear that, the interest rates 
and stock markets are inversely related. As the interest rates go up, stock market 
activities tend to come down. 

Finally, exchange rates directly affect the realized return on 
an investment portfolio with overseas holdings. If one own stock in a foreign company and the 
local currency goes up, the value of the investment also goes up. 
Foreign investment is also related very much to US dollar exchange rate.

Hence, we collect the values of the aforementioned time varying covariates 
during the time range August $5$, 2013, to June $30$, 2015. The pattern of the covariates are 
displayed in Figure \ref{fig:cov}.

\begin{figure}
\begin{subfigure}
\centering
\includegraphics[height=6cm,width=5cm]{cov1.pdf}
\label{fig:realdata_sfig2}
\end{subfigure}
\begin{subfigure}
\centering
\includegraphics[height=6cm,width=5cm]{cov2.pdf}
\label{fig:realdata_sfig3}
\end{subfigure}
\begin{subfigure}
\centering
\includegraphics[height=6cm,width=5cm]{cov3.pdf}
\label{fig:realdata_sfig4}
\end{subfigure}
\caption{Covariates.}
\label{fig:cov}
\end{figure}




We denote these three covariates by $c_1,c_2,c_3$ respectively. Now, our considered $SDE$ models
for national stock exchange data associated with the $15$ companies are the
following:
\begin{equation}
dX_i(t)=(\theta_1^i+\theta_2^ic_1(t)+\theta_3^ic_2(t)+\theta_4^ic_3(t))(\theta_5^i+\theta_6^iX_i(t))dt+A^iX_i(t)^{B^i}dW_i(t),
\label{eq:sde4_appl}
\end{equation}
for $i=1,\cdots,15$. 
%It is important to mention that we are able to find out $467$ 
%data points which implies our data is observed in [0, 467] i.e. T=467.

\subsection{Selection of covariates by Bayes factor}
\label{subsec:covariate_bf}

Among the considered three time varying covariates we now select the best set of covariate combinations
for the $15$ companies among $100$ such sets through Bayes factor, 
computing the log-marginal-likelihoods with respect to the normal prior
on the parameter set, assuming {\it a priori} independence of the parameter components 
with individual means being the corresponding $MLE$ (based on simulated annealing)
and $0.01^2$ variance (relatively small variance ensured numerical stability of th marginal likelihood).
Table \ref{table:companies} provides the sets of covariates for the $15$ companies obtained
by our Bayes factor analysis. Also observe that each of the three covariates occurs about $50\%$ times
among the companies, demonstrating that overall impact of these on national stock exchange is
undeniable.
\begin{table}[h]
\centering
\caption{Company-wise covariates obtained by Bayes factor analysis}
\label{table:companies}
\begin{tabular}{|c||c|}
\hline
Company & Covariates\\
\hline
$1$ & Bank rate\\ 
$2$ & US dollar exchange rate\\
$3$ & None\\
$4$ & None\\
$5$ & Bank rate and US dollar exchange rate\\
$6$ & Bank rate and US dollar exchange rate\\
$7$ & IIP general index and US dollar exchange rate\\
$8$ & Bank rate\\
$9$ & IIP general index and Bank rate\\
$10$ & IIP general index\\
$11$ & IIP general index, Bank rate and US dollar exchange rate\\
$12$ & IIP general index and Bank rate\\
$13$ & US dollar exchange rate\\
$14$ & IIP general index, Bank rate and US dollar exchange rate\\
$15$ & IIP general index\\
\hline
\end{tabular}
\end{table}

\section{Summary and discussion}
\label{sec:conclusion}

This article establishes the asymptotic theory of Bayes factors when the models to be compared are
systems of $SDE$'s consisting of time-dependent covariates and random effects, assuming that the
number of individuals as well as the domains of observations of the individuals increase
indefinitely. Different initial values for different $SDE$'s are also allowed. 
The only instance of related effort in this direction is that of \ctn{Maitra15a}. 
The main difference of our undertaking with that of \ctn{Maitra15a} is that
they assumed the domains of observations to be fixed for the individuals, a consequence
being that incorporation of random effects in their model was not possible from the asymptotic
perspective. Moreover, in their case, a single set of covariates was associated with all the individuals,
but here our random effects set-up allows different sets of time-dependent covariates for different individuals.

To proceed, we first needed to build
an asymptotic theory of Bayes factors for comparing two individual $SDE$'s, rather than two systems of $SDE$'s,
as the domain of observation expands. Our results in this regard, which help
formulate our asymptotic theory for comparing two systems of $SDE$'s using Bayes factors, 
are perhaps also of independent interest, being possibly the first ever results in this direction of research.
Although the relevant variance did not converge to zero when two individual $SDE$'s are compared,
we are able to establish almost sure exponential convergence of the Bayes factor when the number of subjects are allowed to
increase indefinitely.
Importantly, our theory covers both $iid$ and non-$iid$ cases. 


Our simulation studies associated with covariate selection demonstrate that Bayes factor yields
consistent results even in non-asymptotic situations. Bayes factor analysis of a real data on company-wise national
stock exchange also yielded plausible sets of covariates for the companies.

%for non-$iid$ situations 
%readily extends to model comparison 
%problems when one of the models is associated
%with an $iid$ system of $SDE$'s and another with a non-$iid$ system of $SDE$'s.
%For instance, if the true model is associated with an $iid$ system, then $f_{0i}\equiv f_0\equiv f_{\btheta_0}$, and
%the rest of the theory remains the same as our non-$iid$ theory of Bayes factors. The case when the other model is associated 
%with an $iid$ system is analogous.

Note that our current asymptotic Bayes factor theory remains valid
for comparison between $iid$ and non-$iid$ models.
For instance, if the true model consititutes an $iid$ system, then $f_{0i}\equiv f_0\equiv f_{\btheta_0}$; 
the rest remains the same as the theory for our non-$iid$ setting. The situation is analogous when the other model forms
an $iid$ system.




\section*{Acknowledgments}
We are thankful to Dr. Diganta Mukherjee for drawing our attention towards the website 
{\it www.nseindia.com} at which the real data was available. 
The first author gratefully acknowledges her CSIR Fellowship, Govt. of India.

\newpage
\input{supp_T_arxiv}


\normalsize
\bibliographystyle{natbib}
\bibliography{irmcmc}

\end{document}

supp_T_arxiv.tex

\renewcommand\thefigure{S-\arabic{figure}}
\renewcommand\thetable{S-\arabic{table}}
\renewcommand\thesection{S-\arabic{section}}

\setcounter{section}{0}
\setcounter{figure}{0}
\setcounter{table}{0}

\begin{center}
{\bf \Large Supplementary Material}
\end{center}


Throughout, we refer to our main manuscript \ctn{Maitra15b} as MB. 

\begin{comment}
\section{Proof of Lemma 1 of MB}
\label{sec:proof_lemma_1}
We first consider $k=1$. 
Note that due to assumption (H4$^\prime$), %(\ref{eq:H4_prime_1}),
$$E_{\btheta_0}\left(V_{i,\btheta_1}\right)\leq 
T_i K_{\bbeta_1} \left(\underset{s\in [0,T_i]}{\sup}~\phi^2_{i,\bxi_j}(s)\right)
\left(1+\underset{s\in [0,T_i]}{\sup}~E_{\btheta_0}\left(X^2_i(s)\right)+\|\bbeta_1\|^2\right)<\infty,$$
since, by Proposition 1 of \ctn{Maud12}, $\underset{s\in [0,T_i]}{\sup}~E_{\btheta_0}\left(X^{2\ell}_i(s)\right)<\infty$,
for $\ell\geq 1$, and since $ \left(\underset{s\in [0,T_i]}{\sup}~\phi^2_{i,\bxi_j}(s)\right)$ is bounded
above due to continuity of $g_l$; $l=1,\ldots,p$. Hence, (3.6) of MB %(\ref{eq:finite_moment_v1}) 
holds.

Now observe that due to Cauchy-Schwartz and (H4$^\prime$) %(\ref{eq:H4_prime_1}) %and (\ref{eq:H4_prime_2}),
\begin{align}
E_{\btheta_0}\left(V_{i,\btheta_0,\btheta_j}\right)
&=\int_0^{T_i}E_{\btheta_0}\left(\frac{\phi_{i,\bxi_0}(s)\phi_{i,\bxi_j}(s)
b_{\bbeta_j}(X_i(s))b_{\bbeta_0}(X_i(s))}{\sigma^2(X_i(s))}\right)ds\notag\\
&\leq \int_0^{T_i}\left[E_{\btheta_0}\left(\frac{\phi^2_{i,\bxi_j}(s)b^2_{\bbeta_j}(X_i(s))}
{\sigma^2(X_i(s))}\right)\right]^{\frac{1}{2}}
\times \left[E_{\btheta_0}\left(\frac{\phi^2_{i,\bxi_0}(s)b^2_{\bbeta_0}(X_i(s))}
{\sigma^2(X_i(s))}\right)\right]^{\frac{1}{2}}ds\notag\\
&\leq T_i\left(\underset{s\in [0,T_i]}{\sup}~\phi^2_{i,\bxi_j}(s)\right)^{\frac{1}{2}}K^{\frac{1}{2}}_{\bbeta_j} 
\left(1+\underset{s\in [0,T_i]}{\sup}~
E_{\btheta_0}\left(X^2_i(s)\right)+\|\bbeta_j\|^2\right)^{\frac{1}{2}}\notag\\
&\quad\quad\quad\times K^{\frac{1}{2}}_{\bbeta_0}\left(\underset{s\in [0,T_i]}{\sup}~\phi^2_{i,\bxi_0}(s)\right)^{\frac{1}{2}}
\left(1+\underset{s\in [0,T_i]}{\sup}~E_{\btheta_0}\left(X^2_i(s)\right)+\|\bbeta_0\|^2\right)^{\frac{1}{2}}
\notag\\
&<\infty,\notag
\end{align}
by Proposition 1 of \ctn{Maud12} and boundedness of 
$\left(\underset{s\in [0,T_i]}{\sup}~\phi^2_{i,\bxi_j}(s)\right)$. 
%, $\underset{s\in [0,T_i]}{\sup}~E_{\btheta_0}\left(X^{2k}_i(s)\right)<\infty$,
%for $k\geq 1$.
Hence, (3.7) of MB %(\ref{eq:finite_moment_v}) 
holds.
%(\ref{eq:finite_moment_v}) follows from (H4$^\prime$) and (H1$^\prime$). 
Also note that since
$E_{\btheta_0}\left(U_{i,\btheta_j}\right)=E_{\btheta_0}\left(V_{i,\btheta_0,\btheta_j}\right)$
by (2.11) and (2.13), %(\ref{eq:u_v_relation}) and (\ref{eq:zero_mean}), 
(3.5) %(\ref{eq:finite_moment_u}) 
is implied by (3.7) of MB. %(\ref{eq:finite_moment_v}).
To see that the moments are continuous in $\theta_1$, 
let $\left\{\btheta^{(m)}_1\right\}_{m=1}^{\infty}$ 
be a sequence converging to $\tilde\btheta_1$ as $m\rightarrow\infty$. 
%Note that $\btheta^{(m)}_1=(\bxi^{(m)}_1,\bbeta^{(m)}_1)$ and $\tilde\btheta_1=(\tilde\bzeta_1,\tilde\bbeta_1)$. 
Due to (H3$^\prime$), 
$$\frac{\phi^2_{i,\bxi^{(m)}_1}(s)b^2_{\bbeta^{(m)}_1}(X_i(s))}{\sigma^2(X_i(s))}\rightarrow 
\frac{\phi^2_{i,\tilde\bxi_1}(s)b^2_{\tilde\bbeta_1}(X_i(s))}{\sigma^2(X_i(s))}$$
and
$$\frac{\phi_{i,\bxi^{(m)}_1}(s)\phi_{i,\bxi_0}(s)b_{\bbeta^{(m)}_1}(X_i(s))b_{\bbeta_0}(X_i(s))}{\sigma^2(X_i(s))}\rightarrow 
\frac{\phi_{i,\tilde\bxi_1}(s)\phi_{i,\bxi_0}(s)b_{\tilde\bbeta_1}(X_i(s))b_{\bbeta_0}(X_i(s))}{\sigma^2(X_i(s))},$$
as $m\rightarrow\infty$, for any given sample path $\left\{X_i(s):s\in [0,T_i]\right\}$.
Assumption %(\ref{eq:H4_prime_1}) of 
(H4$^\prime$) implies that 
$\frac{\phi^2_{i,\bxi^{(m)}_1}(s)b^2_{\bbeta^{(m)}_1}(X_i(s))}{\sigma^2(X_i(s))}$
is dominated by 
$\underset{\bxi_1\in\mathfrak \bGamma,s\in [0,T_i]}{\sup}~ \phi^2_{i,\bxi_1}(s)\times\underset{\bbeta_1\in\mathfrak B}{\sup}~ K_{\bbeta_1} 
\left(1+\underset{s\in [0,T_i]}{\sup}~\left[X_i(s)\right]^2+\underset{\bbeta_1\in\mathfrak B}{\sup}~\|\bbeta_1\|^2\right)$.
Since $X_i(s)$ is continuous on $[0,T_i]$, (guaranteed by (H2$^\prime$); see \ctn{Maud12}), it follows that
$\int_0^{T_i}\left[X_i(s)\right]^2ds<\infty$, which, in turn guarantees, in conjunction with
compactness of $\mathfrak B$ and $\bGamma$, that the upper bound is integrable. Hence, 
$V_{i,\btheta^{(m)}_1}\rightarrow V_{i,\tilde\btheta_1}$, almost surely.
Now, for all $m\geq 1$, 
\begin{equation*}
V_{i,\btheta^{(m)}_1}< T_i\left(\underset{\bxi_1\in\mathfrak \bGamma,s\in [0,T_i]}{\sup}~ \phi^2_{i,\bxi_1}(s)\right)
\times\left(\underset{\bbeta_1\in\mathfrak B}{\sup}~ K_{\bbeta_1}\right)
\times\left(1+\underset{s\in [0,T_i]}{\sup}~\left[X_i(s)\right]^2+\underset{\bbeta_1\in\mathfrak B}{\sup}~\|\bbeta_1\|^2\right).
\label{eq:v_bound}
\end{equation*}
Since $E_{\btheta_0}\left(\underset{s\in [0,T_i]}{\sup}~\left[X_i(s)\right]^2\right)<\infty$ 
by (3.1) of MB, %(\ref{eq:moment1}), 
it follows that
$E_{\btheta_0}\left(V_{i,\btheta^{(m)}_1}\right)
\rightarrow E_{\btheta_0}\left(V_{i,\tilde\btheta_1}\right)$,
as $\left(\btheta^{(m)}_1\right)\rightarrow \left(\tilde\btheta_1\right)$. 
Hence, $E_{\btheta_0}\left(V_{i,\btheta_1}\right)$
is continuous in $\btheta_1$. 

In the case of $V_{i,\btheta_0,\btheta_1}$, the relevant quantity
$\frac{\phi_{i,\bxi^{(m)}_1}\phi_{i,\bxi_0}b_{\bbeta^{(m)}_1}(X_i(s))b_{\bbeta_0}(X_i(s))}{\sigma^2(X_i(s))}$ 
is dominated by 
the continuous function (hence integrable on $[0,T_i]$) 
\begin{align}
&\left(\underset{\bxi_1\in\bGamma,s\in [0,T_i]}{\sup}~\left|\phi_{i,\bxi_1}(s)\right|\right)
\times\left(\underset{\bbeta_1\in\mathfrak B}{\sup}~ K^{\frac{1}{2}}_{\bbeta_1}\right)\times
\left(1+\underset{s\in [0,T_i]}{\sup}~\left[X_i(s)\right]^2+\underset{\bbeta_1\in\mathfrak B}{\sup}~\|\bbeta_1\|^2\right)\notag\\
&\quad\quad\quad\times% \underset{\bbeta_1\in\bTheta}{\sup}~ 
\left(\underset{s\in [0,T_i]}{\sup}~\left|\phi_{i,\bxi_0}(s)\right|\right)\times
K^{\frac{1}{2}}_{\bbeta_0}
\left(1+\underset{s\in [0,T_i]}{\sup}~\left[X_i(s)\right]^2+\|\bbeta_0\|^2\right),\notag
\end{align}
which ensures $V_{i,\btheta_0,\btheta^{(m)}_1}\rightarrow 
V_{i,\btheta_0,\tilde\btheta_1}$, almost surely.
Using the above bound for
\\
$\frac{\phi_{i,\bxi^{(m)}_1}\phi_{i,\bxi_0}b_{\bbeta^{(m)}_1}(X_i(s))b_{\bbeta_0}(X_i(s))}{\sigma^2(X_i(s))}$, 
it is seen that 
$$V_{i,\btheta_0,\btheta^{(m)}_1}< T_iK_1\left(\underset{s\in [0,T_i]}{\sup}~\left[X_i(s)\right]^4+K_2
\underset{s\in [0,T_i]}{\sup}~\left[X_i(s)\right]^2+K_3\right),$$
for appropriate positive constants $K_1,K_2,K_3$, so that (3.1) of MB %(\ref{eq:moment1}) 
for $k=4$, guarantees that
$E_{\btheta_0}\left(V_{i,\btheta_0,\btheta^{(m)}_1}\right)\rightarrow 
E_{\btheta_0}\left(V_{i,\btheta_0,\tilde\btheta_1}\right)$,
as $\btheta^{(m)}_1\rightarrow \tilde\btheta_1$. 
This shows that $E_{\btheta_0}\left(V_{i,\btheta_0,\btheta_1}\right)$
is continuous as well.
Since $E_{\btheta_0}\left(U_{i,\btheta_1}\right)
=E_{\btheta_0}\left(V_{i,\btheta_0,\btheta_1}\right)$, 
it follows that $E_{\btheta_0}\left(U_{i,\btheta_1}\right)$ is continuous in $\btheta_1$.


We now consider $k\geq 2$.
Note that, due to (H4$^\prime$), and the inequality $(a+b)^k\leq 2^{k-1}(|a|^{k}+|b|^{k})$ for $k\geq 2$ and any $a,b$, 
\begin{align}
E_{\btheta_0}\left(V_{i,\btheta_j}\right)^k
&\leq \left(\underset{s\in[0,T_i]}{\sup}~\left|\phi_{i,\bxi_j}(s)\right|^k\right)
2^{k-1}T_i^kK^k_{\bbeta_j} \left(1+\|\bbeta_j\|^2\right)^k\notag\\
&\quad\quad+\left(\underset{s\in[0,T_i]}{\sup}~\left|\phi_{i,\bxi_j(s)}\right|^k\right)2^{k-1}T_i^kK^k_{\bbeta_j}
E\left(\underset{s\in[0,T_i]}{\sup}~\left[X_i(s)\right]^{2k}\right).\notag
\end{align}
Since $E\left(\underset{s\in[0,T_i]}{\sup}~\left[X_i(s)\right]^{2k}\right)<\infty$ due to (3.1) of MB, %(\ref{eq:moment1}), 
and because $K_{\bbeta_j}$, $\|\bbeta_j\|$ are continuous in compact $\mathfrak B$, and 
$\left(\underset{s\in[0,T_i]}{\sup}~\left|\phi_{i,\bxi_j(s)}\right|^k\right)$ is continuous in compact
$\bGamma$, it holds that $E_{\btheta_0}\left(V_{i,\btheta_j}\right)^k<\infty$. In a similar manner it can be shown that
$E_{\btheta_0}\left(V_{i,\btheta_0,\btheta_1}\right)^k<\infty$. Thus, (3.7) of MB %(\ref{eq:finite_moment_v}) 
follows.

To see that (3.5) of MB %(\ref{eq:finite_moment_u}) 
holds, note that, due to (2.11) of MB %(\ref{eq:u_v_relation}) 
and 
$(a+b)^k\leq 2^{k-1}(|a|^{k}+|b|^{k})$,
\begin{equation}
E_{\btheta_0}\left(U_{i,\btheta_j}\right)^k\leq 2^{k-1}E_{\btheta_0}\left(V_{i,\btheta_0,\btheta_j}\right)^k
+2^{k-1}E_{\btheta_0}\left(\int_0^{T_i}
\frac{\phi_{i,\bxi_j}(s)b_{\bbeta_j}\left(X_i(s)\right)}{\sigma
\left(X_i(s)\right)}dW_i(s)\right)^k. 
\label{eq:u_v_relation2}
\end{equation}
Since, due to (H4$^\prime$), (3.1) of MB %(\ref{eq:moment1}) 
and continuity of $\phi_{i,\bxi_j}$ on compact spaces, 
$$E_{\btheta_0}\left(\int_0^T\left|\frac{\phi_{i,\bxi_j}(s)b_{\bbeta_j}(X_i(s))}
{\sigma(X_i(s))}\right|^kds\right)<\infty,$$
Theorem 7.1 of \ctn{Mao11} (page 39) shows that 
\begin{equation}
E_{\btheta_0}
\left(\left|\int_0^{T_i}\frac{\phi_{i,\bxi_j}(s)b_{\bbeta_j}(X_i(s))}
{\sigma(X_i(s))}dW(s)\right|^k\right)
\leq\left(\frac{k(k-1)}{2}\right)^{\frac{k}{2}}T_i^{\frac{k-2}{2}}
E_{\btheta_0}\left(\int_0^{T_i}\left|\frac{\phi_{i,\bxi_j}(s)b_{\bbeta_j}(X_i(s))}
{\sigma(X_i(s))}\right|^kds\right).
\label{eq:moment2}
\end{equation}
Combining (\ref{eq:u_v_relation2}) with (\ref{eq:moment2}) and the result 
$E_{\btheta_0}\left(V_{i,\btheta_0,\btheta_1}\right)^k<\infty$, it follows that
$E_{\btheta_0}\left(U_{i,\btheta_j}\right)^k<\infty$.

%Due to assumption (H2$^\prime$) (so that (\ref{eq:moment1}) holds) and (H4$^\prime$), 
%(\ref{eq:finite_moment_v}) is easily seen to hold.
%That (\ref{eq:finite_moment_u}) holds follows from (\ref{eq:u_v_relation}), (\ref{eq:zero_mean}),
%It\^{o} isometry, given by
%\begin{equation}
%E_{\btheta_0}\left[\int_0^{T_i}\frac{b_{\bbeta_j}\left(X_i(s)\right)}{\sigma_{\bgamma_j}\left(X_i(s)\right)}dW_i(s)\right]^2
%=\int_0^{T_i}E_{\btheta_0}\left[\frac{b_{\bbeta_j}\left(X_i(s)\right)}{\sigma_{\bgamma_j}\left(X_i(s)\right)}\right]^2ds,
%\label{eq:ito_isometry}
%\end{equation}
%%for $j=0,1$, 
%the inequality $(a+b)^k\leq 2^{k-1}(|a|^{k}+|b|^{k})$ for $k\geq 2$ and any $a,b$, and (\ref{eq:finite_moment_v}).

As regards continuity of the moments for $k\geq 2$, first note that in the context of $k=1$, 
we have shown almost sure continuity of $V_{i,\btheta_1}$ with respect to $\btheta_1$.
Hence, $V^k_{i,\btheta_1}$ is almost surely continuous with respect to $\btheta_1$. That is, 
$\btheta^{(m)}_1\rightarrow\tilde\btheta_1$ 
implies $V^k_{i,\btheta^{(m)}_1}\rightarrow V^k_{i,\tilde\btheta_1}$,
almost surely. Once again, dominated convergence theorem allows us to conclude that
$E_{\btheta_0}\left(V_{i,\btheta^{(m)}_1}\right)^k\rightarrow 
E_{\btheta_0}\left(V_{i,\tilde\btheta_1}\right)^k$,
implying continuity of $E_{\btheta_0}\left(V_{i,\btheta_1}\right)^k$ with respect to $\btheta_1$.
Similarly, it is easy to see that $E_{\btheta_0}\left(V_{i,\btheta_0,\btheta_1}\right)^k$ 
is continuous with respect to $\btheta_1$. 
To see continuity of $E_{\btheta_0}\left(U_{i,\btheta_1}\right)^k$, first note that
$$E_{\btheta_0}\left[\int_0^{T_i}\left(\frac{\phi_{i,\bxi^{(m)}_1}(s)b_{\bbeta^{(m)}_1}(X_i(s))}
{\sigma(X_i(s))}
-\frac{\phi_{i,\tilde\bxi_1}(s)b_{\tilde\bbeta_1}(X_i(s))}
{\sigma(X_i(s))}\right)^2ds\right]\rightarrow 0,$$
as $m\rightarrow\infty$. The result follows as before by first noting pointwise convergence, and then
using (H4$^\prime$) and then (3.1) of MB, %(\ref{eq:moment1}), 
along with (H1$^\prime$) and boundedness
of $\phi_{i,\bxi^{(m)}_1}$. By It\^{o} isometry it holds that
$$E_{\btheta_0}\left[\int_0^{T_i}\frac{\phi_{i,\bxi^{(m)}_1}(s)b_{\bbeta^{(m)}_1}(X_i(s))}
{\sigma(X_i(s))}dW_i(s)
-\int_0^{T_i}\frac{\phi_{i,\tilde\bxi_1}(s)b_{\tilde\bbeta_1}(X_i(s))}{\sigma(X_i(s))}dW_i(s)\right]^2
\rightarrow 0.$$
Hence, 
$$\int_0^{T_i}\frac{\phi_{i,\bxi^{(m)}_1}(s)b_{\bbeta^{(m)}_1}(X_i(s))}{\sigma(X_i(s))}dW_i(s)
\rightarrow \int_0^{T_i}\frac{\phi_{i,\tilde\bxi_1}(s)b_{\tilde\bbeta_1}(X_i(s))}{\sigma(X_i(s))}dW_i(s)$$
in probability, as $m\rightarrow\infty$. Since $V_{i,\btheta_0,\btheta_1^{(m)}}
\rightarrow V_{i,\btheta_0,\tilde\btheta_1}$ 
%$\zeta\left(x^i,\xi^{(m)}_1\right)\rightarrow \zeta\left(x^i,\tilde\xi_1\right)$ 
almost surely as
$m\rightarrow\infty$, it follows from (2.11) of MB %(\ref{eq:u_v_relation}) 
that 
$U_{i,\btheta^{(m)}_1}\rightarrow U_{i,\tilde\btheta_1}$ in probability, so that
$U^k_{i,\btheta^{(m)}_1}\rightarrow U^k_{i,\tilde\btheta_1}$ in probability. Using
(H4$^\prime$), (3.1) of MB %(\ref{eq:moment1}) 
and (H1$^\prime$), it is easily seen, using the same methods
associated with (\ref{eq:u_v_relation2}) and (\ref{eq:moment2}), that
$\underset{m}{\sup}~E_{\btheta_0}\left(U_{i,\btheta^{(m)}_1}\right)^{2k}<\infty$, proving that
$\left\{U^k_{i,\btheta^{(m)}_1}\right\}_{m=1}^{\infty}$ is uniformly integrable.
Hence, $E_{\btheta_0}\left(U_{i,\btheta^{(m)}_1}\right)^k\rightarrow 
E_{\btheta_0}\left(U_{i,\tilde\btheta_1}\right)^k$.
In other words, $E_{\btheta_0}\left(U_{i,\btheta_1}\right)^k$ is continuous in $\btheta_1$.


\section{Proof of Theorem 7 of MB}
\label{sec:proof_theorem_7}
Let us consider the martingale sequence
\[
S_N=\sum_{n=1}^{N}[\log(I_n/I_{n-1})+\mathcal K(f_{0n},\hat f_n)],
\]
which is a martingale because 
$E[\log(I_n/I_{n-1})|X_1,X_2,\ldots,X_{n-1}]=-\mathcal K(f_{0n},\hat f_n)$. 
Using the above it can be verified that if (5.8) of MB %(\ref{eq:convergent_series2}) 
holds, implying
\[
\sum_{n=1}^{\infty} n^{-2} Var\left[\log\frac{I_n}{I_{n-1}}\right]<\infty,
\]
then $S_N/N\rightarrow 0$ almost surely. Therefore
\begin{equation}
N^{-1}\log I_N + N^{-1} \sum_{n=1}^{N}\mathcal K(f_{0n},\hat f_n) \rightarrow 0,
\label{eq:I_n_convergence1}
\end{equation}
almost surely, as $N\rightarrow\infty$.

Now consider $N^{-1}\sum_{i=1}^N\log\frac{f_{0i}(X_i)}{f_i(X_i)}$. If (5.7) of MB %(\ref{eq:sup1}) 
holds, implying
\[
\sum_{i=1}^{\infty} i^{-2} Var\left[\log\frac{f_{0i}(X_i)}{f_i(X_i)}\right]<\infty,
\]
then by Kolmogorov's strong law of large numbers in the independent but non-identical case,
\[
\frac{1}{N}\sum_{i=1}^N\log\frac{f_{0i}(X_i)}{f_i(X_i)}\rightarrow
\mathcal K^{\infty}(f_0,f)
%\underset{N\rightarrow\infty}{\lim}~\frac {1}{N} \sum_{i=1}^N E\left[\log\frac{f_{0i}(X_i)}{f_i(X_i)}\right]
\]
almost surely, as $N\rightarrow\infty$. %We denote this limit by $\mathcal K^{\infty}(f_0,f)$.
%
Let  $\mathcal N_0(c)=\{f: \delta\leq\mathcal K^{\infty}(f_0,f)\leq\delta+c\}$, where $c>0$.
%and let $C_{\delta}(c_N)=\{f:\delta\leq \underset{i}{\inf}~\mathcal K(f_{0i},f_i)<\delta+c_N\}$, for 
%any strictly positive sequence $\left\{c_N\right\}$.
%With the assumption of Kullback-Leibler$(\delta)$ property:
%\[
%\Pi(f_i; \inf_i d_k(f_{0i},f_i)<c)>0
%\]
%Assume that $\pi(\mathcal N_0)>0$ for all $c>\delta\geq 0$.
%
Now, note that,
%\begin{align}
%I_N &=\int\frac{\prod_{i=1}^N f_i(X_i)}{\prod_{i=1}^N f_{0i}(X_i)}\pi (\tilde {df})\notag\\
%& \geq \int_{\mathcal N_0(a_N)\cap C_{\delta}(c_N)} 
%\exp\left( {\sum_{i=1}^N \log \frac {f_i(X_i)}{f_{0i}(X_i)}}\right)\pi (\tilde {df})\notag\\
%& =\int_{\mathcal N_0(a_N)\cap C_{\delta}(c_N)} 
%\exp\left(- {\sum_{i=1}^N \log \frac{f_{0i}(X_i)}{f_i(X_i)}}\right)\pi (\tilde {df})\notag\\
%\end{align}
%Since $\frac{1}{N}\sum_{i=1}^N\log\frac{f_{0i}(X_i)}{f_i(X_i)}\rightarrow \mathcal K^{\infty}(f_0,f)$ 
%almost surely as $N\rightarrow\infty$, and $\mathcal K^{\infty}(f_0,f)\leq\delta+a_N$ 
%on $\mathcal N_0(a_N)\cap C_{\delta}(c_N)$, we have
%\[
%I_N\gtrsim \exp\left\{-N(\delta+a_N)\right\}\pi(\mathcal N_0(a_N)\cap C_{\delta}(c_N)),
%\]
%
\begin{align}
I_N &=\int\frac{\prod_{i=1}^N f_i(X_i)}{\prod_{i=1}^N f_{0i}(X_i)}\pi (\tilde {df})\notag\\
& \geq \int_{\mathcal N_0(c)} 
\exp\left( {\sum_{i=1}^N \log \frac {f_i(X_i)}{f_{0i}(X_i)}}\right)\pi (\tilde {df})\notag\\
& =\int_{\mathcal N_0(c)} 
\exp\left(- {\sum_{i=1}^N \log \frac{f_{0i}(X_i)}{f_i(X_i)}}\right)\pi (\tilde {df}).\notag
\end{align}
By Jensen's inequality,
\begin{align}
\frac{1}{N}\log I_N&\geq -\int_{\mathcal N_0(c)}
\frac{1}{N}\left({\sum_{i=1}^N \log \frac{f_{0i}(X_i)}{f_i(X_i)}}\right)\pi (\tilde {df})
\label{eq:jensen}
\end{align}
The integrand on the right hand side converges to $\mathcal K^{\infty}(f_0,f)$, pointwise for every $f$, given any
sequence $\{X_i\}_{i=1}^{\infty}$ associated with the complement of some null set.
Since, for all such sequences, uniform integrability of the integrand is guaranteed by (5.7) of MB, %(\ref{eq:sup1}), 
it follows that
the right hand side of (\ref{eq:jensen}) converges to $-\int_{\mathcal N_0(c)}\mathcal K^{\infty}(f_0,f)\pi (\tilde {df})$ almost surely.
Hence, almost surely,
\begin{align}
\underset{N}{\liminf}~N^{-1}\log I_N&\geq -\int_{\mathcal N_0(c)}\mathcal K^{\infty}(f_0,f)\pi (\tilde {df})\notag\\
&\geq -(\delta+c)\pi\left(\mathcal N_0(c)\right)\notag\\
&\geq -(\delta+c).\notag
%\label{eq:jensen2}
\end{align}
Since $c>0$ is arbitrary, it follows that
%
%Since $\frac{1}{N}\sum_{i=1}^N\log\frac{f_{0i}(X_i)}{f_i(X_i)}\rightarrow \mathcal K^{\infty}(f_0,f)$ 
%almost surely as $N\rightarrow\infty$, $\mathcal K^{\infty}(f_0,f)\leq\delta+c$ 
%on $\mathcal N_0(c)$, it clearly follows that
%\[
%I_N\gtrsim \exp\left\{-N(\delta+c_N)\right\}\pi(\mathcal N_0(c_N)),
%\]
%where, by the Kullback-Leibler property, $\pi (\mathcal N_0(c_N))>0$ for any $N$.
%In the above, ``$\gtrsim$" indicates that there exists $N_0$ such that ``$\geq$" holds for all $N>N_0$. 
%This implies that, $$\frac{1}{N} \log I_N \gtrsim -\delta-a_N 
%+ \frac{1}{N}\log \pi (\mathcal N_0(c_N)).$$
%Since the second and the third terms on the right hand side of the above approximate inequality is always non-positive, 
%it is clear that
%Choosing $c$ as a sequence $\{c_N\}^\infty_{N=1}$ decreasing to $\delta$ as  $N\rightarrow \infty$,
%we can always get $\log\frac{\Pi (\mathcal N(C_N))}{N}\rightarrow 0$ as $N\rightarrow \infty$. Hence, we have 
\begin{equation}
\underset{N}{\lim\inf}~ N^{-1}\log I_N \geq -\delta, 
\label{eq:liminf_I_N}
\end{equation}
almost surely.
%
%Now consider the $Q^*$ property (\ref{eq:Q_star_non_iid}).
%, that is, $\liminf_n d_k(f_{0n},\hat f_{n,A_n(\delta)})\geq \delta$ whenever 
%$A_n(\delta)=\{f_n:d_k(f_{0n},f_n)\geq \delta\}$ for all $\delta \geq 0$.
%\\
%From the Kullback-Leibler $(\delta)$ property  $\Pi({f_i, \inf_i d_k(f_{0i},f_i)<c})>0$ for all $c>\delta \geq 0$, and 
Now, due to (5.2) of MB %(\ref{eq:prior_delta}) 
it follows that
%$$\pi(f: \underset{i}{\inf}~ \mathcal K(f_{0i},f_i)\geq \delta)=1.$$
$\mathcal K(f_{0n},f_n)\geq \delta$ for all $n$ with probability 1,
so that $\mathcal K(f_{0n},\hat f_n)=\mathcal K(f_{0n},\hat f_{n,A_n(\delta)})$,
where $A_n(\delta)$ is given by (5.5) of MB. %(\ref{eq:A_n}). 
By the $Q^*$ property %(\ref{eq:Q_star_non_iid}) 
it implies that  
\begin{equation*}
\underset{N}{\lim\inf}~ N^{-1} \sum_{n=1}^N \mathcal K(f_{0n},\hat f_n)\geq \delta.
\label{eq:liminf_K_average}
\end{equation*}
Hence, it follows from (\ref{eq:I_n_convergence1}) that
\begin{equation}
\underset{N}{\lim\sup}~ N^{-1} \log I_N \leq -\delta.
\label{eq:limsup_I_N}
\end{equation}
Combining (\ref{eq:liminf_I_N}) and (\ref{eq:limsup_I_N}) it follows that 
$$\underset{N\rightarrow\infty}{\lim}~N^{-1}\log I_N =-\delta,$$
almost surely.


\section{Proof of Lemma 10 of MB}
\label{sec:proof_lemma_9}

The proofs of (6.3) -- (6.5) of MB %(\ref{eq:finite_moment_u_non_iid}) -- (\ref{eq:finite_moment_v_non_iid})
follow in the same way as the proofs of (3.5) -- (3.7) of MB,  
%(\ref{eq:finite_moment_u}) -- (\ref{eq:finite_moment_v}), 
using compactness of $\mathfrak X$, $\mathfrak T$ 
and $\boldcal Z$ in addition to that of $\mathfrak B$ and $\bGamma$.

For the proofs of continuity of the moments, note that as in the $iid$ case,
uniform integrability is ensured by (H4$^\prime$), (3.1) of MB %(\ref{eq:moment1}) 
and compactness
of the sets $\mathfrak B$, $\bGamma$, $\mathfrak X$, $\mathfrak T$ and $\boldcal Z$. 
The rest of the proof is almost the same as the proof
of Theorem 5 of \ctn{Maitra14a}.


\section{Proof of Lemma 11 of MB}
\label{sec:proof_lemma_10}
For notational simplicity, let
\begin{align}
\breve V_{x^i,\bbeta_0}(s)&=\frac{b^2_{\bbeta_0}(X_i(s,x^i))}{\sigma^2(X_i(s,x^i))};\notag\\
\breve V_{x^i,\bbeta_0,\bbeta_j}(s)&=\frac{b_{\bbeta_j}(X_i(s,x^i))
b_{\bbeta_0}(X_i(s,x^i))}{\sigma^2(X_i(s,x^i))};\notag\\
\breve V_{x^i,\bbeta_1}(s)&=\frac{b^2_{\bbeta_1}(X_i(s,x^i))}{\sigma^2(X_i(s,x^i))}.\notag
\end{align}

Continuity of $\mathcal K(f_{\btheta_0,x,T,\bz},f_{\btheta_1,x,T,\bz})$ 
%(or $\mathcal K(f_{\btheta,x,T},f_{\btheta_0,x,T})$) and $\mathcal I(f_{\btheta_0,x,T})$
with respect to $x$ and $T$, the fact that $x^k\rightarrow x^{\infty}$ and 
$T_k\rightarrow T^{\infty}$ as $k\rightarrow\infty$, assumption (H5$^\prime$), and the dominated convergence theorem
together ensure that
\begin{align}
&\underset{n\rightarrow\infty}{\lim}~\frac{\sum_{k=1}^n\mathcal K(f_{\btheta_0,x^k,T_k,\bz_k},
f_{\btheta_1,x^k,T_k,\bz_k})}{n}\notag\\
&=\underset{n\rightarrow\infty}{\lim}~\frac{1}{n}
\sum_{k=1}^n\int_0^{T_k}\left\{\frac{{\phi_{k,\bxi_0}(s)}^2}{2}E_{\btheta_0}(\breve V_{x^k,\bbeta_0}(s))\right.
\notag\\
&\qquad\qquad\left.-\phi_{k,\bxi_0}(s)\phi_{k,\bxi_1}(s)E_{\btheta_0}(\breve V_{x^k,\bbeta_0,\bbeta_1}(s))
+\frac{{\phi_{k,\bxi_1}(s)}^2}{2}E_{\btheta_0}(\breve V_{x^k,\bbeta_1}(s))\right\}ds\notag\\
&=\underset{n\rightarrow\infty}{\lim}~\frac{1}{n}
\int_0^{T_k}\sum_{k=1}^n\left\{\frac{(\xi_{00}+\xi_{10}g_1(z_{k1}(s))
+\cdots+\xi_{p0}g_p(z_{kp}(s)))^2}{2}E_{\btheta_0}(\breve V_{x^k,\bbeta_0}(s))\right.\notag\\
&\quad\quad\left. -(\xi_{00}+\xi_{10}g_1(z_{k1}(s))+\cdots+\xi_{p0}g_p(z_{kp}(s)))(\xi_{01}
+\xi_{11}g_1(z_{k1}(s))+\cdots+\xi_{p1}g_p(z_{kp}(s)))E_{\btheta_0}(\breve V_{x^k,\bbeta_0,\bbeta_1}(s))\right.\notag\\
&\quad\quad\left. +\frac{(\xi_{01}+\xi_{11}g_1(z_{k1}(s))+\cdots+
\xi_{p1}g_p(z_{kp}(s)))^2}{2}E_{\btheta_0}(\breve V_{x^k,\bbeta_1}(s))\right\}ds\notag\\
&=\int_0^{T^{\infty}}\left\{\left(\frac{\xi_{00}^2}{2}+\xi_{00}\sum_{l=1}^p\xi_{l0}c_l(s)
+\frac{1}{2}\sum_{l=1}^p\sum_{m=1}^p\xi_{l0}\xi_{m0}c_l(s)c_m(s)\right)
E_{\btheta_0}(\breve V_{x^{\infty},\bbeta_0}(s))\right.\notag\\
&\quad\quad\left.-\left(\xi_{00}\xi_{01}+\xi_{00}\sum_{l=1}^p\xi_{l1}c_l(s)+\xi_{01}\sum_{l=1}^p\xi_{l0}c_l(s)+
\sum_{l=1}^p\sum_{m=1}^p\xi_{l0}\xi_{m1}c_l(s)c_m(s)\right)E_{\btheta_0}(\breve V_{x^{\infty},\bbeta_0,\bbeta_1}(s))
\right.\notag\\
&\quad\quad\left.+\left(\frac{\xi_{01}^2}{2}+\xi_{01}\sum_{l=1}^p\xi_{l1}c_l(s)
+\frac{1}{2}\sum_{l=1}^p\sum_{m=1}^p\xi_{l1}\xi_{m1}c_l(s)c_m(s)\right)
E_{\btheta_0}(\breve V_{x^{\infty},\bbeta_1}(s))\right\}ds,
%&=\mathcal K(f_{\btheta_0,x^{\infty},T^{\infty},\mathfrak C_0},f_{\btheta_1,x^{\infty},T^{\infty},\mathfrak C_1}),
\label{eq:kl_limit_1}
\end{align}
which is the Kullback-Leibler distance between the models of the same form as
$f_{\btheta_0,x,T,\bz}$ and $f_{\btheta_1,x,T,\bz}$, but with $x$, $T$ and $g_l(z_l(s))$ replaced with
$x^{\infty}$, $T^{\infty}$ and $c_l(s)$. 

\end{comment}

\section{Proof of Lemma 2 of MB}
\label{sec:proof_lemma_14}
Due to compactness of $\bGamma$ it follows, using the form of $\phi_{\bxi_j}$ provided in
(H5$^{\prime\prime}$), that the convergences (3.9), (3.10) and (3.11) of MB %(\ref{eq:phi_conv}), (\ref{eq:phisq_conv})
%and (\ref{eq:phiphi_conv}) 
are uniform over $\bGamma$. The same form shows that the above integrals  
are continuous in $\bxi_1$, for every $T>0$. Hence, due to uniform convergence, the limits 
$\bar\phi^{(1)}_{\bxi_1}$, $\bar\phi^{(2)}_{\bxi_1}$ and $\bar\phi^{(2)}_{\bxi_0,\bxi_1}$
are continuous in $\bxi_1$.



\section{Proof of Lemma 3 of MB}
\label{sec:proof_lemma_15}
The proofs of (i) -- (iv) follow from (H1$^{\prime\prime}$), (H4$^{\prime\prime}$), the results
%(3.9), 
(3.10) and (3.11) of MB
%(\ref{eq:phi_conv}), (\ref{eq:phisq_conv}) and (\ref{eq:phiphi_conv}) 
following from (H5$^{\prime\prime}$), %(\ref{eq:moment1}) 
(3.1) and its asymptotic form (3.2) %(\ref{eq:moment1_T}) 
(with $k=2$), 
using the relation (2.8) of MB. %(\ref{eq:u_v_relation}). 
To prove (v), 
note that, since for any $k\geq 1$, 
it holds, due to (H4$^{\prime\prime}$), (3.1) of MB %(\ref{eq:moment1}) 
and boundedness of $\phi_{\bxi_j}$ on $[0,T]$, that for $j=0,1$,
$$E\int_0^T\left|\frac{\phi_{\bxi_j}(s)b_{\bbeta_j}(s,X(s))}{\sigma(s,X(s))}\right|^{2k}ds<\infty,$$
it follows from Theorem 7.1 of \ctn{Mao11}, page 39, that
\begin{align}
E\left|\int_0^T\frac{\phi_{\bxi_j}(s)b_{\bbeta_j}(s,X(s))}{\sigma(s,X(s))}dW(s)\right|^{2k}
\leq\left(k(2k-1)\right)^{k}T^{k-1}E\int_0^T
\left|\frac{\phi_{\bxi_j}(s)b_{\bbeta_j}(s,X(s))}{\sigma(s,X(s))}\right|^{2k}ds.
\end{align}
Hence, using Chebychev's inequality, it follows that for any $\epsilon>0$,
\begin{align}
&P\left(\left|\frac{1}{T}\int_0^T\frac{\phi_{\bxi_j}(s)b_{\bbeta_j}(s,X(s))}{\sigma(s,X(s))}dW(s)\right|
>\epsilon\right)\notag\\
&\quad\quad\quad<\epsilon^{-2k}\left(k(2k-1)\right)^{k}T^{-(k+1)}E\int_0^T
\left|\frac{\phi_{\bxi_j}(s)b_{\bbeta_j}(s,X(s))}{\sigma(s,X(s))}\right|^{2k}ds.
\end{align}
In particular, if $k=2$ is chosen, then it 
follows from the above inequality, (H4$^{\prime\prime}$) , (3.2) of MB, %(\ref{eq:moment1_T}), 
and boundedness of
$\phi_{\bxi_j}$ on $[0,T]$, that
\begin{equation*}
\sum_{T=1}^{\infty}P\left(\left|\frac{1}{T}
\int_0^T\frac{\phi_{\bxi_j}(s)b_{\bbeta_j}(s,X(s))}{\sigma(s,X(s))}dW(s)\right|>\epsilon\right)
<\infty,
\end{equation*}
proving that 
$$\frac{1}{T}\int_0^T
\frac{\phi_{\bxi_j}(s)b_{\bbeta_j}(s,X(s))}{\sigma(s,X(s))}dW(s)\stackrel{a.s.}{\longrightarrow} 0.$$

To prove (vi), first note that 
\begin{align}
&E\left|\frac{1}{T}\int_0^T\left[\frac{\phi^2_{\bxi_j}(s)b^2_{\bbeta_j}(s,X(s))}{\sigma^2(s,X(s))}
-\phi^2_{\bxi_j}(s)\kappa_j(\bbeta_j)\right]ds\right|^{2k}\notag\\
&\leq T^{-1}E\int_0^T\phi^{4k}_{\bxi_j}(s)\left|\frac{b^2_{\bbeta_j}(s,X(s))}{\sigma^2(s,X(s))}
-\kappa_j(\bbeta_j)\right|^{2k}ds\notag\\
&\leq K_4T^{-1}E\int_0^T\left|\frac{b^2_{\bbeta_j}(s,X(s))}{\sigma^2(s,X(s))}
-\kappa_j(\bbeta_j)\right|^{2k}ds,
\end{align}
for some finite constant $K_4>0$.
The second last inequality is by H\"{o}lder's inequality, and the last inequality
holds because 
$\phi_{\bxi_j}(t)$ is uniformly bounded on $[0,\infty]$ thanks to compactness of $\boldcal Z$ and continuity of 
the functions $g_l;l=1,\ldots,p$. Hence, for any $\epsilon>0$,
\begin{align}
&P\left(\left|\frac{1}{T}\int_0^T\frac{\phi^2_{\bxi_j}(s)b^2_{\bbeta_j}(s,X(s))}{\sigma^2(s,X(s))}ds
-\kappa_j(\bbeta_j)\frac{1}{T}\int_0^T\phi^2_{\bxi_j}(s)ds\right|>\epsilon\right)\notag\\
&\qquad\qquad
<K_4\epsilon^{-2k}T^{-1}E\int_0^T\left|\frac{b^2_{\bbeta_j}(s,X(s))}{\sigma^2(s,X(s))}-\kappa_j(\bbeta_j)\right|^{2k}ds.
\notag
\end{align}
In the same way as the proof of (v), it follows, using the above inequality, 
%(\ref{eq:H4_prime_prime_1}) and (\ref{eq:moment1_T}), 
(3.3) and (3.2) of MB, that
\begin{equation*}
\sum_{T=1}^{\infty}P\left(\left|\frac{1}{T}\int_0^T\frac{\phi^2_{\bxi_j}(s)b^2_{\bbeta_j}(s,X(s))}{\sigma^2(s,X(s))}ds
-\kappa_j(\bbeta_j)\frac{1}{T}\int_0^T\phi^2_{\bxi_j}(s)ds\right|>\epsilon\right)
<\infty.
\end{equation*}
That is, 
$$\frac{1}{T}\int_0^T\frac{\phi^2_{\bxi_j}(s)b^2_{\bbeta_j}(s,X(s))}{\sigma^2(s,X(s))}ds
-\kappa_j(\bbeta_j)\frac{1}{T}\int_0^T\phi^2_{\bxi_j}(s)ds\rightarrow 0,$$
almost surely, as $T\rightarrow\infty$. Since, as $T\rightarrow\infty$, 
$\frac{1}{T}\int_0^T\phi^2_{\bxi_j}(s)ds\rightarrow\bar\phi^{(2)}_{\bxi_j}$
by (3.10) of MB, %(\ref{eq:phisq_conv}), 
the result follows.
Using (3.4) instead of (3.3), %(\ref{eq:H4_prime_prime_2}) instead of (\ref{eq:H4_prime_prime_1}), 
(vii) can be proved in the same way as (vi). 
The proofs of 
(viii) and (ix) follow from (v), (vi) and (vii), using the relation (2.8). %(\ref{eq:u_v_relation}).


\section{Proof of Lemma 4 of MB}
\label{sec:proof_lemma_16}
Using the Cauchy-Schwartz inequality twice we obtain
\begin{align}
&\frac{1}{T}\int_0^T
E\left(\frac{\phi_{\bxi_0}(s)\phi_{\bxi_1}(s)b_{\bbeta_0}(s,X(s))b_{\bbeta_1}(s,X(s))}{\sigma^2(s,X(s))}\right)ds
\notag\\
&\leq\frac{1}{T}\int_0^T
\sqrt{E\left(\frac{\phi^2_{\bxi_0}(s)b^2_{\bbeta_0}(s,X(s))}{\sigma^2(s,X(s))}\right)}
\times \sqrt{E\left(\frac{\phi^2_{\bxi_1}(s)b^2_{\bbeta_1}(s,X(s))}{\sigma^2(s,X(s))}\right)}ds\notag\\
&=\sqrt{\kappa_0(\bbeta_0)\kappa_1(\bbeta_1)+O\left(\exp\left(T^3-T^5\right)\right)}\times
\frac{1}{T}\int_0^T\left|\phi_{\bxi_0}(s)\right|\times\left|\phi_{\bxi_1}(s)\right|ds\notag\\
&\leq \sqrt{\kappa_0(\bbeta_0)\kappa_1(\bbeta_1)+O\left(\exp\left(T^3-T^5\right)\right)}\times
\left(\sqrt{\frac{1}{T}\int_0^T\phi^2_{\bxi_0}(s)ds}\right)\times
\left(\sqrt{\frac{1}{T}\int_0^T\phi^2_{\bxi_1}(s)ds}\right).
\label{eq:phi_kappa_2}
\end{align}
Taking the limit of both sides of (\ref{eq:phi_kappa_2}) as $T\rightarrow\infty$, using (ii) of 
Lemma 3 %\ref{lemma:convergence1} 
and the limits (3.10), %(\ref{eq:phisq_conv}), 
the result %(\ref{eq:phi_kappa_relations})
follows.


\section{Proof of Lemma 5 of MB}
\label{sec:proof_lemma_17}
For any $h\in (0,t)$,
\begin{align}
\frac{I_t}{I_{t-h}} 
&=\frac{\int_{\bTheta} \frac{f_{\btheta_1,t}(X_{0,t})}{f_{\btheta_0,t}(X_{0,t})}\pi (d\btheta_1)}
{\int_{\bTheta} \frac{f_{\btheta_1,{t-h}}(X_{0,t-h})}{f_{\btheta_0,{t-h}}(X_{0,t-h})}\pi (d\btheta_1)}\notag\\
&=\frac{\int_{\bTheta} \exp\left (\left(U_{\btheta_1,t} 
- U_{\btheta_0,t}\right)-\frac{\left(V_{\btheta_1,t}
-V_{\btheta_0,t}\right)}{2}\right) \pi(d\btheta_1)}
{\int_{\bTheta} \exp \left(\left(U_{\btheta_1,t-h} 
- U_{\btheta_0,t-h}\right)-\frac{\left(V_{\btheta_1,t-h}
-V_{\btheta_0,t-h}\right)}{2}\right) \pi(d\btheta_1)}\notag\\
&=\int_{\bTheta} \exp\left(\left(U_{\btheta_1,t-h} 
- U_{\btheta_0,t-h}\right)-\frac{\left(V_{\btheta_1,t-h}
-V_{\btheta_0,t-h}\right)}{2}\right)\notag\\
&\quad\times\frac{\exp\left(U_{\btheta_1,t-h,t}-U_{\btheta_0,t-h,t}
-\frac{\left(V_{\btheta_1,t-h,t}
-V_{\btheta_0,t-h,t}\right)}{2}\right)\pi(d\btheta_1)}
{\int_{\bTheta}\exp\left(\left(U_{\btheta_1,t-h} - U_{\btheta_0,t-h}\right)
-\frac{(V_{\btheta_1,t-h}-V_{\btheta_0,t-h})}{2}\right)\pi(d\btheta_1)}\notag\\
&=E\left[\exp\left(\int_{t-h}^t\frac{\phi_{\bxi_1}(s)b_{\bbeta_1}(s,X(s))}{\sigma^2(s,X(s))}dX(s)
-\int_{t-h}^t\frac{\phi_{\bxi_0}(s)b_{\bbeta_0}(s,X(s))}{\sigma^2(s,X(s))}dX(s)\right.\right.\notag\\
&\left.\left.\quad-\left(\frac{1}{2}
\int_{t-h}^t\frac{\phi_{\bxi_1}^2(s)b_{\bbeta_1}^2(s,X(s))}{\sigma^2(s,X(s))}ds
-\frac{1}{2}\int_{t-h}^t\frac{\phi_{\bxi_0}^2(s)b_{\bbeta_0}^2(s,X(s))}{\sigma^2(s,X(s))}ds\right)\right)
\bigg\vert\mathcal F_{t-h}\right]\notag\\
&=\frac{E\left[\exp\left(\int_{t-h}^t\frac{\phi_{\bxi_1}(s)b_{\bbeta_1}(s,X(s))}{\sigma^2(s,X(s))}dX(s)
-\frac{1}{2}\int_{t-h}^t\frac{\phi_{\bxi_1}^2(s)b_{\bbeta_1}^2(s,X(s))}{\sigma^2(s,X(s))}ds\right)
\bigg |\mathcal F_{t-h}\right]}{\exp\left(\int_{t-h}^t\frac{\phi_{\bxi_0}(s)b_{\bbeta_0}(s,X(s))}{\sigma^2(s,X(s))}dX(s)
-\frac{1}{2}\int_{t-h}^t\frac{\phi_{\bxi_0}^2(s)b_{\bbeta_0}^2(s,X(s))}{\sigma^2(s,X(s))}ds\right)}\notag\\
&=\frac{\hat f_{t-h,t}(X_{t-h,t})}{f_{\btheta_0,t-h,t}(X_{t-h,t})}.
%\label{eq:ration_I_t}
\end{align}
Hence, the result %(\ref{eq:bf_ratio2}) 
holds.


\section{Proof of Theorem 6 of MB}
\label{sec:proof_theorem_18}
Let us consider
\begin{equation}
S_{Tq_n}=Tq_n\sum_{r=0}^{n(T)-1}\left(J'_{rTq_n}+\tilde{\mathcal K}'_{rTq_n}\right), 
\label{eq:Rsum}
\end{equation}
where $q_n=\frac{1}{n(T)}$, where, given $T>0$, $n(T)$ is the 
number of intervals partitioning $[0,T]$ each of length $\frac{T}{n(T)}$. We assume that as $T\rightarrow\infty$,
$\frac{T}{n(T)}\rightarrow 0$.

It follows, using (H9$^{\prime\prime}$), %and (H12$^\prime$), 
that for any $T>0$, 
\begin{align}
E\left(\frac{S_{Tq_n}}{T}\right)&\rightarrow \frac{1}{T}\int_0^T\frac{d}{dt}E_{\btheta_0}\left(\log I_t\right)dt
+\frac{1}{T}\int_0^TE_{\btheta_0}\left[\tilde{\mathcal K}'_t(f_{\btheta_0},\hat f)\right]dt\notag\\
&=E_{\btheta_0}\left(\frac{1}{T}\log I_T\right)
+\frac{1}{T}\int_0^TE_{\btheta_0}\left[\tilde{\mathcal K}'_t(f_{\btheta_0},\hat f)\right]dt,
\label{eq:conv1}
\end{align}
as $n(T)\rightarrow\infty$, for any given $T>0$.
Also, since due to (4.26) of MB, %(\ref{eq:towards_martingale2}),  
%$E[J'_{rTq_n}|\mathcal F_{rTq_n}]=-\tilde{\mathcal K}'_{rTq_n}\left(f_{\btheta_0},\hat f\right)$, 
$E\left(J'_{rTq_n}+\tilde{\mathcal K}'_{rTq_n}|\mathcal F_{rTq_n}\right)=0$,
we must have $E\left(J'_{rTq_n}+\tilde{\mathcal K}'_{rTq_n}\right)
=E\left[E\left(J'_{rTq_n}+\tilde{\mathcal K}'_{rTq_n}|\mathcal F_{rTq_n}\right)\right]=0$, for any $r,T,n(T)$.
Hence, $E\left(\frac{S_{Tq_n}}{T}\right)=0$ for any $T,n(T)$. 
Thus, it follows from (\ref{eq:conv1}), that
\begin{equation}
E_{\btheta_0}\left(\frac{1}{T}\log I_T\right)
+\frac{1}{T}\int_0^TE_{\btheta_0}\left[\tilde{\mathcal K}'_t(f_{\btheta_0},\hat f)\right]dt
\rightarrow 0,\quad\mbox{as}~T\rightarrow\infty.
\label{eq:conv2}
\end{equation}
We now deal with the second term of the left hand side of (\ref{eq:conv2}).
Since, by (H6$^{\prime\prime}$), %the Kullback-Leibler property (\ref{eq:kl_property_t}) that
$$\pi\left(\btheta_1: \underset{t}{\inf}~\tilde{\mathcal K}'_t(f_{\btheta_0},f_{\btheta_1})\geq \delta\right)=1,$$
it holds that $\tilde{\mathcal K}'_t(f_{\btheta_0},f_{\btheta_1})\geq \delta$ for all $t$ with probability 1,
so that 
\[
%g^*(t)=\frac{\mathcal K(f_{0,t,t+h_{n^*}},\hat f_{1,t,t+h_{n^*}})}{h_{n^*}}
%=\frac{\mathcal K(f_{0,t,t+h_{n^*}},\hat f_{1,t,t+h_{n^*},A_t(\delta)})}{h_{n^*}}
\tilde{\mathcal K}'_t(f_{\btheta_0},\hat f)=\tilde{\mathcal K}'_t(f_{\btheta_0},\hat f_{A_t(\delta)}),
\]
where $A_t(\delta)$ is given by (4.19) of MB. %(\ref{eq:A_t}). 
The $Q^*$ property %(\ref{eq:Q_star_t}) 
implies that  
\begin{equation}
\underset{T}{\lim\inf}~ \frac{1}{T}\int_0^TE_{\btheta_0}\left[\tilde{\mathcal K}'_t(f_{\btheta_0},\hat f)\right]dt\geq \delta.
\label{eq:liminf_K_average_T}
\end{equation}
The results (\ref{eq:conv2}) and (\ref{eq:liminf_K_average_T}) imply that
\begin{align}
\underset{T}{\limsup}~ E_{\btheta_0}\left(\frac{1}{T}\log I_T\right)\leq -\delta.
\label{eq:limsup_T}
\end{align}
Now observe that
\begin{align}
I_T&=\int_{\bTheta} \exp\left(U_{\btheta_1,T}
-U_{\btheta_0,T}\right)
\times\exp\left\{-\frac{1}{2}\left(V_{\btheta_1,T}
-V_{\btheta_0,T}\right)\right\}
\pi(d\btheta_1)\notag\\
&\geq 
\int_{\mathcal N_0(c)}\exp\left\{T\left(\frac{U_{\btheta_1,T}}{T}
-\frac{U_{\btheta_0,T}}{T}\right)\right\}\notag\\
&\quad\quad\times\exp\left\{-\frac{T}{2}\left(\frac{V_{\btheta_1,T}}{T}
-\frac{V_{\btheta_0,T}}{T}\right)\right\}
\pi(d\btheta_1),
\label{eq:lower_bound1}
\end{align}
where $c>0$, and
\begin{align}
\mathcal N_0(c)&=\left\{\btheta_1\in\bTheta:\delta\leq\bar{K}^{\infty}\left(f_{\btheta_0},f_{\btheta_1}\right)
\leq\delta+c\right\}\notag\\
&=\left\{\btheta_1\in\bTheta:\delta\leq\frac{\bar\phi^{(2)}_{\bxi_0}}{2}\kappa_0(\bbeta_0)
-\bar\phi^{(2)}_{\bxi_0,\bxi_1}\bar{\kappa}(\bbeta_0,\bbeta_1)
+\frac{\bar\phi^{(2)}_{\bxi_1}}{2}\kappa_1(\bbeta_1)
\leq\delta+c\right\},\notag
\end{align}
the second line following from (4.15) of MB. %(\ref{eq:kl_average_infinite2}).
Using Jensen's inequality, we obtain
\begin{align}
\frac{1}{T}\log\left(I_T\right)&\geq 
\int_{\mathcal N_0(c)}\left[\left(\frac{U_{\btheta_1,T}}{T}
-\frac{U_{\btheta_0,T}}{T}\right)\right.\notag\\
&\quad\quad\left.-\frac{1}{2}\left(\frac{V_{\btheta_1,T}}{T}
-\frac{V_{\btheta_0,T}}{T}\right)\right]
\pi(d\btheta_1).
\label{eq:lower_bound2}
\end{align}
By (vi) -- (ix) of Lemma 3 of MB, %\ref{lemma:convergence1}, 
the integrand of the right hand side of the above
inequality, which we denote by $g_{X_T}(\btheta_1)$, converges to 
$g(\btheta_1)=-\left[\frac{\bar\phi^{(2)}_{\bxi_1}}{2}\kappa_1(\bbeta_1)
-\bar\phi^{(2)}_{\bxi_0,\bxi_1}\bar{\kappa}(\bbeta_0,\bbeta_1)
+\frac{\bar\phi^{(2)}_{\bxi_0}}{2}\kappa_0(\bbeta_0)\right]$, pointwise for every $\btheta_1$, given any 
path of the process $X$ in the complement of the null set. 
Due to (H1$^{\prime\prime}$), (H4$^{\prime\prime}$) and (3.2) of MB, %(\ref{eq:moment1_T}), 
$\underset{T}{\sup}~E_{\btheta_1}\left[g_{X_T}(\btheta_1)\right]^2<\infty$,
so that $\left\{g_{X_T}(\btheta_1)\right\}_{T=1}^{\infty}$ is uniformly integrable. Hence,
$$\int_{\mathcal N_0(c)}g_{X_T}(\btheta_1)\pi(d\btheta_1)\rightarrow 
\int_{\mathcal N_0(c)}g(\btheta_1)\pi(d\btheta_1),$$
given any path of the process $X$ in the complement of the null set.
Let us denote the left hand side of the above by $H_{X_T}$ let $H$ denote the right hand side.
We just proved that $H_{X_T}$ converges to $H$ almost surely. Now observe that
\begin{align}
\underset{T}{\sup}~E_{\btheta_0}\left[H_{X_T}\right]^2
&=\underset{T}{\sup}~E_{\btheta_0}\left[\int_{\mathcal N_0(c)}g_{X_T}(\btheta_1)\pi(d\btheta_1)\right]^2\notag\\
&\leq \int_{\mathcal N_0(c)}\underset{T}{\sup}~E_{\btheta_0}\left[g_{X_T}(\btheta_1)\right]^2\pi(d\btheta_1).\notag
\end{align}
Again, due to (H1$^{\prime\prime}$), (H4$^{\prime\prime}$) and (3.2) of MB, %(\ref{eq:moment1_T}), 
the last expression is finite, proving uniform integrability
of $\left\{H_{X_T}\right\}_{T=1}^{\infty}$. Hence, 
$$\underset{T\rightarrow\infty}{\lim}~E_{\btheta_0}\left(H_{X_T}\right)= E_{\btheta_0}\left(H\right)=H.$$
It follows that
\begin{align}
&\underset{T\rightarrow\infty}{\liminf}~E_{\btheta_0}\left[\frac{1}{T}\log\left(I_T\right)\right]
\geq \underset{T\rightarrow\infty}{\liminf}~E_{\btheta_0}\left(H_{X_T}\right)
=E_{\btheta_0}\left(H\right)\notag\\
&=\int_{\mathcal N_0(c)}g(\btheta_1)\pi(\btheta_1)d\btheta_1\notag\\
&\geq-\left(\delta+c\right)\pi\left(\mathcal N_0(c)\right)\notag\\
&\geq-\left(\delta+c\right).
\end{align}
Since the above holds for arbitrary $c>0$,
it holds that
\begin{equation}
\underset{T\rightarrow\infty}{\lim\inf}~E_{\btheta_0}\left(\frac{1}{T}\log I_T\right)\geq -\delta.
\label{eq:liminf_T}
\end{equation}
Thus (\ref{eq:limsup_T}) and (\ref{eq:liminf_T}) together help us conclude that 
$$E_{\btheta_0}\left(\frac{1}{T}\log I_T\right)\rightarrow-\delta,$$ as $T\rightarrow\infty$.

We now show that the variance of $\frac{1}{T}\log I_T$ is $O(1)$, as $T\rightarrow\infty$.
First note, due to compactness of $\bTheta$, the mean value theorem for integrals
ensure existence of $\acute\btheta_1=(\acute\bbeta_1,\acute\bxi_1)=\acute\btheta_1(W)\in\bTheta$, depending 
on the Wiener process $W$ %as the system has strong solution, 
such that
\begin{equation}
\log I_T=\left(U_{\acute\btheta_1,T}
-U_{\btheta_0,T}\right)
-\frac{1}{2}\left(V_{\acute\btheta_1,T}
-V_{\btheta_0,T}\right).
\label{eq:log_I_T}
\end{equation}

Now note that the results presented in Lemma 3 of MB continue to hold even when
$\btheta_1$ is replaced with $\acute\btheta_1$. Specifically, the following hold
in addition to the results of Lemma 3:
\begin{align}
\quad &\frac{V_{\acute\btheta_1,T}}{T}-\bar \phi^{(2)}_{\acute\bxi_1}\kappa_1(\acute\bbeta_1)
\stackrel{a.s.}{\longrightarrow} 0; \notag\\ %\label{eq:V_convergence}\\
%\quad &\frac{V_{\btheta_0,\btheta_1,T}}{T}\stackrel{a.s.}{\longrightarrow} 
%\bar \phi^{(2)}_{\bxi_0,\bxi_1}\bar{\kappa}(\bbeta_0,\bbeta_1), 
%\label{eq:V_convergence2}\\
%\quad &\frac{U_{\btheta_0,T}}{T}\stackrel{a.s.}{\longrightarrow} 
%\bar\phi^{(2)}_{\bxi_0}\kappa_0(\bbeta_0),
%\label{eq:U_convergence_0}\\
\quad &\frac{U_{\acute\btheta_1,T}}{T}-\bar\phi^{(2)}_{\bxi_0,\acute\bxi_1}\bar{\kappa}(\bbeta_0,\acute\bbeta_1)
\stackrel{a.s.}{\longrightarrow}0.\notag 
\end{align}
It follows that, as $T\rightarrow\infty$,
$$\ell_T=\frac{1}{T}\log I_T-\bar\phi^{(2)}_{\bxi_0,\acute\bxi_1}\bar{\kappa}(\bbeta_0,\acute\bbeta_1)
+\frac{1}{2}\bar \phi^{(2)}_{\acute\bxi_1}\kappa_1(\acute\bbeta_1)
+\frac{1}{2}\bar \phi^{(2)}_{\bxi_0}\kappa_0(\bbeta_0)
\stackrel{a.s.}{\longrightarrow} 0.$$
By uniform integrability arguments, which follow in similar lines as the proofs of Lemmas 1 and 10 of \ctn{Maitra15a} using 
(H4$^{\prime\prime}$), compactness, and Cauchy-Schwartz, it holds that
\begin{equation}
Var_{\btheta_0}\left(\ell_T\right)\rightarrow 0,~\mbox{as}~T\rightarrow\infty.
\label{eq:var_ell}
\end{equation}
Hence,
\begin{align}
Var_{\btheta_0}\left(\frac{1}{T}\log I_T\right)
&=Var_{\btheta_0}\left(\ell_T+\bar\phi^{(2)}_{\bxi_0,\acute\bxi_1}\bar{\kappa}(\bbeta_0,\acute\bbeta_1)
-\frac{1}{2}\bar \phi^{(2)}_{\acute\bxi_1}\kappa_1(\acute\bbeta_1)
-\frac{1}{2}\bar \phi^{(2)}_{\bxi_0}\kappa_0(\bbeta_0)\right)\notag\\
&= Var_{\btheta_0}\left(\ell_T\right)
+Var_{\btheta_0}\left(\bar\phi^{(2)}_{\bxi_0,\acute\bxi_1}\bar{\kappa}(\bbeta_0,\acute\bbeta_1)
-\frac{1}{2}\bar \phi^{(2)}_{\acute\bxi_1}\kappa_1(\acute\bbeta_1)
-\frac{1}{2}\bar \phi^{(2)}_{\bxi_0}\kappa_0(\bbeta_0)\right)\notag\\
&\qquad+2Cov_{\btheta_0}\left(\ell_T,\bar\phi^{(2)}_{\bxi_0,\acute\bxi_1}\bar{\kappa}(\bbeta_0,\acute\bbeta_1)
-\frac{1}{2}\bar \phi^{(2)}_{\acute\bxi_1}\kappa_1(\acute\bbeta_1)
-\frac{1}{2}\bar \phi^{(2)}_{\bxi_0}\kappa_0(\bbeta_0)\right).
\label{eq:var2}
\end{align}
By (\ref{eq:var_ell}), the first term of (\ref{eq:var2}) goes to zero as $T\rightarrow\infty$, and
the third, covariance term tends to zero by Cauchy-Schwartz and (\ref{eq:var_ell}). In other words,
as $T\rightarrow\infty$,
\begin{equation}
\left|Var_{\btheta_0}\left(\frac{1}{T}\log I_T\right)
-Var_{\btheta_0}\left(\bar\phi^{(2)}_{\bxi_0,\acute\bxi_1}\bar{\kappa}(\bbeta_0,\acute\bbeta_1)
-\frac{1}{2}\bar \phi^{(2)}_{\acute\bxi_1}\kappa_1(\acute\bbeta_1)\right)\right|
%-\frac{1}{2}\bar \phi^{(2)}_{\bxi_0}\kappa_0(\bbeta_0)\right)\right|
\rightarrow 0.
\label{eq:var3}
\end{equation}
However, $$Var_{\btheta_0}\left(\bar\phi^{(2)}_{\bxi_0,\acute\bxi_1}\bar{\kappa}(\bbeta_0,\acute\bbeta_1)
-\frac{1}{2}\bar \phi^{(2)}_{\acute\bxi_1}\kappa_1(\acute\bbeta_1)\right)\nrightarrow 0,$$
unless $\bar\phi^{(2)}_{\bxi_0,\acute\bxi_1}\bar{\kappa}(\bbeta_0,\acute\bbeta_1)
-\frac{1}{2}\bar \phi^{(2)}_{\acute\bxi_1}\kappa_1(\acute\bbeta_1)$ is constant almost surely.
It then follows from (\ref{eq:var3}) that
\begin{equation}
Var_{\btheta_0}\left(\frac{1}{T}\log I_T\right)=O(1),~\mbox{as}~T\rightarrow\infty.
\label{eq:var4}
\end{equation}

\section{Proof of Theorem 8 of MB}
\label{sec:proof_theorem_20}
%Assuming the set-up described in Section 2.2 of MB, %\ref{subsec:case2}, 
In our set-up it follows from (2.7) of MB that %(\ref{eq:tilde_I_nT}) that
\begin{equation}
\frac{1}{nT}\log\tilde I_{n,T}
=\frac{1}{n}\sum_{i=1}^n\frac{1}{T}\log I_{i,T}.
\label{eq:bf_convergence_iid}
\end{equation}
In the $iid$ case, given $T>0$, 
using the above form, it follows by the strong law of large numbers, that 
\begin{equation}
\underset{n\rightarrow\infty}{\lim}\frac{1}{nT}\log\tilde I_{n,T}= E_{\btheta_0}\left(\frac{1}{T}\log I_{i,T}\right),
\label{eq:convergence_given_T}
\end{equation}
almost surely. Now, in the $iid$ situation, for each $i$,
$E_{\btheta_0}\left(\frac{1}{T}\log I_{i,T}\right)\rightarrow -\delta$, as $T\rightarrow\infty$.
Hence, taking limit as $T\rightarrow\infty$ on both sides of (\ref{eq:convergence_given_T}) yields
\begin{equation}
\underset{T\rightarrow\infty}{\lim}\underset{n\rightarrow\infty}{\lim}\frac{1}{nT}\log\tilde I_{n,T}
= \underset{T\rightarrow\infty}{\lim}E_{\btheta_0}\left(\frac{1}{T}\log I_{i,T}\right)=-\delta,
\label{eq:convergence_T_n}
\end{equation}
almost surely, proving the theorem.


\section{Proof of Lemma 10 of MB}
\label{sec:proof_lemma_22}
Note that, due to compactness of $\mathfrak X$ and $\boldcal Z$ and continuity of the covariates
in time $t$, there exists $x^*\in\mathfrak X$ and $\bz^*\in\mathfrak Z$, such that
\begin{equation}
\underset{x\in\mathfrak X,\bz\in\mathfrak Z}{\sup}~
\left|\frac{1}{T}E_{\btheta_0}\left(\log I_{x,T,\bz}\right) +\delta(x,\bz)\right|
=\left|\frac{1}{T}E_{\btheta_0}\left(\log I_{x^*,T,\bz^*}\right) +\delta(x^*,\bz^*)\right|
\rightarrow 0,
\label{eq:uniform_convergence}
\end{equation}
as $T\rightarrow\infty$, where the convergence is due to (6.2) of MB. %(\ref{eq:bf_convergence_non_iid}). 
Also, $\frac{1}{T}E_{\btheta_0}\left(\log I_{x,T,\bz}\right)$ is clearly
continuous in $(x,\bz)$ for every $T>0$ (the proof of this follows in the same way as that of Theorem 5 of \ctn{Maitra14a}). 
Combining this with the uniform convergence (\ref{eq:uniform_convergence})
it follows that $\delta(x,\bz)$ is also continuous in $(x,\bz)$.


\section{Proof of Lemma 11 of MB}
\label{sec:proof_lemma_23}
Note that the limit (6.3) of MB %(\ref{eq:delta_convergence}) 
can be represented as
\begin{equation}
\underset{n\rightarrow\infty}{\lim}~\frac{1}{n}\sum_{r=0}^{n-1}
\varrho_{\delta}\left(\frac{r}{n}\right),
\label{eq:Riemann_delta}
\end{equation}
where $\varrho_{\delta}:[0,1]\mapsto\mathbb R^+$ is some continuous function satisfying 
$\varrho_{\delta}\left(\frac{r}{n}\right)=\delta\left(x^{r+1},\bz_{r+1}\right)$ for $r=0,\ldots,n-1$.
For the remaining points $y\in[0,1]$, we set $\varrho_{\delta}(y)=\delta(x,\bz)$,
where $(x,\bz)\in\mathfrak X\times\mathfrak Z$ is such that $\varrho_{\delta}(y)$ is continuous in $y\in[0,1]$. Since 
$\delta(x,\bz)$ is continuous in $(x,\bz)$, $\varrho_{\delta}(y)$ can be thus constructed.
Note that, it is possible to relate $y\in[0,1]$ to $(x,\bz)\in\mathfrak X\times\mathfrak Z$ by some continuous mapping
$G:\mathfrak X\times\mathfrak Z\mapsto[0,1]$, taking $(x,\bz)$ to $y$.
Thus, $\delta^{\infty}$ in (6.3) of MB
%(\ref{eq:delta_convergence}) 
is the limit of the Riemann sum (\ref{eq:Riemann_delta}) associated with the continuous
function $\varrho_{\delta}$; the limit is given by the integral $\int_0^1\varrho_{\delta}(y)dy$. 
Since the domain of integration is $[0,1]$, it follows, using continuity of $\varrho_{\delta}$, 
that the integral is finite. Observe that for any given sequence $\left\{(x^i,\bz_i)\right\}_{i=1}^{\infty}$, one
can construct a continuous function $\varrho_{\delta}$ such that $\delta^{\infty}=\int_0^1\varrho_{\delta}(y)dy$.
In other words,
$\delta^{\infty}$ %in (\ref{eq:delta_convergence})
exists for all sequences $\left\{(x^i,\bz_i)\right\}_{i=1}^{\infty}$.



\section{Proof of Theorem 12 of MB}
\label{sec:proof_theorem_24}
For given $T>0$, it follows from (\ref{eq:var4}), compactness of $\bTheta$, $\mathfrak X$, $\boldcal Z$,
and continuity of the relevant functions $\phi_{\bxi_j}$, 
$b_{\bbeta_j}$, $g_1,\ldots,g_p$, $\kappa_0$, $\kappa_1$ and $\bar\kappa$, that
\begin{equation}
\underset{x\in\mathfrak X,\bz\in\mathfrak Z}{\sup}~Var_{\btheta_0}\left(\frac{1}{T}\log I_{x,T,\bz}\right)<\infty.
\label{eq:var_sup_Z}
\end{equation}
Hence, given $T>0$,
\begin{equation*}
\sum_{i=1}^{\infty}\frac{Var_{\btheta_0}\left(\frac{1}{T}\log I_{i,T}\right)}{i^2}<\infty.
\end{equation*}
It then follows due to Kolmogorov's strong law of large numbers for independent random variables that
\begin{equation}
\underset{n\rightarrow\infty}{\lim}~\frac{1}{n}\sum_{i=1}^n\left(\frac{1}{T}\log I_{i,T}\right)
=\underset{n\rightarrow\infty}{\lim}~\frac{1}{n}\sum_{i=1}^nE_{\btheta_0}\left(\frac{1}{T}\log I_{i,T}\right),
\label{eq:non_iid_convergence1}
\end{equation}
almost surely.

Now observe that the right hand side of (\ref{eq:non_iid_convergence1}) admits the following representation
\begin{equation}
\underset{n\rightarrow\infty}{\lim}~\frac{1}{n}\sum_{r=0}^{n-1}\breve\varrho\left(\frac{r}{n},T\right),
\label{eq:Riemann1}
\end{equation}
where $\breve\varrho(\cdot,T):[0,1]\mapsto\mathbb R$ is some continuous function depending upon $T$ with 
$\breve\varrho(\frac{r}{n},T)=E_{\btheta_0}\left(\frac{1}{T}\log I_{r+1,T}\right)$.% for $r=0,\ldots,n-1$. 
 
Since $E_{\btheta_0}\left(\frac{1}{T}\log I_{x,T,\bz}\right)$ is continuous in $(x,\bz)$, $\breve\varrho(y,T)$ can be constructed 
as in Lemma 11 of MB. %\ref{lemma:delta_limit_existence}.
Then, for almost all $y\in [0,1]$, $\breve\varrho(y,T)\rightarrow -\delta(x,\bz)$ as $T\rightarrow\infty$, for appropriate
$(x,\bz)$ associated with $y$ via $y=G(x,\bz)$ as in Lemma 11. %\ref{lemma:delta_limit_existence}. 
Also, it follows from (\ref{eq:log_I_T}) that 
$\breve\varrho(\cdot,T)$ so constructed is uniformly bounded in $T>0$. 
Thus, the conditions
of the dominated convergence theorem are satisfied.

Since (\ref{eq:Riemann1}) is nothing but the Riemann sum associated with $\breve\varrho(\cdot)$, it follows that
\begin{equation}
\underset{n\rightarrow\infty}{\lim}~\frac{1}{n}\sum_{i=1}^nE_{\btheta_0}\left(\frac{1}{T}\log I_{i,T}\right)
=\int_0^1\breve\varrho(y,T)dy.
\label{eq:towards_DCT}
\end{equation}
By construction of $\breve\varrho(y,T)$, the dominated convergence theorem holds for the right hand side
of (\ref{eq:towards_DCT}). Hence, 
\begin{align}
&\underset{T\rightarrow\infty}{\lim}\underset{n\rightarrow\infty}{\lim}
~\frac{1}{n}\sum_{i=1}^nE_{\btheta_0}\left(\frac{1}{T}\log I_{i,T}\right)
=\underset{T\rightarrow\infty}{\lim}~\int_0^1\breve\varrho(y,T)dy
=\int_0^1\underset{T\rightarrow\infty}{\lim}~\breve\varrho(y,T)dy\notag\\
&\quad\quad=\underset{n\rightarrow\infty}{\lim}~\frac{1}{n}\sum_{i=1}^n
\underset{T\rightarrow\infty}{\lim}~E_{\btheta_0}\left(\frac{1}{T}\log I_{i,T}\right)
=-\underset{n\rightarrow\infty}{\lim}~\frac{1}{n}\sum_{i=1}^n\delta(x^i,\bz_i)=-\delta^{\infty}.
\label{eq:DCT}
\end{align}
Combining (\ref{eq:DCT}) with (\ref{eq:non_iid_convergence1}) it follows that
\begin{equation}
\underset{T\rightarrow\infty}{\lim}\underset{n\rightarrow\infty}{\lim}
~\frac{1}{n}\sum_{i=1}^n\left(\frac{1}{T}\log I_{i,T}\right)
=\underset{T\rightarrow\infty}{\lim}\underset{n\rightarrow\infty}{\lim}
~\frac{1}{n}\sum_{i=1}^nE_{\btheta_0}\left(\frac{1}{T}\log I_{i,T}\right)
=-\delta^{\infty},
\label{eq:non_iid_convergence2}
\end{equation}
almost surely.


bf_T.bbl
\begin{thebibliography}{}

\bibitem[Akaike(1973)Akaike]{Akaike73}
Akaike, H. (1973).
\newblock {I}nformation theory and an extension of the maximum likelihood
  principle.
\newblock In B.~N. Petrov and F.~Csaki, editors, {\em Second International
  Symposium on Information Theory\/}, pages 267--281, Budapest. Academiai
  Kiado.
\newblock Reprinted in S. Kotz and N. L. Johnson (Eds) (1992). {\it
  Breakthroughs in Statistics Volume I: Foundations and Basic Theory}, pp.
  610--624. Springer-Verlag.

\bibitem[Claeskens and Hjort(2008)Claeskens and Hjort]{Claeskens08}
Claeskens, G. and Hjort, N.~L. (2008).
\newblock {\em {M}odel {S}election and {M}odel {A}veraging\/}.
\newblock Cambridge University Press, Cambridge, UK.

\bibitem[Delattre {\em et~al.}(2013)Delattre, Genon-Catalot, and
  Samson]{Maud12}
Delattre, M., Genon-Catalot, V., and Samson, A. (2013).
\newblock {M}aximum {L}ikelihood {E}stimation for {S}tochastic {D}ifferential
  {E}quations with {R}andom {E}ffects.
\newblock {\em Scandinavian Journal of Statistics\/}, {\bf 40}, 322--343.

\bibitem[Dey {\em et~al.}(2000)Dey, Ghosh, and Mallick]{Dey00}
Dey, D.~K., Ghosh, S.~K., and Mallick, B.~K. (2000).
\newblock {G}eneralized {L}inear {M}odels: {A} {B}ayesian {P}erspective.

\bibitem[Fuchs(2013)Fuchs]{Fuchs13}
Fuchs, C. (2013).
\newblock {\em {I}nference for {D}iffusion {P}rocesses: {W}ith {A}pplications
  in {L}ife {S}ciences\/}.
\newblock Springer, New York.

\bibitem[Iacus(2008)Iacus]{Iacus08}
Iacus, S.~M. (2008).
\newblock {\em {S}imulation and {I}nference for {S}tochastic {D}ifferential
  {E}quations: {W}ith {R} {E}xamples\/}.
\newblock Springer, New York.

\bibitem[Jiang(2007)Jiang]{Jiang07}
Jiang, J. (2007).
\newblock {\em {L}inear and {G}eneralized {L}inear {M}ixed {M}odels and {T}heir
  {A}pplications\/}.
\newblock Springer, New York, USA.

\bibitem[Kass and Raftery(1995)Kass and Raftery]{Kass95}
Kass, R.~E. and Raftery, R.~E. (1995).
\newblock Bayes factors.
\newblock {\em Journal of the American Statistical Association\/}, {\bf
  90}(430), 773--795.

\bibitem[Kontoyiannis and Meyn(2003)Kontoyiannis and Meyn]{Konto03}
Kontoyiannis, I. and Meyn, S.~P. (2003).
\newblock {S}pectral {T}heory and {L}imit {T}heorems for {G}eometrically
  {E}rgodic {M}arkov {P}rocesses.
\newblock {\em The Annals of Applied Probability\/}, {\bf 13}, 304--362.

\bibitem[Leander {\em et~al.}(2015)Leander, Almquist, Ahlstr\"{o}m,
  Gabrielsson, and Jirstrand]{Leander15}
Leander, J., Almquist, J., Ahlstr\"{o}m, C., Gabrielsson, J., and Jirstrand, M.
  (2015).
\newblock {M}ixed {E}ffects {M}odeling {U}sing {S}tochastic {D}ifferential
  {E}quations: {I}llustrated by {P}harmacokinetic {D}ata of {N}icotinic {A}cid
  in {O}bese {Z}ucker {R}ats.
\newblock {\em The AAPS Journal\/}, {\bf 17}, 586--596.

\bibitem[Maitra and Bhattacharya(2015)Maitra and Bhattacharya]{Maitra14b}
Maitra, T. and Bhattacharya, S. (2015).
\newblock {O}n {B}ayesian {A}symptotics in {S}tochastic {D}ifferential
  {E}quations with {R}andom {E}ffects.
\newblock {\em Statistics and Probability Letters\/}, {\bf 103}, 148--159.
\newblock Also available at ``http://arxiv.org/abs/1407.3971".

\bibitem[Maitra and Bhattacharya(2016a)Maitra and Bhattacharya]{Maitra15a}
Maitra, T. and Bhattacharya, S. (2016a).
\newblock {A}symptotic {T}heory of {B}ayes {F}actor in {S}tochastic
  {D}ifferential {E}quations: Part {I}.
\newblock Submitted.

\bibitem[Maitra and Bhattacharya(2016b)Maitra and Bhattacharya]{Maitra15b}
Maitra, T. and Bhattacharya, S. (2016b).
\newblock {A}symptotic {T}heory of {B}ayes {F}actor in {S}tochastic
  {D}ifferential {E}quations: Part {II}.
\newblock Submitted.

\bibitem[Maitra and Bhattacharya(2016c)Maitra and Bhattacharya]{Maitra14a}
Maitra, T. and Bhattacharya, S. (2016c).
\newblock {O}n {A}symptotics {R}elated to {C}lassical {I}nference in
  {S}tochastic {D}ifferential {E}quations with {R}andom {E}ffects.
\newblock {\em Statistics and Probability Letters\/}, {\bf 110}, 278--288.
\newblock Also available at ``http://arxiv.org/abs/1407.3968".

\bibitem[Mao(2011)Mao]{Mao11}
Mao, X. (2011).
\newblock {\em {S}tochastic {D}ifferential {E}quations and {A}pplications\/}.
\newblock Woodhead Publishing India Private Limited, New Delhi, India.

\bibitem[M\"{u}ller {\em et~al.}(2013)M\"{u}ller, Scealy, and Welsh]{Muller13}
M\"{u}ller, A., Scealy, J.~L., and Welsh, A.~H. (2013).
\newblock {M}odel {S}election in {L}inear {M}ixed {M}odels.
\newblock {\em Statistical Science\/}, {\bf 28}, 135--167.

\bibitem[Oravecz {\em et~al.}(2011)Oravecz, Tuerlinckx, and
  Vandekerckhove]{Zita11}
Oravecz, Z., Tuerlinckx, F., and Vandekerckhove, J. (2011).
\newblock A hierarchical latent stochastic differential equation model for
  affective dynamics.
\newblock {\em Psychological Methods\/}, {\bf 16}, 468--490.

\bibitem[Overgaard {\em et~al.}(2005)Overgaard, Jonsson, Torn\oe, and
  Madsen]{Overgaard05}
Overgaard, R.~V., Jonsson, N., Torn\oe, C.~W., and Madsen, H. (2005).
\newblock {N}on-{L}inear {M}ixed-{E}ffects {M}odels with {S}tochastic
  {D}ifferential {E}quations: {I}mplementation of an {E}stimation {A}lgorithm.
\newblock {\em Journal of Pharmacokinetics and Pharmacodynamics\/}, {\bf 32},
  85--107.

\bibitem[Roberts and Stramer(2001)Roberts and Stramer]{Robert01}
Roberts, G. and Stramer, O. (2001).
\newblock {O}n {I}nference for {P}artially {O}bserved {N}onlinear {D}iffusion
  {M}odels {U}sing the {M}etropolis-hastings {A}lgorithm.
\newblock {\em Biometrika\/}, {\bf 88}, 603--621.

\bibitem[Schwarz(1978)Schwarz]{Schwarz78}
Schwarz, G. (1978).
\newblock {E}stimating the {D}imension of a {M}odel.
\newblock {\em The Annals of Statistics\/}, {\bf 6}, 461--464.

\bibitem[Sivaganesan and Lingham(2002)Sivaganesan and Lingham]{Siva02}
Sivaganesan, S. and Lingham, R.~T. (2002).
\newblock {O}n the {A}symptotic of the {I}ntrinsic and {F}ractional {B}ayes
  {F}actors for {T}esting {S}ome {D}iffusion {M}odels.
\newblock {\em Annals of the Institute of Statistical Mathematics\/}, {\bf 54},
  500--516.

\end{thebibliography}
