Imprecise Dirichlet process with application to the hypothesis test on the probability that $X\leq Y$
The Dirichlet process (DP) is one of the most popular Bayesian nonparametric models. An open problem with the DP is how to choose its infinite-dimensional parameter (base measure) in case of lack of prior information. In this work we present the Imprecise DP (IDP)---a prior near-ignorance DP-based model that does not require any choice of this probability measure. It consists of a class of DPs obtained by letting the normalized base measure of the DP vary in the set of all probability measures. We discuss the tight connections of this approach with Bayesian robustness and in particular prior near-ignorance modeling via sets of probabilities. We use this model to perform a Bayesian hypothesis test on the probability $P(X\leq Y)$. We study the theoretical properties of the IDP test (e.g., asymptotic consistency), and compare it with the frequentist Mann-Whitney-Wilcoxon rank test that is commonly employed as a test on $P(X\leq Y)$. In particular we will show that our method is more robust, in the sense that it is able to isolate instances in which the aforementioned test is virtually guessing at random.
IDP_ranksum_paper.tex
\documentclass[a4paper,final,3p,reqno,authoryear]{elsarticle}

\usepackage{hyperref}


% \journal{CSDA}
\usepackage{array}

\usepackage{graphicx}

\usepackage{amsmath}
\usepackage{mathptmx}
\usepackage{color}
  \usepackage{bbm}
 \usepackage{amssymb}
  \usepackage{natbib}
%   \usepackage{url}
%  \usepackage{etoolbox}
% \usepackage{times}
% Bayesian Analysis Log listings and useful macros

\setlength{\parskip}{0cm}
\setlength{\parindent}{1em}

% \usepackage{caption}
% \usepackage{subcaption}
% \newcounter{ssection}
% \newcommand{\section}[1]{\vspace{4mm}\refstepcounter{ssection}\par\noindent\textbf{\thessection. #1\\\noindent}}
% 
% \newcounter{ssubsection}
% \newcommand{\subsection}[1]{\vspace{2mm}\refstepcounter{ssubsection}\par\noindent\textbf{\thessection.\thessubsection. #1\\\noindent}}

\newcounter{Mtheorem}
\newenvironment{Mtheorem}[1][]{\refstepcounter{Mtheorem}\par\noindent\textbf{~Theorem~\theMtheorem. #1}\begin{itshape}}% begin code
{\end{itshape} \par}
%\newcounter{transformation}
% \newenvironment{transformation}[1][]{\refstepcounter{transfor}\par\noindent\textit{Transformation~\thetrasf. #1}\begin{itshape}}% begin code
% {\hspace*{\fill}$\blacksquare$ \end{itshape} \par}
\newcounter{trans}
\newenvironment{transformation}[1][]{\refstepcounter{trans}\par\noindent\textbf{~\\Transformation~\theMtrans. #1}\begin{itshape}}% begin code
{ \end{itshape} \par}
\newcounter{Mcorollary}
\newenvironment{Mcorollary}[1][]{\refstepcounter{Mcorollary}\par\noindent\textbf{~Corollary~\theMcorollary. #1}\begin{itshape}}% begin code
{\end{itshape} \par}
\newcounter{Mlemma}
\newenvironment{Mlemma}[1][]{\refstepcounter{Mlemma}\par\noindent\textbf{~Lemma~\theMlemma. #1}\begin{itshape}}% begin code
{\end{itshape}\par}
\newcounter{Proof}
\newenvironment{Proof}[1][]{\refstepcounter{Proof}\par\noindent\textbf{~\\Proof:~}\begin{itshape}}% begin code
{\end{itshape}\par}
% \newtheorem{Mlemma}{Lemma}
%\newtheorem{proposition}{Proposition}
%\newtheorem{Mcorollary}{Corollary}
% \newtheorem{remark}{Remark}
% \newcounter{remark}
% \newenvironment{remark}[1][]{\refstepcounter{remark}\par\noindent\textit{Remark~\theexample. #1}\begin{itshape}}% begin code
% {\hspace*{\fill}$\blacksquare$ \end{itshape}\par}
%\newtheorem{Mexample}{Example}
\newcounter{Mexample}
\newenvironment{Mexample}[1][]{\refstepcounter{Mexample}\par\noindent\textit{~Example~\theMexample. #1}\begin{itshape}}% begin code
{  \end{itshape}\par}
\newcounter{Mdefinition}
\newenvironment{Mdefinition}[1][]{\refstepcounter{Mdefinition}\par\noindent\textbf{~Definition~\theMdefinition. #1}\begin{itshape}}% begin code
{ \end{itshape}\par}
\newcounter{Mproposition}
\newenvironment{Mproposition}[1][]{\refstepcounter{Mproposition}\par\noindent\textbf{~\\Proposition~\theMproposition. #1}\begin{itshape}}% begin code
{ \end{itshape}\par}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\textwidth=31.9pc
\textheight=46.5pc
\oddsidemargin=1pc
\evensidemargin=1pc
\headsep=15pt
%\headheight=.2cm
\topmargin=.6cm
\parindent=1.7pc
\parskip=0pt

% \patchcmd{\thebibliography}{\chapter*}{\section*}{}{}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\setcounter{section}{0}
\def\thesection{\arabic{section}}
\setcounter{page}{1}
\pagestyle{myheadings}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% \begin{document}

% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% 
% \renewcommand{\baselinestretch}{1.2}
% 
% % \markright{
% % \hbox{\footnotesize\rm Statistica Sinica (2013): Preprint}\hfill
% % }
% 
% \markboth{\hfill{\footnotesize\rm Alessio Benavoli, Francesca Mangili, Fabrizio Ruggeri and Marco Zaffalon} \hfill}
% {\hfill {\footnotesize\rm Imprecise Dirichlet Process} \hfill}
% 
% \renewcommand{\thefootnote}{}
% $\ $\par
% 
% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% 
% \fontsize{10.95}{14pt plus.8pt minus .6pt}\selectfont
% \vspace{0.8pc}
% \centerline{\large\bf Imprecise Dirichlet Process  with application  to the hypothesis test }
% \vspace{2pt}
% \centerline{\large\bf on the probability that $X\leq Y$} % Mann-Whitney-Wilcoxon rank test}
% \vspace{.4cm}
% \centerline{Alessio Benavoli$^1$, Francesca Mangili$^1$, Fabrizio Ruggeri$^2$ and Marco Zaffalon$^1$}
% \vspace{.4cm}
% \centerline{\it $^1$ Istituto Dalle Molle di Studi sull'Intelligenza Artificiale (IDSIA), }
% \vspace{2pt}
% \centerline{\it Manno, Switzerland and $^2$ CNR IMATI, Milano, Italy}
% \vspace{.55cm}
% \fontsize{9}{11.5pt plus.8pt minus .6pt}\selectfont
% 
% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% 
% \begin{quotation}
% \noindent {\it Abstract:}
% The Dirichlet process (DP) is one of the most popular Bayesian nonparametric models. 
% An open problem with the DP is how to choose its infinite dimensional parameter (base measure) in case of lack of prior information. 
% In this work we present the Imprecise DP (IDP) -- a prior near-ignorance DP-based model that does not require any choice of this probability measure.
% It consists of a class of DPs obtained by letting the normalized base measure of the DP vary in the set of all probability measures. We discuss the tight connections of this approach with Bayesian robustness and in particular prior near-ignorance modeling via sets of probabilities.
% We use this model to perform a Bayesian hypothesis test on the probability $P(X\leq Y)$. We study the theoretical properties of the IDP test (e.g., asymptotic consistency), and compare it
% with the frequentist Mann-Whitney-Wilcoxon rank test that is commonly employed as a test on $P(X\leq Y)$. In particular we will show that our method is more robust, in the sense that it is able to isolate instances in which the aforementioned  test is virtually guessing at random.
% \par
% 
% \vspace{9pt}
% \noindent {\it Key words and phrases:}
% Bayesian nonparametric test, Imprecise Dirichlet Process, Wilcoxon rank sum
% \par
% \end{quotation}\par

\begin{document}

\begin{frontmatter}

\title{Imprecise Dirichlet process  with application  to the hypothesis test on the probability that $X\leq Y$}
% \tnotetext[mytitlenote]{Fully documented templates are available in the elsarticle package on \href{http://www.ctan.org/tex-archive/macros/latex/contrib/elsarticle}{CTAN}.}

%% Group authors per affiliation:
\author[mymainaddress1]{Alessio Benavoli}
\author[mymainaddress1]{ Francesca Mangili}
\author[mymainaddress2]{Fabrizio Ruggeri}
\author[mymainaddress1]{Marco Zaffalon}


\address[mymainaddress1]{ IPG IDSIA, Manno, Switzerland}
\address[mymainaddress2]{ CNR IMATI, Milano, Italy}

\begin{abstract}
The Dirichlet process (DP) is one of the most popular Bayesian nonparametric models. 
An open problem with the DP is how to choose its infinite-dimensional parameter (base measure) in case of lack of prior information. 
In this work we present the Imprecise DP (IDP)---a prior near-ignorance DP-based model that does not require any choice of this probability measure.
It consists of a class of DPs obtained by letting the normalized base measure of the DP vary in the set of all probability measures. We discuss the tight connections of this approach with Bayesian robustness and in particular prior near-ignorance modeling via sets of probabilities.
We use this model to perform a Bayesian hypothesis test on the probability $P(X\leq Y)$. We study the theoretical properties of the IDP test (e.g., asymptotic consistency), and compare it
with the frequentist Mann-Whitney-Wilcoxon rank test that is commonly employed as a test on $P(X\leq Y)$. In particular we will show that our method is more robust, in the sense that it is able to isolate instances in which the aforementioned  test is virtually guessing at random.
\end{abstract}

\begin{keyword}
Bayesian nonparametric test, Imprecise Dirichlet Process, Wilcoxon rank sum.
\end{keyword}

\end{frontmatter}


\section{Introduction}
\label{sec:1}
The Dirichlet process (DP) is one of the most popular Bayesian nonparametric models. 
It was introduced by \citet{Ferguson1973} as a prior over probability distributions.
In his seminal paper, Ferguson showed that the DP leads to tractable posterior inferences
and can be used for Bayesian analysis of several nonparametric models, such as the estimation of a distribution function, of a mean, of quantiles, of a variance, etc. He also considered the estimation of  $P (X \leq Y )$ assigning independent Dirichlet process priors to the distribution functions of $X$ and $Y$. 
The Mann-Whitney statistic naturally arises in this case. 
\citet{SusarlaRyzin1976} and \citet{Blum1977} extended the results of Ferguson on
estimation of the distribution function in case of right censored data obtaining a Bayesian
version of the Kaplan-Meier estimator.
\citet{dalai1983nonparametric} considered the problem of
estimating  a measure of dependence for a
bivariate distribution. The Bayes estimate is computed using a two-dimension Dirichlet prior 
and Kendall's tau is seen to appear naturally.
A review of other similar applications of the DP can be found in \citep{Phadia2013}.

The beauty of the DP is that most of these results are in closed form and that it
provides a Bayesian justification of  the classic nonparametric estimators. In spite of all these nice properties and of the promising initial outcomes, it is a matter of fact that such a research  has not resulted in the development of  DP-based Bayesian nonparametric procedures for hypothesis testing.
For instance, the most used statistical packages for DP-based Bayesian nonparametric modeling, ``DPpackage'' \citep{Dppackage} and ``Bayesm'' \citep{rossi2010bayesm}, include procedures for density estimation, clustering and regression, but do not include any Bayesian version of the Wilcoxon rank sum, Wilcoxon sign test or other classic nonparametric tests.


It is arguable that this absence may be related to the unsettled question of how to choose the prior ``parameters'' of the DP in case of lack of prior information. 
% The Dirichlet process (DP) is one of the most popular Bayesian nonparametric models. 
% It was introduced by \citet{Ferguson1973} as a prior over probability distributions.
% In his seminal paper, Ferguson showed that the DP leads to tractable posterior inferences
% and can be used for Bayesian analysis of several nonparametric models, e.g., to estimate a distribution function, quantiles, etc..
% Similar results were obtained by \citet{SusarlaRyzin1976}, \citet{Blum1977}, \citet{dalai1983nonparametric} and other authors. See
% \citet{Phadia2013} for a review of these results.
% such for instance the estimate of the probability 
% $P (X<Y)$ by assigning independent Dirichlet process priors to the distribution functions of $X$ and $Y$
% and the Mann-Whitney U statistic is seen to appear naturally. 
% \citet{SusarlaRyzin1976} and \citet{Blum1977} extended the results of Ferguson on
% estimation of the distribution function in case of right censored data obtaining a Bayesian
% version of the Kaplan-Meier estimator.
% \citet{dalai1983nonparametric} considered the problem of
% estimating  a measure of dependence for a
% bivariate distribution. The Bayes estimate is computed using a two-dimension Dirichlet prior 
% and Kendall's tau is seen to appear naturally.
%  Similar results were obtained by \citet{SusarlaRyzin1976,Blum1977,dalai1983nonparametric} 
%  for other classic nonparametric estimators; a review of these and other similar results can be found in \citet{Phadia2013}.
% The beauty of DP is that most of these results are in closed form and that it
% provides a Bayesian justification of  the classic nonparametric estimators.
It is well known in fact that a DP is completely characterized by its prior ``parameters'': the prior strength (or precision), which is a positive scalar number, and the normalized base measure. How should we  choose these prior ``parameters'' in case of lack of prior information? 
The only solution to this problem that has been proposed so far, first by \citet{Ferguson1973} and then by \citet{Rubin1981} under the name of Bayesian Bootstrap (BB), is the limiting DP obtained when the prior strength goes to zero. But the BB model has faced quite some
controversy, since it is not actually noninformative and moreover it assigns zero posterior probability to any set that does not include the observations. We will discuss these two points with more details in Section \ref{sec:PI}. 

In this paper we present an alternative viewpoint to the problem of choosing the prior base measure of the DP in case of lack of prior information that overcomes the above drawbacks of the BB. The model we present generalizes to nonparametrics  earlier ideas developed
in  Bayesian parametric robustness, see \citet{berger1994overview} and \citet{Ruggeri2000} for a review.
Here lack of prior information is expressed in terms of a family $\mathcal{T}$ consisting of all prior probability measures that are compatible with the available prior information. Inferences are then  carried out by considering the whole family $\mathcal{T}$.  In case almost no prior information is available on the parameters of interest, $\mathcal{T}$ should
be as large as possible in order to describe this state of prior ignorance.
The natural candidate for $\mathcal{T}$ to represent complete ignorance is the set of all probability measures.
However, it turns out that the posterior inferences obtained from this set are \textit{vacuous} \citep[Sec.~7.3.7]{walley1991}, i.e., 
 the posterior set   coincides with the prior set. This means that that there is no learning from data. Therefore, the vacuous prior model is not a statistically useful way to model our prior ignorance.
There is then a compromise to be made. \citet{pericchi_walley_91} and \citet{walley1991} suggest, as an alternative, the use of an almost vacuous model which they call ``near-ignorance'' or ``imprecise'' model. This is a model that behaves a priori as a vacuous model for some basic inferences (e.g., prior mean, prior credible regions)  but it always provides non-vacuous posterior inferences. 

While Bayesian robust models have already been extended to the nonparametric setting \citep{ruggeri1994}, 
that has not been the case for near-ignorance models.\footnote{A nonparametric model that uses lower and upper bounds for probabilities to quantify uncertainty
has been proposed by \citet{Augustin2004251,Coolen2009217}. This model is a purely predictive model, based on post-data assumptions, and,
thus, it cannot be used straightforwardly  to perform hypothesis tests.}
The main aim of this  paper is to derive a prior near-ignorance DP, called Imprecise DP (IDP).
This is the class $\mathcal{T}$ of all DPs obtained by fixing the prior strength of the DP and letting the normalized base measure vary in the set of all probability measures. We will show that the IDP  behaves a priori as a vacuous model for all predictive inferences.
This, together with the fact that it is a nonparametric model, allows us to start a statistical analysis with very weak assumptions about the problem of interest. However, contrarily to a full vacuous model,  we will show that the IDP can learn from data.

Moreover, we will employ the IDP to develop a new Bayesian nonparametric hypothesis test on the probability that $X\leq  Y$; we will call this test IDP rank-sum test, due to its similarity with the Mann-Whitney-Wilcoxon (MWW) rank-sum test.  
This hypothesis test is widely applied; for instance, if $X$ and $Y$ are health status measures in a clinical trial, $P(X\leq  Y )$ is, roughly speaking, the probability that the treatment represented by $Y$ is better (not worse) than the treatment represented by $X$. 
A Bayesian nonparametric near-ignorance model presents several advantages with respect to a traditional approach to hypothesis testing.
First of all, the Bayesian approach allows us to formulate the  hypothesis test as a decision problem. This means that we can  verify the evidence in favor of the null hypothesis and not only rejecting it, as well as take decisions that minimize the expected loss.
Second, because of the nonparametric near-ignorance prior, the IDP rank-sum test allows us to start the hypothesis test 
with very weak prior assumptions, much in the direction of letting data speak for themselves.
From a computational point of view, we will show that posterior inferences from the IDP can be derived by computing lower and upper bounds of expectations w.r.t.\
the class of DPs $\mathcal{T}$ and that,  for some inference, these lower and upper bounds can be computed in closed-form (e.g., mean and variance of $P(X\leq  Y )$).
When  no closed form expression exists,  these bounds can be computed by a simple Monte Carlo sampling from two  Dirichlet distributions. This means that we do not need to use stick breaking or other sampling approaches specific for DP. This computational advantage comes for free from prior near-ignorance.

In our view, the IDP rank-sum test appears to be a natural way to complete the work of \citet{Ferguson1973}, who first showed the connection between the expectation of 
$P(X\leq  Y )$ w.r.t.\ the DP and the Mann-Whitney statistic: it develops a Bayesian nonparametric near-ignorance-prior test for
the probability that $X\leq  Y $, which is computationally efficient and that, also for this reason, provides an effective practical alternative to the MWW  test.

Note that, although the IDP test shares several similarities with a standard Bayesian approach, at the same time it embodies a significant change of paradigm when it comes to take decisions.
In fact the IDP rank-sum test has the advantage of producing an \textit{indeterminate} outcome when the decision is \textit{prior-dependent}. In other words, the IDP test suspends the judgment (which can be translated as ``I do not know whether Y is  better than X'') when the option that minimizes the expected loss changes depending on the DP base measure we focus on. Therefore, the IDP-based test is \textit{robust} in the sense that it provides a determinate decision only when all the DPs, in the class the IDP represents, agree on the same decision.  
We will show that the number of indeterminate instances decreases at the accumulation of evidence and thus that the IDP-based test is always asymptotically consistent for $P(X\leq  Y )$.  This is not always true for the MWW test, even though the MWW test is commonly employed as a test about $P(X\leq  Y )$.

Finally, we will compare our IDP test with the MWW test and the DP-based test obtained as the prior strength goes to zero (called BB-DP test).
We empirically show on several different case studies that when the IDP test is indeterminate, 
the MWW and BB-IDP tests are virtually behaving as random guessers.
For a sample size of $20$ observations, the percentage of these instances can arrive up to almost $20\%$.
% This means that once every five decisions of MWW or BB-IDP tests are random guesses.
We regard this surprising result as an important finding, with practical consequences in hypothesis testing. Assume that we are trying to compare the effects of two  medical treatments (``Y  is better than X'') and that, given the available data, the IDP test is indeterminate. In such a situation the MWW test (or the BB-IDP test) always issues a determinate response (for instance, ``I can tell that Y  is better than X''), but it turns out that its response is completely random, like if we were tossing  a coin. In these cases by using MWW we would choose treatment $Y$, but this decision would be based on a random guess. In fact in these instances the MWW test could return the other hypothesis (``it is not true that Y  is better than X'') with equal probability.
On the other side, the IDP test acknowledges the impossibility of making a decision in these cases. Thus, by saying ``I do not know'', the IDP test  provides a richer information to the analyst. The analyst could for instance use this information to collect more data. (Please note that R and Matlab codes of our IDP rank-sum test are freely available at\\ \url{http://ipg.idsia.ch/software/IDP.php}.)


% % 
% % 
% % 
% % 
% % increasing of the number of observations
% % 
% % There are many kinds of predictive inferences and one of the most important is hypothesis testing.
% % 
% % 
% % 
% % 
% % Moreover, what it is the IDP rank-sum test has the advantage of producing an \textit{indeterminate} outcome when the decision is \textit{prior-dependent}. In other words, the IDP test suspends the judgment (I do not know whether ``Y is  better than X'') when the option which minimizes the expected loss depends on the prior. 
% % Therefore, the IDP rank sum test is \textit{robust} in the sense that it provides a determinate decision  only when this decision does not depend on the prior.
% % 
% % 
% % 
% % We will show that IDP behaves a priori as a vacuous model for all
% % predictive inferences. 
% % 
% % 
% % We call this model \textit{prior near-Ignorance DP} (IDP). The term \textit{near-ignorance} has been coined by \citet{pericchi_walley_91} and \citet{walley1991}
% % in the context of Bayesian parametric robustness. One of the aim of this paper is to extend near-ignorance to nonparametric priors and in particular to DP.
% % The adjective near comes from the fact that the prior strength is fixed,
% % In this view, after briefly reviewing prior near-ignorance in Bayesian parametric we show how to define a prior near-Ignorance DP (IDP).
% % and  how prior and posterior inferences can  be obtained from IDP by simply computing lower and upper bounds of expectations.
% %  , but the calculations 
% %  involved are as simple as those for deriving the classic nonparametric estimators.\vspace{-2mm}
% % A second important contribution of this paper is the application of the IDP model to hypothesis testing. In particular, we employ the IDP to develop a Bayesian nonparametric hypothesis test on the probability that $X\leq  Y$; we will call this test IDP rank-sum test due to its similarity with the Mann-Witney-Wilcoxon (MWW) rank-sum test. This hypothesis test is widely applied; for instance, if $X$ and $Y$ are health status measures in a clinical trial, $P(X\leq  Y )$ is, roughly speaking, the probability that the treatment represented by $Y$ is better (not worse) than the treatment represented by $X$. We formulate the  hypothesis test as a decision problem and we show that the obtained test has all the traditional strengths of the Bayesian approach: for instance, it allows verifying the evidence in favor of the null hypothesis and not only rejecting it and it  allows taking decisions which minimize the expected loss.  Moreover, the IDP rank-sum test has the advantage of producing an \textit{indeterminate} outcome when the decision is \textit{prior-dependent}. In other words, the IDP test suspends the judgment (I do not know whether ``Y is  better than X'') when the option which minimizes the expected loss depends on the prior. 
% % Therefore, the IDP rank sum test is \textit{robust} in the sense that it provides a determinate decision  only when this decision does not depend on the prior.
% % 
% % Recently there has been an increasing interest in the development of Bayesian nonparametric procedures for hypothesis testing and, in the context of this paper, it is worth mentioning Bayesian approaches to the two-sample problem. Indeed, given two i.i.d.\ sequences of real-valued observations from two cumulative distribution functions $F_X$ and $F_Y$,  the two-sample problem is to decide whether $F_X=F_Y$. 
% % Bayesian nonparametric approaches to the two-sample problem  have been proposed using Dirichlet Process Mixture (DPM) models or (coupling-optional) Polya tree priors by \citet{borgwardt2009bayesian,holmes2009two,ma2011coupling,RePEc:eee:csdana}.
% % In particular DPM or Polya tree are used to build a Bayes factor for testing whether two (or $k$ in case of \citet{RePEc:eee:csdana}) samples
% % arise from the same distribution.
% %  As for the DP even for these nonparametric priors the choice of the prior parameters (base measure, partition etc.) is critical.
% % In case of lack of prior information the common approach is either to standardize the data and then use some default priors or
% % use a sort of symmetric/uniform prior.
% % A possible drawback of this approach is that it may not perform well when the number of observations is small and, thus, when the
% % choice of the prior parameters has not a negligible effect on the posterior inferences.
% % In this case the decisions may be prior dependent and therefore not robust.
% % Prior near-ignorance models may also be very useful for these and other Bayesian nonparametric tests to address the issue of choosing the prior parameters in case of lack of prior information. 
% % Those approaches could be applied to test $X\leq Y$, since when  $F_X=F_Y$  then $P(X\leq Y)=0.5$ 
% % and therefore we cannot say that $ Y$ is better than $X$. Conversely, it may be possible that $P(X \leq Y)=0.5$ but $F_X\neq F_Y$
% % (e.g., two Normal distributions with same mean but different variance). In this latter case, we cannot use a
% % two-sample test $X\leq Y$. For this reason we do not compare our test with these approaches, but only with the MWW rank sum.
% % MWW test is a two sample test but it uses an estimate 
% % of $P(X\leq  Y )$ to perform the test and therefore, under certain assumptions on the true distributions of $X$ and $Y$ (i.e., location shift model) it 
% % can be used to test $P(X\leq  Y )$.
% % 
% % The only case where they can be compared is for location shift model, i.e., $F_y(z)=F_x(z-\Delta)$, since
% % in this case it is not possible that  $P(X<Y)=0.5$ but $F_X\neq F_Y$.
% % Among the available two-sample tests, the frequentist MWW rank-sum test has proved to be the most powerful in detecting locations shifts. 
% % For this reason, in this paper, we limit to compare our IDP rank-sum test with the MWW rank-sum test. 
% % We will prove that the IDP rank-sum test is always asymptotically consistent for $P(X\leq  Y )$, while this is not always true for the MWW test. Moreover, we will empirically show on several different case studies that, when the IDP test is indeterminate, 
% % the MWW test is practically behaving as a random guesser. The capability of the IDP test to isolate those indeterminate instances is valuable. Assume that we are trying to compare the effects of two  medical treatments (``Y  is better than X'') and that, given the available data, the IDP test is indeterminate. In such situation the MWW test always issues a determinate response (I can tell if ``Y  is better than X''), but it turns out that its response is completely random (like if we were tossing of a coin). On the other side, the IDP test acknowledges the impossibility of making a decision  and thus, the IDP test  provides more information to the analyst. 
% % 
% % Although from the theoretical point of view, our IDP rank-sum test may appear slightly more elaborate than  MWW, its implementation is easy; moreover, ready-to-use R and Matlab code of the IDP rank-sum test can be downloaded from {http://????}.
% % 
% % To the best of our knowledge, a DP based hypothesis test on the probability that MWW rank test has not been  reported in previous literature.
% % Despite these initial promising outcomes,  such a research  did not result in the development of DP-based Bayesian nonparametric procedures for hypothesis testing.
% % 
% % that uses  may be seen as a
% % Observe that these tests generalize to the Bayesian setting classic two-sample nonparametric tests as for instance  the Kolmgorov-Smirnov test, the Cramer-von-Mises test and partially the  MWW rank sum test.
% % % The MWW rank sum test can only partially be listed among the two-sample tests, because it is known that MWW rank sum test is able 
% % % to detect differences in distribution only under location shift models, i.e., $F_Y(z)=F_X(z-\theta)$ (in this case 
% % % it reduces to the \citet{hodges1963}'s estimator). 
% % % Observe that if  $F_X=F_Y$ then $P(X< Y )=0.5$ 
% % In this paper we are testing the probability that  $X< Y$ not the distributions of $X$ and $Y$.
% % This means that if the distributions of $X$ and $Y$ are different but $P(X< Y )=0.5$ (e.g., two Normal distributions with same mean but different variance), we say
% % that $Y$ is not better than $X$, while a two-sample test should state that
% % 
% % 
% % {\color{red}The MWW rank sum test can only partially be listed among the two-sample tests, because it is known that MWW rank sum test is able 
% % to detect differences in distribution only under location shift models, i.e., $F_2(x)=F_1(x-\theta)$ (in this case 
% % it reduces to the \citet{hodges1963}'s estimator). Under location shift models, MWW rank sum test has the highest power
% % among the above aforementioned tests, as shown by \citet[Fig.1.(a)]{RePEc:eee:csdana} - it is in fact used as test for differences in median.
% % While in the case $F_1\neq F_2$ but same median (e.g., two Normal distributions with same mean but different variance), this test
% % does not perform well; it is neither calibrated nor powerful......}
% % 
% % For this reason, we decided to test the null hypothesis P(Z+Z'>=0)=0.5, instead of m=0, to identify a significant difference between classifiers. For this hypothesis, the assumption of symmetry is not necessary in our test. 
% % 
% % As for the DP even for these nonparametric priors the choice of the prior parameters (base measure, partition etc.) is critical.
% % In case of lack of prior information the common approach is either to standardize the data and then use some default priors or
% % use a sort of symmetric/uniform prior (in the Polya tree a dyadic partition and an uniform base measure).
% % A possible drawback of this approach is that it may perform badly when the number of observations is small and, thus, when the
% % prior parameters have not a negligible effect on the posterior inferences.
% % 
% % \citet{hodges1963}
% % 
% % Classic two-sample nonparametric tests are the Kolmgorov-Smirnov test, the Cramer-von-Mises test and the  MWW rank sum test.
% % Although the 
% % 
% % Given two i.i.d.\ sequences of real-valued observations
% % from two cumulative distribution functions $F_1$ and $F_2$, the stochastic ordering problem is to decided whether $F_1(x)\leq F_2(x)$ for all $x \in \mathbb{R}$, while the two-sample problem is to decide whether $F_1=F_2$.
% % A Bayesian method for hypothesis testing of near equalities between  $F_1,F_2$ against a stochastically ordered alternatives has been proposed
% % by \citet{dunson2008bayesian} using a restricted  Dependent DP and based on earlier work from \citet{hoff_2003_biometrika}.
% % They consider an interval null hypothesis and rely on Gibbs sampling for posterior computation.
% % The elicitation of the prior parameters  is a critical part of the analysis.
% % In case of lack of prior information, the authors suggest first to standardize the data and then use some default priors.
% % Concerning the two-sample problems, Bayesian nonparametric approaches have been proposed using Dirichlet Process Mixture (DPM) models or (coupling-optional) Polya tree priors by \citet{borgwardt2009bayesian,holmes2009two,ma2011coupling,RePEc:eee:csdana}.
% % In particular DPM or Polya tree are used to build a Bayes factor for testing whether two (or $k$ in case of \citet{RePEc:eee:csdana}) samples
% % arise from the same distribution.
% % Even in this case the choice of the prior parameters (base measure, partition etc.) is critical.
% % In case of lack of prior information, the common approach is either to standardize the data and then use some default priors or
% % use a sort of symmetric/uniform prior (in the Polya tree a dyadic partition and an uniform base measure).
% % A possible drawback of this approach is that it may perform badly when the number of observations is small and, thus, when the
% % prior parameters have not a negligible effect on the posterior inferences.
% % \citet{hodges1963}
% % 
% % The rest of the paper is organized as follows...
% % 
% % 
% % 

% The aim of this work is twofold.
% 
% Examples of classic two-sample nonparametric tests are Kolmgorov-Smirnov test, Cramer-von-Mises test and only partially  Wilcoxon rank sum test.
% 
% First, we aim to answer to the question: how should we choose the prior probability measure of DP in case of lack of prior information?
% 
% 
% 
% 
% 
% 
% 
% 
% The use of DPM, Polya tree and Bayes factors for hypothesis testing  require the specification of an informative 
% base measure and even the choice of a partition for the Polya tree.
% This means that the prior specification can be problematic in case of lack of prior information. 




\section{Dirichlet process}
\label{sec:DP}
The Dirichlet process was developed by  \citet{Ferguson1973} as a probability distribution on the space of probability distributions. 
Let  $\mathbb{X}$ be a standard Borel space with Borel $\sigma$-field $\mathcal{B}_\mathbb{X}$  and $\mathbb{P}$ be the space
of probability measures on  $(\mathbb{X},\mathcal{B}_{\mathbb{X}})$ equipped with the weak topology and the
corresponding Borel $\sigma$-field $\mathcal{B}_\mathbb{P}$. Let $\mathbb{M}$ 
be the class of all probability measures on $(\mathbb{P},\mathcal{B}_\mathbb{P})$. We call the elements $\mu \in \mathbb{M}$
 nonparametric priors.

An element of  $ \mathbb{M}$ is called a Dirichlet process distribution $\mathcal{D}(\alpha)$ with base measure $\alpha$ if for every finite measurable
partition $B_1,\dots,B_m$ of $\mathbb{X}$, the vector $(P(B_1),\dots,P(B_m))$ has a Dirichlet distribution with parameters $(\alpha(B_1),\dots,\alpha(B_m))$, where $\alpha(\cdot)$ is a finite positive Borel measure on $\mathbb{X}$.
 Consider the partition $B_1=A$ and $B_2=A^c=\mathbb{X}\backslash A$ for some measurable set $A \in \mathbb{X}$, then if $P \sim \mathcal{D}(\alpha)$ from the definition of the DP we have that
$(P(A),P(A^c))\sim Dir(\alpha(A),\alpha(\mathbb{X})-\alpha(A) )$, which is a Beta distribution.
From the moments of the Beta distribution, we can thus derive that:
\begin{equation}
\label{eq:priormom}
\begin{array} {rl}
\mathcal{E}[P(A)]=\dfrac{\alpha(A)}{\alpha(\mathbb{X})},  &\mathcal{E}[(P(A)-\mathcal{E}[P(A)])^2]=\dfrac{\alpha(A)(\alpha(\mathbb{X})-\alpha(A))}{(\alpha(\mathbb{X})^2(\alpha(\mathbb{X})+1))},
\end{array}
\end{equation}
where we have used the calligraphic letter $\mathcal{E}$ to denote  expectation w.r.t.\ the Dirichlet process.
This shows that the normalized measure $\alpha(\cdot)/\alpha(\mathbb{X})$ of the DP reflects the prior expectation of $P$, while the scaling parameter $\alpha(\mathbb{X})$ controls how much $P$ is allowed to deviate from its mean $\alpha(\cdot)/\alpha(\mathbb{X})$.
Let $s = \alpha(\mathbb{X})$ stand for the total mass of $\alpha(\cdot)$  and $\alpha^*(\cdot)=\alpha(\cdot)/s$ stand for the
probability measure obtained by normalizing $\alpha(\cdot)$. If $P \sim \mathcal{D}(\alpha)$, we shall also
describe this by saying $P \sim Dp(s,\alpha^*)$ or, if $\mathbb{X} = \mathbb{R}$, $P \sim Dp(s, G_0)$, where $G_0$ stands
for the cumulative distribution function of $\alpha^*$.

Let $P\sim Dp(s,\alpha^*)$ and $f$ be a real-valued bounded function defined on $(\mathbb{X},\mathcal{B})$. Then the expectation with respect to the Dirichlet process of $E[f]$ is
\begin{equation}
\label{eq:expf}
\mathcal{E}\big[E(f)\big]=\mathcal{E}\left[\int f dP\right]=\int f d\mathcal{E}[P] = \int f d\alpha^*.
\end{equation}
One of the most remarkable properties of the DP priors is that the posterior distribution of $P$ is again a DP.
Let $X_1,\dots,X_n$ be an independent and identically distributed sample from $P$ and $P \sim Dp(s,\alpha^*)$, then the posterior distribution of $P$ given the observations is  
\begin{equation}
\label{eq:DPPost}
P|X_1,\dots,X_n \sim Dp\left(s+n, \frac{s}{s+n} \alpha^*+ \frac{1}{s+n} \sum\limits_{i=1}^n \delta_{X_i}\right),
\end{equation}
where $\delta_{X_i}$ is an atomic probability measure centered at $X_i$. This means that the Dirichlet process satisfies a property of conjugacy, in the sense that the posterior for $P$ is again a Dirichlet process with updated unnormalized base measure $\alpha+ \sum_{i=1}^n \delta_{X_i}$.
From (\ref{eq:DPPost}) and (\ref{eq:priormom})--(\ref{eq:expf}), we can easily derive the posterior mean and variance of $P(A)$ and, respectively, posterior expectation of $f$.
% This means that the Dirichlet process satisfies a property of conjugacy, in the sense that the posterior for $P$ is again a Dirichlet process with updated base measure.
Hereafter we list some useful properties of the DP that will be used in the sequel (see \citet[Ch. 3]{ghosh2003bayesian}).
\begin{description}
 \item[(a)] In case $\mathbb{X} = \mathbb{R}$, since $P$ is completely defined by its cumulative distribution function $F$, a-priori we say $F \sim Dp(s, G_0)$ and a posteriori we can rewrite (\ref{eq:DPPost}) as follows:
%  in an equivalent form. Assume that $F \sim Dp(s, G_0)$,  the posterior distribution of $F$ given $X_1,\dots,X_n$ is:
\begin{equation}
\label{eq:DPCDF}
F|X_1,\dots,X_n \sim Dp\left(s+n, \frac{s}{s+n} G_0+ \frac{1}{s+n} \sum_{i=1}^n I_{[X_i,\infty)}\right),
\end{equation} 
where $I$ is the indicator function.
\item[(b)] Consider an element $\mu \in \mathbb{M}$ which puts all its mass at the probability measure $P=\delta_{x}$ for some $x \in \mathbb{X}$.
This can also be modeled as  $Dp(s,\delta_x)$ for each $s>0$.
\item[(c)] Assume that $P_1 \sim Dp(s_1,\alpha_1^*)$, $P_2 \sim Dp(s_2,\alpha_2^*)$, $(w_1,w_2)\sim Dir(s_1,s_2)$ and $P_1$, $P_2$, $(w_1,w_2)$ are independent, then \citep[Sec. 3.1.1]{ghosh2003bayesian}:
\begin{equation}
 w_1P_1+w_2P_2 \sim Dp\left(s_1+s_2, \frac{s_1}{s_1+s_2} \alpha_1^*+ \frac{s_2}{s_1+s_2} \alpha_2^*\right).
\end{equation} 
\item[(d)] Let $P_x$ have distribution $Dp(s+n, \frac{s}{s+n} \alpha^*+ \frac{1}{s+n} \sum_{i=1}^n \delta_{X_i})$. We can write
\begin{equation} \label{eq:mixing}
P_x=w_0 P+ \sum_{i=1}^n w_i \delta_{X_i},
\end{equation} 
where  $(w_0,w_1,\dots,w_n)\sim Dir(s,1,\dots,1)$ and $P \sim Dp(s,\alpha^*)$ (it follows by (b)-(c)).
\end{description}
% 


\section{Prior ignorance}
\label{sec:PI}
How should we choose the prior parameters $(s,\alpha^*)$ of the DP, in particular the infinite-dimensional  $\alpha^*$, in case of lack of prior information? 
To address this issue, the only prior that has been proposed so far is the limiting DP obtained for $s \rightarrow 0$, which has been introduced under the name of Bayesian Bootstrap (BB) by \citet{Rubin1981}; in fact it can be proven that the BB is asymptotically equivalent (see \citet{Lo1987} and \citet{Weng1989}) to the frequentist bootstrap introduced by  \citet{Efron1979}.

The BB has been criticized on diverse grounds. 
From an a-priori point of view, the main criticism is that taking $s \rightarrow 0$ is far from leading to  a noninformative prior.
\citet{sethuraman1981} have shown that for $s \rightarrow 0$ a measure sampled
from the DP is a degenerated (atomic) measure centered on $X_0$, with $X_0$ distributed according to $\alpha^*$.
% if we consider a sequence of measures $\alpha_r$ such that, $s \rightarrow 0$ and the normalized measure $\alpha_r/s$ converges in the strong sense to the probability measure $\alpha^*$, then the sequence of Dirichlet processes with parameter $\alpha_r$, converges, for $r\rightarrow \infty$, to a random degenerated measure centered on $X_0$, with $X_0$ distributed according to $\alpha^*$. 
% This result depends on the fact that small values of $s$ increase the probability of ties among samples drawn from $P\sim Dp(s,\alpha^*)$ and, thus, as $s$ goes to $0$ the DP tends to concentrate on a single value.
As a further consequence, from an a-posteriori point of view, this choice for the prior gives zero probability to the event that a future observation is different from the previously observed data. 
 \citet{Rubin1981} reports the following extreme example. Consider the probability that $X>C$ where $C$ is a value larger than the largest observed value of $X$, i.e., $X_{(n)}$. The standard BB and bootstrap methods estimate such probability to be $0$ with zero variance, which is untenable if $X$ can assume different values from the $n$ previously observed.
 Rubin also remarks that one should expect a probability that $X$ is greater than or equal to $X_{(n)}$ of about $1/(n+1)$.
This shows that a Dirichlet prior with $s \rightarrow 0$ implies definitely a very strong (and not always reasonable) information about $P$, and hence it cannot be considered a noninformative prior.
On the other side, if we choose a DP prior with $s>0$, the inferences provided by this model will be sensitive to the choice of the normalized measure $\alpha^*$. If, for example, we decide to assign a ``tail'' probability of $1/(n+1)$ to $X>X_{(n)}$, in agreement with Rubin's intuition, the inferences will be different if we assume that the tail probability is concentrated on  $X_{(n)}$ or if we assume that it is spread from $X_{(n)}$ to a very large value of $X$.

To answer to the initial question of this section, we propose the imprecise Dirichlet process (IDP). 
 The main characteristic of the IDP is that it does not require
 any choice of the normalized measure $\alpha^*$, it is a prior near-ignorance model  and solves the issues of the BB.
% method to choose  $(s,\alpha^*)$ in case of lack of prior information: a \textit{prior near-Ignorance DP}. 
% This model  does not require any choice of the normalized measure $\alpha^*$.
% IDP has been derived extending \citet{pericchi_walley_91,walley1991}'s parametric prior near-ignorance models to the DP.
Before introducing the IDP, it is worth to explain  what is a prior near-ignorance model with the example of a parametric model \cite[Sec. 5.3.1]{walley1991}.
% So before introducing this model in the DP setting, it is worth to review  what is a prior ignorance model in the parametric setting.
% Prior Ignorance models have been introduced by \citet{pericchi_walley_91,walley1991} in the context of Bayesian parametric.
% These are models that behave a priori as a vacuous models for some basic inferences (e.g., prior mean, prior credible regions)  but it always provides non-vacuous posterior inferences. 
% 
% A Prior Ignorance model simply consist in a set of probability measures
% 
% introduced a class of priors, called ``near-ignorance priors'' see also \citet{walley1991}.
% This is a model that behaves a priori as a vacuous model for some basic inferences (e.g., prior mean, prior credible regions)  but it always provides non-vacuous posterior inferences.  
\begin{Mexample}
Let $A$ be the event that a particular thumbtack lands pin-up at the next toss. Your information is that there
have been $m$ occurrences of pin up in $n$ previous tosses.
 Using a Bernoulli model, the likelihood
function generated by observing m successes in n trials is then proportional
to $\theta^n(1-\theta)^{n-m}$ where $\theta$ is the chance of pin-up.
To complete the model, we need to specify prior beliefs concerning the
unknown chance $\theta$.
We can use a conjugate Beta prior $p(\theta)=Be(\theta;\alpha,\beta)$,
 where $\alpha,\beta>0$ are the prior parameters of the Beta density.
 A-posteriori we have that $p(\theta|m,n)=Be(\theta;\alpha+m,\beta+n-m)$.
 Thus, the prior and posterior probabilities of $A$ are:
 $$
 P(A)=E[\theta]=t,~~~P(A|m,n)=E[\theta|m,n]=\frac{st+m}{s+n},
 $$
 where $s=\alpha+\beta$ is the prior strength and $t=\alpha/(\alpha+\beta)$ the prior mean.
 The problem is how to choose the parameters $s,t$ in case of lack of prior information.
  \citet[Ch. 5]{walley1991} proposes to use a   prior near-ignorance model.
A  near-ignorance prior model for this example is any set of priors which generates vacuous prior probabilities for the event
of interest $A$, i.e.,
$$
\underline{P}(A)=0,~~\overline{P}(A)=1,
$$
where $\underline{P},\overline{P}$ are lower and upper bounds for $P(A)$. These
vacuous probabilities reflect a complete absence of prior information
concerning $A$. For the Beta prior, since $P(A)=E[\theta]=t$,  the class of priors is simply:
 $$
 p(\theta)\in\left\{Be(\theta;st,s(1-t)):~~0<t<1\right\},
 $$
 for some fixed $s>0$, i.e., this is the set of priors obtained by considering all the Beta densities whose mean parameter $t$ is free to span  the interval  $(0,1)$. %This set of priors  has been called ``Imprecise Beta Model'' by \citet{walley1991.
 Posterior inferences from this model are derived by computing lower and upper posterior bounds;
in the case of event $A$ these bounds are:
 $$
\underline{P}(A|m,n)=\dfrac{m}{s+n},~~\overline{P}(A|m,n)=\dfrac{s+m}{s+n},
$$
where the lower is obtained for $t \rightarrow 0$ and the upper for $t \rightarrow 1$. 
% These both approach the observed relative frequency $m/n$ as the sample
% size $n$ increases. Behaviourally, this means that you are initially unwilling to
% bet on or against $A$ at any odds, because of the near-ignorance property,
% but, as observations are made, you become willing to bet at rates approaching
% the observed relative frequency.
% Note that the set of Beta densities satisfy prior ignorance for other inferences on $\theta$, e.g., non-central
% moments, credible intervals etc..
% Finally, observe  that $s$ is the only parameter that has to be selected by the modeller and determine the degree of ignorance (or robustness)
% of the model, i.e., the difference between the upper and lower bound
% $$
% \overline{P}(A|m,n)-\underline{P}(A|m,n)=\dfrac{s}{s+n}.
% $$
We point the reader to \citet{walley1996} for more details about this model and to \citet{Benavoli20121960}
for an extension of near-ignorance to one-parameter exponential families.
\end{Mexample}

\subsection{Imprecise Dirichlet process}
Before introducing the IDP, we give a formal definition of (nonparametric) prior ignorance
for predictive inferences.
Let $f$ be a  real-valued bounded function on $\mathbb{X}$, we call $E[f]=\int f dP$ a predictive inference about $X$; here $P$
is a probability measure on $(\mathbb{X},\mathcal{B}_{\mathbb{X}})$. Let $\mu \in \mathbb{M}$ be
a nonparametric prior on $P$ and  $\mathcal{E}_{\mu}[E(P)]$ the expectation of $E[f]$ w.r.t.\ $\mu$.
\begin{Mdefinition}
\label{def:PI}
A class of nonparametric priors  $\mathcal{T}\subset \mathbb{M}$ is called
a prior ignorance model for  predictive inferences about $X$,  if  for any  real-valued bounded function $f$ on $\mathbb{X}$ it satisfies:
% Let $f$ be a real-valued bounded function defined on $\mathbb{X}$; we say that
% a set of probability measures $\mathcal{G}$ is a prior ignorance model for $E[f]$ if it holds that:
 \begin{equation}
\label{eq:vacdef}
\underline{\mathcal{E}}[E(f)]=\inf\limits_{\mu \in \mathcal{T}} \mathcal{E}_{\mu}[E(f)] =\inf f, ~~ \overline{\mathcal{E}}[E(f)]=\sup\limits_{\mu \in \mathcal{T}} \mathcal{E}_{\mu}[E(f)]=\sup f,
\end{equation}
where  $\underline{\mathcal{E}}[E(f)]$ and $\overline{\mathcal{E}}[E(f)]$ denote respectively the lower and upper bound of $\mathcal{E}_{\mu}[E(P)]$ 
 calculated w.r.t.\ the class $\mathcal{T}$.
\end{Mdefinition}
From (\ref{eq:vacdef}) it can be observed that the range of $\mathcal{E}_{\mu}[E(f)]$  under the class $\mathcal{T}$ is
the same as the original range of $f$. 
In other words, by specifying the class  $\mathcal{T}$, we are not giving any information
on the value of the expectation of $f$. This means that  the class $\mathcal{T}$ behaves as a vacuous model. 
% All predictive inferences about a rv $X$ can be written as the expected value of a function $f(X)$. For example, for the probability $P(X\in A)$ for some $A \subseteq \mathbb{X}$ can be written as $E(f)$, with $f(x) = I_A(x)$. Given a nonparametric prior $\mu$, we can evaluate $E(f)$ by taking its expected value $\mathcal{E}_{\mu}[E(f)]$ w.r.t. $\mu$. A model of prior ignorance about $E(f)$ can only tell us that the expectation of $E(f)$ is included in the original range of $f$, i.e.,
% $$
% \inf{f} \leq \mathcal{E}_{\mu}[E(f)] \leq \sup{f}
% $$
% where we can replace infimum (supremum) with minimum (maximum) if the infimum (supremum) of $f$ is contained in the range of $f$. This would mean that the prior IDP model does not add any information about $E(f)$ and for this reason is a model of prior ignorance about the value of $E(f)$.
% Therefore, we give the following definition of prior ignorance in the nonparametric setting.
% \begin{Mdefinition}[Prior Ignorance.]
% We say that a set $\mathcal{T^*}$ of probability measures is a prior ignorance model for the predictive inference about $X$ denoted as $E(f)$, with $f$ a real-valued bounded function on $\mathbb{X}$, if it verifies 
% \begin{equation}
% \inf_{\mu \in \mathcal{T^*}} \mathcal{E}_{\mu}[E(f)]=\inf f, ~~ \sup_{\mu \in \mathcal{T^*}} \mathcal{E}_{\mu}[E(f)]=\sup f.
% \end{equation}
% \end{Mdefinition}
We are now ready to define the IDP.
\begin{Mdefinition}[IDP.]
We call  prior imprecise DP the following class of DPs:
 \begin{equation}
\label{eq:IDP}
\mathcal{T}=\left\{Dp(s,\alpha^*): ~\alpha^* \in \mathbb{P}\right\}.
\end{equation}
\end{Mdefinition}
The IDP is the class of DPs obtained for a fixed $s>0$ and by letting the normalized measure $\alpha^*$ to vary in the set of all 
probability measures $\mathbb{P}$ on  $(\mathbb{X}, \mathcal{B}_{\mathbb{X}})$.
\begin{Mtheorem}
\label{th:PI}
The IDP is a model of prior ignorance for all predictive inferences about $X$, i.e., for any  real-valued bounded function $f$ on $\mathbb{X}$ it satisfies:
% Let $f$ be a real-valued bounded function defined on $\mathbb{X}$; we say that
% a set of probability measures $\mathcal{G}$ is a prior ignorance model for $E[f]$ if it holds that:
 \begin{equation}
\label{eq:vac1}
\underline{\mathcal{E}}[E(f)]=\inf f, ~~ \overline{\mathcal{E}}[E(f)]=\sup f,
\end{equation}
where  $\underline{\mathcal{E}}[E(f)]$ and $\overline{\mathcal{E}}[E(f)]$ denote respectively the lower and upper bound of ${\mathcal{E}}[E(f)]$ 
defined in (\ref{eq:expf}) calculated w.r.t.\ the class of DPs (\ref{eq:IDP}).

% Let $f$ be a real-valued bounded function defined on $\mathbb{X}$; we say that
% a set of probability measures $\mathcal{G}$ is a prior ignorance model for $E[f]$ if it holds that:
 %\begin{equation}
%\label{eq:vac1}
%\underline{\mathcal{E}}[E(f)]=\inf f, ~~ \overline{\mathcal{E}}[E(f)]=\sup f,
%\end{equation}
%where  $\underline{\mathcal{E}}[E(f)]$ and $\overline{\mathcal{E}}[E(f)]$ denote respectively the lower and upper bound of ${\mathcal{E}}[E(f)]$ 
%defined in (\ref{eq:expf}) calculated w.r.t.\ the class $\mathcal{T}$ of DPs (\ref{eq:IDP}).

% on $\mathbb{X}$, where  
% $$
% \underline{\mathcal{E}}[E(f)]=\inf_{~\alpha^* \in \mathbb{P}} \mathcal{E}[E(f)|G], ~~~~\overline{\mathcal{E}}[E(f)]=\sup_{G \in \mathcal{G}} \mathcal{E}[E(f)|G],
% $$ 
% denote the lower and, respectively, upper  bound of  $\mathcal{E}[E(f)|G]$, which is the expectation of $E(f)$ with respect to some random probability measure $G$ in $\mathcal{G}$.
\end{Mtheorem}
The proofs of this and the next theorems are in the Appendix.  To show that the IDP is a model of prior ignorance, consider for instance the indicator function $f=I_{A}$ for some $A \subseteq \mathbb{X}$. Since $E[I_A]=P(A)$, from (\ref{eq:expf}) we have that $\mathcal{E}[P(A)]= \int I_A d\alpha^*$.
Then if we choose  $\alpha^*=\delta_{x_l}$ with $x_l\notin A$  and, respectively,  $\alpha^*=\delta_{x_u}$
with $x_u\in A$:
% Equation (\ref{eq:vac1}) becomes
 \begin{equation}
\label{eq:vac1ind}
\underline{\mathcal{E}}[P(A)]= \int I_A d\delta_{x_l}=\min I_A=0,~~ \overline{\mathcal{E}}[P(A)]= \int I_A  d\delta_{x_u}=\max I_A= 1,
\end{equation}
where $\underline{\mathcal{E}}[P(A)]$ and $\overline{\mathcal{E}}[P(A)]$ are the lower and upper
bounds for $\mathcal{E}\big[P(A)\big]$. This is a condition of prior ignorance for $P(A)$,
since we are saying that the only information about $P(A)$ is that $0\leq P(A) \leq 1$.
%  Note that in (\ref{eq:vac1ind}) we have replaced the infimum (supremum) of $f$ with minimum (maximum) because the infimum is contained in the range of $f$.
The lower and upper bounds are obtained from the degenerate DPs $Dp(s,\delta_{x_l})$ and $Dp(s,\delta_{x_u})$, which belong to the class (\ref{eq:IDP}).
Note that, although the lower and upper bounds are obtained by degenerate DPs, to obtain these bounds
we are considering all possible $Dp(s,\alpha^*)$ with $\alpha^* \in \mathbb{P}$ (even the ones with continuous probability measures $\alpha^*$).
% An important characteristic of IDP is that it satisfy prior ignorance for all  predictive inferences about $X$.




% From Theorem \ref{th:PI} it can be observed that the range of ${\mathcal{E}}[E(f)]$  under IDP is
% the same as the original range of $f$. This means that  the prior model IDP does not add any information about $E(f)$ and in this sense is a model of prior ignorance about the value of $E(f)$.
% Note that if the infimum (supremum) of $f$ is contained in the range of $f$, we can replace infimum (supremum) with minimum (maximum),
% as for the case $f=I_A$ in (\ref{eq:vac1ind}).
%From Theorem \ref{th:PI} it can be observed that the range of ${\mathcal{E}}[E(f)]$  under IDP is
%the same as the original range of $f$. This means that  the prior model IDP does not add any information about $E(f)$ and in this sense is a model of prior ignorance about the value of $E(f)$.
% which reflects a complete absence of prior information concerning 
% the expectation of $f$. 
% Equation (\ref{eq:vac1}) becomes
%  \begin{equation}
% \label{eq:vacprob}
% \underline{\mathcal{E}}[P(A)]=\min I_{A} = 0, ~~ \overline{\mathcal{E}}[P(A)]=\max I_{A} = 1,
% \end{equation}
% This is equivalent to say  that the only prior information about $P(A)$ is that $0\leq P(A) \leq 1$ for any $A \subseteq \mathbb{X}$.
% The lower and upper bounds are obtained for instance by these degenerate DPs $Dp(s,\delta_{x_l})$ and $Dp(s,\delta_{x_u})$,
% where $x_l$

%  which means that the lower expected probability of the subset $A$ is zero and the upper is one.

% The set of probability measures $Dp(s, G_0)$ with $G_0 \in \{\text{set of all distributions}\}$ is a model of prior ignorance for predictive inferences, since for any real-valued bounded function $f$ of $\mathbb{X}$ it holds that
% \begin{equation}
% \label{eq:priormeansgen}
% \begin{aligned}
% \underline{\mathcal{E}} \big[E(f)\big]&=\inf\limits_{G_0 \in \mathcal{G}}   \int f(x) dG_0(x)=\inf f,\\
% \overline{\mathcal{E}} \big[E(f)\big]&=\sup\limits_{G_0 \in \mathcal{G}}   \int f(x) dG_0(x)=\sup f.
% \end{aligned}
% \end{equation} 
% where the infimum (supremum) is achieved by taking the limit $\epsilon \rightarrow 0$ of the mixture $dG_0 = \epsilon d\tilde{G}_0 + (1-\epsilon) \delta_x$, where $\tilde{G}_0$ is is diffuse,  which, following \citet[Theorem 3.2.7]{ghosh2003bayesian}, tends to an atomic measure centered at $x=\arg\inf f$ (respectively, at $x=\arg\sup f$). Hereafter we will refer to a prior of this form by writing $dG_0 \rightarrow \delta_x$. 
% Furthermore, the posterior of such limiting prior is a DP with base measure 
% $$
% dG_0 = \tfrac{s}{s+n} [\epsilon d\tilde{G}_0 + (1-\epsilon) \delta_x] + \tfrac{1}{s+n}\sum\limits_{i=1}^n \delta_{X_i}
% $$
% which for $\epsilon \rightarrow 0$ tends to a mixture of atomic measures centered on $x$ and on the observations $X_i$. 
\begin{Mtheorem}[Posterior inference.]
\label{th:DPpost}
Let $X_1,\dots,X_n$ be i.i.d.\ samples from $P$ and $P \sim Dp(s,\alpha^*)$. Then for any  real-valued bounded function $f$ on $\mathbb{X}$, 
the lower and upper bounds of ${\mathcal{E}}[E(f)|X_1,\dots,X_n]$ under the IDP model in (\ref{eq:IDP}) are:
% the posterior distribution of $P$ given the observations is 
% For any  real-valued bounded function $f$ on $\mathbb{X}$ it satisfies:
\begin{equation}
\label{eq:postexp}
\begin{aligned}
\underline{\mathcal{E}}\big[E(f)|X_1,\dots,X_n\big]&= \dfrac{s}{s+n}\inf f+\dfrac{n}{s+n}S_n(f),\\
\overline{\mathcal{E}}\big[E(f)|X_1,\dots,X_n\big]&=  \dfrac{s}{s+n}\sup f+\dfrac{n}{s+n}S_n(f),
\end{aligned}
\end{equation} 
where $S_n(f)=\frac{\sum_{i=1}^n f(X_i)}{n}$.
% and $\underline{\mathcal{E}}[E(f)|X_1,\dots,X_n],\overline{\mathcal{E}}[E(f)|X_1,\dots,X_n]$ denote the lower and upper bounds 
% of ${\mathcal{E}}[E(f)|X_1,\dots,X_n]$ w.r.t.\ the class of DPs (\ref{eq:IDP}).
\end{Mtheorem}
A-posteriori the IDP does not satisfy anymore the prior ignorance property (\ref{eq:vac1}). This means that learning from data takes place under the IDP. 
In fact let $S(f)$ be equal to $\lim_{n\rightarrow \infty} S_n(f)$, a-posteriori  for $n\rightarrow \infty$ we have that:
\begin{equation}
\begin{aligned}
\underline{\mathcal{E}}\big[E(f)|X_1,\dots,X_n\big],\overline{\mathcal{E}}\big[E(f)|X_1,\dots,X_n\big] &\rightarrow S(f),
\end{aligned}
\end{equation} 
i.e., the lower and upper bounds of the posterior expectations converge to $S(f)$, which only depends on data. 
In other words, the effect of prior ignorance vanishes asymptotically:
$$
\overline{\mathcal{E}}\big[E(f)|X_1,\dots,X_n\big]-\underline{\mathcal{E}}\big[E(f)|X_1,\dots,X_n\big]=\dfrac{s}{s+n} (\sup f-\inf f)\rightarrow 0, 
$$
for any finite $s$. To define the IDP, the modeler has only to choose $s$. 
This explains the meaning of the adjective \textit{near} in prior near-ignorance,  because the IDP requires by the modeller the elicitation of a parameter. However, this is a simple elicitation problem for a nonparametric prior, since we only have  to choose the value of a positive scalar (there are not infinitely dimensional parameters left in the IDP model). Section \ref{sec:MWW} gives some guidelines for the choice of this parameter.

% Fixed $s$, for any choice of $\alpha^*$ (continuous or discrete), we always have
% $$
% \underline{\mathcal{E}}\big[E(f)|X_1,\dots,X_n\big] \leq {\mathcal{E}}_{\alpha^*}\big[E(f)|X_1,\dots,X_n\big] \leq \overline{\mathcal{E}}\big[E(f)|X_1,\dots,X_n\big],
% $$
% which means that the IDP is robust, in the sense that encompasses all other posterior inferences obtained for any choice of $\alpha^*$.
% 
% 
% The modeller has only to choose $s$ (actually an upper bound for $s$ would be enough) to completely specify
% IDP. This is a simple elicitation problem for a nonparametric prior we have only to choose the value of a positive scalar
% (there are not infinitely dimensional parameters left). Section \ref{} gives some guidelines to the choice of this parameter.
% this model of prior ignorance satisfies the property of learning\footnote{The property of learning is satisfied if at least one of the posterior (upper and lower) estimates differs from the prior after a sufficiently large, but finite, number of data has been observed} for any bounded functions $f$. 
Observe that IDP solves the two main drawbacks of Bayesian Bootstrap. From the a-priori point of view, 
we have shown in (\ref{eq:vac1}) that the IDP is a model of prior ignorance for predictive inferences. Moreover, the prior distributions considered can assign a non-null probability to unobserved values of $X$. Then, considering Rubin's example about the probability that $X$ is greater than or equal to $X_{(n)}$, which is obtained as the expectation of $f=I_{[C,\infty)}$ with $C>X_{(n)}$, from (\ref{eq:postexp}) we have a-posteriori that $\underline{\mathcal{E}}\big[E(f)|data\big]=0$ and $\overline{\mathcal{E}}\big[E(f)|data\big]=\tfrac{s}{s+n}$. The upper expectation is greater than zero and, for $s=1$, it is equal to $1/(1+n)$. This result is obtained without specifying how the probability of $1/(1+n)$ is spread between the values $X>X_{(n)}$, and thus it is insensitive to the model specification of tail probabilities.
Note that the IDP reduces to the imprecise Dirichlet model proposed by  \citet{walley1996}, see also \cite{Bernard2005123,deCooman2009204}), when we limit ourselves to consider a finite measurable partition $B_1,\dots,B_m$ of $\mathbb{X}$. In this case, the set of priors $\{Dp(s,\alpha^*)$, $\alpha^* \in \mathbb{P}\}$, reduces to a set of Dirichlet distributions with parameters $(s\alpha^*(B_1),\dots, s\alpha^*(B_m))$. 


% 
% 
% An issue with the BB model is that \citet{Rubin1981}: (i) it is not a noninformative model; (ii) it assigns zero posterior probability
% to any set that does not include the observations.
% We will discuss these  two points with more details in Section \ref{sec:BB} and we will show in Section \ref{sec:pign} how our model
% overcomes these issues. 
% 
% In Bayesian nonparametrics a way to tackle this problem is to set a Dirichlet process prior on the distribution of $F$. Then the Bayes estimate of $f$ is fully determined by the choice of $G_0$ and $s$, which represent, respectively, our prior guess about the distribution of $X$, and the strength of this prior guess. Therefore, in the absence of prior information about the distribution of $X$, a frequent choice is to take $s\rightarrow 0$. The limiting posterior process obtained for $s \rightarrow 0$ has been introduced under the name of Bayesian Bootstrap (BB) by \citet{Rubin1981}.
% It can be proven that the BB is asymptotically equivalent \citet{Lo1987, Weng1989} to the frequentist bootstrap introduced by  \citet{Efron1979}.
% 
% The BB has been criticized on diverse grounds. 
% From an a-priori point of view, the main criticism is that taking $s \rightarrow 0$ is far from leading to  a non-informative prior.
% \citet{sethuraman1981} have shown that if we consider a sequence of measures $\alpha_r$ such that, $s \rightarrow 0$ and the normalized measure $\alpha_r/s$ converges in the strong sense to the probability measure $G_0$, then the Dirichlet process $F$ with parameter $\alpha_r$, converges, for $r\rightarrow \infty$, to a random degenerated measure centered on $X_0$, with $X_0$ distributed according to $G_0$. This result depends on the fact that small values of $s$ increase the probability of ties among samples drawn from $P$ and, thus, as $s$ goes to $0$ the DP tends to concentrate on a single value. As a further consequence, from an a-posteriori point of view, this choice for the prior gives zero probability to the event that a future observation is different from the previously observed ones. 
%  \citet{Rubin1981} reports the following extreme example. Consider the probability that $X>C$ where $C$ is a value larger than the largest observed value of $X$, i.e., $X_{(n)}$. The standard BB and bootstrap methods estimate such probability to be $0$ with zero variance, which is untenable if $X$ can assume different values from the $n$ previously observed. Rubin also remarks that one should expect a probability that $X$ is greater than or equal to $X_{(n)}$ of about $1/(n+1)$.
% This shows that a Dirichlet prior with $s \rightarrow 0$ implies definitely a very strong (and not always reasonable) information about $F$, and hence it cannot be considered a noninformative prior.
% On the other side, if we choose a DP prior with $s>0$, the inferences provided by this model will be sensitive to the choice of the prior base distribution $G_0$. If, for example, we decide to assign a ``tail'' probability of $1/(n+1)$ to $X>X_{(n)}$, in agreement with Rubin's intuition, the inferences will be different if we assume that the tail probability is concentrated on  $X_{(n)}$ or if we assume that it is spread from $X_{(n)}$ to a very large value of $X$.
% 
% 
% 
% 
% In Bayesian parametric analysis there are two main avenues to represent ignorance. 
% The first assumes that ignorance can be modeled satisfactorily by a so-called noninformative prior, by which it is meant a prior that contains
% no information about the parameter of interest  $\theta$. The problem is how to define the meaning of ``containing no prior information''.
% It is common to define noninformative  a prior that is invariant under certain re-parametrization of the parameter space, see for instance \citet{jeffreys1983}, \citet[Ch. 6]{berger1985} and \citet[Sec. 5.6.2]{bernardo_94}.
% Critics to noninformative priors is that they are not unique, see for instance \citet[Sec. 3.3.3]{berger1985}, and that they do not really model lack of prior information, but only invariance.
% 
% To solve these issues, many authors suggest that lack of prior information should be expressed in terms of a set $\Gamma$ consisting of all prior distributions
% that are compatible with the available prior information.
% Inferences should then be carried out by considering the whole family $\Gamma$.
% This approach is known as \textit{Bayesian robustness} or \textit{Bayesian sensitivity analysis}, see \citet{berger1994overview} and \citet{Ruggeri2000} for a review.
% For instance, prior information  expressed in form of moments, quantiles, symmetry, etc., 
% can be translated into a set of priors sharing these
% features and posterior inferences can be performed  w.r.t.\ this class. 
% A problem with this approach is that it also requires some prior information to be available.
% The natural candidate for $\Gamma$ to represent complete ignorance is the set of all probability measures.
% However, it turns out that the posterior inferences obtained from this set are \textit{vacuous} \citet[Sec.~7.3.7]{walley1991}, i.e., 
%  the posterior set of probability measures coincides with the prior set of probability measures. This means that there is no learning from data. Therefore, the vacuous prior model is not a statistically useful way to model our prior ignorance.
% There is then a compromise to be made. \citet{pericchi_walley_91} suggest, as an alternative, the use of an almost vacuous model which they call ``near-ignorance priors'' see also \citet{walley1991}.
% This is a model that behaves a priori as a vacuous model for some basic inferences (e.g., prior mean, prior credible regions)  but it always provides non-vacuous posterior inferences.  
% \begin{Mexample}
%  Consider the problem of estimating the bias $\theta \in (0,1)$ of a coin from i.i.d.\ observations. Using a Binomial-Beta model the posterior
%  density for $\theta$ is
%  $$
%  p(\theta|n_h,n_t)=Be(\theta;\alpha+n_h,\beta+n_t),
%  $$
%  where $\alpha,\beta>0$ are the prior parameters of the Beta distribution $n_h,n_t$ are the number of heads and tails in the sample.
% The most used noninformative priors in this case are:  Bayes and Laplace's uniform prior  $\alpha=\beta=1$,  \citet{jeffreys1983}' prior $\alpha=\beta=0.5$ and \citet{haldane1945}'s prior   $\alpha,\beta \rightarrow 0$. They are not unique, they lead to different posterior inferences and, they model invariance, indifference nut not prior ignorance.
% \citet{walley1991}'s near-ignorance prior model instead consists on the class of priors
%  $$
%  p(\theta)\in\left\{Be(\theta;\alpha,\beta):~~\alpha,\beta>0,\alpha+\beta=s\right\},
%  $$
%  i.e., the set of priors obtained by considering all the Beta densities whose parameters $\alpha,\beta$ sum up to the positive constant $s$.
%  This is a near-ignorance model in the sense that for  some basic inferences it provides vacuous answer.
%  For instance, since $E[\theta]=\alpha/(\alpha+\beta)$, a-priori we have that:
%  $$
%  \underline{E}[\theta]=0,~~\overline{E}[\theta]=1,
%  $$
%  where $ \underline{E}, \overline{E}$ denote respectively the lower and upper bound for $E[\theta]$ and are obtained
% for $\alpha\rightarrow 0$ and, respectively, $\beta=0$. By saying that $E[\theta]\in (0,1)$,  we are stating a condition of 
% prior ignorance on the value of the mean of $\theta$. 
% \end{Mexample}


% Examples of classes are:
% $$
% \begin{array}{c}
%  \Gamma_M=\left\{\text{distributions with given mean and variance}\right\},\\
%  \Gamma_Q=\left\{\text{distributions with given quantiles}\right\},\\
% \Gamma_S=\left\{\text{symmetric distributions around zero}\right\}.\\
% % \Gamma_U=\left\{\text{symmetric and unimodal distributions around zero}\right\}.\\
% \end{array}
% $$

% 
% To solve these issues, many authors \citet{berger1994orb,huber1973use,sivaganesan1989ranges,deroberts1981bayesian,pericchi_walley_91,walley1991} suggest that lack of prior information should be expressed in terms of a family $\mathcal{M}$  consisting of all prior distributions
% that are compatible with the available prior information.
% Inferences should then be carried out by considering the whole family $\mathcal{M}$.
% This approach is known as \textit{Bayesian robustness} or \textit{Bayesian sensitivity analysis}---examples are $\epsilon$-contamination models \citet{berger1985,huber1964rel},
% restricted  $\epsilon$-contamination models \citet{sivaganesan1989ranges},
% intervals of measures \citet{deroberts1981bayesian,huber1964rel},
% the density ratio class \citet{deroberts1981bayesian,walley1991}, etc..
% 
% In case almost no prior information is available on $W$, $\mathcal{M}$ should
% be as large as possible in order to describe this state of prior ignorance.
% The natural candidate for $\mathcal{M}$ to represent complete ignorance is the set of all distributions.
% However, it turns out that the posterior inferences obtained from this set are \textit{vacuous} \citet[Sec.~7.3.7]{walley1991}, i.e., 
%  the posterior set of distributions  coincides with the prior set of distributions. This means that our prior beliefs do not change with experience, or, in other words, that there is no learning from data. Therefore, the vacuous prior model is not a statistically useful way to model our prior ignorance.
% There is then a compromise to be made. Walley suggests, as an alternative, the use of an almost vacuous model which he calls ``near-ignorance priors'' \citet{pericchi_walley_91,walley1991}.
% This is a model that behaves a priori as a vacuous model for some basic inferences (e.g., prior mean, prior credible regions)  but it always provides non-vacuous posterior inferences.  
% Near-ignorance models allow us to start a statistical analysis with very weak assumptions about the problem of
% interest and, thus, to implement a genuine objective-minded approach to inference.
% Near-ignorance models have been presented in \citet{pericchi_walley_91,walley1991} to address inference problems in which: (i) the observations are discrete and the unknown $W$ is a vector of variables with bounded components (the so called IDM, Imprecise Dirichlet Model); (ii) the observations are continuous and the unknown $W$ is an unbounded but scalar variable (Gaussian or  double exponential family models \citet{pericchi_walley_91}).
% Near-ignorance often naturally leads to invariance. For instance, IDM is known to satisfy permutation invariance and invariance to coarsening/refinement of the categories \citet{walley1991}. When this is not the case, invariance properties can be imposed to near-ignorance priors as suggested in \citet{pericchi_walley_91}.
% 

% 
% 
% The only solution to this problem that has been proposed so far
% is the limiting DP obtained when the prior base measure goes to zero, which has been first proposed by \citet{Ferguson1973} and then introduced under the name of Bayesian Bootstrap (BB) by \citet{Rubin1981}. An issue with the BB model is that: (i) it is not a noninformative model; (ii) it assigns zero posterior probability
% to any set that does not include the observations. We will discuss these  two points with more details in Section \ref{sec:BB}. 
% 
% In case of lack of prior information, a more natural way to model prior ignorance may be to consider the set of all distributions $G_0$, \citet[Ch. 5]{walley1991} and \citet{pericchi1991robust}.
% In other words, we keep $s$ fixed and assume that
% $$
% G_0 \in \Gamma= \{\text{set of all distributions}\},
% $$
% and then compute the lower and upper expectations for all the functions of interest in the statistical analysis.
% 
% A formal definition of prior ignorance is given below. 
% 
% \begin{Mdefinition}[Prior Ignorance.]












%  ̄
% 
% 
% 
% 
% 
% 
% 
% 
% 
% 
% % The notion of a distribution over distributions was motivated by problems in Bayesian density estimation.
% Let $\mathbb{X}$ be a set and $\mathcal{B}_{\mathbb{X}}$ the $\sigma$-algebra of Borel sets of $\mathbb{X}$, a random probability measure $P$ is distributed according to a Dirichlet Process (DP) with base measure $\alpha$ (a $\sigma$-additive non-null measure on $(\mathbb{X},\mathcal{B}_{\mathbb{X}})$),\footnote{The DP can also be defined by considering a finitely additive measure $\alpha$, however, for the scopes of this paper, we limit ourselves to consider only $\sigma$-additive measures.} if for any finite measurable nontrivial partition $B_1,\dots,B_m$ of $\mathbb{X}$ (i.e., the $B_i$ are measurable, disjoint, and $\cup_{i=1}^m B_i=\mathbb{X}$), the vector $(P(B_1),\dots,P(B_m))$ has a Dirichlet distribution with parameters $(\alpha(B_1),\dots,\alpha(B_m))$. It follows that, for $A \in \mathcal{B}_{\mathbb{X}}$, $P(A)\sim Beta(\alpha(A),\alpha(\mathbb{X})-\alpha(A)))$ and, thus, 
% $$
% \begin{array} {rcl}
% \mathcal{E}[P(A)]=\dfrac{\alpha(A)}{\alpha(\mathbb{X})}, &\text{ and } &\mathcal{E}[(P(A)-\mathcal{E}[P(A)])^2]=\dfrac{\alpha(A)(\alpha(\mathbb{X})-\alpha(A))}{(\alpha(\mathbb{X})^2(\alpha(\mathbb{X})+1))}.
% \end{array}
% $$
% In the following, we assume that $\mathbb{X}\subseteq \mathbb{R}$ and denote the Dirichlet process by $Dp(s,G)$, where $s=\alpha(\mathbb{X})$ is the precision parameter and $G(x)=\alpha(-\infty,x]/s$ is the cumulative distribution function corresponding to the base probability measure $\alpha/s$. 
% Let $X_1,\dots,X_n$ be an independent and identically distributed sample from some unknown probability distribution $F$; taking a Dirichlet process prior $Dp(s,G_0)$ for the distribution of $F$, its posterior probability distribution given the observations is  
% $$
% F|X_1,\dots,X_n \sim Dp\left(s+n, \frac{s}{s+n} G_0+ \frac{1}{s+n} \sum_{i=1}^n I_{[X_i,\infty)}\right),
% $$
% where $I_{[X_i,\infty)}$ is the indicator function. This means that the Dirichlet process satisfies a property of conjugacy, in the sense that the posterior for $F$ is again a Dirichlet process with updated base measure.
% Let $F$ be a Dirichlet process $Dp(s,G)$ and $f$ a real-valued measurable function defined on $(\mathbb{X},\mathcal{B}_{\mathbb{X}})$. If $\int|f|dG<\infty$, then the expectation with respect to the Dirichlet process of $E[f]$ is \citet{Ferguson1973}
% $$
% \mathcal{E}\big[E(f)\big]=\mathcal{E}\left[\int f dF\right]=\int f d\mathcal{E}[F] = \int f dG.
% $$
% 
% % 
% 
% 
% 
% and we will show in Section \ref{sec:pign} how our model
% overcomes these issues.
% 
% 
% He also considered hypothesis testing involving quantiles and estimation of 
% $P (X< Y  )$ assigning independent Dirichlet process priors to the distribution functions of $X$ and $Y$. 
% The sign statistic and Mann-Whitney statistic naturally arise in the two cases. 
% \citet{SusarlaRyzin1976} and \citet{Blum1977} extended the results of Ferguson on
% estimation of the distribution function in case of right censored data obtaining a Bayesian
% version of the Kaplan-Meier estimator.
% \citet{dalai1983nonparametric} considered the problem of
% estimating  a measure of dependence for a
% bivariate distribution. The Bayes estimate is computed using a two-dimension Dirichlet prior 
% and Kendall's tau is seen to appear naturally.
% A review of other similar applications of DP can be found in \citet{Phadia2013}.
% The beauty of DP is that most of these results are in closed form and that it
% provides a Bayesian justification of  the classic nonparametric estimators.
% More recently 
% 
% 
% 
% 
% 
% 





% 
% 
% Somewhat surprisingly, despite these initial outcomes,  such a research  has not resulted in the development of  DP-based Bayesian nonparametric procedures for hypothesis testing.
% % such as Bayesian versions of the Wilcoxon rank sum test, Wilcoxon signed test or other standard nonparametric tests. 
% For instance, the most used statistical packages for DP-based Bayesian nonparametric modeling, ``DPpackage'' \citet{Dppackage} and ``Bayesm'' \citet{rossi2010bayesm}, include procedures for density estimation, clustering and regression but they do not include any Bayesian version of the Wilcoxon rank sum, Wilcoxon sign test or other classic nonparametric tests. Also the current research on Bayesian nonparametrics is  mainly focused on density estimation, clustering and regression, as it can be catched from the recent review   on Bayesian nonparametrics by  \citet{muller2013bayesian} and for instance the recent papers \citet{de2013bayesian}, \citet{di2013simple}.
% 
% The aim of the present work is to fill this gap by proposing a general, simple and conservative method for performing Bayesian nonparametric tests based on DPs.
%  Let $\alpha$ be the finite measure on $\mathbb{X}=\mathbb{R}$  that fully characterizes DP. Define $s=\alpha(\mathbb{X})$, so that $G_0=\alpha/s$ is now a probability measure.
% The idea is to  consider the set of DPs obtained by letting
%  $G_0$  span the set
% $$
% \Gamma=\left\{\textit{set of all distributions}\right\},
% $$
% and to perform inferences using this set of DPs.\footnote{With distributions we mean cumulative distribution functions on $\mathbb{R}$. However, note that DP can be defined on a generic space $\mathbb{X}$.}
% % while the prior strength of DP is fixed to $\alpha(\mathbb{X})=s$. 
% Our approach has two main characteristics.
% First, to perform a Bayesian analysis we only need to specify $\alpha(\mathbb{X})=s$, which is a scalar parameter.
% This means that the elicitation of our model is very easy compared to the case in which, in addition to $s$, we also
% have to provide $G_0$, that is, an infinite dimensional parameter. 
% Second,  our approach allows us to compute posterior inferences, for which no closed form expression exists, by a simple Monte Carlo sampling 
% from the Dirichlet distribution. This means that we do not need to use stick breaking or other sampling approaches specific for DP. 
% In fact, given a sequence of observations $X_1,\dots,X_n$ from $P \sim Dp(s,G_0)$ and exploiting the conjugacy property of DP, we know that a-posteriori:
% $$
% \left\{DP(s+n,G_n): G_n=\frac{s}{s+n} G_0+ \frac{1}{s+n} \sum_{i=1}^n I_{[X_i,+\infty)},~~~G_0\in \Gamma\right\}.
% $$
% Since $G_0$ varies in $\Gamma$, our inferences change depending on the choice of $G_0$. However, we can  compute lower and upper bounds 
% of the  inferences by considering the extreme-case distributions in $\Gamma$.
% It results that the \textit{extreme distributions} that achieve these lower and upper bounds are atomic measures  (Dirac's delta), i.e.,  $G_0=I_{[X_0,+\infty)}$ and $dG_0=\delta_{X_0}$. Thus, to compute the lower (upper) bound we have only to choose where to place this atomic measure, i.e., $X_0$.
% This is in general an easy optimization problem because $X_0$ can be interpreted as an extra observation that has to be chosen so as to minimize or maximize
% the expectation of the function of interest in the analysis.
% Moreover, once we have chosen $dG_0=\delta_{X_0}$, $G_n$ becomes a discrete measure and, thus, from the properties of DP it follows that any distribution $P$ sampled from $DP(s+n,G_n)$ has the  form $P=w_{0}I_{[X_0,+\infty)}  + \sum_{i=1}^{n} w_{i} I_{[X_i,+\infty)}$, 
% where the weights $(w_{0},w_{1},\dots,w_{n})$ are Dirichlet distributed with parameters $(s,1,\dots,1)$ (we do not need stick-breaking).
% 
% Any other inference that we compute by selecting a particular $G_0$ is encompassed by these bounds.
% This approach is particularly effective in hypothesis testing. In fact, given the discrete nature of the decision problem,
% there are cases in which the optimal posterior decision (the one that minimizes the loss) does not change at the varying of $G_0$ in $\Gamma$.
% % In this case, any choice of $G_0$
% % Therefore, we do not need to make $G_0$ explicit.
% Conversely, when this is not the case, it means that in $\Gamma$ there are distributions that lead to
% opposite decisions. 
% In these cases, if some prior information was available, it could  be used 
% to restrict the set $\Gamma$.
% % The available prior information is often not  enough to  reduce $\Gamma$ to a singleton (a single  distribution precisely specified in all details) without introducing some arbitrariness in the elicitation. Arbitrariness that may affect our posterior decisions.
% % More realistically we can expect that the available prior information allows us to restrict $\Gamma$ to a smaller set.
% For instance, information on $G_0$ expressed in form of moments, quantiles, symmetry, etc., 
% can be translated into a set of priors sharing these
% features and the hypothesis test can be performed  again w.r.t.\ this restricted class. Examples of classes are:
% $$
% \begin{array}{c}
%  \Gamma_M=\left\{\text{distributions with given mean and variance}\right\},\\
%  \Gamma_Q=\left\{\text{distributions with given quantiles}\right\},\\
% \Gamma_S=\left\{\text{symmetric distributions around zero}\right\}.\\
% % \Gamma_U=\left\{\text{symmetric and unimodal distributions around zero}\right\}.\\
% \end{array}
% $$
% $ \Gamma$ and these other classes of priors have extensively been studied in Bayesian robustness.
% There is thus a tight connection about the present work and Bayesian robustness. We point the reader to \citet{berger1994overview} and \citet{Ruggeri2000}  for a review of Bayesian robustness and the properties of these classes.
% % Frechet application of some of these classes
% % Although 
% In this paper, we restrict our attention to the largest class $\Gamma$ because it encompasses all the previous cases and can also be employed  in case of lack of prior information -- as it is often the case
% in hypothesis testing.
% It is worth to mention that we can give a subjective interpretation of $\Gamma$ in this setting.
% % In this setting, we can give a subjective intepretation to $\Gamma$.
% % Predictive prior inferences on $X$ are vacuous under the DP when $G_0\in \Gamma$.
% In fact, since for any real-valued function $f$ of $X$, the prior expectation of $f$  w.r.t.\ $P \sim Dp(s,G_0)$ is equal to expectation of $f$  w.r.t.\ $G_0$ and since $G_0$ can be any distribution, it follows
% that the lower expectation of $f$  w.r.t.\ $G_0 \in \Gamma$ coincides with $\inf f$ and the upper with $\sup f$.
% Thus, by choosing $P \sim Dp(s,G_0)$ with $G_0\in \Gamma$, we are assessing our prior
% ignorance about $X$, i.e., for any $f$ we only know that the expectation is in $[\inf f,\sup f]$, which is 
% a vacuous information (we do not know anything). $P \sim Dp(s,G_0)$ with $G_0\in \Gamma$ is therefore a model of prior ignorance for predictive inferences on $X$.
% The use of prior ignorance models in the parametric setting has been proposed by \citet[Ch. 5]{walley1991}, \citet{pericchi1991robust}, \citet{walley1996} and \citet{benavoli2012a} as an alternative to the use of noninformative priors. The present work extends the concepts of prior ignorance to nonparametric models.
% % It should be mentioned that for categorical sets $\mathbb{X}$, our model $P \sim Dp(s,G_0)$ with $G_0\in \Gamma$  reduces to Imprecise Dirichlet model  \citet{walley1996}.
% In the parametric case, the lack of prior information is commonly addressed by considering noninformative (invariant) priors, which are often improper, see for instance \citet{bernardo2009bayesian}.
% In the DP setting, the only noniformative priors that have been proposed so far
% is the limiting DP obtained for $\alpha(\mathbb{X}) \rightarrow 0$, which has been introduced under the name of Bayesian Bootstrap (BB) by \citet{Rubin1981}.
% An issue with the BB model is that \citet{Rubin1981}: (i) it is not a noninformative model; (ii) it assigns zero posterior probability
% to any set that does not include the observations.
% We will discuss these  two points with more details in Section \ref{sec:BB} and we will show in Section \ref{sec:pign} how our model
% overcomes these issues.
% 
% To state the practical effectiveness of our  model, $P \sim Dp(s,G_0)$ with $G_0\in \Gamma$, we apply it
% to implement a  Bayesian version of the Mann-Whitney-Wilcoxon test, which is the main focus of the paper.
% In particular, after briefly reviewing  the  Mann-Whitney-Wilcoxon test, we will discuss how to implement a
%  Bayesian version of the the Mann-Whitney-Wilcoxon test using a DP prior. Then  we will
% discuss the properties and the derivation of  the Bayesian Mann-Whitney-Wilcoxon test for the model
% $P \sim Dp(s,G_0)$ with $G_0\in \Gamma$. We will also show that our test is asymptotically consistent, while this is not always the case
% for the  Mann-Whitney-Wilcoxon test.
% Furthermore,  we will give guidelines for choosing the only unknown parameter of our model, i.e., $\alpha(\mathbb{X})=s$.
% Finally, by means of numerical simulations on artificial as well as real data, we will compare  our test with both the frequentist Mann-Whitney-Wilcoxon test and  the test obtained by taking the limit $s \rightarrow 0$. The simulations will show that our test is more robust, in the sense that it is able to isolate
% instances in which the frequentist test and  the $s \rightarrow 0$ test are almost guessing at random.
% \par


\section{An application to hypothesis testing}
\label{sec:MWW}
Hypothesis testing is an important application of nonparametric statistics.   Recently there has been an increasing interest in the development of Bayesian nonparametric procedures for hypothesis testing.
For instance Bayesian nonparametric approaches to the two-sample problem have been proposed using Dirichlet process mixture models or (coupling-optional) Polya trees priors by \citet{borgwardt2009bayesian,holmes2009two,ma2011coupling,RePEc:eee:csdana}.
% In particular DPM or Polya tree are used to build a Bayes factor for testing whether two (or $k$ in case of \citet{RePEc:eee:csdana}) samples
% arise from the same distribution.
Although   prior near-ignorance  may also be useful in these models and in the two-sample problem, we do not follow this avenue in the present study.
Our focus is instead the hypothesis test $P(X\leq Y)\lesseqgtr P(X>Y)$ (equivalently  $P(X\leq Y)\lesseqgtr 0.5$), given independent random samples of sizes $n_1$ and $n_2$ from two populations. This problem arises, for example, if one wishes to compare the response $X$ of a population with respect to the response $Y$ of a different population in order to establish whether the two populations perform equally well or one population has generally ``better'' responses than the other.

The nonparametric test traditionally applied in such situations is the Mann-Whitney-Wilcoxon (MWW) rank-sum test.
The null hypothesis of the MWW rank-sum test is that the two populations are equal, that is, they come from the same distribution
$F_X(x) = F_Y (x)$. Let $X^{n_1} =\{X_1,\dots,X_{n_1}\}$ and $Y^{n_2} =\{Y_1,\dots,Y_{n_2}\}$ be two sequences of observations 
from the two populations. The MWW test is based on the fact that, if the two populations have the same distribution, the distribution of the linear rank statistic
\begin{equation}
 \label{eq:Umwm}
U=\sum\limits_{i=1}^{n_1}\sum\limits_{j=1}^{n_2} I_{[X_i,\infty)}(Y_j),
\end{equation} 
can be computed by considering all the possible random arrangements of the observations in $X^{n_1}$ and $Y^{n_2}$.
At the increase of $n_1$ and $n_2$, this distribution converges to a Normal distribution with mean and variance given by
\begin{equation}
 \label{eq:Umeanvar}
E\left[\frac{U}{n_1n_2}\right]=\dfrac{1}{2}, ~~Var\left[\frac{U}{n_1n_2}\right]=\dfrac{n_1+n_2}{12n_1n_2}.
\end{equation} 
It is worth to stress that $F_X(x) = F_Y (x)$ implies  $P(X\leq Y)=0.5$ (i.e., it is not true that $Y$ is better than $X$)
but not vice versa. Thus, the MWW test cannot be used in general as a test for $P(X\leq Y)$.
This limitation of the test is due to the choice of the  U statistic as estimator and the need of any frequentist method to specify the distribution of the  statistic under the null hypothesis. The null hypothesis $F_X(x) = F_Y (x)$ is thus selected  to be able to compute
the distribution of the statistic, although, in practice, one is interested in a much weaker hypothesis to test $P(X\leq Y)$ (see \citet{Fay2010} for a detailed discussion).
To overcome this issue of the MWW test, it is often common to assume a location-shift model, which states that the two populations can only differ in locations:
$F_Y(y) = F_X(y - \Delta)$. 
% The location-shift parameter $\Delta$ is also known as treatment effect: if population $X$ is taken as a control population and population $Y$ is the treatment population, then $\Delta$ is the expected effect due to the treatment. 
The goal is then to test the hypothesis that there is no treatment effect $\Delta = 0$ ($P(X\leq Y)=0.5$) versus the alternative $\Delta > 0$
($P(X\leq Y)>0.5$) or $\Delta < 0$ ($P(X\leq Y)<0.5$). 
% This assumption seems more realistic (although much stricter) than the previous one, and thus will be adopted in the numerical applications for fair comparisons of the frequentist and Bayesian tests. 
Under this assumption, the MWW test can be interpreted as a \citet{hodges1963} estimator. On the other side, the Bayesian approach provides the posterior distribution of $P(X\leq Y)$, which can be used to compute the probability of any hypothesis of interest. Therefore, we are not limited in the choice of the null hypothesis.
% 
% 
% However the MWW test has some drawback when applied for testing $P(X\leq Y)=0.5$. 
% Firstly, the most general and realistic alternative hypothesis to $F_X(x) = F_Y (x)$ is that $F_X(x) \neq F_Y (x)$, so that all possible distributions $F_X$ and $F_Y$ are included. Instead, when comparing the null hypothesis $F_X(x) = F_Y (x)$ against, for instance, the alternative hypothesis $P(X\leq Y)>0.5$, we should consider only the  set of all distributions $F_X$ and $F_Y$ except those that do not verify neither the null nor the alternative hypotheses (i.e., verify $P(X<Y)\leq 0.5$ but are not equal) (see \citet{Fay2010} for a detailed discussion). This limitation is due to the fact that the frequentist approach needs to specify the distribution of the U statistic under the null hypothesis, an thus the strong null hypothesis $F_X(x) = F_Y (x)$ is required, although, in practice, one is interested in a much weaker hypothesis. On the other side, the Bayesian approach provides the posterior distribution of $P(X\leq Y)$ based on which the probability of any hypothesis of interest can be computed. Therefore, we are not limited in the choice of the null hypothesis.

Another well-known drawback of the MWW test, which is common to all frequentist tests, is the controversial meaning of the p-value. The Bayesian approach to decision making allows basing the decisions on the value of the expected loss, whose practical meaning is much more intuitive. 
For example, the hypothesis test:
$$
\stackrel{\underbrace{P(X\leq Y)\leq  P(X > Y)}}{P(X\leq Y)\leq  0.5}
 \textit{ vs. } 
\stackrel{\underbrace{P(X\leq Y)>  P(X > Y)}}{P(X\leq Y) >  0.5}
$$
can be performed in a Bayesian way in two steps.
First we define a loss function
\begin{equation}
\label{eq:loss}
 L(P,a) =\left\{\begin{array}{ll}
  K_0 I_{\{P(X\leq Y)> 0.5\}} &\text{ if } a = 0,\\
  K_1 I_{\{P(X\leq Y)\leq  0.5\}} &\text{ if } a = 1.\\
 \end{array}\right.
\end{equation} 
The first row gives the loss we incur by taking the action $a=0$ (i.e., declaring that $P(X\leq Y)\leq  0.5$) when actually $P(X\leq Y) >  0.5$,
while the second row gives the loss we incur by taking the action $a=1$ (i.e., declaring that $P(X\leq Y)> 0.5$) when actually $P(X\leq Y)\leq 0.5$.
Second, we compute the expected value of this loss:
\begin{equation}
\label{eq:exploss}
\mathcal{E}\left[ L(P,a)\right]
 =\left\{\begin{array}{ll}
  K_0 \mathcal{P}\left[P(X\leq Y)> 0.5\right]&\text{ if } a=0,\\
  K_1 \mathcal{P}\left[P(X\leq Y)\leq   0.5\right]&\text{ if } a=1,\\
 \end{array}\right.
\end{equation}
where we have used the calligraphic letter $\mathcal{P}$ to denote the probability w.r.t. the DPs $F_X$ and $F_Y$.
Thus, we choose $a=1$ if
\begin{equation}
\label{eq:exploss_dec}
  K_1 \mathcal{P}\left[P(X\leq Y)\leq   0.5\right] \leq   K_0 \mathcal{P}\left[P(X\leq Y)>  0.5\right] \Rightarrow \mathcal{P}\left[P(X\leq Y)>  0.5\right]> \dfrac{K_1}{K_1+K_0},
\end{equation}
%Since $ \mathcal{P}\left[P(X\leq Y)\leq   \frac{1}{2}\right]=1-\mathcal{P}\left[P(X\leq Y)>  \frac{1}{2}\right]$, this  is equivalent to:
%\begin{equation}
%\label{eq:exploss_dec}
%\mathcal{P}\left[P(X\leq Y)>  \frac{1}{2}\right]> \dfrac{K_1}{K_1+K_0}.
%\end{equation}
or $a=0$ otherwise.
When the above inequality is satisfied, we can declare that $P(X\leq Y)>0.5$ with probability $\frac{K_1}{K_1+K_0}=1-\gamma$  (e.g., $1-\gamma=0.95$).
%Note that if $F_X$ and $F_Y$ are symmetric distributions, $P(X\leq Y)=0.5$ implies that they have the same median. Thus, the test $P(X\leq Y)>0.5$ is a valid test for the median under the location-shift model. On the other side, for different asymmetric distributions $F_X, F_Y$, it may be that $P(X\leq Y)=0.5$ but $F_X$and $F_Y$ have different medians. Thus, $P(X\leq Y)>1/2$ is not a test for the difference in median, but a test on whether ``Y is greater than X''.

Finally, based on the imprecise DP model developed in this paper, we can perform a Bayesian nonparametric test that, besides overcoming the limitation of the frequentist one, is based on extremely weak prior assumptions, and easy to elicit, since it requires only to choose the strength $s$ of the DP instead of its infinite-dimensional parameter $\alpha$. When using the IDP set of priors, we consider for $F_X$ and $F_Y$ all the possible DP priors with strength lower than or equal to $s$ (since all inferences obtained for $s'<s$ are encompassed by those obtained for $s$). All these priors give a posterior probability $\mathcal{P}\left[P(X\leq Y)> 0.5\right]$ included between the lower and upper bounds $\underline{\mathcal{P}}\left[P(X\leq Y)> 0.5\right]$ and $\overline{\mathcal{P}}\left[P(X\leq Y)>  0.5\right]$.
%We can employ the results of Theorem \ref{th:1} and Corollary \ref{co:1} to perform a hypothesis test
%based on the model of prior ignorance developed in this section.
%We thus select $a=1/2$ and, according to the decision (\ref{eq:exploss_dec}) for some $K_0,K_1$ , we verify if
Thus, according to the decision rule in (\ref{eq:exploss_dec}) for some $\gamma = \tfrac{K_0}{K_0+K_1}$, we verify if
$$
\underline{\mathcal{P}}\big[P(X\leq Y)> 0.5|X^{n_1},Y^{n_2}\big]>1-\gamma, ~~\overline{\mathcal{P}}\big[P(X\leq Y)> 0.5|X^{n_1},Y^{n_2}\big]>1-\gamma,
$$
and then:
\begin{enumerate}
 \item if both the inequalities are satisfied we can declare that $P(X\leq Y)$ is greater than $0.5$ with probability larger than $1-\gamma$;
 \item if only one of the inequality is satisfied (which has necessarily to be the one for the upper), we are in an indeterminate situation, i.e., we cannot decide;
\item if both are not satisfied, we can declare that the probability that $P(X\leq Y)$ is greater than $0.5$ is lower than the desired probability of $1-\gamma$.
\end{enumerate}
When our model of prior ignorance returns an indeterminate decision, it means that the evidence from the observations is not enough to declare either that the probability of the hypothesis being true is larger or smaller than the desired value $1-\gamma$; more measurements are necessary to take a decision.

The three cases are respectively depicted  in Figure  \ref{fig:1}.
Observe that the posterior  distributions  of $P(X\leq Y)$, from which the lower and upper probabilities above are derived, give us much more information than the simple result of the hypothesis test. In particular we can derive the posterior lower and upper probabilities of $P(X\leq Y)<0.5$.
For instance, from Figure \ref{fig:1} (d) we can see that $Y$ is not greater than $X$ at $95\%$, but it is evident that $X$ is  greater than $Y$ at $95\%$.
While in the case shown in Figure \ref{fig:1} (b), we can say neither that  $Y$ is  greater than $X$ nor that $X$ is  greater than $Y$. (To distinguish these two cases
it would be more appropriate to perform a ``two-sided'' hypothesis test.)

\begin{figure}[h]
        \centering
        
         \begin{tabular}{cc}
                     \includegraphics[width=0.35\textwidth]{Test_determinateaaa} &  \includegraphics[width=0.35\textwidth]{Test_indeterminateaaa}\\
                      (a) ``$Y$ is greater than $X$'' at $95\%$ &  (b) ``Indeterminate'' at $95\%$\\
                      \includegraphics[width=0.35\textwidth]{Test_determinateno0aaa} & \includegraphics[width=0.35\textwidth]{Test_determinateno1aaa}\\
                      (c) ``$Y$ is not greater than $X$'' at $95\%$ &  (d) ``$Y$ is not greater than $X$'' at $95\%$\\
         \end{tabular}
% 
%         \begin{subfigure}[b]{0.35\textwidth}
%                 \centering
%                 \includegraphics[width=\textwidth]{Test_determinate}
%                 \caption{T: ``$Y$ is greater than $X$'' at $95\%$}
%                 \label{fig:det}
%         \end{subfigure}%
%         ~ %add desired spacing between images, e. g. ~, \quad, \qquad etc.
%           %(or a blank line to force the subfigure onto a new line)
%         \begin{subfigure}[b]{0.35\textwidth}
%                 \centering
%                 \includegraphics[width=\textwidth]{Test_indeterminate}
%                 \caption{T: ``Indeterminate'' at $95\%$}
%                 \label{fig:ind}
%         \end{subfigure}
%         ~ %add desired spacing between images, e. g. ~, \quad, \qquad etc.
%           %(or a blank line to force the subfigure onto a new line)
%         \begin{subfigure}[b]{0.35\textwidth}
%                 \centering
%                 \includegraphics[width=\textwidth]{Test_determinateno0}
%                \caption{T: ``$Y$ is not greater than $X$'' at $95\%$}
%                 \label{fig:detn1}
%         \end{subfigure}
%         \begin{subfigure}[b]{0.35\textwidth}
%                 \centering
%                 \includegraphics[width=\textwidth]{Test_determinateno1}
%                \caption{T:``$Y$ is not greater than $X$'' at $95\%$}
%                 \label{fig:detn0}
%         \end{subfigure}
        \caption{Four possible results of the hypothesis test. The dark and light filled areas correspond respectively to the lower and upper probabilities of 
the event  $P(X\leq Y)>0.5$. The numerical values of these lower and upper probabilities are also reported in the figures. }\label{fig:1}
\end{figure}

In the next section we prove that the IDP is a model of prior ignorance for $P(X\leq Y)$ and derive the posterior results which are necessary to evaluate $\mathcal{P}\left[P(X\leq Y)>  0.5\right]$ and perform the test. Note that, for the moment, we assume that there are not ties between $X$ and $Y$; we will discuss how to account for the presence of  ties in  Section \ref{sec:ties}.

%Ferguson \citet{Ferguson1973} has shown that the U statistic used in the MWW test appears naturally when computing the posterior expectation of $P(X<Y)$ given a DP prior; in particular, the posterior expectation of  $P(X<Y)$ converges to the U statistic as $s \rightarrow 0$. 

\subsection{IDP model for $P(X\leq Y)$}
\label{sec:MWWDP}
Let the samples $X^{n_1}$ and $Y^{n_2}$ be drawn, respectively, from $F_X$ and $F_Y$. As prior for $(F_X,F_Y)$, we assume that $F_X \sim Dp(s_1,G_1)$ and $F_Y \sim Dp(s_2,G_2)$. Hereafter, to simplify the presentation, we take $s_1 = s_2 = s$.
$F_X$ and $F_Y$ are assumed to be independent. 
The probability $P(X\leq Y)$ is given by $P(X\leq Y) =E[I_{[X,\infty)}(Y)] = \int F_X(y) dF_Y(y)$.
As derived by  \citet{Ferguson1973}, by the properties of the Dirichlet process, it follows that a-priori
$\mathcal{E}[P(X\leq Y)] = \int G_1(y) dG_2(y)$. It can be shown that the set of priors $\mathcal{T}$ in (\ref{eq:IDP}) satisfies the condition of prior ignorance also for $P(X\leq Y)$. In fact, since
$\mathcal{E}[P(X\leq Y)]=\int G_1(y)dG_2(y)$, if $G_i \in \mathbb{P}$, we have that
$$
\underline{\mathcal{E}}[P(X\leq Y)]=0, ~~\overline{\mathcal{E}}[P(X\leq Y)]=1,
$$
where the lower (upper) bound is obtained for  $dG_1 = \delta_{X_0}$ and $dG_2 = \delta_{Y_0}$ with $X_0>Y_0$ ($X_0<Y_0$).
Thus, prior ignorance about the mean of $P(X\leq Y)$ is satisfied.
Furthermore, let us consider the probability  of $P(X\leq Y)<0.5$ with respect to the Dirichlet process.  A-priori, for $dG_1 = \delta_{X_0}$ and $dG_2 = \delta_{Y_0}$ we have that
$$
\begin{array} {lll}
\text{if } X_0<Y_0, &\text{ then } \mathcal{P}[P(X\leq Y)=1]=1 &\text{ and thus } \underline{\mathcal{P}}[P(X\leq Y)\leq 0.5]=0\\
\text{if } X_0>Y_0, &\text{ then } \mathcal{P}[P(X\leq Y)=0]=1 &\text{ and thus } \overline{\mathcal{P}}[P(X\leq Y)\leq 0.5]=1. 
\end{array}
$$
A similar reasoning leads to $\underline{\mathcal{P}}[P(X\leq Y)>0.5]=0$, $\overline{\mathcal{P}}[P(X\leq Y)>0.5]=1$, 
thus, prior ignorance about the hypothesis $P(X\leq Y)>0.5$ is also satisfied.
Given the two sequences of measurements, a-posteriori one has:
$$
\mathcal{E}[P(X\leq Y)|X^{n_1},Y^{n_2}] = \int G_{n_1}(y) dG_{n_2}(y),
$$
with $G_{n_i}=\frac{s}{s+n_i} G_i+\frac{1}{s+n_i}\sum_{j=1}^{n_i} I_{[Z_j,\infty)}$, 
where $Z_j=X_j$ for $i=1$ and $Z_j=Y_j$ for $i=2$. It follows that:
 \begin{equation}
\label{eq:expmwmDp}
\begin{array}{rcl}
\mathcal{E}[P(X\leq Y)|X^{n_1},Y^{n_2}] &=& \frac{s}{s+n_1}\frac{s}{s+n_2}\int G_1(y) dG_2(y)+\frac{n_1}{s+n_1}\frac{s}{s+n_2}\frac{1}{n_1}\sum\limits_{j=1}^{n_1} (1-G_2(X_j^-))\\
 &+& \frac{s}{s+n_1}\frac{n_2}{s+n_2}\frac{1}{n_2}\sum\limits_{j=1}^{n_2} G_1(Y_j)+\frac{n_1}{s+n_1}\frac{n_2}{s+n_2}\frac{U}{n_1n_2},\\
\end{array}
\end{equation}
where $1-G_2(X^-)=\int I_{[X,\infty)}dG_2$. 
%The main issue of this approach is how  to select the prior distributions $G_1,G_2$ of the two Dirichlet processes for $X$ and $Y$ in case of lack of prior information. We have seen that in the Bayesian bootstrap (BB) ($s \rightarrow 0$) it holds that $G_{n_i}\rightarrow G^*_{n_i}=\tfrac{1}{n_i}\sum_{j=1}^{n_i} \delta_{Z_j}$, and thus the effect of the prior is eliminated from the posterior. Then the use of $G^*_{n_i}$ as a non-subjective (non-informative) posterior, is considered as one of the few concrete non-informative analysis in Bayesian nonparametrics \citet{ghosh2003bayesian}. However, as already discussed in Section \ref{sec:BB}, it is not completely correct to consider the BB prior as non-informative.
%\subsection{Modelling prior ignorance in the rank sum test}
%\label{sec:Modelling prior ignorance in the rank sum test}
%A way to model prior ignorance in the DP-based rank sum test is by following the approach discussed in Section \ref{sec:pign}.
%In fact, 
Then, the lower and upper posterior bounds of the posterior expectations of $P(X\leq Y)$ given the set of priors $\mathcal{T}$ are:
 \begin{equation}
\label{eq:ranktest}
\begin{array}{rcl}
\underline{\mathcal{E}}[P(X\leq Y)|X^{n_1},Y^{n_2}] &=&\frac{U}{(s+n_1)(s+n_2)},\\
\overline{\mathcal{E}}[P(X\leq Y)|X^{n_1},Y^{n_2}] &=& \frac{U}{(s+n_1)(s+n_2)}+\frac{s(s+n_1+n_2)}{(s+n_1)(s+n_2)}, \\
\end{array}
\end{equation}
obtained in correspondence of the extreme distributions $dG_1 \rightarrow \delta_{X_0}$, $dG_2 \rightarrow \delta_{Y_0}$,
with $X_0>\max(\{Y_0,\dots,Y_{n_1}\})$, $Y_0<\min(\{X_0,\dots,X_{n_2}\})$ (lower) and $X_0<\min(\{Y_0,\dots,Y_{n_1}\})$,  \\ $Y_0>\max(\{X_0,\dots,X_{n_2}\})$ (upper).
% This result provides lower and upper bounds for the expectation of $P(X\leq Y)$ derived by  \citet{Ferguson1973}.
% In this paper we complete the work of \citet{Ferguson1973} by develop
% For any other choice of $G_1,G_2$, it results that
% $$
% \underline{\mathcal{E}}[P(X\leq Y)|X^{n_1},Y^{n_2}]\leq {\mathcal{E}}[P(X\leq Y)|X^{n_1},Y^{n_2}] \leq \overline{\mathcal{E}}[P(X\leq Y)|X^{n_1},Y^{n_2}].
% $$
The posterior probability distribution of $P(X\leq Y)$ w.r.t. the Dirichlet process, which is used to perform the Bayesian test of the difference between the two populations, is, in general, computed numerically (Monte Carlo sampling) by using the stick-breaking construction of the Dirichlet process. We will show in the remaining part of this section that, in correspondence of the discrete priors that give the upper and lower bounds of the posterior distributions of $P(X\leq Y)$, a more efficient procedure can be devised. Consider the limiting posteriors that give the posterior lower and upper expectations in (\ref{eq:ranktest}):
\begin{equation}
\label{eq:basePost}
G_{n_i}(y)=\frac{s}{s+n_i} I_{[Z_{0},\infty)}+\frac{1}{s+n_i}\sum_{j=1}^{n_i} I_{[Z_{j},\infty)},
\end{equation}
where the lower is obtain with $Z_{0}=X_0>\max(\{Y_0,\dots,Y_{n_1}\})$ for $i=1$ and $Z_{0}=Y_0<\min(\{X_0,\dots,X_{n_2}\})$ for $i=2$, and the upper with $Z_{0}=X_0>\max(\{Y_0,\dots,Y_{n_1}\})$ for $i=1$, and $Z_{0}=Y_0<\min(\{X_0,\dots,X_{n_2}\})$ for $i=2$. 

% From the property (d) in Section \ref{sec:DP}, we can then derive the following result. 
\begin{Mlemma}\label{ml:1}
A sample $F_{n_i}$ from the Dirichlet process $Dp(s,G_{n_i})$ with base probability distribution $G_{n_i}$ as that defined in (\ref{eq:basePost}) is given by:
\begin{equation}
\label{eq:sample}
F_{n_i}=w_{i0} I_{[Z_{0},\infty)} + \sum\limits_{j=1}^{n_i} w_{ij} I_{[Z_{j},\infty)},
\end{equation}
where $(w_{i0},w_{i1},\dots,w_{in_i})\sim Dir(s,\stackrel{n_i}{\overbrace{1,\dots,1}})$.
\end{Mlemma}
Since $G_{n_i}$ is a discrete measure, Lemma \ref{ml:1} states that any distribution $F_{n_i}$ sampled from $DP(s+n_i,G_{n_i})$ has the  form (\ref{eq:sample}). Since the probability density functions relative to $F_{n_i}$, i.e., $w_{i0} \delta_{Z_{0}} + \sum\limits_{j=1}^{n_i} w_{ij} \delta_{Z_{j}}$, has a discrete support, we do not need stick-breaking. In fact, once we have selected the support $Z_0,Z_1,\dots,Z_{n_i}$, samples $F_{n_i}$ can simply be obtained by  sampling the weights from the Dirichlet distribution.
Using this fact, we can  derive lower and upper bounds for  ${\mathcal{P}}\big[P(X\leq Y)> c|X^{n_1},Y^{n_2}\big]$
as follows.


\begin{Mtheorem}
\label{th:1}
For any $c\in[0,1]$, it holds that 
\begin{equation}
\label{eq:lowmeandist}
\begin{array}{rcl}
&~& \underline{\mathcal{P}}\big[P(X\leq Y)> c|X^{n_1},Y^{n_2}\big]=P\left[g(w_{1\cdot},w_{2\cdot},X^{n_1}, Y^{n_2})>c\right],\\
\end{array}
\end{equation} 
with $$
g(w_{1\cdot},w_{2\cdot},X^{n_1}, Y^{n_2})=\sum\limits_{j=1}^{n_1}\sum\limits_{k=1}^{n_2} w_{1j}w_{2k} I_{(X_j,\infty)}(Y_k)
$$
% \begin{equation}
% \label{eq:lowmeandist}
% \begin{array}{rcl}
% &~& \underline{\mathcal{F}}\big[P(X\leq Y)\leq|X^{n_1},Y^{n_2}\big]:=\underline{\mathcal{P}}\big[P(X\leq Y)\leq |X^{n_1},Y^{n_2}\big]\vspace{2mm}\\
% &= &\displaystyle{\int \left[w_{10}I_{(X_0,\infty)}(y)+\sum_{j=1}^{n_1} w_{1j} I_{(X_j,\infty)}(y)\right]\left[w_{20}\delta_{Y_0}(y)+\sum_{j=1}^{n_2} w_{2j} \delta_{Y_j}(y)\right]dy}\\
% &=& \sum\limits_{j=1}^{n_1}\sum\limits_{k=1}^{n_2} w_{1j}w_{2k} I_{(X_j,\infty)}(Y_k),\\
% \end{array}
% \end{equation} 
where  $(w_{i0},w_{i1},\dots,w_{in_i})\sim Dir(s,\stackrel{n_i}{\overbrace{1,\dots,1}})$ for $i=1,2$ and $P$ is computed w.r.t.\ these Dirichlet distributions.
The mean and variance of $g(w_{1\cdot},w_{2\cdot},X^{n_1}, Y^{n_2})$ are:
\begin{equation}
\label{eq:lowmeanvar}
 \mu=E_W[W]^TAE_V[V], ~~~\sigma^2= trace[A^TE_W[WW^T]AE_V[VV^T]]-\mu^2,
\end{equation} 
where the expectations $E_W,E_V$ are taken w.r.t.\ the Dirichlet distributions, with  $W=[w_{11},\dots,w_{1n_1}]^T$, $V=[w_{21},\dots,w_{2n_2}]^T$, $E[WW^T]$ and $E[VV^T]$ are $n_i\times n_i$ square-matrix of elements $e_{jk} = (s+n_i)^{-1}(s+n_i+1)^{-1}(1+I_{\{j\}}(k))$ ($i=1$ and $2$, respectively), and $A$ is an $n_1\times n_2$ matrix with elements $a_{jk}=I_{(X_j,\infty)}(Y_k)$.
\end{Mtheorem}

\begin{Mcorollary}
\label{co:1}
For any $c\in[0,1]$, it holds that 
\begin{equation}
\label{eq:upmeandist}
\begin{array}{rcl}
&~& \overline{\mathcal{P}}\big[P(X\leq Y)> c|X^{n_1},Y^{n_2}\big]=P\left[g(w_{1\cdot},w_{2\cdot},X^{n_1}, Y^{n_2})>c\right],\\
\end{array}
\end{equation} 
with
$$
g(w_{1\cdot},w_{2\cdot},X^{n_1}, Y^{n_2})=w_{10}w_{20}+w_{10}\sum\limits_{j=1}^{n_2} w_{2j} +w_{20}\sum\limits_{j=1}^{n_1} w_{1j} +\sum\limits_{j=1}^{n_1}\sum\limits_{k=1}^{n_2} w_{1j}w_{2k} I_{(X_j,\infty)}(Y_k),
$$
where  $(w_{i0},w_{i1},\dots,w_{in_i})\sim Dir(s,\stackrel{n_i}{\overbrace{1,\dots,1}})$ for $i=1,2$.
Consider the augmented vectors $W=[w_{10},w_{11},\dots,w_{1n_1}]^T$, $V=[w_{20},w_{21},\dots,w_{2n_2}]^T$, and the matrix $A$ with elements $a_{jk} = I_{(X_{j-1},\infty)}(Y_{k-1})$ for all $j,k\neq{1}$ and $a_{jk} = 1$ if $j=1$ or $k=1$. The mean and variance of $g(w_{1\cdot},w_{2\cdot},X^{n_1}, Y^{n_2})$ can be computed using the same formulas as in (\ref{eq:lowmeanvar}), where, this time, $E[WW^T]$ and $E[VV^T]$ are $(n_i+1)\times (n_i+1)$ square-matrices ($i=1$ and $2$, respectively) of elements $e_{jk} = (s+n_i)^{-1}(s+n_i+1)^{-1}\tilde{e}_{jk}$ with $\tilde{e}_{jk}=(1+I_{\{j\}}(k))$ for all $j,k\neq{1}$ and $\tilde{e}_{jk} = s(1+s\delta_{jk})$ if $j=1$ or $k=1$.
\end{Mcorollary}
To perform the hypothesis test, we select $c=1/2$ and, according to the decision rule (\ref{eq:exploss_dec}) for some $K_0,~K_1$, we check if
$$
\underline{\mathcal{P}}\big[P(X\leq Y)> \tfrac{1}{2}|X^{n_1},Y^{n_2}\big]>1-\gamma, ~~\overline{\mathcal{P}}\big[P(X\leq Y)> \tfrac{1}{2}|X^{n_1},Y^{n_2}\big]>1-\gamma,
$$
where $\gamma=\tfrac{K_0}{K_0+K_1} \in (0,1)$ (e.g., $1-\gamma=0.95$).
% 
% \begin{figure}[htb]
%  \centering 
% \includegraphics[height=4.5cm]{Test_determinate}
%  \includegraphics[height=4.5cm]{Test_indeterminate}
% \includegraphics[height=4.5cm]{Test_determinateno0}
% \includegraphics[height=4.5cm]{Test_determinateno1}
%     \caption{Correcteness as a function of the difference of the medians $\Delta$ for the case $n_1=n_2=10$}
% \label{fig:1}
% \end{figure}



%\section{Asymptotic consistency}
%% 
%% 
%% Contrarily to the traditional MWW test, which assumes the location-shift model, the DP-based test allows comparing a treatment population with a control by testing, for example, if $P(X\leq Y)>1/2$, without strong assumption about the distribution of $X$ and $Y$. 
%% The DP rank sum test can be also used as a test for the median of the two populations under the assumption that the distribution of $X$ and $Y$ are symmetric, since in this case the condition $P(X\leq Y)=1/2$ implies the equality of the the medians. This is a weaker assumption than the location-shift model assumption required by the MWW rank sum test (for example, it does not require that the distributions $F_1$ and $F_2$ have the same variance). If the distributions $F_1$ and $F_2$ are not symmetric, the hypothesis $P(X\leq Y)>1/2$ can still be tested, but its acceptance does not implies a difference in median. Consider, for example, the following distributions:
%% 
%% \begin{equation}
%% \begin{array}{cc}
%% dF_1 = \begin{cases} 2 & -1 \leq x \leq 0 \\
%% \frac{1}{2} & ~~~0 < x \leq \frac{1}{4} \end{cases} 
%% & 
%% dF_2 = \begin{cases} 2 & -\frac{1}{4} \leq x \leq 0 \\
%% \frac{1}{2} & ~~~0 < x \leq 1 \end{cases} 
%% \end{array}
%% \end{equation}
%% The median of both distributions is in $0$ although the probability $P(X\leq Y)=\int F_1 dF_2 = 11/16$ is larger than $1/2$.  However, under the assumptions of the location-shift model the DP rank sum test is a valid test for the hypothesis $\Delta>0$ against $\Delta \leq 0$ even when the distributions $F_1$ and $F_2$ are not symmetric.
%
%
%From the expression of the lower and upper means in (\ref{eq:ranktest}), it can be verified that for $n_1,n_2\rightarrow \infty$:
%$$
%\underline{\mathcal{E}}[P(X\leq Y)|X^{n_1},Y^{n_2}],\overline{\mathcal{E}}[P(X\leq Y)|X^{n_1},Y^{n_2}]\rightarrow {\mathcal{E}}[P(X\leq Y)|X^{n_1},Y^{n_2}]   = \frac{U}{n_1n_2}.
%$$
%The imprecision (degree of robustness) goes to zero for  $n_1,n_2\rightarrow \infty$ and the expectation ${\mathcal{E}}[P(X\leq Y)|X^{n_1},Y^{n_2}]$
%converges to the Mann-Whitney statistic.
%It is also interesting to check the consistency of the DP rank sum test presented in the previous section.
%% if, under the null hypothesis (same distribution) and at the increase of the number of observations  $n_1,n_2$, the Wilcoxon rank sum test and the model of prior ignorance presented in the previous section give the same result in the hypothesis test.
%To answer this question,  we first prove the following result.
%
%\begin{Mtheorem}
%\label{th:2}
 %Assume that $n_1=n_2=n$, the mean  and the variance (\ref{eq:lowmeanvar}) of  the lower  distribution
%for $n\rightarrow \infty$ converge to $E[U_{ij}]=P(X\leq Y)$ and, respectively, to 
%$$
%\frac{1}{n}Cov[U_{ij}, U_{i,k\neq j}]+\frac{1}{n}Cov[U_{ij},U_{k\neq i,j}],
%$$
%where $U_{rt}=I_{(X_r,+\infty)}(Y_t)$.
%% \frac{1}{n}Cov[U_{ij}, U_{i,k\neq j}]\\
%% \frac{1}{n}Cov[U_{ij},U_{k\neq i,j}],
%% seCov[U_{ij},U_{k\neq i,j}]e \citet[Example 5, Appendix A.1]{Lehmann} for the derivations of the mean and variance of $\frac{U}{n^2}$.
%% Furthermore, the lower distribution becomes asymptotically normal.
%\end{Mtheorem}
%\begin{Proof}
%From Theorem \ref{th:1}, the mean of the lower distribution is
%$$
%\mu_l=E_W[W]^TAE_V[V]=\frac{U}{(s+n)^2},
%$$
%and thus,  for large $n$, it converges to $U/n^2$. % and, thus, to $E[U_{ij}]$, see \citet[Appendix A.5]{Lehmann}
%The convergence of $U/n^2$ to $E[U_{ij}]$ follows by  the fact that $U$ can be replaced by a sum of independent
%random variables, which is asymptotically equivalent to $U$ and to which the law of large numbers can be applied, 
%see  \citet[Appendix A.5]{Lehmann} for more details.
%%  that is the sample mean of $P(X\leq Y)$. % which, in turn, converges to $1/2$. 
%% It remains to prove that  the variances of the lower and upper distributions asymptotically converge to the variance of $U/n^2$.
%For the variance (\ref{eq:lowmeanvar}), first note that we can rewrite
%$$
%E_W[WW^T]=E_V[VV^T]=(D+J_n)\frac{1}{(s+n)(s+n+1)},
%$$
%where  $D$ is the diagonal matrix of ones (identity matrix) and $J_n$ is the $n \times n$ matrix of ones.
%Thus, we have that
%$$
%A^TE_W[WW^T]AE_V[VV^T]=A^T(D+J_n)A(D+J_n)\frac{1}{(s+n)^2(s+n+1)^2},
%$$
%and, for large $n$,
%$$
%\begin{array}{l}
%\dfrac{trace(A^T(D+J_n)A(D+J_n))}{(s+n)^2(s+n+1)^2}\vspace{2mm}\\
%\rightarrow \dfrac{trace(A^TA) +trace(A^TAJ_n)+trace(A^TJ_nA)+trace(A^TJ_nAJ_n)}{n^2(n+1)^2}.
%\end{array}$$
%The above sum has four terms at the numerator:
%$$
%trace(A^TA)=\sum_{i,j} a_{ij}^2=\sum_{i,j} I^2_{(X_i,\infty)}(Y_j),
%$$
%$$
%trace(A^TAJ_n)=\sum_{i,j} a_{ij} \sum_{k} a_{ik}=\sum_{i,j} I^2_{(X_i,\infty)}(Y_j)+\sum_{i,j\neq k} I_{(X_i,\infty)}(Y_j)I_{(X_i,\infty)}(Y_k),
%$$
%$$
%trace(A^TAJ_n)=\sum_{i,j} a_{ij} \sum_{k} a_{kj}=\sum_{i,j} I^2_{(X_i,\infty)}(Y_j)+\sum_{i\neq k,j} I_{(X_i,\infty)}(Y_j)I_{(X_k,\infty)}(Y_j),
%$$
%and
%$$
%trace(A^TJ_nAJ_n)=trace(\mathbbm{1}^TA^T\mathbbm{1}\mathbbm{1}^TA\mathbbm{1})=trace(\mathbbm{1}^TA\mathbbm{1}\mathbbm{1}^TA\mathbbm{1})=U^2,
%$$
%where $\mathbbm{1}$ is the unit vector.
%% Following \citet[Example 5, Appendix A.1]{Lehmann}, let us introduce 
%% $$
%% p_1=P(X\leq Y), ~~p_2=P(X\leq Y,X\leq Y'),~~p_3=P(X\leq Y,X'\leq Y),
%% $$ 
%Then, we have that
%$$
%\begin{array}{l}
%\sigma_l^2=
%\dfrac{3\sum_{i,j} I^2_{(X_i,\infty)}(Y_j)-3n^2\mu_l^2 }{n^2(n+1)^2}
%+\dfrac{\sum_{i,j\neq k} I_{(X_i,\infty)}(Y_j)I_{(X_i,\infty)}(Y_k)-n^2(n-1)\mu_l^2}{n^2(n+1)^2}\\
%+\dfrac{\sum_{i\neq k,j} I_{(X_i,\infty)}(Y_j)I_{(X_k,\infty)}(Y_j)-n^2(n-1)\mu_l^2}{n^2(n+1)^2}
%+\dfrac{3n^2+2n^2(n-1)+n^4}{n^2(n+1)^2}\mu_l^2-\mu_l^2.
%\end{array}
%$$
%Note that, 
%$$
%\left(\frac{3n^2+2n^2(n-1)+n^4}{n^2(n+1)^2}-1\right)\mu_l^2=0
%$$
%and, since the first term in $\sigma_l^2$ goes to zero as $1/n^2$, for large $n$,
%$$
%\begin{array}{rcl}
%\sigma_l^2&\rightarrow &
%% \frac{3\sum_{i,j} I^2_{(X_i,\infty)}(Y_j)-3n^2\mu_l^2 }{n^2(n+1)^2}\\
%\dfrac{\sum_{i,j\neq k} I_{(X_i,\infty)}(Y_j)I_{(X_i,\infty)}(Y_k)-n^2(n-1)\mu_l^2}{n^2(n+1)^2}\\
%&+&\dfrac{\sum_{i\neq k,j} I_{(X_i,\infty)}(Y_j)I_{(X_k,\infty)}(Y_j)-n^2(n-1)\mu_l^2}{n^2(n+1)^2},
%% +\frac{3n^2+2n^2(n-1)+n^4}{n^2(n+1)^2}\mu_l^2-\mu_l^2.
%\end{array}
%$$
%Finally, for large $n$, it can be shown  \citet[Th. 9]{Lehmann} that the right hand side of the above equations tends to
%$\frac{1}{n}Cov[U_{ij}, U_{i,k\neq j}] +\frac{1}{n}Cov[U_{ij},U_{k\neq i,j}]$.
%% $$
%% \begin{array}{rcl}
%% % \sigma_l^2\rightarrow 
%% % \frac{3\sum_{i,j} I^2_{(X_i,\infty)}(Y_j)-3n^2\mu_l^2 }{n^2(n+1)^2}\\
%% \frac{1}{n}Cov[U_{ij}, U_{i,k\neq j}] +\frac{1}{n}Cov[U_{ij},U_{k\neq i,j}].
%% % +\frac{3n^2+2n^2(n-1)+n^4}{n^2(n+1)^2}\mu_l^2-\mu_l^2.
%% \end{array}
%% $$
%\end{Proof}
%The above proof can easily be generalized to the upper distribution (the terms due to $w_{10}$ and $w_{20}$ vanish asymptotically) and to the case $n_1/n_2\neq 1$ (following the same procedure as in \citet[Th. 9]{Lehmann}).
%Theorem \ref{th:2} proves that the mean and variance of the lower and upper distribution of the DP rank sum test converge asymptotically
%to the  mean and variance of the U statistic.
%It is know that the distribution of the U statistic becomes asymptotically normal.
%In \citet{Huskova93}, it is proven that this is true also for the (lower and upper) distribution of the DP rank sum test,
%which  is also asymptotically normal with the same mean and variance of the $U$ statistic.
%% Therefore, we have convergence 
%
%% Assume that the distributions of $X$ and $Y$ are the same, then for $n_1,n_2\rightarrow \infty$ we at the mean and variance of the null hypothesis of the  Wilcoxon rank sum test  are (\ref{eq:Umeanvar}).
%
%\begin{Mcorollary}
 %\label{co:1}
%Assume that the distributions of $X$ and $Y$ are the same (null hypothesis of the  Wilcoxon rank sum test), 
%then for $n_1=n_2=n\rightarrow \infty$ 
%% Considering the mean $\mu$ and variance $\sigma^2$ of the lower distribution of $P(X\leq Y)$, if $X$ and $Y$ have the same distribution, for large $n$
 %it results that
%\begin{eqnarray}
%\label{eq:lowmeanvarres}
 %\mu_l&=&E_W[W]^TAE_V[V]\rightarrow  \dfrac{1}{2}, \\
%\label{eq:lowmeanvarres1}
%\sigma_l^2&=& trace[A^TE_W[WW^T]AE_V[VV^T]]-\mu_l^2\rightarrow \dfrac{1}{6n}.
%\end{eqnarray}
%and the lower distribution converges to a Normal with mean (\ref{eq:lowmeanvarres}) and variance (\ref{eq:lowmeanvarres1}).
%\end{Mcorollary}
%The asymptotic expression of the mean and variance follows directly from Theorem \ref{th:2} by noticing that in this case $E[U_{ij}]=1/2$, 
%$Cov[U_{ij}, U_{i,k\neq j}]=1/12$ and $Cov[U_{ij},U_{k\neq i,j}]=1/12$, see for instance \citet[Sec. 2.3]{Lehmann}.
%
%% Assume that the distributions of $X$ and $Y$ are the same, then for $n_1,n_2\rightarrow \infty$ we know that the distribution of the null hypothesis of  the Wilcoxon rank sum test  converge to a Normal distribution with mean and variance given in (\ref{eq:Umeanvar}).
%% Since (\ref{eq:lowmeandist}) and (\ref{eq:upmeandist}) also converge to a Normal distribution under the null hypothesis, we must check that the mean and variance
%% of (\ref{eq:lowmeandist}) and (\ref{eq:upmeandist}) converge to those in (\ref{eq:Umeanvar}). For simplicity, we will consider only the case $n_1=n_2=n$.
%
%% Before to state this result, we prove the following lemma.
%% \begin{Mlemma}
%%  Consider Theorem \ref{th:1} and assume that (i) $n_1=n_2=n$ the matrix $A$ is upper triangular, then from (\ref{eq:lowmeanvar}) we can derive that
%% \begin{equation}
%% \label{eq:lowmeanvarres}
%%  \mu=E_W[W]^TAE_V[V]=\dfrac{1}{2}, ~~~\sigma^2= trace[A^TE_W[WW^T]AE_V[VV^T]]-\mu^2=\dfrac{n_1+n_2}{12n_1n_2}.
%% \end{equation} 
%% \end{Mlemma}
%% \begin{Proof}
%%  Note that $E_W[W]$ and $E_V[V]$ are, respectively, $n_1\times 1$ and $n_2\times 1$ vectors equal to
%% $$
%% E_W[W]=\left[1,1,\dots,1\right]^T\frac{1}{s+n_1}, ~~E_V[V]=\left[1,1,\dots,1\right]^T\frac{1}{s+n_2}.
%% $$
%% Then, it follows that
%% $$
%% ~
%% $$
%% \end{Proof}
%% \begin{Mtheorem}
%% \label{th:2}
%% Considering the mean $\mu$ and variance $\sigma^2$ of the lower distribution of $P(X\leq Y)$, if $X$ and $Y$ have the same distribution, for large $n$ it results that
%% \begin{eqnarray}
%% \label{eq:lowmeanvarres}
%%  \mu&=&E_W[W]^TAE_V[V]\approx \dfrac{1}{2}, \\
%% \label{eq:lowmeanvarres1}
%% \sigma^2&=& trace[A^TE_W[WW^T]AE_V[VV^T]]-\mu^2\approx \dfrac{1}{6n}.
%% \end{eqnarray}
%% \end{Mtheorem}
%%  \begin{Proof}
%% Consider Theorem \ref{th:1}, without loss of generality we can assume that the observations in the vectors $X^{n}$ and $Y^{n}$ are ordered.
%% Then,  $X_r$ is the  $r$th-order static of the vector $X^{n}$ and converges to the $r$th-quantile for $n\rightarrow \infty$, e.g., for $r=n/2$ it converges to the median.
%% This means that, for $n\rightarrow \infty$, the probability that the observation $Y_q$ is larger than $X_r$ will converge to $1$ if $q>r$, to $0$ if $q<r$ and to $0.5$ if $q=r$. Thus, for $n\rightarrow \infty$ the matrix $A$ in Theorem \ref{th:1} converges to an upper triangular matrix except for the diagonal elements, whose elements will be distributed as $n$ Bernoulli distributions with probability $0.5$. 
%% Since $E_W[W]$ and $E_V[V]$ are $n\times 1$ vectors equal to
%% $$
%% E_W[W]=E_V[V]=\left[1,1,\dots,1\right]^T\frac{1}{s+n},
%% $$
%% then, when $A$ is upper triangular with Bernoulli-distributed diagonal elements, for $n\rightarrow \infty $ it follows that
%% $$
%%  \mu=E_W[W]^TAE_V[V]=\dfrac{n(n-1)+2d}{2(s+n)^2}\rightarrow \dfrac{1}{2}.
%% $$
%% where d is a Binomial random variable with parameters $(n,0.5)$ which, for $n$ large converge to the value $n/2$. 
%% For the variance, write
%% $$
%% A = D + T^+ ~~~~ E_W[WW^T] = E_V[VV^T] = k(I+J_n)
%% $$
%% where $D$ is a diagonal matrix with the Bernoulli random variables $d_1,\dots,d_n$ on the diagonal, $T^+$ is a upper triangular matrix with elements $T_{jk}=1$ if $k>j$ and $T_{jk}=0$ if $k\leq j$, $I$ is the identity matrix, $J_n$ is a matrix of ones and $k$ is the coefficient $1/(s+n)(s+n+1)$.
%% The argument of the trace  is
%% $$
%% k^2(D + T^-)(I + J_n)(D + T^+)(I + J_n) = k^2(D + T^- + DJ_n + T^-J_n)(D + T^+ + DJ_n + T^+J_n) = 
%% $$
%% $$
%% DD + D T^+ + D~DJ_n + D~T^+J_n + T^-D + T^- T^+ + T^- ~DJ_n + T^- ~T^+J_n + DJ_n~D + 
%% $$
%% $$
%% DJ_n~T^+ + DJ_n~DJ_n + DJ_n~T^+J_n + T^-J_n~D + T^-J_n~T^+ + T^-J_n~DJ_n + T^-J_n~T^+J_n
%% $$
%% where $T^-$ is the transpose of $T^+$, $DJ_n$, $T^-J_n$ and $T^+J_n$ are matrices with elements, respectively equal to $d_j$, $j-1$ and $n-j$ at all positions $jk$. The trace is
%% $$
%% trace(D T^+)=trace(T^- D)=0 
%% $$
%% $$
%% trace(DD) = trace(D~DJ_n) = trace(DJ_n~D) = \sum_1^n d_i^2 \rightarrow E[\sum_1^n d_i^2] = n/2
%% $$
%% $$
%% trace(T^- ~DJ_n + D~T^+J_n) = trace(DJ_n~T^+ + T^-J_n~D ) = (n-1) \sum_1^n d_i \rightarrow n(n-1)/2
%% $$
%% $$
%% trace(T^- T^+ ) = \sum_0^{n-1} i = n(n-1)/2 
%% $$
%% $$
%% trace(T^- ~T^+J_n ) = trace(T^-J_n~T^+ ) = \sum_0^{n-1} i^2 = n(n-1)(2n-1)/6 
%% $$
%% $$
%% trace(DJ_n~DJ_n) = (\sum_1^n d_i)^2 \rightarrow n^2/4
%% $$
%% $$
%% trace(DJ_n~T^+J_n ) = trace(T^-J_n~DJ_n ) = (\sum_1^n d_i)(\sum_0^{n-1} i) \rightarrow n^2(n-1)/4  
%% $$
%% $$
%% trace(T^-J_n~T^+J_n) = (\sum_0^{n-1} i)^2 = n^2(n-1)^2/4 
%% $$
%% For $n$ large the trace tends to 
%% $$
%% \frac{\frac{7n^2}{4} + \frac{n(n-1)(2n-1)}{3} + \frac{n^2(n-1)}{2} + \frac{n^2(n-1)^2}{4}}{(s+n)^2(s+n+1)^2}
%% $$
%% 
%% %$$
%% %\left(\dfrac{n^2(n+1)^2}{4}+\dfrac{n(n+1)(2n+1)}{12}+\dfrac{n^2(n+1)}{2}+\dfrac{3n(n+1)}{4}\right)\dfrac{1}{(s+n)^2(s+n+1)^2}.
%% %$$
%% 
%% Hence, by subtracting $\mu^2=\frac{n^4}{4(s+n)^4}$, for large $n$ one gets:
%% 
%% $$
%% \sigma^2 \approx \frac{4 n s^2 + 2 n^2 ( 4 s + 3 s^2 ) + 4 n^3 ( 1 + 3 s + 2 s^2) + n^4 (3 + 10 s) + 2 n^5 }{12 (n + s)^4 (1 + n + s)^2} \approx \frac{1}{6n}.
%% $$
%% 
%% \end{Proof}
%This result can again  be generalized to the upper distribution and to the case $n_1\neq n_2$.
%Since the variance goes to zero, Corollary \ref{co:1} proves that, under the null hypothesis (same distribution), the 
%DP rank sum test and the Wilcoxon rank sum test are asymptotically consistent.
%Furthermore, since the lower and upper distributions converge to the true normal distribution, the DP rank sum test is asymptotically calibrated
%as the Wilcoxon rank sum test.
%
%When the null hypothesis is not true,  i.e., $X,Y$ have different distributions, it is known that the Wilcoxon rank sum test is 
 %asymptotically consistent only in the case $P(X \leq Y)\neq 1/2$.
%While, when the distributions are different but $P(X \leq Y)=1/2$, the Wilcoxon rank sum test is not able to detect  the difference in distribution (e.g.,
%two Normal distributions with equal mean but different variance).
%Conversely, from Theorem \ref{th:2}, it follows that the DP rank sum test is always consistent for the test ``Y greater than X''.
%% This means that they will asymptotically provide the same answers under the null hypothesis and that the DP rank sum test is calibrated.
%To understand the differences between the two tests, in case the null hypothesis (same distribution) is not true, consider the following cases:
%$$
%\begin{array}{rcl}
%X_a&=&\{  1,    2 ,    3,     4,     5,    16,    17,    18,    19,    20\},\\
%Y_a&=&\{  6,     7,     8,     9,    10,    11,    12,   13,    14,    15\},
%\end{array}
%$$
%and
%$$
%\begin{array}{rcl}
%X_b&=&\{ 1,     3,     5,     7,     9,    12,    14,    16,    18,    20\},\\
%Y_b&=&\{  2,     4,     6,     8,    10,    11,    13,    15,    17,    19\}.
%\end{array}
%$$
%In both cases the $U$ statistic is $50$ but, as depicted in Figure \ref{fig:2}, in case (a) the observations $Y$
%are in the middle of the $X$, while in case (b) the observations $X$ and $Y$ alternate.
%For the frequentist test, there is not difference between the two cases (since the $U$ statistic is the same), but this is not the case for the test based on the Dirichlet process.
%Figure \ref{fig:3} shows the lower and upper distributions in the two cases.
%
%\begin{figure}[h]
        %\centering
        %\begin{subfigure}[b]{0.35\textwidth}
                %\centering
                %\includegraphics[width=\textwidth]{multimod}
                %\caption{}
                %\label{fig:mm}
        %\end{subfigure}%
        %~ %add desired spacing between images, e. g. ~, \quad, \qquad etc.
          %%(or a blank line to force the subfigure onto a new line)
        %\begin{subfigure}[b]{0.35\textwidth}
                %\centering
                %\includegraphics[width=\textwidth]{unimod}
                %\caption{}
                %\label{fig:uni}
        %\end{subfigure}
        %\caption{ }\label{fig:2}
%\end{figure}
%
%
%\begin{figure}[h]
        %\centering
        %\begin{subfigure}[b]{0.35\textwidth}
                %\centering
                %\includegraphics[width=\textwidth]{multimod_lu}
                %\caption{}
                %\label{fig:mm}
        %\end{subfigure}%
        %~ %add desired spacing between images, e. g. ~, \quad, \qquad etc.
          %%(or a blank line to force the subfigure onto a new line)
        %\begin{subfigure}[b]{0.35\textwidth}
                %\centering
                %\includegraphics[width=\textwidth]{unimod_lu}
                %\caption{}
                %\label{fig:uni}
        %\end{subfigure}
        %\caption{ }\label{fig:3}
%\end{figure}
%
%Note that, in  case (a) the variance of the distributions is larger than in case (b) (the lower and upper means are equal in the two cases because the $U$ statistic has the same value). This is in agreement with the different distribution of the $U$ statistic when data are generated by two distributions with different support (that would generate data of the type of case (a) in Figure \ref{fig:2}) or by the same distribution (case (b) in figure \ref{fig:2}). Consider for example a case (a) where $n$ samples $X_1,\dots,X_n$ are generated from a mixture of two uniform distributions on $[0, 5]$ and $[15,20]$ each with probability $1/2$ and $n$ samples $Y_1,\dots,Y_n$ are generated from a  uniform distribution on $[5, 15]$ and a case (b) where both samples $X_1,\dots,X_n$ and $Y_1,\dots,Y_n$ are generated from a uniform distribution on $[0, 20]$. In both cases $P(X\leq Y)=1/2$, however, the asymptotic variance  of the $U$ statistic 
%converges to $1/6n$ in case (b) (same distribution), while in case (a), since
%$Cov[U_{ij}, U_{i,k\neq j}]=1/4$ and $Cov[U_{ij},U_{k\neq i,j}]=0$, it converges to $1/4n$ (as seen for DP estimate in figure \ref{fig:3},  the variance is larger in case (a)).
%The distribution of the $U$ statistic in the two cases is shown in Figure \ref{fig:4} for $n_1=n_2=100$.
%A consequence of that is that the Wilcoxon rank sum test is neither consistent (to detect that the two distributions are different) nor  calibrated in this case. In fact, since the true variance is larger than the variance of the null hypothesis, the accuracy of the test will be less than $(1-\gamma)$.
%
%\begin{figure}[h]
		%\centering
		%\includegraphics[width=7cm]{SameUdiffDist}
    %\caption{Distribution of $U/n^2$ in cases (a) and (b) for $n=100$.}
		%\label{fig:4}
%\end{figure}
%
%% When the null hypothesis is not true, the  Wilcoxon rank sum test is still based on the null hypothesis, while this is not the case for the Dirichlet process based test.
%% This difference can be understood by considering the following example.
%% Assume that $n_1=n_2=n=2\ell$ for $\ell=1,2,\dots$ and the following two cases
%% $$
%% c_1: ~~x_{(1)}<y_{(1)}<x_{(2)}<y_{(2)}<\dots<x_{(n)}<y_{(n)},
%% $$
%% and
%% $$
%% c_2: ~~x_{(1)}<x_{(2)}<\dots<x_{(n/2)}<y_{(1)}<y_{(2)}<\dots<y_{(n)}<x_{(n/2+1)}<x_{(n/2_2)}<\dots<x_{(n)}.
%% $$
%% Note that, in both cases, for $n\rightarrow \infty$
%% $$
%% \frac{U}{n_1n_2} \rightarrow\frac{1}{2}
%% $$
%% Thus, one has that
%% $$
%% \begin{array}{rcl}
%% Var[P(X\leq Y)]&=&\sum\limits_{j=1}^{n_1}\sum\limits_{k=1}^{n_2}\dfrac{2}{(s+n_1)(s+n_1+1)}\dfrac{2}{(s+n_2)(s+n_2+1)} I_{[X_i,+\infty)}(Y_k)-E[P(X\leq Y)]^2
%% %&+&
%% \end{array}
%% $$

\subsection{Choice of the prior strength $s$}
\label{sec:s}
The value of $s$ determines how quickly lower and upper posterior expectations converge at the increase
of the number of observations.
A way to select a  value of $s$ is by imposing that
the degree of robustness (indeterminacy) $\overline{\mathcal{E}}[P(X\leq Y)|X^{n_1},Y^{n_2}]-\underline{\mathcal{E}}[P(X\leq Y)|X^{n_1},Y^{n_2}]$
is reduced to a fraction of its prior value ($\overline{\mathcal{E}}[P(X\leq Y)]-\underline{\mathcal{E}}[P(X\leq Y)]=1$)
after one observation $(X_1,Y_1)$. % with $X_1<Y_1$.
Imposing a degree of imprecision close to $1$ after the first observation increases the probability of an indeterminate outcome of the test, whereas, a value close to $0$ makes the test less reliable (in fact the limiting value of $0$ corresponds to the BB which will be shown in  Section \ref{sec:sim} to be less reliable than the IDP). Then, the intermediate value of $1/2$ is a frequent choice in prior-ignorance modeling  \citep{pericchi_walley_91,walley1996}. Although this is a subjective way to choose the degree of conservativeness (indeterminacy), 
we will show in Section \ref{sec:sim} that it represents a reasonable trade-off between the reliability and indeterminacy of the decision.
From (\ref{eq:ranktest})  for $n_1=n_2=1$, it follows that
$$
\begin{array}{l}
\overline{\mathcal{E}}[P(X\leq Y)|X_1,Y_1]-\underline{\mathcal{E}}[P(X\leq Y)|X_1,Y_1]=\frac{s^2+2s}{(s+1)^2}. \\
\end{array}
$$
Thus,  by imposing that,
$$
\frac{s^2+2s}{(s+1)^2}=\frac{1}{2},
$$
we obtain $s= \sqrt{2}-1$. Observe that the lower and upper probabilities produced by a value of $s$ are always contained in the probability intervals produced by the larger value of $s$. Then, whenever we are undecided for $s_1$ we are also for $s_2 > s_1$. Nonetheless, for
large $n$ the distance between the upper and lower probabilities goes to 0, then also the indeterminateness  goes to zero.
% In Section \ref{sec:sim}, we will show that by simulations that the value $s= \sqrt{2}-1$ is a good trade-off between robustness and indeterminacy.



% Observe that, for $s=\sqrt{2}-1$ and $X_1<Y_1$, $n_1=n_2=1$,  the lower
% and upper posterior probabilities of the event  $P(X\leq Y)>0.5$ are:
% $$
% \underline{\mathcal{E}}[I_{P(X\leq Y)>0.5}|X_1,Y_1] = 0.499,~~\overline{\mathcal{E}}[I_{P(X\leq Y)>0.5}|X_1,Y_1] =1.
% $$
% Because the lower probability is less than $0.5$, the choice $s=\sqrt{2}-1$
% implies an indeterminate decision at level $\gamma=0.5$, i.e., we cannot decide if $P(X\leq Y)>0.5$
% or if $P(X\leq Y)\leq 0.5$ in case  $U=1$, $n_1=n_2=1$. At level $\gamma=0.05$, with $n_1=n_2=n$ and $s\geq \sqrt{2}-1$, if $n\leq 4$ we can never decide between the hypothesis $P(X\leq Y)>0.5$ versus $P(X\leq Y)\leq 0.5$. For instance in case $n_1=n_2=4$ and $s=\sqrt{2}-1$, when $X=[1,2,3,4]$ and $Y=[5,6,7,8]$ (the $U$ statistic gets its  maximum value $16$), the lower expectation of $I_{P(X\leq Y)>0.5}$ is equal to $0.946$, while the upper is equal to $1$. This means that we are in the ``no decision region'' and thus cannot decide between the events $P(X\leq Y)>0.5$ and $P(X\leq Y)\leq 0.5$.  Observe that the lower and upper probabilities  produced by a value of $s$ are always contained in the probability intervals produced by larger values of $s$. Then, whenever we are undecided for $s_1$ we are also for $s_2>s_1$. If an earlier decision is desired, e.g. for $n\leq{4}$, one should use smaller values of $s$. Nonetheless, for large $n$ the distance between the upper and lower probabilities goes to $0$ as a consequence of their asymptotic consistency, then also the probability of collecting a sample $X^{n_1}$, $Y^{n_2}$ leading to an indeterminate decision goes to zero. 

% This guarantees that (to be proven)
% $$
% \underline{\mathcal{E}}[I_{P(X\leq Y)>0.5}|X^{n_1},Y^{n_2}]<0.5,
% $$
% which means that in case $n_1=n_2=1$  we will be undecided
% $P(X\leq Y)>0.5$.


\subsection{Managing ties}
\label{sec:ties}
To account for the presence of ties between samples from the two populations ($X_i=Y_j$), the common approach is to test the hypothesis $[P(X< Y) +\frac{1}{2} P(X=Y)]\leq 0.5$ against $[P(X< Y) +\frac{1}{2} P(X=Y)] > 0.5$. 
Since
$$
P(X< Y) +\frac{1}{2} P(X=Y)=E\left[I_{(X,\infty)}(Y)+\tfrac{1}{2}I_{\{X\}}(Y)\right]=E[H(Y-X)],
$$
where $H(\cdot)$ denotes the Heaviside step function, i.e., $H(z)=1$ for $z>0$,  $H(z)=0.5$ for $z=0$ and  $H(z)=0$ for $z<0$, in case of ties the $U$ statistic becomes
 \begin{equation}
\label{eq:Uties}
U=\sum\limits_{i=1}^{n_1} \sum\limits_{j=1}^{n_2} H(Y_j-X_i),
\end{equation}
and it represents the number of pairs $(X_i,Y_j)$ for which $X_i< Y_j$ plus half of the number of pairs $(X_i,Y_j)$ for which $X_i= Y_j$. 
The results presented in Section \ref{sec:MWW} are still valid if we substitute $I_{(X_j,\infty)}(Y_k)$ with $H(Y_k-X_j)$ in matrix $A$. 
%Also the convergence of the distribution of $P(X< Y) +\frac{1}{2} P(X=Y)$ with the distribution of the $U$ statistic under the null hypothesis can be extended to this case by computing the mean $\mu$ and variance $\sigma$ as in (\ref{eq:lowmeanvar}) with the new matrix $A$ and comparing it with the limiting mean and variance of $U$ for large $n$ in case of ties:
%$$
%\mu \approx 1/2, ~~~~ \sigma^2 \approx \frac{1}{6n}-\frac{\sum_{i=1}^k (t_i^3-t_i)}{12n^4}
%$$
%where, said $Z_1,\dots,Z_k$ the different values observed in $X$ and $Y$, $t_i$ is the number of observation in $X$ or $Y$ equal to $Z_i$.




\section{Asymptotic consistency}
\label{sec:Asymptotic}
From the expression of the lower and upper means in (\ref{eq:ranktest}), it can be verified that for $n_1,n_2\rightarrow\infty$:
$$
\underline{\mathcal{E}}[P(X\leq Y)|X^{n_1},Y^{n_2}],\overline{\mathcal{E}}[P(X\leq Y)|X^{n_1},Y^{n_2}] \sim {\mathcal{E}}[P(X\leq Y)|X^{n_1},Y^{n_2}]  \sim \frac{U}{n_1n_2}.
$$
Notice that in this section the symbol $\sim$ will be used to indicate asymptotic equivalence. 
The imprecision (degree of robustness) goes to zero for  $n_1,n_2\rightarrow \infty$ and the expectation ${\mathcal{E}}[P(X\leq Y)|X^{n_1},Y^{n_2}]$
is asymptotically equivalent to the Mann-Whitney statistic  \citep{Ferguson1973}.
The consistency of the IDP rank-sum test can be verified by considering the asymptotic behavior of the posterior lower and upper distributions of $P(X\leq Y)$ and compare it to the asymptotic distribution of the statistic $U/n_1n_2$. For ease of presentation, we limit ourselves to the case $n_1=n_2=n$.
In  \citet[Appendix A.5]{Lehmann} it is proved that $U/n_1n_2$ converges for $n_1,n_2\rightarrow\infty$ to a normal with mean $E[U_{ij}]=P(X\leq Y)$ and variance
\begin{equation} \label{eq:Uvar}
\frac{1}{n}Cov[U_{ij}, U_{i,k\neq j}]+\frac{1}{n}Cov[U_{ij},U_{k\neq i,j}],
\end{equation}
where $U_{rt}=I_{(X_r,\infty)}(Y_t)$ .
In the following theorem an equivalent result is proved for the lower distribution of $P(X \leq Y)$ in the IDP rank-sum test 
\begin{Mtheorem}
\label{th:2}
Assume that $n_1=n_2=n$, for $n\rightarrow \infty$ the IDP rank-sum test lower distribution converges to a normal with mean $E[U_{ij}]=P(X\leq Y)$ and variance given  by Equation (\ref{eq:Uvar}).
\end{Mtheorem}

The above proof can  be easily generalized to the upper distribution (the terms due to $w_{10}$ and $w_{20}$ vanish asymptotically) and to the case $n_1 \neq n_2$ (following the same procedure as in \citet[Th. 9]{Lehmann}). Theorem \ref{th:2} proves that the (upper and lower) distribution of the IDP rank-sum test is asymptotically equivalent to the distribution of the statistic $U/n_1n_2$ and, thus, the IDP rank-sum test  is consistent as a test for $P(X \leq Y)$.
Conversely, the MWW test is only consistent in the case $P(X \leq Y)=0.5$ and $F_X=F_Y$ or $P(X \leq Y) \neq 0.5$ and $F_X \neq F_Y$,
while it is not consistent for $P(X \leq Y) = 0.5$ and $F_X \neq F_Y$. 
For instance if $X \sim N(0,1)$ and $Y \sim N(0,\sigma^2)$ with $\sigma^2>1$,
two Normal distributions with different variance, then $P(X\leq Y)=0.5$ but the distributions are different.
In this case, if we apply MWW test with a significance level $\gamma=0.05$, MWW will return the alternative hypothesis
in approximatively $8.7\%$ of the cases (for a large $\sigma^2$), see \citet[Sec. 25.5]{dasgupta2008asymptotic}.
This means that MWW is not calibrated as a test for $P(X\leq Y)=0.5$ and it is not powerful as a test for  $F_X(x) \neq F_Y (x)$.
Conversely, because of Theorem \ref{th:2}, our IDP test with $\gamma=0.05$ will return the alternative hypothesis
(asymptotically) in  $5\%$ of the cases, which is correct since $P(X\leq Y)=0.5$. 

% Therefore, we have convergence 

% Assume that the distributions of $X$ and $Y$ are the same, then for $n_1,n_2\rightarrow \infty$ we at the mean and variance of the null hypothesis of the  Wilcoxon rank sum test  are (\ref{eq:Umeanvar}).

% \begin{Mcorollary}
%  \label{co:2}
% Assume that the distributions of $X$ and $Y$ are the same (null hypothesis of the  Wilcoxon rank sum test), 
% then for $n_1=n_2=n\rightarrow \infty$ 
% % Considering the mean $\mu$ and variance $\sigma^2$ of the lower distribution of $P(X\leq Y)$, if $X$ and $Y$ have the same distribution, for large $n$
%  it results that
% \begin{eqnarray}
% \label{eq:lowmeanvarres}
%  \mu_l&=&E_W[W]^TAE_V[V]\rightarrow  \dfrac{1}{2}, \\
% \label{eq:lowmeanvarres1}
% \sigma_l^2&=& trace[A^TE_W[WW^T]AE_V[VV^T]]-\mu_l^2 \sim\dfrac{1}{6n},
% \end{eqnarray}
% and the lower distribution converges to a normal with mean (\ref{eq:lowmeanvarres}) and variance (\ref{eq:lowmeanvarres1}).
% \end{Mcorollary}
% The asymptotic expression of the mean and variance follows directly from Theorem \ref{th:2} by noticing that in this case $E[U_{ij}]=0.5$, 
% $Cov[U_{ij}, U_{i,k\neq j}]=1/12$ and $Cov[U_{ij},U_{k\neq i,j}]=1/12$, see for instance \citet[Sec. 2.3]{Lehmann}.

% Assume that the distributions of $X$ and $Y$ are the same, then for $n_1,n_2\rightarrow \infty$ we know that the distribution of the null hypothesis of  the Wilcoxon rank sum test  converge to a Normal distribution with mean and variance given in (\ref{eq:Umeanvar}).
% Since (\ref{eq:lowmeandist}) and (\ref{eq:upmeandist}) also converge to a Normal distribution under the null hypothesis, we must check that the mean and variance
% of (\ref{eq:lowmeandist}) and (\ref{eq:upmeandist}) converge to those in (\ref{eq:Umeanvar}). For simplicity, we will consider only the case $n_1=n_2=n$.

% Before to state this result, we prove the following lemma.
% \begin{Mlemma}
%  Consider Theorem \ref{th:1} and assume that (i) $n_1=n_2=n$ the matrix $A$ is upper triangular, then from (\ref{eq:lowmeanvar}) we can derive that
% \begin{equation}
% \label{eq:lowmeanvarres}
%  \mu=E_W[W]^TAE_V[V]=\dfrac{1}{2}, ~~~\sigma^2= trace[A^TE_W[WW^T]AE_V[VV^T]]-\mu^2=\dfrac{n_1+n_2}{12n_1n_2}.
% \end{equation} 
% \end{Mlemma}
% \begin{Proof}
%  Note that $E_W[W]$ and $E_V[V]$ are, respectively, $n_1\times 1$ and $n_2\times 1$ vectors equal to
% $$
% E_W[W]=\left[1,1,\dots,1\right]^T\frac{1}{s+n_1}, ~~E_V[V]=\left[1,1,\dots,1\right]^T\frac{1}{s+n_2}.
% $$
% Then, it follows that
% $$
% ~
% $$
% \end{Proof}
% \begin{Mtheorem}
% \label{th:2}
% Considering the mean $\mu$ and variance $\sigma^2$ of the lower distribution of $P(X\leq Y)$, if $X$ and $Y$ have the same distribution, for large $n$ it results that
% \begin{eqnarray}
% \label{eq:lowmeanvarres}
%  \mu&=&E_W[W]^TAE_V[V]\approx \dfrac{1}{2}, \\
% \label{eq:lowmeanvarres1}
% \sigma^2&=& trace[A^TE_W[WW^T]AE_V[VV^T]]-\mu^2\approx \dfrac{1}{6n}.
% \end{eqnarray}
% \end{Mtheorem}
%  \begin{Proof}
% Consider Theorem \ref{th:1}, without loss of generality we can assume that the observations in the vectors $X^{n}$ and $Y^{n}$ are ordered.
% Then,  $X_r$ is the  $r$th-order static of the vector $X^{n}$ and converges to the $r$th-quantile for $n\rightarrow \infty$, e.g., for $r=n/2$ it converges to the median.
% This means that, for $n\rightarrow \infty$, the probability that the observation $Y_q$ is larger than $X_r$ will converge to $1$ if $q>r$, to $0$ if $q<r$ and to $0.5$ if $q=r$. Thus, for $n\rightarrow \infty$ the matrix $A$ in Theorem \ref{th:1} converges to an upper triangular matrix except for the diagonal elements, whose elements will be distributed as $n$ Bernoulli distributions with probability $0.5$. 
% Since $E_W[W]$ and $E_V[V]$ are $n\times 1$ vectors equal to
% $$
% E_W[W]=E_V[V]=\left[1,1,\dots,1\right]^T\frac{1}{s+n},
% $$
% then, when $A$ is upper triangular with Bernoulli-distributed diagonal elements, for $n\rightarrow \infty $ it follows that
% $$
%  \mu=E_W[W]^TAE_V[V]=\dfrac{n(n-1)+2d}{2(s+n)^2}\rightarrow \dfrac{1}{2}.
% $$
% where d is a Binomial random variable with parameters $(n,0.5)$ which, for $n$ large converge to the value $n/2$. 
% For the variance, write
% $$
% A = D + T^+ ~~~~ E_W[WW^T] = E_V[VV^T] = k(I+J_n)
% $$
% where $D$ is a diagonal matrix with the Bernoulli random variables $d_1,\dots,d_n$ on the diagonal, $T^+$ is a upper triangular matrix with elements $T_{jk}=1$ if $k>j$ and $T_{jk}=0$ if $k\leq j$, $I$ is the identity matrix, $J_n$ is a matrix of ones and $k$ is the coefficient $1/(s+n)(s+n+1)$.
% The argument of the trace  is
% $$
% k^2(D + T^-)(I + J_n)(D + T^+)(I + J_n) = k^2(D + T^- + DJ_n + T^-J_n)(D + T^+ + DJ_n + T^+J_n) = 
% $$
% $$
% DD + D T^+ + D~DJ_n + D~T^+J_n + T^-D + T^- T^+ + T^- ~DJ_n + T^- ~T^+J_n + DJ_n~D + 
% $$
% $$
% DJ_n~T^+ + DJ_n~DJ_n + DJ_n~T^+J_n + T^-J_n~D + T^-J_n~T^+ + T^-J_n~DJ_n + T^-J_n~T^+J_n
% $$
% where $T^-$ is the transpose of $T^+$, $DJ_n$, $T^-J_n$ and $T^+J_n$ are matrices with elements, respectively equal to $d_j$, $j-1$ and $n-j$ at all positions $jk$. The trace is
% $$
% trace(D T^+)=trace(T^- D)=0 
% $$
% $$
% trace(DD) = trace(D~DJ_n) = trace(DJ_n~D) = \sum_1^n d_i^2 \rightarrow E[\sum_1^n d_i^2] = n/2
% $$
% $$
% trace(T^- ~DJ_n + D~T^+J_n) = trace(DJ_n~T^+ + T^-J_n~D ) = (n-1) \sum_1^n d_i \rightarrow n(n-1)/2
% $$
% $$
% trace(T^- T^+ ) = \sum_0^{n-1} i = n(n-1)/2 
% $$
% $$
% trace(T^- ~T^+J_n ) = trace(T^-J_n~T^+ ) = \sum_0^{n-1} i^2 = n(n-1)(2n-1)/6 
% $$
% $$
% trace(DJ_n~DJ_n) = (\sum_1^n d_i)^2 \rightarrow n^2/4
% $$
% $$
% trace(DJ_n~T^+J_n ) = trace(T^-J_n~DJ_n ) = (\sum_1^n d_i)(\sum_0^{n-1} i) \rightarrow n^2(n-1)/4  
% $$
% $$
% trace(T^-J_n~T^+J_n) = (\sum_0^{n-1} i)^2 = n^2(n-1)^2/4 
% $$
% For $n$ large the trace tends to 
% $$
% \frac{\frac{7n^2}{4} + \frac{n(n-1)(2n-1)}{3} + \frac{n^2(n-1)}{2} + \frac{n^2(n-1)^2}{4}}{(s+n)^2(s+n+1)^2}
% $$
% 
% %$$
% %\left(\dfrac{n^2(n+1)^2}{4}+\dfrac{n(n+1)(2n+1)}{12}+\dfrac{n^2(n+1)}{2}+\dfrac{3n(n+1)}{4}\right)\dfrac{1}{(s+n)^2(s+n+1)^2}.
% %$$
% 
% Hence, by subtracting $\mu^2=\frac{n^4}{4(s+n)^4}$, for large $n$ one gets:
% 
% $$
% \sigma^2 \approx \frac{4 n s^2 + 2 n^2 ( 4 s + 3 s^2 ) + 4 n^3 ( 1 + 3 s + 2 s^2) + n^4 (3 + 10 s) + 2 n^5 }{12 (n + s)^4 (1 + n + s)^2} \approx \frac{1}{6n}.
% $$
% 
% \end{Proof}
% This result can again  be generalized to the upper distribution and to the case $n_1\neq n_2$.
% Since the variance goes to zero, Corollary \ref{co:2} proves that, under the null hypothesis (same distribution), the 
% IDP and the MWW rank-sum test are asymptotically consistent.
% When the null hypothesis is not true, the MWW test is asymptotically consistent only if under the null hypothesis $F_X$ and $F_Y$ verify the location-shift model. If, instead, we wish to compare the null hypothesis $F_X=F_Y$ against the alternative $F_X\neq F_Y$ without restriction on the form of $F_X$ and $F_Y$ it is known that the MWW test is asymptotically consistent only in the case $P(X \leq Y)\neq 0.5$ \citet{Pratt1964, Fay2010}.
% 
% Instead, when the distributions are different but $P(X \leq Y)=0.5$, the power of the MWW test remains small even for large $n$, i.e., it is not able to detect the difference in distribution, since the actual distribution of the $U$ statistic converges to a normal distributions with mean equal to that it would have under the null hypothesis. On the other side, if used to test the null hypothesis $P(X \leq Y) \leq 0.5$ against $P(X\leq Y)> 0.5$, the Wilcoxon rank sum test would not be calibrated under the null hypothesis when $P(X \leq Y)=0.5$ and $F_X\neq F_Y$, since the actual variance of the $U$ statistic would be larger than that under the (correct) null hypothesis $F_X = F_Y$. Clearly this use of the Wilcoxon rank sum test is mistaken; notice however that it is often adopted by practitioners. 
% 
% Conversely, it follows from Theorem \ref{th:2} and Corollary \ref{co:2}, that the IDP rank-sum test is always consistent, and under the null hypothesis $P(X\leq Y)>0.5$ it is asymptotically calibrated, without restriction on the form of $F_X$ and $F_Y$, since the lower and upper distributions converge to the true (normal) limit distribution of the statistic $U/(n_1n_2)$. Then, it is correct to apply it for testing the null hypothesis $P(X \leq Y) \leq 0.5$ against $P(X\leq Y)> 0.5$. 
% % This means that they will asymptotically provide the same answers under the null hypothesis and that the DP rank sum test is calibrated.
% 
% To better understand the differences between the two tests when $F_X\neq F_Y$ and at the same time $P(X \leq Y) = 0.5$, consider the following cases:
% \begin{itemize}
% \item[(a)] the sample $X_1,\dots,X_n$ is generated from a mixture of two uniform distributions on $[0, 5]$ and $[15,20]$ each with probability $0.5$ and the sample $Y_1,\dots,Y_n$ from a  uniform distribution on $[5, 15]$;
% \item[(b)] both samples $X_1,\dots,X_n$ and $Y_1,\dots,Y_n$ are generated from a uniform distribution on $[0, 20]$.
% \end{itemize}
% %The distribution of the $U$ statistic in the two cases is shown in Figure \ref{fig:4} for $n_1=n_2=100$. 
% %
% %\begin{figure}[h]
% 		%\centering
% 		%\includegraphics[width=7cm]{SameUdiffDist}
%     %\caption{Distribution of $U/n^2$ in cases (a) and (b) for $n=100$.}
% 		%\label{fig:4}
% %\end{figure}
% 
% In both cases $P(X\leq Y)=0.5$, but the variance of the $U$ statistic is different and is asymptotically equivalent to $1/4n$ in case (a), since $Cov[U_{ij}, U_{i,k\neq j}]=1/4$ and $Cov[U_{ij},U_{k\neq i,j}]=0$, and to $1/6n$ in case (b) (same distribution). 
% Let us consider the (wrong) test $P(X \leq Y) \leq 0.5$ against $P(X\leq Y)> 0.5$. Based on the asymptotic distributions of the $U$ statistic under the null hypothesis (case (b)), the one-sided Wilcoxon rank sum test rejects the null hypothesis with significance $1-\gamma$ if
% $$
% \dfrac{U}{n^2}>\dfrac{1}{2}+\sqrt{\dfrac{1}{6n}}z_{1-\gamma},
% $$
% In case (b) this condition is verified with probability $\gamma$, while the probability that this condition is verified when $\dfrac{U}{n_1n_2}$ is normal with mean $0.5$ and variance $1/4n$ (case (a)) is
% $$
% P\left(\sqrt{4n}\left(\tfrac{U}{n^2}-\tfrac{1}{2}\right)>\sqrt{\dfrac{2}{3}}z_{1-\gamma}\right),
% $$ 
% which is larger than $\gamma$. Then the test is not calibrated. 
% On the other side, if we consider the (correct) test $F_X = F_Y$ against $F_X\neq F_Y$ the probability of rejection (power) will be 
% $$
% 1-P\left(\sqrt{4n}\left|\tfrac{U}{n_1n_2}-\tfrac{1}{2}\right|<\sqrt{\dfrac{2}{3}}z_{1-\gamma/2}\right).
% $$ 
% which remains very low (about $0.1$ for $\gamma = 0.05$) regardless of the number of samples collected.
% 
% %$$
% %\begin{array}{rcl}
% %X_a&=&\{  1,    2 ,    3,     4,     5,    16,    17,    18,    19,    20\},\\
% %Y_a&=&\{  6,     7,     8,     9,    10,    11,    12,   13,    14,    15\},
% %\end{array}
% %$$
% %and
% %$$
% %\begin{array}{rcl}
% %X_b&=&\{ 1,     3,     5,     7,     9,    12,    14,    16,    18,    20\},\\
% %Y_b&=&\{  2,     4,     6,     8,    10,    11,    13,    15,    17,    19\}.
% %\end{array}
% %$$
% %In both cases the $U$ statistic is $50$ but, as depicted in Figure \ref{fig:2}, in case (a) the observations $Y$
% %are in the middle of the $X$, while in case (b) the observations $X$ and $Y$ alternate.
% %For the frequentist test, there is not difference between the two cases (since the $U$ statistic takes the same value), 
% In the IDP rank-sum test, the lower and upper posterior distributions have larger variance in case (a)  than in case (b) (Figure \ref{fig:3} shows an example of it for $n=10$). Then, in case (a) we need larger values of the $U$ statistic than in case (b) to have a lower probability of $P(X\leq Y) > 0.5$ larger than $1-\gamma$ and thus reject the null hypothesis. This is in agreement with the fact that in case (a) the actual distribution of the $U$ statistic has larger variance and thus large values of $U$ are more probable than in case (b). This explains intuitively why the IDP rank-sum test is asymptotically calibrated even when $F_X\neq F_Y$.
% %
% %\begin{figure}[h]
%         %\centering
%         %\begin{subfigure}[b]{0.35\textwidth}
%                 %\centering
%                 %\includegraphics[width=\textwidth]{multimod}
%                 %\caption{}
%                 %\label{fig:mm}
%         %\end{subfigure}%
%         %~ %add desired spacing between images, e. g. ~, \quad, \qquad etc.
%           %%(or a blank line to force the subfigure onto a new line)
%         %\begin{subfigure}[b]{0.35\textwidth}
%                 %\centering
%                 %\includegraphics[width=\textwidth]{unimod}
%                 %\caption{}
%                 %\label{fig:uni}
%         %\end{subfigure}
%         %\caption{ }\label{fig:2}
% %\end{figure}
% %
% \begin{figure}[h]
%         \centering
%         \includegraphics[width=0.35\textwidth]{multimod_lu}  \includegraphics[width=0.35\textwidth]{unimod_lu}
% %         \begin{subfigure}[b]{0.35\textwidth}
% %                 \centering
% %                 \includegraphics[width=\textwidth]{multimod_lu}
% %                 \caption{}
% %                 \label{fig:mm}
% %         \end{subfigure}%
% %         ~ %add desired spacing between images, e. g. ~, \quad, \qquad etc.
% %           %(or a blank line to force the subfigure onto a new line)
% %         \begin{subfigure}[b]{0.35\textwidth}
% %                 \centering
% %                 \includegraphics[width=\textwidth]{unimod_lu}
% %                 \caption{}
% %                 \label{fig:uni}
% %         \end{subfigure}
%         \caption{Results of the hypothesis test for $n_1=n_2=10$ observations sampled from the distributions of cases (a) and (b). In both cases $U/(n_1n_2)=0.4$}
% 				\label{fig:3}
% \end{figure}


%Note that, in  case (a) the variance of the distributions is larger than in case (b) (while the lower and upper means are equal in the two cases since they depend only on the value of the $U$ statistic). 

% When the null hypothesis is not true, the  Wilcoxon rank sum test is still based on the null hypothesis, while this is not the case for the Dirichlet process based test.
% This difference can be understood by considering the following example.
% Assume that $n_1=n_2=n=2\ell$ for $\ell=1,2,\dots$ and the following two cases
% $$
% c_1: ~~x_{(1)}<y_{(1)}<x_{(2)}<y_{(2)}<\dots<x_{(n)}<y_{(n)},
% $$
% and
% $$
% c_2: ~~x_{(1)}<x_{(2)}<\dots<x_{(n/2)}<y_{(1)}<y_{(2)}<\dots<y_{(n)}<x_{(n/2+1)}<x_{(n/2_2)}<\dots<x_{(n)}.
% $$
% Note that, in both cases, for $n\rightarrow \infty$
% $$
% \frac{U}{n_1n_2} \rightarrow\frac{1}{2}
% $$
% Thus, one has that
% $$
% \begin{array}{rcl}
% Var[P(X\leq Y)]&=&\sum\limits_{j=1}^{n_1}\sum\limits_{k=1}^{n_2}\dfrac{2}{(s+n_1)(s+n_1+1)}\dfrac{2}{(s+n_2)(s+n_2+1)} I_{[X_i,+\infty)}(Y_k)-E[P(X\leq Y)]^2
% %&+&
% \end{array}
% $$


\section{Numerical simulations}
\label{sec:sim}
Consider a Monte Carlo experiment in which $n_1,~n_2$ observations $X,Y$ are generated based  on
$$
X\sim N(0,1),~~Y\sim N(\Delta,1),
$$
with $\Delta$ ranging from $-1.5$ to $1.5$.
To facilitate the comparison of IDP tests with more traditional tests (which never issue indeterminate outcomes) we introduce a new test (called ``50/50 when indeterminate'') which returns the same response as the IDP when this is determinate,
and issues a random answer (with 50/50 chance) otherwise. We want to stress that the test ``50/50 when indeterminate'' has been introduced only for the sake of  comparison. We are not suggesting that when the IDP is indeterminate we should toss a coin to take the decision. On the contrary we claim that the indeterminacy of the IDP is an additional useful information that our approach gives to the analyst.  In these cases she/he knows that (i) her/his posterior decisions would depend on the choice of the prior $G_0$; (ii) deciding between the two hypotheses under test is a difficult problem as shown by the comparison with the Bayesian Bootstrap DP (BB-DP) rank-sum test ($s=0$) and MWW tests. Based on this additional information, the analyst can for example decide to collect additional measurements to  eliminate the indeterminacy (in fact we have seen that when the number of observations goes to infinity the indeterminacy goes to zero).
% We point the reader to \citet{zaffalon2012evaluating} for a broader discussion about indeterminacy and risk-aversion.  

We start by comparing the performance of the BB-DP and IDP tests.
To evaluate the performance of the tests, we have used the loss function defined in (\ref{eq:loss}).
In particular, for each value of $\Delta$ we have performed  $20000$ Monte Carlo runs by generating in each run $n_1=n_2=20$
observations for $X,Y$. 
The average  loss for the cases (i) $K_1=K_2=1$ (i.e., $\gamma=0.5$) (ii) $K_1=1$ and $K_2=9$ (i.e., $\gamma=0.1$) and (iii) $K_1=1$ and $K_2=19$ (i.e., $\gamma=0.05$) is shown in Figure \ref{fig:1bsim} as a function of $\Delta$.
\begin{figure}[h]
 \centerline{\includegraphics[width=12cm]{Figure_con_s=0.eps}}
    \caption{Loss as a function of $\Delta$ for the case $K_0=K_1=1$ (left),  $K_0=1,K_1=9$ (center) and $K_0=1,K_1=19$ (right).}
\label{fig:1bsim}
\end{figure}
%\begin{figure}[htb]
 %\centerline{\includegraphics[height=4.5cm]{Figure_con_s=0_alpha100=50}}
    %\caption{Loss as a function of $\Delta$ for the case $K_0=K_1=1$.}
%\label{fig:1bsim}
%\end{figure}
%\begin{figure}[htb]
 %\centerline{\includegraphics[height=4.5cm]{Figure_con_s=0_alpha100=10}\includegraphics[height=4.5cm]{Figure_con_s=0_alpha100=5_fifty}}
    %\caption{Loss as a function of $\Delta$ for the case $K_0=1,K_1=9$ (left) $K_0=1,K_1=19$ (right).}
%\label{fig:1bsimb}
%\end{figure}
In particular, we report (i) the loss of the BB-DP test; (ii) the loss of the IDP test when it is determinate;
(iii) the indeterminacy of the IDP test, i.e., the number of times it returns an indeterminate response divided by the total number of Monte Carlo runs;
(iv) the loss of the ``50/50 when indeterminate'' test.

From  Figure \ref{fig:1bsim}, it is evident that the performance of the BB-DP and 50/50 tests practically coincide.
Furthermore, since we noticed from experimental evidence that in all cases in which BB-DP is determinate,  BB-DP returns the same response as IDP, 
%(i.e.,in the determinate cases the accuracy of BB-DP is also equal to the diamond curve in Figure \ref{fig:1bsim})
the difference between the two tests is only in the runs  where the IDP is indeterminate. In these runs, BB-DP is clearly guessing at random, since overall it has the same accuracy as the 50/50 test. Therefore,  the IDP is able to isolate several instances in which BB-DP is guessing at random, thus providing useful information to the analyst. 
Assume, for instance, that we are trying to compare the effects of two  medical treatments (``Y  is better than X'')
and that, given the available data, the IDP is indeterminate. In such situation the BB-DP test always issues a determinate response (I can tell if ``Y  is better than X''), but it turns out that its response is virtually random (like if we were tossing a coin). 
On the other side, the IDP acknowledges the impossibility of making a decision and thus, although BB-DP and the IDP (more precisely the ``50/50 when indeterminate'' test) have the same accuracy, the IDP provides more information. 
%The conclusions for the cases $K_1=1$ and $K_2=9$ (or $K_2=19$) are similar (see Figure \ref{fig:1bsim} center and right), in the sense that the performance of the BB-DP and 50/50 tests (almost) coincide.
Note that, in all the cases the maximum percentage of runs in which the IDP is indeterminate is about $18\%$;  this means that 
BB-DP is issuing a random answer in $18\%$ of the cases, which is a large percentage.
For large $|\Delta|$, i.e. when the hypothesis test is easy, there are not indeterminate instances and both the BB-DP and the IDP tests have zero loss.
It is interesting to note that,  for the cases $K_1=1$ and $K_2=9$ (or $K_2=19$) (Figure \ref{fig:1bsim} center and right) it is more risky (we may incur a greater loss) taking the action $a=1$ than $a=0$, and thus the indeterminacy curve is shifted to the $\Delta>0$ quadrant.

We have also compared the IDP test and the one-sided MWW null hypothesis significance test (NHST) implemented  according to the conventional decision criterion,  $p<0.05$.  It is well known that the decision process in NHST is flawed. It is  based on asking
what is the probability of the data statistic if the null hypothesis were true. 
This means that NHST can only reject the null hypothesis ($\Delta \leq 0$), contrarily to a Bayesian analysis that can also accept this hypothesis.
Furthermore, in a Bayesian analysis we have a principled way to determine $\gamma$ (i.e., by means of a loss function) which is lost when putting decisions in the format $p < 0.05$ (or the more vague  $p < 0.1$). 
Because of these differences, it is difficult to compare the Bayesian with the NHST approach, where we do not have a clear interpretation of the significance level.
However, we believe a relatively fair comparison can be carried out by setting $\gamma$ equal to the significance level of the NHST test, so that the decision criteria adopted by the two test are as  similar as possible. 
Figure \ref{fig:1sim} shows the power for the case $\gamma=0.05$, $n_1=n_2=10$ and $n_1=n_2=20$.
\begin{figure}[htb]
 \centerline{\includegraphics[height=4.5cm]{Correctness_n1_10_n2_10_alpha100=5}\includegraphics[height=4.5cm]{Correctness_n1_20_n2_20_alpha100=5}}
    \caption{Power as a function of the difference of the medians $\Delta$ for the case $n_1=n_2=10$ (left) and $n_1=n_2=20$ (right) with $\gamma=0.05$.}
\label{fig:1sim}
\end{figure}
In case $n_1=n_2=20$ (Figure \ref{fig:1sim}, right) it is evident that the performance of the MWW and 50/50 tests practically coincide.
Since it can be verified experimentally that when the IDP is determinate the two tests return the same results, this again suggests that when the IDP is indeterminate we have equal probability that $p < 0.05$ or  $p > 0.05$, as it is shown in Figure \ref{fig:pval}. The IDP test is able to isolate some instances in which also the MWW test is issuing a random answer.
Note that, for $\Delta=0.5$, the maximum percentage of runs in which the IDP test is indeterminate is large, about $18\%$;  this means that 
MWW is issuing a random answer in $18\%$ of the cases. 

\begin{figure}[htb]
  \centerline{\includegraphics[height=4.3cm]{pval}}
    \caption{Distribution of MWW p-values in the IDP indeterminate  cases  for $n_1=n_2=20$, $\gamma=0.05$ and $\Delta=0.5$.}
\label{fig:pval}
\end{figure}

% Looking at the curve of the IP-Dp indeterminacy, it can be observed that it reaches its maximum value
% at $\Delta\approx 0.6$ and then goes to zero for $\Delta > 1.5$.
% For $\Delta=2$ the accuracy of MWW and IP-Dp determinate is one, which means that when the hypothesis test
% is easy (large $\Delta$), there are not indeterminate instances and both tests have accuracy one.
The results for the case $n_1=n_2=10$ (Figure \ref{fig:1sim}, left) lead to similar conclusions. The performance of the MWW and 50/50 tests (almost) coincide.
The 50/50 test is slightly better for $\Delta\leq 0.9$ and slightly worse for $\Delta> 0.9$.
$\Delta=0.9$ is the value  which corresponds to the maximum indeterminacy of the IDP, i.e. $30\%$.
Thus, for $\Delta=0.9$, MWW is guessing at random in $30\%$ of the runs.

It is worth analyzing also the case $\Delta=0$. We know that in this case the frequentist test is calibrated, i.e., when $\gamma=0.05$
the percentage of correct answers is $95\%$ (although it can be noticeably larger for small values of $n_1,~n_2$ since the discreteness of the MWW statistic originates a gap between the chosen $\gamma$ and the actual significance of the MWW test).
Table \ref{tab:1sim} shows the accuracy for $\Delta=0$. The performance of the MWW and 50/50 tests are similar also in this case.
The difference is about $1\%$ (for $n_1=n_2=10$) and $0.5\%$ (for $n_1=n_2=20$). 
%Observe that, the accuracy $0.945$ of the 50/50 test is  obviously equal (on average) to  $0.911+0.068/2$, i.e.,
%the accuracy of IDP in the determinate cases plus half of the indeterminate cases.

\begin{table}
\centering
{\small
 \begin{tabular}{|c|c|c|c}
\hline
& Accuracy $n_1=n_2=10$ & Accuracy $n_1=n_2=20$\\
\hline
\hline
MWW & 0.955 & 0.952\\
50/50 test & 0.945 & 0.947\\
\hline
IDP when determinate & 0.911 & 0.924\\
Indeterminacy & 0.068 & 0.045\\
\hline
 \end{tabular}  }
\caption{Accuracy for  $\Delta=0$ and $\gamma=0.05$.}
\label{tab:1sim}
\end{table}
Also in this case, when the IDP is determinate, it returns the same responses as MWW.  This result holds independently of the choice of $\gamma$, as shown by Figure \ref{fig:2sim} and Table \ref{tab:2sim} where we have repeated the above experiment for $n_1=n_2=20$ with, this time, $\gamma=0.1$ and $\gamma=0.25$.

Finally, Figure \ref{fig:2simb} shows the error (one minus the accuracy) of the IDP test as a function of $s$, when $\gamma=0.1$, $n_1=n_2=20$ and $\Delta=0$.
Clearly, the error of the MWW test is constantly equal to $\gamma=0.1$ (we are under the null hypothesis of MWW).
The error of the IDP test when determinate decreases with $s$, because of the increase of the indeterminacy.
%Note that for $s=0$, the error of the IDP when determinate and the error of the 50/50 test coincide, since there are not indeterminate instances in this case.
The error of the 50/50 test has a convex trend, clearly decreasing for $ s< 0.2$ and increasing for $s>0.5$. 
%It is higher for $s=0$ (Bayesian Boostrap)  than for  and then it increases again for $s>0.5$.
This (together with the other results of this section) may be seen as an empirical confirmation that the choice of $s=\sqrt{2}-1$ is appropriate, since it guarantees a good trade-off between robustness and indeterminacy.

Finally observe that all the above differences/similarities between the three tests appear also 
in the case we consider location-shift models with distributions different from Gaussians (e.g., 
Student-t distribution). These results have been omitted for shortness.
% Similar results can be obtained if instead
% 
% 
% .
% Furthermore the NHST cannot decision of rejectinb the null whenever the p-value is less than $0.05$
% 
% So it 
% Answering this question provides little direct information about the probability
% of parameter values given the data, which is what we want
% to know. 
% 
% NHST is based on sampling distributions generated
% by assuming that the null-hypothesis values of the param-
% eters are true
% 
% ), the
% 
% count  the number of times the true hypothesis is correctly detected 
% among the $15000$ Monte Carlo runs we have generated. This number divided by $15000$ gives the accuracy of the test.\footnote{When
% $\Delta>0$, the accuracy is also called power of test.}
% 
% 
% 
% 
% Our aim is to compare the performance of the one-sided MWW test with  that of the corresponding test based on the prior ignorance Dirichlet process (IP-Dp) model.  
% To evaluate the performance of the test, we count  the number of times the true hypothesis is correctly detected 
% among the $15000$ Monte Carlo runs we have generated. This number divided by $15000$ gives the accuracy of the test.\footnote{When
% $\Delta>0$, the accuracy is also called power of test.}
% \begin{figure}[htb]
%  \centerline{\includegraphics[height=4.5cm]{Correctness_n1_10_n2_10_alpha100=5}\includegraphics[height=4.5cm]{Correctness_n1_20_n2_20_alpha100=5}}
%     \caption{Accuracy as a function of the difference of the medians $\Delta$ for the case $n_1=n_2=10$ (left) and $n_1=n_2=20$ (right) with $\gamma=0.05$.}
% \label{fig:1sim}
% \end{figure}
% The results of the simulations with $\gamma=0.05$ are shown in Figure \ref{fig:1sim} for $n_1=n_2=10$ and $n_1=n_2=20$.
% In particular, Figure \ref{fig:1sim} reports as a function of $\Delta$:
% (i) the accuracy of the MWW test; (ii) the accuracy of the IP-Dp test when it is determinate;
% (iii) the indeterminacy of the IP-Dp test, i.e., the number of times it returns an indeterminate response divided by the total number of Monte Carlo runs;
% (iv) the accuracy of a new test (called ``50/50 when indeterminate'') which returns the same response as IP-Dp when IP-Dp is determinate,
% and issues a random answer (with 50/50 chance) when IP-Dp is indeterminate.
% We have introduced this new artificial test to facilitate the comparison of the MWW and IP-Dp tests.
% 
% From  Figure \ref{fig:1sim}, in case $n_1=n_2=20$,  it is evident that the performance of the MWW and 50/50 tests practically coincide.
% Furthermore, since we noticed from experimental evidence that in all cases in which IP-Dp is determinate,  MWW returns the same response as IP-Dp (i.e.,
% in the determinate cases the accuracy of MWW is also equal to the diamond curve in Figure \ref{fig:1sim}), the difference
% between the two tests is only in the runs  where IP-Dp is indeterminate. In these runs, MWW is clearly guessing at random, since overall it has the same accuracy as the 50/50 test.
% Therefore,  IP-Dp is able to isolate some instances in which MWW is guessing at random. 
% Assume now that we are trying to compare the effects of two  medical treatments (``Y  is better than X'')
% and that, given the available data, IP-Dp is indeterminate. In such situation the MWW always issues a determinate response (I can tell if ``Y  is better than X'' with the desired significance), but it turns out that its response is completely random (like if we were tossing of a coin). On the other side, the IP-Dp acknowledge the impossibility of making a decision (I do not know whether the probability that ``Y  better than X'' is true is larger or smaller than $0.95$) and thus, although  MWW and the IP-Dp (more precisely the 50/50 test derived by IP-Dp) have the same accuracy, the IP-Dp provides more information. Based on this information, we can for example decide to collect 
% additional measurements to  eliminate the indeterminacy.
% 
% Note that, for $\Delta=0.5$, the percentage of runs in which  IP-Dp is indeterminate is about $18\%$;  this means that 
% MWW is issuing a random answer in $18\%$ of the cases, which is a large percentage.
% Looking at the curve of the IP-Dp indeterminacy, it can be observed that it reaches its maximum value
% at $\Delta\approx 0.6$ and then goes to zero for $\Delta > 1.5$.
% For $\Delta=2$ the accuracy of MWW and IP-Dp determinate is one, which means that when the hypothesis test
% is easy (large $\Delta$), there are not indeterminate instances and both tests have accuracy one.
% 
% The conclusions for the case $n_1=n_2=10$ are similar, see Figure \ref{fig:1sim} (left). The performance of the MWW and 50/50 tests (almost) coincide.
% The 50/50 test is slightly better for $\Delta\leq 0.9$ and slightly worse for $\Delta> 0.9$.
% $\Delta=0.9$ is the value  which corresponds to the maximum indeterminacy of IP-Dp, i.e. $30\%$.
% Thus, for $\Delta=0.9$, MWW is guessing at random in $30\%$ of instances.
% 
% It is also worth to analyze the case $\Delta=0$. We know that in this case the frequentist test is calibrated, i.e., when $\gamma=0.05$
% the percentage of correct answers is (at least) $95\%$.\footnote{For small values of $n_1,n_2$, the discreteness of the MWW statistic originates a gap between the chosen $\gamma$ and the actual significance of the MWW test.
% For instance, in case $n_1=n_2=5$ and $\gamma=0.05$, the frequentist test  rejects the null hypothesis with a $p$-value that is less than or equal to $0.0476$, which is obtained for $U=19$, because the exact value $0.05$ is not attainable from the discrete distribution: the $p$-value for $U=20$ is $0.0754$, and thus, around $0.05$, there is a gap of $0.0754-0.0476=0.0278$ between contiguous p-values. Therefore, for small $n$ we expect an accuracy of the MWW noticeably larger than $1-\gamma$.}
% Table \ref{tab:1sim} shows the accuracy for $\Delta=0$. The MWW and 50/50 tests are close also in this case.
% The difference is about $1\%$ (for $n_1=n_2=10$) and $0.5\%$ (for $n_1=n_2=20$). 
% Observe that, the accuracy $0.945$ of the 50/50 test is  obviously equal (on average) to  $0.911+0.068/2$, i.e.,
% the accuracy of IP-Dp in the determinate cases plus half of the indeterminate cases.
% Also in this case, when IP-Dp is determinate, it returns the same responses as MWW.
% 
% \begin{table}
% \centering
%  \begin{tabular}{|c|c|c|c}
% \hline
% & Accuracy $n_1=n_2=10$ & Accuracy $n_1=n_2=20$\\
% \hline
% \hline
% MWW & 0.955 & 0.952\\
% 50/50 test & 0.945 & 0.947\\
% \hline
% IP-Dp when determinate & 0.911 & 0.924\\
% Indeterminacy & 0.068 & 0.045\\
% \hline
%  \end{tabular} 
% \caption{Accuracy for  $\Delta=0$ and $\gamma=0.05$.}
% \label{tab:1sim}
% \end{table} 
% We have repeated the above experiment for the case $n_1=n_2=20$ but with $\gamma=0.1$ and $\gamma=0.25$.
% The results are shown in Figure \ref{fig:2sim} and Table \ref{tab:2sim}.
% Note that, also in this case, the performance of the MWW and 50/50 test almost coincides.
% Thus, the previous considerations appear to hold independently of the choice of $\gamma$.
% 
% Finally, in case $\gamma=0.1$,  $n_1=n_2=20$ and $\Delta=0$, we have computed the error (one minus the accuracy)
% for the IP-Dp test as a function of $s$.
% The error of the MWW test is in this case equal to $\gamma=0.1$ (we are under the null hypothesis).
% From Figure \ref{fig:2simb}, it can be observed that the error of the IP-Dp test when determinate decreases with $s$, because of the increase of the indeterminacy.
% Note that for $s=0$, the error of IP-Dp when determinate and the error of the 50/50 test coincide, since there are not indeterminate instances in this case.
% Looking at the plot relative to the 50/50 test, it can be noticed that it has a convex trend.
% It is higher for $s=0$ (Bayesian Boostrap)  than for $0.3\leq s \leq 0.4$ and then it increases again for $s>0.5$.
% This (together with the other results of this section) may be seen as an empirical confirmation that the choice of $s=\sqrt{2}-1$ is appropriate.
% It guarantees the maximum robustness at the minimum of conservativeness (indeterminacy).




% \subsubsection{Case $\alpha=0.1$}

\begin{table}
\centering
{\small
 \begin{tabular}{|c|c|c|c}
\hline
& Accuracy $\gamma=0.1$ & Accuracy $\gamma=0.25$ \\
\hline
\hline
MWW & 0.8995 & 0.7552\\
50/50 test & 0.8993 & 0.7482\\
\hline
IDP when determinate & 0.8568 & 0.6777\\
IDP indeterminacy & 0.081 & 0.142\\
\hline
 \end{tabular} }
\caption{Accuracy in case $\Delta=0$ for $n_1=n_2=20$}
\label{tab:2sim}
\end{table}

\begin{figure}[htb]
  \centerline{\includegraphics[height=4.5cm]{Correctness_n1_20_n2_20_alpha100=10}\includegraphics[height=4.5cm]{Correctness_n1_20_n2_20_alpha100=25}}
    \caption{Power as a function of the difference of the medians $\Delta$ for $n_1=n_2=20$, $\gamma=0.1$ (left) and $\gamma=0.25$ (right).}
\label{fig:2sim}
\end{figure}

\begin{figure}[htb]
  \centerline{\includegraphics[height=4.5cm]{Correctness_n1_20_n2_10_alpha100=10_Varied_S}}
    \caption{Error as a function of $s$ for $n_1=n_2=20$, $\gamma=0.1$ and $\Delta=0$.}
\label{fig:2simb}
\end{figure}
% 
% \subsection{Different distributions}
% For the case  $n_1=n_2=20$ and $\gamma=0.05$,
% we have performed new simulations changing the distributions of $X,Y$:
% $$
% X\sim St(0,2),~~Y\sim St(\Delta,2)|N(\Delta,1);
% $$
% in the first case, $X,Y$ are generated from two t-Student distributions (with $2$ degrees of freedom) and in the second
% $X$ is from a t-Student distribution and $Y$ from a Normal.
% Figure \ref{fig:3sim} shows the results for these cases.
% When IDP is determinate, the power of MWW coincides with that of IDP,
% while when IDP is indeterminate MWW returns a random response.
% In fact, it can be noticed that the power of the MWW and 50/50 tests are very close also in this case.
% The MWW is slightly better for $\Delta\geq 0.9$ and slightly worse for $\Delta\leq 0.3$.
% This confirms that, also  for non-Normal distributions, the IDP test is able to
% separate the cases  where MWW is issuing a random answer (with 50/50 chance).
% 
% % Table \ref{tab:3sim} reports the results for the case $\Delta=0$.
% % Observe that in the t-Student, Normal case and  $\Delta=0$, although the two distributions satisfy
% % $P(X\leq Y)=0.5$ (they are symmetric with  the same median),
% % the null hypothesis of the MWW test is violated (two different distributions).
% % This means that we are  in a case where $P(X\leq Y)=0.5$  (and thus we cannot say that ``Y  is better than X'')
% % but, since the two distributions are different, the MWW test may reject the null hypothesis.
% % This explains why the accuracy of the MWW test is less than $0.95$ in this case: the test is not calibrated.
% % The difference between IDP test and MWW test in this setting can be better understood from Figure \ref{fig:4sim}, that reports the error
% % in detecting the hypothesis $P(X\leq Y)=0.5$ as a function of $n_1=n_2=n$, for $\gamma=0.05$ and
% % $$
% % X\sim N(0,1),~~Y\sim N(0,100).
% % $$
% % Here, we are considering two Normals with different variance but equal mean.
% % Since the distributions are different, the null hypothesis of the MWW test does not hold but  $P(X\leq Y)=0.5$.
% % From Figure \ref{fig:4sim} it can be observed that the error  of the 50/50 test converges  to  $0.05$ and the indeterminacy of IDP
% % goes to zero at the increase of $n$. This is correct because $P(X\leq Y)=0.5$ and, thus, there is not difference
% % between $X$ and $Y$ (IDP returns a correct answer and   its error is $0.05$).
% % Conversely, the MWW test returns the response  ``Y is greater than X'' in  $8\%$ of the cases and this value does not change with $n$ which means that:
% % (i)  MWW has lower accuracy than IDP in detecting that  $P(X\leq Y)=0.5$; (ii) it is not able to detect  that the two distributions are different
% % with confidence $0.95$. This shows that IDP is a better test for  ``Y is greater than X'' than MWW in this setting.
% % Furthermore, this also shows that IDP is more robust to difference in distributions.
% % Notice that, in this case, since the MWW is not consistent, the IDP can return responses different from those of the MWW even when determinate.
% % This means that 
% 
% 
% \begin{figure}[htb]
%  \centerline{\includegraphics[height=4.5cm]{Correctness_n1_20_n2_20_alpha100=5_student_student}\includegraphics[height=4.5cm]{Correctness_n1_20_n2_20_alpha100=5_norm_student}}
%     \caption{Accuracy as a function of the difference of the medians $\Delta$ for t-Student,t-Student (left) and t-Student,Normal (right) with $n_1=n_2=20$ and $\gamma=0.05$.}
% \label{fig:3sim}
% \end{figure}

% \begin{table}
% \centering
%  \begin{tabular}{|c|c|c|c}
% \hline
%  Accuracy &  t-Student,t-Student & t-Student,Normal \\
% \hline
% \hline
% MWW & 0.9510 & 0.9490\\
% 50/50 test & 0.9452 & 0.9461\\
% \hline
% IDP when determinate & 0.9222 & 0.9256\\
% IDP Indeterminacy & 0.0483 & 0.0430\\
% \hline
%  \end{tabular} 
% \caption{Accuracy for  $\Delta=0$ and $\gamma=0.05$.}
% \label{tab:3sim}
% \end{table} 






% \subsection{Comparison with t-test}
% We have repeated  the  simulations for
% $$
% X\sim N(0,1),~~Y\sim N(\Delta,1),
% $$
% this time comparing the parametric t-test versus the IP-Dp.
% Figure \ref{fig:5sim} shows the results for  $n_1=n_2=20$ and $n_1=n_2=30$.
% The t-test performs better than the 50/50 test for $\Delta>0.5$.
% It is in fact known that in this setting the t-test has greater accuracy than the MWW (that, as we have seen in Figure \ref{fig:1sim}, coincides with the 50/50 test in this case). However, the performance is not very different thus showing that even when the hypothesis of normality is respected, the choice of the nonparametric approach do not implies a great loss of power compared to the parametric t-test. 
% 
% \begin{figure}[htb]
%  \centerline{\includegraphics[height=4.5cm]{Correctness_n1_20_n2_20_alpha100=5_ttest}\includegraphics[height=4.5cm]{Correctness_n1_30_n2_30_alpha100=5_ttest}}
%     \caption{Accuracy as a function of the difference of the medians $\Delta$ for   $\gamma=0.05$, $n_1=n_2=20$ (left) and $n_1=n_2=30$ (right).}
% \label{fig:5sim}
% \end{figure}


% 
% We consider Driver results for all NASCAR races in 1977. The dataset includes the start/finish positions, prize winnings, car manufacturers and laps completed for all participants.
% The aim of the analysis is to compare the car manufacturers with respect to the prize winnings in dollars, for details about the dataset see  \citet{winner2006nascar}.
% The following table reports  a 
% \begin{table}
% \centering
%  \begin{tabular}{|c|c|c|c|c|}
%  \hline
%    & Mercury & Matador & Ford & Dodge \\
%    \hline
%  prize & 5900 & 2358 &   3378 & 3920\\ 
%   races & 36 & 33 & 120 & 160\\
%   \hline
%  \end{tabular} 
% \caption{}
% \end{table} 
% 
% The result of the  IP-Dp test for the comparison Ford vs.Dodge, Matador vs. Dodge,  Mercury vs. Dodge are respectively shown  in Figures \ref{}, while 
% 
% p-value for the one-sided test   0.2284,  0.9888,    0.0247, 0.0884
% 
% 
% 
% % \begin{figure}[h]
% %         \centering
% %         
% %          \begin{tabular}{cc}
% %                      \includegraphics[width=0.35\textwidth]{Ford_vs_Dodge} &  \includegraphics[width=0.35\textwidth]{Maercury_vs_Dodge}\\
% %                       (a) Ford vs. Dodge &  (b) Mercury vs. Dodge\\
% %                       \includegraphics[width=0.35\textwidth]{Matador_vs_Dodge} & \includegraphics[width=0.35\textwidth]{Matador_vs_Ford}\\
% %                       (c) Matador vs. Dodge &  (d) Matador vs. Ford\\
% %          \end{tabular}
% % 
% %         \caption{Four possible results of the hypothesis test. The dark and light filled areas correspond respectively to the lower and upper probabilities of 
% % the event  $P(X\leq Y)>0.5$. The numerical values of these lower and upper probabilities are also reported in the figures. }\label{fig:1}
% % \end{figure}

\section{Conclusions}
In this paper we have proposed a model of prior ignorance for nonparametric inference based on the Dirichlet process (DP), by extending the approach proposed by \citet{pericchi_walley_91} and based on the use of sets of prior distributions. 
We developed a prior near-ignorance DP model (IDP) for inference about a variable $X$ by fixing the prior strength of the DP and  letting the normalized probability measure vary in the set of all distributions. We have proved that the IDP is consistent and a-priori vacuous for all predictive inferences that can be defined as the expectation of a real-valued bounded function of $X$. 
The proposed IDP model has two main merits. First, it removes the need for specifying the infinite-dimensional parameter of the DP (only an upper bound for the strength $s$ of the DP must be assumed a-priori), thus making the elicitation of the prior very easy. Second, it allows computing the posterior inferences for which no closed form expression exists, by a simple Monte Carlo sampling 
from the Dirichlet distribution, thus avoiding more demanding sampling approaches typically used for the DP (e.g., stick breaking). 
Based on this new prior near-ignorance model, we have proposed a general, simple and conservative approach to Bayesian nonparametric tests, and in particular we have developed a robust Bayesian alternative to the Mann-Whitney-Wilcoxon test: the IDP rank-sum test.
We have shown that our test is asymptotically consistent, while this is not always the case for the  Mann-Whitney-Wilcoxon test.
Finally, by means of numerical simulations, we have compared the IDP rank-sum test to the Mann-Whitney-Wilcoxon test and the Bayesian test obtained from the DP when the prior strength goes to zero. Results have shown that the IDP test is more robust, in the sense that it is able to isolate instances in which these tests are almost guessing at random.
% test outperforms both the frequentist Mann-Whitney-Wilcoxon test and  the Bayesian test obtained as a limit of DP when the prior strength goes to zero.
Given these interesting  results, as future work we plan to use this approach to implement Bayesian versions of the most used frequentist nonparametric tests. % such as, for instance, sign test, Wilcoxon signed-rank test, test for censored data, test for bivariate dependence etc.
In the long run, our aim is to build a statistical package for Bayesian nonparametric tests.


\section{Appendix}
{\it Proof of Theorem \ref{th:PI}:}  From (\ref{eq:expf}) assuming that $P\sim Dp(s,\alpha^*)$ one has that ${\mathcal{E}}[E(f)]=\int f d\alpha^*$.
Define $x_l= \arg\inf_{x \in \mathbb{X}} f(x)$ and $x_u= \arg\sup_{x \in \mathbb{X}} f(x)$, then (\ref{eq:vac1}) 
% it is enough to consider the DPs $Dp(s,\delta_{x_l})$ and $Dp(s,\delta_{x_u})$ belonging to the class (\ref{eq:IDP}), where
% $x_l= \arg\inf_{x \in \mathbb{X}} f(x)$ and $x_u= \arg\sup_{x \in \mathbb{X}} f(x)$:
follows by:
{\small
$$
\underline{\mathcal{E}}[E(f)]=\inf_{\alpha^* \in \mathbb{P}} \int f d\alpha^*= \int f d\delta_{x_l}=f(x_l),~~\overline{\mathcal{E}}[E(f)]=\sup_{\alpha^* \in \mathbb{P}} \int f d\alpha^*= \int f d\delta_{x_u}=f(x_u),
$$
}
which are the infimum and supremum of $f$ by definition. The lower and upper bounds are thus obtained by the following degenerate DPs $Dp(s,\delta_{x_l})$  and $Dp(s,\delta_{x_u})$, which belong to the class (\ref{eq:IDP}).
% In case $x_l$ (similar for $x_u$) does not belong to $\mathbb{X}$, the lower bound is not obtained by a DP with atomic base measure, but 
% $Dp(s,\delta_{x_l})$
In case $x_l$ is equal to $\infty$ (or $-\infty$), with $f(x_l)$ we mean $\lim_{x_l \rightarrow  \infty} f(x_l)$,  similar for the upper.\\
\noindent
{\it Proof of Theorem \ref{th:DPpost}:}  By exploiting the fact that  ${\mathcal{E}}[E(f)|X_1,\dots,X_n]=\int f ~d(\tfrac{s}{s+n}\alpha^*+\tfrac{n}{s+n}\frac{1}{n}\sum_{i=1}^n \delta_{X_i})$, the proof is similar to that of Theorem \ref{th:PI} (the lower and upper bounds are again obtained by degenerate DPs $Dp(s,\delta_{x_l})$ and $Dp(s,\delta_{x_u})$).\\
\noindent
{\it Proof of  Lemma \ref{ml:1}:} It follows from the properties (a) and (c) of the DP in Section \ref{sec:DP}.\\
% definition of Dirichlet process and the discreteness of the support of $G_{n_i}$, which imply the discreteness of the support of $F_{n_i}$ and the Dirichlet distribution for the vector of probabilities $(P(\{Z_{0}\}),P(\{Z_{1}\}),\dots, P(\{Z_{n_1}\}))$.\\
\noindent
{\it Proof of Theorem \ref{th:1}:} 
Based on the stick-breaking construction, a sample $F_0$ from the generic DP $Dp(s,G_0)$ can be written as
$F_0(x)=\sum\limits_{k=1}^{\infty} \pi_k \delta_{\tilde{X}_k}$ 
where $\pi_k=\beta_k\prod\limits_{i-1}^{k-1}(1-\beta_i)$, $ \beta_k \sim Beta(1,s)$, and $ X_k \sim G_0$. 
Then, using  (\ref{eq:mixing}), we have that
%Also, given the independent random variables $(w_0,w_1,\dots,w_n)\sim Dir(s,\stackrel{n_i}{\overbrace{1,\dots,1}})$ and distributions $P_0\sim Dp(s,G_0)$,$P_i\sim Dp(1,\delta_{X_i}$, the mixture
%$$
%\sum\limits_{i=0}^n w_i P_i
%$$
%has a DP distribution with base measure $s G_0 + \sum_{i=1}^n \delta_{X_i}$. Then, a sample from the posterior DP given the observations $X_1,\dots,X_n$ can be written as
\begin{equation} \label{eq:stick}
F_{n}(x)= \sum\limits_{i=1}^{n} w_i \delta_{X_i} + w_{0}\sum\limits_{k=1}^{\infty} \pi_{k} \delta_{\tilde{X}_k},
\end{equation}
where $(w_0,w_1,\dots,w_n)\sim Dir(s,\stackrel{n_i}{\overbrace{1,\dots,1}})$. 
Consider the two samples $F_X(x)$ and $F_Y(y)$ from the posterior distributions of $X$ and $Y$ given the generic DP priors $Dp(s,G_{10})$ and $Dp(s,G_{20})$.
%they can be written as
%\begin{equation} \label{eq:stick}
%\begin{array} {ll}
%F_{n_1}(x)=& \sum\limits_{i=1}^{n_1} w_{1i} \delta_{X_i} + w_{10}\sum\limits_{i=n_1+1}^{\infty} \pi_{1i} \delta_{\tilde{X}_i}\\
%F_{n_2}(y)=& \sum\limits_{j=1}^{n_2} w_{2j} \delta_{Y_j} + w_{20}\sum\limits_{j=n_2+1}^{\infty} \pi_{2j} \delta_{\tilde{X}_j}\\
%\end{array}
%\end{equation}
The probability of $P(X\leq Y)>c$ is $\mathcal{P}\big[P(X\leq Y)>c \big] = \mathcal{P}[\int F_{n_1}(y) d F_{n_2}(y)>c ]$.
Then, the posterior lower probability of $P(X\leq Y)> c$ is obtained by minimizing $\int F_{n_1}(y) d F_{n_2}(y)$, which, by  (\ref{eq:stick}), is equal to
\begin{equation} \label{eq:postP}
{\small
\begin{array}{ll}
~&\int \left(\sum\limits_{i=1}^{n_1} w_{1i} I_{(X_i,\infty)}(y)+w_{10} \sum\limits_{k=1}^{\infty} \pi_{1k} I_{(\tilde{X}_k,\infty)}(y)\right)
             \left(\sum\limits_{j=1}^{n_2} w_{2j} \delta_{Y_j}(y) + w_{20} \sum\limits_{l=1}^{\infty} \pi_{2l} \delta_{\tilde{Y}_l}(y)\right)dy\\
=&\sum\limits_{i=1}^{n_1}\sum\limits_{j=1}^{n_2} w_{1i} w_{2j} I_{(X_i,\infty)}(Y_j)
                   +w_{20} \sum\limits_{i=1}^{n_1}\sum\limits_{l=1}^{\infty} w_{1i} \pi_{2l} I_{(X_i,\infty)}(\tilde{Y}_l)\\
                  +&w_{10} \sum\limits_{k=1}^{\infty} \sum\limits_{j=1}^{n_2} \pi_{1k} w_{2j} I_{(\tilde{X}_k,\infty)}(Y_j)
									+w_{10}w_{20} \sum\limits_{k=1}^{\infty}  \sum\limits_{l=1}^{\infty} \pi_{1k} \pi_{2l} I_{(\tilde{X}_k,\infty)}(\tilde{Y}_{l})
\end{array}
}
\end{equation}
The minimum of $\int F_{n_1}(y) d F_{n_2}(y)$ is always found in correspondence of prior DPs such that the posterior probability of sampling $\tilde{X}_k<Y_j,\tilde{Y}_{l}$ or $\tilde{Y}_{l}>X_i$ is zero, so that only the term $\sum\limits_{i=1}^{n_1}\sum\limits_{j=1}^{n_2} w_{1i} w_{2j} I_{(X_i,\infty)}(Y_j)$ remains in  (\ref{eq:postP}). Priors of such kind are, for example, the extreme DP priors that give the posterior lower mean in (\ref{eq:ranktest}) and the posterior Dirichlet process $F_{n_i}$ with base probability $G_{n_i}$ given by (\ref{eq:basePost}).
%First, observe that the posterior lower probability of $P(X\leq Y)> a$ is obtained in correspondence of the extreme DP prior that gives the posterior lower mean in (\ref{eq:ranktest}) and the posterior Dirichlet process $F_{n_i}$ with base probability $G_{n_i}$ given by (\ref{eq:basePost}). Then the lower probability of $P(X\leq Y)>a$ is:
%$$
%\mathcal{P}\big[P(X\leq Y)>a|X^{n_1},Y^{n_2}\big] = \mathcal{P}\left[\displaystyle{\int F_{n_1}(y) d F_{n_2}(y)>a}\right],
%$$
%which, by Lemma \ref{ml:1}, is equal to
%$$
%\begin{array}{ll}
%~&\mathcal{P}\left[\displaystyle{\int \left(w_{10}I_{(X_0,\infty)}(y)+\sum_{j=1}^{n_1} w_{1j} I_{(X_j,\infty)}(y)\right)\left(w_{20}\delta_{Y_0}(y)+\sum_{j=1}^{n_2} w_{2j} \delta_{Y_j}(y)\right)dy}>a\right]\\
%=&\mathcal{P}\left[\sum\limits_{j=1}^{n_1}\sum\limits_{k=1}^{n_2} w_{1j}w_{2k} I_{(X_j,\infty)}(Y_k)>a\right].
%\end{array}
%$$
%
From the property of the Dirichlet distribution, we know that $E[w_{ij}]=1/(s+n_i)$ and, thus, we can rewrite the lower expectation given in the first equation of (\ref{eq:ranktest}) as
$$
\mu= \sum\limits_{j=1}^{n_1}\sum\limits_{k=1}^{n_2} \frac{1}{s+n_1} \frac{1}{s+n_2} I_{(X_j,\infty)}(Y_k)=E_W[W]^TAE_V[V],
$$
For the variance, we have that $\sigma^2=E[ (\sum\limits_{j=1}^{n_1}\sum\limits_{k=1}^{n_2}w_{1j}w_{2k} I_{(X_j,\infty)}(Y_k) )^2]-\mu^2$. Thus, by exploiting the equality
$$
\left(\sum\limits_{j=1}^{n_1}\sum\limits_{k=1}^{n_2}w_{1j}w_{2k}  I_{(X_j,\infty)}(Y_k) \right)^2=W^TAVW^TAV=V^TA^TWW^TAV,
$$
the linearity of expectation and the independence of $W,V$, one obtains
$$
E[V^TA^TWW^TAV]=E_V[V^TA^TE_W[WW^T]AV]=E_V[V^TA^TE_W[WW^T]AV].
$$
Since the result of this product is a scalar, it is equal to its trace and thus we can use the cyclic property  
$trace [E_V[V^TA^TE_W[WW^T]AV]] = trace[A^TE_W[WW^T]AE_V[VV^T]]$, and finally obtain
$\sigma^2= trace[A^TE_W[WW^T]AE_V[VV^T]]-\mu^2$. The proof is easily completed by deriving $E_W[WW^T]$ and $E_V[VV^T]$ from the fact that $w_{ij},w_{kl}$ are independent and
$E_W[w^2_{ij}]=\tfrac{2}{(s+n_i)(s+n_i+1)}$, $E_W[w_{ij}w_{il}]=\tfrac{1}{(s+n_i)(s+n_i+1)}$.\\
% In matrix form, the variance can be rewritten as
% $$
% \begin{array}{rcl}
% \sigma^2&=& trace\left( A^TE_W[WW^T]AE_V[VV^T]- A^TE[W]E[W]^TAE[V]E[V]^T \right),\\
% &=& tr\left( A^TE_W[WW^T]AE_V[VV^T]- A^TE[W]E[W]^TAE[V]E[V]^T \right),\\
% \end{array}
% $$
\noindent
{\it Proof  of Corollary \ref{co:1}:} 
First, observe that the posterior upper probability of $P(X\leq Y)> c$ is obtained in correspondence of the extreme DP prior that gives the posterior upper mean in (\ref{eq:ranktest}) and has base probability $dG_{n_i}$ given by (\ref{eq:basePost}). The probability of $X\leq Y$ for a given realization $F_{n_1}$ of $Dp(s,G_{n_1})$, and $F_{n_2}$ of $Dp(s,G_{n_2})$ is: 
$$
\begin{array}{ll}
~&\mathcal{P}\big[P(X\leq Y)>c|X^{n_1},Y^{n_2}\big] = \mathcal{P}\left[\displaystyle{\int F_{n_1}(y) d F_{n_2}(y)}>c\right] \vspace{2mm}\\
=&\mathcal{P}\left[\displaystyle{\int \left(w_{10}I_{(X_0,\infty)}(y)+\sum_{j=1}^{n_1} w_{1j} I_{(X_j,\infty)}(y)\right)\left(w_{20}\delta_{Y_0}(y)+\sum_{j=1}^{n_2} w_{2j} \delta_{Y_j}(y)\right)dy}>c\right]\\
=&\mathcal{P} \left[ w_{10}w_{20}+w_{10}\sum\limits_{j=1}^{n_2} w_{2j} +w_{20}\sum\limits_{j=1}^{n_1} w_{1j} +\sum\limits_{j=1}^{n_1}\sum\limits_{k=1}^{n_2} w_{1j}w_{2k} I_{(X_j,\infty)}(Y_k)>c\right].
\end{array}
$$
The computations are similar to those in Theorem \ref{th:1}, but in this case we must also consider 
the expectations $E_W[w^2_{i0}]=s(s+1)/(s+n_i)(s+n_i+1)$, $E_W[w_{i0}w_{ij}]=s/(s+n_i)(s+n_i+1)$ for $j>0$.\\
\noindent
{\it Proof of Theorem \ref{th:2}:} 
Our goal is to prove the convergence to a normal distribution of the Bayesian bootstrapped two-sample statistic 
$U_{DP} = \sum\limits_{i,j} w_{1i}w_{2j}I_{[X_i,\infty)}(Y_j)$, which implies the asymptotic normality of the DP rank sum test lower distribution, since the contribution of the prior $G_0$ vanishes asymptotically.
The asymptotic normality of $U_{DP}$ can be proved by means of Lemma 6.1.3. of \citet{Lehmann1998}, which states that given a sequence of random variables $T_n$, the distributions of which tend to a limit distribution $L$, the distribution of another sequence $T_n^*$ satisfying $E[(T_n^*-T_n)^2]\rightarrow 0$ also tends to $L$.  
Said $h(x,y) = I_{[x,\infty)}(y)$, $h_1(x) = E_Y[h(x,Y)]$ and  $h_2(y) = E_X[h(X,y)]$, the theorem will be proved by applying the lemma to 
$$
T_n = \sqrt{n}\left[ \frac{1}{n}(\sum_{i=1}^n h_1(X_i) - \theta) +\frac{1}{n}(\sum_{j=1}^n h_2(Y_j)-\theta)\right]
$$
and $T_n^* = \sqrt{n}(U_{DP}-\theta)$ where $\theta = E[U_{ij}] =  E[h_1(X)] = E[h_2(Y)] $.
$T_n$ is a sum of independent terms and thus, from the central limit theorem, it converges to a Gaussian distribution with mean $0$ and variance $\sigma^2=\sigma^2_1+\sigma^2_2$, where $\sigma^2_1=Var[h_1(X)]$ and $ \sigma^2_2=Var[h_2(Y)])$. Note that 
$$
\begin{array} {l}
\sigma_1^2 = Cov[h(X,Y),h(X,Y')] = Cov[U_{ij}, U_{i,k\neq j}],\\
\sigma_2^2 = Cov[h(X,Y),h(X',Y)] = Cov[U_{ij}, U_{k\neq i,j}].\\
\end{array}
$$
From Theorem \ref{th:1}, the mean of the lower distribution of $U_{DP}$ is $ \mu_l=E_W[W]^TAE_V[V]=\tfrac{U}{(s+n)^2}$, 
and thus, for large $n$, it is asymptotic to $U/n^2$ which converges, in turn, to $E[U_{ij}] = \theta$.%, see \citet[Appendix A.5]{Lehmann}
Then, also $E[T_n^*] = 0$ so that 
$$
E[(T_n^*-T_n)^2] = Var[T_n^*]+Var[T_n]-2 Cov[T_n^*,T_n].
$$
The proof will be completed by showing that $Var[T_n^*] \rightarrow \sigma^2$ and $Cov[T_n^*,T_n] \rightarrow \sigma^2$.
% $$
% \begin{array}{cc}
% Var[T_n^*] \rightarrow \sigma^2, & Cov[T_n^*,T_n] \rightarrow \sigma^2
% \end{array}
% $$
For the variance of $U_{DP}$ (\ref{eq:lowmeanvar}), first note that we can rewrite
$E_W[WW^T]=E_V[VV^T]=(D+J_n)\tfrac{1}{(s+n)(s+n+1)}$
where $D$ is the diagonal matrix of ones (identity matrix) and $J_n$ is the $n \times n$ matrix of ones.
Thus, we have that
$$
A^TE_W[WW^T]AE_V[VV^T]=A^T(D+J_n)A(D+J_n)\frac{1}{(s+n)^2(s+n+1)^2},
$$
and, for large $n$,
$$
\begin{array}{l}
\frac{trace(A^T(D+J_n)A(D+J_n))}{(s+n)^2(s+n+1)^2}\rightarrow \frac{trace(A^TA) +trace(A^TAJ_n)+trace(A^TJ_nA)+trace(A^TJ_nAJ_n)}{n^2(n+1)^2}.
\end{array}$$
The above sum has four terms at the numerator:
{\small
$$
\begin{array}{lcl}
trace(A^TA)&=&\sum_{i,j} a_{ij}^2=\sum_{i,j} I_{(X_i,\infty)}(Y_j),\\
trace(A^TAJ_n)&=&\sum_{i,j} a_{ij} \sum_{k} a_{ik}=\sum_{i,j} I_{(X_i,\infty)}(Y_j)+\sum_{i,j\neq k} I_{(X_i,\infty)}(Y_j)I_{(X_i,\infty)}(Y_k),\\
trace(A^TJ_nA)&=&\sum_{i,j} a_{ij} \sum_{k} a_{kj}=\sum_{i,j} I_{(X_i,\infty)}(Y_j)+\sum_{i\neq k,j} I_{(X_i,\infty)}(Y_j)I_{(X_k,\infty)}(Y_j),\end{array}
$$
}
and $
trace(A^TJ_nAJ_n)=trace(A^T\mathbbm{1}\mathbbm{1}^TA\mathbbm{1}\mathbbm{1}^T)=trace(\mathbbm{1}^TA\mathbbm{1}\mathbbm{1}^TA^T\mathbbm{1})=U^2,
$
where $\mathbbm{1}$ is the unit vector.
% Following \citet[Example 5, Appendix A.1]{Lehmann}, let us introduce 
% $$
% p_1=P(X\leq Y), ~~p_2=P(X\leq Y,X\leq Y'),~~p_3=P(X\leq Y,X'\leq Y),
% $$ 
Then we have that
$$
\begin{array}{l}
\sigma_l^2=
\frac{3\sum_{i,j} I^2_{(X_i,\infty)}(Y_j)-3n^2\mu_l^2 }{n^2(n+1)^2}
+\frac{\sum_{i,j\neq k} I_{(X_i,\infty)}(Y_j)I_{(X_i,\infty)}(Y_k)-n^2(n-1)\mu_l^2}{n^2(n+1)^2}\\
+\frac{\sum_{i\neq k,j} I_{(X_i,\infty)}(Y_j)I_{(X_k,\infty)}(Y_j)-n^2(n-1)\mu_l^2}{n^2(n+1)^2}
+\frac{3n^2+2n^2(n-1)+n^4}{n^2(n+1)^2}\mu_l^2-\mu_l^2.
\end{array}
$$
Note that $(\tfrac{3n^2+2n^2(n-1)+n^4}{n^2(n+1)^2}-1)\mu_l^2=0$ and, since the first term in $\sigma_l^2$ goes to zero as $1/n^2$, for large $n$,
$$
\begin{array}{l}
\sigma_l^2 \rightarrow 
% \frac{3\sum_{i,j} I^2_{(X_i,\infty)}(Y_j)-3n^2\mu_l^2 }{n^2(n+1)^2}\\
\frac{\sum_{i,j\neq k} I_{(X_i,\infty)}(Y_j)I_{(X_i,\infty)}(Y_k)-n^2(n-1)\mu_l^2}{n^2(n+1)^2}
+\frac{\sum_{i\neq k,j} I_{(X_i,\infty)}(Y_j)I_{(X_k,\infty)}(Y_j)-n^2(n-1)\mu_l^2}{n^2(n+1)^2}.
% +\frac{3n^2+2n^2(n-1)+n^4}{n^2(n+1)^2}\mu_l^2-\mu_l^2.
\end{array}
$$
For large $n$, it can be shown  \citet[Th. 9]{Lehmann} that the right-hand side of the above equations tends to
$\frac{1}{n}Cov[U_{ij}, U_{i,k\neq j}] +\frac{1}{n}Cov[U_{ij},U_{k\neq i,j}] = \frac{1}{n}\sigma^2$, and thus $Var[T_n]=Var[\sqrt{n}U_{DP}] \rightarrow \sigma^2$.
For the covariance we have
{\small
$$
\begin{array}{l}
Cov[T_n,T^*_n] = \left(E[U_{DP}\sum\limits_{i=1}^n h_1(X_i)]+E[U_{DP} \sum\limits_{j=1}^n h_2(Y_j)] -2\theta \right)\\
~= \left(E[\sum\limits_{i,j} w_{1i}w_{2j} E_{Y_j}[h(X_i,Y_j)]\sum\limits_{i=1}^n h_1(X_i)]+E[\sum\limits_{i,j} w_{1i}w_{2j} E_{X_i}[h(X_i,Y_j)] \sum\limits_{j=1}^n h_2(Y_j)] -2\theta \right)\\
~= \left(E_{X}[\sum\limits_{i=1}^n E[w_{1i}] h_1(X_i) \sum\limits_{i=1}^n h_1(X_i)]+E_{Y}[\sum\limits_{j=1}^n E[w_{2j}] h_2(Y_j) \sum\limits_{j=1}^n h_2(Y_j)] -2\theta \right)\\
~= \frac{1}{n} \left((\sum\limits_{i=1}^n E_{X}[h_1(X_i))^2]+(\sum\limits_{j=1}^nE_{Y}[ h_2(Y_j))^2] -2\theta \right)= Var[h_1(X)] + Var[h_2(Y)] = \sigma^2 .
\end{array}
$$}



%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section*{Acknowledgements} This work  has been partially supported by the Swiss NSF grant
n.~200020-137680/1.




%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% \noindent{\large\bf References}
% \begin{description}
% \item
% Andrews, D. W. K. (1984). Non-strong mixing autoregressive processes.
%   {\it J. Appl. Probab.} {\bf 21}, 930-934.
% \item
% Avram, F. and Taqqu, M. S. (1987)  Noncentral limit theorems and Appell
% polynomials. {\it Ann. Probab.} {\bf 15}, 767-775.
% \item
% Bradley, R. C. (1986). Basic properties of strong mixing conditions. In {\it
% Dependence
% in Probability and Statistics} (Edited by E. Eberlein and M. S. Taqqu),
% 162-192. Birkh\"auser, Boston.
% \item
% Fox, R. and Taqqu, M. (1987). Central limit theorems for quadratic forms in
% random
% variables having long-range dependence. {\it Probab. Theory Related Fields}
% {\bf 74}, 213-240.
% \end{description}

 \bibliographystyle{elsarticle-harv}
\bibliography{biblio}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% \vskip .65cm
% \noindent
% Istituto Dalle Molle di Studi sull'Intelligenza Artificiale (IDSIA) Manno, Switzerland
% \vskip 2pt
% \noindent
% E-mail: alessio@idsia.ch, francesca@idsia.ch, zaffalon@idsia.ch
% \vskip 2pt
% \noindent
% CNR IMATI, Milano, Italy
% \vskip 2pt
% \noindent
% E-mail: fabrizio@mi.imati.cnr.it
% \vskip .3cm

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\end{document}





IDP_ranksum_paper.bbl
\begin{thebibliography}{33}
\expandafter\ifx\csname natexlab\endcsname\relax\def\natexlab#1{#1}\fi
\expandafter\ifx\csname url\endcsname\relax
  \def\url#1{\texttt{#1}}\fi
\expandafter\ifx\csname urlprefix\endcsname\relax\def\urlprefix{URL }\fi

\bibitem[{Augustin and Coolen(2004)}]{Augustin2004251}
Augustin, T., Coolen, F., 2004. Nonparametric predictive inference and interval
  probability. Journal of Statistical Planning and Inference 124~(2), 251 --
  272.

\bibitem[{Benavoli and Zaffalon(2012)}]{Benavoli20121960}
Benavoli, A., Zaffalon, M., 2012. A model of prior ignorance for inferences in
  the one-parameter exponential family. Journal of Statistical Planning and
  Inference 142~(7), 1960 -- 1979.

\bibitem[{Berger(1994)}]{berger1994overview}
Berger, J.~O., 1994. {An overview of robust Bayesian analysis with discussion}.
  Test 3~(1), 5--124.

\bibitem[{Berger et~al.(2000)Berger, Rios~Insua, and Ruggeri}]{Ruggeri2000}
Berger, J.~O., Rios~Insua, D., Ruggeri, F., 2000. Bayesian robustness. In:
  Rios~Insua, D., Ruggeri, F. (Eds.), Robust Bayesian Analysis. Vol. 152 of
  Lecture Notes in Statistics. Springer New York, pp. 1--32.

\bibitem[{Bernard(2005)}]{Bernard2005123}
Bernard, J.-M., 2005. An introduction to the imprecise dirichlet model for
  multinomial data. International Journal of Approximate Reasoning 39~(2–3),
  123 -- 150, imprecise Probabilities and Their Applications.

\bibitem[{Blum and Susarla(1977)}]{Blum1977}
Blum, J., Susarla, V., 1977. On the posterior distribution of a dirichlet
  process given randomly right censored observations. Stochastic Processes and
  their Applications 5~(3), 207 -- 211.

\bibitem[{Borgwardt and Ghahramani(2009)}]{borgwardt2009bayesian}
Borgwardt, K.~M., Ghahramani, Z., 2009. Bayesian two-sample tests. arXiv
  preprint arXiv:0906.4032.

\bibitem[{Chen and Hanson(2014)}]{RePEc:eee:csdana}
Chen, Y., Hanson, T.~E., 2014. Bayesian nonparametric k-sample tests for
  censored and uncensored data. Computational Statistics and Data Analysis
  71~(C), 335--346.

\bibitem[{Coolen and Augustin(2009)}]{Coolen2009217}
Coolen, F., Augustin, T., 2009. A nonparametric predictive alternative to the
  imprecise dirichlet model: The case of a known number of categories.
  International Journal of Approximate Reasoning 50~(2), 217 -- 230.

\bibitem[{Dalal and Phadia(1983)}]{dalai1983nonparametric}
Dalal, S., Phadia, E., 1983. Nonparametric bayes inference for concordance in
  bivariate distributions. Commun. in Statistics-Theory and Methods 12~(8),
  947--963.

\bibitem[{DasGupta(2008)}]{dasgupta2008asymptotic}
DasGupta, A., 2008. Asymptotic theory of statistics and probability. Springer
  (NY).

\bibitem[{de~Cooman et~al.(2009)de~Cooman, Miranda, and
  Quaeghebeur}]{deCooman2009204}
de~Cooman, G., Miranda, E., Quaeghebeur, E., 2009. Representation insensitivity
  in immediate prediction under exchangeability. International Journal of
  Approximate Reasoning 50~(2), 204 -- 216.

\bibitem[{Efron(1979)}]{Efron1979}
Efron, B., 1979. {Bootstrap Methods: Another Look at the Jackknife}. The Annals
  of Statistics 7~(1), pp. 1--26.

\bibitem[{Fay and Proschan(2010)}]{Fay2010}
Fay, M.~P., Proschan, M.~A., 2010. {Wilcoxon-Mann-Whitney or t-test? On
  assumptions for hypothesis tests and multiple interpretations of decision
  rules}. Statistics Surveys 4, pp. 1--39.

\bibitem[{Ferguson(1973)}]{Ferguson1973}
Ferguson, T.~S., 1973. {A Bayesian Analysis of Some Nonparametric Problems}.
  The Annals of Statistics 1~(2), pp. 209--230.

\bibitem[{Ghosh and Ramamoorthi(2003)}]{ghosh2003bayesian}
Ghosh, J.~K., Ramamoorthi, R., 2003. {Bayesian nonparametrics}. Springer (NY).

\bibitem[{Hodges and Lehmann(1963)}]{hodges1963}
Hodges, J.~L., Lehmann, E.~L., 06 1963. Estimates of location based on rank
  tests. The Annals of Mathematical Statistics 34~(2), 598--611.

\bibitem[{Holmes et~al.(2009)Holmes, Caron, Griffin, and
  Stephens}]{holmes2009two}
Holmes, C., Caron, F., Griffin, J., Stephens, D.~A., 2009. {Two-sample Bayesian
  nonparametric hypothesis testing}. arXiv preprint arXiv:0910.5060.

\bibitem[{Jara et~al.(2011)Jara, Hanson, Quintana, M\"uller, and
  Rosner}]{Dppackage}
Jara, A., Hanson, T., Quintana, F., M\"uller, P., Rosner, G., 2011.
  {DPpackage}: Bayesian semi- and nonparametric modeling in {R}. Journal of
  Statistical Software 40~(5), 1--30.

\bibitem[{Lehmann(1998)}]{Lehmann1998}
Lehmann, E., 1998. {Elements of Large-Sample Theory}. Springer-Verlag New York.

\bibitem[{Lehmann and D'Abrera(1975)}]{Lehmann}
Lehmann, E., D'Abrera, H. J.~M., 1975. {Nonparametrics: statistical methods
  based on ranks}. McGraw-Hill,San Francisco.

\bibitem[{Lo(1987)}]{Lo1987}
Lo, A.~Y., 1987. {A Large Sample Study of the Bayesian Bootstrap}. The Annals
  of Statistics 15~(1).

\bibitem[{Ma and Wong(2011)}]{ma2011coupling}
Ma, L., Wong, W.~H., 2011. {Coupling optional P{\'o}lya trees and the two
  sample problem}. Journal of the American Statistical Association 106~(496).

\bibitem[{Pericchi and Walley(1991)}]{pericchi_walley_91}
Pericchi, L.~R., Walley, P., 1991. {Robust Bayesian credible intervals and
  prior ignorance}. International Statistical Review, 1--23.

\bibitem[{Phadia(2013)}]{Phadia2013}
Phadia, E.~G., 2013. Inference based on complete data. In: Prior Processes and
  Their Applications. Springer, New York, pp. 109--153.

\bibitem[{Rossi and McCulloch(2010)}]{rossi2010bayesm}
Rossi, P., McCulloch, R., 2010. Bayesm: Bayesian inference for
  marketing/micro-econometrics. R package version, 2--2.

\bibitem[{Rubin(1981)}]{Rubin1981}
Rubin, D.~B., 1981. {Bayesian Bootstrap}. The Annals of Statistics 9~(1), pp.
  130--134.

\bibitem[{Ruggeri(2010)}]{ruggeri1994}
Ruggeri, F., 1 2010. {Nonparametric Bayesian robustness}. Chilean Journal of
  Statistics 2, 51--68.

\bibitem[{Sethuraman and Tiwari(1981)}]{sethuraman1981}
Sethuraman, J., Tiwari, R.~C., 1981. {Convergence of Dirichlet measures and the
  interpretation of their parameter}. Defense Technical Information Center.

\bibitem[{Susarla and Van~Ryzin(1976)}]{SusarlaRyzin1976}
Susarla, V., Van~Ryzin, J., 1976. {Nonparametric Bayesian Estimation of
  Survival Curves from Incomplete Observations}. Journal of the American
  Statistical Association 71~(356), pp. 897--902.

\bibitem[{Walley(1991)}]{walley1991}
Walley, P., 1991. Statistical Reasoning with Imprecise Probabilities. Chapman
  and Hall, New York.

\bibitem[{Walley(1996)}]{walley1996}
Walley, P., 1996. {Inferences from multinomial data: learning about a bag of
  marbles}. Journal of the Royal Statistical Society. Series B (Methodological)
  58~(1), 3--57.

\bibitem[{Weng(1989)}]{Weng1989}
Weng, C.-S., 1989. {On a Second-Order Asymptotic Property of the Bayesian
  Bootstrap Mean}. The Annals of Statistics 17~(2), pp. 705--710.

\end{thebibliography}
