Weighted Adaptive Multiple Decision Functions for False Discovery Rate Control
\fnms{Joshua D.
Efforts to develop more efficient multiple hypothesis testing procedures for false discovery rate (FDR) control have focused on incorporating an estimate of the proportion of true null hypotheses (such procedures are called adaptive) or exploiting heterogeneity across tests via some optimal weighting scheme. This paper combines these approaches and provides an assessment of the resulting methodology. Specifically, weighted adaptive multiple decision functions (WAMDFs) that control the FDR when test statistics from true null hypotheses are independent and independent of test statistics from false null hypotheses are provided, and optimal weights for a random effects model are derived. Assessment reveals that, under a weak dependence structure, WAMDFs dominate procedures which are either adaptive or weighted but not both. In particular, they allow for more rejected null hypotheses and have false discovery proportion that is asymptotically less than or equal to the nominal level, even when the employed weights are only positively correlated with optimal weights. The method is demonstrated on a real data set.
Main-Revision-8-2014arxiv.tex
\documentclass[preprint]{imsart}
\usepackage{amsmath,epsfig, amsthm}
\usepackage{natbib}
\usepackage{color}
\usepackage{ulem}
\usepackage{verbatim}
\usepackage{graphicx}
\usepackage{rotating}
\usepackage{verbatim}
\usepackage{amssymb}
\usepackage{xr}
%\externaldocument{Supplement8-2014}
%\newcommand{\mat}[1]{\textrm{\textit{{\textbf{#1}}}}}
\newcommand{\mat}[1]{\text{\mbox{\boldmath$#1$}}}
\newcommand{\bm}[1]{\text{\mbox{\boldmath$#1$}}}
\newtheorem{model}{Model}
\newtheorem{definition}{Definition}
\newtheorem{theorem}{Theorem}
\newtheorem{condition}{Condition N}
\newtheorem{corollary}{Corollary}
\newtheorem{lemma}{Lemma}
\newtheorem{example}{Example}
\newtheorem{proposition}{Proposition}
\newtheorem*{wprocedure}{Weight Selection Procedure}
\newtheorem*{asymptotically optimal WAMDF}{Asymptotically Optimal wamdf}
\newtheorem{ThmProof}{Proof of Theorem}
\newtheorem{LemProof}{Proof of Lemma}
\newtheorem*{tproc}{Threshold Selection Procedure}
\newtheorem*{lemma2}{Lemma S2}
\begin{document}
\begin{frontmatter}

\title{Weighted Adaptive Multiple Decision Functions for False Discovery Rate Control}
\runtitle{Weighted Adaptive Multiple Decision Functions}
\begin{aug}
  \author{\fnms{Joshua D.}  \snm{Habiger}\corref{}\thanksref{t2}\ead[label=e1]{jhabige@okstate.edu}}
 \thankstext{t2}{Joshua Habiger is an Assistant Professor in the Department of Statistics at Oklahoma State University}
 \runauthor{J. Habiger}
  \affiliation{Oklahoma State University}
  \address{301G MSCS building, Stillwater, OK 74078}
\end{aug}

\begin{abstract}
Efforts to develop more efficient multiple hypothesis testing procedures for false discovery rate (FDR) control have focused on incorporating an estimate of the proportion of true null hypotheses (such procedures are called adaptive) or exploiting heterogeneity across tests via some optimal weighting scheme.  This paper combines these approaches and provides an assessment of the resulting methodology.  Specifically, weighted adaptive multiple decision functions (WAMDFs) that control the FDR when test statistics from true null hypotheses are independent and independent of test statistics from false null hypotheses are provided, and optimal weights for a random effects model are derived.  Assessment reveals that, under a weak dependence structure, WAMDFs dominate procedures which are either adaptive or weighted but not both.   In particular, they allow for more rejected null hypotheses and have false discovery proportion that is asymptotically less than or equal to the nominal level, even when the employed weights are only positively correlated with optimal weights.  The method is demonstrated on a real data set.
\end{abstract}


\begin{keyword}[class=MSC]
\kwd[Primary]{62F03}
\kwd[; secondary]{62J15}
\end{keyword}
\begin{keyword}
\kwd{Adaptive}
\kwd{False Discovery Proportion}
\kwd{False Discovery Rate}
\kwd{Heterogeneity}
\kwd{Multiple Decision Function}
\kwd{Weighted}
\kwd{Weighted $p$-Value}
\end{keyword}
\end{frontmatter}


\section{Introduction}\label{sec 1}

High throughput technology routinely generates data sets that allow for hundreds or thousands of null hypotheses to be tested simultaneously.  For example, in \cite{AndHab12}, RNA sequencing technology was used to measure the prevalence of bacteria living near the roots of wheat plants across $i = 1, 2, ..., 5$ treatment groups for each of $m = 1, 2, ..., M = 778$ bacteria, thereby facilitating the simultaneous testing of $778$ null hypotheses.  More specifically, denote the prevalence of bacteria $m$ in the $i$th treatment group by $Y_{im}$ and denote the mean of $Y_{im}$ by $\mu_{im}$.  Assume log$(\mu_{im}) = \beta_{0m} + \beta_{1m}x_i$, where $\beta_{0m}$ and $\beta_{1m}$ are regression parameters and $x_i$ is the total shoot biomass of wheat plants in the $i$th group.  One objective was to determine which bacteria are positively associated with shoot biomass via the testing of the null hypothesis $H_m:\beta_{1m} = 0$ against the alternative hypothesis $K_m:\beta_{1m}>0$ for each $m$.  See Table \ref{data} for a depiction of the data and see \cite{Efr08, DudVan08, Efr10} for other, sometimes called, high dimensional (HD) data sets.
%The standard approach is to first compute a testing $H_m:\beta_{1m} = 0$ can be based on the conditional distribution of the sufficient statistic $T_m = \sum_{i}x_iY_{im}$ given ancillary statistic $\sum_{i=1}^5Y_{im} = n_m$.  See, for example, \cite{McCNel89}.  Here, heterogeneity is attributable to the fact that ancillary statistics vary from test to test.
\begin{table}[t!]\caption{\label{data} Depiction of the data in \cite{AndHab12}.  Shoot biomass $x_i$ in grams for groups $i$ = 1, 2, ..., 5 was 0.86, 1.34, 1.81, 2.37, and 3.00, respectively. Row totals are in the last column. }
\begin{tabular}{ccccccc|c} \hline \hline
Bacteria $m$ && $Y_{1m}$ & $Y_{2m}$ & $Y_{3m}$ & $Y_{4m}$ & $Y_{5m}$ & Total ($n_m$)\\ \cline{1-1} \cline{3-8} \cline{3-8} \cline{1-1}
1 && 0 & 1 & 1 & 0 & 5 & 7\\
2 && 9 & 2 & 0 & 0 & 3 & 14\\
$\vdots$&&$\vdots$&$\vdots$&$\vdots$&$\vdots$&$\vdots$ & $\vdots$\\
778 && 16 & 10 & 29 & 18 & 13 & 81 \\
\hline
\end{tabular}
\end{table}

In general, multiple null hypotheses are simultaneously tested with a multiple testing procedure which, ideally, rejects as many null hypotheses as possible subject to the constraint that some global type 1 error rate is controlled at a prespecified level $\alpha$.  The false discovery rate (FDR) is the most frequently considered error rate in the HD setting.  It is loosely defined as the expected value of the false discovery proportion (FDP), where the FDP is the proportion of erroneously rejected null hypotheses, also called false discoveries, among rejected null hypotheses, or discoveries.  See \cite{Sar07} for other related error rates.  In their seminal paper, \cite{BenHoc95} showed that a step-up procedure based on the \cite{Sim86} line, henceforth referred to as the BH procedure, has FDR  $= \alpha a_0\leq \alpha$ under a certain dependence structure, where $a_0$ is the proportion of true null hypotheses.  Since then, much research has focused on developing more efficient procedures for FDR control.

One approach seeks to control the FDR at a level nearer $\alpha$, as opposed to $\alpha a_0$.  For example, adaptive procedures in \cite{BenHoc00, Sto04, BenKri06, Gav09, LiaNet12} utilize an estimate of $a_0$ and typically have FDR that is greater than $\alpha a_0$ yet still less than or equal to $\alpha$.  \cite{Fin09} proposed nonlinear procedures that ``exhaust the $\alpha$'' in that, loosely speaking, their FDR converges to $\alpha$ under some least favorable configuration as $M$ tends to infinity.

Another approach aims to exploit heterogeneity across hypothesis tests.  \cite{Gen06, BlaRoq08, RoqVan09, Pen11, Pen14} propose a weighted BH-type procedure, where weights are allowed to depend on the power functions of the individual tests or prior probabilities for the states of the null hypotheses.  \cite{SunCai09} and \cite{Hu10} focused on clustered data, where test statistics are heterogeneous across clusters but homogeneous within clusters, while \cite{SunMcL12} considered heteroscedastic standard errors.  Data in Table \ref{data}, for example, are heterogeneous because sample sizes $n_1, n_2, ..., n_M$ vary from test to test, with $n_m$ being as small as $6$ and as large as $911$.  Whatever the nature of the heterogeneity may be, recent literature suggests that it should not be  ignored. Specifically, \cite{RoeWas09} showed that weighted multiple testing procedures generally perform favorably over their unweighted counterparts, especially when the employed weights efficiently exploit heterogeneity. In fact, \cite{SunMcL12} illustrated that procedures which ignore heterogeneity are not only inefficient, but can produce lists of discoveries that are of little scientific interest.

This paper combines approaches for exhausting the $\alpha$ and exploiting heterogeneity and shows that the resulting methodology performs favorably over competing methods.  The general procedure is developed in Sections \ref{sec 2} - \ref{sec 5}.  Section \ref{sec 2} introduces a weighted adaptive multiple decision function (WAMDF) framework and a random effects model that can accommodate many types of heterogeneity, including, but not limited to, those mentioned above.  Additionally, tools which facilitate implementation of the proposed WAMDF, such as weighted $p$-values, are developed.  Section \ref{sec 3} derives optimal weights and explores their operating characteristics.  Section \ref{sec 4} introduces the ``asymptotically optimal'' WAMDF for asymptotic FDP control and Section \ref{sec 5} provides a WAMDF for exact (nonasymptotic) FDR control.

Asymptotic assessment is in Sections \ref{sec 6}.  It is shown that, under a weak dependence structure, WAMDFs dominate other MDFs, asymptotically, in terms of both their FDP and power. Specifically, they are less conservative in that, asymptotically, their FDP is always larger than the FDP of their unadaptive counterpart.  Further, sufficient conditions are provided under which WAMDFs provide asymptotic FDP control and exhaust the $\alpha$. It is shown that these conditions are satisfied in a variety of settings, such as when optimal weights are utilized, when weights are positively correlated with optimal weights, or even in a worst case scenario where the employed weights are independent of optimal weights.  As a corollary, we see that the (linear) unweighted adaptive procedure in \cite{Sto04} is $\alpha$-exhaustive and hence is competitive with (unweighted) nonlinear procedures in \cite{Fin09}.  The notion of ``asymptotically optimal'' is formalized as well.  Simulation studies in Section \ref{sec 7} demonstrate that WAMDFs are more powerful than competing MDFs even if the employed weights are only positively correlated with optimal weights.

In Section \ref{sec 8}, it is demonstrated via the analysis of the data in Table \ref{data}, that WAMDFs can be useful for exploiting heterogeneity in practical applications, even if parameters for the random effects model are not precisely known. Concluding remarks are in Section \ref{sec 9}.  Technical proofs and additional detail regarding the data analysis in Section \ref{sec 8} are in a Supplementary section.
\section{Background} \label{sec 2}
\subsection{Data}
Let $\mat{Z} = (Z_m, m\in\mathcal{M})$ for $\mathcal{M} = \{1, 2, ..., M\}$ be a random vector of test statistics with joint distribution function $F$ and let $\mathcal{F}$ be a model for $F$.  The basic goal is to test null hypotheses $\mat{H} = (H_m, m\in\mathcal{M})$ of the form $H_m:F\in\mathcal{F}_m$, where $\mathcal{F}_m\subseteq\mathcal{F}$ is a submodel for $\mathcal{F}$.  For short, we often denote the state of $H_m$ by $\theta_m = 1 - I(F\in\mathcal{F}_m)$, where $I(\cdot)$ is the indicator function, so that $\theta_m = 0 (1)$ means that $H_m$ is true(false), and denote the state of $\mat{H}$ by $\bm{\theta} = (\theta_m, m\in\mathcal{M})$. Let  $\mathcal{M}_0 = \{m\in\mathcal{M}:\theta_m =0\}$ and $\mathcal{M}_1 = \mathcal{M}\setminus\mathcal{M}_0$ index the set of true and false null hypotheses, respectively, and denote the number of true and false null hypotheses by $M_0 = |\mathcal{M}_0|$ and $M_1 = |\mathcal{M}_1|$, respectively.

To make matters concrete, we often consider the following random effects model for $\mat{Z}$.  For related models see \cite{EfrTib01, GenWas02, Sto03, Gen06, SunCai07,SunCai09, RoqVan09}.  In Model \ref{randeffmodel}, heterogeneity across the $Z_m$s is attributable to prior probabilities $\mat{p} = (p_m, m\in\mathcal{M})$ for the states of the $H_m$s and parameters $\bm{\gamma} = (\gamma_m, m\in\mathcal{M})$, which we refer to as effect sizes, although each $\gamma_m$ could merely index a distribution for $Z_m$ when $H_m$ is false.
%
\begin{model} \label{randeffmodel}
Let $(Z_m, \theta_m, p_m, \gamma_m), m\in\mathcal{M}$ be independent and identically distributed random vectors each with support in $\Re \times\{0,1\}\times [0,1]\times\Re^+$  and with conditional distribution functions
$$F(z_m| \theta_m, p_m, \gamma_m) = (1-\theta_m)F_0(z_m) + \theta_m F_1(z_m|\gamma_m)$$ and $$F(z_m|p_m,\gamma_m) = (1-p_m)F_0(z_m) + p_mF_1(z_m|\gamma_m).$$
Assume further that $F(\gamma_m,p_m) = F(\gamma_m)F(p_m)$, $Var(\gamma_m)<\infty$ and that $p_m$ has mean $1-a_0 \in (0,1)$.
\end{model}
%
Observe that $Z_m$ has distribution function $F_0(\cdot)$ given $H_m:\theta_m = 0$ and has distribution function $F_1(\cdot|\gamma_m)$ otherwise.  It is important to note that parameters $\bm{\theta}$, $\mat{p}$, and $\bm{\gamma}$ are assumed to be random variables in Model \ref{randeffmodel} primarily to facilitate asymptotic analysis, as in \cite{Gen06, BlaRoq08, BlaRoq09,RoqVan09,RoqVil11}. Under Model \ref{randeffmodel}, focus will be on conditional distribution functions $F(\mat{z}|\bm{\theta},\bm{p},\bm{\gamma}) = \prod_{m\in\mathcal{M}}F(z_m|\theta_m,p_m,\gamma_m)$ and $F(\mat{z}|\bm{p},\bm{\gamma}) = \prod_{m\in\mathcal{M}}F(z_m|p_m,\gamma_m)$, and an expectation taken over $\bm{Z}$ with respect to these distributions will be denoted $E[\cdot|\bm{\theta},\bm{p},\bm{\gamma}]$ and $E[\cdot|\bm{p},\bm{\gamma}]$, respectively. We shall write $E[\cdot|A]$ to denote the conditional expectation of $\mat{Z}$ given $(\bm{\theta}, \mat{p},\bm{\gamma})\in A$.  More generally, we write $E[\cdot]$ to denote an expectation taken with respect to some arbitrary $F\in\mathcal{F}$.

\subsection{Multiple decision functions}

A multiple decision function (MDF) framework is used in this paper.  For similar frameworks, see \cite{GenWas04, Sto04, SunCai07, Pen11, Pen14}.  Specifically, each null hypothesis will be tested with the decision function $\delta_m(Z_m;t_m)$ taking values in $\{0,1\}$ and depending on data $Z_m$ and possibly random size threshold $t_m\in[0,1]$, where $\delta_m = 1 (0)$ means that $H_m$ is rejected(retained).  An MDF is denoted by $\bm{\delta}(\mat{Z};\mat{t}) = [\delta_m(Z_m;t_m), m\in\mathcal{M}]$, where $\mat{t} = (t_m, m\in\mathcal{M})$.  For example, suppose that large values of $Z_m$ are evidence against $H_m:\theta_m = 0$ under Model \ref{randeffmodel}.  Then we may define
%
\begin{equation}\label{model1example} \delta_m(Z_m;t_m) = I(Z_m\geq F_0^{-1}(1-t_m)).\end{equation}
%
Observe that $E[\delta_m(Z_m;t_m) |\theta_m = 0] = 1 - F_0(F_0^{-1}(1-t_m)) = t_m$ so that $t_m$ indeed represents the size of $\delta_m$, hence the terminology ``size threshold''.  If $t_m = \alpha/M$ for each $m$, then $\bm{\delta}(\mat{Z};\mat{t})$ represents the well-known Bonferroni procedure.

Throughout this manuscript we assume that, for each $m$, $t_m\mapsto\delta_m(Z_m;t_m)$ is nondecreasing and right continuous with $\delta_m = 0 (1)$ whenever $t_m = 0 (1)$, almost surely, and that $t_m\mapsto E[\delta_m(Z_m;t_m)]$ is continuous and strictly increasing for $t_m\in(0,1)$ with $E[\delta_m(Z_m;t_m)]= t_m$ whenever $m\in\mathcal{M}_0$.  These assumptions are referred to as the nondecreasing-in-size (NS) assumptions and are satisfied, for example, under Model \ref{randeffmodel} for decision functions defined as in (\ref{model1example}).  For additional details and examples, see \cite{HabPen11, Pen11, Hab12}.  %It should be noted that $E[\delta_m(Z_m;t_m)]$ is not continuous in $t_m$ if $Z_m$ has a discrete distribution.  We therefore implicitly assume that if the original test statistic has a discrete distribution, then it has been transformed to have a continuous distribution using, for example, methods in \cite{Ste50} or \cite{HabPen11, Pen11}.  See the Supplemental Article or the aforementioned references for details and discussions.


\subsection{Implementation}
Computation of $\bm{\delta}(\bm{Z};\mat{t})$ is not trivial, especially when $\mat{t}$ is allowed to depend on prior parameters $\mat{p}$ and $\bm{\gamma}$ and data $\mat{Z}$.  To simplify the problem, we break $\mat{t}$ down into the product of a positive valued weight vector $\bm{w} = (w_m, m\in\mathcal{M})$ satisfying $\bar{w} = M^{-1}\sum_{m\in\mathcal{M}} w_m = 1$ and an overall or average threshold $t$.  That is, we write $\mat{t} = t\mat{w}$.  The basic methodology is then operationally implemented in the following two steps:
\begin{enumerate}
\item specify weights $\mat{w}$ and
\item collect data $\mat{Z}=\mat{z}$, find $t$, and compute $\bm{\delta}(\mat{z};t\mat{w})$.
\end{enumerate}
As in \cite{Gen06}, weights are allowed to depend functionally on prior parameters $\mat{p}$ and $\bm{\gamma}$ in Model \ref{randeffmodel} in the first step, while the overall threshold $t$ will be allowed to depend functionally on data $\mat{Z}$ and $\mat{w}$ in the second step.  The Weight Selection Procedure for the first step is formally defined in in Section \ref{sec 3} and the Threshold Selection Procedure for the second step is defined in Section \ref{sec 4}. %Later, we will see that allowing $t$ to depend on data and $\mat{w}$, as opposed to prior parameters $\mat{p}$ and $\bm{\gamma}$ alone, instills a robustness property into the procedure.  That is, this approach allows for FDP/FDP control in settings when prior parameters need not be correctly specified or consistently estimated.  A more detailed discussion will be provided later.

However, before proceeding we develop some foundational tools which facilitate the implementation of the second step.  Define the (unweighted) $p$-value statistic corresponding to $\delta_m$ by
\begin{equation}\label{p}
P_m =\inf\{t_m\in[0,1]:\delta_m(Z_m;t_m) = 1\}.
\end{equation}
This definition, also considered in \cite{HabPen11,Pen11}, has the usual interpretation that $P_m$ is the smallest size $t_m$ allowing for $H_m$ to be rejected and also ensures that
$$
\delta_m(Z_m;t_m) = I(P_m\leq t_m)
$$
almost surely under the NS assumptions.  For example, it can be verified that the $p$-value statistic corresponding to (\ref{model1example}) is $P_m = 1 - F_0(Z_m)$ and that $I(Z_m\leq F_0^{-1}(1-t_m)) = I(P_m\leq t_m)$ almost surely. See \cite{Hab12, HabPen14} for more details or for derivations of more complex $p$-values, such as the $p$-value for the local FDR statistic in \cite{EfrTib01, SunCai07} or for the optimal discovery procedure in \cite{Sto07}.

Now define weighted $p$-value by
%
\begin{equation}\label{weighted p}
Q_m =\inf\{t:\delta_m(Z_m; tw_m)=1\}.
\end{equation}
%
Observe that for $w_m$ fixed and writing $t_m = t w_m$,
$$
P_m = \inf\{tw_m:\delta_m(Z_m;tw_m)=1\} = w_m\inf\{t:\delta_m(Z_m;tw_m) = 1\} = w_mQ_m
$$
almost surely.  Thus, a weighted $p$-value can be computed by $Q_m = P_m/w_m$.  Hence, we have established the following almost surely equivalent expressions for a decision function under the NS assumptions:
\begin{equation}\label{form}
\delta_m(Z_m;t_m) = \delta_m(Z_m;tw_m) = I(P_m\leq t w_m) = I(Q_m\leq t).
\end{equation}

%Now, an MDF can be computed via the following two steps:
%\begin{enumerate}
%\item specify weights $\mat{w}$ and
%\item collect data $\mat{Z}=\mat{z}$ and compute $\bm{\delta}(\mat{z};t\mat{w})$.
%\end{enumerate}
%Model \ref{randeffmodel} and the first two expressions in (\ref{form}) will be used in implementing the first step, while the last two expressions in (\ref{form}) will be used in the second step.  It should be noted that, as in \cite{Gen06}, weights are specified \textit{apriori}, i.e. are only allowed to depend on prior information such as $\mat{p}$ and/or $\bm{\gamma}$ in Model \ref{randeffmodel}.  The overall threshold $t$, will be allowed to depend on $\mat{Z}$.  We will see in Sections \ref{sec 4} and \ref{sec 6} that this restriction significantly simplifies the implementation of the proposed procedure without sacrificing optimality, asymptotically.

\section{Optimal weights} \label{sec 3}
We first focus on optimal weights for Model \ref{randeffmodel} which assume that the overall threshold $t$ in $\bm{\delta}(\mat{Z};t\mat{w})$ is fixed. These weights are directly implementable in a variety of MDFs, such as the weighted Bonferroni-type procedure in \cite{Spj72}, where $t = \alpha/M$.  They are also useful for our proposed WAMDF because, even though it utilizes a data dependent overall threshold and weights must be specified before data are collected, the threshold can be well approximated for large $M$ using only $\mat{p}$ and $\bm{\gamma}$.   We refer to the first set of weights as optimal fixed-t weights and refer to the latter set of weights based on the approximated threshold as ``asymptotically optimal'' for reasons that will be formally justified in Theorem \ref{asymp opt w} (see Section \ref{sec 6}).

\subsection{Optimal fixed-t weights}
For the moment, we focus on $\bm{\delta}(\mat{Z};\mat{t})$ rather than $\bm{\delta}(\mat{Z};t\mat{w})$.  Recall that these expressions are equivalent due to (\ref{as delta}) and the constraint that $t$ is fixed with $\bar{w} = 1$ is equivalent to the constraint that $\bar{t} = t$, where $\bar{t} = M^{-1}\sum_{m\in\mathcal{M}}t_m$. Now, because weights are allowed to depend on $\mat{p}$ and $\bm{\gamma}$ under Model \ref{randeffmodel}, we focus on
%$$G(tw_m|\theta_m, \gamma_m) \equiv E[\delta_m(Z_m;tw_m)|\theta_m, \gamma_m] = (1-\theta_m) tw_m + \theta_m\pi_{\gamma_m}(tw_m)$$ and
\begin{equation}
\label{G}
G_m(t_m) \equiv E[\delta_m(Z_m;t_m)|\bm{p},\bm{\gamma}] = (1-p_m)t_m + p_m\pi_{\gamma_m}(t_m),
\end{equation}
where $\pi_{\gamma_m}(t_m) \equiv E[\delta_m(Z_m;t_m)|\theta_m = 1,\gamma_m]$ is the power function for $\delta_m$.  As in \cite{Gen06, RoqVan09, Pen11}, we assume power functions (as a function of $t_m$) are concave.
\begin{enumerate}
\item[(A1)] For each $m\in\mathcal{M}$, $t_m\mapsto\pi_{\gamma_m}(t_m)$ is concave and twice differentiable for $t_m\in(0,1)$ with $\lim_{t_m\uparrow1}\pi_{\gamma_m}'(t_m) = 0$ and
    $\lim_{t_m\downarrow 0}\pi_{\gamma_m}'(t_m) =\infty$ almost surely, where $\pi_{\gamma_m}'(t_m)$ is the derivative of $\pi_{\gamma_m}(t_m)$ with respect to $t_m$.
\end{enumerate}
This concavity condition is satisfied, for example, under monotone likelihood ratio considerations \citep{Pen11} and under the generalized monotone likelihood ratio (GMLR) condition in \cite{Cao13}.  The GMLR condition states that $g_{1m}(t_m)/g_{0m}(t_m)$ is monotonically decreasing in $t_m$, where $g_{im}(t_m)$ is the derivative of $G_{im}(t_m) \equiv E[\delta_m(Z_m;t_m)|\theta_m = i]$ with respect to $t_m$ for $i = 0, 1$.  In our notation, $g_{1m}(t_m) = \pi_{\gamma_m}'(t_m)$ and, under the NS conditions, $g_{0m}(t_m) = 1$.  Hence, the GMLR condition stipulates that $\pi_{\gamma_m}'(t_m)$ is monotonically decreases, i.e. that $\pi_{\gamma_m}(t_m)$ is concave.

Given $\mat{p}$, $\bm{\gamma}$ and $t$, the goal is to maximize the expected number of correctly rejected null hypotheses
\begin{equation}\label{power}
\pi(\mat{t},\mat{p},\bm{\gamma}) \equiv E\left[\sum_{m\in\mathcal{M}}\theta_m\delta_m(Z_m;t_m) \Big| \bm{\gamma},\mat{p}\right] = \sum_{m\in\mathcal{M}}p_m\pi_{\gamma_m}(t_m),
\end{equation}
subject to the constraint that $\bar{t} = t$.  Theorem \ref{opt solution} describes the form of the solution and states that it exists and is unique.
\begin{theorem}\label{opt solution}
Suppose that (A1) is satisfied and fix $t\in(0,1)$.  Then under Model \ref{randeffmodel} the maximum of $\pi(\mat{t},\mat{p}, \bm{\gamma})$ with respect to $\mat{t}$ subject to constraint $\bar{t} = t$ exists, is unique, and satisfies
\begin{equation}\label{derivative}
\pi_{\gamma_m}'(t_m) = k/p_m
\end{equation}
for every $m\in\mathcal{M}$ and some $k>0$ almost surely.
\end{theorem}

The optimal fixed-t thresholds described in Theorem \ref{opt solution}, and their corresponding weights, can be numerically found as follows.  First, for a fixed value of $k$, denote the solution to equation (\ref{derivative}) in terms of $t_m$ by $t_m(k/p_m, \gamma_m)$ and denote the vector of solutions by $\mat{t}(k,\mat{p},\bm{\gamma}) = [t_m(k/p_m,\gamma_m), m\in\mathcal{M}]$.  In Example \ref{example1} below a closed form expression for $t_m(k/p_m, \gamma_m)$ exists, but in general we may employ any single root finding algorithm to compute each $t_m(k/p_m,\gamma_m)$ because $\pi_{\gamma_m}'(t_m)$ is continuous and monotone in $t_m$ by (A1).  Then to find the optimal fixed-t threshold vector, first find the unique $k^*$ satisfying $\bar{t}_M(k^*,\mat{p},\bm{\gamma}) = t$, where $\bar{t}_M(k,\mat{p},\bm{\gamma}) = M^{-1}\sum_{m\in\mathcal{M}}t_m(k/p_m,\gamma_m)$, and then compute $\mat{t}(k^*,\mat{p},\bm{\gamma})$.  Because $t_m = tw_m$ in our setup and because $\bar{t}_M(k^*,\mat{p},\bm{\gamma}) = t$, each optimal fixed-t weight is recovered via %equation \ref{w star} below.
\begin{equation}\label{w star}
w_m(k^*,\mat{p},\bm{\gamma}) = \frac{t_m(k^*/p_m,\gamma_m)}{\bar{t}_M( k^*,\mat{p},\bm{\gamma})}.
\end{equation} We shall sometimes denote $w_m(k^*,\mat{p},\bm{\gamma})$ by $w_m^*$ for short.  The vector of optimal fixed-t weights is denoted by $\mat{w}(k^*,\mat{p},\bm{\gamma}) = [w_m(k^*,\mat{p},\bm{\gamma}),m\in\mathcal{M}]$ or by $\mat{w}^* = (w_m^*,m\in\mathcal{M})$ for short.

To better understand how the solution is found and related to the values of $p_m$, $\gamma_m$ and $t$ consider the following example.
\begin{example}\label{example1}
Suppose $Z_m|\gamma_m,\theta_m \sim N(\theta_m\gamma_m, 1)$ for $\gamma_m>0$, where $N(a, b)$ represents a Normal distribution with mean $a$ and variance $b$, and consider testing $H_m:\theta_m = 0$.  Denote the standard normal cumulative distribution function and density function by $\Phi(\cdot)$ and $\phi(\cdot)$, respectively, and define $\delta_m(Z_m;t_m) = I(Z_m\geq \Phi^{-1}(1-t_m))$.  The power function is
$$
\pi_{\gamma_m}(t_m) = 1 - \Phi(\Phi^{-1}(1-t_m)-\gamma_m)
$$
and has derivative
$$\pi_{\gamma_m}'(t_m) = \frac{\phi(\Phi^{-1}(1-t_m)-\gamma_m)}{\phi(\Phi^{-1}(1-t_m))}.$$
Setting the derivative equal to $k/p_m$ and solving yields
\begin{equation}\label{t example}
t_m(k/p_m, \gamma_m) = 1 - \Phi\left(0.5\gamma_m + \log(k/p_m)/\gamma_m\right).
\end{equation}
The optimal fixed-$t$ threshold vector is computed by $\mat{t}(k^*,\mat{p},\bm{\gamma})$, where $k^*$ satisfies $\bar{t}_M(k^*,\mat{p},\bm{\gamma}) = t$, and the optimal fixed-$t$ weights are computed as in (\ref{w star}).
\end{example}

First, observe in expression (\ref{t example}) that $t_i(k/p_i, \gamma_i) = t_{j}(k/p_{j},\gamma_{j})$ if $\gamma_i=\gamma_{j}$ and $p_i=p_{j}$ regardless of $k$, and consequently, the optimal fixed-$t$ weight vector is $\bm{1}$ for any $t$ when data are homogeneous. On the other hand, we see that $t_m(k/p_m,\gamma_m)$ is increasing in $p_m$ and hence
$$w_m(k^*,\mat{p},\bm{\gamma}) = M\frac{t_m(k^*/p_m, \gamma_m)}{t_m(k^*/p_m,\gamma_m) + \sum_{j\neq m}t_j(k^*/p_j,\gamma_j)}$$ is increasing in $p_m$.  Hence, if we increase the likelihood that $H_m$ is false then the corresponding optimal weight increases.
\begin{figure}
\epsfig{file = weights, width = 12cm}
\caption{\label{opt figure}A depiction of the optimal thresholds for $M = 2$ hypotheses tests when power functions vary under constraint $0.5(t_1 + t_2) = 0.01$ (left) and $0.5(t_1 + t_2) = 0.05$ (right). }
\end{figure}

The relationship between $w_m(k^*,\mat{p},\bm{\gamma})$ and $\gamma_m$ is more complex. To illustrate, consider testing $M = 2$ null hypotheses and suppose $\gamma_1 = 1.5$, $\gamma_2 = 2.5$, and $p_1=p_2 = 0.5$.  In Figure \ref{opt figure}, observe that for $t = 0.01$, $\bar{t}_M(k^*,\mat{p},\bm{\gamma}) = 0.01$ when $k^* = 6.1$, which gives $t_1(k^*/p_1,\gamma_1) = 0.017$, $t_2(k^*/p_2, \gamma_2) = 0.003$, $w_1^* = 0.003/0.01 = 0.3$ and $w_2^* = 0.017/0.01 = 1.7$.  Note that because $p_1=p_2$, the slopes of the power functions evaluated at $0.003$ and $0.017$, respectively, are equal; see equation (\ref{derivative}).  Now consider fixed threshold $t = 0.05$.  Here $k^* =  1.7$ and the slopes of the power functions evaluated at the solutions are again equal, but now $w_1^* = 0.059/0.05 = 1.18$ and $w_2^* = 0.041/0.05 = 0.82$.  That is, when $t = 0.01$, the hypothesis with the larger effect size is given more weight but when $t = 0.05$ it is given less weight.  For a more detailed discussion on this phenomenon see \cite{Pen11} or Section \ref{sec 8} of the current manuscript.  The important point is that the optimal fixed-$t$ weights depend on the choice of $t$, in addition to the parameters $\mat{p}$ and $\bm{\gamma}$.

\subsection{Asymptotically optimal weights}
In Section \ref{sec 4}, the overall threshold $t$ is chosen using an estimator of the FDP which depends functionally on data $\mat{Z}$; see expressions (\ref{M0 hat}) and (\ref{FDRhat}).  Hence, optimal fixed-$t$ weights, which are not allowed to functionally depend on $\mat{Z}$, are not readily implementable.  To solve this problem, we approximate the FDP estimator using only $\mat{p}$ and $\bm{\gamma}$ and use the resulting ``approximator'' to approximate the data dependent threshold.  Then, the optimal fixed-$t$ weights for the approximated threshold are computed.  The approximator essentially plugs $G_m(t_m(k/p_m,\gamma_m))$ in for each $\delta_m(Z_m; t_m(k/p_m,\gamma_m))$ in (\ref{M0 hat}) and (\ref{FDRhat}).  Formally, denote $\bar{G}_M(\mat{t}(k,\mat{p},\bm{\gamma})) =M^{-1}\sum_{m\in\mathcal{M}}G_m(t_m(k/p_m,\gamma_m))$
and define the FDP approximator by
$$\widetilde{FDP}_M(\mat{t}(k,\mat{p},\bm{\gamma})) = \frac{1-\bar{G}_M(\mat{t}(k,\mat{p},\bm{\gamma}))}{1-\bar{t}_M(k,\mat{p},\bm{\gamma})}\frac{\bar{t}_M(k,\mat{p},\bm{\gamma})}{\bar G_M(\mat{t}(k,\mat{p},\bm{\gamma}))}.$$
Now, the asymptotically optimal weights for the proposed WAMDF are computed as follows for $0<\alpha\leq 1-p_{(M)}$, where $p_{(M)} \equiv \max\{\bm{p}\}$.
\begin{wprocedure} \label{approx t wproc} For $0<\alpha\leq 1 - p_{(M)}$,
\begin{enumerate}
\item[a.] get $k_M^* = \inf\left\{k: \widetilde{FDP}_M(\mat{t}(k,\mat{p},\bm{\gamma})) = \alpha\right\},$ and
\item[b.] for each $m\in\mathcal{M}$, compute $w_m^* = w_m(k_M^*,\mat{p},\bm{\gamma})$ as in (\ref{w star}).
\end{enumerate}
\end{wprocedure}
In Theorem \ref{exist} below, we that see the restriction $0<\alpha\leq 1 - p_{(M)}$ ensures that a solution to $\widetilde{FDP}_M(\mat{t}(k,\mat{p},\bm{\gamma})) = \alpha$ exists.  In practice, this restriction amounts to choosing $\alpha$ and $\mat{p}$ so that $0<\alpha\leq 1-p_m$ for each $m$.  That is, the prior probability that the null hypothesis is true should be at least $\alpha$.  To see why this condition is reasonable, suppose that $1-p_m < \alpha$ for each $m$.  Then we need not consider a weighting scheme or even collect data in the first place because even if all $M$ null hypotheses are rejected, the model stipulates that the expected proportion of false discoveries among the $M$ discoveries is $M^{-1}\sum_{m\in\mathcal{M}}1-p_m < \alpha.$  Hence, if the conditions of Theorem \ref{exist} are not satisfied, we should suspect that the posited model is poor and consider a new model.
%In our analysis, where $p_m$ is viewed as random, the restriction $0<\alpha\leq 1-p_{(M)}$ is satisfied by assuming $\Pr(p_m\leq 1 - \alpha) = 1$, i.e. restricting the support of $p_m$ to $[0, 1-\alpha]$ in Model \ref{randeffmodel}.
\begin{theorem}\label{exist}
Under (A1) and Model \ref{randeffmodel}, $k^*_M$ exists almost surely for $0<\alpha\leq 1-p_{(M)}$.
\end{theorem}

It should be noted that $\widetilde{FDP}_M(\mat{t}(k,\mat{p},\bm{\gamma}))$ need not be monotone in $k$ and hence multiple solutions to $\widetilde{FDP}_M(\mat{t}(k,\mat{p},\bm{\gamma})) = \alpha$ could exist.  We choose the smallest $k$ above because this solution has the largest overall threshold $\bar{t}_M(k,\mat{p},\bm{\gamma})$ (recall $\bar{t}_M(k,\mat{p},\bm{\gamma})$ increases as $k$ decreases) and hence is consistent with our data dependent threshold selection procedure in the next section.  Also note that $\bar{t}_M(k^*_M,\mat{p},\bm{\gamma}) = t$ for some $t\in(0,1)$, i.e. these weights could be viewed as optimal fixed-t weights.  The main difference is that the constraint $\widetilde{FDP}_M(\mat{t}(k_M^*,\mat{p},\bm{\gamma})) = \alpha$ is specified for the asymptotically optimal weights, as opposed to the constraint $\bar{t}_M(k_M^*,\mat{p},\bm{\gamma}) = t$ in Theorem \ref{opt solution}.  We use the notation $k^*_M$, as opposed to $k^*$, to indicate that we are dealing specifically with asymptotically optimal weights and to facilitate asymptotic analysis later.

\section{The procedure}\label{sec 4}
Now we are now in position to formally define the proposed adaptive threshold, which, when used in conjunction with asymptotically optimal weights in $\bm{\delta}(\mat{Z};t\mat{w})$, yields the proposed asymptotically optimal WAMDF.

\subsection{Threshold selection}
For the moment, let $\mat{w}$ be any fixed vector of positive weights satisfying $\bar{w} = 1$.  For brevity, we sometimes suppress the $Z_m$ in each $\delta_m$ and write $\delta_m(tw_m)$ and denote $\bm{\delta}(\mat{Z};t\mat{w})$ by $\bm{\delta}(t\mat{w})$.  Further, denote the number of rejected null hypotheses, or discoveries, at $t\mat{w}$ by $R(t\mat{w}) = \sum_{m\in\mathcal{M}}\delta_m(tw_m)$.

We make use of an ``adaptive'' estimator of the FDP, i.e. it utilizes an estimator of $M_0$ defined by
\begin{equation} \label{M0 hat}
\hat{M}_0(\lambda \mat{w}) = \frac{M - R(\lambda\mat{w}) + 1}{1-\lambda}
\end{equation}
for some fixed tuning parameter $\lambda\in(0,1)$.  This estimator is essentially the weighted version of the estimator in \cite{Sto02} defined by $\hat{M}_0(\lambda\bm{1}) = [M - R(\lambda\bm{1})]/[1-\lambda]$.  For earlier work on the estimation of $M_0$ see \cite{Sch82}.   The idea, in the unweighted setting, is that for $m\in\mathcal{M}_1$, $E[\delta_m(\lambda)]\leq 1$ but the inequality is relatively sharp if all tests have reasonable power and $\lambda$ is chosen sufficiently large.  Hence $$E[M - R(\lambda\bm{1})]  = \sum_{m\in\mathcal{M}} E[1-\delta_m(\lambda)] \geq \sum_{m\in\mathcal{M}_0} E[1-\delta_m(\lambda)] = (1-\lambda)M_0$$ and $E[\hat{M}_0(\lambda\bm{1})] \geq M_0$. That is, $\hat{M}_0$ is positively biased but the bias is minor for suitably chosen $\lambda$ and reasonably powerful tests.  Similar intuition applies for $\hat{M}_0(\lambda\mat{w})$. As in \cite{Sto04}, we add 1 to the numerator in expression (\ref{M0 hat}) to ensure that $\hat{M}_0(\lambda\mat{w})>0$ for finite sample results.  For a discussion on weighted vs. unweighted estimation of $M_0$, see Section \ref{sec 9}.

%Clearly the choice of tuning parameter $\lambda$ governs the behavior of $\hat{M}_0(\lambda\mat{w})$.  For detailed investigations of this issue in the unweighted setting see \cite{Sto04}, \cite{BlaRoq09} or \cite{LiaNet12}.  For now, it is important to note that $\lambda$ and $\mat{w}$ should be chosen in concert so as to avoid $\lambda w_m>1$, which would imply $\delta_m(\lambda w_m) = 1$ almost surely and amount to ignoring data for $H_m$ in the estimation of $M_0$. This condition is formally stated below for clarity.
%\begin{enumerate}
%\item[(A2)] For every $m\in\mathcal{M}$, $0<w_m\lambda<1.$
%\end{enumerate}
%Note that it is necessary to decrease the value of the tuning parameter $\lambda$ if  highly dispersed weights are used. However, for most practical applications, we anticipate $w_m$ is less than 2 or 3 so that $\lambda = 1/2$ or $1/3$ is satisfactory. The adaptive FDP estimator at $t\mat{w}$ is defined

The adaptive FDP estimator is defined by
\begin{equation}\label{FDRhat}
\widehat{FDP}^{\lambda}(t\mat{w}) =
%\left\{\begin{array}{cc}
\frac{\hat{M}_0(\lambda\mat{w}) t}{\max\{R(t\mat{w}), 1\}}.%
% & %t\leq \lambda\\
%1 & t>\lambda
%\end{array}\right.
\end{equation}
The adaptive threshold, which essentially chooses $t$ as large as possible subject to the constraint that the estimate of the FDP is less than or equal to $\alpha$,  is defined by
%
\begin{equation}
\hat t_\alpha^\lambda = \sup\{0 \leq t \leq u: \widehat{FDP}^\lambda(t\mat{w})\leq \alpha\}.\label{t stop}
\end{equation}
%
We assume that $u$, the upper bound for $\hat{t}_\alpha^\lambda$, and the tuning parameter $\lambda$ satisfy condition (A2), given by
\begin{itemize}
\item[(A2)] $\lambda \leq u \leq 1/w_{(M)}$,
\end{itemize}
where $w_{(M)}\equiv \max\{\mat{w}\}$.  This ensures that $\hat t_\alpha^\lambda w_m\leq 1$ and $\lambda w_m\leq 1$ for every $m$.  It should be noted that for $\mat{w} = \bm{1}$ and $u = \lambda$ (which implies $\hat{t}_\alpha^\lambda\leq \lambda$), we recover the unweighted adaptive MDF for finite FDR control in \cite{Sto04}.

In practice, $\hat{t}_{\alpha}^\lambda$ can be difficult to compute.  Alternatively, we may apply the original BH procedure to the weighted $p$-values at level $\alpha/\hat{M}_0(\lambda\mat{w})$.  Note that due to (\ref{form}) we can also use weighted $p$-values to estimate $M_0$ via
$$\hat{M}_0(\lambda\mat{w}) = \frac{M - \sum_{m\in\mathcal{M}}I(Q_m\leq \lambda)+1}{1-\lambda}.
$$
Formally, this threshold selection procedure can be implemented as follows.
\begin{tproc} Fix $\lambda$ and $u$ satisfying (A2).  Then
\begin{enumerate}
\item[a.] compute weighted $p$-values as in (\ref{weighted p}) and get ordered weighted $p$-values $Q_{(1)}\leq Q_{(2)}\leq \ldots \leq Q_{(M)}$.
\item[b.] If $Q_{(m)}>\alpha m/\hat{M}_0(\lambda\mat{w})$ for each $m$, set $j = 0$, otherwise take $$j = \max\left\{m\in\mathcal{M}:Q_{(m)}\leq \alpha/\hat{M}_0(\lambda\mat{w})\right\}.$$
\item[c.] Get $\hat t_{\alpha}^{\lambda *} = \min\{j\alpha/\hat{M}_0(\lambda\mat{w}), u\}$ and compute $\delta_m(Z_m;\hat t_{\alpha}^{\lambda *}w_m) = I(Q_m\leq \hat t_{\alpha}^{\lambda *})$ for each $m$.
\end{enumerate}
\end{tproc}

The WAMDF implemented above is equivalent to $\bm{\delta}(\mat{Z};\hat t_{\alpha}^\lambda\mat{w})$ in that
\begin{equation}\label{implement eqn}
\delta_m(Z_m;\hat t_\alpha^{\lambda}w_m) = I(Q_m\leq  \hat{t}_{\alpha}^\lambda) = I(Q_m\leq \hat{t}_{\alpha}^{\lambda*})
\end{equation}
almost surely for each $m$, i.e. both procedures reject the same set of null hypotheses.  The first equality in (\ref{implement eqn}) follows from (\ref{form}) and the last equality in (\ref{implement eqn}) is a consequence of Lemma 2 in \cite{Sto04}.

\subsection{The asymptotically optimal WAMDF}

The asymptotically optimal WAMDF is formally defined as $\bm{\delta}(\mat{Z};\hat t_{\alpha}^\lambda \mat{w}^*)$ for $0<\alpha\leq 1 - p_{(M)}$ and $\lambda = \bar{t}_M(k_M^*,\mat{p},\bm{\gamma})$, and where $k_M^*$ and $\mat{w}^*$ are defined as in the Weight Selection Procedure.  This particular choice of $\lambda$ is not necessary for ensuring asymptotic FDP or FDR control but, as we will see, ensures that the employed weights are indeed ``asymptotically optimal'' (see Theorem \ref{asymp opt w}) and additionally that (A2) is satisfied if we take $u = 1/w_{(M)}$.  The asymptotically optimal WAMDF is implemented as follows.
\begin{asymptotically optimal WAMDF}To implement the asymptotically optimal WAMDF
\begin{enumerate}
\item specify $(\mat{p}, \bm{\gamma})$, $\alpha\in(0,1-p_{(M)}]$ and compute $\mat{w}^*$ using the Weight Selection Procedure,
\item choose $\lambda = \bar{t}_M(k_M^*,\mat{p},\bm{\gamma})$ and $u$ satisfying (A2), collect data $\mat{Z}=\mat{z}$ and compute $\bm{\delta}(\mat{z};\hat t_\alpha^\lambda\mat{w}^*)$ using the Threshold Selection Procedure.
\end{enumerate}
\end{asymptotically optimal WAMDF}

To illustrate, consider testing $M = 10$ null hypotheses under the setting outlined in Example \ref{example1}, with $p_m = 0.5$ for $m = 1, 2, ..., 10$, $\gamma_m =2$ for $m = 1, 2, \ldots, 5$, $\gamma_m = 3$ for $m = 6, 7, \ldots, 10$, and take $\alpha = 0.05$.  Recall the goal is to test $H_m:\theta_m = 0$ and that decision functions are of the form $\delta_m(Z_m;t_m) = I(Z_m\geq \Phi^{-1}(1-t_m))$.  Further, $p$-values are computed by $P_m = 1 - \Phi(Z_m)$ and recall that weighted $p$-values can be computed by $Q_m = P_m/w_m$.  See Table \ref{illustrate} for summaries of parameters, weights, simulated data, $p$-values, and weighted $p$-values.  As before, the Weight Selection Procedure in step 1 is broken down into 2 sub-steps and the Threshold Selection Procedure in step 2 is split into three sub-steps.  Now, to test these null hypotheses we
\begin{table}\center
\caption{\label{illustrate} A portion of the parameters, data, weights, $p$-values, and weighted $p$-values in columns 1 - 5, respectively.  Each row is sorted in ascending order according to $Q_1, Q_2, ..., Q_M$.}
\begin{tabular}{ccccccc} \hline \hline \scriptsize
$\theta_m$&$\gamma_m$ & $w_m^*$& $Z_m$ & $P_m$ & $Q_m$ & $0.05 m /\hat{M}_0$ \\ \hline
1 & 3 & 0.74 & 3.14 & 0.001 & 0.001 & 0.006 \\
1 & 2 & 1.26 & 2.55 & 0.005 & 0.005 & 0.012 \\
1 & 3 & 0.74 & 2.56 & 0.005 & 0.006 & 0.018 \\ \hline
1 & 2 & 1.26 & 1.47 & 0.070 & 0.062 & 0.024 \\
0 & 2 & 1.74 & 1.17 & 0.121 & 0.106 & 0.030 \\
$\vdots$ & $\vdots$ &$\vdots$ &$\vdots$ &$\vdots$ &$\vdots$ &  \\
0 & 3 & 0.74 & -0.60 & 0.724 & 0.844 & 0.061\\ \hline
\end{tabular}
\end{table}
\begin{enumerate}
\item[1a.] specify $\bm{\gamma}$ (see column 2 of Table \ref{illustrate}), $\mat{p}$ and $\alpha$ and find $k_M^* = 2.52$.
\item[1b.] Compute asymptotically optimal weights $w_m^* = w_m(k_M^*, \bm{p}, \bm{\gamma})$ as in (\ref{w star}).  See column 3 in Table \ref{illustrate}.
\item[2a.] Take $\lambda = \bar{t}_M(k_M^*, \bm{p},\bm{\gamma}) = 0.028$ and $u = 1/1.26 = 0.79$.  Collect data $\mat{Z} = \mat{z}$, compute $p$-values and weighted $p$-values, and order weighted p-values from smallest to largest (see columns 4 - 6 in Table \ref{illustrate}).
\item[2b.] Observe that $Q_{(m)}\leq \alpha m/\hat{M}_0(\lambda\mat{w}^*)$ for $m = 3$ but not for $m = 4, 5, ..., 10$ and hence $\alpha j/\hat{M}_0(\lambda\mat{w}^*) = 0.05 \tfrac{3}{8.23} = 0.013$.
\item[2c.] Compute $\hat{t}_{\alpha}^{\lambda*} = \min\{0.013, 0.79\} = 0.013$ and reject null hypotheses with weighted $p$-values 0.001, 0.005 and 0.006 because their $Q_m$s are less than 0.013.
\end{enumerate}
%Note that repeating sub-steps 2b. - 2c. with $\lambda = 0.5$, as suggested in \cite{Sto04} in the unweighted setting, gives $\hat{M}_0(\lambda\mat{w}^*) = 8.0$ and results in the same set of discoveries.  For a discussion on the choice of $\lambda$, see Section \ref{sec 9}. %It is also worth noting that is not necessary to force $t_{\alpha^\lambda}\leq \lambda$ only requiring asymptotic FDP control as we will see in Section \ref{sec 6}, in which case we may use threshold $\hat t_{\alpha}^{\lambda'} = j\alpha \hat{M}_0(\lambda\mat{w})$.

\section{Finite FDR control}\label{sec 5}
Next an upper bound for the FDR is given for any arbitrary set of fixed or realized weights satisfying $w_m>0$ for each $m$ and $\bar{w} = 1$.  The bound is computed under the following dependence structure for $\mat{Z}$:
\begin{enumerate}
\item[(A3)] $(Z_m, m\in\mathcal{M}_0)$ are mutually independent and independent of the collection $(Z_m, m\in\mathcal{M}_1)$.
\end{enumerate}
This structure was considered in \cite{BenHoc95} (and relaxed in \cite{BenYek01}) to prove FDR control for the original unweighted unadaptive BH procedure.  It was also used in the proof of FDR control for the weighted unadaptive BH procedures in \cite{Gen06, Pen11, Pen14} and for the unweighted adaptive BH procedure in \cite{Sto04}.  Note that (A3) is satisfied under Model \ref{randeffmodel} conditionally upon $\bm{\theta}, \mat{p},\bm{\gamma}$, i.e. for $F(\bm{z}|\bm{\theta},\mat{p},\bm{\gamma})$, but is not limited to this setting.

To define the FDR, let $V(t\mat{w}) = \sum_{m\in\mathcal{M}_0}\delta_m(tw_m)$ denote the number of erroneously rejected null hypotheses (false discoveries) and recall that $R(t\mat{w}) = \sum_{m\in\mathcal{M}}\delta_m(t\mat{w})$ is the number of rejected null hypotheses, or discoveries, for threshold vector $t\mat{w}$.  Define the FDP at $t\mat{w}$ by
\begin{equation}\label{FDP}
FDP(t\mat{w}) = \frac{V(t\mat{w})}{\max\{R(t\mat{w}), 1\}}.
\end{equation}
The FDR at $t\mat{w}$ is defined by $FDR(t\mat{w}) = E[FDP(t\mat{w})]$, where the expectation is taken over $\mat{Z}$ with respect to an arbitrary $F\in\mathcal{F}$, which may include, but is not limited to, $F(\bm{z}|\bm{\theta},\mat{p},\bm{\gamma})$ in Model \ref{randeffmodel}.  %Of interest in this section are distributions for $\mat{Z}$ satisfying (A3).

%It should be noted that, within the context of a random effects model, the above definition of the FDR is sometimes referred to as the ``conditional'' FDR to reflect the fact that the expectation is taken with respect to the distribution of $\mat{Z}$ conditionally upon all other random parameters governing the stochastic behavior of $\mat{Z}$, i.e. $F(\mat{z}|\bm{\theta},\bm{p},\bm{\gamma})$ in Model \ref{randeffmodel}.  This is in contrast to an expectation taken over all random quantities, i.e. over $(\mat{Z},\bm{\theta},\mat{p},\bm{\gamma})$, which gives the ``unconditional FDR''.  It is worth noting if the conditional is less than or equal to $\alpha$ then the unconditional FDR is less than or equal to $\alpha$, but that the converse need not be true.   For more details see \cite{Gen06, BlaRoq08, RoqVan09, RoqVil11}.

The bound is presented in Lemma \ref{bound} below for any distribution function $F$ satisfying (A3).  The focus is on the setting when $M_0\geq 1$ because the FDR is trivially 0 if $M_0 = 0$.   As in \cite{Sto04}, we force $\hat{t}_{\alpha}^\lambda\leq \lambda$ by taking $u = \lambda$ in (\ref{t stop}) to facilitate the use of the Optimal Stopping Theorem in the proof.
\begin{lemma}\label{bound}
Suppose $M_0\geq 1$ and that (A2) and (A3) are satisfied.  Then for $\hat t_{\alpha}^\lambda$ defined as in (\ref{t stop}) with $u = \lambda$,
\begin{equation}\label{lem1 eqn}
FDR(\hat t_\alpha^{\lambda}\mat{w}) \leq \alpha\bar{w}_0\frac{1-\lambda}{1-\lambda\bar{w}_0}[1-(\lambda\bar{w}_0)^{M_0}]\leq\alpha\bar{w}_0\frac{1-\lambda}{1-\lambda\bar{w}_0}, \end{equation}
where $\bar{w}_0 = M_0^{-1}\sum_{m\in\mathcal{M}_0}w_m$ is the mean of the weights corresponding to true null hypotheses.
\end{lemma}
\noindent
Observe that $1-(\lambda\bar{w}_0)^{M_0} \leq 1 $ due to (A2).  Further, if $\mat{w} = \bm{1}$ then $\bar{w}_0 = 1$ and we recover Theorem 3 in \cite{Sto04} as a corollary.

Of course, if $\mat{w}\neq \bm{1}$, the bound in Lemma \ref{bound} is not immediately applicable because $\mathcal{M}_0$ and consequently $\bar{w}_0$ is unobservable. One solution is to utilize an upper bound for $\bar{w}_0$ and adjust the ``$\alpha$'' at which the procedure is applied.  The necessary $\alpha$-adjustment is presented in Theorem \ref{FDRthm}.
\begin{theorem}\label{FDRthm}
Define
$$\alpha^* = \alpha \frac{1}{w_{(M)}}\frac{1-\lambda w_{(M)}}{1-\lambda}.$$
Then under the conditions of Lemma \ref{bound}, $FDR(\hat{t}_{\alpha^*}^\lambda\mat{w})\leq \alpha$.
\end{theorem}
In the next section, we see that $\bar{w}_0$ is typically less than or equal to 1, asymptotically, so that this $\alpha$ adjustment is not needed for large $M$.


\section{Asymptotic results} \label{sec 6}
We first focus on an asymptotic assessment of WAMDFs for arbitrary weights.  In particular, we show that they dominate their unadaptive counterparts in that they always reject more null hypotheses.  This generalizes the results in \cite{Sto04}, where $\mat{w}$ was equal to $\bm{1}$. Then, sufficient conditions for the weights are provided under which WAMDFs control the FDP and are  $\alpha$-exhaustive.  These results are used in the asymptotic analysis of the asymptotically optimal WAMDF.  %Specifically, it is shown that the asymptotically optimal WAMDF controls of the FDP and is $\alpha$-exhaustive under certain weighting schemes considered in the literature, even if weights are misspecified or noisy. Then, formal justification for the terminology ``asymptotically optimal'' weights is provided.

To facilitate asymptotic analysis, denote weight vectors of length $M$ by $\mat{w}_M$ and the $m$th element of $\mat{w}_M$ by $w_{m,M}$.  Further, denote the mean of the weights from true null hypotheses by $\bar{w}_{0,M}$.  Denote the adaptive FDP estimator in (\ref{FDRhat}) by $\widehat{FDP}^\lambda_M(t\bm{w}_M)$ and the FDP in (\ref{FDP}) by $FDP_M(t\bm{w}_M)$.  We will also consider an unadaptive FDP estimator, which uses $M$ in the place of an estimate of $M_0$, defined by
$$\widehat{FDP}_M^0(t\mat{w}_M) = \frac{M t}{\max\{R(t\mat{w}_M), 1\}}.$$
When necessary, we also denote the tuning parameter in (\ref{M0 hat}) by $\lambda_M$ because, as in the asymptotically optimal WAMDF where $\lambda_M = \bar{t}_M(k_M^*,\mat{p},\bm{\gamma})$, it may depend on $M$.

Recall that we assumed $\lambda\leq u \leq 1/w_{(M)}$ in (A2) to ensure that every individual threshold was bounded above by 1.  In our asymptotic analysis, we redefine (A2) as follows:
\begin{enumerate}
\item[(A2)]$\lambda_M\rightarrow \lambda\leq u = 1/k$ almost surely, where $k$ satisfies $\lim_{M\rightarrow \infty} w_{(M)}\leq k$ almost surely.
\end{enumerate}
We will see that (A2) is satisfied, for example, under Model \ref{randeffmodel} and (A1) for the asymptotically optimal WAMDF.   Now, the adaptive threshold is defined as in (\ref{t stop}) and is denoted $\hat t_{\alpha,M}^{\lambda}$.  The unadaptive threshold is defined by
$$\hat t_{\alpha,M}^0 = \sup\{0\leq t\leq u: \widehat{FDP}_M^0(t\mat{w}_M)\leq \alpha\}.$$


\subsection{Arbitrary weights}
Convergence criteria considered here are similar to criteria in \cite{Sto04, Gen06} and allow for weak dependence structures.  See \cite{Bil99}, \cite{Sto03} or see Theorem \ref{optimal conditions} for examples.  Assume for any fixed $t\in(0,u]$ that
%
\begin{enumerate}
\item[(A4)]$R(t\mat{w}_M)/{M} \rightarrow G(t)$ almost surely,
\item[(A5)]$V(t\mat{w}_M)/M \rightarrow a_0 \mu_0 t$ almost surely, for $0<\mu_0<\infty$ and $0<a_0<1$, where $\bar{w}_{0,M} \rightarrow \mu_0$ and  $M_0/M \rightarrow a_0,$ and
\item[(A6)] $t/G(t)$ is strictly increasing and continuous over (0,u) with $$\lim_{t\downarrow 0}\frac{t}{G(t)} = 0 \mbox{\hspace{.1in} and }\lim_{t\uparrow u}\frac{u}{G(u)} \leq 1.$$
\end{enumerate}
%
Here $\mu_0$ is the asymptotic mean of the weights corresponding to true null hypotheses and $a_0$ is the asymptotic proportion of true null hypotheses. The last condition is natural as it will ensure that, asymptotically, the FDP is continuous and increasing in $t$ and takes on value 0, thereby ensuring that it can be controlled.  Note that writing $R(t\mat{w}_M)/M = \sum_{m\in\mathcal{M}}I(Q_m\leq t)/M$ via (\ref{form}), we see that (A4) corresponds to the assumption that the empirical process of the weighted $p$-values converges pointwise to $G(t)$ almost surely.

Asymptotic analysis will focus on comparing random thresholds $\hat t_{\alpha,M}^\lambda$ and $\hat t_{\alpha,M}^0$ to their corresponding asymptotic (nonrandom) thresholds, which are based on the limits of the unadaptive and adaptive FDP estimators. Denote the pointwise limits of the unadaptive FDP estimator, the adaptive FDP estimator and the FDP by
$$FDP_\infty^0(t) = \frac{t}{G(t)}, \hspace{.1 in} FDP_\infty^\lambda(t) = \frac{1-G(\lambda)}{1-\lambda}\frac{t}{G(t)} \mbox{\hspace{.1in} and \hspace{.1in} }
FDP_{\infty}(t) = \frac{a_0\mu_0 t}{G(t)},$$
respectively (see Lemma S1 in the Supplement for verification and details).  Define asymptotic unadaptive and asymptotic adaptive thresholds by
$$t_{\alpha,\infty}^0 = \sup\{0\leq t\leq u:FDP_{\infty}^0(t)\leq \alpha\}$$
and
$$t_{\alpha,\infty}^\lambda = \sup\{0\leq t\leq u:FDP_{\infty}^\lambda(t)\leq \alpha\},$$ respectively.


In Theorem \ref{asymptotic t} we see that both the unadaptive and adaptive thresholds converge to their asymptotic (nonrandom) counterparts, with the asymptotic adaptive threshold being larger than the asymptotic unadaptive threshold.  By virtue of $E[\delta_m(tw_m)]$ being strictly increasing in $t$ for each $m$, it follows that the adaptive procedure will lead to a higher proportion of rejected null hypotheses, asymptotically.
\begin{theorem}\label{asymptotic t}
Fix $\alpha\in(0,1)$.  Then under (A2) and (A4) - (A6)
%
\begin{equation}\label{asymptotic t eqn}
\lim_{M\rightarrow\infty} \hat{t}_{\alpha,M}^0 = t_{\alpha,\infty}^0 \leq \lim_{M\rightarrow\infty}\hat{t}_{\alpha,M}^{\lambda} = t_{\alpha,\infty}^{\lambda}
\end{equation}
almost surely.
\end{theorem}
\noindent

Before focusing on the FDP  it is useful to formally describe the notion of an $\alpha$-exhaustive MDF.  Loosely speaking, \cite{Fin09} referred to an unweighted multiple decision function, say $\bm{\delta}(\hat{t}_{\alpha,M}^*\bm{1}_M)$, as ``asymptotically optimal'' (we will use the terminology $\alpha$-exhaustive) if
$FDR(\hat{t}_{\alpha,M}^*\bm{1}_M)\rightarrow \alpha$ under some least favorable distribution.  A Dirac Uniform (DU) distribution, which we shall define as a distribution that satisfies
$$
E[\delta_m(t)] = \left\{\begin{array}{lll} t & & \mbox {if } m\in\mathcal{M}_0 \\
1 & & \mbox{if }m\in\mathcal{M}_1\end{array}\right.
$$
for every $t\in[0,1]$ and $m\in\mathcal{M}$, was shown to often be least favorable for the FDR in that among all $F$s that satisfy $E[\delta_m(t)] = t$ for every $t\in[0,1]$ when $m\in\mathcal{M}_0$ and dependency structure (A3), $FDR(\hat{t}_{\alpha,M}^*\bm{1}_M)$ is the largest under a DU distribution.  Observe that if (A4) - (A5) are satisfied, then $G(t) = a_0 \mu_0 t + (1-a_0)$ under a DU distribution for $t\leq u$.  Denote this particular $G(t)$ by $G^{DU}(t)$.

%In particular, \cite{Fin09} considered the class of nonlinear procedures that choose threshold $\hat{t}_{\alpha,M}^*$ as one of the solutions to $\hat{F}_M(t) = f_{\alpha}(t)$, where $$f_{\alpha}(t) = \frac{t}{t(1-\alpha) + \alpha}$$ is the nonlinear rejection curve and $\hat{F}_M(t)$ is the empirical CDF of the $p$-values.  %In our notation, we may define $\hat F_M(t) = R(t\bm{1}_M)/M$; see \cite{Hab12} for a formal justification.
%The class of step-up-down (SUD) procedures introduced in \cite{Tam98} was considered for choosing $\hat{t}_{\alpha,M}^*$ and shown to often provide $\alpha$-exhaustion, the notable exception being the step-up procedure which uses the largest solution to $\hat{F}_{M}(t) = f_{\alpha}(t)$.  This procedure fails because the largest solution is $1$, and hence corresponds to automatically rejecting all null hypotheses. %In short, \textit{step-up} procedures cannot be based on $f_\alpha(t)$.  This subtle result is important because step-up procedures are generally more powerful than other step-wise% procedures based on the same rejection curve due to the fact that they utilize the largest possible solution.


To study asymptotic FDP of the WAMDF we consider
\begin{equation}\lim_{M\rightarrow\infty}FDP_M(\hat{t}_{\alpha,M}^0\mat{w}_M)  \leq  \lim_{M\rightarrow\infty}FDP_M(\hat{t}_{\alpha,M}^\lambda\mat{w}_M) \label{asymptotic FDP eqn1} \leq \alpha
\end{equation}
and the following three claims regarding the inequalities:
\begin{enumerate}
\item[(C1)] the first inequality in (\ref{asymptotic FDP eqn1}) is satisfied almost surely,
%\item[(C2)] the equality in (\ref{asymptotic FDP eqn2}) is satisfied almost surely,
\item[(C2)] the second inequality in (\ref{asymptotic FDP eqn1}) is satisfied almost surely, and
\item[(C3)] the second inequality in (\ref{asymptotic FDP eqn1}) is an equality almost surely under a DU distribution.
\end{enumerate}
Informally, Claim (C1) states that the FDP of the WAMDF is asymptotically always larger than the FDP of its unadaptive counterpart and is referred to as the \textit{asymptotically less conservative} claim.  Claim (C2) states that the WAMDF has asymptotic FDP that is less than or equal to $\alpha$ and is referred to as the \textit{asymptotic FDP control} claim.  Claim (C3) is the $\alpha$\textit{-exhaustive} claim and states that the asymptotic FDP of the WAMDF is equal to $\alpha$ under a DU distribution.   Theorem \ref{asymptotic FDP} provides sufficient conditions for each claim.
\begin{theorem} \label{asymptotic FDP}
Fix $\alpha\in (0,1)$ and suppose that (A2) and (A4) - (A6) are satisfied.  Then Claim (C1) holds.  Claim (C2) holds if, additionally, $\mu_0 \leq 1$.  Claim (C3) holds for $0<\alpha\leq FDP_\infty(u)$ if, additionally, $\mu_0 = 1$.
\end{theorem}

Observe that asymptotic FDP control (C2) and $\alpha$-exhaustion (C3) depend on the unobservable value of $\mu_0$, which will necessarily depend on the weighting scheme at hand. In particular, FDP control occurs when $\mu_0 \leq 1$ while $\alpha$-exhaustion occurs when $\mu_0 = 1$.  To motivate the next Theorem, which deals with $\mu_0$, recall in Example \ref{example1} that an optimal fixed-t weight is increasing in $p_m$.  In other words, an optimal weight under Model \ref{randeffmodel} is positively correlated with $\theta_m$, which indicates the state of $H_m$.  It turns out that this positive correlation condition is enough to imply that $\mu_0\leq 1$ and hence is useful for verifying asymptotic FDP control.  In fact, if weights are uncorrelated with the states of the null hypotheses, then $\mu_0 = 1$ and $\alpha$-exhaustion is achieved.  Examples of such weighting schemes are provided later.
\begin{theorem}\label{EFDR}
Suppose that $(W_{m,M} , \theta_{m,M}), m \in\mathcal{M}$ are identically distributed random vectors each with support $\Re^+\times \{0,1\}$ and with $E[W_{m,M}]=1$ and $E[\theta_{m,M}] \in (0,1)$.  Define,
$$\bar{W}_{0,M} = \frac{\sum_{m\in\mathcal{M}}(1-\theta_{m,M})W_{m,M}}{\sum_{m\in\mathcal{M}}(1-\theta_{m,M})}$$
whenever $\bm{\theta}_M \neq \bm{1}_M$ and take $\bar{W}_{0,M} = 1$ otherwise.
Further assume that $\bar{W}_{0,M}\rightarrow \mu_0$ almost surely.  Then $\mu_0\leq 1$ if $Cov(W_{m,M},\theta_{m,M})\geq 0$ and $\mu_0 = 1$ if $Cov(W_{m,M},\theta_{m,M}) = 0$.
\end{theorem}

Before focusing on the asymptotically optimal WAMDF, we first provide Corollary \ref{STS opt}, which is now easily established using Theorems \ref{asymptotic FDP} and \ref{EFDR}.  It states that Claims (C1) - (C3) are satisfied for the unweighted adaptive (linear step-up) procedure defined in \cite{Sto04}, which is denoted $\bm{\delta}(\hat{t}_{\alpha,M}^\lambda\bm{1}_M)$ here.  That is $\bm{\delta}(\hat{t}_{\alpha,M}^\lambda\bm{1}_M)$ is less conservative than $\bm{\delta}(\hat{t}_{\alpha,M}^0\bm{1}_M)$ and provides asymptotic $\alpha$-exhaustive FDP control.
\begin{corollary}\label{STS opt}
Suppose that (A4) - (A6) are satisfied and take $\mat{w}_M = \bm{1}_M$.  Then for any fixed $\lambda\in(0,1)$ and $0 < \alpha\leq a_0$, Claims (C1) - (C3) hold.
\end{corollary}
This Corollary, in particular Claim (C3), suggests that the procedure in \cite{Sto04} is competitive with $\alpha$-exhaustive nonlinear procedures in \cite{Fin09}.  The fact that a DU distribution is the least favorable among such (unweighted) adaptive linear step-up procedures under our weak dependence structure is also interesting as the search for least favorable distributions remains a challenging problem, especially when considering step-up procedures; step-up procedures fail for certain nonlinear procedures in \cite{Fin09} because they utilize threshold $t = 1$ and consequently result in asymptotic FDP equal to $a_0$.  See, \cite{Fin07, RoqVil11, Fin12} for more on nonlinear procedures and least favorable distributions, and see \cite{Tam98} for formal definitions of step-up, step-down and step-up-down procedures.


%It is important to note that, when defining optimality in terms of some type of power, $\alpha$-exhaustion is generally necessary but rarely sufficient.  In fact, we will see that $\bm{\delta}(\hat{t}_{\alpha,M}\mat{w}_M)$ will be ``more powerful'' than the $\alpha$-exhaustive MDF $\bm{\delta}(\hat{t}_{\alpha,M}\bm{1}_M)$ for well chosen $\mat{w}_M$, even in settings when the former MDF need not exhaust the $\alpha$.

\subsection{Asymptotically optimal weights}
\subsubsection{FDP control}
We first verify that the conditions allowing for the WAMDF to provide less conservative asymptotic FDP control (Claims (C1) and (C2)) are satisfied under Model \ref{randeffmodel}, even if the asymptotically optimal weights are perturbed or ``noisy''. As in the previous subsection, weight vectors and elements of weight vectors are indexed by $M$ to facilitate asymptotic arguments.  Further, we sometimes write $\bar{t}_M(k_M^*) = \bar{t}_M(k_M^*,\mat{p},\bm{\gamma})$ for brevity.

Perturbed weights are simulated by multiplying each asymptotically optimal weight by a positive random variable $U_m$ so that, by the law of iterated expectation, perturbed weights are positively correlated with asymptotically optimal weights.  They are formally defined by
\begin{equation}
\tilde{w}_{m,M}(k_M^*,\mat{p},\bm{\gamma}) = U_m w_{m,M}(k_M^*,\mat{p},\bm{\gamma})% = U_m \frac{t_m(k_M^*/p_m,\gamma_m)}{\bar{t}_M(k_M^*,\mat{p},\bm{\gamma})},
\label{perturbed w}
\end{equation}
for each $m$.  For short, a perturbed weight is often denoted by $\tilde{w}_{m,M}$ and the vector of perturbed weights is denoted by $\tilde{\mat{w}}_M(k_M^*,\mat{p},\bm{\gamma})$ or simply by $\tilde{\mat{w}}_M$.  To allow for (A2) to be satisfied we assume that each triplet $(U_m, \gamma_m,p_m)$ has joint distribution satisfying $0\leq U_mt_m(k_M^*/p_m,\gamma_m) \leq 1$ almost surely.  Further, assume that $E[U_m|\mat{p},\bm{\gamma}] = 1$ for each $m$ so that perturbed weights have mean 1.    It should be noted $\tilde{\mat{w}}_M = \mat{w}_M^*$ if $U_m = 1$ for each $m$ (almost surely).  Hence, results regarding perturbed weights immediately carry over to asymptotically optimal weights.

%Recall that assumptions (A4) - (A6) focus on thresholds $0\leq t\leq u$ for $u$ defined as in (\ref{u}) so that $tw_m\in[0,1]$ for every $m$.  Observe that when dealing with perturbed weights, $0\leq t\tilde{w}_{m,M}(k_M^*) \leq 1$ almost surely for $0\leq t\leq \bar{t}_M(k_M^*)$ because $\bar{t}_M(k_M^*)\tilde{w}_{m,M}(k_M^*) = U_mt_m(k_M^*/p_m,\gamma_m)$ by (\ref{perturbed w}) and $0\leq U_mt_m(k_M^*/p_m,\gamma_m)\leq 1$ almost surely by construction.  Hence, $u\geq \bar{t}_M(k_M^*)$ almost surely.

Theorem \ref{optimal conditions} is formally stated below.  Note that we assume $\Pr(p_m\leq 1-\alpha) = 1$ in Model \ref{randeffmodel} so that $\alpha\leq 1-p_{(M)}$ for every $M$ with probability 1, which implies that asymptotically optimal weights and their perturbed versions exist; see Theorem \ref{exist}.
\begin{theorem} \label{optimal conditions}
Suppose that $\Pr(p_m\leq 1-\alpha) = 1$, take $\lambda_M = \bar{t}_M(k_M^*)$, and consider perturbed weights $\tilde{\mat{w}}_M$.  Under Model \ref{randeffmodel} and (A1), (A2) and (A4) - (A6) are satisfied and $\mu_0\leq 1$.  Hence the conditions of Theorem \ref{asymptotic t} are satisfied and Claims (C1) and (C2) hold.
\end{theorem}
%In fact, we can consider a more extreme setting, in which weights are generated independently of $(\bm{\gamma}, \mat{p},\bm{\theta})$ and hence are independent of optimal weights.  In particular, Theorem \ref{indpendent weights} states that $\alpha$-exhaustive asymptotic FDP control is provided in this setting.
%\begin{theorem}\label{independent weights}
%Suppose Model \ref{randeffmodel} and (A1) are satisfied and let $\mat{w}$ be a collection of mutually independent weights with support over $[0,u]$ and with $E[w_m] = 1$ for each $m$.  Further suppose that $\mat{w}$ is independent of $(\bm{\gamma},\bm{p})$.  Then for any fixed $t\in (0, 1/u]$, the conditions of Theorem \ref{asymptotic t} and Claims (C1) - (C3) in Theorem \ref{asymptotic FDP} are satisfied.
%\end{theorem}


\subsubsection{Optimality}

In this subsection, we formally describe what is meant by ``asymptotically optimal'' and also provide some examples of $\alpha$-exhaustive weighting schemes.  To motivate the first Theorem, recall that the asymptotically optimal weights in the Weight Selection Procedure are equivalent to optimal fixed-$t$ weights with $t = \bar{t}_M(k_M^*)$.  However, the asymptotically optimal WAMDF utilizes asymptotic threshold $t_{\alpha,\infty}^\lambda$ (see Theorem \ref{asymptotic t}).  The next theorem states that $\bar{t}_M(k_M^*)\rightarrow t_{\alpha,\infty}^\lambda$ almost surely, i.e. the asymptotically optimal weights are asymptotically equivalent to the optimal fixed-t weights corresponding to Theorem \ref{opt solution}.  Note that we choose $\lambda_M = \bar{t}_M(k_M^*)$ in the asymptotically optimal WAMDF.  Theorem \ref{asymp opt w} need not apply for other specifications of $\lambda_M$.
\begin{theorem} \label{asymp opt w}
Suppose that $\Pr(p_m\leq 1-\alpha) = 1$ and take $\lambda_M = \bar{t}_M(k_M^*)$.  Then under Model \ref{randeffmodel} and (A1), $\bar{t}_M(k_M^*) \rightarrow t_{\alpha,\infty}^{\lambda}$ almost surely.
\end{theorem}
%

The other notion of optimality considered in this paper is the notion of $\alpha$-exhaustion.  The next two corollaries illustrate that Claims (C1) - (C3) (note in particular the $\alpha$-exhaustion Claim (C3)) are satisfied in wide variety of settings.  Corollary \ref{independent weights} states that a WAMDF is $\alpha$-exhaustive in a worst case scenario weighting scheme, i.e. when weights are generated independent of $\bm{\theta}_M$.  This is perhaps not surprising in light of Corollary \ref{STS opt}, where we saw that even choosing $\mat{w}_M = \bm{1}_M$ results in $\alpha$-exhaustion.
\begin{corollary}\label{independent weights}
Under Model 1 and (A1) - (A2), if $\mat{w}_{M}$ are mutually independent weights and independent of $\bm{\theta}_M$ with $E[w_{m,M}] = 1$, then Claims (C1) - (C2) hold for $\alpha \in (0,1)$ and Claim (C3) holds for $0 < \alpha\leq FDP_\infty(u)$.
\end{corollary}

Recall in Theorem \ref{optimal conditions} that optimal weights and their perturbed versions provide less conservative asymptotic FDP control (Claims (C1) and (C2)) under Model \ref{randeffmodel} and (A1).  In Corollary \ref{asymp opt p}, we see that if $p_i = p_j$ for every $i,j$, then optimal weights and their perturbed versions allow for  $\alpha$-exhaustion as well. This setting arises in practice whenever no information for distinguishing prior probabilities $\mat{p}$ from one another is available, but some information for distinguishing effect sizes from one another may be available. For an illustration see Section \ref{sec 8}.  See also \cite{Spj72, Sto07, Pen11}.
\begin{corollary}\label{asymp opt p}
Suppose that the conditions of Theorem \ref{optimal conditions} are satisfied and consider perturbed weights $\tilde{\bm{w}}_M$. If additionally $p_i = p_j$ for every $i,j$ then Claim (C3) holds for $0<\alpha\leq FDP_{\infty}(u)$.
%$$\mu_0 = \lim_{M_0\rightarrow\infty}\frac{1}{M_0}\sum_{m\in\mathcal{M}_0} \tilde{w}_{m,M}(k,\mat{p},\bm{\gamma}) = 1$$
\end{corollary}

It should be noted that $\alpha$-exhaustion need not be achieved when $p_i\neq p_j$ in Model \ref{randeffmodel} for the asymptotically optimal WAMDF.  Hence, though more powerful than its competitors, such as the $\alpha$-exhaustive unweighted MDF $\bm{\delta}(t_{\alpha,M}^\lambda\bm{1}_M)$,  there is room for additional improvement.  A similar phenomenon was observed in \cite{Gen06} in the unadaptive setting, and it was suggested that one potential route for improvement is to incorporate an estimate of $\mu_0$ into the procedure.  However, it is not clear how this objective could be accomplished without sacrificing FDP control, especially when weights may be perturbed.  We leave this as future work.

\section{Simulation} \label{sec 7}

This section compares weighted adaptive MDFs to other MDFs in terms of power and FDP control via simulation.  In particular, for each of $K = 1000$ replications, we generate $Z_m\stackrel{i.i.d.}\sim N(\theta_m\gamma_m,1)$ for $m = 1, 2, ..., 1000$ and compute $\bm{\delta}(\hat{t}_{\alpha,M}^{\lambda}\mat{w}_M)$, $\bm{\delta}(\hat{t}_{\alpha,M}^{0}\mat{w}_M)$, $\bm{\delta}(\hat{t}_{\alpha,M}^{\lambda}\bm{1}_M)$, and $\bm{\delta}(\hat{t}_{\alpha,M}^{0}\bm{1}_M)$ as in Example \ref{example1}, where $\alpha = 0.05$ and $\lambda_M = \bar{t}_M(k_M^*,\mat{p},\bm{\gamma})$. Recall these MDFs are referred to as weighted adaptive (WA), weighted unadaptive (WU), unweighted adaptive (UA), and unweighted unadaptive (UU) procedures, respectively.   The WU procedure is akin to the weighted BH procedure in \cite{Gen06} while the UA procedure is the adaptive BH procedure in \cite{Sto04}.  The UU procedure is the original BH procedure.  The average FDP and average correct discovery proportion (CDP) is computed over the $K$ replications for each procedure, where the CDP is defined by $CDP = \sum_{m\in\mathcal{M}_1}\delta_m/\max\{M_1, 1\}$.

In each simulation experiment, $\gamma_m\stackrel{i.i.d.}\sim Un(1, a)$ for $a = 1, 3, 5$, where $Un(1,a)$ denotes a uniform distribution over $(1,a)$.  Observe that when $a = 1$ the effect sizes are identical while when $a = 3$ or $a = 5$ they vary.  In Simulation 1, we take $p_m = 0.5$ for each $m$ and weighted procedures utilize asymptotically optimal weights.  The data generating mechanism here is similar to settings considered in \cite{Spj72}, \cite{Sto07}, and \cite{Pen11} in that no distinguishing prior probabilities for the states of the $H_m$s are available and results in the WAMDF being both optimally weighted and $\alpha$-exhaustive; see Corollary \ref{asymp opt p}.  This setting also arises in the analysis of the data in Table \ref{data} in the next section.  In Simulation 2, weighted procedures use asymptotically optimal weights as before and the effect sizes vary as before, but the prior probabilities now vary via $p_m\stackrel{i.i.d.}\sim Un(0,1)$.  Thus, the conditions of Claim (C3) are no longer satisfied and the WA procedure is optimally weighted but not $\alpha$-exhaustive (recall the last paragraph of Section \ref{sec 6}).  In Simulation 3, data are generated according to the same mechanism as in Simulation 2, but asymptotically optimal weights are perturbed via $U_m w_{m,M}(k_M^*,\mat{p},\bm{\gamma})$, where $U_m \stackrel{i.i.d.}\sim Un(0,2)$; see expression (\ref{perturbed w}).  Hence, the WA procedure is no longer optimally weighted nor is it $\alpha$-exhaustive, but the employed weights are positively correlated with optimal weights.  Simulation 4 considers a worst case scenario in which weights are generated $w_{m,M}\stackrel{i.i.d.}\sim Un(0,2)$.  Even in this worst case scenario $\alpha$-exhaustion is achieved for the WA procedure by Corollary \ref{independent weights}.  Note that the UA procedure is $\alpha$-exhaustive in all four simulations due to to Corollary \ref{STS opt} while the unadaptive procedures are never $\alpha$-exhaustive.  Results are summarized in Table \ref{simul table}.
\begin{table}[h!]\center \scriptsize
\caption{\label{simul table} The average CDP (FDP) for the UU, UA,  WU, and WA procedures in Simulations  1 - 4.}
\begin{tabular}{cccc}
\multicolumn{4}{c} {Simulation 1}  \\ \hline
\hline & \multicolumn{3}{c}{a}\\ \cline{2-4}
& 1 & 3 & 5 \\ \cline{2-4}
UU &0.007(0.021) &0.390(0.025) & 0.709(0.025)\\
WU &0.007(0.021) &0.397(0.025) & 0.731(0.025)\\
UA &0.007(0.021) &0.437(0.031) & 0.761(0.039) \\
WA &0.007(0.021) &0.442(0.032) & 0.793(0.039)\\ \hline \\

\multicolumn{4}{c} {Simulation 2}  \\ \hline
\hline & \multicolumn{3}{c}{a}\\ \cline{2-4}
& 1 & 3 & 5 \\ \cline{2-4}
UU & 0.007(0.024) & 0.390(0.025) & 0.709(0.025)\\
WU & 0.011(0.008) & 0.434(0.014) & 0.756(0.016)\\
UA & 0.007(0.024) & 0.430(0.031) & 0.761(0.041)\\
WA & 0.011(0.008) & 0.473(0.018) & 0.814(0.026)\\ \hline \\

\multicolumn{4}{c} {Simulation 3} \\ \hline
\hline &  \multicolumn{3}{c}{a}\\ \cline{2-4}
& 1 & 3 & 5 \\ \cline{2-4}
UU & 0.007(0.023)& 0.391(0.025) & 0.709(0.025)\\
WU & 0.013(0.007) & 0.404(0.015) & 0.719(0.016) \\
UA & 0.007(0.023)& 0.430(0.031) & 0.757(0.039) \\
WA & 0.013(0.007)& 0.439(0.019) & 0.774(0.027) \\ \hline \\

\multicolumn{4}{c} {Simulation 4} \\ \hline
\hline &  \multicolumn{3}{c}{a}\\ \cline{2-4}
& 1 & 3 & 5 \\ \cline{2-4}
UU & 0.007(0.025)& 0.391(0.025) & 0.710(0.025)\\
WU & 0.006(0.023) & 0.354(0.025) & 0.682(0.025) \\
UA & 0.007(0.025)& 0.425(0.030) & 0.756(0.039) \\
WA & 0.006(0.023)& 0.387(0.030) & 0.727(0.039) \\ \hline \\
\end{tabular}
\end{table}

The main important point is that the WA procedure dominates all other procedures as long as the employed weights are at least positively correlated with the optimal weights, and it performs nearly as well as other procedures otherwise.    In particular, its FDP is less than or equal to $0.05$ in all simulations and its average CDP is as large or larger than the CDP of all other procedures in the first three simulations.  The WA procedure does have a smaller average CDP than the UA procedure in the worst case scenario (Simulation 4), as we might expect.

Now let us focus on Simulation 1 in more detail.  First, observe that the FDP is increasing in $a$ for both adaptive procedures.  For example the FDP of each procedure is $0.021$ when $a=1$ but is $0.039$ when $a = 5$.  This is to be expected as both adaptive procedures are $\alpha$-exhaustive (see Corollaries \ref{STS opt} and \ref{asymp opt p}) and hence we should expect the FDP to be near 0.05 in high power settings, i.e. for large $a$.  Additionally, the largest gain in power (in terms of the average CDP) of the weighted adaptive procedure over the unweighted adaptive procedure occurs when effect sizes are most heterogeneous.  In particular, when $a = 5$ the average CDP of the WA procedure is $0.793$ while the average CDP of the UA procedure is $0.761$.  When data are homogeneous (a = 1), the CDPs of the procedures are identical.

In Simulation 2, data generating mechanisms are even more heterogeneous as now the $p_m$s also vary.  General conclusions regarding the CDP are the same, with the advantages of the weighted procedures over their unweighted counterparts being more pronounced. For example, the average CDP of the WAMDF for $\gamma_m\stackrel{i.i.d.}\sim Un(1,5)$ increased from 0.793 to 0.814 when allowing $p_m$s to vary, while for the UA procedure the CDP is still 0.761.  %In general, the gain in power of optimally weighted procedures over unweighted procedure should be more pronounced when the degree of heterogeneity is large and less pronounced when data are nearly homogeneous.  %For example, generating $\gamma_m$ via $\Pr(\gamma_m = 1) = \Pr(\gamma_m = 5) = 1/2$ in Simulation 2 rather than generating $\gamma_m$ from a Un(1,5) distribution resulted in an even greater gain in power for the WAMDF over the UA procedure.  The average CDPs were 0.635 and 0.575, respectively.
We also observe that for $a = 5$ the average FDP of the WA procedure is only $0.026$ while the average FDP of the UA procedure is closer to 0.05; it is 0.039.  This is to be expected because, even though the WAMDF will dominate the UA procedure in terms of the average CDP, the UA procedure is $\alpha$-exhaustive while the WAMDF need not be in this setting.

Now consider non-optimal weights in Simulations 3 and 4.  \cite{RoeWas09} concluded that, in the unadaptive setting (the UU and WU procedures), weighted MDFs are robust with respect to weight misspecification in that they generally yield about as many or more rejected null hypotheses as unweighted procedures as long as weights are ``reasonably guessed'' and yield slightly less rejected null hypotheses when weights are poorly guessed. Simulations 3 and 4 confirm their results and further illustrates that the robustness property applies to adaptive procedures.  For example, comparing the unadaptive procedures in Simulation 3, we see that the average CDP of the WU(UU) procedures are 0.013(0.007), 0.404(0.391), and 0.719(0.709) for $a = 1, 3, 5$, respectively.  The average CDP of the WA(UA) procedure is 0.013(0.007), 0.439(0.430), and 0.774(0.757), for $a = 1, 3, 5$, respectively.  That is, when weights are positively correlated with optimal weights, weighted procedures still perform slightly better than their unweighted counterparts.  In the worst case scenario setting in Simulation 4, where weights are independently generated, the FDP is still controlled by the WA procedure, but some loss in power over its unweighted counterpart is observed.  For example, the CDP of the WA(UA) procedure is 0.006(0.007), 0.386(0.425), and 0.727(0.756) when $\gamma = 1, 3,$ and  5, respectively, while the average FDP of the WA(UA) procedure is 0.025(0.023), 0.030(0.030), and 0.039(0.039) when $\gamma = 1, 3, $ and $5$, respectively.  Again the $\alpha$-exhaustion phenomenon for both adaptive procedures is observed in Simulation 4, as Corollaries \ref{STS opt} and \ref{independent weights} suggest.



\section{Data analysis}\label{sec 8}
Next we illustrate the method on the data set outlined in Table \ref{data}. Recall that the goal is to determine which bacteria are positively associated with shoot biomass via the testing of $H_m:\beta_{1m}=0$ vs. $K_m:\beta_{1m}>0$ for each $m$, where $\beta_{1m}$ is the regression coefficient for regressing $\mat{Y}_m = (Y_{im}, i = 1, 2, ..., 5)$ on $\mat{x} = (x_i, i = 1, 2, ..., 5)$ with a log-linear model defined by log$(\mu_{im}) = \beta_{0m} + \beta_{1m}x_i$ for $\mu_{im}$ the mean of $Y_{im}$.  To define the problem within the context of Model \ref{randeffmodel}, assume $\beta_{1m} = \beta\theta_m$ for some $\beta>0$ so that the null hypothesis can be written $H_m:\theta_m = 0$.   Further assume $\mat{Y}_m|(\sum_{i=1}^5Y_{im} = n_m)$ has a multinomial distribution with sample size parameter $n_m$ and probability vector $\mat{p}_m = (p_{im}, i = 1, 2, ..., 5).$ Note that under the log linear model with $\beta_{1m} = \beta\theta_m$, $p_{im} = e^{\beta\theta_m x_i}/[\sum_{i = 1}^5 e^{\beta\theta_m x_i}]$.  Further, $p_{im} = 1/5$ if $\theta_m = 0$.

The $p$-value based on test statistic $T_m = \mat{x}^T\mat{Y}_m$ is defined $P_m = \Pr(T_m\geq t_m|\sum_{i=1}^5Y_{im} = n_m, \theta_m = 0)$.  See \cite{McCNel89} for details.  The power function for $\delta_m(P_m;t_m) = I(P_m\leq t_m)$ is approximated with the power function expression in Example \ref{example1}, where $\gamma_m = \sqrt{n_m} K(\beta)$ for $K(\beta)$ some positive constant depending on $\beta$.  For details on this approximation see the Supplement. The important point is that effect sizes are proportional to $\sqrt{n_m}$.  It is anticipated in \cite{AndHab12} that roughly 1/2 of bacteria will be positively associated with shoot biomass and that, of these, about 1/2 will be detected.  Thus, we choose $p_m = 1/2$ for each $m$ in Model \ref{randeffmodel} and choose $K(\beta)$ so that average power $M^{-1}\sum_m\pi_{\gamma_m}(t_m(k_M^*/p_m, \gamma_m)) = 1/2$.  %, where for any given $K(\beta)$, $k_M^*$ is found as in the Weight Selection Procedure.

\begin{figure}[h!]
\epsfig{file = powdata, width = 5.0in, height = 3.6in}
\caption{\label{powdata}The left panel plots $w_m^*$ and $w_m = 1$ vs. $\gamma_m$, denoted by o and $\triangle$, respectively.  The right panel plots $\pi_{\gamma_m}(tw_m^*)$ and $\pi_{\gamma_m}(t)$ vs. $\gamma_m$, denoted by o and $\triangle$, respectively, where $t = \bar{t}_M(k_M^*)$ is computed as in the Weight Selection Procedure. Not depicted, for readability, are weights and posited power for four $H_m$s with sample sizes 333, 457, 883, 911.  These posited powers and weights were all 1 and 0, respectively.}
\end{figure}
The asymptotically optimal weights and the (posited) power of the unweighted and weighted procedures, given by $\pi_{\gamma_m}(t)$ and $\pi_{\gamma_m}(tw_m^*)$ for $t = \bar{t}_M(k_M^*)$, respectively, are in Figure \ref{powdata}.  Observe that $w_m^*>1$ for the first six points (which actually represents 558 of the 778 hypothesis tests with the smallest sample sizes - many of the $n_m$'s, and consequently the $w_m^*$s, are identical and hence each point represents multiple weights) and resulted in increased power; see the right panel of Figure \ref{powdata}.  Optimal weights for the remaining 220 hypotheses are less than 1 and can even be near 0.  However, in these settings, the power of the weighted decision function is still very near 1.  In short, the optimal strategy borrows weight from tests where the power is anticipated to be near 1, even if the weight is near 0, and redistributes it to tests where increasing the weight can have a more substantial effect on power.  Indeed, for $\alpha = 0.05$ the asymptotically optimal WAMDF resulted in 38 discoveries whereas the unweighted adaptive procedure yielded only 32 discoveries.  While the FDP is unobservable, Corollary \ref{asymp opt p} suggests that it is less than or equal to $0.05$, even if the assumptions under which optimal weights were computed need not be correct.

\section{Concluding Remarks} \label{sec 9}

Efforts to improve upon the original BH procedure have focused on 1) controlling the FDR at a level nearer $\alpha$ or 2) exploiting potential heterogeneity across tests.  This paper combined these ideas using a weighted decision theoretic framework and showed that the resulting procedure is more powerful than procedures which only consider 1) or 2), but not both.  Specifically, we have provided weighted adaptive multiple decision functions that satisfy the $\alpha$-exhaustive optimality criterion considered in \cite{Fin09}, but allow for further improvements via an optimal weighting scheme that incorporates heterogeneity.  For example, in Corollary \ref{STS opt} we saw that the unweighted adaptive procedure is $\alpha$-exhaustive.  However, its weighted version is even more powerful when data are heterogeneous and optimal weights are used; see Theorems \ref{opt solution} and \ref{asymp opt w} and the CDP of the weighted adaptive (WA) vs. the unweighted adaptive (UA) procedures in Simulations 1 and 2 in Section \ref{sec 7}.  In fact, even when weights are not optimal, we saw in Simulation 3 that some gain in power is achieved.

Finite sample results and asymptotic results in this paper are valid under independence and weak dependence conditions, respectively.  %In general, FDR control under stronger dependence assumptions is a challenging problem.
\cite{BenYek01} showed that the unweighted unadaptive BH procedure provides (finite) FDR control under a certain positive dependence structure, and can be modified to control the FDR for arbitrary dependence.  It would be interesting to study the performance of weighted adaptive procedures under arbitrary dependence.  However, obtaining finite sample analytical results, even in the unweighted adaptive setting, appears to be very challenging, especially under arbitrary dependence.  See \cite{GuoSar12} for some results regarding unweighted procedures under a block dependence-type structure.  As for large sample results, \cite{Fan12, DesSto12} provide techniques for transforming test statistics so that they are weakly dependent.  It may be possible to construct weighted adaptive MDFs which make use of these transformed  statistics, but this also requires further study.

%Throughout this paper we have implicity assumed that each test statistics $Z_m$ has a continuous distribution.  While this need not be the case in practice, such as when data are dichotomous in nature or when utilizing nonparametric rank-based procedures, we may, with the aid of an independent auxiliary random variable, transform statistics via $Z_m^* = T(Z_m,U_m)$ so that $Z_m^*$ has a continuous distribution. See \cite{Ste50, Pen11, HabPen11} for details.

Optimal weighting schemes were derived assuming $\bm{\gamma}$ and $\mat{p}$ were known, i.e. we assumed that the nature and degree of heterogeneity is known. In some settings, consistent estimates of these parameters are readily available and WA procedures are easily implemented. See \cite{SunCai09}, \cite{Hu10}, or \cite{SunMcL12} for examples.  In other settings, consistent estimates may not be available and the nature and degree of heterogeneity may not be precisely known.  However, asymptotic FDP control is still provided even if heterogeneity is poorly modeled/posited (see, in particular, Theorem \ref{optimal conditions} and Corollary \ref{independent weights}) and as long as the weighting scheme is reasonable, then some gain in power should be anticipated (see Simulation 3).  Hence, heterogeneity should be incorporated into the multiple testing procedure even if only a reasonable guess for unknown parameters may be available, as in Section \ref{sec 8}, where effect sizes were posited based on the assumption that tests with larger sample sizes are more powerful.

Other estimators for $M_0$ could be considered.  For example, it is possible to use the unweighted estimator from \cite{Sto04} in the WAMDF. % or to consider other specifications of the tuning parameter $\lambda$.
One reason for using the weighted estimator is that this simplifies analytical arguments and the implementation of the WAMDF.  In particular, as we saw in Section \ref{sec 4}, it allows for the WAMDF to be implemented by applying the adaptive BH procedure in \cite{Sto04} to the weighted $p$-values, and facilitates martingale arguments for finite sample results.  Further, under a DU distribution and the dependence structure in (A3), it can be verified that the weighted estimator performs better than the unweighted estimator in that both estimators are consistent but $Var(\hat{M}_0(\lambda\bm{1}))\geq Var(\hat{M}_0(\lambda\mat{w}))$. The latter claim is due to \cite{Hoe56}, where it was shown that the variance of the sum of $n$ independent Bernoulli random variables with average success probability $n^{-1}(p_1 + p_2 + ... +p_n) = p$ is maximized when $p_1 = p_2 = ... = p_n = p$. Of course, this does not imply that the weighted estimator will always perform better than the unweighted estimator.  However, a more detailed assessment of $\hat{M}_0(\lambda\mat{w})$, though warranted, is beyond the scope of the current manuscript.   %Also, for a detailed study of the optimal choice of $\lambda$ in the unweighted setting see \cite{LiaNet12} .

The robustness properties of the asymptotically optimal WAMDF suggest that simpler weighting schemes may still allow for improvements over unweighted adaptive procedures and hence, in some settings, may be an attractive alternative to the more complex asymptotically optimal weighting scheme defined in Section \ref{sec 3}.  For example, \cite{Gen06} considered a binary weighting scheme where each hypothesis was given one of two possible weights and demonstrated that the resulting weighted (unadaptive) procedure could still be more powerful than its unweighted counterpart. It would be interesting to study simpler weighting schemes within the context of adaptive procedures.

\newpage
\bibliographystyle{imsart-nameyear}
\bibliography{FDRweight}
\newpage

\renewcommand{\theequation}{S\arabic{equation}}
\setcounter{equation}{0}
\renewcommand{\thefigure}{S\arabic{figure}}
\setcounter{figure}{0}
\section{Supplementary Material}

\subsection{Proofs of results in Section \ref{sec 3}}\hspace{1in}\\

\noindent\textsc{Proof of Theorem \ref{opt solution}.}
Setting up the Lagrangian
$$L(\mat{t},k) = \pi(\mat{t},\mat{p},\bm{\gamma}) - k\left[\left(\sum_{m\in\mathcal{M}}t_m\right) - Mt\right]$$
and taking derivative with respect to $t_m$ and setting it equal to 0 yields equation (\ref{derivative}).   Now, recall we denote the solution to equation (\ref{derivative}) with respect to $t_m$ by $t_m(k/p_m,\gamma_m)$ and observe $k\mapsto t_m(k/p_m,\gamma_m)$ is continuous and strictly decreasing in $k$ with $\lim_{k\rightarrow \infty}t_m(k/p_m,\gamma_m) = 0$ and $\lim_{k\downarrow 0}t_m(k/p_m,\gamma_m) = 1$ by (A1).  Thus, $\bar{t}_M(k,\mat{p},\bm{\gamma}) = M^{-1}\sum_{m\in\mathcal{M}}t_m(k/p_m,\gamma_m)$ is continuous and strictly decreasing in $k$ with  $\lim_{k\rightarrow \infty}\bar{t}_M(k,\mat{p},\bm{\gamma}) = 0$ and $\lim_{k\downarrow 0}\bar{t}_M(k,\mat{p},\bm{\gamma}) = 1$.  Hence, there exists a unique $k$ satisfying $\bar{t}_{M}(k,\mat{p},\bm{\gamma}) = t$ for any $t\in(0,1)$ and hence a unique collection $[t_m(k/p_m,\gamma_m),m\in\mathcal{M}]$.

To show that the solution is a maximum, it suffices to show that the sequence of the determinants of the principal minors of the bordered hessian matrix, evaluated at the solution, alternates in sign.  The $j$th principle minor of the bordered Hessian matrix is
$$\mat{H}_{j} = \left[\begin{array}{cc}0 & \mat{1}_j\\ \mat{1}_j^T& \mat{D}_j \end{array}\right]$$
where $\mat{D}_j$ is a $j\times j$ diagonal matrix with diagonal elements $d_m = \pi_{\gamma_m}''(t_m)$ and $\bm{1}_j$ is a vector of $1$s of length $j$.  Note that $d_m<0$ at the solution due to (A1).  Now, observe that $|\mat{H}_1| = -1 <0$ where $|\cdot|$ denotes the determinant, and for $j\geq 2$, we have the recursive relation
\begin{equation}
|\mat{H}_j| = d_j|\mat{H}_{j-1}| + (-1)^j\prod_{m=1}^{j-1}(-d_m).\label{H}
\end{equation}
Because $d_j<0$, for $j$ an even (odd) integer each expression on the righthand side of equation (\ref{H}) is positive (negative).  Hence $\{|\mat{H}_j|, j = 1, 2, ...\}$ alternates in sign. The above arguments hold with probability 1. \qed \\

\noindent\textsc{Proof of Theorem \ref{exist}.} Observe that $\widetilde{FDP}_M(\mat{t}(k,\mat{p},\bm{\gamma}))$ is continuous in $k$ under (A1).  Hence, it suffices to show that $\widetilde{FDP}_M(\mat{t}(k,\mat{p},\bm{\gamma}))$ takes on values $0$ and $1-p_{(M)}$ by the Mean Value Theorem.  We first show that $$\lim_{k\downarrow 0}\widetilde{FDP}_M(\mat{t}(k,\mat{p},\bm{\gamma}))\geq 1 - p_{(M)}.$$  Observe that (A1) implies $t_m\leq \pi_{\gamma_m}(t_m)\leq 1$ for $t_m\in[0,1]$ and hence
\begin{equation}\label{ineq FDRM}
\bar{t}_M(k,\mat{p},\bm{\gamma}) \leq  \bar{G}_M(\mat{t}(k,\mat{p},\bm{\gamma})) \leq M^{-1}\left[\sum_{m\in\mathcal{M}}(1-p_m)t_m(k/p_m,\gamma_m) + p_m\right].
\end{equation}
The inequalities in (\ref{ineq FDRM}) imply
\begin{eqnarray*}
\widetilde{FDP}_M(\mat{t}(k,\mat{p},\bm{\gamma})) & = & \frac{\sum_{m\in\mathcal{M}}[1 - G_m(t_m(k/p_m,\gamma_m))]}{\sum_{m\in\mathcal{M}}[1-t_m(k/p_m,\gamma_m)]}\frac{\bar{t}_M(k,\mat{p}, \bm{\gamma})}{\bar G_M(\mat{t}(k,\mat{p},\bm{\gamma}))}\\
&\geq& \frac{\sum_{m\in\mathcal{M}}[1-p_m][1-t_m(k/p_m,\gamma_m)]}{\sum_{m\in\mathcal{M}}[1-t_m(k/p_m,\gamma_m)]}\frac{\bar{t}_M(k,\mat{p}, \bm{\gamma})}{\bar G_M(\mat{t}(k,\mat{p},\bm{\gamma}))}\\
& \geq & \left(1-p_{(M)}\right)\frac{\bar{t}_M(k,\mat{p},\bm{\gamma})}{\bar G_M(\mat{t}(k,\mat{p},\bm{\gamma}))},
\end{eqnarray*}
which converges to $1 - p_{(M)}$ as $k\downarrow 0$ if \begin{equation}
\label{ratio}
\frac{\bar{t}_M(k,\mat{p},\bm{\gamma})}{\bar G_M(\mat{t}(k,\mat{p},\bm{\gamma}))} \rightarrow 1
\end{equation}
as $k\downarrow0$.  To verify (\ref{ratio}), observe that $\lim_{k\downarrow 0}t_m(k/p_m,\gamma_m)= 1$ by (A1) and hence $\bar{t}_M(k, \mat{p},\bm{\gamma}) \rightarrow 1$ as $k\downarrow0$.  This, along with the inequalities in (\ref{ineq FDRM}), imply $\bar G_M(\mat{t}(k,\mat{p},\bm{\gamma}))\rightarrow 1$ as $k\downarrow 0$ and hence (\ref{ratio}) is satisfied.

Now if
\begin{equation} \label{c1}
\lim_{k\rightarrow \infty}\frac{\bar{t}_M(k,\mat{p},\bm{\gamma})}{\bar G_M(\mat{t}(k,\mat{p},\bm{\gamma}))}=0,
\end{equation} then by the first inequality in (\ref{ineq FDRM}) and the definition of $\widetilde{FDP}_M(\mat{t}(k,\mat{p},\bm{\gamma}))$
$$\widetilde{FDP}_M(\mat{t}(k,\mat{p},\bm{\gamma}))\leq \frac{\bar{t}_M(k,\mat{p},\bm{\gamma})}{\bar G_M(\mat{t}(k,\mat{p},\bm{\gamma}))}\rightarrow 0$$
as $k\rightarrow \infty$ and the proof would be complete. Hence, it suffices to show (\ref{c1}).
But because $t_m(k/p_m,\gamma_m)\downarrow 0$ as $k\rightarrow \infty$ and $\pi_{\gamma_m}'(t_m)\rightarrow \infty$ as $t_m\downarrow 0$ by (A1), we have $$\frac{\pi_{\gamma_m}(t_m(k/p_m,\gamma_m))}{t_m(k/p_m,\gamma_m)}\rightarrow \infty$$ as $k \rightarrow \infty$ by H{\^o}pital's rule.  Further for $a_m$, $b_m$,  $m\in\mathcal{M}$ any positive constants,
$$\frac{\sum_{m\in\mathcal{M}}a_m}{\sum_{m\in\mathcal{M}}b_m} = \sum_{m\in\mathcal{M}}\frac{a_m}{b_m}\left(\frac{b_m}{\sum_{m\in\mathcal{M}}b_m}\right)\geq \min\left\{\frac{a_m}{b_m}, m\in\mathcal{M}\right\}.$$
Hence,
$$A(k)\equiv \frac{\sum_{m\in\mathcal{M}}\pi_{\gamma_m}(t_m(k/p_m,\gamma_m))}{\sum_{m\in\mathcal{M}}t_m(k/p_m,\gamma_m)}\geq \min\left\{\frac{\pi_{\gamma_m}(t_m(k/p_m,\gamma_m))}{t_m(k/p_m,\gamma_m)}, m\in\mathcal{M}\right\}\rightarrow \infty$$
as $k\rightarrow \infty$ which implies
\begin{eqnarray*}
\frac{\bar{t}_M(k,\mat{p},\bm{\gamma})}{\bar{G}_M(\mat{t}(k,\mat{p},\bm{\gamma}))}&=& \left[\sum_{m\in\mathcal{M}}\frac{(1-p_m)t_m(k/p_m,\gamma_m)}{\bar{t}_M(k,\mat{p},\bm{\gamma})} + \frac{p_m\pi_{\gamma_m}(t_m(k/p_m,\gamma_m))}{\bar{t}_M(k,\mat{p},\bm{\gamma})}\right]^{-1}\\
&\leq&\left[M(1-p_{(M)}) + M p_{(1)}A(k)\right]^{-1}\rightarrow 0
\end{eqnarray*}
as $k\rightarrow \infty$, where $p_{(1)} \equiv \min\{\bm{p}\}$. Note that the above arguments hold with probability 1.\qed \\

\subsection{Proofs of results in Section \ref{sec 5}}\hspace{1in}\\

\noindent\textsc{Proof of Lemma \ref{bound}.}
The proof uses techniques from the proofs of Theorem 3 in \cite{Sto04} and Theorem 9 in \cite{Pen11,Pen14}. First, observe that because $u = \lambda$, $0\leq \hat{t}_\alpha^\lambda\leq \lambda$ by definition and that if $\hat{t}_\alpha^\lambda = 0$ then $FDR(\hat{t}_\alpha^\lambda\mat{w}) = 0$ trivially.  Let us focus on the setting where $0<\hat{t}_\alpha^\lambda\leq \lambda$.  By the definition of $\hat{t}_\alpha^\lambda$, $\widehat{FDP}^\lambda(\hat{t}_\alpha^\lambda\mat{w})\leq \alpha$ which gives $R(\hat{t}_\alpha^\lambda)\geq \hat{M}_0(\lambda\mat{w}) \hat{t}_\alpha^\lambda/\alpha$ by the definition of $\widehat{FDP}^\lambda(\cdot)$.  Hence,
\begin{eqnarray} %\tag{S1}
FDR(\hat{t}_\alpha^\lambda\mat{w}) &=& E\left[\frac{V( \hat{t}_{\alpha}^\lambda\mat{w})}{R(\hat{t}_\alpha^\lambda\mat{w})}\right]\leq  E\left[\alpha\frac{1}{\hat{M}_0(\lambda\mat{w})}\frac{V(\hat{t}_\alpha^\lambda\mat{w})}{\hat{t}_\alpha^\lambda}\right]\\ &\leq& \label{FDReqn} E\left[\frac{\alpha}{\hat{M}_0(\lambda\mat{w})}\frac{V(\lambda\mat{w})}{\lambda}\right],
\end{eqnarray}
where (\ref{FDReqn}) is established as follows.  First, if $\hat{t}_\alpha^{\lambda} = \lambda$, it is true trivially. Now suppose that $0<\hat{t}_\alpha^{\lambda}<\lambda$. Define filtration $\mathcal{F}_t = \sigma\{\bm{\delta}(s\mat{w}), 0 < t\leq s \leq \lambda\}$ and observe that $\hat{t}_\alpha^{\lambda}$ is a stopping time with respect to $\mathcal{F}_{t}$ (with time running backwards).  Further, for $0< t\leq \lambda$,
$V(t\mat{w})/t$ is a reverse martingale with respect to $\mathcal{F}_{t}$.  This can be verified by noting that for $0 < s\leq t \leq \lambda$
\begin{eqnarray*}
E\left[\frac{V(s\mat{w})}{s}|\mathcal{F}_t\right] &=& \frac{1}{s}\sum_{m\in\mathcal{M}_0} E\left[\delta_m(sw_m)|\mathcal{F}_t\right] \\
&=&\frac{1}{s}\sum_{m\in\mathcal{M}_0} \delta_m(tw_m)E[\delta_m(sw_m)|\delta_m(tw_m)=1, \mathcal{F}_t] \\
&=&\frac{1}{s}\sum_{m\in\mathcal{M}_0} \delta_m(tw_m)E[\delta_m(sw_m)|\delta_m(tw_m)=1]\\
&=&\frac{1}{s}\sum_{m\in\mathcal{M}_0} \delta_m(tw_m)\frac{sw_m}{tw_m}\\
&=&\sum_{m\in\mathcal{M}_0}\frac{\delta_m(tw_m)}{t} \\
&=&\frac{V(t\mat{w})}{t},
\end{eqnarray*}
where first equality follows by the definition of $V(\cdot)$ and the second is due to the fact that $\delta_m(sw_m) = 0$ if $\delta_m(tw_m) = 0$ by the NS assumptions. The third equality is satisfied due to (A3).  The forth equality follows by the fact that $\Pr([\delta_m(sw_m)=1]\cap[\delta_m(tw_m)=1]) = E[\delta_m(sw_m)] = sw_m$ for $m\in\mathcal{M}_0$ and $s\leq \lambda$ under the NS assumptions and under (A2). The forth and fifth equalities follow from some algebra and the definition of $V(\cdot)$, respectively.  Hence, by the law of iterated expectation and the Optional Stopping Theorem \citep{Doo53}
\begin{eqnarray*}
E\left[\frac{\alpha}{\hat{M}_0(\lambda\mat{w})}\frac{V(\hat{t}_\alpha^\lambda\mat{w})}{\hat{t}_\alpha^\lambda}\right]& = &
E\left\{\frac{\alpha}{\hat{M}_0(\lambda\mat{w})}E\left[\frac{V(\hat{t}_\alpha^\lambda\mat{w})}{\hat{t}_\alpha^\lambda}|\mathcal{F}_\lambda\right]\right\}\\
&=&E\left[\frac{\alpha}{\hat{M}_0(\lambda\mat{w})}\frac{V(\lambda\mat{w})}{\lambda}\right].
\end{eqnarray*}
Hence, we have established (\ref{FDReqn}).

Now, note that $M - R(\lambda\mat{w}) = M_0 - V(\lambda\mat{w}) + [M_1 - \sum_{\mathcal{M}_1}\delta_m(\lambda w_m)]\geq M_0 - V(\lambda\mat{w})$. Further observe that $V\mapsto V(\lambda\mat{w})/[M_0-V(\lambda\mat{w}) + 1]$ is convex.  Hence, by Theorem 3 in \cite{Hoe56} and with $p = \lambda\bar{w}_0$
\begin{eqnarray*}
E\left[\frac{V(\lambda\mat{w})}{M_0 - V(\lambda\mat{w})+1}\right] &\leq& \sum_{k=0}^{M_0} \frac{k}{M_0 -k+1}\left(\begin{array}{c}M_0\\k\end{array}\right)p^k(1-p)^{M_0 - k}\\
&=&\frac{p}{1-p}(1-p^{M_0}).
\end{eqnarray*}
The last equality follows from basic calculations. Thus,
\begin{eqnarray*}
E\left[\alpha\frac{1}{\hat{M}_0(\lambda\mat{w})}\frac{V(\lambda\mat{w})}{\lambda}\right] & =
&E\left[\alpha\frac{(1-\lambda)}{M - R(\lambda\mat{w})+1}\frac{V(\lambda\mat{w})}{\lambda}\right] \\
&\leq & \alpha\frac{(1-\lambda)}{\lambda}E\left[\frac{V(\lambda\mat{w})}{M_0 - V(\lambda\mat{w})+1}\right]\\
&=&\alpha\frac{(1-\lambda)}{\lambda} \frac{p}{1-p}(1-p^{M_0}). \\
\end{eqnarray*}
The result follows by plugging $\lambda\bar{w}_0$ in for $p$ in the last expression. \qed \\


\noindent\textsc{Proof of Theorem \ref{FDRthm}.}
From Lemma \ref{bound} and because $\bar{w}_0\leq w_{(M)}$,
\begin{equation*}
FDR(\hat{t}_{\alpha^*}^\lambda\mat{w}) \leq\alpha^*\bar{w}_0\frac{1-\lambda}{1-\lambda\bar{w}_0} = \alpha
\frac{\bar{w}_0}{w_{(M)}}\frac{1-\lambda w_{(M)}}{1-\lambda\bar{w}_0} \leq \alpha. \qed
\end{equation*}
\\


\subsection{Proofs of results in Section \ref{sec 6}}

Before proving Theorem \ref{asymptotic t} the following Glivenko-Cantelli-type Lemma regarding the uniform convergence of the FDP estimators and the FDP is presented.  For similar results in the unweighted adaptive setting see Theorem 6 in \cite{Sto04} or see the proof of Theorem 2 in \cite{Gen06} for the weighted, but unadaptive, setting.  See also \cite{Fin09, Fan12} and references therein for additional results on almost sure convergence of the FDP.
\begin{lemma}\label{lemu} Fix $\delta \in (0,u)$.  Under (A2) and (A4) - (A6),
$$\sup_{\delta\leq t \leq u}|\widehat{FDP}_M^{0}(t\mat{w}_M) - FDP_{\infty}^0(t)|\rightarrow 0,$$
$$\sup_{\delta\leq t\leq u}|\widehat{FDP}_M^{\lambda}(t\mat{w}_M) - FDP_\infty^\lambda(t)|\rightarrow 0,$$ and
$$\sup_{\delta\leq t\leq u}|FDP_M(t\mat{w}_M) - FDP_\infty(t)|\rightarrow 0$$ almost surely.
\end{lemma}
\begin{proof}
In what follows we denote $\max\{R(t\mat{w}_M), 1\}$ by $R(t\mat{w}_M)$ for short.  Observe $R(t\mat{w}_M)$ is nondecreasing in $t$ almost surely by the NS assumptions and $G(t)$ is strictly increasing in $t$ for $0\leq t\leq u$ by (A6).  Hence, for any $\delta\in (0,u),$
\begin{eqnarray*}\sup_{\delta\leq t\leq u}\left|\widehat{FDP}_M^{0}(t\mat{w}_M) - FDP_{\infty}^0(t)\right|&=&
\sup_{\delta\leq t\leq u}\left|\frac{t}{R(t\mat{w}_M)/M} -\frac{t}{G(t)}\right|\\ \\
 &&\hspace{-2.5in}=
\sup_{\delta\leq t\leq u}\left|\frac{t\left[G(t) - R(t\mat{w}_M)/M\right]}{G(t)R(t\mat{w}_M)/M}\right|
 \leq \frac{\sup_{\delta \leq t\leq u}\left|G(t) - R(t\mat{w}_M)/M\right|}{G(\delta)R(\delta\mat{w}_M)/M}\\
&&\hspace{-2.5in} \rightarrow
\frac{0}{G(\delta)^2} = 0 \\
\end{eqnarray*}
almost surely, where the numerator converges to 0 by the Glivenko-Cantelli Theorem and the denominator converges to $G(\delta)^2$ by (A4) and the Continuous Mapping Theorem.

As for the second claim, denote $\hat{a}_{0,M}^\lambda = \hat{M}_0(\lambda_M\mat{w}_M)/M$ and $a_{0,\infty}^\lambda = [1-G(\lambda)]/[1-\lambda]$.  Additionally observe
%
$$\widehat{FDP}_M^\lambda(t\mat{w}_M) = \hat{a}_{0,M}^\lambda \widehat{FDP}_M^0(t\mat{w})\mbox{ \hspace{.1in} and \hspace{.1in} } FDP^\lambda_\infty(t) = a_{0,\infty}^\lambda FDP_{\infty}^0(t),$$
%
Then using the triangle inequality
\begin{eqnarray*}
&&\sup_{\delta\leq t\leq u}\left|\widehat{FDP}^\lambda_M(t\mat{w}_M) - FDP_\infty^\lambda(t)\right| \hspace{1in} \\
&&\hspace{.5in} =\sup_{\delta\leq t\leq u}\left|\hat{a}_{0,M}^\lambda \widehat{FDP}_M^0(t\mat{w}_M) - a_{0,\infty}^\lambda FDP_{\infty}^0(t)\right| \\
&&\hspace{.5in}\leq \left|\hat a_{0,M}^\lambda - a_{0,\infty}^\lambda\right|\times \sup_{\delta\leq t\leq u}\left|\widehat{FDP}_M^0(t\mat{w}_M)\right| \\
&& \hspace{.7 in} + \hspace{.1in} a_{0,\infty}^\lambda\times \sup_{\delta\leq t \leq u}\left|\widehat{FDP}_M^0(t\mat{w}_M) - \widehat{FDP}_{\infty}^0(t)\right| \\
&& \hspace{.5in} < 2\epsilon  + \epsilon, \\
&& \end{eqnarray*}
where the last inequality is satisfied for all large enough $M$ for any $\epsilon>0$.  To verify the last inequality note that $\hat{a}_{0,M}^\lambda \rightarrow a_{0,\infty}^\lambda$ almost surely by (A2), (A4) and the Continuous Mapping Theorem, and hence $|\hat a_{0,M}^\lambda - a_{0,\infty}^\lambda|<\epsilon$ for all large enough $M$.  Further, for all large enough $M$, $$\sup_{\delta\leq t \leq u} \widehat{FDP}_M^0(t\mat{w}_M)< \sup_{\delta \leq t \leq u} FDP_\infty^0(t)\ + \epsilon \leq 2$$
by the first claim of the Lemma and (A6).  Additionally, $G(\lambda)\geq \lambda$ by (A6) and consequently $a_{0,\infty}^\lambda\leq 1$.  Lastly, $\sup_{\delta\leq t \leq u}|\widehat{FDP}_M^0(t\mat{w}_M) - FDP_{\infty}^0(t)|<\epsilon$ for all large enough $M$ by the first claim of the Lemma.

To prove the third claim, we first show that
\begin{eqnarray}
&&\sup_{\delta\leq t\leq u}\left|FDP_M(t\mat{w}_M) - FDP_\infty(t)\right| \nonumber \\
&&\leq\sup_{\delta\leq t\leq u}\left|\frac{V(t\mat{w}_M)}{R(t\mat{w}_M)} - \frac{a_0\mu_0 t}{R(t\mat{w}_M)/M}\right|
 + \sup_{\delta\leq t\leq u}\left|\frac{a_0\mu_0 t}{R(t\mat{w}_M)/M} - \frac{a_0\mu_0 t}{G(t)}\right| \nonumber\\
&&= \sup_{\delta\leq t\leq u}\frac{M}{R(t\mat{w})}\left|\frac{V(t\mat{w}_M)}{M} - a_0\mu_0 t\right|\label{S6}\\
 &&\hspace{.2in} + a_0\mu_0\sup_{\delta\leq t\leq u}\left|\widehat{FDP}_M^0(t\mat{w}_M) - FDP_\infty^0(t)\right| \label{S7}.
\end{eqnarray}
The inequality is a consequence of the triangle inequality and the definitions of $FDP_\infty(t)$ and $FDP_M(t\mat{w}_M)$.  The expression in (\ref{S6}) is verified by factoring out $R(t\mat{w}_M)/M$ in the first expression on the previous line while the expression in (\ref{S7}) follows from factoring out $a_0\mu_0$ in the second expression and by the definitions of $\widehat{FDP}^0_M(t\mat{w}_M)$ and $FDP_\infty^0(t)$.  Now, the quantity in (\ref{S7}) converges to 0 almost surely because $a_0\mu_0$ is bounded under (A5) and by the first claim of the Lemma.  To show that the first expression converges to 0 almost surely, first note for any $t\in(\delta,u]$, because $R(t\mat{w}_M)$ is nondecreasing in $t$, $R(t\mat{w}_M)/M>G(\delta/2)$ and hence that $$\frac{M}{R(t\mat{w}_M)}< \frac{1}{G(\delta/2)}$$ for all large enough $M$. Hence, if
\begin{equation}\label{S8}
\sup_{\delta\leq t \leq u}\left|\frac{V(t\mat{w}_M)}{M} - a_0\mu_0t\right|\rightarrow 0
\end{equation}
almost surely, then
$$\sup_{\delta\leq t\leq u}\frac{M}{R(t\mat{w})}\left|\frac{V(t\mat{w}_M)}{M} - a_0\mu_0 t\right|\leq \frac{\epsilon}{G(\delta/2)}$$
for all large enough $M$ and the proof would be completed since $\epsilon$ is arbitrary and $\delta$ is fixed.  To show (\ref{S8}), first observe that
$E[V(t\mat{w}_M)]/M_0 = \bar{w}_{0,M} t$ under the NS conditions. Also note that by the triangle inequality
\begin{eqnarray}\nonumber
\sup_{\delta\leq t \leq u}\left|\frac{V(t\mat{w}_M)}{M_0} - \mu_0 t\right|&\leq&
\sup_{\delta\leq t \leq u}\left|\frac{V(t\mat{w}_M)}{M_0} - \bar{w}_{0,M} t\right| + \sup_{\delta\leq t \leq u}\left|\bar{w}_{0,M}t - \mu_0 t\right|\\
&\leq& \sup_{\delta\leq t \leq u}\left|\frac{V(t\mat{w}_M)}{M_0} - \bar{w}_{0,M} t\right| + u\left|\bar{w}_{0,M} - \mu_0 \right| \rightarrow 0 \nonumber
\end{eqnarray}
almost surely, where the first quantity converges to 0 by the Glivenko-Cantelli Theorem and the second quantity converges to 0 because $\bar{w}_{0,M}\rightarrow \mu_0$ almost surely under (A5) and because $u\leq 1$.  Thus, again using the triangle inequality
\begin{eqnarray*}
&&\sup_{\delta\leq t \leq u}\left|\frac{V(t\mat{w}_M)}{M} - a_0\mu_0 t\right| = \sup_{\delta\leq t \leq u}\left|\frac{V(t\mat{w}_M)}{M_0}\left[\frac{M_0}{M} + a_0 - a_0\right] - a_0\mu_0 t\right|\\
&&\leq \left|\frac{M_0}{M} - a_0\right|\sup_{\delta\leq t\leq u}\left|\frac{V(t\mat{w}_M)}{M_0}\right| + a_0\sup_{\delta\leq t\leq u}\left|\frac{V(t\mat{w}_M)}{M_0} - \mu_0 t\right|\rightarrow 0 \end{eqnarray*}
almost surely, where the first quantity converges to 0 because $M_0/M\rightarrow a_0$ almost surely under (A5) and because $V(t\mat{w}_M)/M_0\leq 1$, while the second quantity converges to 0 because $a_0\leq 1$ and $V(t\mat{w}_M)/M_0\rightarrow \mu_0 t$.  Hence we have established (\ref{S8}).
\end{proof}


\noindent\textsc{Proof of Theorem \ref{asymptotic t}.}  Let us first focus on the equalities.  Suppose that $t_{\alpha,\infty}^0<u$.  Then $FDP_{\infty}^0(t_{\alpha,\infty}^0) = \alpha$ by the definition of $t_{\alpha,\infty}^0$ and by (A6).  Additionally due to (A6), for any $\epsilon>0$ there exists a $0<\delta<\epsilon$ such that
$$ FDP_{\infty}^0(t_{\alpha,\infty}^0+\delta) < \alpha + \epsilon. $$
Now, Lemma S1 gives $\widehat{FDP}_M^0(t\mat{w}_M)< FDP_{\infty}^0(t_{\alpha,\infty}^0+\delta)$ for $0
\leq t<t_{\alpha,\infty}^0+\delta$ and all large enough $M$.  Hence,  this and (A6) imply $$\hat{t}_{\alpha,M}^0= \sup\left[0\leq t\leq u:\widehat{FDP}_M^0(t\mat{w}_M)\leq \alpha\right]\leq t_{\alpha,\infty}^0 +\delta<t_{\alpha,\infty}^0+\epsilon$$ for all large enough $M$.  Similar arguments give
$\hat{t}_{\alpha,M}^0>t_{\alpha,\infty}^0 -\epsilon$ for all large enough $M$.  Now if $t_{\alpha,\infty}^0 = u$ then
$$t_{\alpha,\infty}^0-\epsilon \leq \hat{t}_{\alpha,M}^0\leq t_{\alpha,\infty}^0 = u$$
for all large enough $M$.  Hence, $|\hat{t}_{\alpha,M}^0 - t_{\alpha,\infty}^0|<\epsilon$ for all large enough $M$ and we conclude $\hat{t}_{\alpha,M}^0\rightarrow t_{\alpha,\infty}^0$ almost surely.  As for the second equality, $FDP^\lambda_\infty(t) = a_{0,\infty}^\lambda FDP_\infty^0(t)$ is also continuous and strictly increasing by (A6) and consequently identical argument apply.  Thus $\hat{t}_{\alpha,M}^{\lambda} \rightarrow t_{\alpha,\infty}^{\lambda}$ almost surely. %for $\lambda \in (0,1)$ and $\hat{t}_{\alpha,M} \rightarrow \hat{t}_{\alpha, \infty}$ for $0<\alpha<a_0\mu_0$ almost surely.

As for the inequality, note that (A6) implies $\lambda\leq G(\lambda)$ which implies
\begin{equation}\label{a0 ineq}
a_{0,\infty}^\lambda = \frac{1-G(\lambda)}{1-\lambda} \leq  1. \end{equation}
Hence,
\begin{equation}\label{asym FDP ineq}
FDP_{\infty}^\lambda(t) = a_{0,\infty}^\lambda FDP_\infty^0(t)\leq FDP_\infty^0(t)
\end{equation} for every $t\in(0,u]$.  This, (A6) and the definitions of $FDP_\infty^0(\cdot)$, $t_{\alpha,\infty}^0 $ and $t_{\alpha,\infty}^\lambda$ imply $t_{\alpha,\infty}^0 \leq t_{\alpha,\infty}^\lambda$. $\qed$\\


\noindent\textsc{Proof of Theorem \ref{asymptotic FDP}.}
By Lemma S1 and (A6), for $0<s<t\leq u$
\begin{eqnarray*}
FDP_M(t\mat{w}_M) - FDP_M(s\mat{w}_M)&>& \\
&& \hspace{-2in}a_0\mu_0 t/G(t) - a_0\mu_0 s/G(s) - 2\sup_{0\leq t\leq u}\left|FDP_M(t\mat{w}_M) - a_0\mu_0t/G(t)\right|\\
&&\hspace{-1in}\rightarrow  a_0\mu_0[t/G(t) - s/G(s)]> 0
\end{eqnarray*}
almost surely.  Claim (C1) is then a consequence of Theorem \ref{asymptotic t} and the Continuous Mapping Theorem.  To verify Claims (C2) and (C3), first observe that by the triangle inequality
\begin{eqnarray*}
&&|FDP_M(\hat{t}_{\alpha,M}^\lambda \mat{w}_M) - FDP_\infty(t_{\alpha,\infty}^\lambda)| \\
&\leq&|FDP_M(\hat{t}_{\alpha,M}^\lambda \mat{w}_M) - FDP_{\infty}(\hat t_{\alpha,M}^\lambda)|
+ |FDP_\infty(\hat{t}_{\alpha,M}^\lambda) - FDP_{\infty}(t_{\alpha,\infty}^\lambda)|.
\end{eqnarray*}
The first quantity converges to 0 almost surely by Lemma S1 and the second quantity converges to 0 almost surely by Theorem \ref{asymptotic t} and the Continuous Mapping Theorem.
Hence, $FDP_M(t_{\alpha,M}^\lambda\mat{w}_M)\rightarrow FDP_\infty(t_{\alpha,\infty}^\lambda)$ almost surely.  Thus to prove Claims (C2) and (C3) it suffices to show that
$FDP_\infty(t_{\alpha,\infty}^\lambda)\leq \alpha$ if $\mu_0\leq 1$, with equality when $G(t)$ is a DU distribution with $\mu_0 =1$ and $FDP_{\infty}^\lambda(u)\geq \alpha$.  To show this, consider the following:
\begin{eqnarray*}
FDP_{\infty}(t_{\alpha,\infty}^\lambda) &=& a_0\mu_0\frac{t_{\alpha,\infty}^\lambda}{G(t_{\alpha,\infty}^\lambda)}\\
&\leq& a_0\frac{t_{\alpha,\infty}^\lambda}{G(t_{\alpha,\infty}^\lambda)}\\
&\leq& \frac{1-G(\lambda)}{1-\lambda}\frac{t_{\alpha,\infty}^\lambda}{G(t_{\alpha,\infty}^\lambda)} \\
&=& FDP^\lambda_{\infty}(t_{\alpha,\infty}^\lambda)\\
&\leq& \alpha.
\end{eqnarray*}
The first equality is due to the definition of $FDP_{\infty}(\cdot)$.  The first inequality is satisfied when $\mu_0\leq 1$ and is an equality when $\mu_0 = 1$.  As for the second inequality, note that $G(\lambda)\leq a_0\lambda + 1-a_0$ when $\mu_0\leq 1$ and $G(\lambda) = a_0\lambda + 1-a_0$ under a DU distribution with $\mu_0 = 1$.  Consequently
$$a_0 = \frac{1-[a_0\lambda + 1-a_0]}{1-\lambda} \leq \frac{1 - G(\lambda)}{1-\lambda}$$
when $\mu_0\leq 1$ and the inequality is an equality when $G$ is a DU distribution with $\mu_0 = 1$.  The last equality is satisfied by the definition of $FDP_{\infty}^\lambda(\cdot)$. The last inequality is satisfied by the definition of $t_{\alpha,\infty}^\lambda$ and is an equality when $G$ is a DU distribution with $\mu_0 = 1$ and $FDP_\infty(u)\geq \alpha$ because these conditions imply $FDP_\infty(u) = FDP_\infty^\lambda(u)\geq \alpha$.  That is, $FDP_\infty(u)$ is continuous and monotone and takes on value $\alpha$.  Hence, $FDP_{\infty}(t_{\alpha,\infty}^\lambda)\leq \alpha$ if $\mu_0\leq 1$ with equality if $G$ is a DU distribution with $\mu_0=1$ and $FDP_\infty(u)\geq \alpha$. $\qed$\\

\noindent\textsc{Proof of Theorem \ref{EFDR}.}
Under the conditions of the Theorem
\begin{eqnarray*}
Cov(W_{m,M},\theta_{m,M}) &=& E[W_{m,M}|\theta_{m,M}=1]E[\theta_{m,M}] - E[W_{m,M}]E[\theta_{m,M}]\\
&=& E[\theta_{m,M}](E[W_{m,M}|\theta_{m,M}=1] - 1).
\end{eqnarray*}
Hence, $Cov(W_{m,M},\theta_{m,M})\geq 0$ implies $E[W_{m,M}|\theta_{m,M}=1]\geq 1$ and consequently $E[W_{m,M}|\theta_{m,M}=0]\leq 1$, with equality if $Cov(W_{m,M},\theta_{m,M}) = 0$.  Hence $E[\bar{W}_{0,M}|\bm{\theta}_M\neq \bm{1}_M] = \mu_0 \leq 1$ with equality if $Cov(W_{m,M},\theta_{m,M})=0$.  The result follows because $\bar{W}_{0,M}\rightarrow \mu_0$ almost surely. $\qed$ \\

\noindent\textsc{Proof of Corollary \ref{STS opt}.}
Observe that $u = 1$ and $\lambda<1$ is fixed.  Hence (A2) is satisfied and (A4) - (A6) are satisfied by the conditions of the Theorem.  Therefore Claim (C1) holds by Theorem \ref{asymptotic FDP}.  Now, additionally note that $\mu_0 = 1$ if $\mat{w}_M = \bm{1}_M$ and that $FDP_{\infty}(1) = a_0 \geq \alpha$ under the conditions of the Theorem.  Thus Claims (C2) and (C3) hold by Theorem \ref{asymptotic FDP}. $\qed$ \\

Before proving Theorem \ref{optimal conditions} we provide Lemma S2.  It will be used to verify that optimal weights are weakly dependent so that decision functions satisfy the weak dependence structure defined in (A4) - (A5).  Below, denote $t_0(k) = E[t_m(k/p_m, \gamma_m)]$ and denote $G(t_0(k)) = E[\delta_m(t_m(k/p_m,\gamma_m))]$, where the expectations are taken over all random quantities, i.e. over $(Z_m,\theta_m,p_m,\gamma_m)$ for some fixed $k>0$.  Further, define
$$\widetilde{FDP}_\infty(t_0(k)) = \frac{1-G(t_0(k))}{1-t_0(k)}\frac{t_0(k)}{G(t_0(k))}$$
and $$k^* = \inf\{k:\widetilde{FDP}_\infty(t_0(k)) = \alpha\},$$
and denote $$\tilde{w}_{m,\infty}(k^*, p_m, \gamma_m) = U_m t_m(k^*/p_m,\gamma_m)/t_0(k^*).$$
\begin{lemma2}\label{as delta}
Suppose that $\Pr(p_m\leq 1-\alpha)$.  Under Model 1 and (A1), $k_M^* \rightarrow k^*$ almost surely and
$$\tilde{w}_{m,M}(k_M^*, \mat{p},\bm{\gamma}) \rightarrow \tilde{w}_{m,\infty}(k^*, p_m, \gamma_m)$$ almost surely.
%Further, for any fixed $t\leq \bar{t}_M(k^*)$,
%$$\lim_{M\rightarrow\infty}\delta_m(t\tilde{w}_{m,M}(k^*,\mat{p},\bm{\gamma})) = \delta_m(t\tilde{w}_{m,\infty}(k,p_m,\gamma_m))$$
%$$G(tw_{m,M}(k^*,\mat{p},\bm{\gamma})\rightarrow G(tw_{m,\infty}(k^*,p_m,\gamma_m))$$
%almost surely for every $m\in\mathcal{M}$.
\end{lemma2}
\begin{proof}
Note that $0<\alpha \leq 1 - p_{(M)}$ with probability 1 so that $k_M^*$ is well defined for $M = 1, 2, ...$ by Theorem \ref{exist}.  Further, observe that $t_m(k^*/p_m,\gamma_m)$, $m = 1, 2, ...$ are i.i.d. continuous random variables taking values in $[0,1]$ under Model \ref{randeffmodel}.  Hence, by the Strong Law of Large numbers $\bar{t}_M(k^*,\mat{p},\bm{\gamma}) \rightarrow t_0(k^*)$ almost surely.  Likewise, $\bar{G}_M(\bm{t}(k^*,\mat{p},\bm{\gamma})\rightarrow G(t_0(k^*))$ almost surely and by the Continuous Mapping Theorem $$\widetilde{FDP}_M(\mat{t}(k^*,\mat{p},\bm{\gamma}))\rightarrow   \widetilde{FDP}_\infty(t_0(k^*))$$ almost surely.  Because further $\widetilde{FDP}_M(\mat{t}(k,\mat{p},\bm{\gamma}))$ and $\widetilde{FDP}_\infty(t_0(k))$ are both continuous in $k$ by (A1), we have from the Continuous Mapping Theorem and the definitions of $k_M^*$ and $k^*$ that $k_M^*\rightarrow k^*$ almost surely.  Thus,
\begin{eqnarray*}
\tilde{\mat{w}}_{m,M}(k_M^*,\mat{p},\bm{\gamma}) &=& U_m\mat{w}_{m,M}(k_M^*,\mat{p},\bm{\gamma})\\
& = & U_m\frac{t_m(k_M^*/p_m,\gamma_m)}{\bar{t}_M(k_M^*,\mat{p},\bm{\gamma})} \\
&\rightarrow&U_m\frac{t_m(k^*/p_m,\gamma_m)}{t_0(k^*)} = \tilde{w}_{m,\infty}(k^*,p_m,\gamma_m)
\end{eqnarray*}
almost surely by the Continuous Mapping Theorem.
%Because the $Z_m$s have continuous distributions, the second claim is a consequence of the first claim and the Continuous Mapping Theorem.

%For the second claim, first define a neighborhood about a discontinuity point $s$ by $A_{m,\epsilon}(s) = \{Z_m:\delta_m(Z_m;s-\epsilon) = 0, \delta_m(Z_m; s+\epsilon) = 1\}$ and observe $\Pr(\bigcap_{\epsilon>0}A_{m,\epsilon}(s))=0$ for every $m\in\mathcal{M}$ due to $t\mapsto E[\delta_m(Z_m;t)]$ being continuous and strictly increasing for every $m\in\mathcal{M}$.  Now, by the first claim, for any arbitrary $\epsilon>0$ we have \begin{eqnarray*}
%&&\{\delta_m(t\tilde{w}_{m,M}(k^*,\mat{p},\bm{\gamma}))\neq \delta_m(t\tilde{w}_{m,\infty}(k^*,p_m,\gamma_m)\}\\&&\subseteq %A_{m,\epsilon}(t\tilde w_{m,\infty}(k^*,p_m,\gamma_m))
%\end{eqnarray*}
%for all large enough $M$.  Hence, $$\Pr(\lim_{M\rightarrow\infty}|\delta_m(t\tilde w_{m,M}(k,\mat{p},\bm{\gamma})) - \delta_m(t\tilde w_{m,\infty}(k,p_m,\gamma_m))|<\epsilon, $$ almost surely.
\end{proof}

\noindent\textsc{Proof of Theorem \ref{optimal conditions}.}
First we verify (A2).  Observe $\lambda_M = \bar{t}_M(k_M^*,\mat{p},\bm{\gamma})\rightarrow t_0(k^*)$ almost surely by the Strong Law of Large Numbers and the Continuous Mapping Theorem, where recall $t_0(k^*) = E[t_m(k^*/p_m,\gamma_m)]$.  Thus, by the definition of $\tilde{w}_{m,M}$
$$\lim_{M\rightarrow\infty}\tilde{w}_{m,M} = \lim_{M\rightarrow\infty}\frac{U_m t_{m,M}(k_M^*/p_m,\gamma_m)}{\bar{t}_M(k_M^*,\mat{p},\bm{\gamma})}\leq \frac{1}{t_0(k^*)}$$ almost surely, where the last inequality is due to the Continuous Mapping Theorem, Lemma (S2) and because $U_mt_m(k_M^*,\mat{p},\bm{\gamma})\leq 1$ almost surely by construction.  That is, (A2) is satisfied with $\lambda = u = 1/t_0(k^*)$.

Before verifying (A4) - (A6) we introduce some notation. Denote
%$$G(t\tilde{w}_{m,M}(k^*,\mat{p},\bm{\gamma}) = E[\delta_m(Z_m;t \tilde{w}_{m,M}(k^*,\mat{p},\bm{\gamma}))|\bm{\gamma},\mat{p},\mat{U}]$$ and
%$$G(t\tilde{w}_{m,\infty}(k^*,p_m,\gamma_m)) = E[\delta_m(Z_m;t \tilde{w}_{m,\infty}(k^*,p_m,\gamma_m))|p_m,\gamma_m,U_m]$$
%and denote
%$$G^{k^*}(t) = E[G(t\tilde{w}_{m,\infty}(k^*,p_m,\gamma_m))]$$
$$G^{k^*}(t) = E[\delta_m(t\tilde{w}_{m,\infty}(k^*,p_m,\gamma_m))]$$
where the expectation is taken over all random quantities, i.e. taken over $(Z_m, \theta_m,p_m,\gamma_m,U_m)$.  Further we sometimes suppress $\mat{p}$ and $\bm{\gamma}$ and write $\tilde{w}_{m,\infty}(k^*) = \tilde{w}_{m,\infty}(k^*,p_m,\gamma_m)$, $\tilde{w}_{m,M}(k^*) = \tilde{w}_{m,M}(k^*,\mat{p},\bm{\gamma})$ and $\tilde{\mat{w}}_M(k^*) = [\tilde{w}_{m,M}(k^*), m\in\mathcal{M}]$.  Further, denote $\tilde{\mat{w}}_\infty(k^*) = [\tilde{w}_{m,\infty}(k^*), m\in\mathcal{M}]$.

Now consider (A4). Observe that $\delta_m(t\tilde{w}_{m,\infty}(k^*)), m = 1, 2, ...$ are i.i.d. Bernoulli random variables with success probability $G^{k^*}(t)$ under Model \ref{randeffmodel} so that
$$\frac{R(t\tilde{\mat{w}}_{\infty}(k^*))}{M} = \frac{\sum_{m\in\mathcal{M}}\delta_m(t\tilde{w}_{m,\infty}(k^*))}{M}\rightarrow G^{k^*}(t)$$ almost surely by the Strong Law of Large Numbers.  Further, by the NS assumptions, Lemma S2, and because $G^{k^*}(t)$ is continuous, we have that for any $\epsilon>0$ there exists an $\epsilon'>0$ such that
\begin{eqnarray*}
\frac{R(t\tilde{\bm{w}}_M(k^*_M))}{M} &=& \frac{\sum_{m\in\mathcal{M}}\delta_m(t\tilde{w}_{m,M}(k_M^*))}{M}<\frac{\sum_{m\in\mathcal{M}}\delta_m(t[\tilde{w}_{m,\infty}(k^*)+\epsilon'])}{M} \\ &<& G^{k^*}(t+t\epsilon')<G^{k^*}(t) + \epsilon
\end{eqnarray*}
for all large enough $M$.  Similar arguments give
$$
\frac{\sum_{m\in\mathcal{M}}\delta_m(t\tilde{w}_{m,M}(k^*))}{M}> G^{k^*}(t) - \epsilon
$$
for all large enough $M$.  Hence, $R(t\tilde{\mat{w}}_{M}(k^*))/M \rightarrow G^{k^*}(t)$ almost surely.  Then because $k_M^*\rightarrow k^*$ almost surely by Lemma S2, $R(t\tilde{\mat{w}}_{M}(k_M^*))/M \rightarrow G^{k^*}(t)$ almost surely by the Continuous Mapping Theorem.

As for (A5), recall the NS conditions give $E[\delta_m(t_m)|\theta_m = 0] = t_m$.  Hence, taking the expectation over all random quantities, we have by the law of iterated expectation
$$E[(1-\theta_m)\delta_m(t \tilde{w}_{m,\infty}(k^*,p_m,\gamma_m))] = a_0\mu_0t,$$
where $a_0 = E[1-\theta_m]$ and $\mu_0 = E[\tilde{w}_{m,\infty}(k^*, p_m,\gamma_m)|\theta_m = 0]$.  Then, arguments akin to those in the proof of (A4) give
$$\frac{V(t\tilde{\mat{w}}_{M}(k_M^*))}{M} = \frac{M_0}{M}\frac{V(t\tilde{\mat{w}}_{M}(k_M^*))}{M_0} \rightarrow a_0 \mu_0 t$$
almost surely.

For (A6), first observe that $G^{k^*}(t) = a_0\mu_0 t + (1-a_0) G_1(t)$ for $t\leq u$,
where $$G_1(t) = E\left[\pi_{{\gamma}_m}(t\tilde{w}_{m,\infty}(k^*,p_m,\gamma_m))\right]$$ and the expectation is taken over all random quantities.  Clearly $t\mapsto G_1(t)$ is concave and twice differentiable because $t\mapsto \pi_{\gamma_m}(t)$ is twice differentiable almost surely by (A1).
To see that $t/G(t)\rightarrow 0$ as $t\downarrow 0$ note that $G_1'(t)\rightarrow \infty$ as $t\downarrow 0$ because $\pi_{\gamma_m}'(t)\rightarrow \infty$ as $t\downarrow 0$ almost surely by (A1).  Hence,
$$\frac{t}{G^{k^*}(t)} = \frac{t}{a_0 \mu_0 t + (1-a_0)G_1(t)}\rightarrow 0$$
as $t\downarrow0$ by an application of H{\^o}ptial's rule.  Clearly, $\lim_{t\uparrow u}t/G^{k^*}(t) \rightarrow u/G^{k^*}(u)$ because $G^{k^*}(t)$ is continuous. To see that $u/G^{k^*}(u) \leq 1$ we establish the following:
%recall $u = \bar{t}_M(k^*)\rightarrow t_0(k^*)$ almost surely so it suffices to show $G^{k^*}(t_0(k^*))\geq t_0(k^*)$.  However, recall $\tilde{w}_{m,\infty}(k^*) = t_0(k^*)^{-1}[U_m t_m(k^*/p_m, \gamma)]$.  Hence
\begin{eqnarray*}
G^{k^*}(u)&=& E[\delta_m(u\tilde{w}_{m,\infty}(k^*))]\\
&=& a_0E[\delta_m(u\tilde{w}_{m,\infty}(k^*))|\theta_m=0] \\
&& + (1-a_0)E[\delta_m(u\tilde{w}_{m,\infty}(k^*))|\theta_m=1] \\
&=& a_0E[u\tilde{w}_{m,\infty}(k^*)] + (1-a_0) E[\pi_{\gamma_m}(u\tilde{w}_{m,\infty}(k^*))] \\
&\geq& a_0E[u\tilde{w}_{m,\infty}(k^*)] + (1-a_0)E[u\tilde{w}_{m,\infty}(k^*)] \\
&=& E[u\tilde{w}_{m,\infty}(k^*)] \\
&=&uE[\tilde{w}_{m,\infty}(k^*)] = u.
\end{eqnarray*}
The first equality is by the definition of $G^{k^*}(u)$ while the second equality is due to the law of iterated expectation. The third is a consequence of the definition of $\pi_{\gamma_m}(t)$ and the NS assumptions.  The inequality is satisfied because $\pi_{\gamma_m}(t)\geq t$ almost surely for every $t\in[0,1]$ under (A1). The forth equality is obvious.  As for the fifth, recall $E[U_m|p_m,\gamma_m] = 1$, $\tilde{w}_{m,\infty}(k^*) = U_m w_{m,\infty}(k^*)$ and that $E[w_{m,\infty}(k^*)] = 1$.  Hence, by the law of iterated expectation $E[\tilde{w}_{m,\infty}(k^*)] = E[w_{m,\infty}(k^*)] = 1.$

To verify that $\mu_0\leq 1$ we make use of Theorem \ref{EFDR} and write $W_{m} = w_{m,M}(k_M^*,\mat{p},\bm{\gamma})$ and $\tilde{W}_m = U_m W_m $ for short.  First let us focus on $Cov(W_m,\theta_m)$.   From the law of iterated expectation,
\begin{equation}\label{cov}
Cov(W_{m}, \theta_m) = E[Cov(W_{m},\theta_m|p_m)] + Cov(E[W_{m}|p_m],p_m).
\end{equation}
Observe that
\begin{eqnarray*}
Cov(W_{m},\theta_m|p_m) & = & E[W_{m}\theta_m|p_m] - E[W_{m}|p_m]E[\theta_m|p_m]\\
 &=& p_mE[W_{m}|p_m] - p_mE[W_{m}|p_m] = 0
\end{eqnarray*}
which implies that the first expectation in (\ref{cov}) is 0. To compute the second expectation, first observe $\pi_{\gamma_m}'(t_m)$ is continuous and strictly decreasing and hence the solution to $\pi_{\gamma_m}'(t_m) = a$, denoted $t_m(a,\gamma_m)$, is continuous and strictly decreasing in $a$ almost surely by (A1).  Hence $t_m(k_M^*/p_m, \gamma_m)$ is strictly increasing and continuous in $p_m$ almost surely.  Thus,
$$E[W_{m}|\mat{p},\bm{\gamma}] = E\left[M\frac{t_m(k_M^*/p_m,\gamma_m)}{t_m(k_M^*/p_m,\gamma_m) + \sum_{j\neq m}t_j(k_M^*/p_j,\gamma_j)}\bigg|\mat{p},\bm{\gamma}\right]$$ is also increasing in $p_m$ almost surely because the function $x/(x + a)$ for $a$ any positive constant is increasing in $x$ for $x>0$.  This implies $E[W_{m}|p_m]$ is also increasing in $p_m$ almost surley which implies $Cov(E[W_{m}|p_m],p_m)\geq 0$. As for $\tilde{W}_{m} = U_mW_{m}$,
$$Cov(\tilde{W}_m,\theta_m) = E[Cov(U_mW_m,\theta_m|W_m)] + Cov(E[U_mW_m|W_m], E[\theta_m|W_m])$$
by the law of iterated expectation.  But $$E[Cov(U_mW_m,\theta_m|W_m)] = E[W_mCov(U_m,\theta_m|W_m)] = 0$$ because $Cov(U_m,\theta_m|W_m)$ is 0 by construction.  Additionally, $$Cov(E[U_mW_m|W_m], E[\theta_m|W_m]) =Cov(W_m,E[\theta_m|W_m])\geq 0$$ because $Cov(W_m,\theta_m)\geq 0$.  Hence, $Cov(\tilde{W}_m,\theta_m)\geq 0$ and thus, by Theorem \ref{EFDR}, $\mu_0\leq 1$.  $\qed$ \\

\noindent\textsc{Proof of Theorem \ref{asymp opt w}.}
First recall from the proof of Theorem \ref{optimal conditions} (where here we take $U_m = 1$ almost surely for every $m$) that $\lambda_M = \bar{t}_M(k_M^*)\rightarrow t_0(k^*)$ Hence,  we have
$$FDP_{\infty}^{\lambda}(t) = \frac{1 - G^{k^*}(t_0(k^*))}{1 - t_0(k^*)}\frac{t}{G^{k^*}(t)}.$$
Further observe that because $t/G^{k^*}(t)$ is strictly increasing by (A6), then $t_0(k^*) = t_{\alpha,\infty}^\lambda$ by the definition of $t_{\alpha,\infty}^\lambda$.  Hence $\bar{t}_M(k_M^*)\rightarrow t_0(k^*) = t_{\alpha,\infty}^\lambda$ almost surely. $\qed$\\


% -  t_{\alpha,\infty}^{t_0(k^*)}| \rightarrow 0$ almost surely.  Additionally, by the definition of $t_{\alpha,\infty}^\lambda$, the continuous mapping theorem, and because $\bar{t}_M(k^*)\rightarrow t_0(k^*)$ almost surely, $| t_{\alpha,\infty}^{\lambda_M} - t_{\alpha,\infty}^{t_0(k^*)}|.$  Hence,
%\begin{eqnarray*}
%|\bar{t}_M(k^*) -  t_{\alpha,\infty}^{\lambda_M}|\leq |\bar{t}_M(k^*) -  t_{\alpha,\infty}^{t_0(k^*)}| + | t_{\alpha,\infty}^{\lambda_M} - %t_{\alpha,\infty}^{t_0(k^*)}| \rightarrow 0. \qed
%\end{eqnarray*}

\noindent\textsc{Proof of Corollary \ref{independent weights}}
First observe that $Cov(w_{m,M},\theta_{m,M}) = 0$ and hence $\mu_0 = 1$ by Theorem \ref{EFDR}.  It therefore suffices to show that (A4) - (A6) are satisfied. But $\delta_m(tw_{m,M})$, $m = 1, 2, ...$ are i.i.d. Bernoulli random variables under Model \ref{randeffmodel} and the conditions of the Theorem.  Hence,
$R(t\mat{w}_M)/M \rightarrow G(t)$ for $G(t) = E[\delta_m(t\mat{w}_{m,M})]$ almost surely by the Strong Law of Large Numbers and (A4) is satisfied.  Likewise $(1-\theta_{m,M})\delta_m(tw_{m,M})$, $m = 1, 2, ...$ are i.i.d. random variable with mean $a_0 t$ under the NS assumptions and the conditions of the Theorem.  Hence,
$$\frac{V(t\mat{w}_M)}{M} = \frac{1}{M}\sum_{m\in\mathcal{M}}(1-\theta_{m,M})\delta_m(tw_{m,M})\rightarrow a_0 t$$ almost surely by the Strong Law of Large Numbers and (A5) is satisfied.  Condition (A6) is verified using arguments identical to those used in verifying (A6) in the proof of Theorem \ref{optimal conditions} with $G^{k^*}(t) = G(t)$ and $w_{m,M} = \tilde{w}_{m,\infty}(k^*)$. \qed \\

\noindent\textsc{Proof of Corollary \ref{asymp opt p}.}
Observe that $\pi(\mat{t},\mat{p},\bm{\gamma})$ is proportional to  $\pi(\mat{t},\mat{1},\bm{\gamma})$ and hence the maximization of $\pi(\mat{t},\mat{p},\bm{\gamma})$ with respect to $\mat{t}$ is free of $p_m$.  Thus $\tilde{w}_{m,M}(k,\mat{p},\bm{\gamma})$ is independent of $p_m$ and hence independent of $\theta_m$.  The result then follows from Theorems \ref{EFDR} and \ref{optimal conditions}. \\$\qed$

\subsection{Data analysis and approximations}


For simplicity and because, as demonstrated in Sections \ref{sec 6} and \ref{sec 7}, the WAMDF is robust with respect to weight misspecification, we approximate power functions using a multivariate normal model for $\mat{Y}_m$.  First, assuming $\bm{Y}_m$ has a Multinomial distribution with probability vector $\mat{p}_m$ and sample size $n_m$, we compute Z-score for $T_m = \mat{x}^T\mat{Y}_m$ by
$$Z_m = \frac{\bm{x}^T\bm{Y}_m - n_m\bar{x}}{\sqrt{\bm{x}^T\hat{\bm{\Sigma}}\bm{x}}},$$
where $\bar{x} = \frac{1}{5}\bm{j}^T\bm{x}$ and $\hat{\bm{\Sigma}} = n_m\{diag(\hat{\mat{p}}_m) - \hat{\mat{p}}_m\hat{\mat{p}}_m^T\}$ for $\hat{\mat{p}}_{m} = \mat{Y}_m/n_m$. By the Central Limit Theorem, Slutsky's Theorem, and the Continuous Mapping Theorem, $Z_m$ is asymptotically normal (in $n_m$) with variance 1 and mean/effect size
$$\gamma_m = \frac{n_m [\bm{x}^T\bm{p}_m - \bar{x}]}{\sqrt{\bm{x}'\bm{\Sigma}\mat{x}}} = \sqrt{n}_m\frac{\sum_ix_i[p_{im} - 1/5]}{\sqrt{\bm{x}^T[diag(\bm{p}_m) - \bm{p}_m\bm{p}_m^T]\bm{x}}} \equiv \sqrt{n_m} K(\beta),$$
where $K(\beta)$ is some constant that depends on $\beta$ (and $\mat{x}$) through $$p_{im} = \frac{e^{\beta x_i}}{\sum_{i = 1}^5 e^{\beta x_i}}.$$ Now, for any specification of $K(\beta)$, each power function is approximated using the power expression in Example \ref{example1} with $\gamma_m = \sqrt{n_m}K(\beta)$.

\textit{Remark:} We should note that results in this paper assume each $Z_m$ is continuous.  While a continuous, say Normal, approximation for the distribution of $Z_m$ may be reasonable, $Z_m$ does technically have a discrete distribution in this particular application.  One route for handling this discreteness is to transform each test statistic so as to have a continuous distribution using an independently generated auxiliary variate, as in \cite{Ste50, HabPen11, Pen11}, while another route is to simply ignore the discreteness and use the Normal approximation.  As in \cite{SunMcL12}, we have chosen the latter approach here for simplicity and because, as demonstrated in Section \ref{sec 6}, the asymptotically optimal WAMDF is robust with respect to weight misspecification.  Intuitively the difference between these two approaches should be negligible when the distributions of the $Z_m$s are well approximated, which appears to be the case here.  See Figure \ref{p-dist} below.  However, a more precise assessment of this phenomenon, though warranted, is beyond the scope of this manuscript.

\begin{figure}[h!]
\epsfig{file = ecdfmult, width = 5in}
\caption{\label{p-dist}The cumulative distribution function of $P_m$ for $n_m = 6$ under $H_m:\theta_m = 0$ is above.  For this data set, $n_m\geq 6$ for each $m$.}
\end{figure}






\end{document}

Main-Revision-8-2014arxiv.bbl
\begin{thebibliography}{49}
% BibTex style file: imsart-nameyear.bst, 2013-01-28
% Default style options (sort=1,type=nameyear).
% Used options (sort=1,type=nameyear).

\bibitem[\protect\citeauthoryear{Anderson and Habiger}{2012}]{AndHab12}
\begin{barticle}[author]
\bauthor{\bsnm{Anderson},~\bfnm{M.}\binits{M.}} \AND
  \bauthor{\bsnm{Habiger},~\bfnm{J.}\binits{J.}}
(\byear{2012}).
\btitle{Characterization and Identification of Productivity-Associated
  Rhizobacteria in Wheat}.
\bjournal{Appl. Environ. Microbiol.}
\bvolume{78}
\bpages{4434 - 444}.
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Benjamini and Hochberg}{1995}]{BenHoc95}
\begin{barticle}[author]
\bauthor{\bsnm{Benjamini},~\bfnm{Y.}\binits{Y.}} \AND
  \bauthor{\bsnm{Hochberg},~\bfnm{Y.}\binits{Y.}}
(\byear{1995}).
\btitle{Controlling the false discovery rate: a practical and powerful approach
  to multiple testing}.
\bjournal{J. R. Stat. Soc. Ser. B. Stat. Methodol.}
\bvolume{57}
\bpages{289--300}.
\bmrnumber{MR1325392 (96d:62143)}
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Benjamini and Hochberg}{2000}]{BenHoc00}
\begin{barticle}[author]
\bauthor{\bsnm{Benjamini},~\bfnm{Yoav}\binits{Y.}} \AND
  \bauthor{\bsnm{Hochberg},~\bfnm{Yosef}\binits{Y.}}
(\byear{2000}).
\btitle{On the Adaptive Control of the False Discovery Rate in Multiple Testing
  with Independent Statistics}.
\bjournal{J. Educ. Behav. Stat.}
\bvolume{25}
\bpages{60-83}.
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Benjamini, Krieger and
  Yekutieli}{2006}]{BenKri06}
\begin{barticle}[author]
\bauthor{\bsnm{Benjamini},~\bfnm{Y.}\binits{Y.}},
  \bauthor{\bsnm{Krieger},~\bfnm{A.~M.}\binits{A.~M.}} \AND
  \bauthor{\bsnm{Yekutieli},~\bfnm{D.}\binits{D.}}
(\byear{2006}).
\btitle{Adaptive linear step-up procedures that control the false discovery
  rate}.
\bjournal{Biometrika}
\bvolume{93}
\bpages{491--507}.
\bmrnumber{MR2261438 (2007j:62024)}
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Benjamini and Yekutieli}{2001}]{BenYek01}
\begin{barticle}[author]
\bauthor{\bsnm{Benjamini},~\bfnm{Yoav}\binits{Y.}} \AND
  \bauthor{\bsnm{Yekutieli},~\bfnm{Daniel}\binits{D.}}
(\byear{2001}).
\btitle{The control of the false discovery rate in multiple testing under
  dependency}.
\bjournal{Ann. Statist.}
\bvolume{29}
\bpages{1165--1188}.
\bdoi{10.1214/aos/1013699998}
\bmrnumber{1869245 (2002i:62135)}
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Billingsley}{1999}]{Bil99}
\begin{bbook}[author]
\bauthor{\bsnm{Billingsley},~\bfnm{Patrick}\binits{P.}}
(\byear{1999}).
\btitle{Convergence of probability measures},
\bedition{second} ed.
\bseries{Wiley Series in Probability and Statistics: Probability and
  Statistics}.
\bpublisher{John Wiley \& Sons Inc.}, \baddress{New York}.
\bnote{A Wiley-Interscience Publication}.
\bdoi{10.1002/9780470316962}
\bmrnumber{1700749 (2000e:60008)}
\end{bbook}
\endbibitem

\bibitem[\protect\citeauthoryear{Blanachar and Roquain}{2008}]{BlaRoq08}
\begin{barticle}[author]
\bauthor{\bsnm{Blanachar},~\bfnm{Gilles}\binits{G.}} \AND
  \bauthor{\bsnm{Roquain},~\bfnm{Etienne}\binits{E.}}
(\byear{2008}).
\btitle{Two simple sufficient conditions for {FDR} control}.
\bjournal{Electron. J. Stat.}
\bvolume{2}
\bpages{963--992}.
\bdoi{10.1214/08-EJS180}
\bmrnumber{2448601 (2009k:62184)}
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Blanchard and Roquain}{2009}]{BlaRoq09}
\begin{barticle}[author]
\bauthor{\bsnm{Blanchard},~\bfnm{G.}\binits{G.}} \AND
  \bauthor{\bsnm{Roquain},~\bfnm{E.}\binits{E.}}
(\byear{2009}).
\btitle{Adaptive FDR control under independence and dependence}.
\bjournal{J. Mach. Learn. Res.}
\bvolume{10}
\bpages{2837 --2831}.
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Cai and Sun}{2009}]{SunCai09}
\begin{barticle}[author]
\bauthor{\bsnm{Cai},~\bfnm{T.~Tony}\binits{T.~T.}} \AND
  \bauthor{\bsnm{Sun},~\bfnm{Wenguang}\binits{W.}}
(\byear{2009}).
\btitle{Simultaneous testing of grouped hypotheses: finding needles in multiple
  haystacks}.
\bjournal{J. Amer. Statist. Assoc.}
\bvolume{104}
\bpages{1467--1481}.
\bdoi{10.1198/jasa.2009.tm08415}
\bmrnumber{2597000 (2011d:62020)}
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Cao, Sun and Kosorok}{2013}]{Cao13}
\begin{barticle}[author]
\bauthor{\bsnm{Cao},~\bfnm{Hongyuan}\binits{H.}},
  \bauthor{\bsnm{Sun},~\bfnm{Wenguang}\binits{W.}} \AND
  \bauthor{\bsnm{Kosorok},~\bfnm{Michael~R.}\binits{M.~R.}}
(\byear{2013}).
\btitle{The optimal power puzzle: scrutiny of the monotone likelihood ratio
  assumption in multiple testing}.
\bjournal{Biometrika}
\bvolume{100}
\bpages{495--502}.
\bdoi{10.1093/biomet/ast001}
\bmrnumber{3068449}
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Desai and Storey}{2012}]{DesSto12}
\begin{barticle}[author]
\bauthor{\bsnm{Desai},~\bfnm{Keyur~H.}\binits{K.~H.}} \AND
  \bauthor{\bsnm{Storey},~\bfnm{John~D.}\binits{J.~D.}}
(\byear{2012}).
\btitle{Cross-dimensional inference of dependent high-dimensional data}.
\bjournal{J. Amer. Statist. Assoc.}
\bvolume{107}
\bpages{135--151}.
\bdoi{10.1080/01621459.2011.645777}
\bmrnumber{2949347}
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Doob}{1953}]{Doo53}
\begin{bbook}[author]
\bauthor{\bsnm{Doob},~\bfnm{J.}\binits{J.}}
(\byear{1953}).
\btitle{Stochastic Processes}.
\bpublisher{John Wiley and Sons}, \baddress{New York}.
\end{bbook}
\endbibitem

\bibitem[\protect\citeauthoryear{Dudoit and van~der Laan}{2008}]{DudVan08}
\begin{bbook}[author]
\bauthor{\bsnm{Dudoit},~\bfnm{Sandrine}\binits{S.}} \AND
  \bauthor{\bparticle{van~der} \bsnm{Laan},~\bfnm{Mark~J.}\binits{M.~J.}}
(\byear{2008}).
\btitle{Multiple testing procedures with applications to genomics}.
\bseries{Springer Series in Statistics}.
\bpublisher{Springer}, \baddress{New York}.
\bdoi{10.1007/978-0-387-49317-6}
\bmrnumber{2373771 (2009j:62004)}
\end{bbook}
\endbibitem

\bibitem[\protect\citeauthoryear{Efron}{2008}]{Efr08}
\begin{barticle}[author]
\bauthor{\bsnm{Efron},~\bfnm{B.}\binits{B.}}
(\byear{2008}).
\btitle{Microarrays, Empirical Bayes and the Two-Group Model}.
\bjournal{Statistical Science}
\bvolume{23}
\bpages{1--22}.
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Efron}{2010}]{Efr10}
\begin{bbook}[author]
\bauthor{\bsnm{Efron},~\bfnm{Bradley}\binits{B.}}
(\byear{2010}).
\btitle{Large-scale inference}.
\bseries{Institute of Mathematical Statistics (IMS) Monographs}
\bvolume{1}.
\bpublisher{Cambridge University Press}, \baddress{Cambridge}.
\bnote{Empirical Bayes methods for estimation, testing, and prediction}.
\bmrnumber{2724758 (2012a:62006)}
\end{bbook}
\endbibitem

\bibitem[\protect\citeauthoryear{Efron et~al.}{2001}]{EfrTib01}
\begin{barticle}[author]
\bauthor{\bsnm{Efron},~\bfnm{Bradley}\binits{B.}},
  \bauthor{\bsnm{Tibshirani},~\bfnm{Robert}\binits{R.}},
  \bauthor{\bsnm{Storey},~\bfnm{John~D.}\binits{J.~D.}} \AND
  \bauthor{\bsnm{Tusher},~\bfnm{Virginia}\binits{V.}}
(\byear{2001}).
\btitle{Empirical {B}ayes analysis of a microarray experiment}.
\bjournal{J. Amer. Statist. Assoc.}
\bvolume{96}
\bpages{1151--1160}.
\bdoi{10.1198/016214501753382129}
\bmrnumber{1946571}
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Fan, Han and Gu}{2012}]{Fan12}
\begin{barticle}[author]
\bauthor{\bsnm{Fan},~\bfnm{Jianqing}\binits{J.}},
  \bauthor{\bsnm{Han},~\bfnm{Xu}\binits{X.}} \AND
  \bauthor{\bsnm{Gu},~\bfnm{Weijie}\binits{W.}}
(\byear{2012}).
\btitle{Estimating false discovery proportion under arbitrary covariance
  dependence}.
\bjournal{J. Amer. Statist. Assoc.}
\bvolume{107}
\bpages{1019--1035}.
\bdoi{10.1080/01621459.2012.720478}
\bmrnumber{3010887}
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Finner, Dickhaus and Roters}{2007}]{Fin07}
\begin{barticle}[author]
\bauthor{\bsnm{Finner},~\bfnm{Helmut}\binits{H.}},
  \bauthor{\bsnm{Dickhaus},~\bfnm{Thorsten}\binits{T.}} \AND
  \bauthor{\bsnm{Roters},~\bfnm{Markus}\binits{M.}}
(\byear{2007}).
\btitle{Dependency and false discovery rate: asymptotics}.
\bjournal{Ann. Statist.}
\bvolume{35}
\bpages{1432--1455}.
\bdoi{10.1214/009053607000000046}
\bmrnumber{2351092 (2008j:62020)}
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Finner, Dickhaus and Roters}{2009}]{Fin09}
\begin{barticle}[author]
\bauthor{\bsnm{Finner},~\bfnm{Helmut}\binits{H.}},
  \bauthor{\bsnm{Dickhaus},~\bfnm{Thorsten}\binits{T.}} \AND
  \bauthor{\bsnm{Roters},~\bfnm{Markus}\binits{M.}}
(\byear{2009}).
\btitle{On the false discovery rate and an asymptotically optimal rejection
  curve}.
\bjournal{Ann. Statist.}
\bvolume{37}
\bpages{596--618}.
\bdoi{10.1214/07-AOS569}
\bmrnumber{2502644 (2010f:62198)}
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Finner, Gontscharuk and
  Dickhaus}{2012}]{Fin12}
\begin{barticle}[author]
\bauthor{\bsnm{Finner},~\bfnm{Helmut}\binits{H.}},
  \bauthor{\bsnm{Gontscharuk},~\bfnm{Veronika}\binits{V.}} \AND
  \bauthor{\bsnm{Dickhaus},~\bfnm{Thorsten}\binits{T.}}
(\byear{2012}).
\btitle{False discovery rate control of step-up-down tests with special
  emphasis on the asymptotically optimal rejection curve}.
\bjournal{Scand. J. Stat.}
\bvolume{39}
\bpages{382--397}.
\bdoi{10.1111/j.1467-9469.2012.00791.x}
\bmrnumber{2927031}
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Gavrilov, Benjamini and Sarkar}{2009}]{Gav09}
\begin{barticle}[author]
\bauthor{\bsnm{Gavrilov},~\bfnm{Yulia}\binits{Y.}},
  \bauthor{\bsnm{Benjamini},~\bfnm{Yoav}\binits{Y.}} \AND
  \bauthor{\bsnm{Sarkar},~\bfnm{Sanat~K.}\binits{S.~K.}}
(\byear{2009}).
\btitle{An adaptive step-down procedure with proven {FDR} control under
  independence}.
\bjournal{Ann. Statist.}
\bvolume{37}
\bpages{619--629}.
\bdoi{10.1214/07-AOS586}
\bmrnumber{2502645 (2010f:62199)}
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Genovese, Roeder and Wasserman}{2006}]{Gen06}
\begin{barticle}[author]
\bauthor{\bsnm{Genovese},~\bfnm{C.}\binits{C.}},
  \bauthor{\bsnm{Roeder},~\bfnm{K.}\binits{K.}} \AND
  \bauthor{\bsnm{Wasserman},~\bfnm{L.}\binits{L.}}
(\byear{2006}).
\btitle{False discovery control with {$p$}-value weighting}.
\bjournal{Biometrika}
\bvolume{93}
\bpages{509--524}.
\bmrnumber{MR2261439 (2008c:62018)}
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Genovese and Wasserman}{2002}]{GenWas02}
\begin{barticle}[author]
\bauthor{\bsnm{Genovese},~\bfnm{C.}\binits{C.}} \AND
  \bauthor{\bsnm{Wasserman},~\bfnm{L.}\binits{L.}}
(\byear{2002}).
\btitle{Operating characteristic and extensions of the false discovery rate
  procedure}.
\bjournal{J. R. Stat. Soc. Ser. B Stat Methodol.}
\bvolume{64}
\bpages{499-517}.
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Genovese and Wasserman}{2004}]{GenWas04}
\begin{barticle}[author]
\bauthor{\bsnm{Genovese},~\bfnm{Christopher}\binits{C.}} \AND
  \bauthor{\bsnm{Wasserman},~\bfnm{Larry}\binits{L.}}
(\byear{2004}).
\btitle{A stochastic process approach to false discovery control}.
\bjournal{Ann. Statist.}
\bvolume{32}
\bpages{1035--1061}.
\bdoi{10.1214/009053604000000283}
\bmrnumber{2065197 (2005k:62149)}
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Guo and Sarkar}{2012}]{GuoSar12}
\begin{barticle}[author]
\bauthor{\bsnm{Guo},~\bfnm{W.}\binits{W.}} \AND
  \bauthor{\bsnm{Sarkar},~\bfnm{S.}\binits{S.}}
(\byear{2012}).
\btitle{Adaptive controls of FWER and FDR under block dependence}.
\bjournal{avaliable at web.njit.edu/~wguo}.
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Habiger}{2012}]{Hab12}
\begin{barticle}[author]
\bauthor{\bsnm{Habiger},~\bfnm{Joshua}\binits{J.}}
(\byear{2012}).
\btitle{A method for modifying multiple testing procedures}.
\bjournal{J. Statist. Plann. Inference}
\bvolume{142}
\bpages{2227--2231}.
\bdoi{10.1016/j.jspi.2012.02.010}
\bmrnumber{2903425}
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Habiger and Pe{\~n}a}{2011}]{HabPen11}
\begin{barticle}[author]
\bauthor{\bsnm{Habiger},~\bfnm{J.}\binits{J.}} \AND
  \bauthor{\bsnm{Pe{\~n}a},~\bfnm{E.}\binits{E.}}
(\byear{2011}).
\btitle{Randomized p-values and nonparametric procedures in multiple testing}.
\bjournal{J. Nonparametr. Stat.}
\bvolume{23}
\bpages{583-604}.
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Habiger and Pe{\~n}a}{2014}]{HabPen14}
\begin{barticle}[author]
\bauthor{\bsnm{Habiger},~\bfnm{Joshua~D.}\binits{J.~D.}} \AND
  \bauthor{\bsnm{Pe{\~n}a},~\bfnm{Edsel~A.}\binits{E.~A.}}
(\byear{2014}).
\btitle{Compound {$p$}-value statistics for multiple testing procedures}.
\bjournal{J. Multivariate Anal.}
\bvolume{126}
\bpages{153--166}.
\bdoi{10.1016/j.jmva.2014.01.007}
\bmrnumber{3173088}
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Hoeffding}{1956}]{Hoe56}
\begin{barticle}[author]
\bauthor{\bsnm{Hoeffding},~\bfnm{Wassily}\binits{W.}}
(\byear{1956}).
\btitle{On the distribution of the number of successes in independent trials}.
\bjournal{Ann. Math. Statist.}
\bvolume{27}
\bpages{713--721}.
\bmrnumber{0080391 (18,240f)}
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Hu, Zhao and Zhou}{2010}]{Hu10}
\begin{barticle}[author]
\bauthor{\bsnm{Hu},~\bfnm{James~X.}\binits{J.~X.}},
  \bauthor{\bsnm{Zhao},~\bfnm{Hongyu}\binits{H.}} \AND
  \bauthor{\bsnm{Zhou},~\bfnm{Harrison~H.}\binits{H.~H.}}
(\byear{2010}).
\btitle{False discovery rate control with groups}.
\bjournal{J. Amer. Statist. Assoc.}
\bvolume{105}
\bpages{1215--1227}.
\bdoi{10.1198/jasa.2010.tm09329}
\bmrnumber{2752616 (2012b:62252)}
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Liang and Nettleton}{2012}]{LiaNet12}
\begin{barticle}[author]
\bauthor{\bsnm{Liang},~\bfnm{Kun}\binits{K.}} \AND
  \bauthor{\bsnm{Nettleton},~\bfnm{Dan}\binits{D.}}
(\byear{2012}).
\btitle{Adaptive and dynamic adaptive procedures for false discovery rate
  control and estimation}.
\bjournal{J. R. Stat. Soc. Ser. B Stat. Methodol.}
\bvolume{74}
\bpages{163--182}.
\bdoi{10.1111/j.1467-9868.2011.01001.x}
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{McCullagh and Nelder}{1989}]{McCNel89}
\begin{bbook}[author]
\bauthor{\bsnm{McCullagh},~\bfnm{P.}\binits{P.}} \AND
  \bauthor{\bsnm{Nelder},~\bfnm{J.~A.}\binits{J.~A.}}
(\byear{1989}).
\btitle{Generalized linear models}.
\bseries{Monographs on Statistics and Applied Probability}.
\bpublisher{Chapman \& Hall, London}
\bnote{Second edition [of MR0727836]}.
\bdoi{10.1007/978-1-4899-3242-6}
\bmrnumber{3223057}
\end{bbook}
\endbibitem

\bibitem[\protect\citeauthoryear{Pe{\~n}a, Habiger and Wu}{2011}]{Pen11}
\begin{barticle}[author]
\bauthor{\bsnm{Pe{\~n}a},~\bfnm{Edsel~A.}\binits{E.~A.}},
  \bauthor{\bsnm{Habiger},~\bfnm{Joshua~D.}\binits{J.~D.}} \AND
  \bauthor{\bsnm{Wu},~\bfnm{Wensong}\binits{W.}}
(\byear{2011}).
\btitle{Power-enhanced multiple decision functions controlling family-wise
  error and false discovery rates}.
\bjournal{Ann. Statist.}
\bvolume{39}
\bpages{556--583}.
\bdoi{10.1214/10-AOS844}
\bmrnumber{2797856 (2012b:62055)}
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Pe{\~n}a, Habiger and Wu}{2014}]{Pen14}
\begin{barticle}[author]
\bauthor{\bsnm{Pe{\~n}a},~\bfnm{EdselA.}\binits{E.}},
  \bauthor{\bsnm{Habiger},~\bfnm{JoshuaD.}\binits{J.}} \AND
  \bauthor{\bsnm{Wu},~\bfnm{Wensong}\binits{W.}}
(\byear{2014}).
\btitle{Classes of multiple decision functions strongly controlling FWER and
  FDR}.
\bjournal{Metrika}
\bpages{1-33}.
\bdoi{10.1007/s00184-014-0516-6}
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Roeder and Wasserman}{2009}]{RoeWas09}
\begin{barticle}[author]
\bauthor{\bsnm{Roeder},~\bfnm{Kathryn}\binits{K.}} \AND
  \bauthor{\bsnm{Wasserman},~\bfnm{Larry}\binits{L.}}
(\byear{2009}).
\btitle{Genome-wide significance levels and weighted hypothesis testing}.
\bjournal{Statist. Sci.}
\bvolume{24}
\bpages{398--413}.
\bdoi{10.1214/09-STS289}
\bmrnumber{2779334 (2011k:62307)}
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Roquain and van~de Wiel}{2009}]{RoqVan09}
\begin{barticle}[author]
\bauthor{\bsnm{Roquain},~\bfnm{Etienne}\binits{E.}} \AND
  \bauthor{\bparticle{van~de} \bsnm{Wiel},~\bfnm{Mark~A.}\binits{M.~A.}}
(\byear{2009}).
\btitle{Optimal weighting for false discovery rate control}.
\bjournal{Electron. J. Stat.}
\bvolume{3}
\bpages{678--711}.
\bdoi{10.1214/09-EJS430}
\bmrnumber{2521216 (2010k:62307)}
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Roquain and Villers}{2011}]{RoqVil11}
\begin{barticle}[author]
\bauthor{\bsnm{Roquain},~\bfnm{Etienne}\binits{E.}} \AND
  \bauthor{\bsnm{Villers},~\bfnm{Fanny}\binits{F.}}
(\byear{2011}).
\btitle{Exact calculations for false discovery proportion with application to
  least favorable configurations}.
\bjournal{Ann. Statist.}
\bvolume{39}
\bpages{584--612}.
\bdoi{10.1214/10-AOS847}
\bmrnumber{2797857 (2012b:62255)}
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Sarkar}{2007}]{Sar07}
\begin{barticle}[author]
\bauthor{\bsnm{Sarkar},~\bfnm{S.}\binits{S.}}
(\byear{2007}).
\btitle{Stepup procedures controlling generalized {FWER} and generalized
  {FDR}}.
\bjournal{Ann. Statist.}
\bvolume{35}
\bpages{2405--2420}.
\bdoi{10.1214/009053607000000398}
\bmrnumber{MR2382652 (2009e:62260)}
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Schweder and Spjotvoll}{1982}]{Sch82}
\begin{barticle}[author]
\bauthor{\bsnm{Schweder},~\bfnm{T.}\binits{T.}} \AND
  \bauthor{\bsnm{Spjotvoll},~\bfnm{E.}\binits{E.}}
(\byear{1982}).
\btitle{Plots of P-Values to Evaluate Many Tests Simultaneously}.
\bjournal{Biometrika}
\bvolume{69}
\bpages{pp. 493-502}.
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Simes}{1986}]{Sim86}
\begin{barticle}[author]
\bauthor{\bsnm{Simes},~\bfnm{R.~J.}\binits{R.~J.}}
(\byear{1986}).
\btitle{An improved {B}onferroni procedure for multiple tests of significance}.
\bjournal{Biometrika}
\bvolume{73}
\bpages{751--754}.
\bmrnumber{MR897872 (88d:62043)}
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Spj{\o}tvoll}{1972}]{Spj72}
\begin{barticle}[author]
\bauthor{\bsnm{Spj{\o}tvoll},~\bfnm{E.}\binits{E.}}
(\byear{1972}).
\btitle{On the optimality of some multiple comparison procedures}.
\bjournal{Ann. Math. Statist.}
\bvolume{43}
\bpages{398--411}.
\bmrnumber{MR0301871 (46 \#\#1026)}
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Stevens}{1950}]{Ste50}
\begin{barticle}[author]
\bauthor{\bsnm{Stevens},~\bfnm{W.~L.}\binits{W.~L.}}
(\byear{1950}).
\btitle{Fiducial limits of the parameter of a discontinuous distribution}.
\bjournal{Biometrika}
\bvolume{37}
\bpages{117--129}.
\bmrnumber{0035955 (12,37e)}
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Storey}{2002}]{Sto02}
\begin{barticle}[author]
\bauthor{\bsnm{Storey},~\bfnm{J.}\binits{J.}}
(\byear{2002}).
\btitle{A direct approach to false discovery rates}.
\bjournal{J. R. Stat. Soc. Ser. B Stat Methodol.}
\bvolume{64}
\bpages{479 - 498}.
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Storey}{2003}]{Sto03}
\begin{barticle}[author]
\bauthor{\bsnm{Storey},~\bfnm{J.}\binits{J.}}
(\byear{2003}).
\btitle{The positive false discovery rate: a Bayesian interpretation and the
  q-Value}.
\bjournal{Ann. Statist.}
\bvolume{31}
\bpages{2012 - 2035}.
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Storey}{2007}]{Sto07}
\begin{barticle}[author]
\bauthor{\bsnm{Storey},~\bfnm{John~D.}\binits{J.~D.}}
(\byear{2007}).
\btitle{The optimal discovery procedure: a new approach to simultaneous
  significance testing}.
\bjournal{J. R. Stat. Soc. Ser. B. Stat. Methodol.}
\bvolume{69}
\bpages{347--368}.
\bdoi{10.1111/j.1467-9868.2007.005592.x}
\bmrnumber{2323757}
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Storey, Taylor and Siegmund}{2004}]{Sto04}
\begin{barticle}[author]
\bauthor{\bsnm{Storey},~\bfnm{J.~D.}\binits{J.~D.}},
  \bauthor{\bsnm{Taylor},~\bfnm{J.~E.}\binits{J.~E.}} \AND
  \bauthor{\bsnm{Siegmund},~\bfnm{D.}\binits{D.}}
(\byear{2004}).
\btitle{Strong control, conservative point estimation and simultaneous
  conservative consistency of false discovery rates: A unified approach}.
\bjournal{J. R. Stat. Soc. Ser. B. Stat. Methodol.}
\bvolume{66}
\bpages{187--205}.
\bmrnumber{MR2035766 (2004k:62056)}
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Sun and Cai}{2007}]{SunCai07}
\begin{barticle}[author]
\bauthor{\bsnm{Sun},~\bfnm{Wenguang}\binits{W.}} \AND
  \bauthor{\bsnm{Cai},~\bfnm{T.~Tony}\binits{T.~T.}}
(\byear{2007}).
\btitle{Oracle and adaptive compound decision rules for false discovery rate
  control}.
\bjournal{J. Amer. Statist. Assoc.}
\bvolume{102}
\bpages{901--912}.
\bdoi{10.1198/016214507000000545}
\bmrnumber{MR2411657}
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Sun and McLain}{2012}]{SunMcL12}
\begin{barticle}[author]
\bauthor{\bsnm{Sun},~\bfnm{Wenguang}\binits{W.}} \AND
  \bauthor{\bsnm{McLain},~\bfnm{Alexander~C.}\binits{A.~C.}}
(\byear{2012}).
\btitle{Multiple testing of composite null hypotheses in heteroscedastic
  models}.
\bjournal{J. Amer. Statist. Assoc.}
\bvolume{107}
\bpages{673--687}.
\bdoi{10.1080/01621459.2012.664505}
\bmrnumber{2980076}
\end{barticle}
\endbibitem

\bibitem[\protect\citeauthoryear{Tamhane, Liu and Dunnett}{1998}]{Tam98}
\begin{barticle}[author]
\bauthor{\bsnm{Tamhane},~\bfnm{Ajit~C.}\binits{A.~C.}},
  \bauthor{\bsnm{Liu},~\bfnm{Wei}\binits{W.}} \AND
  \bauthor{\bsnm{Dunnett},~\bfnm{Charles~W.}\binits{C.~W.}}
(\byear{1998}).
\btitle{A generalized step-up-down multiple test procedure}.
\bjournal{Canad. J. Statist.}
\bvolume{26}
\bpages{353--363}.
\bdoi{10.2307/3315516}
\bmrnumber{1648451}
\end{barticle}
\endbibitem

\end{thebibliography}
