Inference in partially identified models with many moment inequalities using Lasso\thanks{We thank useful comments and suggestions from the participants in the 2015 World Congress in Montreal, the Second International Workshop in Financial Econometrics, and the seminar at University of Maryland. Bugni acknowledges support by National Institutes of Health under grant no.\ 40-4153-00-0-85-399. % The research of the second author was supported by National Science Foundation Grant SES-1530534. Bredahl Kock acknowledges support from CREATES - Center for Research in Econometric Analysis of Time Series (DNRF78), funded by the Danish National Research Foundation. Lahiri acknowledges support from National Science Foundation under grant no.\ DMS 130068.
% Federico A. Bugni\footnote{ % Department of Economics
% Duke University
% \url{federico.bugni@duke.edu
Federico A. Bugni
Department of Economics
Duke University %
% \url{federico.bugni@duke.edu
This paper considers inference in a partially identified moment (in)equality model with many moment inequalities. We propose a novel two-step inference procedure that combines the methods proposed by \cite{chernozhukov/chetverikov/kato:2014c (\citetalias{chernozhukov/chetverikov/kato:2014c, hereafter) with a first-step moment inequality selection based on the Lasso. Our method controls size uniformly, both in underlying parameter and data distribution. Also, the power of our method compares favorably with that of the corresponding two-step method in \citetalias{chernozhukov/chetverikov/kato:2014c for large parts of the parameter space, both in theory and in simulations. Finally, our Lasso-based first step is straightforward to implement. % This paper considers the problem of inference in a partially identified moment (in)equality model with possibly many moment inequalities. Our contribution is to propose a novel two-step new inference method based on the combination of two ideas. On the one hand, our test statistic and critical values are based on those proposed by \cite{chernozhukov/chetverikov/kato:2014c (\citetalias{chernozhukov/chetverikov/kato:2014c, hereafter). On the other hand, we propose a new first step selection procedure based on the Lasso. Some of the advantages of our two-step inference method are that (i) it can be used to conduct hypothesis tests and to construct confidence sets for the true parameter value that are uniformly valid, both in the underlying parameter $\theta$ and the distribution of the data; (ii) our test is asymptotically optimal in a minimax sense and (iii) our method has better power than \citetalias{chernozhukov/chetverikov/kato:2014c in large parts of the parameter space, both in theory and in simulations. Finally, we show that the Lasso-based first step can be implemented with a thresholding least squares procedure that makes it extremely simple to compute. \vspace{0.3in \noindent Keywords and phrases: Many moment inequalities, self-normalizing sum, multiplier bootstrap, empirical bootstrap, Lasso, inequality selection. \noindent \medskip \noindent JEL classification: C13, C23, C26.
JoE6.tex
	%  Paper ``Inference in partially identified models with many moment inequalities using Lasso'' by Bugni, Caner, Kock, Lahiri
%  Created by Federico Bugni on May 28, 2012.
%  Copyright (c) 2012 Duke University. All rights reserved.
\documentclass[10pt]{article}\synctex=1 % linkear
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{geometry}
\usepackage{commath}
\usepackage{booktabs}
\usepackage{lscape}
\geometry{letterpaper,left=1.0in,right=1.0in,top=1.2in,bottom=1.0in}
\usepackage{xr}
% \externaldocument{JasaSupplement}
%-------- Bibliography ---------------------------------------
\bibliographystyle{ecta}
\usepackage[authoryear]{natbib}
\renewcommand{\theequation}{\thesection.\arabic{equation}}
\newcommand{\myref}[2]{\hyperref[#1]{#2}}
\numberwithin{equation}{section}

%-------- PACKAGES ------------------------------------------
\usepackage{latexsym}
\usepackage{graphicx} % include figures
\usepackage{enumerate}
\usepackage{setspace}
% \onehalfspacing
\doublespacing
% \setstretch{1.20}
\usepackage[toc,title,titletoc,header]{appendix}
\usepackage[bottom]{footmisc}   % forces the footnotes to go at the bottom of page.
\usepackage[pdfborder={0 0 0}]{hyperref}

% Lines
\interfootnotelinepenalty=10000
\usepackage{lineno}
\setpagewiselinenumbers
%%\modulolinenumbers[2]
% \linenumbers

%-------- ENVIRONMENTS ------------------------------------------
\newtheorem{theorem}{Theorem}[section]
\newtheorem{lemma}{Lemma}[section]
\newtheorem{corollary}{Corollary}[section]
\newtheorem{assumption}{Assumption}[section]
\newtheorem{summary}{Summary}[section]
\theoremstyle{definition}
\newtheorem{definition}{Definition}[section]
\newtheorem{example}{Example}[section]
\newtheorem{exercise}{Exercise}[section]
\newtheorem{remark}{Remark}[section]
\newtheorem{conclusion}{Conclusion}[section]

\theoremstyle{remark}
\newtheorem{comment}{Comment}[section]
\newtheorem{conjecture}{Conjecture}[section]
\newtheorem{algorithm}{Algorithm}[section]
\newcounter{assumptionM}
\newcounter{assumptionA}
\def\theassumptionM{M.\arabic{assumptionM}}
\def\theassumptionA{A.\arabic{assumptionA}}
\setcounter{assumptionM}{0}
\setcounter{assumptionA}{0}
\newenvironment{assumptionA}[1][]{\refstepcounter{assumptionA}\medskip\noindent{\textbf{Assumption \theassumptionA. #1}}}{\medskip}
\newenvironment{assumptionM}[1][]{\refstepcounter{assumptionM}\medskip\noindent{\textbf{Assumption \theassumptionM. #1}}}{\medskip}

%-----New commands---------------
\newcommand{\sgn}{\mathrm{sign}}
%-----------

\hypersetup{
  colorlinks=true,linkcolor=blue, citecolor=blue, filecolor=blue, 
  linkbordercolor={0 1 1}, citebordercolor={1 0 0},
  pdftitle={Inference in partially identified models with many moment inequalities using Lasso},
  pdfauthor={Bugni, Caner, Kock, and Lahiri},
  pdfcreator={\LaTeX\ with package \flqq hyperref\frqq},
  pdfsubject={},
}

\defcitealias{chernozhukov/chetverikov/kato:2014c}{CCK14}


\begin{document}
\sloppy

\title{
Inference in partially identified models with many moment inequalities using Lasso\thanks{We thank useful comments and suggestions from the participants in the 2015 World Congress in Montreal, the Second International Workshop in Financial Econometrics, and the seminar at University of Maryland. Bugni acknowledges support by National Institutes of Health under grant no.\ 40-4153-00-0-85-399. 
% The research of the second author was supported by National Science Foundation Grant SES-1530534.
Bredahl Kock acknowledges support from CREATES - Center for Research in Econometric Analysis of Time Series (DNRF78), funded by the Danish National Research Foundation. Lahiri acknowledges support from National Science Foundation under grant no.\ DMS 130068.}
}

% \author{
% Federico A. Bugni\footnote{
% Department of Economics,
% Duke University,
% \url{federico.bugni@duke.edu}}
% \and
% Mehmet Caner\footnote{
% Department of Economics,
% Ohio State University,
% \url{mcanerosu@gmail.com}}
% \and
% Anders Bredahl Kock\footnote{
% Department of Economics and Business,
% Aarhus University,
% \url{akock@econ.au.dk}}
% \and
% {Soumendra Lahiri}\footnote{
% Department of Statistics,
% North Carolina State University,
% \url{snlahiri@ncsu.edu}}
% }
\author{
Federico A. Bugni\\
Department of Economics\\
Duke University
% \\
% \url{federico.bugni@duke.edu}
\and
Mehmet Caner\\
Department of Economics\\
Ohio State University
% \\
% \url{mcanerosu@gmail.com}
\and
Anders Bredahl Kock\\
Department of Economics \& Business\\
Aarhus University
% \\
% \url{akock@econ.au.dk}
\and
{Soumendra Lahiri}\\
Department of Statistics\\
North Carolina State University
% \\
% \url{snlahiri@ncsu.edu}
}

\maketitle

\begin{abstract}
	This paper considers inference in a partially identified moment (in)equality model with many moment inequalities. We propose a novel two-step inference procedure that combines the methods proposed by \cite{chernozhukov/chetverikov/kato:2014c} (\citetalias{chernozhukov/chetverikov/kato:2014c}, hereafter) with a first-step moment inequality selection based on the Lasso. Our method controls size uniformly, both in underlying parameter and data distribution. Also, the power of our method compares favorably with that of the corresponding two-step method in \citetalias{chernozhukov/chetverikov/kato:2014c} for large parts of the parameter space, both in theory and in simulations. Finally, our Lasso-based first step is straightforward to implement. 
	
	
		% This paper considers the problem of inference in a partially identified moment (in)equality model with possibly many moment inequalities. Our contribution is to propose a novel two-step new inference method based on the combination of two ideas. On the one hand, our test statistic and critical values are based on those proposed by \cite{chernozhukov/chetverikov/kato:2014c} (\citetalias{chernozhukov/chetverikov/kato:2014c}, hereafter). On the other hand, we propose a new first step selection procedure based on the Lasso. Some of the advantages of our two-step inference method are that (i) it can be used to conduct hypothesis tests and to construct confidence sets for the true parameter value that are uniformly valid, both in the underlying parameter $\theta$ and the distribution of the data; (ii) our test is asymptotically optimal in a minimax sense and (iii) our method has better power than \citetalias{chernozhukov/chetverikov/kato:2014c} in large parts of the parameter space, both in theory and in simulations. Finally, we show that the Lasso-based first step can be implemented with a thresholding least squares procedure that makes it extremely simple to compute.
 \vspace{0.3in}

\noindent \textit{Keywords and phrases}: 
Many moment inequalities, self-normalizing sum, multiplier bootstrap, empirical bootstrap, Lasso, inequality selection.

\noindent \textit{\medskip \noindent JEL classification}: C13, C23, C26.
\end{abstract}

%\vfill

%\thispagestyle{empty} % no page number \clearpage
%\pagebreak % The paper begins in a new page
%\setcounter{page}1 % Set the next page number to 1
% \tableofcontents
% \pagebreak % The paper begins in a new page
%-------- End Title Page -----------------------------

\newpage
\section{Introduction}

This paper contributes to the growing literature on inference in partially identified econometric models defined by many unconditional moment (in)equalities, i.e.,\ inequalities and equalities. Consider an economic model with a parameter $\theta $ belonging to a parameter space $\Theta $, whose main prediction is that the true value of $\theta $, denoted by $ \theta _{0}$, satisfies a collection of moment (in)equalities. This model is partially identified, i.e., the restrictions of the model do not necessarily restrict $\theta _{0}$ to a single value, but rather they constrain it to belong to a certain set, called the identified set. The literature on partially identified  models discusses several examples of economic models that satisfy this structure, such as selection problems, missing data, or multiplicity of equilibria (see, e.g., \cite{manski:1995} and \cite{tamer:2003}).

The first contributions in the literature of partially identified moment (in)equalities focus on the case in which there is a fixed and finite number of moment (in)equalities, both unconditionally\footnote{ These include \cite{CHT07}, \cite{andrews/berry/jiabarwick:2004}, \cite{imbens/manski:2004}, \citet{galichon/henry:2006,galichon/henry:2013}, \cite{beresteanu/molinari:2008}, \cite{romano/shaikh:2008}, \cite{rosen:2008}, \cite{andrews/guggenberger:2009}, \cite{stoye:2009}, \cite{andrews/soares:2010}, \citet{bugni:2010,bugni:2015}, \cite{canay:2010}, \cite{romano/shaikh:2010}, \cite{andrews/jiabarwick:2012}, \cite{bontemps/magnac/maurin:2012}, \cite{bugni/canay/guggenberger:2012}, \cite{romano/shaikh/wolf:2014}, and \cite{pakes/porter/ho/ishii:2015}, among others.} and conditionally\footnote{These include \cite{kim:2008}, \cite{ponomareva:2010}, \citet{armstrong:2014,armstrong:2015}, \cite{chetverikov:2013}, \cite{andrews/shi:2013}, and \cite{chernozhukov/lee/rosen:2013}, among others.}. In practice, however, there are many relevant econometric models that produce a large set of moment conditions (even infinitely many). As several references in the literature point out (e.g.\ \citet{menzel:2009,menzel:2014}), the associated inference problems cannot be properly addressed by an asymptotic framework with a fixed number of moment (in)equalities.\footnote{As pointed out by \cite{chernozhukov/chetverikov/kato:2014c}, this is true even for conditional moment (in)equality models (which typically produce an infinite number of unconditional moment (in)equalities). As they explain, the unconditional moment (in)equalities generated by conditional moment (in)equality models inherit the structure from the conditional moment conditions, which limits the underlying econometric model.} To address this issue, \cite{chernozhukov/chetverikov/kato:2014c} (hereafter referred to as \citetalias{chernozhukov/chetverikov/kato:2014c}) obtain inference results in a partially identified model with \textit{many} moment (in)equalities.\footnote{See also the related technical contributions in \citet{chernozhukov/chetverikov/kato:2013a,chernozhukov/chetverikov/kato:2013b,chernozhukov/chetverikov/kato:2014a,chernozhukov/chetverikov/kato:2014b}.} According to this asymptotic framework, the number of moment (in)equalities, denoted by $p$, is allowed to be larger than the sample size $n$. In fact, the asymptotic framework allows $p$ to be an increasing function of $n$ and even to grow at certain exponential rates. Furthermore, \citetalias{chernozhukov/chetverikov/kato:2014c} allow their moment (in)equalities to be ``unstructured'', in the sense that they do not impose restrictions on the correlation structure of the sample moment conditions.\footnote{This feature distinguishes their framework from a standard conditional moment (in)equality model. While conditional moment conditions can generate an uncountable set of unconditional moment (in)equalities, their covariance structure is greatly restricted by the conditioning structure.} For these reasons, \citetalias{chernozhukov/chetverikov/kato:2014c} represents a significant advancement relative to the previous literature on inference in moment (in)equalities.

This paper builds on the inference method proposed in \citetalias{chernozhukov/chetverikov/kato:2014c}. Their goal is to test whether a collection of $p$ moment inequalities simultaneously holds or not. In order to implement their test they propose a test statistic based on the maximum of $p$ Studentized statistics and several methods to compute the critical values. Their critical values may include a first stage inequality selection procedure with the objective of detecting slack moment inequalities, thus increasing the statistical power. According to their simulation results, including a first stage can result in significant power gains.

Our contribution is to propose a new inference method based on the combination of two ideas. On the one hand, our test statistic and critical values are based on those proposed by \citetalias{chernozhukov/chetverikov/kato:2014c}. On the other hand, we propose a new first stage selection procedure based on the Lasso. The Lasso was first proposed in the seminal contribution by \cite{tibshirani:1996} as a regularization technique in the linear regression model. Since then, this method has found wide use as a dimension reduction technique in large dimensional models with strong theoretical underpinnings.\footnote{For excellent reviews of this method see, e.g., \cite{belloni/chernozhukov:2011}, \cite{buhlmann/vandegeer:2011}, \cite{fan/lv/qi:2011}, and \cite{hastie/tibshirani/wainwright:2015}.} It is precisely these powerful shrinkage properties that serve as motivation to consider the Lasso as a procedure to separate out and select binding moment inequalities from the non-binding ones. Our Lasso first step inequality selection can be combined with any of the second step inference procedures in \citetalias{chernozhukov/chetverikov/kato:2014c}: self-normalization, multiplier bootstrap, or empirical bootstrap.

The present paper considers using the Lasso to select moments in a partially identified moment (in)equality model. In the context of point identified problems, there is an existing literature that proposes the Lasso to address estimation and moment selection in GMM settings. In particular, \cite{caner:2009} introduce Lasso type GMM-Bridge estimators to estimate structural parameters in a general model. The problem of selection of moment in GMM is studied in \cite{liao:2013} and \cite{cheng/liao:2015}. In addition, \cite{caner/zhang:2014} and \cite{caner/han/lee:2016} find a method to estimate parameters in GMM with diverging number of moments/parameters, and selecting valid moments among many valid or invalid moments respectively. In addition, \cite{fan/liao/yao:2015} consider the problem of inference in high dimensional models with sparse alternatives. Finally, \cite{caner/fan:2015} propose a hybrid two-step estimation procedure based on Generalized Empirical Likelihood, where instruments are chosen in a first-stage using an adaptive Lasso procedure.

We obtain the following results for our two-step Lasso inference methods. First, we provide conditions under which our methods are uniformly valid, both in the underlying parameter $\theta $ and the distribution of the data. According to the literature in moment (in)equalities, obtaining uniformly valid asymptotic results is important to guarantee that the asymptotic analysis provides an accurate approximation to finite sample results.\footnote{In these models, the limiting distribution of the test statistic is discontinuous in the slackness of the moment inequalities, while its finite sample distribution does not exhibit such discontinuities. In consequence, asymptotic results obtained for any fixed distribution (i.e.\ pointwise asymptotics) can be grossly misleading, and possibly producing confidence sets that undercover (even asymptotically). See \cite{imbens/manski:2004}, \cite{andrews/guggenberger:2009}, \cite{andrews/soares:2010}, and \cite{andrews/shi:2013} (Section 5.1).} Second, by virtue of results in \citetalias{chernozhukov/chetverikov/kato:2014c}, all of our proposed tests are asymptotically optimal in a minimax sense. Third, we compare the power of our methods to the corresponding one in \citetalias{chernozhukov/chetverikov/kato:2014c}, both in theory and in simulations. Since our two-step procedure and the corresponding one in \citetalias{chernozhukov/chetverikov/kato:2014c} share the second step, our power comparison is a comparison of the Lasso-based first-step vis-\`a-vis the ones in \citetalias{chernozhukov/chetverikov/kato:2014c}. On the theory front, we obtain a region of underlying parameters under which the power of our method dominates that of \citetalias{chernozhukov/chetverikov/kato:2014c}. We also conduct extensive simulations to explore the practical consequences of our theoretical findings. Our simulations indicate that a Lasso-based first step is usually as powerful as the one in \citetalias{chernozhukov/chetverikov/kato:2014c}, and can sometimes be more powerful. Fourth, we show that our Lasso-based first step is straightforward to implement.

The remainder of the paper is organized as follows. Section \ref{sec:Setup} describes the inference problem and introduces our assumptions. Section \ref{sec:Lasso} introduces the Lasso as a method to distinguish binding moment inequalities from non-binding ones and Section \ref{sec:Inference} considers inference methods that use the Lasso as a first step. Section \ref{sec:Power} compares the power properties of inference methods based on the Lasso with the ones in the literature. Section \ref{sec:MonteCarlos} provides evidence of the finite sample performance using Monte Carlo simulations. Section \ref{sec:Conclusions} concludes. Proofs of the main results and several intermediate results are reported in the appendix. 

Throughout the paper, we use the following notation. For any set $S$, $|S|$ denotes its cardinality, and for any vector $x \in \mathbb{R}^{d}$, $||x||_{1}\equiv \sum_{i=1}^{d}|x_i|$.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%5
\section{Setup}\label{sec:Setup}


For each $\theta \in \Theta $, let $X(\theta):\Omega \to \mathbb{R}^{k}$ be a $k$-dimensional random variable with distribution $P(\theta )$ and mean $\mu (\theta )\equiv E_{P(\theta )}[X(\theta)]\in \mathbb{R} ^{k}$. Let $\mu _{j}(\theta )$ denote the $j$th component of $\mu(\theta)$ so that $\mu (\theta )=\{\mu _{j}(\theta )\}_{j\leq k}$.
%Let $P\equiv \{P(\theta ):\theta \in \Theta \}$ denote the collection of distributions, which is
% assumed to belong to the set $\mathcal{P}$.
The main tenet of the econometric model is that the true parameter value $\theta _{0}$ satisfies the following collection of $p$ moment inequalities and $v\equiv k-p$ moment equalities: 
\begin{eqnarray} 
	\mu _{j}(\theta _{0}) &\leq &0\text{ for }j=1,\ldots ,p, \notag \\
	\mu _{j}(\theta _{0}) &=&0\text{ for }j=p+1,\ldots ,k. \label{eq:MI}
\end{eqnarray} 
As in \citetalias{chernozhukov/chetverikov/kato:2014c}, we are implicitly allowing the collection $P$ of distributions of $X(\theta)$ and the number of moment (in)equalities, $k = p+v$ to depend on $n$. In particular, we are primarily interested in the case in which $p=p_{n}\to \infty$ and $v=v_{n}\to \infty$ as $n \to \infty$, but the subscripts will be omitted to keep the notation simple. In particular, $p$ and $v$ can be much larger than the sample size and increase at rates made precise in Section \ref{sec:Assumptions}. We allow the econometric model to be partially identified, i.e., the moment (in)equalities in Eq.\ \eqref{eq:MI} do not necessarily restrict $\theta _{0}$ to a single value, but rather they constrain it to belong to the identified set, denoted by $\Theta _{I}(P)$. By definition, the identified set is as follows:
\begin{equation}
\Theta _{I}(P)~\equiv ~\cbr[4]{ \theta \in \Theta :\cbr[4]{
\begin{array}{l}
	\mu _{j}(\theta )\leq 0\text{ for }j=1,\ldots ,p, \\ 
	\mu _{j}(\theta )=0\text{ for }j=p+1,\ldots ,k.
\end{array}} }.  \label{eq:IdSet}
\end{equation}

Our goal is to test whether a particular parameter value $\theta \in \Theta $ is a possible candidate for the true parameter value $\theta _{0} \in \Theta_{I}(P)$. In other words, we are interested in testing:
\begin{equation}
	H_{0}:\theta _{0}=\theta~~ \text{ vs. }~~H_{1}:\theta _{0}\not=\theta .
	\label{eq:HypTest}
\end{equation}
By definition, the identified set is composed of all parameters that are observationally equivalent to the true parameter value $\theta _{0}$, i.e., every parameter value in $\Theta _{I}(P)$ is a candidate for $\theta _{0}$. In this sense, $\theta=\theta _{0}$ is observationally equivalent to $\theta \in \Theta _{I}(P)$ and so the hypothesis test in Eq.\ \eqref{eq:HypTest} can be equivalently reexpressed as:
%\begin{equation}
\begin{align}
&\quad 	H_{0}:\theta \in \Theta _{I}(P)\quad \text{vs.}\quad H_{1}:\theta \not\in \Theta _{I}(P),
	\notag \\[.05in]
\text{i.e.,}\quad 
H_{0} &:\cbr[4]{\begin{array}{l}
	\mu _{j}(\theta )\leq 0\text{ for all }j=1,\ldots ,p, \\ 
	\mu _{j}(\theta )=0\text{ for all }j=p+1,\ldots ,k.
\end{array}}\quad \text{vs.}\quad H_{1} : \mbox{``not $H_0$''}.
 \label{eq:HypTest2}
\end{align}
 
In this paper, we propose a procedure to implement the hypothesis test in Eq.\ \eqref{eq:HypTest} (or, equivalently, Eq.\ \eqref{eq:HypTest2}) with a given significance level $\alpha \in (0,1)$ based on a random sample of $ X(\theta)\sim P(\theta)$, denoted by $X^{n}(\theta) \equiv \{X_{i}(\theta)\}_{i\leq n}$. The inference procedure will reject the null hypothesis whenever a certain test statistic $ T_{n}(\theta )$ exceeds a critical value $c_{n}(\alpha ,\theta )$, i.e.,
\begin{equation}
\phi _{n}(\alpha ,\theta )~\equiv ~ 1[T_{n}(\theta )>c_{n}(\alpha ,\theta )],
\label{eq:HT}
\end{equation}
where $1[\cdot]$ denotes the indicator function.
  By the duality between hypothesis tests and confidence sets, a confidence set for $\theta _{0}$ can be constructed by collecting all parameter values for which the inference procedure is not rejected, i.e.,
\begin{equation}
C_{n}(1-\alpha )~\equiv ~\{\theta \in \Theta :T_{n}(\theta ) \leq c_{n}(\alpha ,\theta )\}. \label{eq:CS}
\end{equation}
Our formal results will have the following structure. Let $\mathcal{P}$ denote a set of probability distributions. We will show that for all $P \in \mathcal{P}$ and under $H_0$,
\begin{equation}
	P\del[1]{T_{n}(\theta ) > c_{n}(\alpha ,\theta )}~~\leq ~~ \alpha + o(1). \label{eq:CSvalid2}
\end{equation}
Moreover, the convergence in Eq.\ \eqref{eq:CSvalid2} will be shown to occur uniformly over both $P \in \mathcal{P}$ and $\theta\in \Theta$. This uniform size control result in Eq.\ \eqref{eq:CSvalid2} has important consequences regarding our inference problem. First, this result immediately implies that the hypothesis test procedure in Eq.\ \eqref{eq:HT} uniformly controls asymptotic size i.e., for all $\theta \in \Theta$ and under $H_0:\theta_0 = \theta$,
\begin{equation}
	\underset{n\to \infty }{\lim \sup }~\sup_{P\in \mathcal{P} }~E[\phi _{n}(\alpha ,\theta )]~~\leq ~~ \alpha. \label{eq:HTvalid}
\end{equation}
Second, the result also implies that the confidence set in Eq.\ \eqref{eq:CS} is asymptotically uniformly valid, i.e.,
\begin{equation}
	\underset{n\to \infty }{\lim \inf }~\inf_{P\in \mathcal{P} }~\inf_{\theta \in \Theta _{I}(P)}~P\del[1]{\theta\in C_{n}(1-\alpha )}~~\geq ~~1-\alpha. \label{eq:CSvalid}
\end{equation}

The rest of the section is organized as follows. Section \ref{sec:Assumptions} specifies the assumptions on the probability space $\mathcal{P}$ that are required for our analysis. All the inference methods described in this paper share the test statistic $ T_{n}(\theta )$ and differ only in the critical value $c_{n}(\alpha ,\theta )$. The common test statistic is introduced and described in Section \ref{sec:TestStat}.

\subsection{Assumptions}\label{sec:Assumptions} 

The collection of distributions $P\equiv \{P(\theta ):\theta \in \Theta \}$ are assumed 
to satisfy the following assumptions.

\begin{assumptionA} \label{ass:Basic}
	For every $\theta \in \Theta$, let ${X}^{n}(\theta) \equiv \{X_{i}(\theta)\}_{i\leq n}$ be i.i.d.\ $k$-dimensional random vectors distributed according to $P(\theta)$. Further, let $E_{P(\theta)}[X_{1j}(\theta)]\equiv \mu_{j}(\theta) $ and $Var_{P(\theta)}[X_{1j}(\theta)] \equiv \sigma^2_{j}(\theta) >0$, where $X_{ij}(\theta)$ denotes the $j$ component of $X_i(\theta)$.
\end{assumptionA}

\begin{assumptionA} \label{ass:Rates}
	For some $\delta \in (0,1]$, $ \max_{j=1,\dots ,k} \sup_{\theta \in \Theta } ( E_{P(\theta)}[|X_{1j}(\theta)|^{2+\delta }]) ^{1/( 2+\delta ) } \equiv M_{n,2+\delta }<\infty $ and $M_{n,2+\delta }^{2+\delta }( \ln (2k-p) ) ^{( 2+\delta) /2}n^{-\delta /2}\to 0$.
\end{assumptionA}

\begin{assumptionA} \label{ass:Rates2}
For some $c\in (0,1)$, $( n^{-( 1-c) /2}\ln (2k-p)+n^{-3/2}( \ln (2k-p)) ^{2}) B_{n}^{2}\to 0$, where $\sup_{\theta \in \Theta}( E_{P(\theta)}[\max_{j=1,\dots,k}|Z_{1j}(\theta)|^{4 }]) ^{1/4 }\equiv B_{n}  <\infty $ and $Z_{ij}(\theta) \equiv (X_{ij}(\theta) - \mu_{j}(\theta))/ \sigma_{j}(\theta)$.
\end{assumptionA}

\begin{assumptionA} \label{ass:Rates3}
		For some $c\in (0,1/2)$ and $C>0$, $\max \{M_{n,3}^{3},M_{n,4}^{2},B_{n}\}^{2}\ln ( (2k-p)n) ^{7/2}\leq Cn^{1/2-c}$, where $M_{n,2+\delta }$ and $B_{n}$ are as in Assumptions \ref{ass:Rates}-\ref{ass:Rates2}.
\end{assumptionA}

We now briefly describe these assumptions. Assumption \ref{ass:Basic} is standard in microeconometric applications. Assumption \ref{ass:Rates} has two parts. The first part requires that $X_{ij}(\theta)$ has finite $(2+\delta)$-moments for all $ j=1,\dots,k$. The second part limits the rate of growth of $M_{n,2+\delta }$ and the number of moment (in)equalities. Notice that $M_{n,2+\delta }$ is a function of the sample size because $ \max_{j=1,\dots ,k} \sup_{\theta \in \Theta } ( E_{P(\theta)}[|X_{1j}(\theta)|^{2+\delta }]) ^{1/( 2+\delta ) }$ is function of $P$ and $k=v+p$, both of which could depend on $n$. Also, notice that $2k-p = 2v+p$, i.e., the total number of moment inequalities $p$ plus twice the number of moment equalities $v$, all of which could depend on $n$. Assumption \ref{ass:Rates2} could be interpreted in a similar fashion as Assumption \ref{ass:Rates}, except that it refers to the standardized random variable $Z_{ij}(\theta)\equiv (X_{ij}(\theta) - \mu_{j}(\theta))/ \sigma_{j}(\theta)$. Assumption \ref{ass:Rates3} is a technical assumption that is used to control the size of the bootstrap test in \citetalias{chernozhukov/chetverikov/kato:2014c}.\footnote{We point out that Assumptions \ref{ass:Basic}-\ref{ass:Rates3} are tailored for the construction of confidence sets in Eq.\ \eqref{eq:CS} in the sense that all the relevant constants are defined uniformly in $\theta \in \Theta$. If we were only interested in the hypothesis testing problem  for a particular value of ${\theta}$, then the previous assumptions could be replaced by their ``pointwise'' versions at the parameter value of interest.}

\subsection{Test statistic}\label{sec:TestStat}
Throughout the paper, we consider the following test statistic:
\begin{equation}
T_{n}(\theta) ~\equiv~ \max \cbr[3]{\max_{j=1,\dots ,p}\frac{\sqrt{n}\hat{\mu}_{j}(\theta)}{\hat{ \sigma}_{j}(\theta)},\max_{s=p+1,\dots
,k}\frac{\sqrt{n}\left\vert \hat{\mu} _{s}(\theta)\right\vert }{\hat{\sigma}_{s}(\theta)}},
\label{eq:TestStat}
\end{equation}
where, for $j=1,\dots ,k$,
$
\hat{\mu}_{j}(\theta) \equiv \frac{1}{n}\sum_{i=1}^{n}X_{ij}(\theta)$  and 
$\hat{\sigma}_{j}^{2}(\theta) \equiv \frac{1}{n}\sum_{i=1}^{n}\left( X_{ij}(\theta)-\hat{\mu} _{j}(\theta)\right) ^{2}
$.
Note that Eq.\ \eqref{eq:TestStat} is not properly defined if $\hat{\sigma}_{j}^{2}(\theta)=0$ for some $j=1,\dots,k$ and, in such cases, we use the convention that $C/0 \equiv \infty \times 1[C>0] -\infty \times 1[C<0] $.

The test statistic is identical to that in \citetalias{chernozhukov/chetverikov/kato:2014c} with the exception that we allow for the presence of moment equalities. By definition, large values of $T_n(\theta)$ are an indication that $H_0: \theta = \theta_0$ is likely to be violated, leading to the hypothesis test in Eq.\ \eqref{eq:HT}. The remainder of the paper considers several procedures to construct critical values that can be associated to this test statistic.

\section{Lasso as a first step moment selection procedure}\label{sec:Lasso}

In order to propose a critical value for our test statistic $T_{n}(\theta)$, we need to approximate its distribution under the null hypothesis. According to the econometric model in Eq.\ \eqref{eq:MI}, the true parameter satisfies $p$ moment inequalities and $v$ moment equalities. By definition, the moment equalities are always binding under the null hypothesis. On the other hand, the moment inequalities may or may not be binding, and a successful approximation of the asymptotic distribution depends on being able to distinguish between these two cases. Incorporating this information into the hypothesis testing problem is one of the key issues in the literature on inference in partially identified moment (in)equality models.

In their seminal contribution, \citetalias{chernozhukov/chetverikov/kato:2014c} is the first paper in the literature to conduct inference in a partially identified model with many unstructured moment inequalities. Their paper proposes several procedures to select binding moment inequalities from non-binding based on three approximation methods: self-normalization (SN), multiplier bootstrap (MB), and empirical bootstrap (EB). Our relative contribution is to propose a novel approximation method based on the Lasso. By definition, the Lasso penalizes parameters values by their $\ell_{1}$-norm, with the ability of producing parameter estimates that are exactly equal to zero. This powerful shrinkage property is precisely what motivates us to consider the Lasso as a first step moment selection procedure in a model with many moment (in)equalities. As we will soon show, the Lasso is an excellent method to detect binding moment inequalities from non-binding ones, and this information can be successfully incorporated into an inference procedure for many moment (in)equalities.

For every $\theta \in \Theta$, let $J(\theta)$ denote the true set of binding moment inequalities, i.e., $J(\theta)~\equiv~ \{j=1,\dots ,p~:~\mu _{j}(\theta)\geq 0\}$. Let $\mu_{I}(\theta) \equiv \{\mu_j(\theta)\}_{j=1}^{p}$ denote the moment vector for the moment inequalities and let $\hat{\mu}_{I}(\theta) \equiv \{\hat{\mu}_j(\theta)\}_{j=1}^{p}$ denote its sample analogue. In order to detect binding moment inequalities, we consider the weighted Lasso estimator of $\mu_{I}(\theta)$, given by:
\begin{equation}
\hat{\mu}_{L}(\theta) ~\equiv~  \underset{t\in \mathbb{R} ^{p}}{\arg \min } \cbr[3]{ \left( \hat{\mu}_{I}(\theta) -t\right)^{\prime }\hat{W}(\theta)\left( \hat{\mu}_{I}(\theta)-t\right) + \lambda _{n}\left\Vert \hat{W}(\theta)^{1/2}t\right\Vert
_{1}}, \label{eq:Lasso0}
\end{equation}
where $\lambda _{n}$ is a positive penalization sequence that controls the amount of regularization and $\hat{W}(\theta)$ is a positive definite weighting matrix. To simplify the computation of the Lasso estimator, we impose $\hat{W}(\theta) \equiv diag\{ 1/\hat{\sigma}_{j}(\theta)^{2}\} _{j=1}^{p}$. As a consequence, Eq.\ \eqref{eq:Lasso0} becomes:
\begin{equation}
\hat{\mu}_{L}(\theta)  ~=~ \cbr[3]{ \underset{m\in \mathbb{R} }{\arg \min }\left\{ \left( {\hat{ \mu}_{j}(\theta)}-m\right) ^{2}+\lambda _{n}\hat{\sigma}_{j}(\theta)|m|\right\}} _{j=1}^{p}.\label{eq:Lasso1}
\end{equation}

Notice that instead of using the Lasso in one $p$-dimensional model we instead use it in $p$ one-dimensional models. As we shall see later, $\hat{\mu}_{L}(\theta)$ in Eq.\ \eqref{eq:Lasso1} is closely linked to the soft-thresholded least squares estimator, which implies that its computation is straightforward.  The Lasso estimator $\hat{\mu}_{L}(\theta)$ implies a Lasso-based estimator of $J(\theta)$, given by:
\begin{equation}
\hat{J}_{L}(\theta) ~\equiv~ \{j=1,\dots ,p:\hat{\mu}_{j,L}(\theta)/\hat{\sigma}_{j}(\theta) ~\geq~ -\lambda _{n}\}.
\label{eq:BindingLasso}
\end{equation}
In order to implement this procedure, we need to choose the sequence $\lambda_n$, which determines the degree of regularization imposed by the Lasso. A higher value of $\lambda_{n}$ will produce a larger number of moment inequalities considered to be binding, resulting in a lower rejection rate. In consequence, this is a critical choice for our inference methodology. According to our theoretical results,
 a suitable choice of $\lambda_n$ is given by:
\begin{equation}
\lambda _{n}=(4/3+\varepsilon) n^{-1/2}\left( {M}_{n,2+\delta }^{2}n^{-\delta /(2+\delta )}-n^{-1}\right) ^{-1/2} \label{eq:LambdaConcrete}
\end{equation}
for any arbitrary $\varepsilon>0$. Assumption \ref{ass:Rates} implies that $\lambda _{n}$ in Eq.\ \eqref{eq:LambdaConcrete} satisfies $\lambda_n\to 0$. Notice that Eq.\ \eqref{eq:LambdaConcrete} is infeasible as it depends on the unknown expression ${M}_{n,2+\delta }$. In practice, one can replace this unknown expression with its sample analogue:
	$$\hat{M}^{2}_{n,2+\delta } =  \max_{j=1,\dots ,k} ~\sup_{\theta \in \Theta } ~ \left(n^{-1} \sum\nolimits_{i=1}^{n}|X_{ij}(\theta)|^{2+\delta }\right) ^{2/( 2+\delta ) }.$$  
%%%%%%%%%%%%%%%%%%%
In principle, a more rigorous choice of $\lambda_{n}$ can be implemented via a modified BIC method designed for divergent number of parameters as in \cite{wang/li/leng:2009} or \cite{caner/han/lee:2016}.\footnote{Nevertheless, it is unclear whether the asymptotic properties of this method carry over to our partially identified moment (in)equality model. We consider that a rigorous handling of these issues is beyond the scope of this paper.}

As explained earlier, our Lasso procedure is used as a first step in order to detect binding moment inequalities from non-binding ones. The following result formally establishes that our Lasso procedure includes all binding ones with a probability that approaches one, uniformly.

\begin{lemma} \label{lem:LassoNoOverFit}
	Assume Assumptions \ref{ass:Basic}-\ref{ass:Rates2}, and let $\lambda_n$ be as in Eq.\ \eqref{eq:LambdaConcrete}. Then,
\begin{eqnarray*}
P[J(\theta)\subseteq \hat{J}_{L}(\theta)] ~\geq~ 1-2p\exp \Big( -
\frac{n^{\delta /(2+\delta )}}{2M_{n,2+\delta }^{2}}\Big) \left[ 1+K\Big( \frac{M_{n,2+\delta }}{n^{\delta /(2(2+\delta ))}}
                         +1\Big) ^{2+\delta }\right] +\tilde{K} n^{-c} ~=~ 1 + o(1),
\end{eqnarray*}
where $K,\tilde{K}$ are universal constants and the convergence is uniform in all parameters $\theta \in \Theta$ and distributions $P$ that satisfy the assumptions in the statement.
\end{lemma}

Thus far, our Lasso estimator of the binding constrains in Eq.\ \eqref{eq:BindingLasso} has been defined in terms of the solution of the $p$-dimensional minimization problem in Eq.\ \eqref{eq:Lasso1}. We conclude the subsection by providing an equivalent closed form solution for this set.

\begin{lemma}\label{lem:LassoClosedForm}
	Eq.\ \eqref{eq:BindingLasso} can be equivalently reexpressed as follows:	
	\begin{equation}
	\hat{J}_{L}(\theta) ~=~ \{j=1,\dots ,p: {\hat{\mu}_{j}(\theta)}/{\hat{\sigma}_{j}(\theta)}\geq -{3}\lambda_n/{2}\}.
	\label{eq:BindingLasso2}
	\end{equation}
\end{lemma}

Lemma \ref{lem:LassoClosedForm} is a very important computational aspect of our methodology. This result reveals that $\hat{J}_{L}(\theta) $ can be computed by comparing standardized sample averages with a modified threshold of $-{3}\lambda_n/{2}$. In other words, our Lasso-based first stage can be implemented without the need of solving the $p$-dimensional minimization problem in Eq.\ \eqref{eq:Lasso1}.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Inference methods with Lasso first step}\label{sec:Inference}

In the remainder of the paper we show how to conduct inference in our partially identified many moment (in)equality model by combining the Lasso-based first step in Section \ref{sec:Lasso} with a second step based on the inference methods proposed by \citetalias{chernozhukov/chetverikov/kato:2014c}. In particular, Section \ref{sec:SN} combines our Lasso-based first step with their self-normalization approximation, while Section \ref{sec:Boot} combines it with their bootstrap approximations.

\subsection{Self-normalization approximation}\label{sec:SN}

Before describing our self-normalization (SN) approximation with Lasso first stage, we first describe the ``plain vanilla'' SN approximation without first stage moment selection. Our treatment extends the SN method proposed by \citetalias{chernozhukov/chetverikov/kato:2014c} to the presence of moment equalities. 

As a preliminary step, we now define the SN approximation to the $(1-\alpha)$-quantile of $T_{n}(\theta)$ in a hypothetical moment (in)equality model composed of $|J|$ moment inequalities and $k-p$ moment equalities, given by:
\begin{equation}
c_{n}^{SN}(|J|,\alpha)\equiv \left\{
\begin{array}{ll}
0 & \text{if }2(k-p)+|J|=0, \\
\tfrac{\Phi ^{-1}\left( 1-\alpha /(2(k-p)+|J|)\right) }{\sqrt{1-\left( \Phi ^{-1}\left( 1-\alpha /(2(k-p)+|J|)\right) \right) ^{2}/n}} & \text{if } 2(k-p)+|J|>0.
\end{array}
\right.
 \label{eq:CVSN_oracle}
\end{equation}
Lemma \ref{lem:SNSize} in the appendix shows that $c_{n}^{SN}(|J|,\alpha)$ provides asymptotic uniform size control in a hypothetical moment (in)equality model with $|J|$ moment inequalities and $k-p$ moment equalities under Assumptions \ref{ass:Basic}-\ref{ass:Rates}. The main difference between this result and \citetalias{chernozhukov/chetverikov/kato:2014c} (Theorem 4.1) is that we allow for the presence of moment equalities. Since our moment (in)equality model has $|J|=p$ moment inequalities and $k-p$ moment equalities, we can define the regular (i.e.\ one-step) SN approximation method by using $|J|=p$ in Eq.\ \eqref{eq:CVSN_oracle}, i.e.,
\begin{equation*}
c_{n}^{SN,1S}(\alpha )\equiv c_{n}^{SN}(p,\alpha )=\tfrac{\Phi ^{-1}\left( 1-\alpha /(2k-p)\right) }{\sqrt{1-\left( \Phi ^{-1}\left( 1-\alpha /(2k-p)\right) \right) ^{2}/n}}.
% \label{eq:SN1S-CV}
\end{equation*}
The following result is a corollary of Lemma \ref{lem:SNSize}.

\begin{theorem}[One-step SN approximation] \label{thm:SN1Scorollary}
Assume Assumptions \ref{ass:Basic}-\ref{ass:Rates}, $\alpha \in (0,0.5)$, and that $H_{0}$ holds. Then,
\[
P\del[1]{T_n(\theta) > c_{n}^{SN,1S}(\alpha )} ~\leq~ \alpha  + \alpha Kn^{-\delta /2}M_{n,2+\delta }^{2+\delta }
\Big(1+\Phi ^{-1}\left( 1-\alpha /(2k-p)\right) \Big)^{2+\delta }
= \alpha + o(1),
\]
where $K$ is a universal constant and the convergence is uniform in all parameters $\theta \in \Theta$ and distributions $P$ that satisfy the assumptions in the statement.
\end{theorem}

By definition, this SN approximation considers all moment inequalities in the model as binding. A more powerful test can be constructed by using the data to reveal which moment inequalities are slack.  In particular, \citetalias{chernozhukov/chetverikov/kato:2014c} propose a two-step SN procedure which combines a first step moment inequality based on SN methods and the second step SN critical value in Theorem \ref{thm:SN1Scorollary}. If we adapt their procedure to the presence of moment equalities, this would be given by:
\begin{align}
c^{SN,2S}(\theta,\alpha) ~&\equiv~ c_{n}^{SN}(|\hat{J}_{SN}(\theta)|,\alpha - 2 \beta_{n} )
\label{eq:SN2S-CV}
\end{align}
with:
\begin{align*}
\hat{J}_{SN}(\theta) ~&\equiv ~ \Big\{j\in \{1,\dots,p\}~:
~\sqrt{n}\hat{\mu}_j(\theta)/\hat{\sigma}_j(\theta)>-2c^{SN,1S}(\beta_n)\Big\},  
% \label{eq:SN2S-Set}
\end{align*}
where $\{\beta_n\}_{n\ge 1}$ is an arbitrary sequence of constants in $(0,\alpha/3)$. By extending arguments in \citetalias{chernozhukov/chetverikov/kato:2014c} to include moment equalities, one can show that inference based on the critical value $c^{SN,2S}(\theta,\alpha)$ in Eq.\ \eqref{eq:SN2S-CV} is asymptotically valid in a uniform sense.

In this paper, we propose an alternative SN procedure by using our Lasso-based first step. In particular, we define the following two-step Lasso SN critical value:
\begin{eqnarray}
c_{n}^{SN,L}(\theta, \alpha )~\equiv~ c_{n}^{SN}(|\hat{J}_{L}(\theta)|,\alpha) ,
\label{eq:cn-SNLasso}
\end{eqnarray}
where $\hat{J}_{L}(\theta)$ is as in Eq.\ \eqref{eq:BindingLasso2}. The following result shows that an inference method based on our two-step Lasso SN critical value is asymptotically valid in a uniform sense.

\begin{theorem}[Two-step Lasso SN approximation] \label{thm:Lasso2SSize}
	Assume Assumptions \ref{ass:Basic}-\ref{ass:Rates2}, $\alpha \in (0,0.5)$, and that $H_{0}$ holds, and let $\lambda_n$ be as in Eq.\ \eqref{eq:LambdaConcrete}. Then,
\begin{align*}
&P\del[1]{T_{n}(\theta)>c_{n}^{SN,L}(\theta,\alpha)}\\
 &\leq \alpha
+\sbr[4]{\begin{array}{c}
\alpha Kn^{-\delta /2}M_{n,2+\delta }^{2+\delta }(1+\Phi ^{-1}\left( 1-\alpha /(2k-p)\right) )^{2+\delta }+ \\
4p\exp\del[2]{-2^{-1}n^{\delta /(2+\delta )}M_{n,2+\delta }^{-2}}\sbr[2]{1+K\left( n^{-\delta /(2(2+\delta ))} M_{n,2+\delta }+1\right) ^{2+\delta }} +2\tilde{K}n^{-c}
\end{array}}\\
&=\alpha +o(1),
\end{align*}
where $K,\tilde{K}$ are universal constants and the convergence is uniform in all parameters $\theta \in \Theta$ and distributions $P$ that satisfy the assumptions in the statement.
\end{theorem}

We now compare our two-step SN Lasso method with the SN methods in \citetalias{chernozhukov/chetverikov/kato:2014c}. Since all inference methods share the test statistic, the only difference lies in the critical values. While the one-step SN critical values considers all $p$ moment inequalities as binding, our two-step SN Lasso critical value considers only $|\hat{J}_{L}(\theta)|$ moment inequalities as binding. Since $|\hat{J}_{L}(\theta)|\leq p$ and $c_{n}^{SN}(\alpha ,|J|)$ is weakly increasing in $|J|$ (see Lemma \ref{lem:CSNincreasing} in the appendix), then our two-step SN method results in a weakly larger rejection probability for all sample sizes. In contrast, the comparison between $c_{n}^{SN,L}(\theta,\alpha)$ and $c_{n}^{SN,2S}(\theta,\alpha)$ is not straightforward as these differ in two aspects. First, the set of binding constrains $\hat{J}_{SN}(\theta)$ according to SN differs from the set of binding constrains $\hat{J}_{L}(\theta)$ according to the Lasso. Second, the quantile of the critical values are different: the two-step SN method in Eq.\ \eqref{eq:SN2S-CV} considers the $\alpha - 2\beta_{n}$ quantile while the Lasso-based method considers the usual $\alpha$ quantile. As a result of these differences, the comparison of these critical values is ambiguous and so is the resulting power comparison. This topic will be discussed in further detail in Section \ref{sec:Power}.


\subsection{Bootstrap methods}\label{sec:Boot}

\citetalias{chernozhukov/chetverikov/kato:2014c} also propose two bootstrap approximation methods: multiplier bootstrap (MB) and empirical bootstrap (EB). Relative to the SN approximation, bootstrap methods have the advantage of taking into account the dependence between the coordinates of $\{\sqrt{n}\hat{\mu}_{j}({\theta})/\hat{\sigma}_{j}(\theta)\}_{j=1}^{p}$ involved in the definition of the test statistic $T_{n}(\theta)$.

As in the previous subsection, we first define the bootstrap approximation to the $(1-\alpha)$-quantile of $T_n(\theta)$ in a hypothetical moment (in)equality model composed of moment inequalities indexed by the set $J$ and the $k-p$ moment equalities. The corresponding MB and EB approximations are denoted by $c_{n}^{MB}(\theta, J, \alpha)$ and $c_{n}^{EB}(\theta, J, \alpha)$, respectively, and are computed as follows.

\begin{algorithm}\label{alg:MB}\textbf{Multiplier bootstrap (MB) }
\begin{enumerate}
\item Generate i.i.d.\ standard normal random variables $\{\epsilon _{i}\}_{i = 1 }^{n}$, and independent of the data $X^{n}(\theta)$.
\item Construct the multiplier bootstrap test statistic:
\begin{equation*}
W_{n}^{MB}(\theta, J)=\max \cbr[4]{\max_{j\in J}\frac{\frac{1}{\sqrt{n}}\sum_{i=1}^{n} \epsilon _{i}( X_{ij}(\theta)-\hat{\mu}_{j}(\theta)) }{\hat{\sigma}_{j}(\theta)} ,\max_{s=p+1,\dots ,k}\frac{\frac{1}{\sqrt{n}}|\sum_{i=1}^{n} \epsilon _{i}( X_{is}(\theta)-\hat{\mu}_{s}(\theta)) |}{\hat{\sigma}_{s}(\theta)}}.
\end{equation*}
\item Calculate $c_{n}^{MB}(\theta, J, \alpha)$ as the conditional $(1-\alpha )$-quantile of $W_{n}^{MB}(\theta, J)$ (given $X^{n}(\theta)$).
\end{enumerate}
\end{algorithm}

\begin{algorithm}\label{alg:EB}\textbf{Empirical bootstrap (EB)}
\begin{enumerate}
\item Generate a bootstrap sample $\{ X ^{*}_{i}(\theta)\}_{i = 1 }^{n}$ from the data, i.e., an i.i.d.\ draw from the empirical distribution of $X^{n}(\theta)$.
\item Construct the empirical bootstrap test statistic:
\begin{equation*}
W_{n}^{EB}(\theta, J)=\max \cbr[4]{ \max_{j\in J}\frac{\frac{1}{\sqrt{n}}\sum_{i=1}^{n} ( X^{*}_{ij}(\theta)-\hat{\mu}_{j}(\theta)) }{\hat{\sigma}_{j}(\theta)} ,\max_{s=p+1,\dots ,k}\frac{\frac{1}{\sqrt{n}}|\sum_{i=1}^{n} ( X^{*}_{is}(\theta)-\hat{\mu}_{s}(\theta)) |}{\hat{\sigma}_{s}(\theta)}}.
\end{equation*}
\item Calculate $c_{n}^{EB}(\theta, J, \alpha)$ as the conditional $(1-\alpha )$-quantile of $W_{n}^{EB}(\theta, J)$ (given $X^{n}(\theta)$).
\end{enumerate}
\end{algorithm}

All the results in the remainder of the section will apply to both versions of the bootstrap, and under the same assumptions. For this reason, we can use $c_{n}^{B}(\theta, J, \alpha)$ to denote the bootstrap critical value where $B \in \{MB,EB\}$ represents either MB or EB. Lemma \ref{lem:BootSize} in the appendix shows that $c_{n}^{B}(\theta, J, \alpha)$ for $B \in \{MB,EB\}$ provides asymptotic uniform size control in a hypothetical moment (in)equality model composed of moment inequalities indexed by the set $J$ and the $k-p$ moment equalities under Assumptions \ref{ass:Basic} and \ref{ass:Rates3}. As in Section \ref{sec:SN}, the main difference between this result and \citetalias{chernozhukov/chetverikov/kato:2014c} (Theorem 4.3) is that we allow for the presence of the moment equalities. Since our moment (in)equality model has $|J|=p$ moment inequalities and $k-p$ moment equalities, we can define the regular (i.e.\ one-step) MB or EB approximation method by using $|J|=p$ in Algorithm \ref{alg:MB} or \ref{alg:EB}, respectively, i.e.,
\begin{equation*}
	c_{n}^{B,1S}(\theta, \alpha )\equiv c_{n}^{B}(\theta ,\{1,\dots,p\},\alpha),
	% \label{eq:B1S-CV}
\end{equation*} 
where $c_{n}^{B}(\theta,J,\alpha )$ is as in Algorithm \ref{alg:MB} if $B=MB$ or Algorithm \ref{alg:EB} if $B=EB$. The following result is a corollary of Lemma \ref{lem:BootSize}.

\begin{theorem}[One-step bootstrap approximation]\label{thm:B1Scorollary}
	Assume Assumptions \ref{ass:Basic}, \ref{ass:Rates3}, $\alpha \in (0,0.5)$, and that $H_{0}$ holds.  Then,
\begin{equation*}
P\del[1]{T_{n}(\theta)>c_{n}^{B,1S}(\theta, \alpha ) }~\le~ \alpha +\tilde{C}n^{-\tilde{c}},
\end{equation*}
where $\tilde{c},\tilde{C}>0$ are positive constants that only depend on the constants $c,C$ in Assumption \ref{ass:Rates3}. 
Furthermore, if $\mu _{j}(\theta)=0$ for all $j=1,\dots ,p$, then
\begin{equation*}
	\vert P\del[1]{ T_{n}(\theta)>c_{n}^{B,1S}(\theta, \alpha )} -\alpha  \vert \leq \tilde{C}n^{-\tilde{c}}.
\end{equation*}
Finally, the proposed bounds are uniform in all parameters $\theta \in \Theta$ and distributions $P$ that satisfy the assumptions in the statement.
\end{theorem}

As in the SN approximation method, the regular (one-step) bootstrap approximation considers all moment inequalities in the model as binding. A more powerful bootstrap-based test can be constructed using the data to reveal which moment inequalities are slack. However, unlike in the SN approximation method, Theorem \ref{thm:B1Scorollary} shows that the size of the test using the bootstrap critical values converges to $\alpha$ when all the moment inequalities are binding. This difference comes from the fact that the bootstrap can better approximate the correlation structure in the moment inequalities, which is not taken into account by the SN approximation. As we will see in simulations, this translates into power gains in favor of the bootstrap.

\citetalias{chernozhukov/chetverikov/kato:2014c} propose a two-step bootstrap procedure, combining a first step moment inequality based on the bootstrap with the second step bootstrap critical value in Theorem \ref{thm:B1Scorollary}.\footnote{They also consider the so-called ``hybrid'' procedures in which the first step can be based on one approximation method (e.g.\ SN approximation) and the second step could be based on another approximation method (e.g.\ bootstrap). While these are not explicitly addressed in this section they are included in the Monte Carlo section.} If we adapt their procedure to the presence of moment equalities, this would be given by:
\begin{align}
c^{B,2S}(\theta,\alpha) ~\equiv ~c_{n}^{B}(\theta, \hat{J}_{B}(\theta), \alpha - 2 \beta_{n})
\label{eq:B2S-CV}
\end{align}
with:
\begin{align*}
\hat{J}_{B}(\theta) ~\equiv ~ \{j\in \{1,\dots,p\}~:~\sqrt{n}\hat{\mu}_j(\theta)/\hat{\sigma}_j(\theta)
>-2c^{B,1S}(\alpha,\beta_n)\},
% \label{eq:B2S-Set}
\end{align*}
where $\{\beta_n\}_{n\ge 1}$ is an arbitrary sequence of constants in $(0,\alpha/2)$. Again, by extending arguments in \citetalias{chernozhukov/chetverikov/kato:2014c} to the presence of moment equalities, one can show that an inference method based on the critical value $c^{B,2S}(\theta,\alpha)$ in Eq.\ \eqref{eq:B2S-CV} is asymptotically valid in a uniform sense.

This paper proposes an alternative bootstrap procedure by using our Lasso-based first step. For $B \in \{MB,EB\}$, define the following two-step Lasso bootstrap critical value:
\begin{equation}
c_{n}^{B,L}(\theta, \alpha )~\equiv~ c_{n}^{B}(\theta, \hat{J}_{L}(\theta), \alpha) ,
\label{eq:cn-BLasso}
\end{equation}
where $\hat{J}_{L}(\theta)$ is as in Eq.\ \eqref{eq:BindingLasso2}, and $c_{n}^{B}(\theta,J,\alpha )$ is as in Algorithm \ref{alg:MB} if $B=MB$ or Algorithm \ref{alg:EB} if $B=EB$. The following result shows that an inference method based on our two-step Lasso bootstrap critical value is asymptotically valid in a uniform sense.

\begin{theorem}[Two-step Lasso bootstrap approximation]\label{lem:Lasso2SBootstrap}
Assume  Assumptions \ref{ass:Basic}, \ref{ass:Rates}, \ref{ass:Rates2}, \ref{ass:Rates3}, $\alpha \in (0,0.5)$, and that $H_{0}$ holds, and let $\lambda_n$ be as in Eq.\ \eqref{eq:LambdaConcrete}. Then, for $B\in \{MB,EB\}$, 
\begin{eqnarray*}
	&&P \del[1]{ T_n(\theta) > c_n^{B,L} (\theta , \alpha)} \\
	&&\leq \alpha +  \tilde{C} n^{-\tilde{c}} + C n^{-c} + 2\tilde{K} n^{-c}+ 4p \exp\del[2]{2^{-1} n^{\delta/(2+\delta)}/ M_{n,2+\delta}^2} \sbr[2]{ 1+ K (M_{n,2+\delta}/n^{\delta/(2(2+\delta)} + 1 )^{2+ \delta}} \\
	&& = \alpha + o(1),
\end{eqnarray*}
where $\tilde{c},\tilde{C}>0$ are positive constants that only depend on the constants $c,C$ in Assumption \ref{ass:Rates3}, $K,\tilde{K}$ are universal constants, and the convergence is uniform in all parameters $\theta \in \Theta$ and distributions $P$ that satisfy the assumptions in the statement. 
Furthermore, if $\mu_j(\theta) =0$ for all $1 \le j \le p$ and
\begin{equation}
	\tilde{K} n^{-c}+ 2p \exp\del[2]{2^{-1} n^{\delta/(2+\delta)}/ M_{n,2+\delta}^2} \sbr[2]{ 1+ K (M_{n,2+\delta}/n^{\delta/(2(2+\delta)} + 1 )^{2+ \delta}}  \leq \tilde{C} n^{-\tilde{c}},\label{eq:RestrictionOnParams}
\end{equation}
then,
\begin{equation*}
|P \del[1]{T_n(\theta) > c_n^{B,L} (\theta , \alpha)} - \alpha |\leq 3 \tilde{C} n^{- \tilde{c}} + C n^{-c} = o(1),
\end{equation*}
where all constants are as defined earlier and the convergence is uniform in all parameters $\theta \in \Theta$ and distributions $P$ that satisfy the assumptions in the statement.
\end{theorem}

By repeating arguments at the end of Section \ref{sec:SN}, it follows that our two-step bootstrap method results in a larger rejection probability than the one-step bootstrap method for all sample sizes.\footnote{To establish this result, we now use Lemma \ref{lem:CvBootIncreasing} instead of Lemma \ref{lem:CSNincreasing}.} Also, the comparison between $c_{n}^{B,L}(\theta,\alpha)$ and $c_{n}^{B,2S}(\theta,\alpha)$ is not straightforward as these differ in the same two aspects described Section \ref{sec:SN}. This comparison will be the topic of the next section.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Power comparison}\label{sec:Power}

\citetalias{chernozhukov/chetverikov/kato:2014c} show that all of their inference methods satisfy uniform asymptotic size control under appropriate assumptions. Theorems \ref{thm:Lasso2SSize} and \ref{lem:Lasso2SBootstrap} show that our Lasso-based two-step inference methods also satisfy uniform asymptotic size control under similar assumptions. Given these results, the natural next step is to compare these inference methods in terms of criteria related to power.

One possible such criterion is minimax optimality, i.e., the ability that a test has of rejecting departures from the null hypothesis at the fastest possible rate (without loosing uniform size control). \citetalias{chernozhukov/chetverikov/kato:2014c} show that all their proposed inference methods are asymptotically optimal in a minimax sense, even in the absence of any inequality selection (i.e.\ defined as in Theorems \ref{thm:SN1Scorollary} and \ref{thm:B1Scorollary} in the presence of moment equalities). Since our Lasso-based inequality selection can only reduce the number of binding moment inequalities (thus increasing rejection), we can also conclude that all of our two-step Lasso-based inference methods (SN, MB, and EB) are also asymptotically optimal in a minimax sense. In other words, minimax optimality is a desirable property that is satisfied by all tests under consideration and, thus, cannot be used as a criterion to distinguish between them.

Thus, we proceed to compare our Lasso-based inference procedures with those proposed by \citetalias{chernozhukov/chetverikov/kato:2014c} in terms of rejection rates. Since all inference methods share the test statistic $T_{n}(\theta)$, the power comparison depends exclusively on the critical values.

\subsection{Comparison with one-step methods}

As pointed out in previous sections, our Lasso-based two-step inference methods will always be more powerful than the corresponding one-step analogue, i.e., 
\begin{eqnarray*}
	P \del[1]{T_n(\theta) > c_{n}^{SN,L}(\theta,\alpha)} &\geq& P \del[1]{ T_n(\theta) > c_{n}^{SN,1S}(\alpha)} \\
	P \del[1]{ T_n(\theta) > c_{n}^{B,L}(\theta,\alpha)} &\geq& P \del[1]{T_n(\theta) > c_{n}^{B,1S}(\theta,\alpha)}~~\forall B \in \{MB,EB\},
\end{eqnarray*}
for all $\theta \in \Theta$ and $n \in \mathbb{N}$. 
This is a direct consequence of the fact that one-step critical values are based on considering all moment inequalities as binding, while the Lasso-based first-step will restrict attention to the subset of them that are sufficiently close to binding, i.e., $\hat{J}_{L}(\theta) \subseteq \{1,\dots,p\}$.

\subsection{Comparison with two-step methods}

The comparison between our two-step Lasso procedures and the two-step methods in \citetalias{chernozhukov/chetverikov/kato:2014c} is not straightforward for two reasons. First, the set of binding inequalities according to the Lasso might be different from the other methods. Second, our Lasso-based methods considers the usual $\alpha$ quantile while the other two-step methods consider the $\alpha - 2\beta_{n}$ quantile for a sequence of positive constants $\{ \beta_{n} \}_{ n\geq 1}$.

To simplify the discussion, we focus exclusively on the case where the moment (in)equality model is only composed of inequalities, i.e., $k=p$, which is precisely the setup in \citetalias{chernozhukov/chetverikov/kato:2014c}. This is done for simplicity of exposition, the introduction of moment equalities would not qualitatively change the conclusions that follow.

We begin by comparing the two-step SN method with the two-step Lasso SN method. For all $\theta \in \Theta$ and $n \in \mathbb{N}$, our two-step Lasso SN method will have more power than the two-step SN method if and only if $ c_{n}^{SN,L}(\theta,\alpha)~\leq~ c_{n}^{SN,2S}(\alpha)$. By inspecting the formulas in \citetalias{chernozhukov/chetverikov/kato:2014c}, this occurs if and only if:
\begin{equation}
	|\hat{J}_L(\theta)|~\leq~ \frac{\alpha}{\alpha-2\beta_n}|\hat{J}_{SN}(\theta)|,
	 \label{eq:PowerAdvSN2}
\end{equation}
where, by definition, $\{ \beta_{n} \}_{ n\geq 1}$ satisfies $\beta_n\leq \alpha/3$. We provide sufficient conditions for Eq.\ \eqref{eq:PowerAdvSN2} in the following result.

\begin{theorem}\label{thm:powercompSN} 
	For all $\theta \in \Theta$ and $n \in \mathbb{N}$,
	\begin{equation}
		\hat{J}_L(\theta)~~\subseteq~~ \hat{J}_{SN}(\theta) 
		\label{eq:JComparisonSN}
	\end{equation}
	implies
	\begin{equation}
		P \del[1]{ T_n(\theta) > c_{n}^{SN,L}(\theta,\alpha)} ~~\geq~~ P \del[1]{T_n(\theta) > c_{n}^{SN,2S}(\alpha)}.
		\label{eq:PowerComparisonSN}
	\end{equation}
	In turn, Eq.\ \eqref{eq:JComparisonSN} occurs under any of the following circumstances:	
		\begin{align}
		&\frac{4}{3}c_{n}^{SN}( \beta _{n})\geq\sqrt{n}\lambda _{n}, ~~~ \mbox{or,}
                \label{eq:highlevel}
		\\
\beta_n\leq 0.1,~~ M_{n,2+\delta}^2n^{2/(2+\delta)}\geq 2, ~~\mbox{and}~~ &
\ln\del[2]{\frac{p}{2\beta_n\sqrt{2\pi}}}
\geq
\frac{9}{8}(\frac{4}{3}+\varepsilon)^2n^{\delta/(2+\delta)}M_{n,2+\delta}^{-2},\label{eq:lowlevel}
\end{align}
where $\varepsilon >0$ is as in Eq.\ \eqref{eq:LambdaConcrete}.
\end{theorem}

Theorem \ref{thm:powercompSN} provides two sufficient conditions under which our two-step Lasso SN method will have greater or equal power than the two-step SN method in \citetalias{chernozhukov/chetverikov/kato:2014c}. The power difference is a direct consequence of Eq.\ \eqref{eq:JComparisonSN}, i.e., our Lasso-based first step inequality selection procedure chooses a subset of the inequalities in the SN-based first step. The first sufficient condition, Eq.\ \eqref{eq:highlevel}, is sharper than the second one, Eq.\ \eqref{eq:lowlevel}, but the second one is of lower level and, thus, easier to interpret and understand. Eq.\ \eqref{eq:lowlevel} is composed of three statements and only the third one could be considered restrictive. The first one, $\beta \leq 10\%$, is non-restrictive as \citetalias{chernozhukov/chetverikov/kato:2014c} require that $\beta_n\leq \alpha/3$ and the significance level $\alpha$ is typically less than $30\%$. The second, $M_{n,2+\delta}^2n^{2/(2+\delta)}\geq 2$, is also non-restrictive since $M_{n,2+\delta}^2$ is a non-decreasing sequence of positive constants and $n^{2/(2+\delta)}\to\infty$.

	
In principle, Theorem \ref{thm:powercompSN} allows for the possibility of the inequality in Eq.\ \eqref{eq:PowerComparisonSN} being an equality. However, in cases in which the Lasso-based first step selects a strict subset of the moment inequalities chosen by the SN method (i.e.\ the inclusion in Eq.\ \eqref{eq:JComparisonSN} is strict), the inequality in Eq.\ \eqref{eq:PowerComparisonSN} can be strict. In fact, the inequality in Eq.\ \eqref{eq:PowerComparisonSN} can be strict even in cases in which the Lasso-based and SN-based first step agree on the set of binding moment inequalities. The intuition for this is that our Lasso-based method considers the usual $\alpha$-quantile while the other two-step methods consider the $(\alpha - 2\beta_{n})$-quantile for the sequence of positive constants $\{ \beta_{n} \}_{ n\geq 1}$. This slight difference always plays in favor of the Lasso-based first step having more power.\footnote{This is clearly shown in Designs 5-6 of our Monte Carlos. In these cases, both first-step methods to agree on the correct set of binding moment inequalities (i.e.\ $\hat{J}_L(\theta)=\hat{J}_{SN}(\theta)$). Nevertheless, the slight difference in quantiles produces small but positive power advantage in favor of methods that use the Lasso in a first stage.}

% \begin{remark}\label{rem:Comparisons}
% 	Under the sufficient conditions of Theorem \ref{thm:powercompSN} the power advantage results also extend to our two-step bootstrap-Lasso and the hybrid two-step method based on the SN approximation-bootstrap. The reason for this is that the sufficient conditions imply Eq.\ \eqref{eq:JComparisonSN} and this, together with the fact that our Lasso procedure uses the $\alpha$ quantile while the hybrid procedure uses the $\alpha - 2\beta_{n}$ quantile, would both result in a relative power advantage in favor of our two-step bootstrap Lasso.
% \end{remark}

The relevance of Theorem \ref{thm:powercompSN} depends on the generality of the sufficient conditions in Eq.\ \eqref{eq:highlevel} and \eqref{eq:lowlevel}. Figure \ref{fig:SN} provides heat maps that indicate combinations of values of $M_{n,2+\delta}$ and $p$ under which Eqs.\ \eqref{eq:highlevel} and \eqref{eq:lowlevel} are satisfied. The graphs clearly show these conditions are satisfied for a large portion of the parameter space. In fact, the region in which Eq.\ \eqref{eq:highlevel} fails to hold is barely visible. In addition, the graph also confirms that Eq.\ \eqref{eq:highlevel} applies more generally than Eq.\ \eqref{eq:lowlevel}.

\begin{figure}
\begin{center}
\includegraphics[width=8cm, height=8cm]{C41_highlevel.png}
\includegraphics[width=8cm, height=8cm]{C41_closedform.png}
\caption{Consider a moment inequality model with $n=400$, $\beta_n=0.1\%$, $C=2$, $M=M_{n,2+\delta} \in [0,10]$, and $k=p\in \{1,\dots,1000\}$. The left (right) panel shows in red the configurations $(p,M)$ that do not satisfy Eq.\ \eqref{eq:highlevel} (Eq.\ \eqref{eq:lowlevel}, respectively).}
\label{fig:SN}
\end{center}
\end{figure}

\begin{remark}\label{rem:FiniteSN}
	Notice that the power comparison in Theorem \ref{thm:powercompSN} is a finite sample result. In other words, under any of the sufficient conditions Theorem in \ref{thm:powercompSN}, the rejection of the null hypothesis by an inference method with SN-based first step implies the same outcome for the corresponding inference method with Lasso-based first step. Expressed in terms of confidence sets, the confidence set with our Lasso first step will be a subset of the corresponding confidence set with a SN first step.
\end{remark}

To conclude the section, we now compare the power of the two-step bootstrap procedures. 

\begin{theorem}\label{thm:power_compB}
	Assume Assumption \ref{ass:Rates3} and let $B \in \{MB,EB\}$. 
	
	\underline{Part 1:} For all $\theta \in \Theta$ and $n \in \mathbb{N}$, 
	\begin{align}
		% \{\hat{J}_L(\theta) ~\subseteq~ \hat{J}_{B}(\theta)
% 		\} ~~\subseteq ~~\{c_{n}^{B,L}(\theta,\alpha) ~\leq~  c_{n}^{B,2S}(\alpha)\}.
\hat{J}_L(\theta) ~\subseteq~ \hat{J}_{B}(\theta)
		\label{eq:JComparisonB}
	\end{align}
	implies
	\begin{align}
		P (T_n(\theta) > c_{n}^{B,2S}(\alpha)) ~\leq~ P (T_n(\theta) > c_{n}^{B,L}(\theta,\alpha)).
		\label{eq:PowerComparisonBpre}
	\end{align}
	\underline{Part 2:} Eq.\ \eqref{eq:JComparisonB} occurs with probability approaching one, i.e.,
	\begin{equation}
		P\del[1]{\hat{J}_L(\theta)\subseteq \hat{J}_{B}(\theta)} ~\ge~ 1-{C}n^{-{c}} \label{eq:JComparisonBstock}
	\end{equation}
	under the following sufficient conditions: $M_{n,2+\delta}^2n^{2/(2+\delta)}\geq 2$, $\beta_n\geq {C}n^{-{c}}$ for some $C,c>0$, and any one of the following conditions:
		%
\begin{align}
1-\Phi \left( \frac{3}{2^{3/2}}(\frac{4}{3}+\varepsilon )n^{\delta /(2(2+\delta ))}M_{n,2+\delta }^{-1}\right)
         &  \geq 3\beta _{n},  \qquad \mbox{or,} \label{eq:2Ssuff}
\\
\sqrt{(1-\rho(\theta) )\log (p)/2}-\sqrt{2\log (1/[1-3\beta _{n}])}
& \geq \frac{3}{ 2^{3/2}}(\frac{4}{3}+\varepsilon )n^{\delta /(2(2+\delta ))}M_{n,2+\delta }^{-1},
\label{eq:2Ssuff2}
\end{align}
where $\rho(\theta) \equiv \max_{j_1\neq j_2}corr[X_{j_1}(\theta),X_{j_2}(\theta)]$.
% denotes the maximal pairwise correlation between entries of $X(\theta)$.\\
%
\\
	\underline{Part 3:} Under any of the sufficient conditions in part 2,
	\begin{equation}
		P \del[1]{T_n(\theta) > c_{n}^{B,2S}(\alpha)} ~\leq~ P\del[1]{T_n(\theta) > c_{n}^{B,L}(\theta,\alpha)} + {C}n^{-{c}} \label{eq:PowerComparisonB}
	\end{equation}
\end{theorem}

Theorem \ref{thm:power_compB} provides sufficient conditions under which any power advantage of the two-step bootstrap method in \citetalias{chernozhukov/chetverikov/kato:2014c} relative to our two-step bootstrap Lasso vanishes as the sample size diverges to infinity. Specifically, Eq.\ \eqref{eq:PowerComparisonB} indicates that, under any of the sufficient conditions, this power advantage does not exceed $\tilde{C}n^{-\tilde{c}}$. As in the SN approximation, this relative power difference is a direct consequence of Eq.\ \eqref{eq:JComparisonB}, i.e., our Lasso-based first step inequality selection procedure chooses a subset of the inequalities in the bootstrap-based first step. 

The relevance of the result in Theorem \ref{thm:power_compB} depends on the generality of the sufficient condition. This condition has three parts. The first part, i.e., $M_{n,2+\delta}^2n^{2/(2+\delta)}\geq 2$, was already argued to be non-restrictive since $M_{n,2+\delta}^2$ is a non-decreasing sequence of positive constants and $n^{2/(2+\delta)}\to\infty$. The second part, i.e., $\beta_n\geq{C}n^{-{c}}$ is also considered mild as $\{\beta_n\}_{n\ge 1}$ is a sequence of positive constants and $ {C}n^{-{c}}$ converges to zero. The third part is Eq.\ \eqref{eq:2Ssuff} or \eqref{eq:2Ssuff2} and we deem it to be the more restrictive condition of the three. In the case of the latter, this condition can be understood as imposing an upper bound on the maximal pairwise correlation within the moment inequalities of the model. 

%\begin{center}
%	{\bf ADD BOOTSTRAP HEAT MAPS AROUND HERE}
%\end{center}


%\end{document}
%\input{Appendix/MC-paper}
\section{Monte Carlo simulations}\label{sec:MonteCarlos}

We now use Monte Carlo simulations to investigate the finite sample properties of our tests and to compare them to those proposed by \citetalias{chernozhukov/chetverikov/kato:2014c}. Our simulation setup follows closely the moment inequality model considered in their Monte Carlo simulation section. For a hypothetical fixed parameter value $\theta \in \Theta$, we generate data according to the following equation:
\begin{align*}
X_{i}(\theta)~=~\mu(\theta) + A'\epsilon_{i}~~~~~i=1,\dots,n = 400,
\end{align*}
where $\Sigma(\theta)=A'A$, $\epsilon_i=(\epsilon_{i,1},\dots, \epsilon_{i,p})$, and $p \in \{ 200,500,1000\}$. We simulate $\{\epsilon_i\}_{i=1}^{n}$ to be i.i.d.\ with $E[\epsilon_i]= {\bf 0}_{p}$ and $Var[\epsilon_i]=\mathbf{I}_{p\times p}$, and so $\{X_{i}(\theta)\}_{i=1}^{n}$ are i.i.d.\ with $E[X_{i}(\theta)]=\mu(\theta)$ and $Var[X_{i}(\theta)]=\Sigma(\theta)$. This model satisfies the moment (in)equality model in Eq.\ \eqref{eq:MI} if and only if $\mu(\theta)\leq {\bf 0}_{p}$. In this context, we are interested in implementing the hypothesis test in Eqs.\ \eqref{eq:HypTest} (or, equivalently, Eq.\ \eqref{eq:HypTest2}) with a significance level of $\alpha = 5\%$.

We simulate $\epsilon_i =(\epsilon_{i,1},\dots, \epsilon_{i,p})$ to be i.i.d.\ according to two distributions: (i) $\epsilon_{i,j}$ follows a $t$-distribution with four degrees of freedom divided by $\sqrt{2}$, i.e., $ \epsilon_{i,j} \sim t_4/\sqrt{2}$ and (ii) $\epsilon_{i,j} \sim U(-\sqrt{3},\sqrt{3})$. Note that both of these choices satisfy $E[\epsilon_i]= {\bf 0}_{p}$ and $Var[\epsilon_i]=\mathbf{I}_{p\times p}$. Since $(\epsilon_{i,1},\dots, \epsilon_{i,p})$ are i.i.d., the correlation structure across moment inequalities depends entirely on $\Sigma(\theta)$, for which we consider two possibilities: (i) $\Sigma(\theta)_{[j,k]}=1[j=k]+\rho \cdot 1[j\neq k]$ and (ii) a Toeplitz structure, i.e., $\Sigma(\theta)_{[j,k]}=\rho^{|j-k|}$ with $\rho \in \{ 0,0.5,0.9\}$. We repeat all experiments $2,000$ times.

The description of the model is completed by specifying $\mu(\theta)$, given in Table \ref{tab:ParameterChoices}. We consider ten different specifications of $\mu(\theta)$ which, in combination with the rest of the parameters, results in fourteen simulation designs. Our first eight simulation designs correspond exactly to those in \citetalias{chernozhukov/chetverikov/kato:2014c}, half of which satisfy the null hypothesis and half of which do not. We complement these simulations with six designs that do not satisfy the null hypothesis. The additional designs are constructed so that the moment inequalities that agree with the null hypothesis are only slightly or moderately negative.\footnote{For reasons of brevity, these additional designs only consider $\Sigma(\theta)$ with a Toeplitz structure. We carried out the same designs with equicorrelated $\Sigma(\theta)$ and obtained qualitatively similar results. These are available from the authors, upon request.} As the slackness of these inequalities decreases, it becomes harder for two-step inference methods to correctly classify the non-binding moment conditions as such. As a consequence, these new designs will help us understand which two-step inference procedures have better ability in detecting slack moment inequalities.

\begin{table}[h]
	% \begin{center}
	% \hspace{-1cm}
	\scalebox{1}{\begin{tabular}{ccccc}
		\hline\hline
		Design no. & $\{\mu_{j}(\theta):j \in \{1,\dots,p\}\}$ & $\Sigma(\theta)$ & Hypothesis & \citetalias{chernozhukov/chetverikov/kato:2014c} Design no. \\
		\hline
		1 &$-0.8\cdot 1[j>0.1p]$ & Equicorrelated & $H_0$ & 2\\
		2 &$-0.8\cdot 1[j>0.1p]$ & Toeplitz & $H_0$ & 4\\
		3 &$0$ & Equicorrelated & $H_0$ & 1\\
		4 &$0$ & Toeplitz & $H_0$ & 3\\
		5 &$0.05$ & Equicorrelated & $H_1$ & 5\\
		6 &$0.05$ & Toeplitz & $H_1$ & 7\\
		7 &$-0.75\cdot 1[j>0.1p]+0.05\cdot 1[j\leq 0.1p]$ & Equicorrelated & $H_1$ & 6\\
		8 &$-0.75\cdot 1[j>0.1p]+0.05\cdot 1[j\leq 0.1p]$ & Toeplitz & $H_1$ & 8\\
		9 &$-0.6\cdot 1[j>0.1p]+0.05\cdot 1[j\leq 0.1p]$ & Toeplitz & $H_1$ & New\\
		10 &$-0.5\cdot 1[j>0.1p]+0.05\cdot 1[j\leq 0.1p]$ & Toeplitz & $H_1$ & New\\
		11 &$-0.4\cdot 1[j>0.1p]+0.05\cdot 1[j\leq 0.1p]$ & Toeplitz & $H_1$ & New\\
		12 &$-0.3\cdot 1[j>0.1p]+0.05\cdot 1[j\leq 0.1p]$ & Toeplitz & $H_1$ & New\\
		13 &$-0.2\cdot 1[j>0.1p]+0.05\cdot 1[j\leq 0.1p]$ & Toeplitz & $H_1$ & New\\
		14 &$-0.1\cdot 1[j>0.1p]+0.05\cdot 1[j\leq 0.1p]$ & Toeplitz & $H_1$ & New\\
	\hline\hline
	\end{tabular}}
	% \end{center}
	\caption{Parameter choices in our simulations.}
	\label{tab:ParameterChoices}
\end{table}

We implement all the inference methods described in Table \ref{tab:InfMethods}. These include all of the procedures described in previous sections some additional ``hybrid'' methods (i.e.\ MB-H and EB-H). The bootstrap based methods are implemented with $B=1,000$ bootstrap replications. Finally, for our Lasso-based first step, we use:
\begin{align}
{\lambda}_n~=~C\cdot n^{-1/2}\del[1]{\hat{M}_{n,3}^2n^{-1/3}-n^{-1}}^{-1},
\label{eq:LambdaMCs}
\end{align}
with $C\in\{2,4,6\}$ and $\hat{M}_{n,3}\equiv \max_ {j = 1,\dots,p} (n^{-1}\sum\nolimits_{i=1}^n|X_{ij}(\theta)|^3)^{1/3}$. This corresponds to the empirical analogue of Eq.\ \eqref{eq:LambdaConcrete} when $\delta=1$ and $\varepsilon\in \{2/3, 8/3, 14/3\}$.

% \begin{table}
% 	\scalebox{1}{\begin{tabular}{lcccc}
% 		\hline\hline
% 		Method & No.\ of steps & First step & Second step & Parameters\\
% 		\hline
% 		SN Lasso & Two & Lasso & Self-normalization  &   $C \in \{2,4,6\}$ in  Eq.\ \eqref{eq:LambdaMCs}  \\
% 		MB Lasso & Two & Lasso & Multiplier bootstrap  & $C \in \{2,4,6\}$ in Eq.\ \eqref{eq:LambdaMCs}  \\
% 		EB Lasso & Two & Lasso & Empirical bootstrap  & $C \in \{2,4,6\}$ in Eq.\ \eqref{eq:LambdaMCs} \\
% 		SN-1S & One & None & Self-normalization & None \\
% 		SN-2S & Two & Self-normalization & Self-normalization & None \\
% 		MB-1S & One & None & Multiplier bootstrap & None \\
% 		MB-H  & Two & Self-normalization & Multiplier bootstrap  & $\beta_{n} \in \{0.01\%,0.1\%,1\%\}$ \\
% 		MB-2S & Two & Multiplier bootstrap & Multiplier bootstrap & $\beta_{n} \in \{0.01\%,0.1\%,1\%\}$\\
% 		EB-1S & One & None & Empirical bootstrap & None \\
% 		EB-H  & Two & Self-normalization & Empirical bootstrap  & $\beta_{n} \in \{0.01\%,0.1\%,1\%\}$ \\
% 		EB-2S & Two & Empirical bootstrap & Empirical bootstrap & $\beta_{n} \in \{0.01\%,0.1\%,1\%\}$\\
% 		\hline\hline
% 	\end{tabular}}
% 	% \end{center}
% 	\caption{Inference methods.}
% 	\label{tab:InfMethods}
% \end{table}

\begin{table}
	\begin{center}
	\scalebox{1}{
	\begin{tabular}{lcccc}
		\hline\hline 
		Method & No.\ of steps & First step & Second step & Parameters\\
		\hline
		SN Lasso & Two & Lasso & SN  &   $C \in \{2,4,6\}$ in  Eq.\ \eqref{eq:LambdaMCs}  \\
		MB Lasso & Two & Lasso & MB  & $C \in \{2,4,6\}$ in Eq.\ \eqref{eq:LambdaMCs}  \\
		EB Lasso & Two & Lasso & EB  & $C \in \{2,4,6\}$ in Eq.\ \eqref{eq:LambdaMCs} \\
		SN-1S & One & None & SN & None \\
		SN-2S & Two & SN & SN & None \\
		MB-1S & One & None & MB & None \\
		MB-H  & Two & SN & MB  & $\beta_{n} \in \{0.01\%,0.1\%,1\%\}$ \\
		MB-2S & Two & MB & MB & $\beta_{n} \in \{0.01\%,0.1\%,1\%\}$\\
		EB-1S & One & None & EB & None \\
		EB-H  & Two & SN & EB  & $\beta_{n} \in \{0.01\%,0.1\%,1\%\}$ \\
		EB-2S & Two & EB & EB & $\beta_{n} \in \{0.01\%,0.1\%,1\%\}$\\
		\hline\hline 
	\end{tabular}}
	\end{center}
	\caption{Inference methods.}
	\label{tab:InfMethods}
\end{table}


We shall begin by considering the simulation designs in \citetalias{chernozhukov/chetverikov/kato:2014c} as reported in Tables \ref{tab1}-\ref{tab8}. The first four tables are concerned with the finite sample size control. The general finding is that all tests under consideration are very rarely over-sized. The maximal size observed for our procedures is 7.15 (e.g.\ EB Lasso in Designs 3-4, $p=1,000$, $\rho=0$, and uniform errors) while the corresponding number for \citetalias{chernozhukov/chetverikov/kato:2014c} is 7.25 (e.g.\ EB-1S in Designs 3-4, $p=1,000$, $\rho=0$, and uniform errors). Some procedures, such as SN-1S, can be heavily under-sized. Our simulations reveal that in order to achieve empirical rejection rates close to $\alpha = 5\%$ under the null hypothesis, one requires using a two-step inference procedure with a bootstrap-based second step (either MB or EB). 
% If we adopt them separately, neither a two step inference procedure nor a bootstrap-based inference method are a guarantee of avoiding undersized tests. In particular, notice that a two-step procedure with a second step based on self-normalization results in undersized tests when $\rho=0.9$, while a single-step bootstrap procedure results in undersized tests for all values of $\rho$.

Before turning to the individual setups for power comparison, let us remark that a first step based on our Lasso procedure compares favorably with a first step based on SN. For example, SN-Lasso with $C=2$ has more or equal power than SN-2S with $\beta_n=0.1\%$. While the differences may often be small, this finding is in line with the power comparison in Section \ref{sec:Power}.

Tables \ref{tab5}-\ref{tab8} contain the designs used by \citetalias{chernozhukov/chetverikov/kato:2014c} to gauge the power of their tests. Tables \ref{tab5} and \ref{tab6} consider the case where all moment inequalities are violated. Since none of the moment conditions are slack, there is no room for power gains based on a first-step inequality selection procedure. In this sense, it is not surprising that the first step choice makes no difference in these designs. For example, the power of SN-Lasso is identical to the one of SN-1S while the power of SN-2S is also close to the one of SN-1S. However, the SN-2S has lower power than SN-1S for some values of $\beta_{n}$ while the power of SN Lasso appears to be invariant to the choice of $C$. The latter is in accordance with our previous findings. The bootstrap still improves power for high values of $\rho$.

Next, we consider Tables \ref{tab7} and \ref{tab8}. In this setting, $90\%$ of the moment conditions have $\mu_j(\theta)=-0.75$ and our results seem to suggest that this value is relative far away from being binding. We deduce this from the fact that all first-step selection methods agree on the set of binding moment conditions, producing very similar power results. Table \ref{tab15} shows the percentage of moment inequalities retained by each of the first-step procedures in Design 8. When the error terms are $t$-distributed, all first-step procedures retain around $10\%$ of the inequalities which is also the fraction that are truly binding (and, in this case, violated). Thus, all two-step inference procedures are reasonably powerful. When the error terms are uniformly distributed, all first-step procedures have an equal tendency to aggressively remove slack inequalities. However, we have seen from the size comparisons that this does not seem to result in oversized tests. Finally, we notice that the power of our procedures hardly varies with the choice of $C$.

The overall message of the simulation results in Designs 1-8 is that our Lasso-based procedures are comparable in terms of size and power to the ones proposed by \citetalias{chernozhukov/chetverikov/kato:2014c}.

Tables \ref{tab9}-\ref{tab14} present simulations results for Designs 9-14. These correspond to modifications of the setup in Design 8 in which progressively decrease the degree of slackness of the non-binding moment inequalities from $-0.75$ to values between $-0.6$ and $-0.1$.

Tables \ref{tab9}-\ref{tab10} shows results for Designs 9 and 10. As in the case of Design 8, the degree of slackness of the non-binding moment inequalities is still large enough so that it can be correctly detected by all first first-step selection methods.  

As Table \ref{tab11} shows, this pattern changes in Design 11. In this case, the MB Lasso with $C=2$ has a rejection rate that is at least 20 percentage points higher than the most powerful procedure in \citetalias{chernozhukov/chetverikov/kato:2014c}. For example, with $t$-distributed errors, $p=1,000$, and $\rho=0$, our MB Lasso with $C=2$ has a rejection rate of 71.40\% whereas the MB-2S with  $\beta_n = 0.01\%$ has a rejection rate of 20.55\%. Table \ref{tab16} holds the key to these power differences. Ideally, a powerful procedure should retain only the $10\%$ of the moment inequalities that are binding (in this case, violated). The Lasso-based selection indeed often retains close to $10\%$ of the inequalities for $C \in \{2,4\}$. On the other hand, SN-based selection can sometimes retain more than $90\%$ of the inequalities (e.g.\ see $t$-distributed errors, $p=1,000$, and $\rho=0$).

The power advantage in favor of the Lasso-based first step is also present in Design 12 as shown in Table \ref{tab12}. In this case, the MB Lasso with $C=2$ has a rejection rate which is at least $15$ percentage points higher than the most powerful procedure in \citetalias{chernozhukov/chetverikov/kato:2014c}. For $t$-distributed errors, the MB Lasso always has a rejection rate that is at least $20$ percentage points higher than its competitors and sometimes more than $50$ percentage points (e.g.\ $p=1,000$ and $\rho=0$). As in the previous design, this power gain mainly comes from the Lasso being better at removing the slack moment conditions.

Table \ref{tab13} shows the results for Design 13. For $t$-distributed errors, the MB Lasso with $C=2$ has a higher rejection rate than the most powerful procedure of \citetalias{chernozhukov/chetverikov/kato:2014c} (which is often MB-1S) by at least $5$ percentage points. Sometimes the difference is larger than $45$ percentage points (e.g.\ see $p=1,000$ and $\rho=0$). For uniformly distributed errors, there seems to be no significant difference between our procedures and the ones in \citetalias{chernozhukov/chetverikov/kato:2014c}; all of them have relatively low power.

Design 14 is our last experiment and it is shown in Table \ref{tab14}. In this case, the degree of slackness of the non-binding moment inequalities is so small that it cannot be detected by any of the first-step selection methods. As a consequence, there are very little differences among the various inference procedures and all of them exhibit relatively low power.

The overall message from Tables \ref{tab9}-\ref{tab14} is that our Lasso-based inference procedures can have higher power than those in \citetalias{chernozhukov/chetverikov/kato:2014c} when the slack moment inequalities are difficult to distinguish from zero. 
% Finally, we note that Designs 7-14 coincide in having $10\%$ of the moment inequalities being violated. We have also conducted experiments when this percentage drops to $5\%$ or $1\%$ and our Lasso-based procedure remains powerful in these settings.\footnote{These are omitted from the paper for reasons of brevity. These are available from the authors, upon request.}

\section{Conclusions}\label{sec:Conclusions}

	This paper considers the problem of inference in a partially identified moment (in)equality model with possibly many moment inequalities. Our contribution is to propose a novel two-step inference method based on the combination of two ideas. On the one hand, our test statistic and critical values are based on those proposed by \citetalias{chernozhukov/chetverikov/kato:2014c}. On the other hand, we propose a new first step selection procedure based on the Lasso. Our two-step inference method can be used to conduct hypothesis tests and to construct confidence sets for the true parameter value.

	Our inference method has very desirable properties. First, under reasonable conditions, it is uniformly valid, both in underlying parameter $\theta $ and distribution of the data. Second, by virtue of results in \citetalias{chernozhukov/chetverikov/kato:2014c}, our test is asymptotically optimal in a minimax sense. Third, the power of our method compares favorably with that of the corresponding two-step method in \citetalias{chernozhukov/chetverikov/kato:2014c}, both in theory and in simulations. On the theory front, we provide sufficient conditions under which the power of our method dominates. These can sometimes represent a significant part of the parameter space. Our simulations indicate that our inference methods are usually as powerful as the corresponding ones in \citetalias{chernozhukov/chetverikov/kato:2014c}, and can sometimes be more powerful. 
	% In particular, our simulations reveal that the Lasso-based first step delivers more power in designs with sparse alternatives, i.e., when only few of the moment (in)equalities are violated.
	Fourth, our Lasso-based first step is straightforward to implement.

{\tiny

\begin{landscape}
\begin{table}
\scalebox{0.6}{
\begin{tabular}{cccccccccccccccccccccccccccccc}
\toprule Density & $p$ & $\rho$ & \multicolumn{9}{c}{Our methods} & \multicolumn{18}{c}{\citetalias{chernozhukov/chetverikov/kato:2014c}'s methods} \\
\cmidrule(lr){4-12}\cmidrule(lr){13-30}
 & & & \multicolumn{3}{c}{SN Lasso} & \multicolumn{3}{c}{MB Lasso} & \multicolumn{3}{c}{EB Lasso} & SN-1S & \multicolumn{3}{c}{SN-2S} & MB-1S & \multicolumn{3}{c}{MB-H} & \multicolumn{3}{c}{MB-2S} & EB-1S & \multicolumn{3}{c}{EB-H} & \multicolumn{3}{c}{EB-2S} \\
\cmidrule(lr){4-6}\cmidrule(lr){7-9}\cmidrule(lr){10-12}\cmidrule(lr){14-16}\cmidrule(lr){18-20}\cmidrule(lr){21-23}\cmidrule(lr){25-27}\cmidrule(lr){28-30}
 & & & C=2 & C=4 & C=6 & C=2 & C=4 & C=6 & C=2 & C=4 & C=6 & & 0.01\% & 0.10\% & 1.00\% & & 0.01\% & 0.10\% & 1.00\% & 0.01\% & 0.10\% & 1.00\% & & 0.01\% & 0.10\% & 1.00\% & 0.01\% & 0.10\% & 1.00\% \\
\midrule
$t_{4}/\sqrt{2}$ & 200 & 0 & 4.10 & 4.00 & 4.00 & 5.20 & 5.00 & 5.00 & 4.55 & 4.50 & 4.50 & 0.40 & 3.95 & 3.85 & 2.30 & 0.55 & 4.80 & 4.50 & 2.50 & 4.85 & 4.50 & 2.50 & 0.50 & 4.35 & 4.40 & 2.65 & 4.75 & 4.55 & 2.70 \\
 & & 0.5 & 3.05 & 3.05 & 3.05 & 5.10 & 5.10 & 5.10 & 4.85 & 4.85 & 4.85 & 0.25 & 3.05 & 2.95 & 1.85 & 0.85 & 5.05 & 4.80 & 2.80 & 5.05 & 4.80 & 2.85 & 0.85 & 4.85 & 4.60 & 3.10 & 4.75 & 4.65 & 2.85 \\
 & & 0.9 & 0.90 & 0.90 & 0.90 & 5.00 & 5.00 & 5.00 & 5.15 & 5.15 & 5.15 & 0.05 & 0.90 & 0.85 & 0.45 & 2.80 & 4.95 & 4.80 & 3.10 & 4.95 & 4.80 & 3.10 & 2.80 & 5.15 & 4.85 & 2.95 & 5.10 & 4.95 & 3.10 \\
 & 500 & 0 & 4.45 & 4.35 & 4.35 & 5.65 & 5.45 & 5.45 & 5.00 & 4.85 & 4.85 & 0.50 & 4.30 & 4.30 & 2.85 & 0.65 & 5.25 & 5.20 & 3.40 & 5.30 & 5.20 & 3.40 & 0.45 & 4.55 & 4.55 & 3.10 & 4.85 & 4.75 & 3.15 \\
 & & 0.5 & 2.75 & 2.75 & 2.75 & 5.70 & 5.70 & 5.70 & 5.60 & 5.60 & 5.60 & 0.35 & 2.70 & 2.70 & 1.60 & 1.30 & 5.65 & 5.55 & 3.50 & 5.65 & 5.55 & 3.50 & 1.20 & 5.45 & 5.30 & 3.25 & 5.60 & 5.35 & 3.35 \\
 & & 0.9 & 0.55 & 0.55 & 0.55 & 5.80 & 5.80 & 5.80 & 5.60 & 5.60 & 5.60 & 0.15 & 0.55 & 0.55 & 0.40 & 2.75 & 5.80 & 5.60 & 3.10 & 5.80 & 5.60 & 3.10 & 2.90 & 5.60 & 5.40 & 3.15 & 5.65 & 5.40 & 3.00 \\
 & 1000 & 0 & 5.05 & 4.85 & 4.80 & 5.85 & 5.80 & 5.80 & 5.55 & 5.35 & 5.35 & 0.45 & 4.65 & 4.65 & 3.00 & 0.55 & 5.55 & 5.45 & 3.60 & 5.60 & 5.40 & 3.60 & 0.45 & 4.85 & 4.90 & 3.40 & 4.95 & 4.80 & 3.15 \\
 & & 0.5 & 2.10 & 2.10 & 2.10 & 4.80 & 4.80 & 4.80 & 4.70 & 4.70 & 4.70 & 0.25 & 2.10 & 2.10 & 1.40 & 1.15 & 4.80 & 4.65 & 2.95 & 4.80 & 4.65 & 2.95 & 1.20 & 4.50 & 4.50 & 2.90 & 4.60 & 4.45 & 2.60 \\
 & & 0.9 & 0.55 & 0.55 & 0.55 & 4.60 & 4.60 & 4.60 & 4.65 & 4.65 & 4.65 & 0.05 & 0.55 & 0.55 & 0.35 & 2.75 & 4.55 & 4.40 & 2.70 & 4.60 & 4.40 & 2.70 & 2.65 & 4.60 & 4.40 & 2.65 & 4.70 & 4.60 & 2.85 \\
\midrule
$U(-\sqrt{3},\sqrt{3})$& 200 & 0 & 5.00 & 5.00 & 5.00 & 5.60 & 5.60 & 5.60 & 5.80 & 5.80 & 5.80 & 0.45 & 4.95 & 4.85 & 3.20 & 0.55 & 5.55 & 5.45 & 3.65 & 5.55 & 5.45 & 3.65 & 0.55 & 5.90 & 5.65 & 3.70 & 5.70 & 5.45 & 3.65 \\
 & & 0.5 & 3.10 & 3.10 & 3.10 & 5.20 & 5.20 & 5.20 & 5.05 & 5.05 & 5.05 & 0.40 & 3.10 & 3.05 & 2.00 & 1.15 & 5.20 & 4.90 & 3.15 & 5.20 & 4.90 & 3.15 & 1.15 & 5.15 & 5.00 & 3.10 & 5.15 & 5.00 & 3.15 \\
 & & 0.9 & 1.10 & 1.10 & 1.05 & 4.85 & 4.85 & 4.85 & 5.05 & 5.05 & 5.05 & 0.25 & 1.10 & 1.10 & 0.65 & 2.80 & 4.85 & 4.70 & 3.00 & 4.85 & 4.70 & 3.00 & 2.75 & 5.05 & 4.95 & 3.00 & 4.85 & 4.75 & 3.05 \\
 & 500 & 0 & 5.05 & 5.05 & 5.05 & 5.70 & 5.70 & 5.70 & 6.05 & 6.05 & 6.05 & 0.80 & 5.05 & 5.05 & 3.10 & 1.10 & 5.65 & 5.45 & 3.70 & 5.65 & 5.45 & 3.70 & 1.05 & 6.10 & 5.90 & 3.85 & 5.70 & 5.50 & 3.85 \\
 & & 0.5 & 2.60 & 2.60 & 2.60 & 5.35 & 5.35 & 5.35 & 5.40 & 5.40 & 5.40 & 0.40 & 2.55 & 2.45 & 1.60 & 1.10 & 5.30 & 5.15 & 3.25 & 5.30 & 5.15 & 3.25 & 1.05 & 5.45 & 5.15 & 3.15 & 5.20 & 5.10 & 3.00 \\
 & & 0.9 & 0.75 & 0.75 & 0.75 & 5.05 & 5.05 & 5.05 & 4.70 & 4.70 & 4.70 & 0.05 & 0.75 & 0.75 & 0.45 & 2.95 & 5.05 & 4.75 & 3.05 & 5.05 & 4.75 & 3.05 & 2.90 & 4.70 & 4.50 & 3.00 & 4.85 & 4.75 & 3.00 \\
 & 1000 & 0 & 4.50 & 4.50 & 4.50 & 5.85 & 5.85 & 5.85 & 5.55 & 5.55 & 5.55 & 0.55 & 4.50 & 4.40 & 3.00 & 0.70 & 5.80 & 5.45 & 3.65 & 5.80 & 5.45 & 3.65 & 0.80 & 5.60 & 5.40 & 3.65 & 5.30 & 5.20 & 3.75 \\
 & & 0.5 & 2.15 & 2.15 & 2.15 & 4.95 & 4.95 & 4.95 & 5.10 & 5.10 & 5.10 & 0.40 & 2.15 & 2.00 & 1.50 & 1.40 & 4.95 & 4.85 & 2.75 & 4.95 & 4.85 & 2.75 & 1.35 & 5.15 & 5.00 & 2.95 & 5.25 & 4.95 & 2.75 \\
 & & 0.9 & 0.45 & 0.45 & 0.45 & 5.20 & 5.20 & 5.20 & 5.20 & 5.20 & 5.20 & 0.00 & 0.45 & 0.40 & 0.30 & 2.75 & 5.20 & 4.80 & 2.70 & 5.20 & 4.80 & 2.70 & 2.80 & 5.20 & 5.05 & 2.75 & 5.15 & 4.95 & 3.05 \\
\bottomrule
\end{tabular}
}
\caption{Simulation results in Design 1: $\mu_{j}(\theta)=-0.8\cdot 1[j>0.1 p]$, $\Sigma(\theta)$ equicorrelated}
\label{tab1}
\end{table}

\begin{table}
\scalebox{0.6}{
\begin{tabular}{cccccccccccccccccccccccccccccc}
\toprule Density & $p$ & $\rho$ & \multicolumn{9}{c}{Our methods} & \multicolumn{18}{c}{\citetalias{chernozhukov/chetverikov/kato:2014c}'s methods} \\
\cmidrule(lr){4-12}\cmidrule(lr){13-30}
 & & & \multicolumn{3}{c}{SN Lasso} & \multicolumn{3}{c}{MB Lasso} & \multicolumn{3}{c}{EB Lasso} & SN-1S & \multicolumn{3}{c}{SN-2S} & MB-1S & \multicolumn{3}{c}{MB-H} & \multicolumn{3}{c}{MB-2S} & EB-1S & \multicolumn{3}{c}{EB-H} & \multicolumn{3}{c}{EB-2S} \\
\cmidrule(lr){4-6}\cmidrule(lr){7-9}\cmidrule(lr){10-12}\cmidrule(lr){14-16}\cmidrule(lr){18-20}\cmidrule(lr){21-23}\cmidrule(lr){25-27}\cmidrule(lr){28-30}
 & & & C=2 & C=4 & C=6 & C=2 & C=4 & C=6 & C=2 & C=4 & C=6 & & 0.01\% & 0.10\% & 1.00\% & & 0.01\% & 0.10\% & 1.00\% & 0.01\% & 0.10\% & 1.00\% & & 0.01\% & 0.10\% & 1.00\% & 0.01\% & 0.10\% & 1.00\% \\
\midrule
$t_{4}/\sqrt{2}$ & 200 & 0 & 4.10 & 4.00 & 4.00 & 5.20 & 5.00 & 5.00 & 4.55 & 4.50 & 4.50 & 0.40 & 3.95 & 3.85 & 2.30 & 0.55 & 4.80 & 4.50 & 2.50 & 4.85 & 4.50 & 2.50 & 0.50 & 4.35 & 4.40 & 2.65 & 4.75 & 4.55 & 2.70 \\
 & & 0.5 & 4.40 & 4.40 & 4.40 & 5.30 & 5.20 & 5.20 & 5.05 & 5.00 & 5.00 & 0.35 & 4.40 & 4.20 & 2.70 & 0.50 & 5.05 & 5.10 & 3.30 & 5.10 & 5.05 & 3.30 & 0.45 & 4.90 & 4.90 & 3.05 & 5.15 & 4.95 & 3.20 \\
 & & 0.9 & 2.30 & 2.30 & 2.30 & 5.20 & 5.20 & 5.20 & 5.00 & 4.95 & 4.95 & 0.35 & 2.30 & 2.30 & 1.55 & 0.70 & 5.10 & 4.85 & 3.35 & 5.15 & 4.85 & 3.35 & 0.65 & 4.85 & 4.80 & 3.35 & 5.30 & 5.20 & 3.10 \\
 & 500 & 0 & 4.45 & 4.35 & 4.35 & 5.65 & 5.45 & 5.45 & 5.00 & 4.85 & 4.85 & 0.50 & 4.30 & 4.30 & 2.85 & 0.65 & 5.25 & 5.20 & 3.40 & 5.30 & 5.20 & 3.40 & 0.45 & 4.55 & 4.55 & 3.10 & 4.85 & 4.75 & 3.15 \\
 & & 0.5 & 4.35 & 4.25 & 4.25 & 5.60 & 5.45 & 5.45 & 5.15 & 5.10 & 5.10 & 0.40 & 4.20 & 4.15 & 2.70 & 0.45 & 5.35 & 5.25 & 3.25 & 5.40 & 5.25 & 3.25 & 0.45 & 4.85 & 4.70 & 3.00 & 4.75 & 4.55 & 3.05 \\
 & & 0.9 & 1.75 & 1.70 & 1.70 & 5.20 & 5.20 & 5.20 & 5.30 & 5.30 & 5.30 & 0.10 & 1.65 & 1.60 & 1.15 & 0.35 & 5.10 & 4.95 & 2.70 & 5.15 & 4.95 & 2.70 & 0.35 & 5.00 & 5.15 & 2.60 & 5.25 & 4.90 & 2.55 \\
 & 1000 & 0 & 5.05 & 4.85 & 4.80 & 5.85 & 5.80 & 5.80 & 5.55 & 5.35 & 5.35 & 0.45 & 4.65 & 4.65 & 3.00 & 0.55 & 5.55 & 5.45 & 3.60 & 5.60 & 5.40 & 3.60 & 0.45 & 4.85 & 4.90 & 3.40 & 4.95 & 4.80 & 3.15 \\
 & & 0.5 & 4.55 & 4.15 & 4.10 & 5.80 & 5.60 & 5.60 & 5.50 & 5.40 & 5.35 & 0.45 & 4.00 & 3.90 & 2.35 & 0.65 & 5.45 & 5.25 & 3.35 & 5.55 & 5.30 & 3.35 & 0.60 & 4.95 & 5.00 & 3.05 & 4.75 & 4.60 & 2.85 \\
 & & 0.9 & 1.90 & 1.85 & 1.85 & 5.25 & 5.20 & 5.20 & 4.95 & 4.95 & 4.95 & 0.20 & 1.85 & 1.70 & 1.15 & 0.35 & 5.15 & 4.90 & 2.90 & 5.15 & 4.90 & 2.90 & 0.25 & 4.85 & 4.75 & 2.85 & 4.95 & 4.85 & 2.90 \\
\midrule
$U(-\sqrt{3},\sqrt{3})$& 200 & 0 & 5.00 & 5.00 & 5.00 & 5.60 & 5.60 & 5.60 & 5.80 & 5.80 & 5.80 & 0.45 & 4.95 & 4.85 & 3.20 & 0.55 & 5.55 & 5.45 & 3.65 & 5.55 & 5.45 & 3.65 & 0.55 & 5.90 & 5.65 & 3.70 & 5.70 & 5.45 & 3.65 \\
 & & 0.5 & 4.65 & 4.65 & 4.65 & 5.55 & 5.55 & 5.55 & 5.45 & 5.45 & 5.45 & 0.55 & 4.65 & 4.50 & 3.00 & 0.65 & 5.50 & 5.30 & 3.80 & 5.50 & 5.30 & 3.80 & 0.80 & 5.45 & 5.20 & 3.85 & 5.95 & 5.85 & 3.40 \\
 & & 0.9 & 2.25 & 2.25 & 2.25 & 5.25 & 5.25 & 5.25 & 5.15 & 5.15 & 5.15 & 0.55 & 2.25 & 2.15 & 1.50 & 0.85 & 5.15 & 4.90 & 3.40 & 5.15 & 4.90 & 3.40 & 0.80 & 5.20 & 5.15 & 3.50 & 5.15 & 5.05 & 3.45 \\
 & 500 & 0 & 5.05 & 5.05 & 5.05 & 5.70 & 5.70 & 5.70 & 6.05 & 6.05 & 6.05 & 0.80 & 5.05 & 5.05 & 3.10 & 1.10 & 5.65 & 5.45 & 3.70 & 5.65 & 5.45 & 3.70 & 1.05 & 6.10 & 5.90 & 3.85 & 5.70 & 5.50 & 3.85 \\
 & & 0.5 & 4.55 & 4.55 & 4.55 & 5.70 & 5.70 & 5.70 & 5.30 & 5.30 & 5.30 & 0.55 & 4.50 & 4.35 & 2.80 & 0.65 & 5.65 & 5.50 & 3.50 & 5.65 & 5.50 & 3.50 & 0.60 & 5.30 & 5.25 & 3.35 & 5.95 & 5.75 & 3.15 \\
 & & 0.9 & 1.65 & 1.65 & 1.65 & 5.20 & 5.20 & 5.20 & 5.00 & 5.00 & 5.00 & 0.15 & 1.65 & 1.65 & 1.00 & 0.35 & 5.15 & 4.90 & 2.60 & 5.15 & 4.90 & 2.60 & 0.35 & 5.00 & 5.00 & 3.00 & 4.80 & 4.65 & 2.90 \\
 & 1000 & 0 & 4.50 & 4.50 & 4.50 & 5.85 & 5.85 & 5.85 & 5.55 & 5.55 & 5.55 & 0.55 & 4.50 & 4.40 & 3.00 & 0.70 & 5.80 & 5.45 & 3.65 & 5.80 & 5.45 & 3.65 & 0.80 & 5.60 & 5.40 & 3.65 & 5.30 & 5.20 & 3.75 \\
 & & 0.5 & 4.70 & 4.70 & 4.70 & 6.15 & 6.15 & 6.15 & 5.90 & 5.90 & 5.90 & 0.80 & 4.70 & 4.50 & 2.60 & 0.90 & 6.10 & 5.90 & 3.35 & 6.10 & 5.90 & 3.35 & 0.95 & 5.90 & 5.70 & 3.60 & 6.05 & 5.65 & 3.50 \\
 & & 0.9 & 2.40 & 2.40 & 2.40 & 5.55 & 5.55 & 5.55 & 5.30 & 5.30 & 5.30 & 0.20 & 2.40 & 2.40 & 1.60 & 0.50 & 5.50 & 5.35 & 3.50 & 5.50 & 5.35 & 3.50 & 0.40 & 5.30 & 5.25 & 3.50 & 5.35 & 5.10 & 3.55 \\
\bottomrule
\end{tabular}
}
\caption{Simulation results in Design 2: $\mu_{j}(\theta)=-0.8\cdot 1[j>0.1 p]$, $\Sigma(\theta)$ Toeplitz}
\label{tab2}
\end{table}

\begin{table}
\scalebox{0.6}{
\begin{tabular}{cccccccccccccccccccccccccccccc}
\toprule Density & $p$ & $\rho$ & \multicolumn{9}{c}{Our methods} & \multicolumn{18}{c}{\citetalias{chernozhukov/chetverikov/kato:2014c}'s methods} \\
\cmidrule(lr){4-12}\cmidrule(lr){13-30}
 & & & \multicolumn{3}{c}{SN Lasso} & \multicolumn{3}{c}{MB Lasso} & \multicolumn{3}{c}{EB Lasso} & SN-1S & \multicolumn{3}{c}{SN-2S} & MB-1S & \multicolumn{3}{c}{MB-H} & \multicolumn{3}{c}{MB-2S} & EB-1S & \multicolumn{3}{c}{EB-H} & \multicolumn{3}{c}{EB-2S} \\
\cmidrule(lr){4-6}\cmidrule(lr){7-9}\cmidrule(lr){10-12}\cmidrule(lr){14-16}\cmidrule(lr){18-20}\cmidrule(lr){21-23}\cmidrule(lr){25-27}\cmidrule(lr){28-30}
 & & & C=2 & C=4 & C=6 & C=2 & C=4 & C=6 & C=2 & C=4 & C=6 & & 0.01\% & 0.10\% & 1.00\% & & 0.01\% & 0.10\% & 1.00\% & 0.01\% & 0.10\% & 1.00\% & & 0.01\% & 0.10\% & 1.00\% & 0.01\% & 0.10\% & 1.00\% \\
\midrule
$t_{4}/\sqrt{2}$ & 200 & 0 & 3.20 & 3.05 & 3.05 & 4.35 & 4.30 & 4.30 & 3.65 & 3.55 & 3.55 & 3.05 & 3.05 & 2.95 & 1.55 & 4.30 & 4.25 & 4.10 & 2.15 & 4.25 & 4.10 & 2.15 & 3.60 & 3.60 & 3.50 & 2.00 & 3.45 & 3.35 & 1.95 \\
 & & 0.5 & 1.45 & 1.45 & 1.45 & 4.60 & 4.60 & 4.60 & 4.15 & 4.15 & 4.15 & 1.45 & 1.45 & 1.40 & 1.00 & 4.60 & 4.60 & 4.40 & 2.60 & 4.60 & 4.40 & 2.60 & 4.20 & 4.20 & 4.05 & 2.55 & 4.35 & 4.15 & 2.45 \\
 & & 0.9 & 0.25 & 0.25 & 0.25 & 5.20 & 5.20 & 5.20 & 5.30 & 5.30 & 5.30 & 0.25 & 0.25 & 0.20 & 0.15 & 5.20 & 5.15 & 4.90 & 3.10 & 5.15 & 4.90 & 3.10 & 5.30 & 5.30 & 4.80 & 3.00 & 5.15 & 4.95 & 2.95 \\
 & 500 & 0 & 3.00 & 2.95 & 2.95 & 4.45 & 4.25 & 4.25 & 3.50 & 3.50 & 3.50 & 2.95 & 2.95 & 2.90 & 1.55 & 4.25 & 4.20 & 3.95 & 2.20 & 4.20 & 3.95 & 2.20 & 3.55 & 3.55 & 3.50 & 1.90 & 3.70 & 3.50 & 2.20 \\
 & & 0.5 & 1.45 & 1.45 & 1.45 & 5.65 & 5.65 & 5.65 & 5.05 & 5.05 & 5.05 & 1.45 & 1.45 & 1.35 & 0.85 & 5.65 & 5.65 & 5.65 & 3.45 & 5.65 & 5.65 & 3.45 & 5.10 & 5.10 & 5.00 & 3.15 & 5.10 & 5.00 & 3.15 \\
 & & 0.9 & 0.20 & 0.20 & 0.20 & 5.45 & 5.45 & 5.45 & 5.55 & 5.55 & 5.55 & 0.20 & 0.20 & 0.20 & 0.15 & 5.45 & 5.45 & 5.25 & 3.50 & 5.45 & 5.25 & 3.50 & 5.55 & 5.55 & 5.35 & 2.95 & 5.35 & 5.25 & 3.00 \\
 & 1000 & 0 & 3.40 & 3.20 & 3.20 & 4.65 & 4.40 & 4.40 & 3.75 & 3.70 & 3.70 & 3.20 & 3.15 & 3.05 & 1.80 & 4.40 & 4.40 & 4.25 & 2.75 & 4.40 & 4.25 & 2.75 & 3.70 & 3.70 & 3.60 & 1.95 & 3.90 & 3.85 & 2.25 \\
 & & 0.5 & 1.40 & 1.40 & 1.40 & 4.25 & 4.25 & 4.25 & 3.85 & 3.85 & 3.85 & 1.40 & 1.40 & 1.35 & 0.90 & 4.25 & 4.25 & 4.10 & 2.90 & 4.25 & 4.10 & 2.90 & 3.85 & 3.85 & 3.75 & 2.55 & 4.20 & 4.05 & 2.75 \\
 & & 0.9 & 0.15 & 0.15 & 0.15 & 4.40 & 4.40 & 4.40 & 4.15 & 4.15 & 4.15 & 0.15 & 0.15 & 0.15 & 0.05 & 4.40 & 4.40 & 4.15 & 2.50 & 4.40 & 4.15 & 2.50 & 4.15 & 4.15 & 4.00 & 2.50 & 4.35 & 4.10 & 2.60 \\
\midrule
$U(-\sqrt{3},\sqrt{3})$& 200 & 0 & 3.85 & 3.85 & 3.85 & 5.20 & 5.20 & 5.20 & 4.90 & 4.90 & 4.90 & 3.85 & 3.85 & 3.75 & 2.00 & 5.20 & 5.15 & 4.90 & 2.85 & 5.15 & 4.90 & 2.85 & 4.95 & 4.90 & 4.70 & 2.95 & 5.30 & 5.10 & 2.80 \\
 & & 0.5 & 1.95 & 1.95 & 1.95 & 4.65 & 4.65 & 4.65 & 4.55 & 4.55 & 4.55 & 1.95 & 1.95 & 1.85 & 1.45 & 4.65 & 4.65 & 4.65 & 2.80 & 4.65 & 4.65 & 2.80 & 4.55 & 4.55 & 4.45 & 2.75 & 4.70 & 4.45 & 2.75 \\
 & & 0.9 & 0.40 & 0.40 & 0.40 & 4.95 & 4.95 & 4.95 & 5.20 & 5.20 & 5.20 & 0.40 & 0.40 & 0.40 & 0.30 & 4.95 & 4.95 & 4.85 & 2.85 & 4.95 & 4.85 & 2.85 & 5.20 & 5.20 & 5.05 & 2.90 & 4.85 & 4.65 & 2.90 \\
 & 500 & 0 & 4.20 & 4.20 & 4.20 & 5.60 & 5.60 & 5.60 & 5.65 & 5.65 & 5.65 & 4.20 & 4.20 & 4.00 & 2.50 & 5.60 & 5.45 & 5.20 & 3.35 & 5.45 & 5.20 & 3.35 & 5.75 & 5.70 & 5.55 & 3.60 & 5.75 & 5.55 & 3.35 \\
 & & 0.5 & 1.50 & 1.50 & 1.50 & 5.05 & 5.05 & 5.05 & 4.70 & 4.70 & 4.70 & 1.50 & 1.50 & 1.50 & 1.10 & 5.05 & 5.05 & 4.95 & 2.80 & 5.05 & 4.95 & 2.80 & 4.75 & 4.70 & 4.65 & 2.45 & 5.00 & 4.75 & 2.65 \\
 & & 0.9 & 0.10 & 0.10 & 0.10 & 5.10 & 5.10 & 5.10 & 4.50 & 4.50 & 4.50 & 0.10 & 0.05 & 0.05 & 0.05 & 5.10 & 5.10 & 4.85 & 2.85 & 5.10 & 4.85 & 2.85 & 4.60 & 4.55 & 4.35 & 2.80 & 4.60 & 4.50 & 2.65 \\
 & 1000 & 0 & 5.05 & 5.05 & 5.05 & 6.95 & 6.95 & 6.95 & 7.15 & 7.15 & 7.15 & 5.05 & 5.00 & 4.95 & 3.05 & 6.95 & 6.95 & 6.85 & 4.30 & 6.95 & 6.85 & 4.30 & 7.25 & 7.20 & 7.15 & 4.20 & 7.20 & 6.95 & 4.50 \\
 & & 0.5 & 1.40 & 1.40 & 1.40 & 5.90 & 5.90 & 5.90 & 5.55 & 5.55 & 5.55 & 1.40 & 1.40 & 1.40 & 1.05 & 5.90 & 5.85 & 5.65 & 3.70 & 5.85 & 5.65 & 3.70 & 5.55 & 5.55 & 5.45 & 3.75 & 5.85 & 5.65 & 3.90 \\
 & & 0.9 & 0.05 & 0.05 & 0.05 & 5.10 & 5.10 & 5.10 & 5.05 & 5.05 & 5.05 & 0.05 & 0.05 & 0.05 & 0.05 & 5.10 & 5.10 & 4.95 & 3.30 & 5.10 & 4.95 & 3.30 & 5.05 & 5.05 & 4.85 & 3.40 & 5.60 & 5.30 & 3.45 \\
\bottomrule
\end{tabular}
}
\caption{Simulation results in Design 3: $\mu_{j}(\theta)=0$ for all $j=1,\dots,p$, $\Sigma(\theta)$ equicorrelated.}
\label{tab3}
\end{table}

\begin{table}
\scalebox{0.6}{
\begin{tabular}{cccccccccccccccccccccccccccccc}
\toprule Density & $p$ & $\rho$ & \multicolumn{9}{c}{Our methods} & \multicolumn{18}{c}{\citetalias{chernozhukov/chetverikov/kato:2014c}'s methods} \\
\cmidrule(lr){4-12}\cmidrule(lr){13-30}
 & & & \multicolumn{3}{c}{SN Lasso} & \multicolumn{3}{c}{MB Lasso} & \multicolumn{3}{c}{EB Lasso} & SN-1S & \multicolumn{3}{c}{SN-2S} & MB-1S & \multicolumn{3}{c}{MB-H} & \multicolumn{3}{c}{MB-2S} & EB-1S & \multicolumn{3}{c}{EB-H} & \multicolumn{3}{c}{EB-2S} \\
\cmidrule(lr){4-6}\cmidrule(lr){7-9}\cmidrule(lr){10-12}\cmidrule(lr){14-16}\cmidrule(lr){18-20}\cmidrule(lr){21-23}\cmidrule(lr){25-27}\cmidrule(lr){28-30}
 & & & C=2 & C=4 & C=6 & C=2 & C=4 & C=6 & C=2 & C=4 & C=6 & & 0.01\% & 0.10\% & 1.00\% & & 0.01\% & 0.10\% & 1.00\% & 0.01\% & 0.10\% & 1.00\% & & 0.01\% & 0.10\% & 1.00\% & 0.01\% & 0.10\% & 1.00\% \\
\midrule
$t_{4}/\sqrt{2}$ & 200 & 0 & 3.20 & 3.05 & 3.05 & 4.35 & 4.30 & 4.30 & 3.65 & 3.55 & 3.55 & 3.05 & 3.05 & 2.95 & 1.55 & 4.30 & 4.25 & 4.10 & 2.15 & 4.25 & 4.10 & 2.15 & 3.60 & 3.60 & 3.50 & 2.00 & 3.45 & 3.35 & 1.95 \\
 & & 0.5 & 3.85 & 3.80 & 3.80 & 4.90 & 4.85 & 4.85 & 4.50 & 4.45 & 4.45 & 3.80 & 3.80 & 3.80 & 2.45 & 4.85 & 4.85 & 4.55 & 3.05 & 4.85 & 4.55 & 3.05 & 4.55 & 4.55 & 4.20 & 2.85 & 4.55 & 4.45 & 2.95 \\
 & & 0.9 & 2.40 & 2.40 & 2.40 & 5.90 & 5.90 & 5.90 & 5.55 & 5.55 & 5.55 & 2.40 & 2.40 & 2.35 & 1.75 & 5.90 & 5.90 & 5.80 & 3.60 & 5.90 & 5.80 & 3.60 & 5.55 & 5.55 & 5.45 & 3.50 & 5.65 & 5.40 & 3.20 \\
 & 500 & 0 & 3.00 & 2.95 & 2.95 & 4.45 & 4.25 & 4.25 & 3.50 & 3.50 & 3.50 & 2.95 & 2.95 & 2.90 & 1.55 & 4.25 & 4.20 & 3.95 & 2.20 & 4.20 & 3.95 & 2.20 & 3.55 & 3.55 & 3.50 & 1.90 & 3.70 & 3.50 & 2.20 \\
 & & 0.5 & 3.90 & 3.75 & 3.75 & 5.70 & 5.45 & 5.45 & 5.00 & 4.85 & 4.85 & 3.75 & 3.75 & 3.45 & 1.95 & 5.45 & 5.45 & 5.30 & 3.05 & 5.45 & 5.30 & 3.05 & 4.95 & 4.95 & 4.55 & 2.45 & 4.50 & 4.40 & 2.55 \\
 & & 0.9 & 2.35 & 2.35 & 2.35 & 5.20 & 5.20 & 5.20 & 4.60 & 4.55 & 4.55 & 2.35 & 2.35 & 2.20 & 1.45 & 5.20 & 5.15 & 5.00 & 2.85 & 5.15 & 5.00 & 2.85 & 4.55 & 4.55 & 4.45 & 2.90 & 4.70 & 4.65 & 2.65 \\
 & 1000 & 0 & 3.40 & 3.20 & 3.20 & 4.65 & 4.40 & 4.40 & 3.75 & 3.70 & 3.70 & 3.20 & 3.15 & 3.05 & 1.80 & 4.40 & 4.40 & 4.25 & 2.75 & 4.40 & 4.25 & 2.75 & 3.70 & 3.70 & 3.60 & 1.95 & 3.90 & 3.85 & 2.25 \\
 & & 0.5 & 3.70 & 3.60 & 3.60 & 5.30 & 5.20 & 5.20 & 4.70 & 4.65 & 4.65 & 3.60 & 3.60 & 3.40 & 1.95 & 5.15 & 5.15 & 4.95 & 3.15 & 5.15 & 4.95 & 3.15 & 4.70 & 4.70 & 4.55 & 2.80 & 4.35 & 4.25 & 2.80 \\
 & & 0.9 & 2.30 & 2.30 & 2.30 & 5.20 & 5.15 & 5.15 & 5.00 & 4.85 & 4.85 & 2.30 & 2.30 & 2.30 & 1.55 & 5.15 & 5.10 & 5.05 & 3.20 & 5.10 & 5.05 & 3.20 & 4.95 & 4.95 & 4.65 & 2.90 & 4.90 & 4.80 & 2.80 \\
\midrule
$U(-\sqrt{3},\sqrt{3})$& 200 & 0 & 3.85 & 3.85 & 3.85 & 5.20 & 5.20 & 5.20 & 4.90 & 4.90 & 4.90 & 3.85 & 3.85 & 3.75 & 2.00 & 5.20 & 5.15 & 4.90 & 2.85 & 5.15 & 4.90 & 2.85 & 4.95 & 4.90 & 4.70 & 2.95 & 5.30 & 5.10 & 2.80 \\
 & & 0.5 & 4.35 & 4.35 & 4.35 & 5.20 & 5.20 & 5.20 & 5.25 & 5.25 & 5.25 & 4.35 & 4.35 & 4.05 & 2.95 & 5.20 & 5.15 & 5.05 & 3.35 & 5.15 & 5.05 & 3.35 & 5.40 & 5.35 & 5.10 & 3.50 & 5.30 & 5.10 & 3.45 \\
 & & 0.9 & 2.50 & 2.50 & 2.50 & 5.40 & 5.40 & 5.40 & 5.30 & 5.30 & 5.30 & 2.50 & 2.50 & 2.40 & 1.95 & 5.40 & 5.40 & 5.15 & 3.85 & 5.40 & 5.15 & 3.85 & 5.40 & 5.35 & 5.25 & 3.70 & 5.35 & 5.20 & 3.65 \\
 & 500 & 0 & 4.20 & 4.20 & 4.20 & 5.60 & 5.60 & 5.60 & 5.65 & 5.65 & 5.65 & 4.20 & 4.20 & 4.00 & 2.50 & 5.60 & 5.45 & 5.20 & 3.35 & 5.45 & 5.20 & 3.35 & 5.75 & 5.70 & 5.55 & 3.60 & 5.75 & 5.55 & 3.35 \\
 & & 0.5 & 3.85 & 3.85 & 3.85 & 5.25 & 5.25 & 5.25 & 5.15 & 5.15 & 5.15 & 3.85 & 3.85 & 3.75 & 2.55 & 5.25 & 5.15 & 5.05 & 3.60 & 5.15 & 5.05 & 3.60 & 5.20 & 5.20 & 5.05 & 3.35 & 5.20 & 5.10 & 3.25 \\
 & & 0.9 & 2.75 & 2.75 & 2.75 & 5.35 & 5.35 & 5.35 & 5.55 & 5.55 & 5.55 & 2.75 & 2.75 & 2.70 & 1.50 & 5.35 & 5.35 & 5.25 & 3.30 & 5.35 & 5.25 & 3.30 & 5.70 & 5.70 & 5.50 & 3.50 & 5.30 & 5.25 & 3.45 \\
 & 1000 & 0 & 5.05 & 5.05 & 5.05 & 6.95 & 6.95 & 6.95 & 7.15 & 7.15 & 7.15 & 5.05 & 5.00 & 4.95 & 3.05 & 6.95 & 6.95 & 6.85 & 4.30 & 6.95 & 6.85 & 4.30 & 7.25 & 7.20 & 7.15 & 4.20 & 7.20 & 6.95 & 4.50 \\
 & & 0.5 & 5.00 & 5.00 & 5.00 & 6.75 & 6.75 & 6.75 & 6.90 & 6.90 & 6.90 & 5.00 & 5.00 & 4.90 & 3.05 & 6.75 & 6.75 & 6.70 & 4.55 & 6.75 & 6.70 & 4.55 & 6.95 & 6.90 & 6.75 & 4.50 & 6.90 & 6.85 & 4.55 \\
 & & 0.9 & 2.95 & 2.95 & 2.95 & 6.40 & 6.40 & 6.40 & 6.30 & 6.30 & 6.30 & 2.95 & 2.95 & 2.95 & 1.85 & 6.40 & 6.40 & 6.20 & 3.95 & 6.40 & 6.20 & 3.95 & 6.35 & 6.35 & 6.25 & 4.15 & 6.40 & 6.20 & 4.25 \\
\bottomrule
\end{tabular}
}
\caption{Simulation results in Design 4: $\mu_{j}(\theta)=0$ for all $j=1,\dots,p$, $\Sigma(\theta)$ Toeplitz.}
\label{tab4}
\end{table}


\begin{table}
\scalebox{0.6}{
\begin{tabular}{cccccccccccccccccccccccccccccc}
\toprule Density & $p$ & $\rho$ & \multicolumn{9}{c}{Our methods} & \multicolumn{18}{c}{\citetalias{chernozhukov/chetverikov/kato:2014c}'s methods} \\
\cmidrule(lr){4-12}\cmidrule(lr){13-30}
 & & & \multicolumn{3}{c}{SN Lasso} & \multicolumn{3}{c}{MB Lasso} & \multicolumn{3}{c}{EB Lasso} & SN-1S & \multicolumn{3}{c}{SN-2S} & MB-1S & \multicolumn{3}{c}{MB-H} & \multicolumn{3}{c}{MB-2S} & EB-1S & \multicolumn{3}{c}{EB-H} & \multicolumn{3}{c}{EB-2S} \\
\cmidrule(lr){4-6}\cmidrule(lr){7-9}\cmidrule(lr){10-12}\cmidrule(lr){14-16}\cmidrule(lr){18-20}\cmidrule(lr){21-23}\cmidrule(lr){25-27}\cmidrule(lr){28-30}
 & & & C=2 & C=4 & C=6 & C=2 & C=4 & C=6 & C=2 & C=4 & C=6 & & 0.01\% & 0.10\% & 1.00\% & & 0.01\% & 0.10\% & 1.00\% & 0.01\% & 0.10\% & 1.00\% & & 0.01\% & 0.10\% & 1.00\% & 0.01\% & 0.10\% & 1.00\% \\
\midrule
$t_{4}/\sqrt{2}$ & 200 & 0 & 70.30 & 70.25 & 70.25 & 75.70 & 75.70 & 75.70 & 73.15 & 73.15 & 73.15 & 70.25 & 70.10 & 68.95 & 53.85 & 75.70 & 75.60 & 74.55 & 61.55 & 75.60 & 74.55 & 61.55 & 73.55 & 73.50 & 72.50 & 58.05 & 73.10 & 71.90 & 58.60 \\
 & & 0.5 & 22.15 & 22.15 & 22.15 & 37.75 & 37.75 & 37.75 & 36.75 & 36.75 & 36.75 & 22.15 & 22.10 & 21.60 & 16.55 & 37.75 & 37.70 & 37.25 & 28.50 & 37.70 & 37.25 & 28.50 & 37.00 & 36.80 & 36.10 & 28.15 & 37.00 & 36.25 & 27.95 \\
 & & 0.9 & 4.40 & 4.40 & 4.40 & 27.80 & 27.80 & 27.80 & 27.20 & 27.20 & 27.20 & 4.40 & 4.40 & 4.25 & 2.80 & 27.80 & 27.75 & 27.00 & 20.20 & 27.75 & 27.00 & 20.20 & 27.30 & 27.25 & 26.55 & 20.05 & 27.00 & 26.35 & 20.20 \\
 & 500 & 0 & 77.50 & 77.45 & 77.45 & 84.10 & 84.05 & 84.05 & 80.60 & 80.55 & 80.55 & 77.45 & 77.40 & 76.10 & 62.00 & 84.05 & 83.90 & 83.30 & 70.90 & 83.90 & 83.30 & 70.90 & 80.75 & 80.65 & 79.85 & 66.05 & 80.60 & 79.90 & 65.90 \\
 & & 0.5 & 20.95 & 20.95 & 20.95 & 38.35 & 38.35 & 38.35 & 37.05 & 37.05 & 37.05 & 20.95 & 20.90 & 20.75 & 15.70 & 38.35 & 38.25 & 37.85 & 30.05 & 38.25 & 37.85 & 30.05 & 37.30 & 37.25 & 36.50 & 28.45 & 36.95 & 36.70 & 28.90 \\
 & & 0.9 & 3.35 & 3.35 & 3.35 & 28.30 & 28.30 & 28.30 & 28.10 & 28.10 & 28.10 & 3.35 & 3.35 & 3.25 & 2.10 & 28.30 & 28.30 & 27.45 & 21.00 & 28.30 & 27.45 & 21.00 & 28.15 & 28.15 & 27.30 & 20.55 & 28.15 & 27.60 & 20.20 \\
 & 1000 & 0 & 81.70 & 81.65 & 81.65 & 89.15 & 89.10 & 89.10 & 84.65 & 84.50 & 84.50 & 81.65 & 81.55 & 80.45 & 66.30 & 89.10 & 89.00 & 88.05 & 77.75 & 89.00 & 88.05 & 77.75 & 84.95 & 84.95 & 84.05 & 70.60 & 84.95 & 84.10 & 70.75 \\
 & & 0.5 & 17.50 & 17.50 & 17.50 & 36.65 & 36.65 & 36.65 & 34.85 & 34.85 & 34.85 & 17.50 & 17.45 & 17.20 & 13.90 & 36.65 & 36.55 & 35.70 & 28.30 & 36.55 & 35.70 & 28.30 & 35.05 & 34.95 & 34.40 & 26.55 & 35.30 & 34.45 & 26.75 \\
 & & 0.9 & 2.05 & 2.05 & 2.05 & 26.20 & 26.20 & 26.20 & 26.00 & 26.00 & 26.00 & 2.05 & 2.05 & 2.05 & 1.65 & 26.20 & 26.15 & 25.70 & 19.05 & 26.15 & 25.70 & 19.05 & 26.20 & 26.15 & 25.15 & 18.45 & 25.70 & 24.90 & 18.15 \\
\midrule
$U(-\sqrt{3},\sqrt{3})$& 200 & 0 & 70.00 & 70.00 & 70.00 & 76.50 & 76.50 & 76.50 & 76.30 & 76.30 & 76.30 & 70.00 & 69.80 & 68.90 & 55.90 & 76.50 & 76.40 & 75.80 & 62.10 & 76.40 & 75.80 & 62.10 & 76.65 & 76.60 & 75.60 & 63.55 & 76.35 & 75.50 & 62.90 \\
 & & 0.5 & 22.50 & 22.50 & 22.50 & 38.20 & 38.20 & 38.20 & 37.95 & 37.95 & 37.95 & 22.50 & 22.50 & 21.75 & 17.30 & 38.20 & 38.05 & 37.40 & 29.00 & 38.05 & 37.40 & 29.00 & 38.20 & 38.10 & 37.25 & 29.25 & 38.15 & 37.35 & 29.25 \\
 & & 0.9 & 4.20 & 4.20 & 4.20 & 27.30 & 27.30 & 27.30 & 27.80 & 27.80 & 27.80 & 4.20 & 4.20 & 4.05 & 2.90 & 27.30 & 27.15 & 26.65 & 20.25 & 27.15 & 26.65 & 20.25 & 27.80 & 27.75 & 27.30 & 20.15 & 27.65 & 27.20 & 19.95 \\
 & 500 & 0 & 77.30 & 77.30 & 77.30 & 83.70 & 83.70 & 83.70 & 84.45 & 84.45 & 84.45 & 77.30 & 77.25 & 76.25 & 61.90 & 83.70 & 83.65 & 82.80 & 71.25 & 83.65 & 82.80 & 71.25 & 84.75 & 84.75 & 84.00 & 72.30 & 84.25 & 83.40 & 72.00 \\
 & & 0.5 & 20.50 & 20.50 & 20.50 & 40.20 & 40.20 & 40.20 & 39.70 & 39.70 & 39.70 & 20.50 & 20.45 & 19.90 & 16.00 & 40.20 & 40.10 & 39.45 & 29.95 & 40.10 & 39.45 & 29.95 & 40.00 & 39.75 & 39.20 & 30.05 & 39.55 & 38.65 & 30.15 \\
 & & 0.9 & 2.70 & 2.70 & 2.70 & 28.70 & 28.70 & 28.70 & 27.95 & 27.95 & 27.95 & 2.70 & 2.70 & 2.70 & 2.15 & 28.70 & 28.70 & 28.05 & 20.65 & 28.70 & 28.05 & 20.65 & 28.10 & 28.05 & 27.40 & 20.55 & 28.00 & 27.50 & 20.70 \\
 & 1000 & 0 & 83.40 & 83.40 & 83.40 & 90.00 & 90.00 & 90.00 & 89.95 & 89.95 & 89.95 & 83.40 & 83.35 & 82.05 & 68.50 & 90.00 & 89.85 & 89.20 & 79.20 & 89.85 & 89.20 & 79.20 & 90.35 & 90.25 & 89.85 & 79.70 & 90.05 & 89.35 & 80.05 \\
 & & 0.5 & 20.55 & 20.55 & 20.55 & 40.65 & 40.65 & 40.65 & 40.10 & 40.10 & 40.10 & 20.55 & 20.55 & 20.45 & 16.60 & 40.65 & 40.55 & 39.70 & 30.85 & 40.55 & 39.70 & 30.85 & 40.15 & 40.10 & 39.50 & 31.65 & 39.80 & 39.40 & 31.45 \\
 & & 0.9 & 2.30 & 2.30 & 2.30 & 28.25 & 28.25 & 28.25 & 28.00 & 28.00 & 28.00 & 2.30 & 2.30 & 2.20 & 1.55 & 28.25 & 28.25 & 28.00 & 20.80 & 28.25 & 28.00 & 20.80 & 28.15 & 28.10 & 27.65 & 20.45 & 28.55 & 27.75 & 20.40 \\
\bottomrule
\end{tabular}
}
\caption{Simulation results in Design 5: $\mu_{j}(\theta)=0.05$ for all $j=1,\dots p$, $\Sigma(\theta)$ equicorrelated.}
\label{tab5}
\end{table}

\begin{table}
\scalebox{0.6}{
\begin{tabular}{cccccccccccccccccccccccccccccc}
\toprule Density & $p$ & $\rho$ & \multicolumn{9}{c}{Our methods} & \multicolumn{18}{c}{\citetalias{chernozhukov/chetverikov/kato:2014c}'s methods} \\
\cmidrule(lr){4-12}\cmidrule(lr){13-30}
 & & & \multicolumn{3}{c}{SN Lasso} & \multicolumn{3}{c}{MB Lasso} & \multicolumn{3}{c}{EB Lasso} & SN-1S & \multicolumn{3}{c}{SN-2S} & MB-1S & \multicolumn{3}{c}{MB-H} & \multicolumn{3}{c}{MB-2S} & EB-1S & \multicolumn{3}{c}{EB-H} & \multicolumn{3}{c}{EB-2S} \\
\cmidrule(lr){4-6}\cmidrule(lr){7-9}\cmidrule(lr){10-12}\cmidrule(lr){14-16}\cmidrule(lr){18-20}\cmidrule(lr){21-23}\cmidrule(lr){25-27}\cmidrule(lr){28-30}
 & & & C=2 & C=4 & C=6 & C=2 & C=4 & C=6 & C=2 & C=4 & C=6 & & 0.01\% & 0.10\% & 1.00\% & & 0.01\% & 0.10\% & 1.00\% & 0.01\% & 0.10\% & 1.00\% & & 0.01\% & 0.10\% & 1.00\% & 0.01\% & 0.10\% & 1.00\% \\
\midrule
$t_{4}/\sqrt{2}$ & 200 & 0 & 70.30 & 70.25 & 70.25 & 75.70 & 75.70 & 75.70 & 73.15 & 73.15 & 73.15 & 70.25 & 70.10 & 68.95 & 53.85 & 75.70 & 75.60 & 74.55 & 61.55 & 75.60 & 74.55 & 61.55 & 73.55 & 73.50 & 72.50 & 58.05 & 73.10 & 71.90 & 58.60 \\
 & & 0.5 & 64.10 & 64.05 & 64.05 & 71.15 & 71.15 & 71.15 & 69.05 & 69.00 & 69.00 & 64.05 & 64.00 & 62.60 & 49.40 & 71.15 & 71.00 & 70.10 & 57.85 & 71.00 & 70.10 & 57.85 & 69.30 & 69.20 & 68.35 & 54.00 & 69.20 & 68.10 & 54.85 \\
 & & 0.9 & 33.70 & 33.70 & 33.70 & 53.05 & 53.05 & 53.05 & 52.35 & 52.35 & 52.35 & 33.70 & 33.65 & 33.00 & 25.15 & 53.05 & 53.05 & 52.35 & 41.35 & 53.05 & 52.35 & 41.35 & 52.60 & 52.40 & 51.40 & 40.70 & 52.35 & 51.35 & 40.00 \\
 & 500 & 0 & 77.50 & 77.45 & 77.45 & 84.10 & 84.05 & 84.05 & 80.60 & 80.55 & 80.55 & 77.45 & 77.40 & 76.10 & 62.00 & 84.05 & 83.90 & 83.30 & 70.90 & 83.90 & 83.30 & 70.90 & 80.75 & 80.65 & 79.85 & 66.05 & 80.60 & 79.90 & 65.90 \\
 & & 0.5 & 72.90 & 72.80 & 72.80 & 81.65 & 81.55 & 81.55 & 78.35 & 78.30 & 78.30 & 72.75 & 72.75 & 71.75 & 58.20 & 81.55 & 81.45 & 80.55 & 67.40 & 81.45 & 80.55 & 67.40 & 78.60 & 78.50 & 77.60 & 64.80 & 78.35 & 77.25 & 64.25 \\
 & & 0.9 & 43.50 & 43.50 & 43.50 & 65.40 & 65.40 & 65.40 & 64.50 & 64.50 & 64.50 & 43.50 & 43.45 & 42.10 & 31.60 & 65.40 & 65.30 & 64.15 & 52.10 & 65.30 & 64.15 & 52.10 & 64.75 & 64.65 & 63.70 & 51.25 & 64.75 & 63.45 & 50.95 \\
 & 1000 & 0 & 81.70 & 81.65 & 81.65 & 89.15 & 89.10 & 89.10 & 84.65 & 84.50 & 84.50 & 81.65 & 81.55 & 80.45 & 66.30 & 89.10 & 89.00 & 88.05 & 77.75 & 89.00 & 88.05 & 77.75 & 84.95 & 84.95 & 84.05 & 70.60 & 84.95 & 84.10 & 70.75 \\
 & & 0.5 & 79.45 & 79.40 & 79.40 & 87.75 & 87.75 & 87.75 & 83.65 & 83.65 & 83.65 & 79.40 & 79.35 & 78.55 & 65.15 & 87.75 & 87.65 & 86.50 & 76.15 & 87.65 & 86.50 & 76.15 & 84.15 & 84.05 & 83.40 & 71.30 & 84.75 & 84.00 & 71.35 \\
 & & 0.9 & 51.25 & 51.25 & 51.25 & 73.30 & 73.25 & 73.25 & 71.55 & 71.50 & 71.50 & 51.25 & 51.25 & 50.25 & 39.00 & 73.25 & 73.20 & 72.50 & 59.35 & 73.20 & 72.50 & 59.35 & 72.10 & 72.00 & 71.05 & 58.00 & 71.95 & 70.70 & 58.35 \\
\midrule
$U(-\sqrt{3},\sqrt{3})$& 200 & 0 & 70.00 & 70.00 & 70.00 & 76.50 & 76.50 & 76.50 & 76.30 & 76.30 & 76.30 & 70.00 & 69.80 & 68.90 & 55.90 & 76.50 & 76.40 & 75.80 & 62.10 & 76.40 & 75.80 & 62.10 & 76.65 & 76.60 & 75.60 & 63.55 & 76.35 & 75.50 & 62.90 \\
 & & 0.5 & 65.25 & 65.25 & 65.25 & 72.65 & 72.65 & 72.65 & 73.00 & 73.00 & 73.00 & 65.25 & 65.15 & 63.50 & 51.85 & 72.65 & 72.50 & 71.55 & 59.20 & 72.50 & 71.55 & 59.20 & 73.50 & 73.35 & 72.20 & 59.95 & 73.10 & 72.05 & 59.65 \\
 & & 0.9 & 34.10 & 34.10 & 34.10 & 53.45 & 53.45 & 53.45 & 53.90 & 53.90 & 53.90 & 34.10 & 34.10 & 33.30 & 25.70 & 53.45 & 53.20 & 52.00 & 41.85 & 53.20 & 52.00 & 41.85 & 54.25 & 54.10 & 53.15 & 41.85 & 53.80 & 52.40 & 41.35 \\
 & 500 & 0 & 77.30 & 77.30 & 77.30 & 83.70 & 83.70 & 83.70 & 84.45 & 84.45 & 84.45 & 77.30 & 77.25 & 76.25 & 61.90 & 83.70 & 83.65 & 82.80 & 71.25 & 83.65 & 82.80 & 71.25 & 84.75 & 84.75 & 84.00 & 72.30 & 84.25 & 83.40 & 72.00 \\
 & & 0.5 & 74.60 & 74.60 & 74.60 & 83.55 & 83.55 & 83.55 & 83.10 & 83.10 & 83.10 & 74.60 & 74.55 & 73.60 & 59.85 & 83.55 & 83.45 & 83.10 & 69.75 & 83.45 & 83.10 & 69.75 & 83.45 & 83.40 & 82.55 & 70.85 & 83.60 & 82.60 & 70.40 \\
 & & 0.9 & 42.85 & 42.85 & 42.85 & 64.80 & 64.80 & 64.80 & 65.70 & 65.70 & 65.70 & 42.85 & 42.80 & 41.70 & 32.90 & 64.80 & 64.80 & 64.15 & 53.10 & 64.80 & 64.15 & 53.10 & 66.05 & 66.05 & 65.25 & 53.30 & 64.90 & 64.20 & 53.40 \\
 & 1000 & 0 & 83.40 & 83.40 & 83.40 & 90.00 & 90.00 & 90.00 & 89.95 & 89.95 & 89.95 & 83.40 & 83.35 & 82.05 & 68.50 & 90.00 & 89.85 & 89.20 & 79.20 & 89.85 & 89.20 & 79.20 & 90.35 & 90.25 & 89.85 & 79.70 & 90.05 & 89.35 & 80.05 \\
 & & 0.5 & 80.50 & 80.50 & 80.50 & 88.70 & 88.70 & 88.70 & 89.15 & 89.15 & 89.15 & 80.50 & 80.35 & 79.55 & 66.55 & 88.70 & 88.70 & 87.90 & 77.00 & 88.70 & 87.90 & 77.00 & 89.25 & 89.20 & 88.30 & 77.25 & 88.85 & 88.05 & 78.15 \\
 & & 0.9 & 53.30 & 53.30 & 53.30 & 75.35 & 75.35 & 75.35 & 74.65 & 74.65 & 74.65 & 53.30 & 53.20 & 52.40 & 42.30 & 75.35 & 75.10 & 74.25 & 62.50 & 75.10 & 74.25 & 62.50 & 75.30 & 75.20 & 74.05 & 62.65 & 75.95 & 74.90 & 62.75 \\
\bottomrule
\end{tabular}
}
\caption{Simulation results in Design 6: $\mu_{j}(\theta)=0.05$ for all $j=1,\dots p$, $\Sigma(\theta)$ Toeplitz.}
\label{tab6}
\end{table}

\begin{table}
\scalebox{0.6}{
\begin{tabular}{cccccccccccccccccccccccccccccc}
\toprule Density & $p$ & $\rho$ & \multicolumn{9}{c}{Our methods} & \multicolumn{18}{c}{\citetalias{chernozhukov/chetverikov/kato:2014c}'s methods} \\
\cmidrule(lr){4-12}\cmidrule(lr){13-30}
 & & & \multicolumn{3}{c}{SN Lasso} & \multicolumn{3}{c}{MB Lasso} & \multicolumn{3}{c}{EB Lasso} & SN-1S & \multicolumn{3}{c}{SN-2S} & MB-1S & \multicolumn{3}{c}{MB-H} & \multicolumn{3}{c}{MB-2S} & EB-1S & \multicolumn{3}{c}{EB-H} & \multicolumn{3}{c}{EB-2S} \\
\cmidrule(lr){4-6}\cmidrule(lr){7-9}\cmidrule(lr){10-12}\cmidrule(lr){14-16}\cmidrule(lr){18-20}\cmidrule(lr){21-23}\cmidrule(lr){25-27}\cmidrule(lr){28-30}
 & & & C=2 & C=4 & C=6 & C=2 & C=4 & C=6 & C=2 & C=4 & C=6 & & 0.01\% & 0.10\% & 1.00\% & & 0.01\% & 0.10\% & 1.00\% & 0.01\% & 0.10\% & 1.00\% & & 0.01\% & 0.10\% & 1.00\% & 0.01\% & 0.10\% & 1.00\% \\
\midrule
$t_{4}/\sqrt{2}$ & 200 & 0 & 52.15 & 52.15 & 52.15 & 55.45 & 55.40 & 55.40 & 54.20 & 54.20 & 54.20 & 11.80 & 51.15 & 50.55 & 39.50 & 13.75 & 54.65 & 54.10 & 42.85 & 55.20 & 54.15 & 42.85 & 12.80 & 52.90 & 52.70 & 41.55 & 53.00 & 52.05 & 42.05 \\
 & & 0.5 & 27.15 & 27.15 & 27.15 & 35.35 & 35.35 & 35.35 & 34.55 & 34.55 & 34.55 & 7.10 & 26.90 & 26.35 & 20.90 & 15.15 & 35.15 & 34.10 & 27.20 & 35.25 & 34.10 & 27.20 & 14.80 & 34.50 & 34.10 & 26.65 & 34.85 & 34.30 & 26.45 \\
 & & 0.9 & 10.80 & 10.80 & 10.80 & 28.25 & 28.25 & 28.25 & 28.25 & 28.25 & 28.25 & 2.20 & 10.75 & 10.60 & 7.85 & 19.15 & 28.25 & 27.75 & 20.85 & 28.25 & 27.75 & 20.85 & 19.20 & 28.35 & 27.60 & 20.70 & 28.10 & 27.80 & 20.55 \\
 & 500 & 0 & 59.40 & 59.20 & 59.20 & 64.70 & 64.65 & 64.65 & 62.70 & 62.60 & 62.60 & 14.15 & 58.05 & 57.85 & 45.20 & 17.95 & 63.30 & 62.90 & 50.50 & 63.70 & 62.95 & 50.50 & 15.70 & 59.80 & 60.35 & 48.15 & 60.85 & 60.00 & 47.40 \\
 & & 0.5 & 25.35 & 25.35 & 25.35 & 36.65 & 36.65 & 36.65 & 36.05 & 36.05 & 36.05 & 7.70 & 25.05 & 24.75 & 19.75 & 16.00 & 36.15 & 35.65 & 28.35 & 36.30 & 35.60 & 28.35 & 15.05 & 35.45 & 34.95 & 27.50 & 35.80 & 35.30 & 27.55 \\
 & & 0.9 & 7.75 & 7.75 & 7.75 & 28.30 & 28.30 & 28.30 & 28.20 & 28.20 & 28.20 & 1.85 & 7.75 & 7.50 & 6.05 & 20.20 & 28.25 & 27.75 & 21.10 & 28.25 & 27.75 & 21.10 & 19.60 & 28.10 & 27.65 & 20.70 & 28.15 & 27.65 & 21.05 \\
 & 1000 & 0 & 66.20 & 66.10 & 66.10 & 71.40 & 71.25 & 71.25 & 69.10 & 69.00 & 68.95 & 16.15 & 64.00 & 63.80 & 50.90 & 19.70 & 69.40 & 69.35 & 56.25 & 70.30 & 69.45 & 56.25 & 17.95 & 64.05 & 65.20 & 53.00 & 65.25 & 64.75 & 52.50 \\
 & & 0.5 & 22.75 & 22.75 & 22.75 & 36.05 & 36.05 & 36.05 & 35.45 & 35.45 & 35.45 & 6.25 & 22.50 & 22.35 & 18.00 & 17.00 & 35.70 & 34.95 & 27.60 & 35.90 & 35.00 & 27.60 & 15.55 & 34.30 & 34.15 & 26.50 & 34.65 & 34.05 & 26.55 \\
 & & 0.9 & 5.00 & 5.00 & 5.00 & 26.90 & 26.90 & 26.90 & 26.35 & 26.35 & 26.35 & 1.40 & 5.00 & 4.95 & 3.60 & 19.05 & 26.85 & 26.15 & 19.40 & 26.85 & 26.15 & 19.40 & 18.30 & 26.35 & 25.65 & 19.40 & 26.25 & 25.80 & 19.30 \\
\midrule
$U(-\sqrt{3},\sqrt{3})$& 200 & 0 & 50.75 & 50.75 & 50.75 & 54.05 & 54.05 & 53.90 & 53.30 & 53.30 & 53.30 & 11.85 & 50.70 & 49.85 & 38.80 & 13.80 & 54.05 & 52.75 & 41.80 & 54.05 & 52.75 & 41.80 & 14.20 & 53.55 & 52.65 & 42.20 & 54.05 & 53.10 & 41.85\\
 & & 0.5 & 26.75 & 26.75 & 26.75 & 35.15 & 35.15 & 35.10 & 35.25 & 35.25 & 35.25 & 7.10 & 26.70 & 26.25 & 20.90 & 14.35 & 35.15 & 34.65 & 27.25 & 35.15 & 34.65 & 27.25 & 15.05 & 35.30 & 34.50 & 26.70 & 34.95 & 34.40 & 27.25 \\
 & & 0.9 & 9.60 & 9.60 & 9.60 & 27.85 & 27.85 & 27.85 & 27.80 & 27.80 & 27.80 & 2.30 & 9.60 & 9.30 & 6.95 & 19.20 & 27.70 & 27.25 & 20.40 & 27.70 & 27.25 & 20.40 & 18.70 & 27.80 & 27.30 & 20.10 & 27.95 & 27.40 & 20.10 \\
 & 500 & 0 & 59.10 & 59.10 & 59.05 & 63.25 & 63.25 & 63.25 & 64.45 & 64.45 & 64.35 & 14.00 & 59.05 & 58.00 & 45.80 & 16.80 & 63.05 & 62.25 & 50.50 & 63.05 & 62.25 & 50.50 & 17.50 & 64.50 & 63.00 & 50.15 & 64.15 & 63.40 & 50.40 \\
 & & 0.5 & 26.15 & 26.15 & 26.10 & 36.80 & 36.80 & 36.80 & 36.55 & 36.55 & 36.55 & 7.25 & 26.00 & 25.40 & 19.60 & 16.55 & 36.80 & 36.05 & 29.40 & 36.80 & 36.05 & 29.40 & 16.45 & 36.70 & 35.90 & 28.80 & 36.65 & 35.65 & 28.55 \\
 & & 0.9 & 7.10 & 7.10 & 7.10 & 28.00 & 28.00 & 28.00 & 27.75 & 27.75 & 27.75 & 1.80 & 7.05 & 6.95 & 5.00 & 20.35 & 27.90 & 27.25 & 21.15 & 27.90 & 27.25 & 21.15 & 20.40 & 27.80 & 27.15 & 20.75 & 28.25 & 27.40 & 20.75 \\
 & 1000 & 0 & 64.55 & 64.55 & 64.55 & 70.15 & 70.15 & 70.10 & 69.65 & 69.65 & 69.60 & 15.75 & 64.45 & 63.45 & 48.40 & 20.05 & 70.05 & 68.95 & 54.15 & 70.05 & 68.95 & 54.15 & 20.40 & 70.25 & 69.00 & 55.00 & 69.85 & 69.15 & 55.40 \\
 & & 0.5 & 24.90 & 24.90 & 24.85 & 37.50 & 37.50 & 37.50 & 37.40 & 37.40 & 37.40 & 6.75 & 24.75 & 24.35 & 19.20 & 17.45 & 37.45 & 36.70 & 29.40 & 37.45 & 36.70 & 29.40 & 17.70 & 37.40 & 36.90 & 29.60 & 37.10 & 36.25 & 29.80 \\
 & & 0.9 & 5.75 & 5.75 & 5.70 & 27.65 & 27.65 & 27.65 & 27.55 & 27.55 & 27.55 & 1.15 & 5.75 & 5.70 & 3.65 & 20.60 & 27.60 & 26.95 & 20.95 & 27.60 & 26.95 & 20.95 & 20.20 & 27.65 & 26.85 & 20.00 & 27.60 & 27.00 & 20.35 \\
\bottomrule
\end{tabular}
}
\caption{Simulation results in Design 7: $\mu_{j}(\theta)=-0.75\cdot 1[j>0.1 p]+0.05\cdot 1[j\leq 0.1 p]$, $\Sigma(\theta)$ equicorrelated.}
\label{tab7}
\end{table}

\begin{table}
\scalebox{0.6}{
\begin{tabular}{cccccccccccccccccccccccccccccc}
\toprule Density & $p$ & $\rho$ & \multicolumn{9}{c}{Our methods} & \multicolumn{18}{c}{\citetalias{chernozhukov/chetverikov/kato:2014c}'s methods} \\
\cmidrule(lr){4-12}\cmidrule(lr){13-30}
 & & & \multicolumn{3}{c}{SN Lasso} & \multicolumn{3}{c}{MB Lasso} & \multicolumn{3}{c}{EB Lasso} & SN-1S & \multicolumn{3}{c}{SN-2S} & MB-1S & \multicolumn{3}{c}{MB-H} & \multicolumn{3}{c}{MB-2S} & EB-1S & \multicolumn{3}{c}{EB-H} & \multicolumn{3}{c}{EB-2S} \\
\cmidrule(lr){4-6}\cmidrule(lr){7-9}\cmidrule(lr){10-12}\cmidrule(lr){14-16}\cmidrule(lr){18-20}\cmidrule(lr){21-23}\cmidrule(lr){25-27}\cmidrule(lr){28-30}
 & & & C=2 & C=4 & C=6 & C=2 & C=4 & C=6 & C=2 & C=4 & C=6 & & 0.01\% & 0.10\% & 1.00\% & & 0.01\% & 0.10\% & 1.00\% & 0.01\% & 0.10\% & 1.00\% & & 0.01\% & 0.10\% & 1.00\% & 0.01\% & 0.10\% & 1.00\% \\
\midrule
$t_{4}/\sqrt{2}$ & 200 & 0 & 52.15 & 52.15 & 52.15 & 55.45 & 55.40 & 55.40 & 54.20 & 54.20 & 54.20 & 11.80 & 51.15 & 50.55 & 39.50 & 13.75 & 54.65 & 54.10 & 42.85 & 55.20 & 54.15 & 42.85 & 12.80 & 52.90 & 52.70 & 41.55 & 53.00 & 52.05 & 42.05 \\
 & & 0.5 & 43.60 & 43.55 & 43.55 & 48.40 & 48.40 & 48.40 & 47.25 & 47.20 & 47.20 & 10.15 & 42.95 & 42.15 & 32.60 & 12.50 & 48.10 & 47.20 & 35.80 & 48.10 & 47.30 & 35.85 & 11.50 & 46.25 & 46.00 & 35.90 & 47.35 & 46.55 & 35.50 \\
 & & 0.9 & 18.85 & 18.85 & 18.85 & 33.95 & 33.95 & 33.90 & 34.15 & 34.15 & 34.15 & 4.85 & 18.70 & 18.50 & 14.25 & 8.45 & 33.50 & 32.90 & 24.45 & 33.65 & 32.90 & 24.45 & 8.60 & 33.75 & 33.10 & 24.40 & 32.95 & 32.45 & 24.30 \\
 & 500 & 0 & 59.40 & 59.20 & 59.20 & 64.70 & 64.65 & 64.65 & 62.70 & 62.60 & 62.60 & 14.15 & 58.05 & 57.85 & 45.20 & 17.95 & 63.30 & 62.90 & 50.50 & 63.70 & 62.95 & 50.50 & 15.70 & 59.80 & 60.35 & 48.15 & 60.85 & 60.00 & 47.40 \\
 & & 0.5 & 53.45 & 53.45 & 53.45 & 59.40 & 59.30 & 59.30 & 57.55 & 57.45 & 57.45 & 12.45 & 52.65 & 51.95 & 41.55 & 15.55 & 58.05 & 57.80 & 46.60 & 58.65 & 57.75 & 46.65 & 14.60 & 55.65 & 55.40 & 44.45 & 56.10 & 55.40 & 44.55 \\
 & & 0.9 & 25.75 & 25.75 & 25.75 & 41.60 & 41.60 & 41.60 & 41.25 & 41.25 & 41.25 & 6.00 & 25.70 & 25.25 & 18.50 & 11.20 & 41.00 & 40.80 & 31.50 & 41.25 & 40.80 & 31.55 & 10.75 & 40.45 & 40.15 & 31.10 & 41.50 & 40.80 & 31.05 \\
 & 1000 & 0 & 66.20 & 66.10 & 66.10 & 71.40 & 71.25 & 71.25 & 69.10 & 69.00 & 68.95 & 16.15 & 64.00 & 63.80 & 50.90 & 19.70 & 69.40 & 69.35 & 56.25 & 70.30 & 69.45 & 56.25 & 17.95 & 64.05 & 65.20 & 53.00 & 65.25 & 64.75 & 52.50 \\
 & & 0.5 & 59.60 & 59.35 & 59.35 & 65.75 & 65.65 & 65.60 & 64.10 & 64.05 & 64.00 & 15.40 & 57.65 & 57.45 & 46.45 & 19.80 & 64.40 & 64.25 & 52.40 & 65.10 & 64.40 & 52.55 & 18.00 & 59.80 & 60.55 & 49.45 & 61.80 & 61.05 & 49.90 \\
 & & 0.9 & 30.80 & 30.75 & 30.75 & 46.85 & 46.85 & 46.85 & 46.25 & 46.25 & 46.25 & 6.90 & 30.50 & 30.00 & 23.55 & 13.60 & 46.25 & 45.75 & 36.75 & 46.70 & 45.90 & 36.75 & 13.50 & 45.75 & 45.40 & 37.05 & 45.20 & 45.05 & 36.30 \\
\midrule
$U(-\sqrt{3},\sqrt{3})$& 200 & 0 & 50.75 & 50.75 & 50.75 & 54.05 & 54.05 & 53.90 & 53.30 & 53.30 & 53.30 & 11.85 & 50.70 & 49.85 & 38.80 & 13.80 & 54.05 & 52.75 & 41.80 & 54.05 & 52.75 & 41.80 & 14.20 & 53.55 & 52.65 & 42.20 & 54.05 & 53.10 & 41.85 \\
 & & 0.5 & 44.80 & 44.80 & 44.70 & 49.75 & 49.75 & 49.70 & 49.65 & 49.65 & 49.65 & 9.95 & 44.75 & 43.80 & 33.65 & 11.95 & 49.70 & 48.70 & 37.75 & 49.70 & 48.70 & 37.75 & 12.25 & 49.75 & 49.15 & 38.25 & 49.65 & 49.00 & 37.70 \\
 & & 0.9 & 19.50 & 19.50 & 19.45 & 34.65 & 34.65 & 34.60 & 34.20 & 34.20 & 34.20 & 4.75 & 19.50 & 19.25 & 14.90 & 8.85 & 34.50 & 34.00 & 24.55 & 34.50 & 34.00 & 24.55 & 9.15 & 34.35 & 33.75 & 25.15 & 34.20 & 33.45 & 25.50 \\
 & 500 & 0 & 59.10 & 59.10 & 59.05 & 63.25 & 63.25 & 63.25 & 64.45 & 64.45 & 64.35 & 14.00 & 59.05 & 58.00 & 45.80 & 16.80 & 63.05 & 62.25 & 50.50 & 63.05 & 62.25 & 50.50 & 17.50 & 64.50 & 63.00 & 50.15 & 64.15 & 63.40 & 50.40 \\
 & & 0.5 & 52.75 & 52.75 & 52.70 & 58.00 & 58.00 & 57.95 & 57.50 & 57.50 & 57.45 & 13.40 & 52.70 & 51.60 & 40.15 & 16.40 & 57.85 & 57.05 & 44.85 & 57.85 & 57.05 & 44.85 & 16.65 & 57.80 & 56.80 & 45.15 & 57.80 & 57.15 & 45.70 \\
 & & 0.9 & 25.70 & 25.70 & 25.70 & 40.65 & 40.65 & 40.60 & 39.90 & 39.90 & 39.90 & 5.65 & 25.65 & 25.00 & 19.25 & 11.05 & 40.60 & 40.15 & 31.75 & 40.60 & 40.15 & 31.75 & 11.55 & 39.90 & 39.30 & 32.05 & 40.25 & 39.95 & 31.95 \\
 & 1000 & 0 & 64.55 & 64.55 & 64.55 & 70.15 & 70.15 & 70.10 & 69.65 & 69.65 & 69.60 & 15.75 & 64.45 & 63.45 & 48.40 & 20.05 & 70.05 & 68.95 & 54.15 & 70.05 & 68.95 & 54.15 & 20.40 & 70.25 & 69.00 & 55.00 & 69.85 & 69.15 & 55.40 \\
 & & 0.5 & 58.65 & 58.65 & 58.65 & 65.75 & 65.75 & 65.70 & 65.30 & 65.30 & 65.30 & 14.75 & 58.55 & 57.45 & 44.25 & 18.55 & 65.65 & 64.70 & 51.55 & 65.65 & 64.70 & 51.55 & 19.20 & 65.60 & 64.05 & 51.35 & 65.40 & 64.20 & 52.45 \\
 & & 0.9 & 30.65 & 30.65 & 30.65 & 46.55 & 46.55 & 46.55 & 45.85 & 45.85 & 45.75 & 6.80 & 30.60 & 29.95 & 23.35 & 13.40 & 46.35 & 45.55 & 36.55 & 46.35 & 45.55 & 36.55 & 13.05 & 46.00 & 45.05 & 36.40 & 46.55 & 45.90 & 35.90 \\
\bottomrule
\end{tabular}
}
\caption{Simulation results in Design 8: $\mu_{j}(\theta)=-0.75\cdot 1[j>0.1 p]+0.05\cdot 1[j\leq 0.1 p]$, $\Sigma(\theta)$ Toeplitz.}
\label{tab8}
\end{table}

%%
\begin{table}
\scalebox{0.6}{
\begin{tabular}{cccccccccccccccccccccccccccccc}
\toprule Density & $p$ & $\rho$ & \multicolumn{9}{c}{Our methods} & \multicolumn{18}{c}{\citetalias{chernozhukov/chetverikov/kato:2014c}'s methods} \\
\cmidrule(lr){4-12}\cmidrule(lr){13-30}
 & & & \multicolumn{3}{c}{SN Lasso} & \multicolumn{3}{c}{MB Lasso} & \multicolumn{3}{c}{EB Lasso} & SN-1S & \multicolumn{3}{c}{SN-2S} & MB-1S & \multicolumn{3}{c}{MB-H} & \multicolumn{3}{c}{MB-2S} & EB-1S & \multicolumn{3}{c}{EB-H} & \multicolumn{3}{c}{EB-2S} \\
\cmidrule(lr){4-6}\cmidrule(lr){7-9}\cmidrule(lr){10-12}\cmidrule(lr){14-16}\cmidrule(lr){18-20}\cmidrule(lr){21-23}\cmidrule(lr){25-27}\cmidrule(lr){28-30}
 & & & C=2 & C=4 & C=6 & C=2 & C=4 & C=6 & C=2 & C=4 & C=6 & & 0.01\% & 0.10\% & 1.00\% & & 0.01\% & 0.10\% & 1.00\% & 0.01\% & 0.10\% & 1.00\% & & 0.01\% & 0.10\% & 1.00\% & 0.01\% & 0.10\% & 1.00\% \\
\midrule
$t_{4}/\sqrt{2}$ & 200 & 0 & 52.15 & 52.15 & 50.65 & 55.45 & 55.40 & 53.90 & 54.20 & 54.20 & 52.45 & 11.80 & 40.50 & 47.80 & 38.85 & 13.75 & 43.20 & 50.90 & 46.55 & 51.10 & 51.15 & 42.60 & 12.80 & 40.50 & 48.05 & 39.90 & 47.05 & 48.15 & 40.45 \\
 & & 0.5 & 43.60 & 43.55 & 41.75 & 48.40 & 48.40 & 46.75 & 47.25 & 47.20 & 44.80 & 10.15 & 33.55 & 39.15 & 32.30 & 12.50 & 37.50 & 44.50 & 40.85 & 44.55 & 44.75 & 35.55 & 11.50 & 35.95 & 42.80 & 35.20 & 42.55 & 43.00 & 34.65 \\
 & & 0.9 & 18.85 & 18.85 & 14.95 & 33.95 & 33.95 & 26.40 & 34.15 & 34.15 & 25.95 & 4.85 & 16.15 & 17.90 & 14.20 & 8.45 & 25.70 & 31.75 & 29.35 & 32.10 & 32.10 & 24.45 & 8.60 & 24.95 & 31.25 & 24.15 & 30.45 & 31.10 & 24.15 \\
 & 500 & 0 & 59.40 & 59.20 & 58.80 & 64.70 & 64.65 & 64.40 & 62.70 & 62.60 & 61.90 & 14.15 & 40.65 & 52.00 & 44.35 & 17.95 & 46.65 & 56.65 & 51.10 & 56.90 & 57.85 & 49.60 & 15.70 & 40.30 & 51.45 & 44.55 & 48.80 & 50.20 & 44.65 \\
 & & 0.5 & 53.45 & 53.45 & 53.10 & 59.40 & 59.30 & 58.80 & 57.55 & 57.45 & 56.70 & 12.45 & 38.15 & 47.15 & 40.35 & 15.55 & 43.10 & 52.80 & 48.45 & 52.75 & 53.50 & 46.05 & 14.60 & 38.85 & 48.80 & 42.05 & 47.20 & 48.45 & 42.50 \\
 & & 0.9 & 25.75 & 25.75 & 23.80 & 41.60 & 41.60 & 38.45 & 41.25 & 41.25 & 37.50 & 6.00 & 18.05 & 23.55 & 18.40 & 11.20 & 28.15 & 36.40 & 35.20 & 38.05 & 38.45 & 31.45 & 10.75 & 27.30 & 35.40 & 30.20 & 36.60 & 36.95 & 30.45 \\
 & 1000 & 0 & 66.20 & 66.10 & 65.90 & 71.40 & 71.25 & 71.15 & 69.10 & 69.00 & 68.30 & 16.15 & 41.80 & 54.35 & 48.25 & 19.70 & 48.15 & 59.90 & 54.10 & 61.80 & 62.70 & 54.90 & 17.95 & 40.25 & 52.70 & 48.05 & 49.15 & 51.45 & 47.35 \\
 & & 0.5 & 59.60 & 59.35 & 59.20 & 65.75 & 65.65 & 65.50 & 64.10 & 64.05 & 63.45 & 15.40 & 38.00 & 49.95 & 44.70 & 19.80 & 44.85 & 57.20 & 51.15 & 58.55 & 59.35 & 51.40 & 18.00 & 38.65 & 49.75 & 45.55 & 49.00 & 50.45 & 46.05 \\
 & & 0.9 & 30.80 & 30.75 & 30.40 & 46.85 & 46.85 & 45.20 & 46.25 & 46.25 & 44.45 & 6.90 & 20.30 & 27.30 & 23.30 & 13.60 & 30.25 & 39.95 & 39.35 & 41.25 & 41.90 & 36.35 & 13.50 & 28.65 & 38.40 & 35.50 & 39.50 & 40.15 & 35.10 \\
\midrule
$U(-\sqrt{3},\sqrt{3})$& 200 & 0 & 50.75 & 50.75 & 15.45 & 54.05 & 54.00 & 17.85 & 53.30 & 53.30 & 18.05 & 11.85 & 45.35 & 49.10 & 38.80 & 13.80 & 48.45 & 52.30 & 48.25 & 53.20 & 52.45 & 41.80 & 14.20 & 48.60 & 52.50 & 42.20 & 53.40 & 52.90 & 41.85 \\
 & & 0.5 & 44.80 & 44.80 & 14.30 & 49.75 & 49.75 & 16.65 & 49.65 & 49.65 & 16.95 & 9.95 & 39.05 & 43.50 & 33.65 & 11.95 & 43.60 & 48.15 & 44.15 & 48.40 & 48.35 & 37.75 & 12.25 & 43.80 & 48.70 & 38.25 & 48.65 & 48.75 & 37.70 \\
 & & 0.9 & 19.50 & 19.50 & 7.10 & 34.65 & 34.65 & 12.50 & 34.20 & 34.20 & 12.55 & 4.75 & 17.35 & 19.20 & 14.90 & 8.85 & 28.40 & 33.40 & 30.30 & 33.60 & 33.65 & 24.55 & 9.15 & 28.70 & 33.20 & 25.15 & 33.20 & 33.10 & 25.50 \\
 & 500 & 0 & 59.10 & 59.10 & 18.90 & 63.25 & 63.25 & 23.10 & 64.45 & 64.45 & 23.55 & 14.00 & 46.40 & 56.60 & 45.75 & 16.80 & 51.05 & 60.85 & 58.25 & 61.00 & 61.30 & 50.50 & 17.50 & 51.65 & 61.90 & 50.15 & 62.10 & 62.35 & 50.40 \\
 & & 0.5 & 52.75 & 52.75 & 18.75 & 58.00 & 57.95 & 21.80 & 57.50 & 57.50 & 22.25 & 13.40 & 40.65 & 50.05 & 40.15 & 16.40 & 45.45 & 55.90 & 53.10 & 55.70 & 55.90 & 44.85 & 16.65 & 46.00 & 55.90 & 45.10 & 55.95 & 56.30 & 45.70 \\
 & & 0.9 & 25.70 & 25.70 & 9.25 & 40.65 & 40.65 & 15.55 & 39.90 & 39.90 & 15.50 & 5.65 & 19.15 & 24.45 & 19.25 & 11.05 & 29.80 & 38.65 & 36.85 & 38.75 & 38.95 & 31.75 & 11.55 & 30.15 & 37.95 & 32.05 & 37.95 & 39.05 & 31.95 \\
 & 1000 & 0 & 64.55 & 64.55 & 21.95 & 70.15 & 70.15 & 26.70 & 69.65 & 69.60 & 27.55 & 15.75 & 43.60 & 58.95 & 48.25 & 20.05 & 49.60 & 65.40 & 64.00 & 65.25 & 66.40 & 54.00 & 20.40 & 50.50 & 65.80 & 54.95 & 65.45 & 66.85 & 55.35 \\
 & & 0.5 & 58.65 & 58.65 & 21.55 & 65.75 & 65.70 & 26.25 & 65.30 & 65.30 & 26.80 & 14.75 & 39.90 & 54.00 & 44.05 & 18.55 & 46.25 & 60.80 & 58.85 & 61.55 & 62.15 & 51.55 & 19.20 & 46.70 & 60.60 & 51.25 & 61.45 & 62.05 & 52.45 \\
 & & 0.9 & 30.65 & 30.65 & 11.90 & 46.55 & 46.55 & 19.85 & 45.85 & 45.85 & 19.60 & 6.80 & 20.85 & 28.40 & 23.25 & 13.40 & 31.60 & 41.60 & 41.25 & 43.25 & 43.55 & 36.40 & 13.05 & 31.85 & 41.30 & 36.15 & 42.50 & 43.30 & 35.80 \\
\bottomrule
\end{tabular}
}
\caption{Simulation results in Design 9: $\mu_{j}(\theta)=-0.6\cdot 1[j>0.1 p]+0.05\cdot 1[j\leq 0.1 p]$, $\Sigma(\theta)$ Toeplitz.}
\label{tab9}
\end{table}

\begin{table}
\scalebox{0.6}{
\begin{tabular}{cccccccccccccccccccccccccccccc}
\toprule Density & $p$ & $\rho$ & \multicolumn{9}{c}{Our methods} & \multicolumn{18}{c}{\citetalias{chernozhukov/chetverikov/kato:2014c}'s methods} \\
\cmidrule(lr){4-12}\cmidrule(lr){13-30}
 & & & \multicolumn{3}{c}{SN Lasso} & \multicolumn{3}{c}{MB Lasso} & \multicolumn{3}{c}{EB Lasso} & SN-1S & \multicolumn{3}{c}{SN-2S} & MB-1S & \multicolumn{3}{c}{MB-H} & \multicolumn{3}{c}{MB-2S} & EB-1S & \multicolumn{3}{c}{EB-H} & \multicolumn{3}{c}{EB-2S} \\
\cmidrule(lr){4-6}\cmidrule(lr){7-9}\cmidrule(lr){10-12}\cmidrule(lr){14-16}\cmidrule(lr){18-20}\cmidrule(lr){21-23}\cmidrule(lr){25-27}\cmidrule(lr){28-30}
 & & & C=2 & C=4 & C=6 & C=2 & C=4 & C=6 & C=2 & C=4 & C=6 & & 0.01\% & 0.10\% & 1.00\% & & 0.01\% & 0.10\% & 1.00\% & 0.01\% & 0.10\% & 1.00\% & & 0.01\% & 0.10\% & 1.00\% & 0.01\% & 0.10\% & 1.00\% \\
\midrule
$t_{4}/\sqrt{2}$ & 200 & 0 & 52.15 & 51.95 & 42.45 & 55.45 & 55.25 & 45.45 & 54.20 & 53.90 & 42.20 & 11.80 & 17.75 & 27.65 & 31.40 & 13.75 & 20.80 & 31.05 & 34.25 & 32.50 & 34.40 & 36.50 & 12.80 & 18.40 & 28.50 & 31.85 & 27.85 & 29.05 & 32.70 \\
 & & 0.5 & 43.60 & 43.45 & 32.85 & 48.40 & 48.30 & 36.90 & 47.25 & 47.10 & 34.55 & 10.15 & 15.15 & 23.95 & 27.45 & 12.50 & 18.40 & 27.10 & 31.10 & 28.30 & 29.90 & 31.35 & 11.50 & 17.15 & 24.95 & 29.00 & 24.60 & 25.80 & 28.95 \\
 & & 0.9 & 18.85 & 18.60 & 8.10 & 33.95 & 32.80 & 14.45 & 34.15 & 32.80 & 13.85 & 4.85 & 6.70 & 10.60 & 12.55 & 8.45 & 11.55 & 17.15 & 20.85 & 19.40 & 21.05 & 22.65 & 8.60 & 11.45 & 16.40 & 19.65 & 18.85 & 20.15 & 21.85 \\
 & 500 & 0 & 59.40 & 59.10 & 56.85 & 64.70 & 64.60 & 61.70 & 62.70 & 62.50 & 57.60 & 14.15 & 19.50 & 27.05 & 31.55 & 17.95 & 22.95 & 30.95 & 32.40 & 33.40 & 35.20 & 39.05 & 15.70 & 20.35 & 27.40 & 30.40 & 26.30 & 27.05 & 30.65 \\
 & & 0.5 & 53.45 & 53.45 & 49.35 & 59.40 & 59.30 & 54.70 & 57.55 & 57.25 & 51.20 & 12.45 & 16.65 & 24.90 & 29.60 & 15.55 & 20.80 & 29.35 & 31.30 & 31.60 & 33.30 & 36.75 & 14.60 & 18.40 & 26.15 & 29.70 & 25.30 & 26.20 & 30.75 \\
 & & 0.9 & 25.75 & 25.65 & 15.75 & 41.60 & 41.50 & 25.85 & 41.25 & 40.75 & 24.65 & 6.00 & 7.50 & 11.55 & 13.90 & 11.20 & 13.25 & 18.05 & 22.00 & 20.80 & 21.90 & 26.50 & 10.75 & 12.40 & 17.15 & 21.55 & 19.20 & 20.70 & 24.65 \\
 & 1000 & 0 & 66.20 & 66.05 & 64.85 & 71.40 & 71.20 & 70.15 & 69.10 & 68.85 & 65.40 & 16.15 & 19.40 & 25.85 & 30.40 & 19.70 & 23.80 & 31.80 & 30.80 & 34.80 & 36.85 & 40.45 & 17.95 & 20.35 & 26.10 & 28.50 & 25.50 & 26.50 & 29.75 \\
 & & 0.5 & 59.60 & 59.35 & 57.65 & 65.75 & 65.55 & 64.15 & 64.10 & 63.85 & 60.30 & 15.40 & 19.05 & 25.45 & 30.15 & 19.80 & 23.60 & 30.90 & 31.15 & 32.60 & 33.90 & 37.55 & 18.00 & 21.30 & 26.50 & 29.75 & 26.25 & 27.10 & 30.75 \\
 & & 0.9 & 30.80 & 30.75 & 23.75 & 46.85 & 46.80 & 36.80 & 46.25 & 45.95 & 35.50 & 6.90 & 8.55 & 12.50 & 15.40 & 13.60 & 15.40 & 20.40 & 23.35 & 22.95 & 23.80 & 28.60 & 13.50 & 15.00 & 19.30 & 23.10 & 21.40 & 22.45 & 26.50 \\
\midrule
$U(-\sqrt{3},\sqrt{3})$& 200 & 0 & 50.75 & 37.65 & 11.85 & 54.05 & 41.10 & 13.85 & 53.30 & 41.00 & 14.05 & 11.85 & 16.85 & 29.40 & 35.35 & 13.80 & 19.30 & 32.20 & 41.85 & 33.45 & 35.85 & 39.20 & 14.20 & 19.85 & 32.55 & 38.85 & 34.20 & 36.80 & 39.45 \\
 & & 0.5 & 44.80 & 33.60 & 10.15 & 49.75 & 37.50 & 12.00 & 49.65 & 38.00 & 12.30 & 9.95 & 14.85 & 25.25 & 30.75 & 11.95 & 17.40 & 27.90 & 37.20 & 29.80 & 32.10 & 35.00 & 12.25 & 17.90 & 29.35 & 34.75 & 30.90 & 32.50 & 35.65 \\
 & & 0.9 & 19.50 & 15.90 & 4.80 & 34.65 & 25.50 & 8.90 & 34.20 & 25.60 & 9.05 & 4.75 & 7.00 & 11.40 & 13.70 & 8.85 & 12.00 & 18.10 & 24.10 & 20.85 & 22.15 & 23.35 & 9.15 & 12.30 & 18.05 & 22.35 & 21.20 & 23.05 & 24.35 \\
 & 500 & 0 & 59.10 & 44.90 & 14.00 & 63.25 & 49.70 & 16.85 & 64.45 & 50.30 & 17.30 & 14.00 & 17.20 & 26.60 & 35.75 & 16.80 & 21.50 & 31.50 & 41.15 & 33.85 & 35.95 & 42.95 & 17.50 & 22.05 & 32.30 & 40.15 & 34.75 & 37.10 & 42.85 \\
 & & 0.5 & 52.75 & 41.10 & 13.40 & 58.00 & 46.10 & 16.50 & 57.50 & 46.85 & 16.55 & 13.40 & 16.90 & 23.55 & 31.75 & 16.40 & 20.05 & 28.25 & 37.90 & 30.85 & 33.10 & 38.65 & 16.65 & 20.35 & 28.55 & 37.20 & 31.70 & 33.45 & 39.20 \\
 & & 0.9 & 25.70 & 20.45 & 5.80 & 40.65 & 32.15 & 11.10 & 39.90 & 31.85 & 11.50 & 5.65 & 7.60 & 11.75 & 15.15 & 11.05 & 12.90 & 18.65 & 23.70 & 22.05 & 23.55 & 27.70 & 11.55 & 13.15 & 18.70 & 23.35 & 21.55 & 23.05 & 27.70 \\
 & 1000 & 0 & 64.55 & 49.05 & 15.75 & 70.15 & 55.40 & 20.15 & 69.65 & 55.70 & 19.85 & 15.75 & 18.65 & 25.60 & 33.20 & 20.05 & 23.30 & 30.50 & 38.90 & 33.25 & 35.10 & 42.70 & 20.40 & 23.75 & 31.15 & 39.25 & 35.10 & 36.55 & 42.95 \\
 & & 0.5 & 58.65 & 46.95 & 14.80 & 65.75 & 54.00 & 18.60 & 65.30 & 53.70 & 18.95 & 14.75 & 17.50 & 23.70 & 31.10 & 18.55 & 21.50 & 28.45 & 36.65 & 31.80 & 32.30 & 39.60 & 19.20 & 22.40 & 29.50 & 36.55 & 32.30 & 34.25 & 40.20 \\
 & & 0.9 & 30.65 & 25.85 & 6.90 & 46.55 & 38.40 & 13.50 & 45.85 & 38.15 & 12.95 & 6.80 & 8.15 & 11.95 & 15.10 & 13.40 & 15.00 & 19.80 & 25.25 & 23.00 & 24.20 & 29.50 & 13.05 & 14.95 & 19.65 & 25.35 & 23.50 & 24.50 & 28.90 \\
\bottomrule
\end{tabular}
}
\caption{Simulation results in Design 10: $\mu_{j}(\theta)=-0.5\cdot 1[j>0.1 p]+0.05\cdot 1[j\leq 0.1 p]$, $\Sigma(\theta)$ Toeplitz.}
\label{tab10}
\end{table}



\begin{table}
\scalebox{0.6}{
\begin{tabular}{cccccccccccccccccccccccccccccc}
\toprule Density & $p$ & $\rho$ & \multicolumn{9}{c}{Our methods} & \multicolumn{18}{c}{\citetalias{chernozhukov/chetverikov/kato:2014c}'s methods} \\
\cmidrule(lr){4-12}\cmidrule(lr){13-30}
 & & & \multicolumn{3}{c}{SN Lasso} & \multicolumn{3}{c}{MB Lasso} & \multicolumn{3}{c}{EB Lasso} & SN-1S & \multicolumn{3}{c}{SN-2S} & MB-1S & \multicolumn{3}{c}{MB-H} & \multicolumn{3}{c}{MB-2S} & EB-1S & \multicolumn{3}{c}{EB-H} & \multicolumn{3}{c}{EB-2S} \\
\cmidrule(lr){4-6}\cmidrule(lr){7-9}\cmidrule(lr){10-12}\cmidrule(lr){14-16}\cmidrule(lr){18-20}\cmidrule(lr){21-23}\cmidrule(lr){25-27}\cmidrule(lr){28-30}
 & & & C=2 & C=4 & C=6 & C=2 & C=4 & C=6 & C=2 & C=4 & C=6 & & 0.01\% & 0.10\% & 1.00\% & & 0.01\% & 0.10\% & 1.00\% & 0.01\% & 0.10\% & 1.00\% & & 0.01\% & 0.10\% & 1.00\% & 0.01\% & 0.10\% & 1.00\% \\
\midrule
$t_{4}/\sqrt{2}$ & 200 & 0 & 52.10 & 48.25 & 26.70 & 55.45 & 51.55 & 30.10 & 54.20 & 49.10 & 27.05 & 11.80 & 12.10 & 12.90 & 13.15 & 13.75 & 14.05 & 15.20 & 14.65 & 15.90 & 16.30 & 16.50 & 12.80 & 13.10 & 13.95 & 13.70 & 14.50 & 14.35 & 14.10 \\
 & & 0.5 & 43.60 & 38.80 & 19.15 & 48.40 & 43.30 & 22.65 & 47.25 & 40.85 & 21.00 & 10.15 & 10.40 & 11.20 & 11.45 & 12.50 & 12.65 & 13.55 & 13.30 & 14.30 & 14.35 & 14.70 & 11.50 & 11.90 & 12.60 & 12.30 & 13.20 & 13.10 & 12.90 \\
 & & 0.9 & 18.85 & 12.35 & 5.75 & 33.95 & 20.65 & 10.25 & 34.15 & 19.60 & 10.20 & 4.85 & 4.85 & 5.00 & 5.00 & 8.45 & 8.45 & 8.90 & 9.20 & 9.30 & 9.30 & 10.65 & 8.60 & 8.60 & 9.00 & 8.15 & 9.30 & 9.25 & 9.80 \\
 & 500 & 0 & 59.40 & 58.35 & 42.65 & 64.70 & 63.55 & 47.30 & 62.70 & 60.75 & 42.05 & 14.15 & 14.35 & 14.85 & 13.85 & 17.95 & 17.90 & 18.70 & 16.40 & 19.70 & 19.55 & 17.80 & 15.70 & 16.05 & 16.70 & 14.40 & 16.75 & 16.55 & 14.70 \\
 & & 0.5 & 53.45 & 51.90 & 34.95 & 59.40 & 57.70 & 40.00 & 57.55 & 55.05 & 36.40 & 12.45 & 12.50 & 12.90 & 11.75 & 15.55 & 15.75 & 16.10 & 14.90 & 16.70 & 16.55 & 15.75 & 14.60 & 14.60 & 14.90 & 12.95 & 15.05 & 15.05 & 13.25 \\
 & & 0.9 & 25.75 & 20.60 & 9.60 & 41.60 & 33.35 & 16.85 & 41.25 & 32.10 & 15.90 & 6.00 & 6.00 & 6.05 & 5.40 & 11.20 & 11.20 & 11.30 & 10.70 & 11.90 & 11.90 & 11.30 & 10.75 & 10.75 & 10.85 & 9.30 & 11.40 & 11.40 & 10.25 \\
 & 1000 & 0 & 66.20 & 65.60 & 56.10 & 71.40 & 70.75 & 61.15 & 69.10 & 67.20 & 55.30 & 16.15 & 16.20 & 16.50 & 13.40 & 19.70 & 19.80 & 19.85 & 17.20 & 20.55 & 20.35 & 18.60 & 17.95 & 18.05 & 18.05 & 14.40 & 18.00 & 18.05 & 14.45 \\
 & & 0.5 & 59.60 & 58.70 & 48.25 & 65.75 & 65.25 & 54.30 & 64.10 & 62.45 & 49.50 & 15.40 & 15.45 & 15.65 & 13.80 & 19.80 & 19.90 & 20.20 & 17.10 & 20.50 & 20.65 & 18.35 & 18.00 & 18.05 & 18.10 & 15.15 & 18.50 & 18.50 & 15.50 \\
 & & 0.9 & 30.80 & 28.50 & 14.45 & 46.85 & 42.25 & 23.55 & 46.25 & 41.80 & 22.95 & 6.90 & 6.95 & 6.90 & 6.15 & 13.60 & 13.60 & 13.55 & 12.20 & 14.10 & 14.05 & 12.70 & 13.50 & 13.55 & 13.50 & 10.15 & 13.60 & 13.35 & 11.40 \\
\midrule
$U(-\sqrt{3},\sqrt{3})$& 200 & 0 & 50.70 & 13.40 & 11.85 & 53.95 & 15.25 & 13.80 & 53.25 & 15.65 & 14.00 & 11.85 & 11.85 & 12.75 & 13.25 & 13.80 & 13.95 & 14.95 & 16.20 & 15.45 & 15.55 & 16.00 & 14.20 & 14.30 & 15.10 & 15.40 & 15.30 & 15.40 & 16.15 \\
 & & 0.5 & 44.75 & 11.40 & 9.95 & 49.75 & 13.70 & 11.95 & 49.65 & 13.80 & 12.20 & 9.95 & 10.20 & 10.80 & 11.15 & 11.95 & 12.05 & 12.95 & 14.35 & 13.80 & 13.65 & 14.35 & 12.25 & 12.35 & 13.30 & 13.35 & 13.90 & 14.20 & 14.55 \\
 & & 0.9 & 19.45 & 5.75 & 4.75 & 34.65 & 10.20 & 8.85 & 34.20 & 10.45 & 9.05 & 4.75 & 4.80 & 5.15 & 5.25 & 8.85 & 8.95 & 9.35 & 10.15 & 10.00 & 10.05 & 11.10 & 9.15 & 9.35 & 9.70 & 9.30 & 10.50 & 10.40 & 11.35 \\
 & 500 & 0 & 59.10 & 15.60 & 14.00 & 63.25 & 19.10 & 16.80 & 64.40 & 19.50 & 17.30 & 14.00 & 14.00 & 14.05 & 12.60 & 16.80 & 16.90 & 17.05 & 17.60 & 18.25 & 17.90 & 16.45 & 17.50 & 17.45 & 17.90 & 15.70 & 19.15 & 18.95 & 17.05 \\
 & & 0.5 & 52.75 & 15.55 & 13.40 & 57.95 & 18.80 & 16.40 & 57.50 & 19.00 & 16.40 & 13.40 & 13.40 & 13.65 & 12.15 & 16.40 & 16.50 & 16.60 & 16.95 & 17.80 & 17.60 & 16.50 & 16.65 & 16.70 & 17.15 & 15.05 & 17.85 & 17.85 & 16.65 \\
 & & 0.9 & 25.70 & 7.30 & 5.65 & 40.65 & 12.85 & 11.05 & 39.80 & 12.80 & 11.45 & 5.65 & 5.75 & 5.95 & 5.15 & 11.05 & 11.00 & 11.10 & 11.05 & 11.60 & 11.60 & 11.30 & 11.55 & 11.55 & 11.55 & 9.75 & 11.65 & 11.75 & 11.30 \\
 & 1000 & 0 & 64.55 & 18.10 & 15.75 & 70.05 & 22.60 & 20.05 & 69.55 & 22.75 & 19.85 & 15.75 & 15.70 & 15.60 & 12.80 & 20.05 & 20.10 & 20.05 & 19.50 & 20.65 & 20.35 & 18.10 & 20.40 & 20.35 & 20.50 & 17.50 & 21.50 & 21.00 & 18.30 \\
 & & 0.5 & 58.65 & 17.50 & 14.75 & 65.70 & 21.60 & 18.55 & 65.30 & 22.25 & 18.95 & 14.75 & 14.75 & 14.75 & 12.40 & 18.55 & 18.55 & 18.70 & 18.50 & 19.45 & 19.25 & 16.80 & 19.20 & 19.20 & 19.30 & 16.45 & 20.05 & 19.70 & 17.05 \\
 & & 0.9 & 30.65 & 8.65 & 6.80 & 46.55 & 15.95 & 13.40 & 45.75 & 15.65 & 12.90 & 6.80 & 6.80 & 6.80 & 5.80 & 13.40 & 13.35 & 13.30 & 12.50 & 13.85 & 13.85 & 12.40 & 13.05 & 13.10 & 12.80 & 10.80 & 13.70 & 13.60 & 11.95 \\
\bottomrule
\end{tabular}
}
\caption{Simulation results in Design 11: $\mu_{j}(\theta)=-0.4\cdot 1[j>0.1 p]+0.05\cdot 1[j\leq 0.1 p]$, $\Sigma(\theta)$ Toeplitz.}
\label{tab11}
\end{table}


\begin{table}
\scalebox{0.6}{
\begin{tabular}{cccccccccccccccccccccccccccccc}
\toprule Density & $p$ & $\rho$ & \multicolumn{9}{c}{Our methods} & \multicolumn{18}{c}{\citetalias{chernozhukov/chetverikov/kato:2014c}'s methods} \\
\cmidrule(lr){4-12}\cmidrule(lr){13-30}
 & & & \multicolumn{3}{c}{SN Lasso} & \multicolumn{3}{c}{MB Lasso} & \multicolumn{3}{c}{EB Lasso} & SN-1S & \multicolumn{3}{c}{SN-2S} & MB-1S & \multicolumn{3}{c}{MB-H} & \multicolumn{3}{c}{MB-2S} & EB-1S & \multicolumn{3}{c}{EB-H} & \multicolumn{3}{c}{EB-2S} \\
\cmidrule(lr){4-6}\cmidrule(lr){7-9}\cmidrule(lr){10-12}\cmidrule(lr){14-16}\cmidrule(lr){18-20}\cmidrule(lr){21-23}\cmidrule(lr){25-27}\cmidrule(lr){28-30}
 & & & C=2 & C=4 & C=6 & C=2 & C=4 & C=6 & C=2 & C=4 & C=6 & & 0.01\% & 0.10\% & 1.00\% & & 0.01\% & 0.10\% & 1.00\% & 0.01\% & 0.10\% & 1.00\% & & 0.01\% & 0.10\% & 1.00\% & 0.01\% & 0.10\% & 1.00\% \\
\midrule
$t_{4}/\sqrt{2}$ & 200 & 0 & 51.60 & 30.90 & 15.75 & 55.15 & 33.95 & 17.70 & 53.50 & 31.50 & 16.45 & 11.80 & 11.75 & 11.70 & 8.15 & 13.75 & 13.65 & 13.30 & 11.20 & 13.70 & 13.35 & 10.30 & 12.80 & 12.75 & 12.45 & 9.05 & 12.90 & 12.50 & 9.15 \\
 & & 0.5 & 42.85 & 23.15 & 12.85 & 48.05 & 26.60 & 15.60 & 46.75 & 24.70 & 14.40 & 10.15 & 10.15 & 9.80 & 6.75 & 12.50 & 12.50 & 12.10 & 10.05 & 12.55 & 12.15 & 8.95 & 11.50 & 11.55 & 11.05 & 8.15 & 11.55 & 11.00 & 7.95 \\
 & & 0.9 & 18.25 & 6.25 & 5.10 & 31.75 & 11.05 & 9.00 & 31.65 & 11.10 & 9.05 & 4.85 & 4.85 & 4.70 & 3.60 & 8.45 & 8.45 & 8.35 & 7.45 & 8.45 & 8.35 & 6.50 & 8.60 & 8.55 & 8.20 & 6.20 & 8.25 & 8.20 & 6.15 \\
 & 500 & 0 & 58.90 & 45.85 & 23.75 & 64.60 & 51.40 & 28.40 & 62.35 & 45.70 & 24.05 & 14.15 & 14.10 & 14.00 & 9.85 & 17.95 & 17.75 & 17.25 & 13.85 & 17.80 & 17.25 & 12.50 & 15.70 & 15.65 & 15.30 & 11.20 & 15.45 & 15.10 & 10.95 \\
 & & 0.5 & 53.30 & 38.85 & 20.15 & 59.35 & 44.25 & 24.70 & 57.05 & 40.20 & 22.35 & 12.45 & 12.45 & 12.05 & 8.40 & 15.55 & 15.55 & 15.20 & 11.95 & 15.55 & 15.20 & 11.10 & 14.60 & 14.60 & 14.25 & 10.25 & 14.50 & 13.85 & 10.00 \\
 & & 0.9 & 25.30 & 11.55 & 7.00 & 40.80 & 18.95 & 12.15 & 39.60 & 17.60 & 11.55 & 6.00 & 6.00 & 5.90 & 3.85 & 11.20 & 11.20 & 10.80 & 9.15 & 11.20 & 10.80 & 7.75 & 10.75 & 10.75 & 10.45 & 7.30 & 10.60 & 10.45 & 7.15 \\
 & 1000 & 0 & 66.00 & 58.00 & 36.15 & 71.30 & 63.10 & 41.45 & 68.70 & 58.00 & 35.50 & 16.15 & 16.15 & 15.65 & 10.90 & 19.70 & 19.65 & 19.25 & 15.55 & 19.65 & 19.25 & 14.20 & 17.95 & 17.95 & 17.50 & 12.05 & 17.55 & 17.25 & 12.10 \\
 & & 0.5 & 59.55 & 50.40 & 28.95 & 65.65 & 56.85 & 35.20 & 63.75 & 51.75 & 30.70 & 15.40 & 15.40 & 15.20 & 11.45 & 19.80 & 19.75 & 19.50 & 15.95 & 19.75 & 19.50 & 14.65 & 18.00 & 17.95 & 17.50 & 13.10 & 17.95 & 17.80 & 13.05 \\
 & & 0.9 & 30.60 & 17.05 & 9.10 & 46.20 & 26.95 & 16.70 & 45.40 & 26.15 & 16.25 & 6.90 & 6.90 & 6.75 & 4.70 & 13.60 & 13.60 & 13.50 & 11.05 & 13.60 & 13.50 & 9.05 & 13.50 & 13.50 & 13.10 & 8.75 & 13.05 & 12.50 & 8.50 \\
\midrule
$U(-\sqrt{3},\sqrt{3})$& 200 & 0 & 37.30 & 11.85 & 11.85 & 40.30 & 13.80 & 13.80 & 39.95 & 14.00 & 14.00 & 11.85 & 11.65 & 11.50 & 8.30 & 13.80 & 13.80 & 13.60 & 12.20 & 13.85 & 13.60 & 10.60 & 14.20 & 14.20 & 13.70 & 10.45 & 13.55 & 13.35 & 10.10 \\
 & & 0.5 & 32.90 & 10.10 & 9.95 & 36.85 & 11.95 & 11.95 & 37.45 & 12.20 & 12.20 & 9.95 & 9.95 & 9.75 & 7.65 & 11.95 & 11.90 & 11.70 & 10.80 & 11.90 & 11.70 & 9.15 & 12.25 & 12.25 & 11.85 & 9.15 & 12.10 & 11.75 & 9.25 \\
 & & 0.9 & 15.75 & 4.75 & 4.75 & 24.90 & 8.85 & 8.85 & 25.05 & 9.05 & 9.05 & 4.75 & 4.75 & 4.65 & 3.90 & 8.85 & 8.75 & 8.70 & 7.70 & 8.75 & 8.70 & 6.30 & 9.15 & 9.15 & 8.95 & 6.35 & 8.95 & 8.80 & 6.25 \\
 & 500 & 0 & 43.85 & 14.00 & 14.00 & 48.15 & 16.80 & 16.80 & 48.95 & 17.30 & 17.30 & 14.00 & 14.00 & 13.55 & 8.95 & 16.80 & 16.80 & 16.30 & 15.10 & 16.85 & 16.30 & 11.85 & 17.50 & 17.45 & 17.00 & 12.45 & 17.30 & 17.10 & 11.95 \\
 & & 0.5 & 39.85 & 13.40 & 13.40 & 44.50 & 16.45 & 16.40 & 45.35 & 16.50 & 16.40 & 13.40 & 13.40 & 13.15 & 9.40 & 16.40 & 16.35 & 16.20 & 14.60 & 16.40 & 16.20 & 12.25 & 16.65 & 16.55 & 16.25 & 12.20 & 16.40 & 16.00 & 12.45 \\
 & & 0.9 & 19.85 & 5.75 & 5.65 & 30.85 & 11.10 & 11.05 & 30.90 & 11.45 & 11.45 & 5.65 & 5.65 & 5.40 & 3.90 & 11.05 & 11.00 & 10.65 & 9.50 & 11.00 & 10.70 & 7.90 & 11.55 & 11.50 & 11.20 & 7.65 & 10.90 & 10.55 & 7.65 \\
 & 1000 & 0 & 46.55 & 15.75 & 15.75 & 53.25 & 20.10 & 20.05 & 53.65 & 19.85 & 19.85 & 15.75 & 15.65 & 15.25 & 10.60 & 20.05 & 20.00 & 19.50 & 17.80 & 20.00 & 19.50 & 14.30 & 20.40 & 20.30 & 19.75 & 14.55 & 20.65 & 19.90 & 14.75 \\
 & & 0.5 & 45.00 & 14.75 & 14.75 & 52.50 & 18.55 & 18.55 & 51.55 & 18.95 & 18.95 & 14.75 & 14.75 & 14.40 & 10.90 & 18.55 & 18.50 & 18.15 & 17.10 & 18.50 & 18.15 & 13.90 & 19.20 & 19.15 & 18.75 & 14.10 & 19.50 & 18.85 & 13.95 \\
 & & 0.9 & 24.35 & 6.80 & 6.80 & 37.00 & 13.45 & 13.40 & 36.60 & 12.90 & 12.90 & 6.80 & 6.80 & 6.70 & 5.15 & 13.40 & 13.30 & 13.05 & 11.35 & 13.30 & 13.05 & 8.80 & 13.05 & 13.05 & 12.70 & 8.90 & 13.25 & 12.75 & 9.00 \\\bottomrule
\end{tabular}
}
\caption{Simulation results in Design 12: $\mu_{j}(\theta)=-0.3\cdot 1[j>0.1 p]+0.05\cdot 1[j\leq 0.1 p]$, $\Sigma(\theta)$ Toeplitz.}
\label{tab12}
\end{table}

\begin{table}
\scalebox{0.6}{
\begin{tabular}{cccccccccccccccccccccccccccccc}
\toprule Density & $p$ & $\rho$ & \multicolumn{9}{c}{Our methods} & \multicolumn{18}{c}{\citetalias{chernozhukov/chetverikov/kato:2014c}'s methods} \\
\cmidrule(lr){4-12}\cmidrule(lr){13-30}
 & & & \multicolumn{3}{c}{SN Lasso} & \multicolumn{3}{c}{MB Lasso} & \multicolumn{3}{c}{EB Lasso} & SN-1S & \multicolumn{3}{c}{SN-2S} & MB-1S & \multicolumn{3}{c}{MB-H} & \multicolumn{3}{c}{MB-2S} & EB-1S & \multicolumn{3}{c}{EB-H} & \multicolumn{3}{c}{EB-2S} \\
\cmidrule(lr){4-6}\cmidrule(lr){7-9}\cmidrule(lr){10-12}\cmidrule(lr){14-16}\cmidrule(lr){18-20}\cmidrule(lr){21-23}\cmidrule(lr){25-27}\cmidrule(lr){28-30}
 & & & C=2 & C=4 & C=6 & C=2 & C=4 & C=6 & C=2 & C=4 & C=6 & & 0.01\% & 0.10\% & 1.00\% & & 0.01\% & 0.10\% & 1.00\% & 0.01\% & 0.10\% & 1.00\% & & 0.01\% & 0.10\% & 1.00\% & 0.01\% & 0.10\% & 1.00\% \\
\midrule
$t_{4}/\sqrt{2}$ & 200 & 0 & 39.05 & 15.50 & 12.40 & 42.00 & 17.70 & 14.30 & 38.85 & 16.10 & 13.25 & 11.80 & 11.75 & 11.70 & 7.80 & 13.75 & 13.65 & 13.30 & 9.75 & 13.65 & 13.30 & 9.75 & 12.80 & 12.75 & 12.45 & 8.80 & 12.90 & 12.40 & 8.80 \\
 & & 0.5 & 30.85 & 12.80 & 10.50 & 34.10 & 15.25 & 12.95 & 32.35 & 14.00 & 11.95 & 10.15 & 10.15 & 9.80 & 6.45 & 12.50 & 12.50 & 12.05 & 8.80 & 12.50 & 12.05 & 8.80 & 11.50 & 11.50 & 11.05 & 7.85 & 11.50 & 11.00 & 7.60 \\
 & & 0.9 & 9.45 & 5.10 & 4.90 & 15.40 & 8.95 & 8.55 & 14.90 & 8.95 & 8.60 & 4.85 & 4.85 & 4.65 & 3.45 & 8.45 & 8.45 & 8.35 & 6.15 & 8.45 & 8.35 & 6.15 & 8.60 & 8.55 & 8.20 & 6.10 & 8.25 & 8.20 & 5.80 \\
 & 500 & 0 & 51.75 & 22.85 & 16.75 & 56.55 & 27.20 & 20.25 & 52.20 & 23.25 & 17.70 & 14.15 & 14.10 & 14.00 & 9.75 & 17.95 & 17.75 & 17.25 & 12.25 & 17.75 & 17.25 & 12.25 & 15.70 & 15.65 & 15.30 & 11.20 & 15.45 & 15.10 & 10.90 \\
 & & 0.5 & 44.25 & 19.25 & 13.85 & 50.10 & 23.90 & 17.30 & 46.40 & 20.75 & 15.75 & 12.45 & 12.45 & 12.05 & 8.20 & 15.55 & 15.55 & 15.15 & 10.90 & 15.55 & 15.15 & 10.90 & 14.60 & 14.60 & 14.25 & 10.15 & 14.50 & 13.85 & 9.90 \\
 & & 0.9 & 15.40 & 7.05 & 6.10 & 24.80 & 12.05 & 11.30 & 22.90 & 11.50 & 10.80 & 6.00 & 6.00 & 5.90 & 3.85 & 11.20 & 11.20 & 10.80 & 7.65 & 11.20 & 10.80 & 7.65 & 10.75 & 10.75 & 10.45 & 7.30 & 10.60 & 10.45 & 7.00 \\
 & 1000 & 0 & 59.95 & 32.25 & 21.05 & 65.55 & 38.60 & 24.40 & 60.70 & 32.05 & 21.75 & 16.15 & 16.15 & 15.65 & 10.90 & 19.70 & 19.65 & 19.25 & 14.15 & 19.65 & 19.25 & 14.15 & 17.95 & 17.95 & 17.50 & 12.00 & 17.55 & 17.25 & 12.05 \\
 & & 0.5 & 53.50 & 27.00 & 18.45 & 59.70 & 32.90 & 23.10 & 54.95 & 28.60 & 20.50 & 15.40 & 15.40 & 15.20 & 11.35 & 19.80 & 19.75 & 19.50 & 14.50 & 19.75 & 19.50 & 14.50 & 18.00 & 17.95 & 17.45 & 13.10 & 17.95 & 17.80 & 12.95 \\
 & & 0.9 & 22.25 & 9.00 & 7.00 & 33.70 & 16.25 & 14.00 & 32.50 & 15.90 & 13.85 & 6.90 & 6.90 & 6.75 & 4.70 & 13.60 & 13.60 & 13.50 & 9.05 & 13.60 & 13.50 & 9.05 & 13.50 & 13.50 & 13.10 & 8.75 & 13.05 & 12.50 & 8.50 \\
\midrule
$U(-\sqrt{3},\sqrt{3})$& 200 & 0 & 13.70 & 11.85 & 11.85 & 15.75 & 13.80 & 13.80 & 15.80 & 14.00 & 14.00 & 11.85 & 11.65 & 11.50 & 8.20 & 13.80 & 13.80 & 13.60 & 10.35 & 13.80 & 13.60 & 10.35 & 14.20 & 14.20 & 13.70 & 10.30 & 13.50 & 13.35 & 9.85 \\
 & & 0.5 & 11.80 & 9.95 & 9.95 & 14.15 & 11.95 & 11.95 & 14.30 & 12.20 & 12.20 & 9.95 & 9.95 & 9.75 & 7.55 & 11.95 & 11.90 & 11.70 & 9.05 & 11.90 & 11.70 & 9.05 & 12.25 & 12.25 & 11.85 & 9.00 & 12.05 & 11.75 & 9.00 \\
 & & 0.9 & 5.85 & 4.75 & 4.75 & 10.40 & 8.85 & 8.85 & 10.55 & 9.05 & 9.05 & 4.75 & 4.75 & 4.65 & 3.80 & 8.85 & 8.75 & 8.70 & 6.00 & 8.75 & 8.70 & 6.00 & 9.15 & 9.15 & 8.95 & 6.25 & 8.95 & 8.80 & 5.85 \\
 & 500 & 0 & 16.05 & 14.00 & 14.00 & 19.85 & 16.80 & 16.80 & 20.20 & 17.30 & 17.30 & 14.00 & 14.00 & 13.55 & 8.95 & 16.80 & 16.80 & 16.30 & 11.80 & 16.80 & 16.30 & 11.80 & 17.50 & 17.45 & 17.00 & 12.35 & 17.30 & 17.10 & 11.75 \\
 & & 0.5 & 15.90 & 13.40 & 13.40 & 19.30 & 16.40 & 16.40 & 19.30 & 16.40 & 16.40 & 13.40 & 13.40 & 13.15 & 9.40 & 16.40 & 16.35 & 16.20 & 12.15 & 16.35 & 16.20 & 12.15 & 16.65 & 16.55 & 16.25 & 12.15 & 16.40 & 16.00 & 12.30 \\
 & & 0.9 & 7.40 & 5.65 & 5.65 & 12.85 & 11.05 & 11.05 & 12.90 & 11.45 & 11.45 & 5.65 & 5.65 & 5.40 & 3.75 & 11.05 & 11.00 & 10.65 & 7.90 & 11.00 & 10.65 & 7.90 & 11.55 & 11.50 & 11.15 & 7.60 & 10.90 & 10.55 & 7.60 \\
 & 1000 & 0 & 18.60 & 15.75 & 15.75 & 23.30 & 20.05 & 20.05 & 23.25 & 19.85 & 19.85 & 15.75 & 15.65 & 15.25 & 10.60 & 20.05 & 20.00 & 19.50 & 14.15 & 20.00 & 19.50 & 14.15 & 20.40 & 20.30 & 19.75 & 14.55 & 20.65 & 19.85 & 14.75 \\
 & & 0.5 & 17.80 & 14.75 & 14.75 & 22.30 & 18.55 & 18.55 & 22.60 & 18.95 & 18.95 & 14.75 & 14.75 & 14.40 & 10.90 & 18.55 & 18.50 & 18.15 & 13.85 & 18.50 & 18.15 & 13.85 & 19.20 & 19.15 & 18.75 & 14.10 & 19.50 & 18.85 & 13.85 \\
 & & 0.9 & 8.65 & 6.80 & 6.80 & 15.95 & 13.40 & 13.40 & 15.60 & 12.90 & 12.90 & 6.80 & 6.80 & 6.70 & 5.15 & 13.40 & 13.30 & 13.05 & 8.65 & 13.30 & 13.05 & 8.65 & 13.05 & 13.05 & 12.70 & 8.90 & 13.25 & 12.75 & 8.95\\
\bottomrule
\end{tabular}
}
\caption{Simulation results in Design 13: $\mu_{j}(\theta)=-0.2\cdot 1[j>0.1 p]+0.05\cdot 1[j\leq 0.1 p]$, $\Sigma(\theta)$ Toeplitz.}
\label{tab13}
\end{table}

\begin{table}
\scalebox{0.6}{
\begin{tabular}{cccccccccccccccccccccccccccccc}
\toprule Density & $p$ & $\rho$ & \multicolumn{9}{c}{Our methods} & \multicolumn{18}{c}{\citetalias{chernozhukov/chetverikov/kato:2014c}'s methods} \\
\cmidrule(lr){4-12}\cmidrule(lr){13-30}
 & & & \multicolumn{3}{c}{SN Lasso} & \multicolumn{3}{c}{MB Lasso} & \multicolumn{3}{c}{EB Lasso} & SN-1S & \multicolumn{3}{c}{SN-2S} & MB-1S & \multicolumn{3}{c}{MB-H} & \multicolumn{3}{c}{MB-2S} & EB-1S & \multicolumn{3}{c}{EB-H} & \multicolumn{3}{c}{EB-2S} \\
\cmidrule(lr){4-6}\cmidrule(lr){7-9}\cmidrule(lr){10-12}\cmidrule(lr){14-16}\cmidrule(lr){18-20}\cmidrule(lr){21-23}\cmidrule(lr){25-27}\cmidrule(lr){28-30}
 & & & C=2 & C=4 & C=6 & C=2 & C=4 & C=6 & C=2 & C=4 & C=6 & & 0.01\% & 0.10\% & 1.00\% & & 0.01\% & 0.10\% & 1.00\% & 0.01\% & 0.10\% & 1.00\% & & 0.01\% & 0.10\% & 1.00\% & 0.01\% & 0.10\% & 1.00\% \\
\midrule
$t_{4}/\sqrt{2}$ & 200 & 0 & 15.50 & 12.05 & 11.85 & 17.80 & 13.95 & 13.75 & 16.15 & 12.95 & 12.75 & 11.80 & 11.75 & 11.70 & 7.80 & 13.75 & 13.65 & 13.30 & 9.75 & 13.65 & 13.30 & 9.75 & 12.80 & 12.75 & 12.45 & 8.80 & 12.90 & 12.40 & 8.80 \\
 & & 0.5 & 12.80 & 10.20 & 10.15 & 15.10 & 12.55 & 12.50 & 14.00 & 11.60 & 11.55 & 10.15 & 10.15 & 9.80 & 6.45 & 12.50 & 12.50 & 12.05 & 8.80 & 12.50 & 12.05 & 8.80 & 11.50 & 11.50 & 11.05 & 7.85 & 11.50 & 11.00 & 7.60 \\
 & & 0.9 & 5.10 & 4.85 & 4.85 & 8.95 & 8.50 & 8.45 & 8.95 & 8.50 & 8.50 & 4.85 & 4.85 & 4.65 & 3.45 & 8.45 & 8.45 & 8.35 & 6.15 & 8.45 & 8.35 & 6.15 & 8.60 & 8.55 & 8.20 & 6.10 & 8.25 & 8.20 & 5.80 \\
 & 500 & 0 & 22.00 & 15.05 & 14.30 & 26.10 & 18.80 & 18.05 & 22.20 & 16.25 & 15.75 & 14.15 & 14.10 & 14.00 & 9.75 & 17.95 & 17.75 & 17.25 & 12.25 & 17.75 & 17.25 & 12.25 & 15.70 & 15.65 & 15.30 & 11.20 & 15.45 & 15.10 & 10.90 \\
 & & 0.5 & 18.45 & 12.80 & 12.55 & 22.60 & 15.90 & 15.60 & 19.75 & 14.85 & 14.65 & 12.45 & 12.45 & 12.05 & 8.20 & 15.55 & 15.55 & 15.15 & 10.90 & 15.55 & 15.15 & 10.90 & 14.60 & 14.60 & 14.25 & 10.15 & 14.50 & 13.85 & 9.90 \\
 & & 0.9 & 7.30 & 6.05 & 6.00 & 12.65 & 11.25 & 11.20 & 11.90 & 10.70 & 10.70 & 6.00 & 6.00 & 5.90 & 3.85 & 11.20 & 11.20 & 10.80 & 7.65 & 11.20 & 10.80 & 7.65 & 10.75 & 10.75 & 10.45 & 7.30 & 10.60 & 10.45 & 7.00 \\
 & 1000 & 0 & 27.30 & 18.00 & 16.60 & 33.45 & 21.90 & 20.25 & 27.05 & 19.45 & 18.20 & 16.15 & 16.15 & 15.65 & 10.90 & 19.70 & 19.65 & 19.25 & 14.15 & 19.65 & 19.25 & 14.15 & 17.95 & 17.95 & 17.50 & 12.00 & 17.55 & 17.25 & 12.05 \\
 & & 0.5 & 24.55 & 16.85 & 15.80 & 29.35 & 21.40 & 20.00 & 25.70 & 19.00 & 18.05 & 15.40 & 15.40 & 15.20 & 11.35 & 19.80 & 19.75 & 19.50 & 14.50 & 19.75 & 19.50 & 14.50 & 18.00 & 17.95 & 17.45 & 13.10 & 17.95 & 17.80 & 12.95 \\
 & & 0.9 & 9.10 & 6.95 & 6.90 & 16.35 & 13.70 & 13.60 & 15.65 & 13.60 & 13.45 & 6.90 & 6.90 & 6.75 & 4.70 & 13.60 & 13.60 & 13.50 & 9.05 & 13.60 & 13.50 & 9.05 & 13.50 & 13.50 & 13.10 & 8.75 & 13.05 & 12.50 & 8.50 \\
\midrule
$U(-\sqrt{3},\sqrt{3})$& 200 & 0 & 11.85 & 11.85 & 11.85 & 13.85 & 13.80 & 13.80 & 14.00 & 14.00 & 14.00 & 11.85 & 11.65 & 11.50 & 8.20 & 13.80 & 13.80 & 13.60 & 10.35 & 13.80 & 13.60 & 10.35 & 14.20 & 14.20 & 13.70 & 10.30 & 13.50 & 13.35 & 9.85 \\
 & & 0.5 & 10.15 & 9.95 & 9.95 & 11.95 & 11.95 & 11.95 & 12.30 & 12.20 & 12.20 & 9.95 & 9.95 & 9.75 & 7.55 & 11.95 & 11.90 & 11.70 & 9.05 & 11.90 & 11.70 & 9.05 & 12.25 & 12.25 & 11.85 & 9.00 & 12.05 & 11.75 & 9.00 \\
 & & 0.9 & 4.80 & 4.75 & 4.75 & 8.85 & 8.85 & 8.85 & 9.05 & 9.05 & 9.05 & 4.75 & 4.75 & 4.65 & 3.80 & 8.85 & 8.75 & 8.70 & 6.00 & 8.75 & 8.70 & 6.00 & 9.15 & 9.15 & 8.95 & 6.25 & 8.95 & 8.80 & 5.85 \\
 & 500 & 0 & 14.00 & 14.00 & 14.00 & 16.85 & 16.80 & 16.80 & 17.30 & 17.30 & 17.30 & 14.00 & 14.00 & 13.55 & 8.95 & 16.80 & 16.80 & 16.30 & 11.80 & 16.80 & 16.30 & 11.80 & 17.50 & 17.45 & 17.00 & 12.35 & 17.30 & 17.10 & 11.75 \\
 & & 0.5 & 13.40 & 13.40 & 13.40 & 16.45 & 16.40 & 16.40 & 16.50 & 16.40 & 16.40 & 13.40 & 13.40 & 13.15 & 9.40 & 16.40 & 16.35 & 16.20 & 12.15 & 16.35 & 16.20 & 12.15 & 16.65 & 16.55 & 16.25 & 12.15 & 16.40 & 16.00 & 12.30 \\
 & & 0.9 & 5.75 & 5.65 & 5.65 & 11.10 & 11.05 & 11.05 & 11.45 & 11.45 & 11.45 & 5.65 & 5.65 & 5.40 & 3.75 & 11.05 & 11.00 & 10.65 & 7.90 & 11.00 & 10.65 & 7.90 & 11.55 & 11.50 & 11.15 & 7.60 & 10.90 & 10.55 & 7.60 \\
 & 1000 & 0 & 15.75 & 15.75 & 15.75 & 20.15 & 20.05 & 20.05 & 19.85 & 19.85 & 19.85 & 15.75 & 15.65 & 15.25 & 10.60 & 20.05 & 20.00 & 19.50 & 14.15 & 20.00 & 19.50 & 14.15 & 20.40 & 20.30 & 19.75 & 14.55 & 20.65 & 19.85 & 14.75 \\
 & & 0.5 & 14.75 & 14.75 & 14.75 & 18.60 & 18.55 & 18.55 & 18.95 & 18.95 & 18.95 & 14.75 & 14.75 & 14.40 & 10.90 & 18.55 & 18.50 & 18.15 & 13.85 & 18.50 & 18.15 & 13.85 & 19.20 & 19.15 & 18.75 & 14.10 & 19.50 & 18.85 & 13.85 \\
 & & 0.9 & 6.80 & 6.80 & 6.80 & 13.45 & 13.40 & 13.40 & 12.90 & 12.90 & 12.90 & 6.80 & 6.80 & 6.70 & 5.15 & 13.40 & 13.30 & 13.05 & 8.65 & 13.30 & 13.05 & 8.65 & 13.05 & 13.05 & 12.70 & 8.90 & 13.25 & 12.75 & 8.95 \\
\bottomrule
\end{tabular}
}
\caption{Simulation results in Design 14: $\mu_{j}(\theta)=-0.1\cdot 1[j>0.1 p]+0.05\cdot 1[j\leq 0.1 p]$, $\Sigma(\theta)$ Toeplitz.}
\label{tab14}
\end{table}


\begin{table}
\centering
\scalebox{0.70}{
\begin{tabular}{ccccccccccccccc}
\toprule
Density & $p$ & $\rho$ & \multicolumn{3}{c}{Our methods} & \multicolumn{9}{c}{\citetalias{chernozhukov/chetverikov/kato:2014c}'s methods} \\
\cmidrule(lr){7-15}
 & & & \multicolumn{3}{c}{Lasso selection} & \multicolumn{3}{c}{SN selection} & \multicolumn{3}{c}{MB selection} & \multicolumn{3}{c}{EB Selection} \\
\cmidrule(lr){4-6}\cmidrule(lr){7-9}\cmidrule(lr){10-12}\cmidrule(lr){13-15}
 & & & C=2 & C=4 & C=6 & 0.01\% & 0.10\% & 1.00\% & 0.01\% & 0.10\% & 1.00\% & 0.01\% & 0.10\% & 1.00\% \\
\midrule
$t_{4}/\sqrt{2}$ & 200 & 0 & 9.99 & 10.00 & 10.00 & 10.35 & 10.15 & 10.06 & 10.16 & 10.13 & 10.05 & 10.25 & 10.17 & 10.06 \\
 & & 0.5 & 9.99 & 10.00 & 10.00 & 10.26 & 10.11 & 10.04 & 10.12 & 10.10 & 10.04 & 10.17 & 10.11 & 10.04 \\
 & & 0.9 & 10.00 & 10.00 & 10.01 & 10.06 & 10.02 & 10.01 & 10.02 & 10.02 & 10.01 & 10.03 & 10.02 & 10.01 \\
 & 500 & 0 & 9.97 & 10.00 & 10.00 & 10.49 & 10.21 & 10.10 & 10.23 & 10.19 & 10.09 & 10.37 & 10.25 & 10.10 \\
 & & 0.5 & 9.98 & 10.00 & 10.00 & 10.37 & 10.17 & 10.08 & 10.18 & 10.14 & 10.07 & 10.27 & 10.18 & 10.07 \\
 & & 0.9 & 9.99 & 10.00 & 10.00 & 10.11 & 10.05 & 10.02 & 10.04 & 10.04 & 10.02 & 10.06 & 10.04 & 10.02 \\
 & 1000 & 0 & 9.94 & 10.00 & 10.00 & 10.64 & 10.27 & 10.12 & 10.29 & 10.23 & 10.10 & 10.49 & 10.32 & 10.12 \\
 & & 0.5 & 9.95 & 10.00 & 10.00 & 10.49 & 10.20 & 10.09 & 10.22 & 10.17 & 10.08 & 10.35 & 10.23 & 10.09 \\
 & & 0.9 & 9.99 & 10.00 & 10.00 & 10.14 & 10.05 & 10.02 & 10.05 & 10.04 & 10.02 & 10.07 & 10.05 & 10.02 \\
\midrule
$U(-\sqrt{3},\sqrt{3})$ & 200 & 0 & 2.00 & 2.00 & 2.00 & 2.00 & 2.00 & 2.00 & 2.00 & 2.00 & 2.00 & 2.00 & 2.00 & 2.00 \\
 & & 0.5 & 2.00 & 2.00 & 2.00 & 2.00 & 2.00 & 2.00 & 2.00 & 2.00 & 2.00 & 2.00 & 2.00 & 2.00 \\
 & & 0.9 & 2.00 & 2.00 & 2.01 & 2.00 & 2.00 & 2.00 & 2.00 & 2.00 & 2.00 & 2.00 & 2.00 & 2.00 \\
 & 500 & 0 & 5.00 & 5.00 & 5.01 & 5.00 & 5.00 & 5.00 & 5.00 & 5.00 & 5.00 & 5.00 & 5.00 & 5.00 \\
 & & 0.5 & 5.00 & 5.00 & 5.01 & 5.00 & 5.00 & 5.00 & 5.00 & 5.00 & 5.00 & 5.00 & 5.00 & 5.00 \\
 & & 0.9 & 5.00 & 5.00 & 5.01 & 5.00 & 5.00 & 5.00 & 5.00 & 5.00 & 5.00 & 5.00 & 5.00 & 5.00 \\
 & 1000 & 0 & 10.00 & 10.00 & 10.02 & 10.00 & 10.00 & 10.00 & 10.00 & 10.00 & 10.00 & 10.00 & 10.00 & 10.00 \\
 & & 0.5 & 10.00 & 10.00 & 10.01 & 10.00 & 10.00 & 10.00 & 10.00 & 10.00 & 10.00 & 10.00 & 10.00 & 10.00 \\
 & & 0.9 & 10.00 & 10.00 & 10.02 & 10.00 & 10.00 & 10.00 & 10.00 & 10.00 & 10.00 & 10.00 & 10.00 & 10.00 \\
\bottomrule
\end{tabular}
}
\caption{Percentage of moment inequalities retained by first step selection procedures in Design 8: $\mu_{j}(\theta)=-0.75\cdot 1[j>0.1 p]+0.05\cdot 1[j\leq 0.1 p]$, $\Sigma(\theta)$ Toeplitz.}
\label{tab15}
\end{table}

\begin{table}
\centering
\scalebox{0.70}{
\begin{tabular}{ccccccccccccccc}
\toprule
Density & $p$ & $\rho$ & \multicolumn{3}{c}{Our methods} & \multicolumn{9}{c}{\citetalias{chernozhukov/chetverikov/kato:2014c}'s methods} \\
\cmidrule(lr){7-15}
 & & & \multicolumn{3}{c}{Lasso selection} & \multicolumn{3}{c}{SN selection} & \multicolumn{3}{c}{MB selection} & \multicolumn{3}{c}{EB Selection} \\
\cmidrule(lr){4-6}\cmidrule(lr){7-9}\cmidrule(lr){10-12}\cmidrule(lr){13-15}
 & & & C=2 & C=4 & C=6 & 0.01\% & 0.10\% & 1.00\% & 0.01\% & 0.10\% & 1.00\% & 0.01\% & 0.10\% & 1.00\% \\
\midrule
$t_{4}/\sqrt{2}$ & 200 & 0 & 9.99 & 12.01 & 46.98 & 95.88 & 81.21 & 49.76 & 78.87 & 74.71 & 44.79 & 82.96 & 79.09 & 47.60 \\
 & & 0.5 & 9.99 & 12.93 & 55.22 & 96.32 & 82.13 & 50.27 & 79.80 & 75.76 & 45.03 & 82.90 & 78.99 & 47.15 \\
 & & 0.9 & 10.00 & 26.27 & 86.88 & 97.22 & 84.36 & 51.58 & 78.14 & 73.37 & 37.67 & 78.91 & 74.25 & 38.46 \\
 & 500 & 0 & 9.97 & 10.41 & 25.12 & 98.06 & 89.18 & 63.79 & 86.63 & 83.75 & 57.92 & 90.44 & 87.96 & 61.93 \\
 & & 0.5 & 9.98 & 10.62 & 31.38 & 98.33 & 89.97 & 64.65 & 87.31 & 84.48 & 58.51 & 90.38 & 87.87 & 61.67 \\
 & & 0.9 & 10.00 & 16.18 & 70.57 & 98.89 & 91.81 & 66.98 & 86.58 & 83.40 & 52.79 & 88.21 & 85.11 & 53.90 \\
 & 1000 & 0 & 9.95 & 10.14 & 16.32 & 98.94 & 93.19 & 73.30 & 90.72 & 88.60 & 67.23 & 94.05 & 92.39 & 71.51 \\
 & & 0.5 & 9.96 & 10.20 & 19.76 & 99.11 & 93.79 & 74.24 & 91.45 & 89.38 & 67.98 & 94.05 & 92.41 & 71.49 \\
 & & 0.9 & 9.99 & 12.54 & 55.08 & 99.42 & 95.12 & 76.52 & 91.05 & 88.81 & 63.52 & 92.20 & 90.07 & 65.04 \\
\midrule
$U(-\sqrt{3},\sqrt{3})$ & 200 & 0 & 2.00 & 17.04 & 20.00 & 19.60 & 17.27 & 10.54 & 16.73 & 15.95 & 9.38 & 16.66 & 15.88 & 9.33 \\
 & & 0.5 & 2.00 & 16.40 & 20.00 & 19.58 & 17.22 & 10.49 & 16.71 & 15.95 & 9.32 & 16.63 & 15.84 & 9.20 \\
 & & 0.9 & 2.00 & 15.35 & 20.00 & 19.55 & 17.10 & 10.43 & 15.83 & 14.88 & 7.57 & 15.68 & 14.75 & 7.49 \\
 & 500 & 0 & 5.01 & 41.99 & 50.00 & 49.62 & 46.61 & 34.35 & 45.25 & 44.09 & 30.97 & 45.20 & 43.98 & 30.77 \\
 & & 0.5 & 5.01 & 40.28 & 50.00 & 49.59 & 46.55 & 34.26 & 45.21 & 43.99 & 30.85 & 45.21 & 43.96 & 30.72 \\
 & & 0.9 & 5.00 & 37.28 & 50.00 & 49.56 & 46.40 & 34.17 & 43.90 & 42.36 & 26.88 & 43.84 & 42.26 & 26.82 \\
 & 1000 & 0 & 10.02 & 83.05 & 100.00 & 99.63 & 96.22 & 78.71 & 94.10 & 92.53 & 72.32 & 93.88 & 92.26 & 71.79 \\
 & & 0.5 & 10.01 & 79.39 & 100.00 & 99.62 & 96.12 & 78.53 & 94.00 & 92.39 & 71.98 & 93.82 & 92.14 & 71.51 \\
 & & 0.9 & 10.01 & 72.46 & 100.00 & 99.58 & 95.92 & 78.06 & 91.86 & 89.74 & 64.80 & 91.92 & 89.76 & 64.87 \\
\bottomrule
\end{tabular}
}
\caption{Percentage of moment inequalities retained by first step selection procedures in Design 11: $\mu_{j}(\theta)=-0.4\cdot 1[j>0.1 p]+0.05\cdot 1[j\leq 0.1 p]$, $\Sigma(\theta)$ Toeplitz.}
\label{tab16}
\end{table}

\end{landscape}

}

\newpage

\appendix

%%%%%%%%%%%%%%%%%%% -- DIVIDER - %%%%%%%%%%%%%%%%%%%

\section{Appendix}
\begin{small}

Throughout the appendix, we omit the dependence of all expressions on $\theta$. Furthermore, LHS and RHS abbreviate ``left hand side'' and ``right hand side'', respectively.

\subsection{Auxiliary results}

%%%%%%%%%%%%%%%%%%% -- DIVIDER - %%%%%%%%%%%%%%%%%%%

\begin{lemma} \label{lem:SampleBound}
Assume Assumptions \ref{ass:Basic}-\ref{ass:Rates}. Then, for any $\gamma $ s.t.\ $\sqrt{n}\gamma /\sqrt{1+\gamma ^{2}}\in [ 0,n^{\delta /(2(2+\delta ))}M_{n,2+\delta }^{-1}]$,
\begin{equation}
P\del[2]{\max_{j=1,\dots ,p}\vert \hat{\mu}_{j}-\mu _{j}\vert / \hat{\sigma} _{j}>\gamma} \leq 2p(1-\Phi (\sqrt{n}\gamma /\sqrt{ 1+\gamma ^{2}}))[ 1+Kn^{-\delta /2}M_{n,2+\delta }^{2+\delta }( 1+ \sqrt{n}\gamma /\sqrt{ 1+\gamma ^{2}}) ^{2+\delta }] , \label{eq:Lemma1Eq1}
\end{equation}
where $K$ is a universal constant.
\end{lemma}
%%%%%%%%%%%%%%%%%%% -- DIVIDER - %%%%%%%%%%%%%%%%%%%
\begin{proof}
	
For any $i=1,\dots ,n$ and $j=1,\dots ,p$, let $Z_{ij}\equiv ( X_{ij}-\mu _{j})/\sigma _{j}$ and $U_{j}\equiv \sqrt{n} \sum_{i=1}^{n} (Z_{ij}/n)/\sqrt{ \sum_{i=1}^{n} (Z_{ij}^{2}/n)}$. We divide the rest of the proof into three steps.

\underline{Step 1.} By definition, $\sqrt{n}(\hat{\mu}_{j}-\mu _{j})/\hat{\sigma}_{j}=U_{j}/\sqrt{ 1-U_{j}^{2}/n}$ and so
\begin{equation}
	\sqrt{n}|\hat{\mu}_{j}-\mu _{j}|/\hat{\sigma}_{j}=|U_{j}|/\sqrt{ 1-|U_{j}|^{2}/n}.\label{eq:reexpressionU}
\end{equation}
Since the RHS of Eq.\ \eqref{eq:reexpressionU} is increasing in $|U_{j}|$, it follows that:
\begin{eqnarray}
\cbr[2]{\max_{j=1,\dots ,p}|\hat{\mu}_{j}-\mu _{j}|/\hat{\sigma}_{j}>\gamma } =\cbr[2]{\max_{1\leq j\leq p}|U_{j}|/\sqrt{1-|U_{j}|^{2}/n}>\sqrt{n}\gamma } 
\subseteq \cbr[2]{\max_{1\leq j\leq p}|U_{j}|\geq \sqrt{n}\gamma /\sqrt{1+\gamma ^{2}}}.
\label{eq:SNbound0}
\end{eqnarray}

\underline{Step 2.} For every $j=1,\dots ,p,$ $\{Z_{ij}\}_{i=1}^{n}$ is a sequence of independent random variables with $E[Z_{ij}]=0$, $E[Z_{ij}^{2}]=1$, and $ E[|Z_{ij}|^{2+\delta }]\leq M_{n,2+\delta }^{2+\delta }<\infty $. If we let $ S_{nj}=\sum_{i=1}^{n}Z_{ij}$, $V_{nj}^{2}=\sum_{i=1}^{n}Z_{ij}^{2}$, and $ 0<D_{nj}=[n^{-1}\sum_{i=1}^{n}E[|Z_{ij}|^{2+\delta }]]^{1/(2+\delta )}\leq M_{n,2+\delta }<\infty $, then \citetalias{chernozhukov/chetverikov/kato:2014c} (Lemma A.1) implies that for all $t\in [ 0,n^{\delta /(2(2+\delta ))}D_{nj}^{-1}]$,
\begin{equation}
	\envert[2]{\frac{P(S_{nj}/V_{nj}\geq t)}{1-\Phi (t)}-1}\leq Kn^{-\delta /2}D_{nj}^{2+\delta }(1+t)^{2+\delta },
	\label{eq:SNbound1}
\end{equation}
where $K$ is a universal constant.

By using that $U_{j} = S_{nj}/V_{nj}$, $D_{nj}\leq M_{n,2+\delta }$, and applying Eq.\ \eqref{eq:SNbound1} to $t=\sqrt{n}\gamma /\sqrt{1+\gamma ^{2}}$, it follows that for any $\gamma $ s.t.\ $\sqrt{n}\gamma /\sqrt{1+\gamma ^{2}}\in [ 0,n^{\delta /(2(2+\delta ))}M_{n,2+\delta }^{-1}]$,
\begin{eqnarray*}
\envert[1]{P\del[2]{U_{j}\geq \sqrt{n}\gamma /\sqrt{1+\gamma ^{2}}}-(1-\Phi (\sqrt{n}\gamma / \sqrt{1+\gamma ^{2}}))} \leq 
Kn^{-\delta /2}D_{nj}^{2+\delta }\del[1]{1-\Phi (\sqrt{ n }\gamma /\sqrt{1+\gamma ^{2}})}\del[1]{1+\sqrt{n}\gamma /\sqrt{1+\gamma ^{2}} }^{2+\delta }.
\end{eqnarray*}
Thus, for any $\gamma $ s.t.\ $\sqrt{n}\gamma /\sqrt{1+\gamma ^{2}}\in [ 0,n^{\delta /(2(2+\delta ))}M_{n,2+\delta }^{-1}]$,
\begin{eqnarray}
\sum_{j=1}^{p}P\del[2]{U_{j}\geq \sqrt{n}\gamma /\sqrt{1+\gamma ^{2}}}\leq
p\del[1]{1-\Phi (\sqrt{n}\gamma /\sqrt{1+\gamma ^{2}})}\sbr[1]{1+Kn^{-\delta /2}M_{n,2+\delta }^{2+\delta }(1+\sqrt{n}\gamma /\sqrt{1+\gamma ^{2}})^{2+\delta }}. \label{eq:SNbound2}
\end{eqnarray}
By applying the same argument for $-Z_{ij}$ instead of $Z_{ij}$, it follows that for any $\gamma $ s.t.\ $\sqrt{n}\gamma /\sqrt{1+\gamma ^{2}}\in [ 0,n^{\delta /(2(2+\delta ))}M_{n,2+\delta }^{-1}]$,
\begin{eqnarray}
\sum_{j=1}^{p}P\del[2]{-U_{j}\geq \sqrt{n}\gamma /\sqrt{1+\gamma ^{2}}}\leq 
p\del[1]{1-\Phi (\sqrt{n}\gamma /\sqrt{1+\gamma ^{2}})}\sbr[1]{1+Kn^{-\delta /2}M_{n,2+\delta }^{2+\delta }(1+\sqrt{n}\gamma /\sqrt{1+\gamma ^{2}})^{2+\delta }}. \label{eq:SNbound3}
\end{eqnarray}

\underline{Step 3.} Consider the following argument.
\begin{eqnarray*}
P\del[2]{\max_{j=1,\dots ,p}|\hat{\mu}_{j}-\mu _{j}|/\hat{\sigma}_{j}>\gamma }
&\leq &P\del[2]{\max_{1\leq j\leq p}|U_{j}|\geq \sqrt{n}\gamma /\sqrt{1+\gamma ^{2}} } \\
&\leq &\sum_{j=1}^{p}P\del[2]{|U_{j}|\geq \sqrt{n}\gamma /\sqrt{1+\gamma ^{2}}} \\
&\leq &\sum_{j=1}^{p}P\del[2]{U_{j}\geq \sqrt{n}\gamma /\sqrt{1+\gamma ^{2}} }+\sum_{j=1}^{p}P\sbr[2]{-U_{j}\geq \sqrt{n}\gamma /\sqrt{1+\gamma ^{2}}} \\
&\leq &2p\del[2]{1-\Phi (\sqrt{n}\gamma /\sqrt{1+\gamma ^{2}})}\sbr[2]{1+Kn^{-\delta /2}M_{n,2+\delta }^{2+\delta }\del[1]{1+\sqrt{n}\gamma /\sqrt{1+\gamma ^{2}} }^{2+\delta }},
\end{eqnarray*}
where the first inequality follows from Eq.\ \eqref{eq:SNbound0}, the second inequality follows from Bonferroni bound, and the fourth inequality follows from Eqs.\ \eqref{eq:SNbound2} and \eqref{eq:SNbound3}.
\end{proof}

%%%%%%%%%%%%%%%%%%% -- DIVIDER - %%%%%%%%%%%%%%%%%%%

\begin{lemma} \label{lem:SampleBound2}
Assume Assumptions \ref{ass:Basic}-\ref{ass:Rates} and let $\{\gamma _{n}\}_{n\geq 1}\subseteq \mathbb{R}$ satisfy $\gamma _{n}\geq \gamma _{n}^{\ast }$ for all $n$ sufficiently large, where
\begin{eqnarray}
\gamma _{n}^{\ast } \equiv  n^{-1/2}( M_{n,2+\delta }^{2}n^{-\delta /(2+\delta )}-n^{-1}) ^{-1/2} =( nM_{n,2+\delta }^{2+\delta }) ^{-1/(2+\delta )}( 1-( nM_{n,2+\delta }^{2+\delta }) ^{-2/(2+\delta )}) ^{-1/2}\to 0. \label{Eq:LambdaStar}
\end{eqnarray}
Then,
\begin{eqnarray}
P\del[2]{ \max_{j=1,\dots ,p}\vert \hat{\mu}_{j}-\mu _{j}\vert / \hat{\sigma} _{j}>\gamma _{n}} \leq 2p\exp \del[1]{ -2^{-1}n^{\delta /(2+\delta )}/M_{n,2+\delta }^{2}} \sbr[1]{ 1+K( M_{n,2+\delta }/n^{\delta /(2(2+\delta ))}+1) ^{2+\delta }} \to 0. \label{eq:SampleBoundEq}
\end{eqnarray}
\end{lemma}
%%%%%%%%%%%%%%%%%%% -- DIVIDER - %%%%%%%%%%%%%%%%%%%
\begin{proof}
First, note that the convergence to zero in Eq.\ \eqref{Eq:LambdaStar} follows from $ nM_{n,2+\delta }^{2+\delta }\to \infty $. Since $\gamma _{n}\geq \gamma _{n}^{\ast }$, Eq. \eqref{eq:SampleBoundEq} holds if we show:
\begin{eqnarray}
P\del[2]{ \max_{j=1,\dots ,p}\vert \hat{\mu}_{j}-\mu _{j}\vert / \hat{\sigma} _{j}>\gamma _{n}^*} \leq 2p\exp \del[1]{ -2^{-1}n^{\delta /(2+\delta )}/M_{n,2+\delta }^{2}} \sbr[1]{ 1+K( M_{n,2+\delta }/n^{\delta /(2(2+\delta ))}+1) ^{2+\delta }} \to 0. \label{eq:SampleBoundEq2}
\end{eqnarray}
As we show next, Eq.\ \eqref{eq:SampleBoundEq2} follows from using Lemma \ref{lem:SampleBound} with $\gamma =\gamma _{n}^{\ast }$. This choice of $\gamma$ implies $\sqrt{n}\gamma _{n}^{\ast }/\sqrt{1+(\gamma _{n}^{\ast })^{2} }=n^{\delta /(2(2+\delta ))}M_{n,2+\delta }^{-1}$ making $\gamma =\gamma_n^*$ a valid choice in Lemma \ref{lem:SampleBound}. Then, Lemma \ref{lem:SampleBound} with $\gamma =\gamma _{n}^{\ast }$ implies that:
\begin{eqnarray*}
 P\del[2]{\max_{j=1,\dots ,p}\vert \hat{\mu}_{j}-\mu _{j}\vert / \hat{\sigma} _{j}>\gamma _{n}^{\ast }}
 &\leq& 
2p\del[1]{1-\Phi (n^{\delta /(2(2+\delta ))}M_{n,2+\delta }^{-1})}\sbr[1]{ 1+Kn^{-\delta /2}M_{n,2+\delta }^{2+\delta }( 1+n^{\delta /(2(2+\delta ))}M_{n,2+\delta }^{-1}) ^{2+\delta }} \\
& \leq& 2p\exp \del[1]{-2^{-1}n^{\delta /(2+\delta )}/M_{n,2+\delta }^{2}}\sbr[1]{ 1+K( n^{-\delta /(2(2+\delta ))}M_{n,2+\delta }+1) ^{2+\delta } },
\end{eqnarray*}
where we have used that $1-\Phi(t)\leq e^{-t^2/2}$. We now show that the RHS of the above display converges to zero by Assumption \ref{ass:Rates}. First, notice that $ M_{n,2+\delta }^{(2+\delta )}(\ln ( 2k-p) )^{(2+\delta )/2}n^{-\delta /2}\to 0$. Next, $( 2k-p) >1$ implies that $M_{n,2+\delta }^{(2+\delta )}n^{-\delta /2}\to 0$ and, in turn, this implies that $n^{-\delta /(2(2+\delta ))}M_{n,2+\delta }\to 0$. Furthermore, notice that $M_{n,2+\delta }^{(2+\delta )}(\ln ( 2k-p) )^{(2+\delta )/2}n^{-\delta /2}\to 0$, $M_{n,2+\delta }^{(2+\delta )}(\ln ( 2k-p) )^{(2+\delta )/2}n^{-\delta /2}>0$, and $( 2k-p) \geq p$ implies that $n^{\delta /(2+\delta )}(M_{n,2+\delta }^{2}\ln p)^{-1}\to \infty $. This implies that:
\[
p\exp \del[1]{-2^{-1}n^{\delta /(2+\delta )}/M_{n,2+\delta }^{2}]}=\exp \del[2]{\ln p\sbr[1]{1-2^{-1}[n^{\delta /(2+\delta )}(M_{n,2+\delta }^{2}\ln p)^{-1}]}} \to 0,
\]
completing the proof.
\end{proof}

%%%%%%%%%%%%%%%%%%% -- DIVIDER - %%%%%%%%%%%%%%%%%%%

% \begin{lemma}\label{lem:Sigma}
%  For every $c\in ( 0,1)$,
%  \[
%  P[ \max_{j=1,\dots ,p}\vert \hat{\sigma}_{j}/\sigma _{j}-1\vert > K( n^{-( 1-c) /2}B_{n}^{2}\ln p+n^{-3/2}B_{n}^{2}( \ln p) ^{2}) ] ~\leq~ \tilde{K} n^{-c},
%  \]
%  where $B_{n}\equiv E[\max_{j=1,\dots ,p}Z_{1,j}^{4}]^{1/4}$ and $K,\tilde{K}$ are universal constants.
%  \end{lemma}
%  %%%%%%%%%%%%%%%%%%% -- DIVIDER - %%%%%%%%%%%%%%%%%%%
%  \begin{proof}
%  This result is shown in \citet[Lemma A.5]{chernozhukov/chetverikov/kato:2014c}.
%  \end{proof}


%%%%%%%%%%%%%%%%%%% -- DIVIDER - %%%%%%%%%%%%%%%%%%%

\begin{proof}[Proof of Lemma \ref{lem:LassoNoOverFit}]
 By definition, $J\subseteq J_{I}$ where $J_{I}$ is as defined in the proof of Theorem \ref{thm:Lasso2SSize}. Then, the result is a corollary of Step 2 in the proof of Theorem \ref{thm:Lasso2SSize}.
\end{proof}


%%%%%%%%%%%%%%%%%%% -- DIVIDER - %%%%%%%%%%%%%%%%%%%
\begin{proof}[Proof of Lemma \ref{lem:LassoClosedForm}]
	Fix $j=1,\dots ,p$ arbitrarily. \citet[Eq.\ (2.5)]{buhlmann/vandegeer:2011} implies that the Lasso estimator in Eq.\ \eqref{eq:Lasso1} satisfies:
	\begin{align}
	\hat{\mu}_{L,j} = \sgn(\hat{\mu}_{j}) \times \max \{|\hat{\mu}_j|-\hat{\sigma}_j{\lambda_n}/{2}, 0\}~~~\forall j=1,\dots,p. \label{eq:ClosedForm1}
	\end{align}
	
	To complete the proof, it suffices to show that:
	\begin{equation}
	\{\hat{\mu}_{L,j}\geq -\hat{\sigma}_{j}\lambda _{n}\}~~=~~ \{\hat{\mu }_{j}\geq -3\hat{\sigma}_{j}\lambda _{n}/2\}. \label{eq:ClosedForm2}
	\end{equation}
	We divide the verification into four cases. First, consider that $\hat{\sigma }_{j}=0$. If so, $-\hat{\sigma}_{j}\lambda _{n}=-3\hat{\sigma} _{j}\lambda _{n}/2=0$ and $\hat{\mu}_{L,j}=sign( \hat{\mu} _{j}) \times \max \{ \vert \hat{\mu}_{j}\vert ,0\} =\hat{\mu}_{j}$, and so Eq.\ \eqref{eq:ClosedForm2} holds. Second, consider that $\hat{\sigma}_{j}>0$ and $\hat{\mu}_{j}\geq 0$. If so, $\hat{\mu}_{j}\geq 0\geq -3\hat{\sigma}_{j}\lambda _{n}/2$ and so the RHS condition in Eq.\ \eqref{eq:ClosedForm2} is satisfied. In addition, Eq.\ \eqref{eq:ClosedForm1} implies that $\hat{\mu}_{L,j}\geq 0\geq -\hat{\sigma} _{j}\lambda _{n}$ and so the LHS of condition in Eq.\ \eqref{eq:ClosedForm2} is also satisfied. Thus, Eq.\ \eqref{eq:ClosedForm2} holds. Third, consider that $\hat{\sigma}_{j}>0$ and $\hat{\mu}_{j}\in [ -\hat{\sigma} _{j}\lambda _{n}/2,0)$. If so, $\hat{\mu}_{j}\geq - \hat{\sigma}_{j}\lambda _{n}/2\geq -3\hat{\sigma}_{j}\lambda _{n}/2$ and so the RHS condition in Eq.\ \eqref{eq:ClosedForm2} is satisfied. In addition, Eq.\ \eqref{eq:ClosedForm1} implies that $\hat{\mu}_{L,j}=0\geq -\hat{\sigma}_{j}\lambda _{n}$ and so the LHS of condition in Eq.\ \eqref{eq:ClosedForm2} is also satisfied. Thus, Eq.\ \eqref{eq:ClosedForm2} holds. Fourth and finally, consider that $\hat{\sigma}_{j}>0$ and $\hat{\mu}_{j}<-\hat{\sigma}_{j}\lambda _{n}/2$. Then, Eq.\ \eqref{eq:ClosedForm1} implies that $\hat{\mu}_{L,j}=\hat{\mu}_{j}+\hat{\sigma} _{j}\lambda _{n}/2$ and so Eq.\ \eqref{eq:ClosedForm2} holds.
\end{proof}


\subsection{Results for the self-normalization approximation}

%%%%%%%%%%%%%%%%%%% -- DIVIDER - %%%%%%%%%%%%%%%%%%%
\begin{lemma} \label{lem:CSNincreasing}
For any $\pi \in (0,0.5]$, $n\in \mathbb{N}$, and $d\in \{0,1\dots ,2k-p\}$, define the function:
\[
CV(d)\equiv \left\{
\begin{array}{ll}
0 & \text{if }d=0 ,\\
\frac{\Phi ^{-1}( 1-\pi /d) }{\sqrt{1-( \Phi ^{-1}( 1-\pi /d) ) ^{2}/n}} & \text{if }d>0.
\end{array}
\right.
\]
Then, $CV:\{0,1\dots, 2k-p\}\to \mathbb{R}_{+}$ is weakly increasing for $n$ sufficiently large.
\end{lemma}
%%%%%%%%%%%%%%%%%%% -- DIVIDER - %%%%%%%%%%%%%%%%%%%
\begin{proof}
First, we show that $CV(d)\leq CV(d+1)$ for $d=0$. To see this, use that $\pi \leq 0.5$ such that $\Phi ^{-1}( 1-\pi) \geq 0$, implying that $CV(1)\geq 0=CV(0)$.

Second, we show that $CV(d)\leq CV(d+1)$ for any $d>0$. To see this, notice that $CV(d)$ and $CV(d+1)$ are both the result of the composition $g_{1}(g_{2}(\cdot )):\{ 1\dots ,2k-p\} \to \mathbb{R}$ where:
\begin{eqnarray*}
	g_{1}(y) &\equiv&y/\sqrt{1-y^{2}/n}:[0,\sqrt{n})\to \mathbb{R}_{+} \\
	g_{2}(d) &\equiv&\Phi ^{-1}( 1-{\pi }/{d}) :\{ 1\dots ,2k-p\} \to \mathbb{R}.
\end{eqnarray*}
We first show that $g_{1}(g_{2}(\cdot ))$ is properly defined by verifying that the range of $g_{2}$ is included in support of $g_{1}$. Notice that $g_{2}$ is an increasing function and so $ g_{2}(d)\in \lbrack g_{2}(1),g_{2}(2(k-p)+p)]=[\Phi ^{-1}( 1-\pi ) ,\Phi ^{-1}( 1-\pi /(2k-p)) ]$. For the lower bound, $\pi \leq 0.5$ implies that $\Phi ^{-1}( 1-\pi ) \geq 0$. For the upper bound, consider the following argument. On the one hand, $( 1-\Phi ( \sqrt{n} )) \leq \exp ( -n/2)/2$ holds for all $n$ large enough. On the other hand, Assumption \ref{ass:Rates} implies that $\exp(-n/2)/2\leq {\pi}/(2k-p)$. By combining these two, we conclude that $\Phi ^{-1}( 1-\pi /(2k-p)) \leq \sqrt{n}$ for all $n$ large enough, as desired. From here, the monotonicity of $CV(d)$ follows from the fact that $g_{1}$ and $ g_{2}$ are both weakly increasing functions and so $CV(d)=g_{1}(g_{2}(d)) \leq g_{1}(g_{2}(d+1))=CV(d+1)$.
\end{proof}

%%%%%%%%%%%%%%%%%%% -- DIVIDER - %%%%%%%%%%%%%%%%%%%
\begin{lemma}\label{lem:SNSize}
Assume Assumptions \ref{ass:Basic}-\ref{ass:Rates}, $\alpha \in (0,0.5)$, and that $H_{0}$ holds. For any non-stochastic set $L\subseteq \{1,\dots ,p\}$, define:
\begin{eqnarray*}
	T_{n}(L) &\equiv &\max \left\{ \max_{j\in L}{\sqrt{n}\hat{\mu}_{j}}/{\hat{ \sigma}_{j}},\max_{s=p+1,\dots ,k}{\sqrt{n}\vert \hat{\mu} _{s}\vert }/{\hat{\sigma}_{s}} \right\} \\
c_{n}^{SN}(|L|,\alpha) &\equiv &\tfrac{\Phi ^{-1}( 1-\alpha /(2(k-p)+|L|)) }{\sqrt{1-( \Phi ^{-1}( 1-\alpha /(2(k-p)+|L|)) ) ^{2}/n}}.
\end{eqnarray*}
Then,
\begin{equation*}
	P\del[1]{T_{n}(L)>c_{n}^{SN}(|L|,\alpha)} \leq \alpha +R_{n},
\end{equation*}
where $R_{n}\equiv \alpha Kn^{-\delta /2}M_{n,2+\delta }^{2+\delta }(1+\Phi ^{-1}( 1-\alpha /(2k-p)) )^{2+\delta }\to 0$ and $K$ is a universal constant.
\end{lemma}
%%%%%%%%%%%%%%%%%%% -- DIVIDER - %%%%%%%%%%%%%%%%%%%
\begin{proof}
	% If there were no equalities, then the result would follow from replacing $\{1,\dots ,p\}$ with $L$ in the proof of \citetalias{chernozhukov/chetverikov/kato:2014c} (Theorem 4.1).
	Under $H_{0}$, $\sqrt{n}\hat{\mu}_{j}/\hat{\sigma}_{j}\leq \sqrt{n}( \hat{\mu}_{j}-\mu _{j}) /\hat{\sigma}_{j}$ for all $j\in L$ and $\sqrt{n} \vert \hat{\mu}_{s}\vert /\hat{\sigma}_{s}=\sqrt{n}\vert \hat{\mu}_{s}-\mu _{s}\vert /\hat{\sigma}_{s}$ for $s=p+1,\dots ,k$. From this, we deduce that:
\begin{eqnarray*}
T_{n}(L) &=&\max \cbr[2]{ \max_{j\in L}{\sqrt{n}\hat{\mu}_{j}}/{\hat{ \sigma}_{j}},\max_{s=p+1,\dots ,k}{\sqrt{n}\vert \hat{\mu} _{s}\vert }/{\hat{\sigma}_{s}}} \\
&\leq &\max \cbr[2]{ \max_{j\in L}{\sqrt{n}( \hat{\mu}_{j}-\mu _{j}) }/{\hat{\sigma}_{j}},\max_{s=p+1,\dots ,k}{\sqrt{n} \vert \hat{\mu}_{s}-\mu _{s}\vert }/{\hat{\sigma}_{s}}} =T_{n}^{\ast }(L).
\end{eqnarray*}
For any $i=1,\dots ,n$ and $j=1,\dots ,k$, let $Z_{ij}\equiv (X_{ij}-\mu _{j})/\sigma _{j}$ and $U_{j}\equiv \sqrt{n}
\sum_{i=1}^{n} (Z_{ij}/n)/\sqrt{\sum_{i=1}^{n} (Z_{ij}^2/n)}$. It then follows that $\sqrt{n}[\hat{\mu}_{j}-\mu _{j}]/\hat{\sigma}_{j}=U_{j}/\sqrt{ 1-U_{j}^{2}/n}$ and so,
\begin{eqnarray*}
\sqrt{n}(\hat{\mu}_{j}-\mu _{j})/\hat{\sigma}_{j} &=&U_{j}/\sqrt{ 1-|U_{j}|^{2}/n} \\
\sqrt{n}|\hat{\mu}_{j}-\mu _{j}|/\hat{\sigma}_{j} &=&|U_{j}|/\sqrt{ 1-|U_{j}|^{2}/n}.
\end{eqnarray*}
Notice that the expressions on the RHS are increasing in $U_{j}$ and $ |U_{j}|$, respectively. Therefore, for any $c\geq 0 $,
\begin{eqnarray*}
\{ T_{n}^{\ast }(L)>c\} &=&\cbr[2]{ \max_{j\in L}{\sqrt{n} ( \hat{\mu}_{j}-\mu _{j}) }/{\hat{\sigma}_{j}}>c} \cup \cbr[2]{ \max_{s=p+1,\dots ,k}{\sqrt{n}\vert \hat{\mu}_{s}-\mu _{s}\vert }/{\hat{\sigma}_{s}}>c} \\
&=&\cbr[2]{ \max_{j\in L}U_{j}/\sqrt{1-|U_{j}|^{2}/n}>c} \cup \cbr[2]{ \max_{s=p+1,\dots ,k}|U_{s}|/\sqrt{1-|U_{s}|^{2}/n}>c} \\
&=&\cbr[2]{ \max_{j\in L}U_{j}>c/\sqrt{1+c^{2}/n}} \cup \cbr[2]{ \max_{s=p+1,\dots ,k}|U_{s}|>c/\sqrt{1+c^{2}/n}} .
\end{eqnarray*}
From here, we conclude that for all $c\geq 0 $ such that:
 $c/\sqrt{1+c^{2}/n}\in [0,n^{\delta /(2(2+\delta ))}M_{n,2+\delta }^{-1}]$,
\begin{eqnarray}
P(T_{n}(L)>c) &\leq& P(T_{n}^{\ast }(L)>c) \notag \\
&\leq& P\del[3]{ \cbr[2]{ \max_{j\in L}U_{j}>c/\sqrt{1+c^{2}/n}} \cup \cbr[2]{ \max_{s=p+1,\dots ,k}|U_{s}|>c/\sqrt{1+c^{2}/n}} }\notag  \\
&\leq& \sum_{j\in L}P\del[2]{ U_{j}>c/\sqrt{1+c^{2}/n}} +\sum_{s=p+1}^{k}P\del[2]{ |U_{s}|>c/\sqrt{1+c^{2}/n}} \notag \\
&\leq& \sum_{j\in L}P\del[2]{ U_{j}>c/\sqrt{1+c^{2}/n}} +\sum_{s=p+1}^{k}P\del[2]{U_{s}>c/\sqrt{1+c^{2}/n}}+\sum_{g=p+1}^{k}P\del[2]{-U_{g}>c/ \sqrt{1+c^{2}/n}} \notag \\
&\leq& (2(k-p)+|L|) \del[2]{1-\Phi (c/\sqrt{1+c^{2}/n})} \sbr[2]{1+Kn^{-\delta /2}M_{n,2+\delta }^{2+\delta }(1+c/\sqrt{1+c^{2}/n})^{2+\delta }} \label{eq:KeyBound},
\end{eqnarray}
where the first inequality follows from $T_{n}(L)\leq T_{n}^{\ast }(L)$, the third inequality is based on a Bonferroni bound, the last inequality follows from Eqs.\ \eqref{eq:SNbound1}-\eqref{eq:SNbound2} in Lemma \ref{lem:SampleBound} upon choosing $\gamma=c/\sqrt{n}$ in that result.

We are interested in applying Eq.\ \eqref{eq:KeyBound} with $c=c_{n}^{SN}(|L|,\alpha)$ which satisfies:
\begin{equation}
	(2(k-p)+|L|) \del[2]{1-\Phi ( c_{n}^{SN}(|L|,\alpha)/\sqrt{ 1+c_{n}^{SN}(|L|,\alpha)^{2}/n})}= \alpha.
	\label{eq:CoverageDefn}
\end{equation}
Before doing this, we need to verify that this is a valid choice, i.e., we need to verify that, for all sufficiently large $n$,
\begin{equation}
	c_{n}^{SN}(|L|,\alpha)/\sqrt{1+c_{n}^{SN}(|L|,\alpha)^{2}/n} 
	% = \Phi ^{-1}( 1-\alpha /(2(k-p)+|L|))
	~~\in~~ [0,n^{\delta /(2(2+\delta ))}M_{n,2+\delta }^{-1}].
\end{equation}
On the one hand, note that $c_{n}^{SN}(|L|,\alpha)\geq 0$ implies that $ c_{n}^{SN}(|L|,\alpha)/\sqrt{1+c_{n}^{SN}(\alpha ,|L|)^{2}/n}\geq 0$. On the other hand, note that, by definition, $c_{n}^{SN}(|L|,\alpha)/\sqrt{1+c_{n}^{SN}(|L|,\alpha)^{2}/n} = \Phi ^{-1}( 1-\alpha /(2(k-p)+|L|))$ and so it suffices to show that $\Phi ^{-1}( 1-\alpha /(|L|+2(k-p))) M_{n,2+\delta }n^{-\delta /(2(2+\delta ))} \to 0$. To show this, note that $\Phi ^{-1}( 1-\alpha /(2(k-p)+|L|)) \leq \sqrt{2\ln ( (|L|+2(k-p))/\alpha ) }\leq \sqrt{2\ln ( (2k-p)/\alpha ) }$, where the first inequality uses that $1-\Phi (t)\leq \exp ( -t^{2}/2) $ for any $t>0$ and the second inequality follows from $|L|\leq p$. These inequalities and $\ln ( (2k-p)/\alpha ) M_{n,2+\delta }^{2}n^{-\delta /(2+\delta )}\to 0$ (by Assumption \ref{ass:Rates}) complete the verification.

Therefore, by Eq.\ \eqref{eq:KeyBound} with $c=c_{n}^{SN}(|L|,\alpha)$ we conclude that:
\begin{eqnarray*}
P(T_{n}>c_{n}^{SN}(|L|,\alpha)) ~\leq~ \alpha +\alpha Kn^{-\delta /2}M_{n,2+\delta }^{2+\delta }(1+\Phi ^{-1}( 1-\alpha /(2(k-p)+|L|)) )^{2+\delta }  ~\leq~ \alpha +R_{n},
\end{eqnarray*}
where the first inequality uses Eq.\ \eqref{eq:CoverageDefn} and the second inequality follows from the definition $R_{n}$ and $f(x)\equiv \Phi ^{-1}( 1-\alpha /(2(k-p)+x)) $ being increasing and $|L|\leq p$. To conclude the proof, it suffices to show that $R_{n}\to 0$. To this end, consider the following argument:
\begin{eqnarray*}
R_{n} &\equiv &\alpha Kn^{-\delta /2}M_{n,2+\delta }^{2+\delta }(1+\Phi ^{-1}( 1-\alpha /(2k-p)) )^{2+\delta } \\
&\leq &\alpha 2^{1+\delta}Kn^{-\delta /2}M_{n,2+\delta }^{2+\delta }(1+\vert \Phi ^{-1}( 1-\alpha /(2k-p)) \vert ^{2+\delta }) \\
&\leq &\alpha 2^{1+\delta} K n^{-\delta /2}M_{n,2+\delta }^{2+\delta } (1 + {2}^{1/2}( \ln ( (2k-p)/\alpha ) ) ^{( 2+\delta ) /2}))=o(1),
\end{eqnarray*}
where the first inequality uses the convexity of $f(x)=x^{2+\delta}$ and $\delta>0$ and Jensen's Inequality to show $(1+a)^{2+\delta}\leq 2^{1+\delta}(1+a^{2+\delta})$ for any $a>0$, the second inequality follows from $1-\Phi (t)\leq \exp ( -t^{2}/2) $ for any $t>0$ and so $\Phi ^{-1}(1-\alpha /(2k-p) )\leq \sqrt{2\ln ( (2k-p) /\alpha ) }$, and the convergence to zero is based on $n^{-\delta /2}M_{n,2+\delta }^{2+\delta }(\ln (2k-p) )^{(2+\delta )/2}\to 0$ (by Assumption \ref{ass:Rates}) which for $2k-p>1$ implies that $n^{-\delta /2}M_{n,2+\delta }^{2+\delta }\to 0 $.
\end{proof}
%%%%%%%%%%%%%%%%%%% -- DIVIDER - %%%%%%%%%%%%%%%%%%%
\begin{proof}[Proof of Theorem \ref{thm:SN1Scorollary}]
	This result follows from Lemma \ref{lem:SNSize} with $L=\{1,\dots ,p\}$.
\end{proof}

%%%%%%%%%%%%%%%%%%% -- DIVIDER - %%%%%%%%%%%%%%%%%%%
\begin{proof}[Proof of Theorem \ref{thm:Lasso2SSize}]
This proof follows similar steps than \citetalias{chernozhukov/chetverikov/kato:2014c} (Proof of Theorem 4.2). Let us define the sequence of sets:
\begin{equation*}
	J_{I} ~\equiv~ \{j=1,\dots ,p:\mu _{j}/\sigma _{j}\geq -3\lambda _{n}/4 \}
\end{equation*}
We divide the proof into three steps.

\underline{Step 1.} We show that $\hat{\mu}_{j}\leq 0$ for all $j\in J_{I}^{c}$ with high probability, i.e., for any $c\in (0,1)$,
\[
P\del[1]{ \cup _{j\in J_{I}^{c}}\{\hat{\mu}_{j}>0\}}
\leq
2p\exp \del[1]{ -2^{-1}n^{\delta /(2+\delta )}/M_{n,2+\delta }^{2}} \sbr[1]{ 1+K( M_{n,2+\delta }/n^{\delta /(2(2+\delta ))}+1) ^{2+\delta }} +\tilde{K} n^{-c}\to 0,
\]
where $K$ and $\tilde{K}$ are universal constants.

First, we show that for any $r\in (0,1)$,
\[
 \cbr[1]{\cup _{j\in J_{I}^{c}}\{\hat{\mu}_{j}>0\}} \cap \cbr[2]{ \sup_{j=1,\dots ,p}\vert \hat{\sigma}_{j}/\sigma _{j}-1\vert \leq r/(1+r) } \subseteq \cbr[2]{ \sup_{j=1,\dots ,p}\vert \hat{\mu}_{j}-\mu _{j}\vert /\hat{\sigma }_{j}>(1-r)\lambda _{n}3/4 }.
\]
To see this, suppose that there is an index $j=1,\dots ,p$ s.t.\ $\mu _{j}/\sigma _{j}<-\lambda _{n}3/4 $ and $\hat{\mu}_{j}>0$. Then, $\vert \hat{\mu}_{j}-\mu _{j}\vert /\hat{\sigma}_{j}>\lambda_{n}(3/4) ( \sigma _{j}/\hat{\sigma}_{j}) $. In turn, $ \sup_{j=1,\dots ,p}\vert 1- \hat{\sigma}_{j}/\sigma _{j}\vert \leq r/(1+r)$ implies that $\vert 1- \sigma _{j}/\hat{\sigma} _{j}\vert \leq r$ and so $( \sigma _{j}/\hat{\sigma}_{j})\lambda _{n}3/4 \geq  ( 1-r)\lambda _{n}3/4 $. By combining these, we conclude that $ \sup_{j=1,\dots ,p}\vert \hat{\mu}_{j}-\mu _{j}\vert /\hat{\sigma }_{j}>( 1-r)\lambda _{n}(3/4) $.

Based on this, consider the following derivation for any $r\in (0,1)$,
\begin{align}
P( \cup _{j\in J_{I}^{c}}\{\hat{\mu}_{j}>0\})~&=~\left\{
\begin{array}{c}
P( \cup _{j\in J_{I}^{c}}\{\hat{\mu}_{j}>0\}\cap \sup_{j=1,\dots ,p}\vert \hat{\sigma}_{j}/\sigma _{j}-1\vert \leq r/(1+r)) + \\
P( \cup _{j\in J_{I}^{c}}\{\hat{\mu}_{j}>0\}\cap \sup_{j=1,\dots ,p}\vert \hat{\sigma}_{j}/\sigma _{j}-1\vert >r/(1+r))
\end{array}
\right\} \notag\\
~&\leq~ P\del[2]{\sup_{j=1,\dots ,p}\vert \hat{\mu}_{j}-\mu _{j}\vert /\hat{\sigma}_{j}> ( 1-r)\lambda _{n}3/4 } +
P\del[2]{ \sup_{j=1,\dots ,p}\vert \hat{\sigma }_{j}/\sigma _{j}-1\vert >r/(1+r)}.\label{eq:step1_r_step}
\end{align}
By evaluating Eq.\ \eqref{eq:step1_r_step} with $r=r_{n}=(((n^{-( 1-c) /2}\ln p+n^{-3/2}( \ln p) ^{2}) B_{n}^{2}) ^{-1}-1)^{-1}\to 0$ (by Assumption \ref{ass:Rates2}), we deduce that:
\begin{align*}
P( \cup _{j\in J_{I}^{c}}\{\hat{\mu}_{j}>0\}) ~\leq~ 2p\exp ( -2^{-1}n^{\delta /(2+\delta )}/M_{n,2+\delta }^{2}) [ 1+K( M_{n,2+\delta }/n^{\delta /(2(2+\delta ))}+1) ^{2+\delta }] +\tilde{K}n^{-c},
\end{align*}
where the first term is a consequence of Lemma \ref{lem:SampleBound2}, $r_{n}\to 0$, and
$ ( 1-r_{n})\lambda _{n}3/4 \geq n^{-1/2}( M_{n,2+\delta }^{2}n^{-\delta /(2+\delta )}-n^{-1}) ^{-1/2}$ for all $n$ sufficiently large, and the second term is a consequence of \citetalias{chernozhukov/chetverikov/kato:2014c} (Lemma A.5) and $ r_{n}/(1+r_{n})=[ n^{-( 1-c) /2}\ln p+n^{-3/2}( \ln p) ^{2}] B_{n}^{2}\to 0$.

\underline{Step 2.} We show that $J_{I}\subseteq \hat{J}_{L}$ with high probability, i.e.,
\begin{equation*}
	P(J_{I}\subseteq \hat{J}_{L}) ~\geq~ 1- 2p\exp ( -2^{-1}n^{\delta /(2+\delta )}/M_{n,2+\delta }^{2}) [ 1+K( M_{n,2+\delta }/n^{\delta /(2(2+\delta ))}+1) ^{2+\delta }] +\tilde{K} n^{-c},
\end{equation*}
where $K,\tilde{K}$ are uniform constants.

First, we show that for any $r\in (0,1)$,
\begin{equation*}
	 \cbr[2]{\{ J_{I}\not\subseteq \hat{J}_{L}\} \cap \cbr[2]{ \sup_{j=1,\dots ,p}\vert \hat{\sigma}_{j}/\sigma _{j}-1\vert \leq r/(1+r) } } ~\subseteq~ \cbr[2]{ \sup_{j=1,\dots ,p}\vert \hat{\mu}_{j}-\mu _{j}\vert /\hat{\sigma}_{j}>\lambda _{n}(1-r)3/4}.
\end{equation*}
To see this, consider the following argument. Suppose that $j\in J_{I}$ and $j\not\in \hat{J}_{L}$, i.e., $\mu _{j}/\sigma _{j}\geq -\lambda _{n}3/{4} $ and $\hat{\mu}_{L,j}/\hat{\sigma} _{j}< -\lambda _{n}$ or, equivalently by Eq.\ \eqref{eq:ClosedForm2},  $\hat{\mu}_{j}/\hat{\sigma} _{j}< -\lambda _{n}{3}/{2}$. Then, $\vert \mu _{j} - \hat{\mu}_j \vert / \hat{\sigma}_{j}>\lambda _{n}[\frac{3}{2}-\frac{3}{4} (\sigma _{j}/\hat{ \sigma}_{j})]$. In turn, $\sup_{j=1,\dots ,p}\vert 1 - \hat{\sigma}_j/ \sigma_j \vert \leq r/(1+r)$ implies that $\vert \sigma _{j}/\hat{\sigma}_{j}-1\vert \leq r$ and so $\lambda _{n}[\frac{3}{2}-\frac{3}{4}(\sigma _{j}/\hat{\sigma}_{j})]\geq \lambda _{n}(1-r)3/4$. By combining these, we conclude that $ \sup_{j=1,\dots ,p}\vert \hat{\mu}_{j}-\mu _{j}\vert /\hat{ \sigma}_{j}>\lambda _{n}(1-r)3/4$, as desired.

Based on this, consider the following derivation for any $r\in (0,1)$,
\begin{eqnarray*}
P(J_{I}\not\subseteq \hat{J}_{L}) &=&\left\{
\begin{array}{c}
	P \left( \{ J_{I}\not\subseteq \hat{J}_{L}\} \cap \{ \sup_{j=1,\dots ,p}\vert \hat{\sigma}_{j}/\sigma _{j}-1\vert \leq r/(1+r)\}  \right)\\
	 +P \left( \{ J_{I}\not\subseteq \hat{J} _{L}\} \cap \{ \sup_{j=1,\dots ,p}\vert \hat{\sigma} _{j}/\sigma _{j}-1\vert >r/(1+r) \} \right)
\end{array}
\right\}
\\
&\leq& P\del[2]{\sup_{j=1,\dots ,p}\vert \hat{\mu}_{j}-\mu _{j}\vert /\hat{\sigma}_{j}>\lambda _{n}(1-r)3/4} +
P\del[2]{\sup_{j=1,\dots ,p}\vert \hat{\sigma }_{j}/\sigma _{j}-1\vert >r/(1+r) } .
\end{eqnarray*}
Notice that the expression on the RHS is exactly the RHS of Eq.\ \eqref{eq:step1_r_step}. Consequently, by evaluating this equation in $r=r_{n}$ and repeating arguments used in step 1, the desired result follows.

\underline{Step 3.} We now complete the argument. Consider the following derivation:
\begin{align*}
\left\{ \{ T_{n}>c_{n}^{SN,L}(\alpha) \}\cap \{J_{I}\subseteq \hat{J}_{L}\} \cap \{\cap _{j\in J_{I}^{c}} \{\hat{\mu}_{j}\leq 0\}\} \right\} &\subseteq \left\{ \{T_{n}>c_{n}^{SN}(|J_{I}|,\alpha )\} \cap \{\cap _{j\in J_{I}^{c}}\cbr[1]{\hat{\mu}_{j}\leq 0}\} \right\}\\
&\subseteq \cbr[3]{\max \cbr[2]{ \max_{j\in J_{I}}\frac{\sqrt{n}\hat{\mu}_{j}}{\hat{ \sigma}_{j}},\max_{s=p+1,\dots ,k}\frac{\sqrt{n}\vert \hat{\mu} _{s}\vert }{\hat{\sigma}_{s}}}>c_{n}^{SN}(\alpha ,|J_{I}|)} ,
\end{align*}
where we have used $c_{n}^{SN,L}(\alpha) = c_{n}^{SN}(\alpha ,|\hat{J}_L|)$, Lemma \ref{lem:CSNincreasing} (in that $c_{n}^{SN}(\alpha ,d)$ is a non-negative increasing function of $d\in \{0,1\dots ,2k-p\}$), and we take $\max_{j\in J_{I}}\sqrt{n}\hat{\mu}_{j}/\hat{\sigma}_{j}=-\infty $ if $ J_{I}=\emptyset $. Thus,
\begin{align}
P( T_{n}>c_{n}^{SN,L}(\alpha)) &=\left\{
\begin{array}{c}
P( \{ T_{n}>c_{n}^{SN,L}(\alpha)\} \cap \{ \{ J_{I}\subseteq \hat{J}_{L}\} \cap \{ \cap _{j\in J_{I}^{c}}\{\hat{\mu}_{j}\leq 0\}\} \} ) + \\
P( \{ T_{n}>c_{n}^{SN,L}(\alpha)\} \cap \{ \{ J_{I}\not\subseteq \hat{J}_{L}\} \cup \{ \cup _{j\in J_{I}^{c}}\{\hat{\mu}_{j}>0\}\} \} )
\end{array}
\right\}  \nonumber \\
&\leq 
P\del[3]{ \max \cbr[2]{ \max_{j\in J_{I}}\frac{\sqrt{n}\hat{\mu}_{j}}{\hat{ \sigma}_{j}},\max_{s=p+1,\dots ,k}\frac{\sqrt{n}\vert \hat{\mu} _{s}\vert }{\hat{\sigma}_{s}}} >c_{n}^{SN}(\alpha ,|J_{I}|)} + P( J_{I}\not\subseteq \hat{J}_{L}) +P( \cup _{j\in J_{I}^{c}}\{\hat{\mu}_{j}>0\}) \nonumber \\
&\leq \alpha +\left\{
\begin{array}{c}
\alpha Kn^{-\delta /2}M_{n,2+\delta }^{2+\delta }(1+\Phi ^{-1}( 1-\alpha /(2k-p)) )^{2+\delta }+ \\
4p\exp ( -2^{-1}n^{\delta /(2+\delta )}/M_{n,2+\delta }^{2}) [ 1+K( M_{n,2+\delta }/n^{\delta /(2(2+\delta ))}+1) ^{2+\delta }] +2\tilde{K} n^{-c}
\end{array}
\right\}  \nonumber \\
&\leq \alpha +o(1), 
\end{align}
where the third line uses Lemma \ref{lem:SNSize} and steps 1 and 2, and the convergence in the last line holds uniformly in the manner required by the result.
\end{proof}


%%%%%%%%%%%%%%%%%%% -- DIVIDER - %%%%%%%%%%%%%%%%%%%

\subsection{Results for the bootstrap approximation}
%%%%%%%%%%%%%%%%%%% -- DIVIDER - %%%%%%%%%%%%%%%%%%%
\begin{lemma} \label{lem:BootSize}
Assume Assumptions \ref{ass:Basic}, \ref{ass:Rates3}, $\alpha \in (0,0.5)$, and that $H_{0}$ holds. For any non-stochastic set $L\subseteq \{1,\dots ,p\}$, define:
\[
T_{n}(L) ~\equiv~ \max \cbr[2]{ \max_{j\in L}{\sqrt{n}\hat{\mu}_{j}}/{\hat{\sigma} _{j}},\max_{s=p+1,\dots ,k}{\sqrt{n}\vert \hat{\mu} _{s}\vert }/{\hat{\sigma}_{s}}},
\]
and let $c_{n}^{B}(L,\alpha)$ with $B\in \{MB,EB\}$ denote the conditional $(1-\alpha )$-quantile based on the bootstrap.
Then,
\[
P(T_{n}(L)>c_{n}^{B}(L,\alpha)) \leq \alpha +\tilde{C}n^{- \tilde{c}},
\]
where $\tilde{c},\tilde{C}>0$ are positive constants that only depend on the constants $c,C$ in Assumption \ref{ass:Rates3}. Furthermore, if $\mu _{j}=0$ for all $j\in L$ then:
\[
\vert P( T_{n}(L)>c_{n}^{B}(L,\alpha)) -\alpha \vert \leq \tilde{C}n^{-\tilde{c}}.
\]
Finally, since $\tilde{c},\tilde{C}$ depend only on the constants $c,C$ in Assumption \ref{ass:Rates3}, the proposed bounds are uniform in all parameters $\theta \in \Theta$ and distributions $P$ that satisfy the assumptions in the statement.
\end{lemma}
%%%%%%%%%%%%%%%%%%% -- DIVIDER - %%%%%%%%%%%%%%%%%%%
\begin{proof}
In the absence of moment equalities equalities, this results follow from replacing $\{1,\dots ,p\}$ with $L$ in \citetalias{chernozhukov/chetverikov/kato:2014c} (proof of Theorem 4.3). As we show next, our proof can be completed by simply redefining the set of moment inequalities by adding the moment equalities as two sets of inequalities with reversed sign.

Define $A = A(L)\equiv  L\cup \{p+1,\dots ,k\}\cup \{k+1,\dots ,2k-p\}$ with $|A|=|L|+2(k-p)$ and for any $i=1,\dots ,n$, define the following $|A|$-dimensional auxiliary data vector:
\[
X_{i}^{E}\equiv \cbr[1]{ \{X_{ij}\}_{j\in L}^{\prime },\{X_{is}\}_{s=p+1,\dots ,k}^{\prime },\{-X_{is}\}_{s=p+1,\dots ,k}^{\prime }} ^{\prime }.
\]
Based on these definitions, we modify all expressions analogously, e.g.,
\begin{eqnarray*}
\mu ^{E} &=&\{ \{\mu _{j}\}_{j\in L}^{\prime },\{\mu _{s}\}_{s=p+1,\dots ,k}^{\prime },\{-\mu _{s}\}_{s=p+1,\dots ,k}^{\prime }\} ^{\prime }, \\
\sigma ^{E} &=&\{ \{\sigma _{j}\}_{j\in L}^{\prime },\{\sigma _{s}\}_{s=p+1,\dots ,k}^{\prime },\{\sigma _{s}\}_{s=p+1,\dots ,k}^{\prime }\} ^{\prime },
\end{eqnarray*}
and notice that $H_{0}$ is equivalently re-written as $\mu ^{E}\leq \mathbf{0}_{|A|}$.

In the new notation, the test statistic is re-written as $T_{n}(L)=\max_{j\in A} {\sqrt{n}\hat{ \mu}_{j}^{E}}/{\hat{\sigma}_{j}^{E}}$, and the critical values can re-written analogously. In particular, the MB and EB test statistics are respectively defined as follows:
\begin{eqnarray*}
	W_{n}^{MB}(L)&=&\max_{j\in A} {\sqrt{n}\sum_{i=1}^{n}  \epsilon _{i}( X_{ij}^{E}-\hat{\mu}_{j}^{E}) }/{\hat{\sigma }_{j}^{E}},\\
	W_{n}^{EB}(L)&=&\max_{j\in A} {\sqrt{n}\sum_{i=1}^{n}  ( X_{ij}^{*,E}-\hat{\mu}_{j}^{E}) }/{\hat{\sigma }_{j}^{E}}.
\end{eqnarray*}
Given this setup, the result follows immediately from \citetalias{chernozhukov/chetverikov/kato:2014c} (Theorem 4.3).
\end{proof}


%%%%%%%%%%%%%%%%%%% -- DIVIDER - %%%%%%%%%%%%%%%%%%%
\begin{proof}[Proof of Theorem \ref{thm:B1Scorollary}.]
	This result follows from Lemma \ref{lem:BootSize} with $|L|=\{1,\dots ,p\}$.
\end{proof}

%%%%%%%%%%%%%%%%%%% -- DIVIDER - %%%%%%%%%%%%%%%%%%%

\begin{lemma}\label{lem:CvBootIncreasing}
For any $\alpha \in (0,0.5)$, $n\in \mathbb{N}$, $B\in \{MB,EB\}$, and $L_{1}\subseteq L_{2}\subseteq \{1,\dots ,p\}$, % let $c_{n}^{B}(\alpha ,L)$ denote the conditional $(1-\alpha )$-quantile based on multiplier bootstrap (MB) or empirical bootstrap (EB), i.e., $B\in \{MB,EB\}$. Then,
\[
c_{n}^{B}(L_{1},\alpha)~\leq~ c_{n}^{B}(L_{2},\alpha ).
\]
Furthermore, under the above assumptions, $P\del[1]{c_{n}^{B}(L_{1},\alpha)\geq 0}\geq 1-Cn^{-c}$, where $c,C$ are universal constants.
\end{lemma}
%%%%%%%%%%%%%%%%%%% -- DIVIDER - %%%%%%%%%%%%%%%%%%%
\begin{proof}
By definition, $L_{1}\subseteq L_{2}$ implies that $W_{n}^{B}(L_{1})\leq W_{n}^{B}(L_{2})$ which, in turn, implies $c_{n}^{B}(L_{1},\alpha)\leq c_{n}^{B}(L_{2},\alpha)$. 

We now turn to the second result. If the model has at least one moment equality, then $W_n^B(L_{1})\geq 0$ and so $c_{n}^{B}(\alpha ,L_{1})\geq 0$. If the model has no moment equalities, then we consider consider a different argument depending on the type of bootstrap procedure being implemented.

First, consider MB. Conditionally on the sample, $W_{n}^{MB}(L_{1})=\max_{j\in L}{(1/\sqrt{n})\sum_{i=1}^{n} \epsilon _{i}\left( X_{ij}-\hat{\mu}_{j}\right)  }/{\hat{\sigma}_{j}}$ is the maximum of $L_{1}$ zero mean Gaussian random variables. Thus, $\alpha \in (0,0.5)$ implies that $c_{n}^{MB}(\alpha ,L_{1})\geq 0$. 

Second, consider EM. Let $c_{0}(L_{1},\alpha)$ denote the $(1-\alpha)$-quantile of $ \max_{j \in L_{1}}Y_{j}$ with $\{Y_{j}\}_{j \in L_{1}}\sim N(\mathbf{0},E [ \tilde{Z}\tilde{Z}'] )$ with $\tilde{Z} = \{Z_j\}_{j \in L_{1}}$ and $Z$ as in Assumption \ref{ass:Rates2}. At this point, we apply \citetalias{chernozhukov/chetverikov/kato:2014c} (Eq.\ (66)) to our hypothetical model with the moment inequalities indexed by $L_{1}$. Applied to this model, their Eq.\ (66) yields:
\begin{equation}
	P\del[1]{c_{n}^{EB}(L_{1},\alpha)\geq c_0(L_{1},\alpha +\gamma_{n})}~\geq~ 1-Cn^{-c},\label{eq:66inCCK}
\end{equation}
where $\gamma_{n} \equiv \zeta_{n2}+\nu_{n}+8\zeta_{n1}\sqrt{\log p} \in (0, 2Cn^{-c})$, for sequences $\{(\zeta_{n1},\zeta_{n2},\nu_{n})\}_{n\ge 1}$ and universal positive constants $(c,C)$, all specified in \citetalias{chernozhukov/chetverikov/kato:2014c}. Since $\alpha<0.5$ and $\gamma_{n}<2Cn^{-c}$, it follows that for all $n$ sufficiently large, $\alpha + \gamma_{n}<0.5$ and so $c_0(\alpha + \gamma_{n},L_{1})>0$. The desired result follows from combining this with Eq.\ \eqref{eq:66inCCK}.
\end{proof}

%%%%%%%%%%%%%%%%%%% -- DIVIDER - %%%%%%%%%%%%%%%%%%%

\begin{proof}[Proof of Theorem \ref{lem:Lasso2SBootstrap}]
This proof follows similar steps than \citetalias{chernozhukov/chetverikov/kato:2014c} (Proof of Theorem 4.4). Let us define the sequence of sets:
\begin{equation*}
	J_{I} ~\equiv~ \{j=1,\dots ,p:\mu _{j}/\sigma _{j}\geq -3\lambda _{n}/4 \}
\end{equation*}
We divide the proof into three steps. Steps 1-2 are exactly as in the proof of Theorem \ref{thm:Lasso2SSize} so they are omitted.

\underline{Step 3.} Defining $T_{n}(J_{I})$ as in Lemma \ref{lem:BootSize} and consider the following derivation:
\begin{align*}
&\cbr[2]{T_{n}>c_{n}^{B}(\hat{J}_L,\alpha)}\cap  \cbr[2]{J_{I}\subseteq \hat{J}_{L}}\cap \cbr[2]{\cap _{j\in J_{I}^{c}}\cbr[1]{\hat{\mu}_{j}\leq 0}}\cap\cbr[2]{c_{n}^{B}(J_I,\alpha)\geq 0}\\
&\subseteq
\cbr[2]{T_{n}>c_{n}^{B}(J_{I},\alpha)}\cap \cbr[2]{\cap _{j\in J_{I}^{c}}\cbr[1]{\hat{\mu}_{j}\leq 0}}\cap\cbr[2]{c_{n}^{B}(\alpha ,J_I)\geq 0}\\
&\subseteq 
\cbr[2]{T_n(J_I)>c_{n}^{B}(J_{I},\alpha)} ,
\end{align*}
where the first inclusion follows from Lemma \ref{lem:CvBootIncreasing}, and the second inclusion follows from noticing that ${\cap _{j\in J_{I}^{c}}\{\hat{\mu}_{j}\leq 0}\}$ and $\{T_{n}>c_{n}^{B}(\alpha ,J_{I})\geq 0\}$ implies that $\{T_n(J_I)>c_{n}^{B}(\alpha ,J_{I})\}$. Thus,
\begin{align}
P\del[1]{T_{n}>c_{n}^{B,L}(\alpha)} &= P\del[1]{T_{n}>c_{n}^{B}(\hat{J}_{L},\alpha)} \notag\\
 &=\cbr[4]{
\begin{array}{c}
P\del[1]{ \{ T_{n}>c_{n}^{B}(\hat{J}_{L},\alpha)\} \cap \{ \{ J_{I}\subseteq \hat{J}_{L}\} \cap \{ \cap _{j\in J_{I}^{c}}\{\hat{\mu}_{j}\leq 0\}\} \cap\cbr[0]{c_{n}^{B}(\alpha ,J_I)\geq 0}\} } + \\
P\del[1]{ \{ T_{n}>c_{n}^{B}(\hat{J}_{L},\alpha)\} \cap \{ \{ J_{I}\not\subseteq \hat{J}_{L}\} \cup \{ \cup _{j\in J_{I}^{c}}\{\hat{\mu}_{j}>0\}\} \cup\cbr[0]{c_{n}^{B}(\alpha ,J_I)< 0}\} }
\end{array}
}   \notag \\
&\leq P( T_{n}(J_{I})>c_{n}^{B}(J_{I},\alpha)) +P( J_{I}\not\subseteq \hat{J}_{L}) +P( \cup _{j\in J_{I}^{c}}\{\hat{ \mu}_{j}>0\})+P(c_{n}^{B}(\alpha ,J_I)< 0) \nonumber \\
&\leq \alpha + Cn^{-c}+ \tilde{C} n^{-\tilde{c}} + 4p\exp ( -2^{-1}n^{\delta /(2+\delta )}M_{n,2+\delta }^{-2}) [ 1+K( n^{-\delta /(2(2+\delta ))}M_{n,2+\delta }+1) ^{2+\delta }] +2\tilde{K} n^{-c}  \notag \\
&\leq \alpha +o(1),  \label{eq:Derivation2SMBLasso}
\end{align}
where the convergence in the last line is uniform in the manner required by the result. The third line of Eq.\ \eqref{eq:Derivation2SMBLasso} uses Lemmas \ref{lem:BootSize} and \ref{lem:CvBootIncreasing} as well as steps 1 and 2.

We next turn to the second part of the result. By the case under consideration, $\mu={\bf 0}_{p}$ and so $J_{I} = \{ 1, \dots, p \}$. Thus, in this case, $\{J_{I} \subseteq \hat{J}_{L}\} = \{ \hat{J}_{L} = J_{I} = \{1,\dots,p\}\}$. By this and step 2 of Theorem \ref{thm:Lasso2SSize}, it follows that:
\begin{equation}
	P\del[1]{ \hat{J}_{L} = J_{I} = \{1,\dots,p\}} ~\geq~ 1 - 2p\exp ( -2^{-1}n^{\delta /(2+\delta )}/M_{n,2+\delta }^{2}) [ 1+K( M_{n,2+\delta }/n^{\delta /(2(2+\delta ))}+1) ^{2+\delta }] + \tilde{K} n^{-c},
	\label{eq:mehmet1}
\end{equation}
% where
% \begin{equation}
% 	R_{n} \equiv  2p\exp ( -2^{-1}n^{\delta /(2+\delta )}/M_{n,2+\delta }^{2}) [ 1+K( M_{n,2+\delta }/n^{\delta /(2(2+\delta ))}+1) ^{2+\delta }] + \tilde{K} n^{-c}
% 	\label{eq:mehmet2}
% \end{equation}
where $K,\tilde{K}$ are uniform constants. In turn, notice that $\{\hat{J}_{L} = J_{I} = \{1,\dots,p\} \}$ implies that $c_n^{B,1S}  (\alpha) = c_n^{B} ({J}_{I},\alpha) = c_n^{B} (\hat{J}_{L},\alpha) = c_n^{B,L} (\alpha)$. Thus,
\begin{eqnarray}
 P ( T_n > c_n^{B,L} (\alpha) )&=& P ( \{T_n > c_n^{B,L} ( \alpha)\} \cap \{\hat{J}_{L} = J_{I} = \{1,\dots,p\}\}) + P ( \{T_n > c_n^{B,L} ( \alpha)\} \cap \{\hat{J}_{L} = J_{I} = \{1,\dots,p\}\}^{c})  \notag\\
& \ge& P ( \{T_n > c_n^{B,1S} ( \alpha)\} \cap \{\hat{J}_{L} = J_{I} = \{1,\dots,p\}\}) \notag\\
% & =& P ( T_n > c_n^{B,1S} ( \alpha) ) - P ( \{T_n > c_n^{B,1S} ( \alpha)\} \cap \{\hat{J}_{L} = J_{I} = \{1,\dots,p\}\}^{c}) \notag\\
& \ge& P ( T_n > c_n^{B,1S} ( \alpha) ) - P ( \{\hat{J}_{L} = J_{I} = \{1,\dots,p\}\}^{c}) \notag \\
& \ge&  \alpha - 2\tilde{C} n^{- \tilde{c}},\label{eq:Derivation2SMBLasso2}
\end{eqnarray}
where the last inequality uses the second result in Theorem \ref{thm:B1Scorollary}, Eq.\ \eqref{eq:RestrictionOnParams}, and Eq.\ \eqref{eq:mehmet1}. If we combine this with Eq.\ \eqref{eq:Derivation2SMBLasso}, the result follows.
\end{proof}


\subsection{Results for power comparison}

%%%%%%%%%%%%%%%%%%% -- DIVIDER - %%%%%%%%%%%%%%%%%%%
\begin{proof}[Proof of Theorem  \ref{thm:powercompSN}]
	The arguments in the main text show that Eq.\ \eqref{eq:JComparisonSN} implies Eq.\ \eqref{eq:PowerComparisonSN}. To complete the proof, it suffices to show that the two sufficient conditions imply Eq.\ \eqref{eq:JComparisonSN}. By definition and Lemma \ref{lem:LassoClosedForm},
\begin{eqnarray*}
\hat{J}_{SN} &=&\{j=1,\dots,p:\hat{\mu}_{j}/\hat{\sigma}_{j}\geq -2c_{n}^{SN,1S}(\beta _{n})/\sqrt{n}\}, \\
\hat{J}_{L} &=&\{j=1,\dots,p:\hat{\mu}_{j,L}/\hat{\sigma}_{j}\geq -\lambda _{n}\}=\{j=1,\dots ,p:\hat{\mu}_{j}/\hat{\sigma}_{j}\geq -\lambda _{n}3/2\},
\end{eqnarray*}

\underline{Condition 1.} We show this by contradiction, i.e., suppose that Eq.\ \eqref{eq:highlevel} and $\hat{J}_{L}\not\subseteq \hat{J}_{SN}$ hold. By the latter, $ \exists j=1,\dots ,p$ s.t.\ $j\in \hat{J}_{L}\cap \hat{J}_{SN}^{c}$, i.e., $-2c_{n}^{SN,1S}(\beta _{n})/\sqrt{n}>{\hat{\mu}_{j}}/{\hat{\sigma}_{j}}\geq - \lambda _{n}3/2$, which implies that $c_{n}^{SN,1S}(\beta _{n})4/3 < \sqrt{n} \lambda _{n}$, contradicting Eq.\ \eqref{eq:highlevel}.

\underline{Condition 2.} By definition, $c_{n}^{SN,1S}(\beta _{n})4/3 \geq \sqrt{n}\lambda _{n}$ is equivalent to
\begin{equation}
\del[1]{\Phi ^{-1}(1-\beta _{n}/p)} ^{2}~\geq~ n\lambda _{n}^{2}\frac{9}{16}. \label{eq:InequalityForPower}
\end{equation}
The remainder of the proof shows that Eq.\ \eqref{eq:InequalityForPower} holds under Eq.\ \eqref{eq:lowlevel}. 

First, we establish a lower bound for the LHS of Eq.\ \eqref{eq:InequalityForPower}. For any $x\geq 1$, consider the following inequalities:
\[
1-\Phi (x)~\geq~ \frac{1}{x+1/x}\frac{1}{\sqrt{2\pi }}e^{-x^{2}/2}~\geq~ \frac{1}{2x}\frac{1}{\sqrt{2\pi }}e^{-x^{2}/2}~\geq~ \frac{1}{2\sqrt{2\pi }}e^{-x^{2}},
\]
where the first inequality holds for all $x>0$ by \citet[Eq.\ (10)]{gordon:1941}, the second inequality holds by $x\geq 1$ and so $x>1/x$, and the third inequality holds by $e^{-x^{2}/2}\leq 1/x$ for all $x>0$. Note that for $\beta _{n}\leq 10\% $ and $p\geq 1$, $\Phi ^{-1}( 1-\beta _{n}/p) \geq 1$. Evaluating the previous display at $x=\Phi ^{-1}( 1-\beta _{n}/p) $ yields:
\begin{equation}
\del[1]{\Phi ^{-1}(1-\beta _{n}/p)} ^{2}~\geq~ \ln \del[2]{ \frac{p}{2 \sqrt{2\pi }\beta _{n}}} . \label{eq:InequalityForPowerRHS}
\end{equation}
Second, we establish an upper bound for the RHS of Eq.\ \eqref{eq:InequalityForPower}. 
By Eq.\ \eqref{eq:LambdaConcrete},
\begin{align}
n\lambda _{n}^{2}~=~(4/3+\varepsilon)^{2}\frac{n}{n^{2/(2+\delta )}M_{n,2+\delta }^{2}-1}~\leq~ 
% (4/3+\varepsilon)^{2}\frac{2n}{n^{2/(2+\delta )}M_{n,2+\delta }^{2}}=
2(4/3+\varepsilon)^{2}n^{\delta /(2+\delta )}M_{n,2+\delta }^{-2},\label{eq:nlambda}
\end{align}
where the last inequality used that $1/(x-1)\leq 2/x$ for $x\geq 2$ and that $n^{2/(2+\delta)}M_{n,2+\delta}^2\geq 2$. Thus,
\begin{equation}
\frac{9}{16}n\lambda _{n}^{2}~\leq~ \frac{18}{16}(4/3+\varepsilon)^{2}n^{\delta /(2+\delta )}M_{n,2+\delta }^{-2} ~=~ \frac{9}{8}(4/3+\varepsilon)^{2}n^{\delta /(2+\delta )}M_{n,2+\delta }^{-2}. \label{eq:InequalityForPowerLHS}
\end{equation}
To conclude the proof, notice that Eq.\ \eqref{eq:InequalityForPower} follows directly from combining Eqs.\ \eqref{eq:lowlevel}, \eqref{eq:InequalityForPowerRHS}, and \eqref{eq:InequalityForPowerLHS}.
\end{proof}

%%%%%%%%%%%%%%%%%%% -- DIVIDER - %%%%%%%%%%%%%%%%%%%
\begin{proof}[Proof of Theorem  \ref{thm:power_compB}]
	This result has several parts. 
	
	\underline{Part 1:} The same arguments used for SN method imply that Eq.\ \eqref{eq:JComparisonB} implies Eq.\ \eqref{eq:PowerComparisonBpre}.
	
	\underline{Part 2:} By definition and Lemma \ref{lem:LassoClosedForm},
		\begin{eqnarray*}
		\hat{J}_{B} &=&\{ j=1,\dots ,p:\hat{\mu}_{j}/\hat{\sigma}_{j}\geq -2c_{n}^{B,1S}( \beta _{n}) /\sqrt{n}\} , \\
		\hat{J}_{L} &=&\{ j=1,\dots ,p:\hat{\mu}_{j,L}/\hat{\sigma}_{j}\geq -\lambda _{n}\}=   \{j=1,\dots ,p:\hat{\mu}_{j}/\hat{\sigma}_{j}\geq -\lambda _{n}3/2\},
		\end{eqnarray*}

		Suppose that $\hat{J}_{L}\subseteq \hat{J}_{B}$ does not occur, i.e., $ \exists j\in \hat{J}_{L}\cap
	\hat{J}_{B}^{c}$ s.t.\ $-2c_{n}^{B}( \beta _{n}) /\sqrt{n}>\hat{\mu}_{j}/\hat{\sigma} _{j}\geq -\lambda _{n}3/2$. From this, we conclude that:
		\[
		\{ c_{n}^{B}( \beta _{n}) 4/3\geq \lambda _{n}\sqrt{n} \}  ~\subseteq~ \{ \hat{J}_{L}\subseteq \hat{J}_{B}\} .
		\]

		Let $c_{0}(3\beta _{n})$ denote the $(1-3\beta _{n})$-quantile of $ \max_{1\leq j\leq p}Y_{j}$ with $(Y_{1},\ldots ,Y_{p})\sim N(\mathbf{0},E [ ZZ'] )$ with $Z$ as in Assumption \ref{ass:Rates2}. In the remainder of this step, we consider two strategies to establish the following result:
		\begin{equation}
		c_{0}(3\beta _{n})4/3~\geq~ \lambda _{n}\sqrt{n}.  \label{eq:Strategies}
		\end{equation}
		Under Eq.\ \eqref{eq:Strategies}, we can conclude that:
		\[
		\{ c_{n}^{B}( \beta _{n}) \geq c_{0}(3\beta _{n})\} ~\subseteq~ \{ c_{n}^{B}( \beta _{n}) 4/3\geq \lambda _{n}\sqrt{n}\} ~\subseteq~ \{ \hat{J}_{L}\subseteq \hat{J} _{B}\} .
		\]
		From this and since $c_{0}(\cdot )$ is decreasing, we conclude that for any $\mu _{n}\leq 3\beta _{n}$,
	\begin{equation}
		P\del[1]{ \hat{J}_{L}\subseteq \hat{J}_{B}} ~\geq~ P\del[1]{c_{n}^{B}( \beta _{n}) ~\geq~ c_{0}(3\beta _{n})} ~\geq~ P\del[1]{ c_{n}^{B}( \beta _{n}) \geq c_{0}(\mu _{n})}.
		\label{eq:PowerGivenStrategies}
	\end{equation}

		To complete the proof, it suffices to provide a uniformly high lower bound for the RHS of Eq.\ \eqref{eq:PowerGivenStrategies}. To this end, we consider \citetalias{chernozhukov/chetverikov/kato:2014c} (Eq.\ (66)) at the following values: $\alpha =\beta _{n}$, $\nu _{n}=Cn^{-c}$, and $(\zeta _{n2},\zeta _{n1})$ s.t.\ $\zeta _{n2}+8\zeta _{n1}\sqrt{\ln p}\leq Cn^{-c}$. Under our assumptions, these choices yield $\mu _{n}\equiv \beta _{n}+\zeta _{n2}+v_{n}+8\zeta _{n1}\sqrt{\log p}\leq \beta _{n}+2Cn^{-c}\leq 3\beta _{n}$. By plugging these on \citetalias{chernozhukov/chetverikov/kato:2014c} (Eq.\ (66)), the RHS of Eq.\ \eqref{eq:PowerGivenStrategies} exceeds $1-Cn^{-c}$, as desired.

		To complete the proof the step, we now describe the two strategies that can be used to show Eq.\ \eqref{eq:Strategies}. The first strategy relies on Eq.\ \eqref{eq:2Ssuff} and the second strategy relies on Eq.\ \eqref{eq:2Ssuff2}.
	
		\underline{Strategy 1.} By definition, 
		\begin{equation}
		c_{0}(3\beta _{n})~\geq~ \Phi ^{-1}(1-3\beta _{n}),  \label{eq:P2_B}
		\end{equation}
		By combining Eqs.\ \eqref{eq:2Ssuff}, \eqref{eq:nlambda}, and \eqref{eq:P2_B}, it follows that:
		\[
		c_{0}(3\beta _{n})4/3~\geq~ \Phi ^{-1}(1-3\beta _{n})4/3~\geq~ \sqrt{2} (4/3+\varepsilon )n^{\delta /(2(2+\delta ))}M_{n,2+\delta }^{-1}\geq \sqrt{n} \lambda _{n}.
		\]

		\underline{Strategy 2.} First, the Borell-Cirelson-Sudakov inequality (see, e.g., \citet[Theorem 5.8]{boucheron/lugosi/massart:2013}), implies that for $x\geq 0$,
		\begin{equation}
		P\del[2]{\max_{1\leq j\leq p}Y_{j}~\leq~ E[ \max_{1\leq j\leq p}Y_{j} ] -x} ~\leq~ e^{-x^{2}/2},
		\label{eq:Piece0}
		\end{equation}
		where we used that the diagonal $E[ZZ']$ is a vector of ones. Equating the RHS of Eq.\ \eqref{eq:Piece0} to $(1-3\beta _{n})$ yields $x=\sqrt{ 2\log (1/[1-3\beta _{n}])}$ such that:
		\begin{equation}
		c_{0}(3\beta _{n})~\geq~ E[ \max_{1\leq j\leq p}Y_{j}] -\sqrt{2\log (1/[1-3\beta _{n}])}. \label{eq:Piece1}
		\end{equation}
		We now provide a lower bound for the first term on the RHS of Eq.\ \eqref{eq:Piece1}. Consider the following derivation:
		\begin{eqnarray}
		E[ \max_{1\leq j\leq p}Y_{j}] ~\geq~ \min_{i\neq j}\sqrt{ E(Y_{i}-Y_{j})^{2}\log (p)/2} ~\geq ~\sqrt{2(1-\rho )\log (p)/2},\label{eq:Piece2}
		\end{eqnarray}
		where the first inequality follows from Sudakov's minorization inequality (see, e.g., \citet[Theorem 13.4]{boucheron/lugosi/massart:2013}) and the second inequality follows from $E[ZZ']$ having a diagonal elements equal to one and the maximal absolute correlation less that $\rho $. Eqs.\ \eqref{eq:Piece1}-\eqref{eq:Piece2} imply that:
		\begin{equation}
		c_{0}(3\beta _{n})~\geq~ \sqrt{(1-\rho )\log (p)/2}-\sqrt{2\log (1/[1-3\beta _{n}])}. \label{eq:P2_C}
		\end{equation}
		By combining Eqs.\ \eqref{eq:2Ssuff2}, \eqref{eq:nlambda}, and \eqref{eq:P2_C}, it follows that:
		\begin{eqnarray*}
		c_{0}(3\beta _{n})4/3 \geq 4/3( \sqrt{(1-\rho )\log (p)/2}-\sqrt{ 2\log (1/[1-3\beta _{n}])})  \geq\sqrt{2}(4/3+\varepsilon )n^{\delta /(2(2+\delta ))}M_{n,2+\delta }^{-1}\geq \sqrt{n}\lambda _{n}.
		\end{eqnarray*}

	\underline{Part 3:} Consider the following argument.
	\begin{align*}
		P(T_{n}\geq c_{n}^{B,2S}( \alpha ) ) &=P(
		T_{n}\geq c_{n}^{B,2S}( \alpha ) \cap \hat{J}_{L}\subseteq \hat{J}_{B}) +P(T_{n}\geq c_{n}^{B,2S}( \alpha )\cap \hat{J}_{L}\not\subseteq \hat{J}_{B})\\
		&\leq  P(T_{n}\geq c_{n}^{B,L}( \alpha )) +P(\hat{J}_{L}\not\subseteq \hat{J}_{B})  \\
		&\leq P(T_{n}\geq c_{n}^{B,L}( \alpha )) + {C}n^{-{c}},
	\end{align*}
	where the first inequality uses part 1, and the second inequality uses that the sufficient conditions imply Eq.\ \eqref{eq:JComparisonBstock}.
\end{proof}

\end{small}
 % \newpage

\bibliography{BIBLIOGRAPHY.bib}
% \bibliography{../../../BIBLIOGRAPHY}

\end{document}
JoE6.bbl
\begin{thebibliography}{51}
\newcommand{\enquote}[1]{``#1''}
\expandafter\ifx\csname natexlab\endcsname\relax\def\natexlab#1{#1}\fi

\bibitem[\protect\citeauthoryear{Andrews, Berry, and {Jia-Barwick}}{Andrews
  et~al.}{2004}]{andrews/berry/jiabarwick:2004}
\textsc{Andrews, D. W.~K., S.~Berry, and P.~{Jia-Barwick}} (2004):
  \enquote{Confidence Regions for Parameters in Discrete Games with Multiple
  Equilibria with an Application to Discount Chain Store Location,} Mimeo: Yale
  University and M.I.T.

\bibitem[\protect\citeauthoryear{Andrews and Guggenberger}{Andrews and
  Guggenberger}{2009}]{andrews/guggenberger:2009}
\textsc{Andrews, D. W.~K. and P.~Guggenberger} (2009): \enquote{Validity of
  Subsampling and ``Plug-in Asymptotic'' Inference for Parameters Defined by
  Moment Inequalities,} \emph{Econometric Theory}, 25, 669--709.

\bibitem[\protect\citeauthoryear{Andrews and {Jia-Barwick}}{Andrews and
  {Jia-Barwick}}{2012}]{andrews/jiabarwick:2012}
\textsc{Andrews, D. W.~K. and P.~{Jia-Barwick}} (2012): \enquote{Inference for
  Parameters Defined by Moment Inequalities: A Recommended Moment Selection
  Procedure,} \emph{Econometrica}, 80, 2805--2826.

\bibitem[\protect\citeauthoryear{Andrews and Shi}{Andrews and
  Shi}{2013}]{andrews/shi:2013}
\textsc{Andrews, D. W.~K. and X.~Shi} (2013): \enquote{Inference Based on
  Conditional Moment Inequalities,} \emph{Econometrica}, 81, 609--666.

\bibitem[\protect\citeauthoryear{Andrews and Soares}{Andrews and
  Soares}{2010}]{andrews/soares:2010}
\textsc{Andrews, D. W.~K. and G.~Soares} (2010): \enquote{Inference for
  Parameters Defined by Moment Inequalities Using Generalized Moment
  Selection,} \emph{Econometrica}, 78, 119--157.

\bibitem[\protect\citeauthoryear{Armstrong}{Armstrong}{2014{\natexlab{a}}}]{armstrong:2015}
\textsc{Armstrong, T.~B.} (2014{\natexlab{a}}): \enquote{Asymptotically Exact
  Inference in Conditional Moment Inequality Models,} \emph{Journal of
  Econometrics}, 186, 51--65.

\bibitem[\protect\citeauthoryear{Armstrong}{Armstrong}{2014{\natexlab{b}}}]{armstrong:2014}
---\hspace{-.1pt}---\hspace{-.1pt}--- (2014{\natexlab{b}}): \enquote{Weighted
  {KS} Statistics for Inference on Conditional Moment Inequalities,}
  \emph{Journal of Econometrics}, 181, 92--116.

\bibitem[\protect\citeauthoryear{Belloni and Chernozhukov}{Belloni and
  Chernozhukov}{2011}]{belloni/chernozhukov:2011}
\textsc{Belloni, A. and V.~Chernozhukov} (2011): \emph{High dimensional sparse
  econometric models: An introduction}, Springer.

\bibitem[\protect\citeauthoryear{Beresteanu and Molinari}{Beresteanu and
  Molinari}{2008}]{beresteanu/molinari:2008}
\textsc{Beresteanu, A. and F.~Molinari} (2008): \enquote{Asymptotic Properties
  for a Class of Partially Identified Models,} \emph{Econometrica}, 76,
  763--814.

\bibitem[\protect\citeauthoryear{Bontemps, Magnac, and Maurin}{Bontemps
  et~al.}{2012}]{bontemps/magnac/maurin:2012}
\textsc{Bontemps, C., T.~Magnac, and E.~Maurin} (2012): \enquote{Set Identified
  Linear Models,} \emph{Econometrica}, 80, 1129--1155.

\bibitem[\protect\citeauthoryear{Boucheron, Lugosi, and Massart}{Boucheron
  et~al.}{2013}]{boucheron/lugosi/massart:2013}
\textsc{Boucheron, S., G.~Lugosi, and P.~Massart} (2013): \emph{Concentration
  Inequalities: A Nonasymptotic Theory of Independence}, Oxford University
  Press.

\bibitem[\protect\citeauthoryear{Bugni}{Bugni}{2010}]{bugni:2010}
\textsc{Bugni, F.~A.} (2010): \enquote{Bootstrap Inference in Partially
  Identified Models Defined by Moment Inequalities: Coverage of the Identified
  Set,} \emph{Econometrica}, 78, 735--753.

\bibitem[\protect\citeauthoryear{Bugni}{Bugni}{2015}]{bugni:2015}
---\hspace{-.1pt}---\hspace{-.1pt}--- (2015): \enquote{A comparison of
  inferential methods in partially identified models in terms of error in
  coverage probability (Formerly circulated as ``Bootstrap Inference in
  Partially Identified Models Defined by Moment Inequalities: Coverage of the
  Elements of the Identified Set''),} \emph{Econometric Theory}, FirstView,
  1--56.

\bibitem[\protect\citeauthoryear{Bugni, Canay, and Guggenberger}{Bugni
  et~al.}{2012}]{bugni/canay/guggenberger:2012}
\textsc{Bugni, F.~A., I.~A. Canay, and P.~Guggenberger} (2012):
  \enquote{Distortions of Asymptotic Confidence Size in Locally Misspecified
  Moment Inequality Models,} \emph{Econometrica}, 80, 1741--1768.

\bibitem[\protect\citeauthoryear{B{\"u}hlmann and {van de Geer}}{B{\"u}hlmann
  and {van de Geer}}{2011}]{buhlmann/vandegeer:2011}
\textsc{B{\"u}hlmann, P. and S.~{van de Geer}} (2011): \emph{Statistics for
  high-dimensional data: methods, theory and applications}, Springer Science \&
  Business Media.

\bibitem[\protect\citeauthoryear{Canay}{Canay}{2010}]{canay:2010}
\textsc{Canay, I.~A.} (2010): \enquote{{E.L.} Inference for Partially
  Identified Models: Large Deviations Optimality and Bootstrap Validity,}
  \emph{Journal of Econometrics}, 156, 408--425.

\bibitem[\protect\citeauthoryear{Caner}{Caner}{2009}]{caner:2009}
\textsc{Caner, M.} (2009): \enquote{Lasso Type GMM Estimator,}
  \emph{Econometric Theory}, 25, 270--290.

\bibitem[\protect\citeauthoryear{Caner and Fan}{Caner and
  Fan}{2015}]{caner/fan:2015}
\textsc{Caner, M. and Q.~M. Fan} (2015): \enquote{Hybrid GEL Estimators:
  Instrument Selection with Adaptive Lasso,} \emph{Journal of Econometrics},
  187, 256--274.

\bibitem[\protect\citeauthoryear{Caner, Han, and Lee}{Caner
  et~al.}{2016}]{caner/han/lee:2016}
\textsc{Caner, M., X.~Han, and Y.~Lee} (2016): \enquote{Adaptive elastic net
  GMM estimation with many Invalid Moment Conditions: Simultaneous Model and
  Moment Selection,} Forthcoming: Journal of Business and Economics Statistics.

\bibitem[\protect\citeauthoryear{Caner and Zhang}{Caner and
  Zhang}{2014}]{caner/zhang:2014}
\textsc{Caner, M. and H.~Zhang} (2014): \enquote{Adaptive Elastic Net GMM with
  Diverging Number of Moments,} \emph{Journal of Business and Economics
  Statistics}, 32, 30--47.

\bibitem[\protect\citeauthoryear{Cheng and Liao}{Cheng and
  Liao}{2015}]{cheng/liao:2015}
\textsc{Cheng, X. and Z.~Liao} (2015): \enquote{Select the Valid and Relevant
  Moment Conditions: A One-step Procedure for GMM with Many Moments,}
  \emph{Journal of Econometrics}, 186, 443--464.

\bibitem[\protect\citeauthoryear{Chernozhukov, Chetverikov, and
  Kato}{Chernozhukov
  et~al.}{2013{\natexlab{a}}}]{chernozhukov/chetverikov/kato:2013b}
\textsc{Chernozhukov, V., D.~Chetverikov, and K.~Kato} (2013{\natexlab{a}}):
  \enquote{Comparison and anti-concentration bounds for maxima of Gaussian
  random vectors,} Working paper. Forthcoming in Probability Theory Related
  Fields.

\bibitem[\protect\citeauthoryear{Chernozhukov, Chetverikov, and
  Kato}{Chernozhukov
  et~al.}{2013{\natexlab{b}}}]{chernozhukov/chetverikov/kato:2013a}
---\hspace{-.1pt}---\hspace{-.1pt}--- (2013{\natexlab{b}}): \enquote{Gaussian
  approximations and multiplier bootstrap for maxima of sums of
  high-dimensional random vectors,} \emph{Annals of Statistics}, 41,
  2786--2819.

\bibitem[\protect\citeauthoryear{Chernozhukov, Chetverikov, and
  Kato}{Chernozhukov
  et~al.}{2014{\natexlab{a}}}]{chernozhukov/chetverikov/kato:2014a}
---\hspace{-.1pt}---\hspace{-.1pt}--- (2014{\natexlab{a}}):
  \enquote{Anti-concentration and honest, adaptive confidence bands,}
  \emph{Annals of Statistics}, 42, 1787--1818.

\bibitem[\protect\citeauthoryear{Chernozhukov, Chetverikov, and
  Kato}{Chernozhukov
  et~al.}{2014{\natexlab{b}}}]{chernozhukov/chetverikov/kato:2014b}
---\hspace{-.1pt}---\hspace{-.1pt}--- (2014{\natexlab{b}}): \enquote{Central
  Limit Theorems and Bootstrap in High Dimensions,} Working paper.

\bibitem[\protect\citeauthoryear{Chernozhukov, Chetverikov, and
  Kato}{Chernozhukov
  et~al.}{2014{\natexlab{c}}}]{chernozhukov/chetverikov/kato:2014c}
---\hspace{-.1pt}---\hspace{-.1pt}--- (2014{\natexlab{c}}): \enquote{Testing
  Many Moment Inequalities,} Working paper.

\bibitem[\protect\citeauthoryear{Chernozhukov, Hong, and Tamer}{Chernozhukov
  et~al.}{2007}]{CHT07}
\textsc{Chernozhukov, V., H.~Hong, and E.~Tamer} (2007): \enquote{Estimation
  and Confidence Regions for Parameter Sets in Econometric Models,}
  \emph{Econometrica}, 75, 1243--1284.

\bibitem[\protect\citeauthoryear{Chernozhukov, Lee, and Rosen}{Chernozhukov
  et~al.}{2013{\natexlab{c}}}]{chernozhukov/lee/rosen:2013}
\textsc{Chernozhukov, V., S.~Lee, and A.~M. Rosen} (2013{\natexlab{c}}):
  \enquote{Intersection Bounds: Estimation and Inference,} \emph{Econometrica},
  81, 667--737.

\bibitem[\protect\citeauthoryear{Chetverikov}{Chetverikov}{2013}]{chetverikov:2013}
\textsc{Chetverikov, D.} (2013): \enquote{Adaptive Test of Conditional Moment
  Inequalities,} Unpublished manuscript.

\bibitem[\protect\citeauthoryear{Fan, Liao, and Yao}{Fan
  et~al.}{2015}]{fan/liao/yao:2015}
\textsc{Fan, J., Y.~Liao, and J.~Yao} (2015): \enquote{Power Enhancement in
  High-Dimensional Cross-Sectional Tests,} \emph{Econometrica}, 84, 1496--1541.

\bibitem[\protect\citeauthoryear{Fan, Lv, and Qi}{Fan
  et~al.}{2011}]{fan/lv/qi:2011}
\textsc{Fan, J., J.~Lv, and L.~Qi} (2011): \enquote{Sparse high dimensional
  models in economics,} \emph{Annual review of economics}, 3, 291--317.

\bibitem[\protect\citeauthoryear{Galichon and Henry}{Galichon and
  Henry}{2006}]{galichon/henry:2006}
\textsc{Galichon, A. and M.~Henry} (2006): \enquote{Inference in Incomplete
  Models,} Mimeo: Ecole Polytechnique, Paris - Department of Economic Sciences
  and Pennsylvania State University.

\bibitem[\protect\citeauthoryear{Galichon and Henry}{Galichon and
  Henry}{2013}]{galichon/henry:2013}
---\hspace{-.1pt}---\hspace{-.1pt}--- (2013): \enquote{Dilation Bootstrap: A
  methodology for constructing confidence regions with partially identified
  models,} \emph{Journal of Econometrics}, 177, 109--115.

\bibitem[\protect\citeauthoryear{Gordon}{Gordon}{1941}]{gordon:1941}
\textsc{Gordon, R.~D.} (1941): \enquote{Values of Mills' ratio of area to
  bounding ordinate and of the normal probability integral for large values of
  the argument,} \emph{The Annals of Mathematical Statistics}, 12, 364--366.

\bibitem[\protect\citeauthoryear{Hastie, Tibshirani, and Wainwright}{Hastie
  et~al.}{2015}]{hastie/tibshirani/wainwright:2015}
\textsc{Hastie, T., R.~Tibshirani, and M.~Wainwright} (2015): \emph{Statistical
  Learning with Sparsity: The Lasso and Generalizations}, CRC Press.

\bibitem[\protect\citeauthoryear{Imbens and Manski}{Imbens and
  Manski}{2004}]{imbens/manski:2004}
\textsc{Imbens, G. and C.~F. Manski} (2004): \enquote{Confidence Intervals for
  Partially Identified Parameters,} \emph{Econometrica}, 72, 1845--1857.

\bibitem[\protect\citeauthoryear{Kim}{Kim}{2008}]{kim:2008}
\textsc{Kim, K.} (2008): \enquote{Set Estimation and Inference with Models
  Characterized by Conditional Moment Inequalities,} Mimeo: Michigan State
  University.

\bibitem[\protect\citeauthoryear{Liao}{Liao}{2013}]{liao:2013}
\textsc{Liao, Z.} (2013): \enquote{Adaptive GMM Shrinkage Estimation with
  Consistent Moment Selection,} \emph{Econometric Theory}, 29, 1--48.

\bibitem[\protect\citeauthoryear{Manski}{Manski}{1995}]{manski:1995}
\textsc{Manski, C.~F.} (1995): \emph{Identification Problems in the Social
  Sciences}, Harvard University Press.

\bibitem[\protect\citeauthoryear{Menzel}{Menzel}{2009}]{menzel:2009}
\textsc{Menzel, K.} (2009): \enquote{Essays of set estimation and inference
  with moment inequalities,} Ph.D. Thesis dissertation, MIT.

\bibitem[\protect\citeauthoryear{Menzel}{Menzel}{2014}]{menzel:2014}
---\hspace{-.1pt}---\hspace{-.1pt}--- (2014): \enquote{Consistent estimation
  with many moment inequalities,} \emph{Journal of Econometrics}, 182,
  329–350.

\bibitem[\protect\citeauthoryear{Pakes, Porter, Ho, and Ishii}{Pakes
  et~al.}{2015}]{pakes/porter/ho/ishii:2015}
\textsc{Pakes, A., J.~Porter, K.~Ho, and J.~Ishii} (2015): \enquote{Moment
  Inequalities and Their Application,} \emph{Econometrica}, 83, 315--334.

\bibitem[\protect\citeauthoryear{Ponomareva}{Ponomareva}{2010}]{ponomareva:2010}
\textsc{Ponomareva, M.} (2010): \enquote{Inference in Models Defined by
  Conditional Moment Inequalities with Continuous Covariates,} Mimeo: Northern
  Illinois University.

\bibitem[\protect\citeauthoryear{Romano and Shaikh}{Romano and
  Shaikh}{2008}]{romano/shaikh:2008}
\textsc{Romano, J.~P. and A.~M. Shaikh} (2008): \enquote{Inference for
  Identifiable Parameters in Partially Identified Econometric Models,}
  \emph{Journal of Statistical Planning and Inference}, 138, 2786--2807.

\bibitem[\protect\citeauthoryear{Romano and Shaikh}{Romano and
  Shaikh}{2010}]{romano/shaikh:2010}
---\hspace{-.1pt}---\hspace{-.1pt}--- (2010): \enquote{Inference for the
  Identified Set in Partially Identified Econometric Models,}
  \emph{Econometrica}, 78, 169--211.

\bibitem[\protect\citeauthoryear{Romano, Shaikh, and Wolf}{Romano
  et~al.}{2014}]{romano/shaikh/wolf:2014}
\textsc{Romano, J.~P., A.~M. Shaikh, and M.~Wolf} (2014): \enquote{A Practical
  Two-Step Method for Testing Moment Inequalities,} \emph{Econometrica}, 82,
  1979--2002.

\bibitem[\protect\citeauthoryear{Rosen}{Rosen}{2008}]{rosen:2008}
\textsc{Rosen, A.~M.} (2008): \enquote{Confidence Sets for Partially Identified
  Parameters that Satisfy a Finite Number of Moment Inequalities,}
  \emph{Journal of Econometrics}, 146, 107--117.

\bibitem[\protect\citeauthoryear{Stoye}{Stoye}{2009}]{stoye:2009}
\textsc{Stoye, J.} (2009): \enquote{More on Confidence Intervals for Partially
  Identified Parameters,} \emph{Econometrica}, 77, 299--1315.

\bibitem[\protect\citeauthoryear{Tamer}{Tamer}{2003}]{tamer:2003}
\textsc{Tamer, E.} (2003): \enquote{Incomplete Simultaneous Discrete Response
  Model with Multiple Equilibria,} \emph{Review of Economic Studies}, 70,
  147--165.

\bibitem[\protect\citeauthoryear{Tibshirani}{Tibshirani}{1996}]{tibshirani:1996}
\textsc{Tibshirani, R.} (1996): \enquote{Regression shrinkage and selection via
  the lasso,} \emph{Journal of the Royal Statistical Society, Series B
  (Methodological)}, 58, 267--288.

\bibitem[\protect\citeauthoryear{Wang, Li, and Leng}{Wang
  et~al.}{2009}]{wang/li/leng:2009}
\textsc{Wang, H., B.~Li, and C.~Leng} (2009): \enquote{Shrinkage tuning
  parameter selection with a diverging number of parameters,} \emph{J. R.
  Statist. Soc. B}, 71, 671--683.

\end{thebibliography}
