Minimax Capacity Loss under Sub-Nyquist Universal Sampling
Yuxin Chen
Andrea J. Goldsmith
This paper considers the capacity of sub-sampled analog channels when the sampler is designed to operate independent of instantaneous channel realizations. A compound multiband Gaussian channel with unknown subband occupancy is considered, with perfect channel state information available at both the receiver and the transmitter. We restrict our attention to a general class of periodic sub-Nyquist samplers, which subsumes as special cases sampling with periodic modulation and filter banks. We evaluate the loss due to channel-independent (universal) sub-Nyquist design through a sampled capacity loss metric, that is, the gap between the undersampled channel capacity and the Nyquist-rate capacity. We investigate sampling methods that minimize the worst-case (minimax) capacity loss over all channel states. A fundamental lower bound on the minimax capacity loss is first developed, which depends only on the band sparsity ratio and the undersampling factor, modulo a residual term that vanishes at high signal-to-noise ratio. We then quantify the capacity loss under Landau-rate sampling with periodic modulation and low-pass filters, when the Fourier coefficients of the modulation waveforms are randomly generated and independent (resp. i.i.d. Gaussian-distributed), termed independent random sampling (resp. Gaussian sampling). Our results indicate that with exponentially high probability, independent random sampling and Gaussian sampling achieve minimax sampled capacity loss in the Landau-rate and super-Landau-rate regime, respectively. While identifying a deterministic minimax sampling scheme is in general intractable, our results highlight the power of randomized sampling methods, which are optimal in a universal design sense. Along the way, we derive concentration of several log-determinant functions, which might be of independent interest.
MinimaxSampling.tex
%% LyX 2.0.6 created this file.  For more info, see http://www.lyx.org/.
%% Do not edit unless you really know what you are doing.
\documentclass[10pt,english]{article}
\usepackage[T1]{fontenc}
\usepackage[latin9]{inputenc}
\usepackage{geometry}
\geometry{verbose,tmargin=1in,bmargin=1in,lmargin=1in,rmargin=1in}
\usepackage{array}
\usepackage{verbatim}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{graphicx}
\usepackage{esint}

\makeatletter

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% LyX specific LaTeX commands.
%% Because html converters don't know tabularnewline
\providecommand{\tabularnewline}{\\}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% User specified LaTeX commands.

\makeatother

\usepackage{amsthm}\usepackage{dsfont}\usepackage{array}\usepackage{mathrsfs}\usepackage{cite}\usepackage{comment}\usepackage{mathrsfs}
%\usepackage{hyperref}

\onecolumn

\makeatother

\usepackage{babel}

\makeatother

\usepackage{babel}
\begin{document}
\bibliographystyle{IEEEtran}


\title{Minimax Capacity Loss under Sub-Nyquist Universal Sampling}


\author{Yuxin Chen, Andrea J. Goldsmith, and Yonina C. Eldar%
\thanks{Y. Chen is with the Department of Electrical Engineering and the Department
of Statistics, Stanford University, Stanford, CA 94305, USA (email:
yxchen@stanford.edu). A. J. Goldsmith is with the Department of Electrical
Engineering, Stanford University, Stanford, CA 94305, USA (email:
andrea@ee.stanford.edu). Y. C. Eldar is with the Department of Electrical
Engineering, Technion, Israel Institute of Technology Haifa, Israel
32000 (email: yonina@ee.technion.ac.il). The contact author is Y.
Chen. This work was supported in part by the NSF under grant CCF-0939370
and CIS-1320628, the AFOSR under MURI Grant FA9550-09-1-0643, and
BSF Transformative Science Grant 2010505. It has been presented in
part at the IEEE International Symposium on Information Theory (ISIT)
2013. Manuscript date: \today.%
}}

\maketitle
\theoremstyle{plain}\newtheorem{lem}{\textbf{Lemma}}\newtheorem{theorem}{\textbf{Theorem}}\newtheorem{corollary}{\textbf{Corollary}}\newtheorem{prop}{\textbf{Proposition}}\newtheorem{fct}{\textbf{Fact}}\newtheorem{remark}{\textbf{Remark}}

\theoremstyle{definition}\newtheorem{definition}{\textbf{Definition}}\newtheorem{example}{\textbf{Example}}
\begin{abstract}
This paper considers the capacity of sub-sampled analog channels when
the sampler is designed to operate independent of instantaneous channel
realizations. A compound multiband Gaussian channel with unknown subband
occupancy is considered, with perfect channel state information available
at both the receiver and the transmitter. We restrict our attention
to a general class of periodic sub-Nyquist samplers, which subsumes
as special cases sampling with periodic modulation and filter banks. 

We evaluate the loss due to channel-independent (universal) sub-Nyquist
design through a \emph{sampled capacity loss} metric, that is, the
gap between the undersampled channel capacity and the Nyquist-rate
capacity. We investigate sampling methods that minimize the worst-case
(minimax) capacity loss over all channel states. A fundamental lower
bound on the minimax capacity loss is first developed, which depends
only on the band sparsity ratio and the undersampling factor, modulo
a residual term that vanishes at high signal-to-noise ratio. We then
quantify the capacity loss under Landau-rate sampling with periodic
modulation and low-pass filters, when the Fourier coefficients of
the modulation waveforms are randomly generated and independent (resp.
i.i.d. Gaussian-distributed), termed independent random sampling (resp.
Gaussian sampling). Our results indicate that with exponentially high
probability, independent random sampling and Gaussian sampling achieve
minimax sampled capacity loss in the Landau-rate and super-Landau-rate
regime, respectively. While identifying a deterministic minimax sampling
scheme is in general intractable, our results highlight the power
of randomized sampling methods, which are optimal in a universal design
sense. Along the way, we derive concentration of several log-determinant
functions, which might be of independent interest.
\end{abstract}
\textbf{Index Terms}: Channel capacity, sub-Nyquist sampling, universal
sampling, non-asymptotic random matrix, minimaxity, log-determinant,
concentration of measure, universality phenomenon, compressed sensing,
discrete-time sparse vector channel


\section{Introduction}

The maximum rate of information that can be conveyed through an analog
communication channel largely depends on the sampling technique and
rate employed at the receiver end. In wideband communication systems,
hardware and cost limitations often preclude sampling at or above
the Nyquist rate, which presents a major bottleneck in transferring
wideband and energy-efficient receiver design paradigms from theory
to practice. Understanding the effects upon capacity of sub-Nyquist
sampling is thus crucial in circumventing this bottleneck.

In practice, receiver hardware and, in particular, sampling mechanisms
are typically static and hence designed based on a family of possible
channel realizations. During operation, the actual channel realization
will vary over this class of channels. Since the sampler is typically
integrated into the hardware and difficult to change during system
operation, it needs to be designed independent of instantaneous channel
state information (CSI). This has no effect if the sampling rate employed
is commensurate with the maximum bandwidth (or the Nyquist rate) of
the channel family. However, at the sub-Nyquist sampling rate regime,
the sampler design significantly impacts the information rate achievable
over different channel realizations. As was shown in \cite{ChenEldarGoldsmith2012},
the capacity-maximizing sub-Nyquist sampling mechanism for a given
linear time-invariant (LTI) channel depends on specific channel realizations.
In time-varying channels, \emph{sampled capacity loss} relative to
the Nyquist-rate capacity is necessarily incurred due to channel-independent
(universal) sub-Nyquist sampler design. Moreover, it turns out that
the capacity-optimizing sampler for a given channel structure might
result in very low data rate for other channel realizations. 

In this paper, our goal is to explore universal design of a sub-Nyquist
sampling system that is robust against the uncertainty and variation
of instantaneous channel realizations, based on sampled capacity loss
as a metric. In particular, we investigate the fundamental lower limit
of sampled capacity loss in some overall sense (as will be detailed
as minimax capacity loss in Section \ref{sub:Universal-Sampling}),
and design a sub-Nyquist sampling system for which the capacity loss
can be uniformly controlled and optimized over all possible channel
realizations. 

\begin{comment}
For concreteness, we focus on a compound multiband Gaussian channel,
whereby at any timeframe only a subset of subbands are available to
the transmitter. Our goal is to explore universal design of a sub-Nyquist
sampling system that is robust against the uncertainty and variation
of instantaneous channel realizations, based on channel capacity as
a metric. 
\end{comment}



\subsection{Related Work}

In various scenarios, sampling at or above the Nyquist rate is not
necessary for preserving signal information if certain signal structures
are appropriately exploited \cite{ButSte1992,MisEld2011}. Take multiband
signals for example, that reside within several subbands over a wide
spectrum. If the spectral support is known, then the sampling rate
necessary for perfect signal reconstruction is the spectral occupancy,
termed the \textit{Landau rate} \cite{Landau1967}. Such signals admit
perfect recovery when sampled at rates approaching the Landau rate,
assuming appropriately chosen sampling sets (e.g. \cite{HerleyWong1999,VenkataramaniBresler2001}).
Inspired by recent ``compressive sensing'' \cite{CandRomTao06,Don2006,eldar2012compressed}
ideas, spectrum-blind sub-Nyquist samplers have also been developed
for multiband signals \cite{MisEld2010Theory2Practice}, pulse streams
\cite{BluDraVetMarCou2008,GedTurEld2010}, etc. These sampling-theoretic
works, however, were not based on capacity as a metric in the sampler
design.

On the other hand, the Shannon-Nyquist sampling theorem has frequently
been used to investigate analog waveform channels (e.g. \cite{Med2000,MedGal2002,Bello1963,BiglieriProakisShamai1998,ForUng1998}).
One key paradigm to determine or bound the channel capacity is converting
the continuous-time channel into a set of parallel discrete-time channels,
under the premise that sampling, when it is performed at or above
the Nyquist rate, preserves information. In addition, the effects
upon capacity of oversampling have been investigated in the presence
of quantization \cite{Sha1994,KochLap2010}. However, none of these
works considered the effect of reduced-rate sampling upon capacity.
Another recent line of work \cite{PelegShamai} investigated the tradeoff
between sparse coding and subsampling in AWGN channels, but did not
consider capacity-achieving input distributions. 

Our recent work \cite{ChenGolEld2010,ChenEldarGoldsmith2012} established
a new framework for investigating the capacity of linear time invariant
(LTI) Gaussian channels under a broad class of sub-Nyquist sampling
strategies, including filter-bank and modulation-bank sampling \cite{Papoulis1977,MisEld2010Theory2Practice}
and, more generally, time-preserving sampling. We showed that periodic
sampling or, more simply, sampling with a filter bank, is sufficient
to approach maximum capacity among all sampling structures under a
given sampling rate constraint, assuming that perfect CSI is available
at both the receiver and the transmitter. 

Practical communication systems often involve time-varying channels,
e.g. wireless slow fading channels \cite{BiglieriProakisShamai1998,ElGamalKim2012}.
Many of these channels can be modeled as a channel with state (see
a detailed survey in \cite[Chapter 7]{ElGamalKim2012}), where the
channel variation is captured by a state that may be fixed over a
long transmission block, or, more simply, a compound channel \cite{BlackwellBriemanThomasian1959}
whereby the channel realization lies within a collection of possible
channels \cite{EffrosGoldsmithLiang2010}. One class of compound channel
models concerns multiband Gaussian channels, whereby the instantaneous
frequency support active for transmission resides within several continuous
intervals, spread over a wide spectrum. This model naturally arises
in several wideband communication systems, including time division
multiple access systems and cognitive radio networks, as will be discussed
in Section \ref{sub:Compound-Multiband-Channel}. However, to the
best of our knowledge, no prior work has investigated, from a capacity
perspective, a universal (channel-independent) sub-Nyquist sampling
paradigm that is robust to channel variations in the above channel
models.

Finally, we note that the design of optimal sampling / sensing matrices
have recently been investigated in discrete-time settings from a information
theoretical perspective. In particular, Donoho \emph{et. al.} \cite{DonohoJavanmardMontanari2012}
assert that: random and band-diagonal sampling systems admit perfect
signal recovery from an information theoretically minimal number of
samples. However, the optimality was not defined based on channel
capacity as a metric, but instead based on a fundamental rate-distortion
limit \cite{WuVerdu2012}. 

\begin{comment}
While optimal Nyquist-rate sampling can be channel-independent since
it is information preserving, both the capacity-maximizing input and
sampling systems depend on specific channel realizations under sub-Nyquist
sampling. Although instantaneous CSI may be estimated during transmission
for compound channels, the sampling systems often need to be designed
based on all possible channel realizations rather than instantaneous
realizations. This arises from the fact that samplers are typically
integrated into the hardware and are difficult to change during system
operation. 
\end{comment}



\subsection{Main Contributions}

In this paper, we consider a compound multiband channel, whereby the
channel bandwidth $W$ is divided into $n$ continuous subbands and,
at each timeframe, only $k$ subbands are active for transmission.
We consider the class of periodic sampling systems (i.e. a system
that consists of a periodic preprocessor and a recurrent sampling
set, detailed in Section \ref{sub:Sampled-Channel-Capacity-Minimax})
with period $n/W$ and sampling rate $f_{\text{s}}=mW/n$ for some
integer $m$ ($m\leq n$). Under this model, we define $\beta:=k/n$
as the band sparsity ratio, and $\alpha:=m/n$ as the undersampling
factor. The sampling mechanism is termed a Landau-rate sampling (resp.
super-Landau-rate sampling) system if $f_{\text{s}}$ is equal to
(resp. greater than) the spectral size of the instantaneous channel
support. Our contributions are as follows.
\begin{enumerate}
\item We derive, in Theorem \ref{Theorem-Minimax-Det-Upper-Bound}, a fundamental
lower bound on the largest sampled capacity loss (defined in Section
\ref{sec:Problem-Formulation-Minimax}) incurred by any channel-independent
sampler, under both Landau-rate and super-Landau-rate sampling. This
lower bound depends only on the band sparsity ratio and the undersampling
factor, modulo a small residual term that vanishes when SNR and $n$
increase. The bound is derived by observing that at each frequency
within $[0,W/n]$, the exponential sum of the capacity loss over all
states $\boldsymbol{s}$ is independent of the sampling system, except
for a relatively small residual term that vanishes with SNR.
\item Theorem \ref{theorem-Minimax-Sampler-BWLimited-Achievability} characterizes
the sampled capacity loss under a class of periodic sampling with
periodic modulation (of period $n/W$) and low-pass filters with passband
$\left[0,W/n\right]$, when the Fourier coefficients of the modulation
waveforms are randomly generated and independent (termed \emph{independent
random sampling}). We demonstrate that with exponentially high probability,
the sampled capacity loss matches the fundamental lower bound of Theorem
\ref{Theorem-Minimax-Det-Upper-Bound} uniformly across all channel
realizations. This implies that independent sampling achieves the
minimum worst-case (or \emph{minimax}) capacity loss among all periodic
sampling methods with period $n/W$. To be more concrete, an independent
random sampling system achieves minimax sampled capacity loss as long
as the Fourier coefficients of the modulation waveforms are independently
sub-Gaussian \cite{Vershynin2012} distributed with matching moments
up to the second order. This universality phenomenon occurs due to
sharp concentration of spectral measures of large random matrices
\cite{GuionnetZeitouni2000}.
\item For a large portion of the super-Landau-rate regime, we quantify the
sampled capacity loss under independent random sampling when the Fourier
coefficients of the modulation waveforms are i.i.d. Gaussian-distributed
(termed \emph{Gaussian random sampling}), as stated in Theorem \ref{theorem-Minimax-Sampler-BWLimited-Achievability-Oversampling}.
With exponentially high probability, Gaussian random sampling achieves
minimax capacity loss among all periodic sampling with period $n/W$. 
\item Similar results for a discrete-time sparse vector channel with unknown
channel support can be delivered as an immediate consequence of our
analysis framework. When the number of measurements is equal to the
channel support size, independent random sensing matrices are minimax
in terms of channel-blind sampler design. When the sample size exceeds
the channel support size, Gaussian sensing matrices achieve minimax
sampled capacity loss.  
\item In order to establish the optimality of random sampling, we derive
sharp concentration of several log-determinant functions for i.i.d.
random matrix ensembles, which might be of independent interest for
other works involving log-determinant metrics. 
\end{enumerate}

\subsection{Organization}

The remainder of this paper is organized as follows. In Section \ref{sec:Problem-Formulation-Minimax},
we introduce our system model of compound multiband channels. A metric
called sampled capacity loss, and a minimax sampler are defined with
respect to sampled channel capacity. We then determine in Section
\ref{sec:Minimax-Capacity-Regret} the minimax capacity loss that
is achievable within the class of periodic sampling systems. Specifically,
we develop a lower bound on the minimax capacity loss in Section \ref{sub:The-Converse-Minimax}.
Its achievability under Landau-rate and super-Landau-rate sampling
are treated in Section \ref{sub:Achievability-Landau-rate-Minimax}
and Section \ref{sub:Achievability-Super-Landau-Minimax}, respectively.
Along the way, we derive concentration of several log-determinant
functions in Section \ref{sub:Concentration-of-Log-Determinant}.
Section \ref{sub:Implications-Minimax} summarizes the key observation
and implications from our results. We present in Section \ref{sec:Connections-with-Discrete-Time}
extensions to discrete-time sparse vector channels, and discuss connections
with compressed sensing literature. Section \ref{sec:Conclusion-Minimax}
closes the paper with a short summary of our findings and potential
future directions. 


\subsection{Notation}

We define the following two functions: $\log^{\epsilon}(x):=\log\left(\max\left(\epsilon,x\right)\right)$,
and $\det^{\epsilon}\boldsymbol{X}:=\prod_{i}\max\left(\lambda_{i}\left(\boldsymbol{X}\right),\epsilon\right)$.
Denote by $\mathcal{H}(\beta):=-\beta\log\beta-(1-\beta)\log(1-\beta)$
the binary entropy function, and $\mathcal{H}\left(\left\{ x_{1},\cdots,x_{n}\right\} \right):=-\sum_{i=1}^{n}x_{i}\log x_{i}$
the more general entropy function. The standard notation $f(n)=\mathcal{O}\left(g(n)\right)$
means there exists a constant $c$ (not necessarily positive) such
that $\left|f(n)\right|\leq cg(n)$, $f(n)=\Theta\left(g(n)\right)$
means there exist constants $c_{1}$ and $c_{2}$ such that $c_{1}g(n)\leq f(n)\leq c_{2}g(n)$,
$f(n)=\omega\left(g(n)\right)$ means that $\lim_{n\rightarrow\infty}\frac{g(n)}{f(n)}=0$,
and $f(n)=o\left(g(n)\right)$ indicates that $\lim_{n\rightarrow\infty}\frac{f(n)}{g(n)}=0$.
For a matrix $\boldsymbol{A}$, we use $\boldsymbol{A}_{i*}$ and
$\boldsymbol{A}_{*i}$ to denote the $i$th row and $i$th column
of $\boldsymbol{A}$, respectively. We let $[n]$ denote the set $\left\{ 1,2,\cdots,n\right\} $.
For any set $\mathcal{A}\subseteq[n]$, we denote by ${\mathcal{A} \choose k}$
the set of all $k$-combinations of $\mathcal{A}$. In particular,
we write ${[n] \choose k}$ for the set of all $k$-element subsets
of $\left\{ 1,2,\cdots,n\right\} $. We also use $\text{card}\left(\mathcal{A}\right)$
to denote the cardinality of a set $\mathcal{A}$. Let $\boldsymbol{W}$
be a $p\times p$ random matrix that can be expressed as $\boldsymbol{W}=\Sigma_{i=1}^{n}\boldsymbol{Z}_{i}\boldsymbol{Z}_{i}^{\top}$,
where $\boldsymbol{Z}_{i}\sim\mathcal{N}\left(0,\boldsymbol{\Sigma}\right)$
are jointly independent Gaussian vectors. Then $\boldsymbol{W}$ is
said to have a central Wishart distribution with $n$ degrees of freedom
and covariance matrix $\Sigma$, denoted as $\boldsymbol{W}\sim\mathcal{W}_{p}(n,\boldsymbol{\Sigma})$.
Our notation is summarized in Table \ref{tab:Summary-of-Notation-Nonuniform}. 

\begin{table}
\caption{\label{tab:Summary-of-Notation-Nonuniform}Summary of Notation and
Parameters}


\centering{}%
\begin{tabular}{l>{\raggedright}p{4.2in}}
$\mathcal{H}(x)$  & binary entropy function, i.e. $\mathcal{H}(x)=-x\log x-(1-x)\log(1-x)$ \tabularnewline
$h(t)$,$H(f)$  & impulse response, and frequency response of the LTI analog channel\tabularnewline
$\mathcal{S}_{\eta}(f)$ & power spectral density of the noise $\eta(t)$\tabularnewline
$f_{\text{s}}$, $T_{\text{s}}$ & aggregate sampling rate, and the corresponding sampling interval ($T_{\text{s}}=1/f_{\text{s}}$)\tabularnewline
$W$, $W_{0}$ & channel bandwidth, size of instantaneous channel support\tabularnewline
$n,m,k$ & number of subbands, number of sampling branches, number of subbands
being simultaneously active\tabularnewline
$\alpha=f_{\text{s}}/W=m/n$, $\beta=k/n$ & undersampling factor, sparsity ratio\tabularnewline
$\boldsymbol{Q}$, $\boldsymbol{Q}^{\text{w}}=\left(\boldsymbol{Q}\boldsymbol{Q}^{*}\right)^{-\frac{1}{2}}\boldsymbol{Q}$ & sampling matrix, whitened sampling matrix\tabularnewline
$L_{\boldsymbol{s}}^{\boldsymbol{Q}}$ & capacity loss associated with a sampling matrix $\boldsymbol{Q}$
given state $\boldsymbol{s}$\tabularnewline
$\boldsymbol{A}_{i*}$, $\boldsymbol{A}_{*i}$ & $i$th row of $\boldsymbol{A}$, $i$th column of $\boldsymbol{A}$\tabularnewline
$\text{card}\left(\mathcal{A}\right)$ & cardinality of a set $\mathcal{A}$\tabularnewline
$[n]$ & $[n]:=\left\{ 1,2,\cdots,n\right\} $\tabularnewline
${\mathcal{A} \choose k}$,${[n] \choose k}$  & set of all $k$-element subsets of $\mathcal{A}$, set of all $k$-element
subsets of $[n]$\tabularnewline
$\mathcal{W}_{p}\left(n,\boldsymbol{\Sigma}\right)$ & $p\times p$-dimensional central Wishart distribution with $n$ degrees
of freedom and covariance matrix $\boldsymbol{\Sigma}$ \tabularnewline
$\mathbb{Z}$, $\mathbb{R}$ & set of integers, set of real numbers\tabularnewline
\end{tabular}
\end{table}



\section{Problem Formulation and Preliminaries\label{sec:Problem-Formulation-Minimax}}


\subsection{Compound Multiband Channel\label{sub:Compound-Multiband-Channel}}

We consider a compound multiband Gaussian channel. The channel has
a total bandwidth $W$, and is divided into $n$ continuous subbands
each of bandwidth $W/n$. A state $\boldsymbol{s}\in{[n] \choose k}$
is generated, which dictates the channel support and realization%
\footnote{Note that in practice, $n$ is typically a large number. For instance,
the number of subcarriers ranges from 128 to 2048 in LTE \cite{ghosh2010fundamentals,LTEWhitePaper}.%
}. Specifically, given a state $\boldsymbol{s}$, the channel is an
LTI filter with impulse response $h_{\boldsymbol{s}}(t)$ and frequency
response $H_{\boldsymbol{s}}(f)=\int_{-\infty}^{\infty}h_{\boldsymbol{s}}(t)\exp(-j2\pi ft)\text{d}t$.
It is assumed throughout that there exists a general function $H(f,\boldsymbol{s})$
such that for every $f$ and $\boldsymbol{s}$, $H_{\boldsymbol{s}}(f)$
can be expressed as 
\[
H_{\boldsymbol{s}}(f)=H(f,\boldsymbol{s}){\bf 1}_{\boldsymbol{s}}(f),\quad\text{where}\quad\boldsymbol{1}_{\boldsymbol{s}}(f)=\begin{cases}
1,\quad & \text{if }f\text{ }\text{lies within subbands at indices from }\boldsymbol{s},\\
0, & \text{else}.
\end{cases}
\]


A transmit signal $x(t)$ with a power constraint $P$ is passed through
this multiband channel, which yields a channel output
\begin{equation}
r_{\boldsymbol{s}}(t)=h_{\boldsymbol{s}}(t)*x(t)+\eta(t),\label{eq:ChannelModel-1}
\end{equation}
where $\eta(t)$ is stationary zero-mean Gaussian noise with power
spectral density $\mathcal{S}_{\eta}(f)$. We assume that perfect
CSI is available at both the transmitter and the receiver. 

The above model subsumes as special cases the following communication
scenarios.
\begin{itemize}
\item \textbf{Time Division Multiple Access Model.} In this setting the
channel is shared by a set of different users. At each timeframe,
one of the users is selected for transmission. The receiver (e.g.
the base station) allocates a subset of subbands to the designated
sender over that timeframe. 
\item \textbf{White-Space Cognitive Radio Network.} In a white-space cognitive
radio network, cognitive users exploit spectrum holes unoccupied by
primary users and utilize them for communications. Since the locations
of the spectrum holes change over time, the spectral subbands available
to cognitive users is varying over time.
\end{itemize}

\subsection{Sampled Channel Capacity\label{sub:Sampled-Channel-Capacity-Minimax}}

We aim to design a sampler that works at rates below the Nyquist rate
(i.e. the channel bandwidth $W$). In particular, we consider the
class of periodic sampling systems, which subsume the most widely
used sampling mechanisms in practice.


\subsubsection{Periodic Sampling}

The class of periodic sampling systems is defined in \cite[Section IV]{ChenEldarGoldsmith2012},
which we restate as follows.

\begin{definition}[\textbf{Periodic Sampling}]\label{definition-PeriodicSampling}Consider
a sampling system consisting of a preprocessor with an impulse response
$q(t,\tau)$ followed by a sampling set $\Lambda=\left\{ t_{k}\mid k\in\mathbb{Z}\right\} $.
A linear sampling system is said to be periodic with period $T_{q}$
and sampling rate $f_{s}$ ($f_{s}T_{q}\in\mathbb{Z}$) if for every
$t,\tau\in\mathbb{R}$ and every $k\in\mathbb{Z}$, we have
\begin{equation}
q(t,\tau)=q(t+T_{q},\tau+T_{q});\quad\quad t_{k+f_{s}T_{q}}=t_{k}+T_{q}.
\end{equation}
\end{definition}

Consider a periodic sampling system $\mathcal{P}$ with period $T_{q}=n/W$
and sampling rate $f_{\text{s}}:=mW/n$ for some integer $m$. A special
case consists of sampling with a combination of filter banks and periodic
modulation with period $n/W$, as illustrated in Fig. \ref{fig:SamplingModulationFilter}(a).
Specifically, the sampling system comprises $m$ branches, where at
each branch, the channel output is passed through a pre-modulation
filter, modulated by a periodic waveform of period $T_{q}$, and then
filtered with a post-modulation filter followed by uniform sampling
at rate $f_{\text{s}}/m$. The channel capacity in a sub-sampled LTI
Gaussian channel has been derived in \cite[Theorem 5]{ChenEldarGoldsmith2012}.
As will be shown later, in the high SNR regime, employing water-filling
power allocation harvests only marginal capacity gain relative to
equal power allocation, as will be shown later. For this reason and
for mathematical convenience, we only restate below the sampled channel
capacity under uniform power allocation, which suffices for us to
bound the fundamental minimax limit as well as the convergence rate
with SNR. Specifically, if we denote by $\boldsymbol{s}_{i}$ and
$f_{i}$ the $i$th smallest element in $\boldsymbol{s}$ and the
lowest frequency of the $i$th subband, respectively, and define $\boldsymbol{H}_{\boldsymbol{s}}(f)$
as a $k\times k$ diagonal matrix obeying
\[
\left(\boldsymbol{H}_{\boldsymbol{s}}(f)\right)_{ii}=\frac{\left|H_{\boldsymbol{s}}\left(f_{\boldsymbol{s}_{i}}+f\right)\right|}{\sqrt{\mathcal{S}_{\eta}\left(f_{\boldsymbol{s}_{i}}+f\right)}},
\]
then the sampled channel capacity, when specialized to our setting,
is given as follows. 

\begin{theorem}[{\bf Sampled Capacity with Equal Power Allocation}\cite{ChenEldarGoldsmith2012}]\label{thm:PeriodicSampledCapacity}Consider
a channel with total bandwidth $W$ and instantaneous band sparsity
ratio $\beta:=\frac{k}{n}$. Assume perfect CSI at both the transmitter
and the receiver, and equal power allocation employed over active
subbands. If a periodic sampler $\mathcal{P}$ with period $n/W$
and sampling rate $f_{\text{s}}=\frac{m}{n}W$ is employed, then the
sampled channel capacity at a given state $\boldsymbol{s}$ is given
by
\begin{equation}
C_{\boldsymbol{s}}^{\boldsymbol{Q}}=\frac{1}{2}{\displaystyle \int}_{0}^{\frac{W}{n}}\log\det\left(\boldsymbol{I}_{m}+\frac{P}{\beta W}\boldsymbol{Q}^{\mathrm{w}}(f)\boldsymbol{H}_{\boldsymbol{s}}^{2}(f)\boldsymbol{Q}^{\mathrm{w}*}(f)\right)\mathrm{d}f,\label{eq:CapacityPeriodicSampling}
\end{equation}
where $\boldsymbol{Q}^{\mathrm{w}}(f):=\left(\boldsymbol{Q}(f)\boldsymbol{Q}^{*}(f)\right)^{-1/2}\boldsymbol{Q}(f)$.
Here, $\boldsymbol{Q}(f)$ is an $m\times n$ matrix that only depends
on $\mathcal{P}$, and $\boldsymbol{Q}_{\boldsymbol{s}}(f)$ denotes
the submatrix consisting of the columns of $\boldsymbol{Q}(f)$ at
indices of $\boldsymbol{s}$. \end{theorem}

In general, $\boldsymbol{Q}(f)$ is a function that varies with $f$.
Unless otherwise specified, we call $\boldsymbol{Q}(\cdot)$ the \emph{sampling
coefficient function} and $\boldsymbol{Q}^{\text{w}}(\cdot)$ the
\emph{whitened} \emph{sampling coefficient function} with respect
to the sampling system $\mathcal{P}$. Note that $\boldsymbol{Q}^{\text{w}}(f)\boldsymbol{Q}^{\text{w}*}(f)=\boldsymbol{I}$.

\begin{figure}
\begin{centering}
\begin{tabular}{cc}
\includegraphics[scale=0.35]{XamlingStructure.pdf} & \includegraphics[scale=0.35]{XamlingStructure_LPF.pdf}\tabularnewline
(a) & (b)\tabularnewline
\end{tabular}
\par\end{centering}

\caption{\label{fig:SamplingModulationFilter}(a) Sampling with modulation
and filter banks. The channel output $r(t)$ is passed through $m$
branches, each consisting of a pre-modulation filter, a periodic modulator
and a post-modulation filter followed by a uniform sampler with sampling
rate $f_{\text{s}}/m$. (b) Sampling with a modulation bank and low-pass
filters. The channel output $r(t)$ is passed through $m$ branches,
each consisting of a modulator with modulation waveform $q_{i}(t)$
and a low-pass filter of pass band $\left[0,f_{\text{s}}/m\right]$
followed by a uniform sampler with sampling rate $f_{\text{s}}/m$.}
\end{figure}



\subsubsection{Flat Sampling Coefficient Function}

A special class of periodic sampling concerns the ones whose $\boldsymbol{Q}(\cdot)$
are flat over $\left[0,f_{\text{s}}/m\right]$, in which case we can
use an $m\times n$ matrix $\boldsymbol{Q}$ to represent the sampling
coefficient function, termed a \emph{sampling coefficient matrix}.
This class of sampling systems can be realized through the $m$-branch
sampling system illustrated in Fig. \ref{fig:SamplingModulationFilter}(b).
In the $i$th branch, the channel output is modulated by a periodic
waveform $q_{i}(t)$ of period $n/W$, passed through a low-pass filter
with pass band $\left[0,f_{\text{s}}/m\right]$, and then uniformly
sampled at rate $f_{\text{s}}/m$, where the Fourier transform of
$q_{i}(t)$ obeys $\mathcal{F}\left(q_{i}(t)\right)=\sum_{l=1}^{n}\boldsymbol{Q}_{i,l}\delta\left(f-lW/n\right)$.
In this paper, a sampling system within this class is said to be \emph{(independent)
random sampling} if the entries of $\boldsymbol{Q}$ are randomly
generated (and are independent). In addition, a sampling system is
termed \emph{Gaussian sampling} if the entries of $\boldsymbol{Q}$
are i.i.d. Gaussian distributed.

It turns out that this simple class of sampling structures is sufficient
to achieve overall robustness in terms of sampled capacity loss, provided
that the entries of $\boldsymbol{Q}$ are sub-Gaussian with zero mean
and unit variance, as will be detailed in Section \ref{sec:Minimax-Capacity-Regret}. 


\subsection{Universal Sampling\label{sub:Universal-Sampling}}

As was shown in \cite{ChenEldarGoldsmith2012}, the optimal sampling
mechanism for a given LTI channel with perfect CSI extracts out a
frequency set with the highest SNR and hence suppresses aliasing.
Such alias-suppressing sampler may achieve very low capacity for some
channel realizations. In this paper, we desire a sampler that operates
independent of the instantaneous CSI, and our objective is to design
a single linear sampling system that achieves to within a minimal
gap of the Nyquist-rate capacity across all possible channel realizations.
The availability of CSI to the transmitter, the receiver and the sampler
is illustrated in Fig. \ref{fig:ProblemFormulation-Minimax}.

\begin{comment}
Consider for instance two states $\boldsymbol{s}$ and $\hat{\boldsymbol{s}}$
such that $\boldsymbol{s}\cap\hat{\boldsymbol{s}}=\emptyset$. The
sampler optimized for $\boldsymbol{s}$ achieves zero capacity under
state $\hat{\boldsymbol{s}}$, and the rate loss relative to Nyquist-rate
capacity scales in SNR. 
\end{comment}


\vspace{10pt}


\begin{figure}[htbp]
\begin{centering}
\textsf{\includegraphics[scale=0.5]{ProblemFormulationState}} 
\par\end{centering}

\caption{\label{fig:ProblemFormulation-Minimax}At each timeframe, a state
is generated from a finite set $\mathcal{S}$, which dictates the
channel realization $H_{\boldsymbol{s}}(f)$. Both the transmitter
and the receiver have perfect CSI, while the sampler operates independently
of $\boldsymbol{s}$.}
\end{figure}



\subsubsection{Sampled Capacity Loss}

Universal sub-Nyquist samplers suffer from information rate loss relative
to Nyquist-rate capacity. In this subsection, we make formal definitions
of this metric.

For any state $\boldsymbol{s}$, when equal power allocation is performed
over active subbands, the Nyquist-rate capacity can be written as
\begin{align}
C_{\boldsymbol{s}}^{P_{\text{eq}}} & ={\displaystyle \int}_{0}^{W/n}\frac{1}{2}\log\det\left(\boldsymbol{I}_{k}+\frac{P}{\beta W}\boldsymbol{H}_{\boldsymbol{s}}^{2}(f)\right)\mathrm{d}f,\label{eq:MatrixFormCs}
\end{align}
which is a special case of (\ref{eq:CapacityPeriodicSampling}). In
contrast, if power control at the transmitter side is allowed, then
the Nyquist-rate capacity is given by
\begin{align}
C_{\boldsymbol{s}}^{\text{opt}} & ={\displaystyle \int}_{0}^{W/n}\frac{1}{2}\sum_{i=1}^{k}\log^{+}\left(\nu\left(\boldsymbol{H}_{\boldsymbol{s}}(f)\right)_{ii}^{2}\right)\mathrm{d}f,\label{eq:MatrixFormCsWaterFilling}
\end{align}
where $\nu$ is determined by the equation
\begin{align}
P & ={\displaystyle \int}_{0}^{W/n}\sum_{i=1}^{k}\left(\nu-\frac{1}{\left(\boldsymbol{H}_{\boldsymbol{s}}(f)\right)_{ii}^{2}}\right)^{+}\mathrm{d}f.\label{eq:PowerLevelWaterFilling}
\end{align}
We can then make formal definitions of sampled capacity loss as follows.

\begin{definition}[{\bf Sampled Capacity Loss}]For any sampling coefficient
function $\boldsymbol{Q}(\cdot)$ and any given state $\boldsymbol{s}$,
we define the sampled capacity loss without power control as
\[
L_{\boldsymbol{s}}^{\boldsymbol{Q}}:=C_{\boldsymbol{s}}^{P_{\text{eq}}}-C_{\boldsymbol{s}}^{\boldsymbol{Q}},
\]
and define the sampled capacity loss with optimal power control as
\[
L_{\boldsymbol{s}}^{\boldsymbol{Q},\text{opt}}:=C_{\boldsymbol{s}}^{\text{opt}}-C_{\boldsymbol{s}}^{\boldsymbol{Q}}.
\]
\end{definition}These metrics quantify the capacity gaps relative
to Nyquist-rate capacity due to \emph{universal} (channel-independent)
sub-Nyquist sampling design. When sampling is performed at or above
the Landau rate (which is equal to $kW/n$ in our case) but below
the Nyquist rate, these gaps capture the rate loss due to channel-independent
sampling relative to channel-optimized design, either with or without
power control. 

For notational convenience, for an $m\times n$ matrix $\boldsymbol{M}$,
we denote by $L_{\boldsymbol{s}}^{\boldsymbol{M}}$ and $L_{\boldsymbol{s}}^{\boldsymbol{M},\text{opt}}$
the capacity loss with respect to a sampling coefficient function
$\boldsymbol{Q}(f)\equiv\boldsymbol{M}$, which is flat across $\left[0,W/n\right]$. 


\subsubsection{Minimax sampler}

Frequently used in the theory of statistics (e.g. \cite{LehmannCasella1998}),
minimaxity is a metric that seeks to minimize the loss function in
some overall sense, defined as follows.

\begin{definition}[{\bf Minimax Sampler}]A sampling system associated
with a sampling coefficient function $\boldsymbol{Q}^{\text{m}}$,
which minimizes the worst-case capacity loss, that is, which satisfies
\[
\max_{\boldsymbol{s}\in{[n] \choose k}}L_{\boldsymbol{s}}^{\boldsymbol{Q}^{\text{m}}}=\inf_{\boldsymbol{Q}(\cdot)}\max_{\boldsymbol{s}\in{[n] \choose k}}L_{\boldsymbol{s}}^{\boldsymbol{Q}},
\]
is called a minimax sampler with respect to the state alphabet ${[n] \choose k}$.\end{definition}

The minimax criteria is of interest for designing a sampler robust
to all possible channel states, that is, achieving to within a minimal
gap relative to maximum capacity for all channel realizations. It
aims to control the rate loss across all states in a uniform manner,
as illustrated in Fig. \ref{fig:CapacityLoss}. Note that the minimax
sampler is in general different from the one that maximizes the lowest
capacity among all states (worst-case capacity). While the latter
guarantees an optimal worst-case capacity that can be achieved regardless
of which channel is realized, it may result in significant capacity
loss in many states with large Nyquist-rate capacity, as illustrated
in Fig. \ref{fig:CapacityLoss}. In contrast, a desired minimax sampler
controls the capacity loss for every single state $\boldsymbol{s}$,
and allows for robustness over all channel states with universal channel-independent
sampling. It turns out that in the compound multiband channels,
\[
\forall\boldsymbol{s},\quad L_{\boldsymbol{s}}^{\boldsymbol{Q}^{\text{m}}}=\max_{\tilde{\boldsymbol{s}}\in{[n] \choose k}}L_{\tilde{\boldsymbol{s}}}^{\boldsymbol{Q}^{\text{m}}}
\]
except for some vanishingly small residual terms, as will be shown
in the next section.

\begin{figure}
\begin{centering}
\emph{\includegraphics[scale=0.35]{CapacityLossGeneral}}
\par\end{centering}

\caption{\label{fig:CapacityLoss}Minimax sampler v.s. the sampler that maximizes
worst-case capacity, when sampling is channel-independent and performed
below the Nyquist rate. The blue solid line represents the Nyquist-rate
(analog) capacity, the black dotted line represents the capacity achieved
by minimax sampler, the orange dashed illustrates the Nyquist-rate
capacity minus the minimax capacity loss, while the purple dashed
line corresponds to maximum worst-case capacity. }
\end{figure}


\begin{comment}
aa


\subsubsection{Bayes sampler}

Another way to minimize this risk function in an overall sense is
to consider average risk optimality \cite{LehmannCasella1998}, which
concerns minimizing the average capacity loss under some prior measure
$\mu$ on ${\bf s}$. The corresponding sampler based on Bayesian
statistics is defined as follows.

\begin{definition}A sampling system associated with a sampling matrix
$\boldsymbol{Q}^{\text{B}}$, which minimizes the average capacity
loss under a prior distribution $\mu$ on $\boldsymbol{s}$, that
is, which satisfies
\[
{\displaystyle \int}L_{\boldsymbol{s}}^{\boldsymbol{Q}^{\text{B}}}\mathrm{d}\mu(\boldsymbol{s})=\inf_{\boldsymbol{Q}}{\displaystyle \int}L_{\boldsymbol{s}}^{\boldsymbol{Q}}\mathrm{d}\mu(\boldsymbol{s}),
\]
is called a Bayes sampling system.\end{definition}

The Bayes sampler optimizes the risk based on prior information. In
fact, the Bayes sampler is equivalent to maximizing under the prior
distribution $\mu$ the Bayesian capacity metric
\[
C_{\mu}:={\displaystyle \int}C_{\boldsymbol{s}}^{\boldsymbol{Q}}\mathrm{d}\mu(\boldsymbol{s})
\]
over all sampling matrices $\boldsymbol{Q}$. This metric quantifies
the maximum number of bits per second that, for a given sampler $\mathcal{P}$,
can be conveyed through the channel with zero probability of error
under the assumptions that
\begin{itemize}
\item the channel support is drawn from $\mu(\cdot)$; 
\item both the transmitter and the receiver know the instantaneous channel
support, but only the receiver knows instantaneous CSI; 
\item the sampler is oblivious to instantaneous channel states. \end{itemize}
\end{comment}



\section{Minimax Sampled Capacity Loss\label{sec:Minimax-Capacity-Regret}}

The minimax sampled capacity loss problem can be cast as minimizing
$\max_{\boldsymbol{s}\in\mathcal{S}}L_{\boldsymbol{s}}^{\boldsymbol{Q}}$
over all sampling coefficient functions $\boldsymbol{Q}(f)$. In general,
the problem is non-convex in $\boldsymbol{Q}(f)$, and hence it is
difficult to identify the optimal sampling systems. Nevertheless,
the minimax capacity loss can be quantified reasonably well at moderate-to-high
SNR. It turns out that under both Landau-rate sampling and a large
regime of super-Landau-rate sampling, the minimax capacity loss can
be well approached by random sampling. 

Define the undersampling factor $\alpha:=m/n$, and recall that the
band sparsity ratio is $\beta:=k/n$. Our main results are summarized
in the following theorem.

\begin{theorem}\label{theorem-Minimax-Final}Consider any sampling
coefficient function $\boldsymbol{Q}(\cdot)$ with an undersampling
factor $\alpha$, and let the sparsity ratio be $\beta$. Define 
\begin{align}
\mathsf{SNR}_{\min} & :=\frac{P}{\beta W}\inf_{0\leq f\leq W,\boldsymbol{s}\in{[n] \choose k}}\frac{\left|H(f,\boldsymbol{s})\right|^{2}}{\mathcal{S}_{\eta}(f)},\label{eq:DefnSNRmin}\\
\mathsf{SNR}_{\max} & :=\frac{P}{\beta W}\sup_{0\leq f\leq W,\boldsymbol{s}\in{[n] \choose k}}\frac{\left|H(f,\boldsymbol{s})\right|^{2}}{\mathcal{S}_{\eta}(f)},\label{eq:DefnSNRmax}
\end{align}
and suppose that $\mathsf{SNR}_{\max}\geq1$.

(i) (Landau-rate sampling) If $\alpha=\beta$ (or $k=m$), then
\begin{equation}
\inf_{\boldsymbol{Q}}\max_{\boldsymbol{s}\in{[n] \choose k}}L_{\boldsymbol{s}}^{\boldsymbol{Q}}=\frac{W}{2}\left\{ \mathcal{H}\left(\beta\right)+\mathcal{O}\left(\min\left\{ \frac{\sqrt{\small\mathsf{SNR}_{\max}}}{\sqrt{n}},\frac{\log n+\log\small\mathsf{SNR}_{\max}}{n^{1/4}}\right\} \right)+\Delta_{\mathrm{L}}\right\} ,\label{eq:MinimaxCapacityGapLandauFinal}
\end{equation}
where
\[
-\frac{2}{\sqrt{\small\mathsf{SNR}_{\min}}}\leq\Delta_{\mathrm{L}}\leq\frac{\beta}{\small\mathsf{SNR}_{\min}}.
\]


(ii) (Super-Landau-rate sampling) Suppose that there is a small constant
$\delta>0$ such that $\alpha-\beta\geq\delta$ and $1-\alpha-\beta\geq\delta$.
Then
\begin{equation}
\inf_{\boldsymbol{Q}}\max_{\boldsymbol{s}\in{[n] \choose k}}L_{\boldsymbol{s}}^{\boldsymbol{Q}}=\frac{W}{2}\left\{ \mathcal{H}\left(\beta\right)-\alpha\mathcal{H}\left(\frac{\beta}{\alpha}\right)+\mathcal{O}\left(\frac{\log n}{n^{1/3}}\right)+\Delta_{\mathrm{SL}}\right\} ,\label{eq:MinimaxCapacityGapSuperLandauFinal}
\end{equation}
where
\[
-\frac{2}{\sqrt{\small\mathsf{SNR}_{\min}}}\leq\Delta_{\mathrm{SL}}\leq\frac{\beta}{\small\mathsf{SNR}_{\min}}.
\]
\end{theorem}

\begin{remark}Note that $\mathcal{H}(\cdot)$ denotes the binary
entropy function. Its appearance is due to the fact that it is a tight
estimate of the rate function of binomial coefficients. \end{remark}

Theorem \ref{theorem-Minimax-Final} provides a tight characterization
of the minimax sampled capacity loss relative to the Nyquist-rate
capacity, under both Landau-rate sampling and super-Landau-rate sampling.
Note that the Landau-rate sampling regime in (i) is not a special
case of the super-Landau-rate regime considered in (ii). For instance,
if $\beta>1/2$, then $\alpha+\beta>1$, which falls within a regime
not accounted for by Theorem \ref{theorem-Minimax-Final}(ii).

The capacity loss expressions (\ref{eq:MinimaxCapacityGapLandauFinal})
and (\ref{eq:MinimaxCapacityGapSuperLandauFinal}) contain residual
terms that vanish for large $n$ and high SNR. In the regime where
$\frac{\small\mathsf{SNR}_{\max}}{n}\ll1$ and $\small\mathsf{SNR}_{\min}\gg1$,
these fundamental minimax limits are approximately equal to a constant
modulo a vanishing residual term. Since the Nyquist rate capacity
scales as $\Theta\left(W\log\mathsf{SNR}\right)$, our results indicate
that the ratio of the minimax capacity loss to the Nyquist-rate capacity
vanishes at a rate $\Theta\left(1/\log\mathsf{SNR}\right)$. 

Note that even if we allow power control at the transmitter side,
the results are still valid at high SNR. This is summarized in the
following theorem. 

\begin{theorem}\label{theorem-Minimax-Final-water-filling}Consider
the metric $L_{\boldsymbol{s}}^{\boldsymbol{Q},\mathrm{opt}}$ with
power control. Under all other conditions of Theorem \ref{theorem-Minimax-Final},
the bounds (\ref{eq:MinimaxCapacityGapLandauFinal}) and (\ref{eq:MinimaxCapacityGapSuperLandauFinal})
(with $L_{\boldsymbol{s}}^{\boldsymbol{Q}}$ replaced by $L_{\boldsymbol{s}}^{\boldsymbol{Q},\mathrm{opt}}$)
continue to hold if $\Delta_{\mathrm{L}}$ and $\Delta_{\mathrm{SL}}$
are respectively replaced by some residuals $\Delta_{\mathrm{L}}^{\mathrm{opt}}$
and $\Delta_{\mathrm{SL}}^{\mathrm{opt}}$ that satisfy
\[
-\frac{2}{\sqrt{\small\mathsf{SNR}_{\min}}}\leq\Delta_{\mathrm{L}}^{\mathrm{opt}},\Delta_{\mathrm{SL}}^{\mathrm{opt}}\leq\frac{\beta+\overline{A}}{\small\mathsf{SNR}_{\min}},
\]
where $\overline{A}$ is a constant defined as
\[
\overline{A}:=\min\left\{ \frac{\max_{\boldsymbol{s}\in{[n] \choose k}}\int_{0}^{W}\frac{\left|H(f,\boldsymbol{s})\right|^{2}}{\mathcal{S}_{\eta}(f)}\mathrm{d}f}{\beta W\inf_{0\leq f\leq W,\boldsymbol{s}\in{[n] \choose k}}\frac{\left|H(f,\boldsymbol{s})\right|^{2}}{\mathcal{S}_{\eta}(f)}},\frac{\sup_{0\leq f\leq W,\boldsymbol{s}\in{[n] \choose k}}\frac{\left|H(f,\boldsymbol{s})\right|^{2}}{\mathcal{S}_{\eta}(f)}}{\inf_{0\leq f\leq W,\boldsymbol{s}\in{[n] \choose k}}\frac{\left|H(f,\boldsymbol{s})\right|^{2}}{\mathcal{S}_{\eta}(f)}}\right\} .
\]
\end{theorem}

Theorem \ref{theorem-Minimax-Final-water-filling} demonstrates that
if the average-to-minimum ratio $\frac{\overline{\small\mathsf{SNR}}}{\small\mathsf{SNR}_{\min}}$
is bounded by a constant (where $\overline{\mathsf{SNR}}:=\max_{\boldsymbol{s}\in{[n] \choose k}}\frac{P}{\beta W}\int_{0}^{W}\frac{\left|H(f,\boldsymbol{s})\right|^{2}}{\mathcal{S}_{\eta}(f)}\mathrm{d}f$),
then the minimax sampled capacity gap with power control remains almost
the same as that with power control within a vanishingly small gap
per unit bandwidth at high SNR and large $n$. Note that the constant
$\overline{A}$ given in Theorem \ref{theorem-Minimax-Final-water-filling}
is fairly conservative, and can be refined with finer tuning or algebraic
techniques.

Theorem \ref{theorem-Minimax-Final-water-filling} can be delivered
as an immediate consequence of Theorem \ref{theorem-Minimax-Final}
if we can quantify the gap between $C_{\boldsymbol{s}}^{P_{\text{eq}}}$
and $C_{\boldsymbol{s}}^{\mathrm{opt}}$. In fact, the capacity benefits
of using power control at high SNR regime is no larger than $\mathcal{O}(\mathsf{SNR}^{-1})$
per unit bandwidth. See Appendix \ref{sec:Proof-of-Theorem-Minimax-Final-water-filling}
for details. For this reason, our analysis is mainly devoted to $L_{\boldsymbol{s}}^{\boldsymbol{Q}}$,
which corresponds to the capacity loss relative to Nyquist-rate capacity
with uniform power allocation. 

The proof of Theorem \ref{theorem-Minimax-Final} involves the verification
of two parts: a converse part that provides a fundamental lower bound
on the minimax sampled capacity loss, and an achievability part that
provides a sampling scheme to approach this bound. As we show, the
class of sampling systems with periodic modulation followed by low-pass
filters, as illustrated in Fig. \ref{fig:SamplingModulationFilter}(b),
is sufficient to approach the minimax sampled capacity loss. 

Throughout the remainder of the paper, we suppose that the noise is
of unit power spectral density $\mathcal{S}_{\eta}(f)\equiv1$ unless
otherwise specified. Note that this incurs no loss of generality since
we can always include a noise-whitening LTI filter at the first stage
of the sampling system. 

\begin{comment}
\begin{theorem}\label{theorem-Minimax-Sampler}Let ${\bf G}$ be
drawn from a $K\times N$ dimensional Gaussian ensemble where each
entry is i.i.d. drawn from $\mathcal{N}\left(0,1/N\right)$. Then
if $N\gg K$, the sampler associated with the coefficient matrix
\[
{\bf Q}^{\text{m}}=\left({\bf G}{\bf G}^{T}\right)^{-\frac{1}{2}}{\bf G}
\]
is approximately minimax with respect to the capacity loss risk function. 

\end{theorem}

\begin{proof}

Under our assumptions, the sampled output satisfies 
\[
\left[\begin{array}{cccc}
q_{11} & q_{12} & \cdots & q_{1N}\\
q_{21} & q_{22} & \cdots & q_{2N}\\
\vdots & \vdots & \vdots & \vdots\\
q_{K1} & q_{K2} & \cdots & q_{kN}
\end{array}\right]\left[\begin{array}{cccc}
H_{1}\\
 & H_{2}\\
 &  & \ddots\\
 &  &  & H_{N}
\end{array}\right]\left[\begin{array}{c}
X_{1}\\
X_{2}\\
\vdots\\
X_{N}
\end{array}\right]+\left[\begin{array}{c}
\tilde{\eta}_{1}\\
\tilde{\eta}_{2}\\
\vdots\\
\tilde{\eta}_{K}
\end{array}\right]
\]
where $\eta$ is white noise. Denote by $[s]$ the index set associated
with the state sequence $s$, the capacity loss given state $s$ becomes
\[
\frac{W}{2N}\log\det\left(I+\frac{P}{W}{\bf Q}_{[s]}{\bf H}_{[s]}^{2}{\bf Q}_{[s]}^{T}\right)-\frac{W}{2N}\log\det\left(I+\frac{P}{W}{\bf H}_{[s]}^{2}\right),
\]
where ${\bf Q}_{[s]}$ denotes a $K$-dimensional squared matrix consisting
of all columns with index belonging to the index set $[s]$, and ${\bf H}_{[s]}$
denotes the $K$-dimensional diagonal matrix consisting of $\left\{ H_{i}\mid i\in[s]\right\} $.
In the high SNR regime, this capacity loss can be approximated by
\begin{align*}
 & -\frac{W}{2N}\log\det\left(I+\frac{P}{W}{\bf Q}_{[s]}{\bf H}_{[s]}^{2}{\bf Q}_{[s]}^{T}\right)+\frac{W}{2N}\log\det\left(I+\frac{P}{W}{\bf H}_{[s]}^{2}\right)\\
\approx & -\frac{W}{2N}\log\det\left(\frac{P}{W}{\bf Q}_{[s]}{\bf H}_{[s]}^{2}{\bf Q}_{[s]}^{T}\right)+\frac{W}{2N}\log\det\left(\frac{P}{W}{\bf H}_{[s]}^{2}\right)=-\frac{W}{2N}\log\det\left({\bf Q}_{[s]}{\bf Q}_{[s]}^{T}\right)-\frac{W}{2N}\log\det\left(\frac{P}{W}{\bf H}_{[s]}^{2}\right)+\frac{W}{2N}\log\det\left(\frac{P}{W}{\bf H}_{[s]}^{2}\right)\\
= & -\frac{W}{2N}\log\det\left({\bf Q}_{[s]}{\bf Q}_{[s]}^{T}\right)=-\frac{WK}{2N}\frac{1}{K}\log\det\left({\bf Q}_{[s]}{\bf Q}_{[s]}^{T}\right)\\
= & -\frac{f_{s}}{2}\frac{1}{K}\log\det\left({\bf Q}_{[s]}{\bf Q}_{[s]}^{T}\right)
\end{align*}
 Therefore, our goal becomes how to evaluate 
\[
\max_{{\bf Q}}\quad\min_{s\in\mathcal{S}_{K}}\det\left({\bf Q}_{[s]}{\bf Q}_{[s]}^{T}\right)
\]
The following lemma provides an upper bound on $\max_{{\bf Q}}\min_{[s]}\det\left({\bf Q}_{[s]}{\bf Q}_{[s]}^{T}\right)$,
whose proof can be found in Appendix \ref{sec:Proof-of-Theorem-Minimax-Det-Upper-Bound}

\end{proof}

\textbf{Implications}: The sampling coefficient matrix generated from
i.i.d. Gaussian ensembles is near minimax. Although a closed-form
minimax coefficient matrix is not easy to get, we can easily generate
it using a random matrix while still achieving near-optimal performance.
\end{comment}



\subsection{The Converse \label{sub:The-Converse-Minimax}}

We need to show that the minimax sampled capacity loss under any channel-independent
sampler cannot be lower than (\ref{eq:MinimaxCapacityGapLandauFinal})
and (\ref{eq:MinimaxCapacityGapSuperLandauFinal}). This is given
by the following theorem, which takes into account the entire regime
including the situation where $\alpha+\beta>1$.

\begin{theorem}\label{Theorem-Minimax-Det-Upper-Bound}Consider any
Riemann-integrable sampling coefficient function $\boldsymbol{Q}(\cdot)$
with an undersampling factor $\alpha:=m/n$. Suppose the sparsity
ratio $\beta:=k/n$ satisfies $\beta\leq\alpha\leq1$. The minimax
capacity loss can be lower bounded by
\begin{equation}
\inf_{\boldsymbol{Q}}\max_{\boldsymbol{s}\in{[n] \choose k}}L_{\boldsymbol{s}}^{\boldsymbol{Q}}\geq\frac{W}{2}\left\{ \mathcal{H}\left(\beta\right)-\alpha\mathcal{H}\left(\frac{\beta}{\alpha}\right)-\frac{2}{\sqrt{\small\mathsf{SNR}_{\min}}}-\frac{\log\left(n+1\right)}{n}\right\} .\label{eq:MinimaxCapacityGapLowerBoundLargeN}
\end{equation}


\end{theorem}

For a given $\beta$, the bound is decreasing in $\alpha$. While
the active channel bandwidth is smaller than the total bandwidth,
the noise (even though the SNR is large) is scattered over the entire
bandwidth. Thus, none of the universal sub-Nyquist sampling strategies
are information preserving, and increasing the sampling rate can always
harvest capacity gain.


\subsection{Achievability with Landau-rate Sampling ($\alpha=\beta$) \label{sub:Achievability-Landau-rate-Minimax}}

Consider the achievability part when the sampling rate equals the
active frequency bandwidth ($\beta=\alpha$). In general, it is very
difficult to find a deterministic solution to approach the lower bound
(\ref{eq:MinimaxCapacityGapLowerBoundLargeN}). A special instance
of sampling methods that we can analyze concerns the case in which
the sampling coefficient functions are flat over $\left[0,W/n\right]$
and whose coefficients are generated in a random fashion. It turns
out that as $n$ grows large, the capacity loss achievable by random
sampling approaches the lower bound (\ref{eq:MinimaxCapacityGapLowerBoundLargeN})
uniformly across all realizations. The results are stated in Theorem
\ref{theorem-Minimax-Sampler-BWLimited-Achievability} after introducing
a class of sub-Gaussian measure below.

\begin{definition}A measure $\nu$ on $\mathbb{R}$ satisfies the
logarithmic Sobolev inequality (LSI) with constant $c_{\text{LS}}$
if, for any differentiable function $g$,
\[
{\displaystyle \int}g^{2}\log\frac{g^{2}}{\int g^{2}\mathrm{d}\nu}\mathrm{d}\nu\leq2c_{\text{LS}}{\displaystyle \int}\left|g'\right|^{2}\mathrm{d}\nu.
\]
\end{definition}

\begin{remark}A probability measure obeying the LSI possesses sub-Gaussian
tails, and a large class of sub-Gaussian measures satisfies this inequality
for some constant. See \cite{GuionnetZeitouni2000,raginsky2012concentration}
for examples. In particular, the standard Gaussian measure satisfies
this inequality with constant $c_{\mathrm{LS}}=1$ (e.g. \cite{MassartPicard2007}).
\end{remark}

\begin{theorem}\label{theorem-Minimax-Sampler-BWLimited-Achievability}Let
$\boldsymbol{M}\in\mathbb{R}^{k\times n}$ be a random matrix such
that $\boldsymbol{M}_{ij}$'s are jointly independent symmetric random
variables with zero mean and unit variance. In addition, suppose that
$\boldsymbol{M}_{ij}$ satisfies either of the following conditions:

(a) $\boldsymbol{M}_{ij}$ is bounded in magnitude by an absolute
constant $D$;

(b) The probability measure of $\boldsymbol{M}_{ij}$ satisfies the
LSI with a bounded constant $c_{\mathrm{LS}}$.

If $\mathsf{SNR}_{\max}\geq1$, then there exist absolute constants
$c_{0},c_{1},C>0$ such that
\begin{equation}
\max_{\boldsymbol{s}\in{[n] \choose k}}L_{\boldsymbol{s}}^{\boldsymbol{M}}\leq\frac{W}{2}\left(\mathcal{H}\left(\beta\right)+c_{1}\min\left\{ \frac{\sqrt{\small\mathsf{SNR}_{\max}}}{\sqrt{n}},\frac{\log n+\log\small\mathsf{SNR}_{\max}}{n^{1/4}}\right\} +\frac{\beta}{\small\mathsf{SNR}_{\min}}\right)\label{eq:MinimaxGapGeneralLowerBound-1}
\end{equation}
with probability exceeding $1-C\exp\left(-c_{0}n\right)$.\end{theorem}

Theorem \ref{theorem-Minimax-Sampler-BWLimited-Achievability} demonstrates
that independent random sampling achieves minimax sampled capacity
loss, which is $\frac{1}{2}\mathcal{H}\left(\beta\right)$ per unit
bandwidth modulo some vanishing residual term. In fact, our analysis
demonstrates that the sampled capacity loss approaches the minimax
limit uniformly over all states $\boldsymbol{s}$. Another interesting
observation is the universality phenomenon, i.e. a broad class of
sub-Gaussian ensembles, as long as the entries are jointly independent
with matching moments, suffices to generate minimax samplers. 


\subsection{Achievability with Super-Landau-Rate Sampling $\left(\alpha>\beta,\alpha+\beta<1\right)$
\label{sub:Achievability-Super-Landau-Minimax}}

So far we have considered the case in which the sampling rate is equal
to the spectral support. While the active bandwidth for transmission
is smaller than the total bandwidth, the noise (even though the SNR
is large) is scattered over the entire bandwidth. This indicates that
none of the sub-Nyquist sampling strategies preserves all information
contents conveyed through the noisy channel, unless they know the
channel support. One may thus hope that increasing the sampling rate
would improve the achievable information rate. The achievability result
for super-Landau-rate sampling is stated in the following theorem. 

\begin{theorem}\label{theorem-Minimax-Sampler-BWLimited-Achievability-Oversampling}Let
$\boldsymbol{M}\in\mathbb{R}^{m\times n}$ be a Gaussian matrix such
that $\boldsymbol{M}_{ij}$'s are independently drawn from $\mathcal{N}\left(0,1\right)$.
Suppose that $1-\alpha-\beta\geq\varepsilon$ and $\alpha-\beta\geq\varepsilon$
for some small constant $\varepsilon>0$. Then there exist universal
constants $C,c>0$ such that
\[
\max_{\boldsymbol{s}\in{[n] \choose k}}L_{\boldsymbol{s}}^{\boldsymbol{M}}\leq\frac{W}{2}\left[\mathcal{H}(\beta)-\alpha\mathcal{H}\left(\frac{\beta}{\alpha}\right)+\frac{c\log n}{n^{1/3}}+\frac{\beta}{\small\mathsf{SNR}_{\min}}\right]
\]
with probability exceeding $1-C\exp\left(-n\right)$.\end{theorem}

Theorem \ref{theorem-Minimax-Sampler-BWLimited-Achievability-Oversampling}
indicates that i.i.d. Gaussian sampling approaches the minimax capacity
loss (which is about $\frac{1}{2}\mathcal{H}(\beta)-\frac{1}{2}\alpha\mathcal{H}\left(\frac{\beta}{\alpha}\right)$
per Hertz) to within vanishingly small gap. As will be shown in our
proof, with exponentially high probability, the sampled capacity loss
for all states are equivalent and coincide with the fundamental minimax
limit. In contrast to Theorem \ref{theorem-Minimax-Sampler-BWLimited-Achievability},
we restrict our attention to Gaussian sampling, which suffices for
the proof of Theorem \ref{theorem-Minimax-Final}.


\section{Equivalent Algebraic Problems}

Our main results in Section \ref{sec:Minimax-Capacity-Regret} are
established by investigating three equivalent algebraic problems.
Recall that $\frac{P}{\beta W}\boldsymbol{H}_{\boldsymbol{s}}^{2}\succeq\text{SNR}_{\min}\boldsymbol{I}_{k}$.
Define $\boldsymbol{Q}_{\boldsymbol{s}}^{\text{w}}:=\left(\boldsymbol{Q}\boldsymbol{Q}^{*}\right)^{-\frac{1}{2}}\boldsymbol{Q}_{\boldsymbol{s}}$.
Simple manipulations yield
\begin{align}
L_{\boldsymbol{s}}^{\boldsymbol{Q}}= & -\frac{1}{2}{\displaystyle \int}_{0}^{W/n}\log\det\left(\boldsymbol{I}_{m}+\frac{P}{\beta W}\boldsymbol{Q}_{\boldsymbol{s}}^{\text{w}}(f)\boldsymbol{H}_{\boldsymbol{s}}^{2}(f)\boldsymbol{Q}_{\boldsymbol{s}}^{\text{w}*}(f)\right)\mathrm{d}f+\frac{1}{2}{\displaystyle \int}_{0}^{W/n}\log\det\left(\boldsymbol{I}_{k}+\frac{P}{\beta W}\boldsymbol{H}_{\boldsymbol{s}}^{2}(f)\right)\mathrm{d}f\nonumber \\
= & -\frac{1}{2}{\displaystyle \int}_{0}^{W/n}\log\det\left(\boldsymbol{I}_{k}+\frac{P}{\beta W}\boldsymbol{H}_{\boldsymbol{s}}(f)\boldsymbol{Q}_{\boldsymbol{s}}^{\text{w}*}(f)\boldsymbol{Q}_{\boldsymbol{s}}^{\text{w}}(f)\boldsymbol{H}_{\boldsymbol{s}}(f)\right)\mathrm{d}f\nonumber \\
 & \quad\quad+\frac{1}{2}{\displaystyle \int}_{0}^{W/n}\log\det\left(\frac{P}{\beta W}\boldsymbol{H}_{\boldsymbol{s}}^{2}(f)\right)\mathrm{d}f+\frac{\beta W}{2}\Delta_{\boldsymbol{s}}\\
= & -\frac{1}{2}{\displaystyle \int}_{0}^{W/n}\log\det\left(\frac{\beta W}{P}\boldsymbol{H}_{\boldsymbol{s}}^{-2}(f)+\boldsymbol{Q}_{\boldsymbol{s}}^{\text{w}*}(f)\boldsymbol{Q}_{\boldsymbol{s}}^{\text{w}}(f)\right)\mathrm{d}f+\frac{\beta W}{2}\Delta_{\boldsymbol{s}},\label{eq:GsQApproximationLandauSampling}
\end{align}
where $\Delta_{\boldsymbol{s}}$ denotes some residual term. In particular,
$\Delta_{\boldsymbol{s}}$ can be bounded as
\begin{equation}
0\leq\Delta_{\boldsymbol{s}}\leq\frac{1}{\small\mathsf{SNR}_{\min}}.\label{eq:BoundResidualDelta}
\end{equation}
This is an immediate consequence of the following observation: for
any $k\times k$ positive semidefinite matrix $\boldsymbol{A}$, 
\begin{equation}
0\leq\frac{1}{k}\log\det\left(\boldsymbol{I}_{k}+\boldsymbol{A}\right)-\frac{1}{k}\log\det\left(\boldsymbol{A}\right)=\frac{1}{k}\sum_{i=1}^{k}\log\left(1+\frac{1}{\lambda_{i}\left(\boldsymbol{A}\right)}\right)\leq\frac{1}{\lambda_{\min}{\bf \left(\boldsymbol{A}\right)}}.\label{eq:EstimateLogIplusA-LotA}
\end{equation}


Recall that $\mathsf{SNR}_{\min}:=\frac{P}{\beta W}\inf_{0\leq f\leq W}\left|H(f)\right|^{2}$
and $\mathsf{SNR}_{\max}:=\frac{P}{\beta W}\sup_{0\leq f\leq W}\left|H(f)\right|^{2}$.
Therefore, $\frac{\beta W}{P}\boldsymbol{H}_{\boldsymbol{s}}^{-2}$
can be bounded as
\begin{equation}
\frac{1}{\small\mathsf{SNR}_{\max}}\boldsymbol{I}_{k}\preceq\frac{\beta W}{P}\boldsymbol{H}_{\boldsymbol{s}}^{-2}\preceq\frac{1}{\small\mathsf{SNR}_{\min}}\boldsymbol{I}_{k}.\label{eq:BoundHInverse}
\end{equation}
This bound together with (\ref{eq:GsQApproximationLandauSampling})
makes $\det\left(\epsilon\boldsymbol{I}_{k}+\boldsymbol{Q}_{\boldsymbol{s}}^{\text{w}*}\boldsymbol{Q}_{\boldsymbol{s}}^{\text{w}}\right)$
a quantity of interest (for some small $\epsilon$). In the sequel,
we provide tight bounds on several algebraic problems, which in turn
establish Theorems \ref{Theorem-Minimax-Det-Upper-Bound}-\ref{theorem-Minimax-Sampler-BWLimited-Achievability-Oversampling}.
The proofs of these results rely heavily on \emph{non-asymptotic}
(random) matrix theory. In particular, the proofs for achievability
bounds are established based on concentration of log-determinant functions,
which we provide as well in this section. 


\subsection{The Converse}

Note that $\boldsymbol{Q}^{\text{w}}(f)$ has orthonormal rows. The
following theorem investigates the properties of $\det\left(\epsilon\boldsymbol{I}_{k}+\boldsymbol{B}_{\boldsymbol{s}}^{*}\boldsymbol{B}_{\boldsymbol{s}}\right)$
for any $m\times n$ matrix $\boldsymbol{B}$ that has orthonormal
rows. This, together with the Riemann integrability assumption of
$\boldsymbol{Q}(f)$, establishes Theorem \ref{Theorem-Minimax-Det-Upper-Bound}.

\begin{theorem}\label{theorem-Evaluate-Det-IplusBB}(1) Consider
any $m\times n$ matrix $\boldsymbol{B}$ $\left(n\geq m\geq k\right)$
that satisfies $\boldsymbol{B}\boldsymbol{B}^{*}=\boldsymbol{I}_{m}$,
and denote by $\boldsymbol{B}_{\boldsymbol{s}}$ the $m\times k$
submatrix of $\boldsymbol{B}$ with columns coming from the index
set $\boldsymbol{s}$. Then for any $\epsilon>0$, one has 
\begin{align}
{m \choose k}\leq\sum_{\boldsymbol{s}\in{[n] \choose k}}\det\left(\epsilon\boldsymbol{I}_{m}+\boldsymbol{B}_{\boldsymbol{s}}^{*}\boldsymbol{B}_{\boldsymbol{s}}\right) & =\sum_{l=0}^{k}{n-l \choose k-l}{m \choose l}\epsilon^{k-l}\label{eq:SumDetIplusBB}\\
 & \leq{m \choose k}\left(1+\sqrt{\epsilon}\right)^{n+k}.
\end{align}


(2) For any positive integer $p$, suppose that $\boldsymbol{B}_{1},\cdots,\boldsymbol{B}_{p}$
are all $m\times n$ matrices such that $\boldsymbol{B}_{i}\boldsymbol{B}_{i}^{*}=\boldsymbol{I}_{m}$.
Then,
\begin{align}
\min_{\boldsymbol{s}\in{[n] \choose k}}\frac{1}{np}\sum_{i=1}^{p}\log\det\left(\epsilon\boldsymbol{I}_{k}+\left(\boldsymbol{B}_{i}\right)_{\boldsymbol{s}}^{*}\left(\boldsymbol{B}_{i}\right)_{\boldsymbol{s}}\right) & \leq\frac{1}{n}\log{m \choose k}-\frac{1}{n}\log{n \choose k}+2\sqrt{\epsilon}\label{eq:MaximinLogDetIplusBB-NoStirling}\\
 & \leq\alpha\mathcal{H}\left(\frac{\beta}{\alpha}\right)-\mathcal{H}\left(\beta\right)+2\sqrt{\epsilon}+\frac{\log\left(n+1\right)}{n}.\label{eq:MaximinLogDetIplusBB}
\end{align}
\end{theorem}

Note that $\boldsymbol{Q}^{\text{w}}(f)$ has orthonormal rows for
any $f$, and $\boldsymbol{Q}^{\text{w}}(f)$ is assumed to be Riemann
integrable. For any $\delta>0$, we can find a sufficiently large
$p$ such that
\[
{\displaystyle \int}_{0}^{W/n}\log\det\left(\epsilon\boldsymbol{I}_{k}+\boldsymbol{Q}_{\boldsymbol{s}}^{\text{w}*}(f)\boldsymbol{Q}_{\boldsymbol{s}}^{\text{w}}(f)\right)\mathrm{d}f\leq\delta+\frac{W}{pn}\sum_{i=1}^{p}\log\det\left(\epsilon\boldsymbol{I}_{k}+\boldsymbol{Q}_{\boldsymbol{s}}^{\text{w}*}\left(\frac{iW}{pn}\right)\boldsymbol{Q}_{\boldsymbol{s}}^{\text{w}}\left(\frac{iW}{pn}\right)\right).
\]
Since $\delta$ can be arbitrarily small, applying Theorem \ref{theorem-Evaluate-Det-IplusBB}
immediately yields that for any $\boldsymbol{Q}(\cdot)$:
\[
\min_{\boldsymbol{s}\in{[n] \choose k}}{\displaystyle \int}_{0}^{W/n}\log\det\left(\epsilon\boldsymbol{I}_{k}+\boldsymbol{Q}_{\boldsymbol{s}}^{\text{w}*}(f)\boldsymbol{Q}_{\boldsymbol{s}}^{\text{w}}(f)\right)\mathrm{d}f\leq W\left\{ \alpha\mathcal{H}\left(\frac{\beta}{\alpha}\right)-\mathcal{H}\left(\beta\right)+2\sqrt{\epsilon}+\frac{\log\left(n+1\right)}{n}\right\} .
\]
This together with (\ref{eq:GsQApproximationLandauSampling}), (\ref{eq:BoundResidualDelta})
and (\ref{eq:BoundHInverse}) leads to
\begin{align*}
\inf_{\boldsymbol{Q}}\max_{\boldsymbol{s}\in{[n] \choose k}}L_{\boldsymbol{s}}^{\boldsymbol{Q}} & \geq\frac{W}{2}\left\{ \mathcal{H}\left(\beta\right)-\alpha\mathcal{H}\left(\frac{\beta}{\alpha}\right)-\frac{2}{\sqrt{\small\mathsf{SNR}_{\min}}}-\frac{\log\left(n+1\right)}{n}\right\} ,
\end{align*}
which completes the proof of Theorem \ref{Theorem-Minimax-Det-Upper-Bound}.


\subsection{Achievability (Landau-rate Sampling)}

When it comes to the achievability part, the major step is to quantify
$\det(\epsilon\boldsymbol{I}_{k}+(\boldsymbol{M}\boldsymbol{M}^{\top})^{-1}\boldsymbol{M}_{\boldsymbol{s}}\boldsymbol{M}_{\boldsymbol{s}}^{\top})$
for every $\boldsymbol{s}\in{[n] \choose k}$. Interestingly, this
quantity can be uniformly controlled due to the concentration of spectral
measure of random matrices \cite{GuionnetZeitouni2000}. This is stated
in the following theorem, which demonstrates the achievability for
Landau-rate sampling.

\begin{theorem}\label{theorem-Minimax-RMT-LandauRate}Let $\boldsymbol{M}\in\mathbb{R}^{k\times n}$
be a random matrix of independent entries, and let $0<\epsilon<1$
denote some constant. Under the conditions of Theorem \ref{Theorem-Minimax-Det-Upper-Bound},
there exist absolute constants $c_{0},$ $c_{1}$, $C>0$ such that
\begin{align}
-\mathcal{H}\left(\beta\right)-c_{1}\min\left\{ \frac{1}{\sqrt{n\epsilon}},\frac{\log n+\log\frac{1}{\epsilon}}{n^{1/4}}\right\}  & \leq\min_{\boldsymbol{s}\in{[n] \choose k}}\frac{1}{n}\log\det\left(\epsilon\boldsymbol{I}_{k}+\left(\boldsymbol{M}\boldsymbol{M}^{\top}\right)^{-1}\boldsymbol{M}_{\boldsymbol{s}}\boldsymbol{M}_{\boldsymbol{s}}^{\top}\right)\\
 & \leq-\mathcal{H}\left(\beta\right)+2\sqrt{\epsilon}+\frac{\log\left(n+1\right)}{n}
\end{align}
with probability exceeding $1-C\exp\left(-c_{0}n\right)$.\end{theorem}

Putting Theorem \ref{theorem-Minimax-RMT-LandauRate} and equations
(\ref{eq:GsQApproximationLandauSampling}), (\ref{eq:BoundResidualDelta})
and (\ref{eq:BoundHInverse}) together establishes that
\begin{align*}
\max_{\boldsymbol{s}\in{[n] \choose k}}L_{\boldsymbol{s}}^{\boldsymbol{M}} & \leq\frac{W}{2}\left[\mathcal{H}\left(\beta\right)+\mathcal{O}\left(\min\left\{ \sqrt{\frac{\small\mathsf{SNR}_{\max}}{n}},\frac{\log n+\log\small\mathsf{SNR}_{\max}}{n^{1/4}}\right\} \right)+\frac{\beta}{\mathsf{SNR}_{\min}}\right]
\end{align*}
with exponentially high probability, as claimed in Theorem \ref{theorem-Minimax-Sampler-BWLimited-Achievability}.


\subsection{Achievability (Super-Landau-rate Sampling)}

Instead of studying a large class of sub-Gaussian random ensembles%
\footnote{The proof argument for Landau-rate sampling cannot be readily carried
over to super-Landau regime since $\boldsymbol{M}_{\boldsymbol{s}}$
is now a tall matrix, and hence we cannot separate $\boldsymbol{M}_{\boldsymbol{s}}$
and $\boldsymbol{M}\boldsymbol{M}^{*}$ easily. %
}, the following theorem focuses on i.i.d. Gaussian matrices, which
establishes the optimality of Gaussian random sampling for the super-Landau
regime.

\begin{theorem}\label{theorem-Minimax-RMT-SuperLandau}Let $\boldsymbol{M}\in\mathbb{R}^{m\times n}$
be an i.i.d. random matrix satisfying $\boldsymbol{M}_{ij}\sim\mathcal{N}(0,1)$.
Suppose that $1-\alpha-\beta\geq\zeta$ and $\alpha-\beta\geq\zeta$
for some small constant $\zeta>0$. Then there exist absolute constants
$c$, $C>0$ such that 
\begin{align*}
-\mathcal{H}(\beta)+\alpha\mathcal{H}\left(\frac{\beta}{\alpha}\right)+\frac{c\log n}{n^{1/3}} & \leq\min_{\boldsymbol{s}\in{[n] \choose k}}\frac{1}{n}\log\det\left(\epsilon\boldsymbol{I}_{k}+\boldsymbol{M}_{\boldsymbol{s}}^{\top}\left(\boldsymbol{M}\boldsymbol{M}^{\top}\right)^{-1}\boldsymbol{M}_{\boldsymbol{s}}\right)\\
 & \leq-\mathcal{H}\left(\beta\right)+\alpha\mathcal{H}\left(\frac{\beta}{\alpha}\right)+2\sqrt{\epsilon}+\frac{\log\left(n+1\right)}{n}
\end{align*}
with probability at least $1-C\exp\left(-n\right)$. \end{theorem}

Combining Theorem \ref{theorem-Minimax-RMT-SuperLandau} and equations
(\ref{eq:GsQApproximationLandauSampling}), (\ref{eq:BoundResidualDelta})
and (\ref{eq:BoundHInverse}) implies that
\[
L_{\boldsymbol{s}}^{\boldsymbol{M}}\leq\frac{W}{2}\left[\mathcal{H}(\beta)-\alpha\mathcal{H}\left(\frac{\beta}{\alpha}\right)+\mathcal{O}\left(\frac{\log n}{n^{1/3}}\right)+\frac{\beta}{\small\mathsf{SNR}_{\min}}\right]
\]
with exponentially high probability. 


\subsection{Concentration of Log-Determinant for Random Matrices\label{sub:Concentration-of-Log-Determinant}}

It has been shown above that the achievability bounds can be established
by demonstrating measure concentration of certain log-determinant
functions. In fact, the concentration of log-determinant has been
studied in the random matrix literature as a key ingredient in demonstrating
universality laws for spectral statistics (e.g. \cite[Proposition 48]{TaoVu2012Nonhermitian}).
However, these bounds are only shown to hold with overwhelming probability
(i.e. with probability at least $1-e^{-\omega\left(\log n\right)}$),
which are not sharp enough for our purpose. As a result, we provide
sharper concentration results of log-determinant in this subsection. 

\begin{lem}\label{lemma-ConcentrationLogDetPlusConstant}Suppose
that $k/p\in(0,1]$, and that $\epsilon>0$. Consider a random matrix
$\boldsymbol{A}=\mathbb{R}^{k\times p}$ where $\boldsymbol{A}_{ij}$
are symmetric and jointly independent with zero mean and unit variance. 

(a) If $\boldsymbol{A}_{ij}$'s are bounded in magnitude by a constant
$D$, then for any $\delta>\frac{8D\sqrt{\pi}}{\sqrt{\epsilon}}$:
\begin{equation}
\mathbb{P}\left(\log\det\left(\epsilon\boldsymbol{I}+\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)<\mathbb{E}\left[\log\det\left(\frac{2\epsilon}{e}\boldsymbol{I}+\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]-\delta\right)\leq4\exp\left(-\frac{\epsilon\delta^{2}}{16D^{2}}\right);\label{eq:ConcentrationLogDetPlusConstant}
\end{equation}


\begin{equation}
\mathbb{P}\left(\log\det\left(\epsilon\boldsymbol{I}+\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)>\mathbb{E}\left[\log\det\left(\frac{e\epsilon}{2}\boldsymbol{I}+\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]+\delta\right)\leq4\exp\left(-\frac{\epsilon e\delta^{2}}{32D^{2}}\right).\label{eq:ConcentrationLogDetPlusConstant-2}
\end{equation}


(b) $\boldsymbol{A}_{ij}$'s satisfy the LSI with uniformly bounded
constant $c_{\mathrm{LS}}$, then for any $\delta>0$:
\begin{equation}
\mathbb{P}\left(\left|\log\det\left(\epsilon\boldsymbol{I}+\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)-\mathbb{E}\left[\log\det\left(\epsilon\boldsymbol{I}+\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]\right|>\delta\right)\leq2\exp\left(-\frac{\epsilon\delta^{2}}{2c_{\mathrm{LS}}}\right).\label{eq:ConcentrationLogDetPlusConstant-1}
\end{equation}
\end{lem}

\begin{proof}See Appendix \ref{sec:Proof-of-Lemma-ConcentrationLogDetPlusConstant}.\end{proof}

The concentration results for $\frac{1}{k}\log\text{det}\left(\epsilon\boldsymbol{I}+\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)$
will be useful if we can quantify $\mathbb{E}\left[\frac{1}{k}\log\text{det}\left(\epsilon\boldsymbol{I}+\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]$,
as established in the following lemma.

\begin{lem}\label{lemma-Expected-Log-Determinant-Plus-Constant}Let
$\boldsymbol{A}\in\mathbb{R}^{k\times k}$ be a random matrix such
that all entries are jointly independent with zero mean and unit variance.
For any $\epsilon\in\left(\frac{4}{k},\frac{1}{e^{2}}\right)$, we
have
\begin{equation}
\mathbb{E}\left[\frac{1}{k}\log\det\left(\epsilon\boldsymbol{I}+\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]\leq\frac{1}{k}\log\mathbb{E}\left[\det\left(\epsilon\boldsymbol{I}+\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]\leq-1+\frac{1.5\log\left(ek\right)}{k}+2\sqrt{\epsilon}\log\frac{1}{\epsilon}.\label{eq:LogExpectedDetMSquarePlusConstantUB}
\end{equation}


Additionally, under Condition (a) or (b) of Lemma \ref{lemma-ConcentrationLogDetPlusConstant},
one has, for any $\frac{1}{k}<\epsilon\leq0.8$,
\begin{equation}
\mathbb{E}\left[\frac{1}{k}\log\det\left(\epsilon\boldsymbol{I}+\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]\geq-1+\frac{\log k}{2k}-\mathcal{O}\left(\frac{1}{k\epsilon}\right).\label{eq:LogExpectedDetMSquarePlusConstantLB}
\end{equation}
In particular, if $\boldsymbol{A}_{ij}\sim\mathcal{N}\left(0,1\right)$,
then for any $0<\epsilon\leq0.8$,
\begin{equation}
\mathbb{E}\left[\frac{1}{k}\log\det\left(\epsilon\boldsymbol{I}+\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]\geq-1+\frac{\log k}{2k}-\frac{2}{k\epsilon}.\label{eq:LogExpectedDetMSquarePlusConstantGaussian}
\end{equation}
\end{lem}\begin{proof}See Appendix \ref{sec:Proof-of-lemma-Expected-Log-Determinant-Plus-Constant}.\end{proof}

One issue concerning Lemmas \ref{lemma-ConcentrationLogDetPlusConstant}
and \ref{lemma-Expected-Log-Determinant-Plus-Constant} is that the
bounds might become useless when $\epsilon$ is extremely small. To
mitigate this issue, we obtain another concentration lower bound in
the following lemma.

\begin{lem}\label{lemma:logdet_I_AA_LB}Let $\boldsymbol{A}\in\mathbb{R}^{k\times k}$
be a random matrix satisfying the assumptions of Lemma \ref{lemma-ConcentrationLogDetPlusConstant}.
There exists an absolute constant $c_{14}>0$ such that for any $\frac{1}{\sqrt{k}}\leq\tau<k$
and $0<\epsilon<1$, 
\[
\frac{1}{k}\log\det\left(\epsilon\boldsymbol{I}+\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\geq-1-\frac{c_{14}\tau^{1/4}}{k^{1/4}}\left(\log\frac{1}{\epsilon}+\log k\right)
\]
with probability at least $1-4\exp\left(-\tau k\right)$.\end{lem}

\begin{proof}See Appendix \ref{sec:Proof-of-Lemma-logdet_I_AA_LB}.\end{proof}

Lemmas \ref{lemma-ConcentrationLogDetPlusConstant}, \ref{lemma-Expected-Log-Determinant-Plus-Constant},
and \ref{lemma:logdet_I_AA_LB} taken collectively allow us to derive
the concentration of $\log\text{det}\left(\epsilon\boldsymbol{I}+\frac{1}{k}\boldsymbol{M}_{\boldsymbol{s}}\boldsymbol{M}_{\boldsymbol{s}}^{\top}\right)$
uniformly over all $\boldsymbol{s}\in{[n] \choose k}$ as follows.

\begin{lem}\label{lem:MinLogdetMs_Landau}Let $\boldsymbol{M}\in\mathbb{R}^{m\times n}$
be a random matrix satisfying the conditions of Theorem \ref{Theorem-Minimax-Det-Upper-Bound}.
Then for any constant $0<\epsilon<1$,
\begin{equation}
\forall\boldsymbol{s}:\text{ }\frac{1}{n}\log\det\left(\epsilon\boldsymbol{I}+\frac{1}{k}\boldsymbol{M}_{\boldsymbol{s}}\boldsymbol{M}_{\boldsymbol{s}}^{\top}\right)\geq-1-\mathcal{O}\left(\min\left\{ \frac{1}{\sqrt{n\epsilon}},\frac{\log\frac{1}{\epsilon}+\log n}{n^{1/4}}\right\} \right)\label{eq:ConcentrationFinalLogDetPlusConst}
\end{equation}
holds with probability exceeding $1-2\left(\frac{2}{e}\right)^{n}$. 

In particular, if $\boldsymbol{M}_{ij}\sim\mathcal{N}(0,1)$ are jointly
independent, then for any $0<\epsilon\leq0.8$,
\begin{align}
\forall\boldsymbol{s}:\text{ }\frac{1}{k}\log\det\left(\epsilon\boldsymbol{I}+\frac{1}{k}\boldsymbol{M}_{\boldsymbol{s}}\boldsymbol{M}_{\boldsymbol{s}}^{\top}\right)\geq-1-\frac{2}{k\epsilon}-\sqrt{\frac{2}{\beta k\epsilon}}\label{eq:Gaussian_Landau_Ms}
\end{align}
with probability at least $1-\left(\frac{2}{e}\right)^{n}$.\end{lem}

\begin{proof}Fix $\delta=c_{5}\sqrt{k/\epsilon}$ for some numerical
constant $c_{5}>0$ in Lemma \ref{lemma-ConcentrationLogDetPlusConstant}.
Combining Lemmas \ref{lemma-ConcentrationLogDetPlusConstant}, \ref{lemma-Expected-Log-Determinant-Plus-Constant},
and \ref{lemma:logdet_I_AA_LB} yields the following: under the assumptions
of Lemma  \ref{lemma-ConcentrationLogDetPlusConstant}, for any small
constant $\epsilon>0$ and some sufficiently large $c_{5}$ we have
\begin{equation}
\frac{1}{k}\log\text{det}\left(\epsilon\boldsymbol{I}+\frac{1}{k}\boldsymbol{M}_{\boldsymbol{s}}\boldsymbol{M}_{\boldsymbol{s}}^{\top}\right)\geq-1+\frac{\log k}{2k}-\mathcal{O}\left(\min\left\{ \frac{1}{\sqrt{n\epsilon}},\frac{\log\frac{1}{\epsilon}+\log n}{n^{1/4}}\right\} \right)\label{eq:ConcentrationLogDetPlusEpsilon}
\end{equation}
with probability exceeding $1-4e^{-n}$. Since there are at most ${n \choose k}\leq\frac{1}{2}\cdot2^{n}$
different $\boldsymbol{s}$, applying the union bound establishes
(\ref{eq:ConcentrationFinalLogDetPlusConst}). Similarly, for the
ensemble $\boldsymbol{M}_{ij}\sim\mathcal{N}\left(0,1\right)$, Lemmas
\ref{lemma-ConcentrationLogDetPlusConstant} and \ref{lemma-Expected-Log-Determinant-Plus-Constant}
indicates that
\begin{align*}
\frac{1}{k}\log\det\left(\epsilon\boldsymbol{I}+\frac{1}{k}\boldsymbol{M}_{\boldsymbol{s}}\boldsymbol{M}_{\boldsymbol{s}}^{\top}\right) & \geq-1+\frac{\log k}{2k}-\frac{2}{k\epsilon}-\sqrt{\frac{2}{\beta k\epsilon}}
\end{align*}
with probability at least $1-2e^{-n}$. The proof is then complete
via the union bound. \end{proof}

Another important class of log-determinant function takes the form
of $\frac{1}{n}\log\det\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)$.
The concentration of such functions for i.i.d. rectangular random
matrices is characterized in the following lemmas.

\begin{lem}\label{lemma-Rectangular-Nonhermitian-LogDet}Let $\alpha:=m/n$
be a fixed constant independent from $\left(m,n\right)$, and that
$\alpha\in\left[\delta,1-\delta\right]$ for some small constant $\delta>0$.
Let $\boldsymbol{A}\in\mathbb{R}^{m\times n}$ be a random matrix
such that all entries are symmetric and jointly independent with zero
mean and unit variance. Under Condition (a) or (b) of Lemma \ref{lemma-ConcentrationLogDetPlusConstant},
there exist universal constants $c_{7},C_{7}>0$ independent of $n$
such that
\begin{equation}
\left|\frac{1}{n}\log\det\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)-\left(1-\alpha\right)\log\frac{1}{1-\alpha}+\alpha\right|\leq\frac{1}{\sqrt{n}}\label{eq:logdetAA_UB}
\end{equation}
with probability exceeding $1-C_{7}\exp\left(-c_{7}n\right)$.\end{lem}

\begin{proof}See Appendix \ref{sec:Proof-of-lemma-Rectangular-Nonhermitian-LogDet}.\end{proof}

This result can be made more explicit for Gaussian ensembles as follows.

\begin{lem}\label{lem:GaussianLogDet_LB}Suppose that $\boldsymbol{A}\in\mathbb{R}^{m\times n}$
is a random matrix with independent entries satisfying $\boldsymbol{A}_{ij}\sim\mathcal{N}(0,1)$.
Suppose that there exists a small constant $\delta>0$ such that $\alpha:=\frac{m}{n}\in[\delta,1-\delta]$. 

(1) For any $\epsilon>0$ and any $\tau>0$, 
\begin{equation}
\frac{\mathrm{card}\left\{ i\mid\lambda_{i}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)<\epsilon\right\} }{n}<\frac{\alpha}{1-\alpha-\frac{1}{n}}\epsilon+\frac{4\sqrt{\alpha\tau}}{\sqrt{n\epsilon}}\label{eq:SmallEvalueCardinalityBound-1}
\end{equation}
with probability exceeding $1-2\exp\left(-\tau n\right)$. 

(2) For any $n>\max\left\{ \frac{2}{1-\sqrt{\alpha}},7\right\} $
and any $\tau>0$, 
\begin{equation}
\frac{1}{n}\log\det\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\leq\left(1-\alpha\right)\log\frac{1}{1-\alpha}-\alpha+\frac{2\log n}{n}+\frac{5\sqrt{\alpha}}{\left(1-\sqrt{\alpha}-\frac{2}{n}\right)}\frac{\tau}{n}\label{eq:logdetAA_UB_Gaussian}
\end{equation}
with probability exceeding $1-2\exp\left(-2\tau^{2}\right)$.

(3) For any $n>\max\left\{ \frac{6.414}{1-\alpha}\cdot e^{\frac{\tau}{1-\alpha}},\left(\frac{\alpha}{1-\alpha-\frac{1}{n}}+4\sqrt{\alpha\tau}\right)^{3}\right\} $
and any $\tau>0$,
\begin{align}
\frac{1}{n}\log\det\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right) & >\left(1-\alpha\right)\log\frac{1}{1-\alpha}-\alpha-\left(\frac{2\alpha}{1-\alpha-\frac{1}{n}}+11\sqrt{\alpha\tau}\right)\frac{\log n}{n^{1/3}}-\frac{6\alpha}{n^{2/3}}\label{eq:Gaussian_LB_logdetAA}
\end{align}
with probability exceeding $1-5\exp(-\tau n)$.\end{lem}

\begin{proof}See Appendix \ref{sec:Proof-of-Lemma-GaussianLogDet_LB}.\end{proof}

The last log-determinant function considered here takes the form of
$\log\det\left(\epsilon\boldsymbol{I}+\boldsymbol{A}^{\top}\boldsymbol{B}^{-1}\boldsymbol{A}\right)$
for some independent random matrices $\boldsymbol{A}$ and $\boldsymbol{B}$,
as stated in the following lemma.

\begin{lem}\label{lemma-Expected-Log-Determinant-LambdaStatistics}Suppose
that $m>k$ and $m/k$ is bounded away from%
\footnote{In other words, there exists a small constant $\delta>0$ such that
$m/k\geq1+\delta$.%
} 1. Let matrix $\boldsymbol{A}=\mathbb{R}^{m\times k}$ and $\boldsymbol{B}\in\mathbb{R}^{m\times m}$
be two independent random matrices such that $\boldsymbol{A}_{ij}\sim\mathcal{N}\left(0,1\right)$
are jointly independent, and $\boldsymbol{B}\sim\mathcal{W}_{m}\left(n-k,\boldsymbol{I}_{m}\right)$.
Then for any $\tau>\frac{1}{n^{1/3}}$,
\begin{align*}
 & \frac{1}{n}\log\det\left(\epsilon\boldsymbol{I}_{k}+\boldsymbol{A}^{\top}\boldsymbol{B}^{-1}\boldsymbol{A}\right)\geq-\left(\alpha-\beta\right)\log\left(\alpha-\beta\right)+\alpha\log\alpha\\
 & \quad\quad\quad\quad\quad\quad+\left(1-\alpha-\beta\right)\log\left(1-\frac{\beta}{1-\alpha}\right)-\beta\log\left(1-\alpha\right)-\frac{c_{8}\tau\log n}{n^{1/3}}
\end{align*}
with probability exceeding $1-C_{8}\exp\left(-\tau^{2}n\right)$ for
some absolute constants $c_{8},C_{8}>0$. \end{lem}

\begin{proof}See Appendix \ref{sec:Proof-of-lemma-Expected-Log-Determinant-LambdaStatistics}.\end{proof}


\section{Discussion}


\subsection{Implications of Main Results \label{sub:Implications-Minimax}}

Under both Landau-rate and super-Landau-rate sampling, the minimax
capacity loss depends almost entirely on $\beta$ and $\alpha$. In
this subsection, we summarize several key insights from the main theorems. 


\subsubsection{The Converse}

Our analysis demonstrates that at high SNR, the loss $L_{\boldsymbol{s}}^{\boldsymbol{Q}}$
depends almost solely on the quantity
\[
d\left(\boldsymbol{Q}(f),\boldsymbol{s},\epsilon\right):=\det\left(\epsilon\boldsymbol{I}_{k}+\boldsymbol{Q}_{\boldsymbol{s}}^{\text{w}*}(f)\boldsymbol{Q}_{\boldsymbol{s}}^{\text{w}}(f)\right)
\]
for small $\epsilon>0$, which is approximately the exponential of
capacity loss at a given pair $\left(\boldsymbol{s},f\right)$. In
fact, the key observation underlying the proof of Theorem \ref{theorem-Evaluate-Det-IplusBB}
is that for any $f$, the sum
\[
\sum_{\boldsymbol{s}\in{[n] \choose k}}d\left(\boldsymbol{Q}(f),\boldsymbol{s},\epsilon\right)
\]
is a \emph{constant} independent of the sampling coefficient function
$\boldsymbol{Q}$. In other words, at any given $f$, the exponential
sum of capacity loss over all states $\boldsymbol{s}$ is invariable
regardless of what samplers we employ. 

This invariant quantity is critical in identifying the minimax sampling
method. In fact, it motivates us to seek a sampling method that achieves
equivalent performance over all states $\boldsymbol{s}$. Large random
matrices exhibit sharp concentration of spectral measure, and hence
become a natural candidate to attain minimaxity.


\subsubsection{Landau-rate Sampling}

When sampling is performed at the Landau rate, the minimax capacity
loss per unit Hertz is almost solely determined by the entropy function
$\mathcal{H}(\beta)$. Specifically, when $n$ and $k$ are sufficiently
large, the minimax limit depends only on the sparsity ratio $\beta=k/n$
rather than $\left(n,k\right)$. Some implications from Theorem \ref{Theorem-Minimax-Det-Upper-Bound}
and Theorem \ref{theorem-Minimax-Sampler-BWLimited-Achievability}
are as follows.

\begin{figure}
\begin{centering}
\begin{tabular}{cc}
\includegraphics[scale=0.4]{BinaryEntropy} & \includegraphics[scale=0.4]{BinaryEntropyNormalized}\tabularnewline
(a) & (b)\tabularnewline
\end{tabular}
\par\end{centering}

\caption{\label{fig:BinaryEntropyFunction}Plot (a) illustrates $\mathcal{H}\left(\beta\right)/2$
v.s. the sparsity ratio $\beta$, which characterizes the fundamental
minimax capacity loss pert Hertz within a vanishing gap. Plot (b)
illustrates $\mathcal{H}\left(\beta\right)/(2\beta)$ v.s. $\beta$,
which corresponds approximately to the normalized capacity loss per
Hertz.}
\end{figure}

\begin{enumerate}
\item The capacity loss per unit Hertz is illustrated in Fig. \ref{fig:BinaryEntropyFunction}(a).
The capacity loss vanishes when $\beta\rightarrow1$, since Nyquist-rate
sampling is information preserving. The capacity loss divided by $\beta$
is plotted in Fig. \ref{fig:BinaryEntropyFunction}(b), which provides
a normalized view of the capacity loss. It can be seen that the normalized
loss decreases monotonically with $\beta$, indicating that the loss
is more severe in sparse channels. Note that this is different from
an LTI channel whereby sampling at the Landau rate is sufficient to
preserve all information. When the channel state is uncertain, increasing
the sampling rate above the Landau rate (but below the Nyquist rate)
effectively increases the SNR, and hence allows more information to
be harvested from the noisy sampled output.
\item The capacity loss incurred by independent random sampling meets the
fundamental minimax limit for Landau-rate sampling uniformly across
all states $\boldsymbol{s}$, which reveals that with exponentially
high probability, random sampling is optimal in terms of universal
sampling design. The capacity achievable by random sampling exhibits
very sharp concentration around the minimax limit uniformly across
all states $\boldsymbol{s}\in{[n] \choose k}$.
\item A\emph{ universality} phenomenon that arises in large random matrices
(e.g. \cite{TaoVu2012Covariance}) leads to the fact that the minimaxity
of random sampling matrices does not depend on the particular distribution
of the coefficients. For a large class of sub-Gaussian measure, as
long as all entries are jointly independent with matching moments
up to the second order, the sampling mechanism it generates is minimax
with exponentially high probability. 
\end{enumerate}
\begin{comment}
For a large pair $\left(n,k\right)$, the capacity loss achievable
by random sampling does not vary with $n$ and $k$ except for a vanishingly
small gap. One implication of this result is that independent random
sampling with more branches, while imposing a large burden on hardware
design, does not allow a better capacity loss to be achieved. For
example, we do not harvest any capacity gain if we employ $kN$-branch
independent random sampling where each branch has sampling rate $f_{\text{s}}/kN$.
It is unclear whether a more delicate random sampling method with
more branches can improve the capacity.
\end{comment}



\subsubsection{Super-Landau-Rate Sampling}

The random sampling analyzed in Theorem \ref{theorem-Minimax-Sampler-BWLimited-Achievability-Oversampling}
only involves Gaussian random sampling, and we have not shown universality
results. Some implications under super-Landau-rate sampling are as
follows.

\begin{figure}
\begin{centering}
\includegraphics[scale=0.5]{SuperLandau3DPlot}\includegraphics[scale=0.5]{SuperLandau3DPlot_Normalized}
\par\end{centering}

\begin{centering}
\includegraphics[scale=0.5]{SuperLandauPlot}\includegraphics[scale=0.5]{SuperLandauPlot_Normalized}
\par\end{centering}

\caption{\label{fig:SuperLandauBinaryEntropyFunction}The function $\frac{1}{2}\mathcal{H}\left(\beta\right)-\frac{\alpha}{2}\mathcal{H}\left(\beta/\alpha\right)$
v.s. the sparsity ratio $\beta$ and the undersampling factor $\alpha$.
Here, $\frac{1}{2}\mathcal{H}\left(\beta\right)-\frac{\alpha}{2}\mathcal{H}\left(\beta/\alpha\right)$
characterizes the fundamental minimax capacity loss pert Hertz within
a vanishing gap.}
\end{figure}

\begin{enumerate}
\item Similar to the case with Landau-rate sampling, Gaussian sampling achieves
minimax capacity loss uniformly across all states $\boldsymbol{s}$
within a large super-Landau-rate regime. The capacity gap is illustrated
in Fig. \ref{fig:SuperLandauBinaryEntropyFunction}. It can be observed
from the plot that increasing the $\alpha/\beta$ ratio improves the
capacity gap, shrinks the locus and shifts it leftwards. 
\item Theorem \ref{theorem-Minimax-Sampler-BWLimited-Achievability-Oversampling}
only concerns i.i.d. Gaussian random sampling instead of more general
independent random sampling. While we conjecture that the universality
phenomenon continues to hold for other jointly independent random
ensembles with sub-Gaussian tails and appropriate matching moments,
the mathematical analysis turns out to be more tricky than in the
Landau-rate sampling case. 
\item The capacity gain by sampling above the Landau rate depends on the
undersampling factor $\alpha$ as well. Specifically, the capacity
benefit per unit bandwidth due to super-Landau sampling is captured
by the term $\frac{1}{2}\alpha\mathcal{H}\left(\beta/\alpha\right)$.
When $\alpha\rightarrow1$, the capacity loss per Hertz reduces to
\[
\frac{1}{2}\mathcal{H}\left(\beta\right)-\frac{1}{2}\alpha\mathcal{H}\left(\frac{\beta}{\alpha}\right)=0,
\]
meaning that there is effectively no capacity loss under Nyquist-rate
sampling. This agrees with the fact that Nyquist-rate sampling is
information preserving.
\end{enumerate}

\section{Connections with Discrete-Time Sparse Channels \label{sec:Connections-with-Discrete-Time}}


\subsection{Minimax Sampler in Discrete-time Sparse Channels}

Our results have straightforward extensions to discrete-time sparse
vector channels. Specifically, consider a collection of $n$ parallel
channels. The channel input $\boldsymbol{x}\in\mathbb{R}^{n}$ is
passed though the channel and contaminated by Gaussian noise $\boldsymbol{n}\sim\mathcal{N}\left(0,\boldsymbol{I}_{n}\right)$,
yielding a channel output
\[
\boldsymbol{r}=\boldsymbol{H}\boldsymbol{x}+\boldsymbol{n},
\]
where $\boldsymbol{H}$ is some diagonal channel matrix. In particular,
at each asymptotically long timeframe, a state $\boldsymbol{s}\in{[n] \choose k}$
is generated, which dictates the set of channels available for transmission,
i.e. the transmitter can only send $\boldsymbol{x}$ at indices in
$\boldsymbol{s}$. One can then obtain $m$ measurements of the channel
output through a sensing matrix $\boldsymbol{Q}\in\mathbb{R}^{m\times n}$,
i.e. the measurements $\boldsymbol{y}\in\mathbb{R}^{m}$ can be expressed
as
\[
\boldsymbol{y}=\boldsymbol{Q}\boldsymbol{r}=\boldsymbol{Q}\left(\boldsymbol{H}\boldsymbol{x}+\boldsymbol{n}\right).
\]
The goal is then to identify a sensing matrix $\boldsymbol{Q}$ that
minimizes the worst-case capacity loss over all states $\boldsymbol{s}\in{[n] \choose k}$. 

If we abuse our notation $\boldsymbol{L}_{\boldsymbol{s}}^{\boldsymbol{Q}}$
(resp. $\boldsymbol{L}_{\boldsymbol{s}}^{\boldsymbol{Q},\text{opt}}$)
to denote the capacity loss at state $\boldsymbol{s}$ relative to
Nyquist-rate capacity without (resp. with) power control, then the
following results are immediate.

\begin{theorem}Define $\mathsf{SNR}_{\min}:=\frac{P}{k}\inf_{1\leq i\leq n}\left|\boldsymbol{H}_{ii}\right|$
and $\mathsf{SNR}_{\max}:=\frac{P}{k}\sup_{1\leq i\leq n}\left|\boldsymbol{H}_{ii}\right|$,
and suppose that $\mathsf{SNR}_{\max}\geq1$.

(i) (Landau-rate sampling) If $\alpha=\beta$ ($k=m$), then
\begin{equation}
\inf_{\boldsymbol{Q}}\max_{\boldsymbol{s}\in{[n] \choose k}}\frac{L_{\boldsymbol{s}}^{\boldsymbol{Q}}}{n}=\frac{1}{2}\left\{ \mathcal{H}\left(\beta\right)+\mathcal{O}\left(\min\left\{ \sqrt{\frac{\small\mathsf{SNR}_{\max}}{n}},\frac{\log n+\log\small\mathsf{SNR}_{\max}}{n^{1/4}}\right\} \right)+\Delta_{\mathrm{L}}\right\} ,
\end{equation}
and
\begin{equation}
\inf_{\boldsymbol{Q}}\max_{\boldsymbol{s}\in{[n] \choose k}}\frac{L_{\boldsymbol{s}}^{\boldsymbol{Q},\mathrm{opt}}}{n}=\frac{1}{2}\left\{ \mathcal{H}\left(\beta\right)+\mathcal{O}\left(\min\left\{ \sqrt{\frac{\small\mathsf{SNR}_{\max}}{n}},\frac{\log n+\log\small\mathsf{SNR}_{\max}}{n^{1/4}}\right\} \right)+\Delta_{\mathrm{L}}^{\mathrm{wf}}\right\} .
\end{equation}


(ii) (Super-Landau-rate sampling) Suppose that there is a small constant
$\delta>0$ such that $\alpha-\beta\geq\delta$ and $1-\alpha-\beta\geq\delta$.
Then
\begin{equation}
\inf_{\boldsymbol{Q}}\max_{\boldsymbol{s}\in{[n] \choose k}}\frac{L_{\boldsymbol{s}}^{\boldsymbol{Q}}}{n}=\frac{1}{2}\left\{ \mathcal{H}\left(\beta\right)-\alpha\mathcal{H}\left(\frac{\beta}{\alpha}\right)+\mathcal{O}\left(\frac{\log n}{n^{1/3}}\right)+\Delta_{\mathrm{SL}}\right\} ,
\end{equation}
and
\begin{equation}
\inf_{\boldsymbol{Q}}\max_{\boldsymbol{s}\in{[n] \choose k}}\frac{L_{\boldsymbol{s}}^{\boldsymbol{Q},\mathrm{opt}}}{n}=\frac{1}{2}\left\{ \mathcal{H}\left(\beta\right)-\alpha\mathcal{H}\left(\frac{\beta}{\alpha}\right)+\mathcal{O}\left(\frac{\log n}{n^{1/3}}\right)+\Delta_{\mathrm{SL}}^{\mathrm{wf}}\right\} .
\end{equation}


Here, $\Delta_{\mathrm{L}}$, $\Delta_{\mathrm{L}}^{\mathrm{opt}}$,
$\Delta_{\mathrm{SL}}$ and $\Delta_{\mathrm{SL}}^{\mathrm{opt}}$
are some residual terms satisfying
\[
-\frac{2}{\sqrt{\small\mathsf{SNR}_{\min}}}\leq\Delta_{\mathrm{L}},\Delta_{\mathrm{SL}}\leq\frac{\beta}{\small\mathsf{SNR}_{\min}},\quad\text{and}\quad-\frac{2}{\sqrt{\small\mathsf{SNR}_{\min}}}\leq\Delta_{\mathrm{L}}^{\mathrm{opt}},\Delta_{\mathrm{SL}}^{\mathrm{opt}}\leq\frac{\beta+\overline{A}}{\small\mathsf{SNR}_{\min}},
\]
where $\overline{A}$ is a constant defined as
\[
\overline{A}:=\min\left\{ \frac{\mathrm{tr}\left(\boldsymbol{H}\boldsymbol{H}^{*}\right)}{k\min_{1\leq i\leq n}\left|\boldsymbol{H}_{ii}^{2}\right|},\frac{\max_{1\leq i\leq n}\left|\boldsymbol{H}_{ii}^{2}\right|}{\min_{1\leq i\leq n}\left|\boldsymbol{H}_{ii}^{2}\right|}\right\} .
\]
\end{theorem}

The key observations are that in a discrete-time sparse vector channel,
the minimax capacity loss per degree of freedom again depends only
on $\beta$ and $\alpha$ modulo a vanishingly small gap. Independent
random sensing matrices and i.i.d. Gaussian sensing matrices are minimax
in terms of a channel-blind sensing matrix design, in the Landau-rate
and super-Landau-rate regimes, respectively.


\subsection{Connections with Sparse Recovery}


\subsubsection{Restricted Isometry Property (RIP)}

The readers familiar with compressed sensing \cite{Don2006,CandRomTao06}
may naturally wonder whether the optimal sampling matrices $\boldsymbol{M}$
satisfy the restricted isometry property (RIP). An RIP constant $\delta_{k}$
with respect to a matrix $\boldsymbol{M}\in\mathbb{C}^{m\times n}$
is defined (e.g. \cite{CanTao2005Decoding}) as the smallest quantity
such that
\[
\left(1-\delta_{k}\right)\left\Vert \boldsymbol{c}\right\Vert \leq\left\Vert \boldsymbol{M}_{\boldsymbol{s}}\boldsymbol{c}\right\Vert \leq\left(1+\delta_{k}\right)\left\Vert \boldsymbol{c}\right\Vert 
\]
holds for any vector $\boldsymbol{c}$ and any $\boldsymbol{s}$ of
size at most $k$ (recall that $\boldsymbol{M}_{\boldsymbol{s}}$
is a submatrix consisting of $k$ columns of $\boldsymbol{M}$). This
quantity measures how close $\boldsymbol{M}_{\boldsymbol{s}}$'s are
to orthonormal systems, and the existence of a small RIP constant
that does not scale with $\left(n,k,m\right)$ typically enables exact
sparse recovery from noiseless measurements with a sensing matrix
$\boldsymbol{M}$ \cite{CanTao2005Decoding}. 

Nevertheless, RIP is not necessary for approaching the minimax capacity
loss. Consider the Landau-rate sampling regime for example. When the
entries of $\boldsymbol{M}$ are independently generated under conditions
of Theorem \ref{theorem-Minimax-RMT-LandauRate}, one typically has
(e.g. \cite{Tao2012RMT})
\[
\sigma_{\min}\left(\boldsymbol{M}_{\boldsymbol{s}}\right)=\mathcal{O}\left(\frac{1}{k}\right),
\]
which cannot be bounded away from 0 by a constant. On the other hand,
there is no guarantee that a restricted isometric matrix $\boldsymbol{M}$
is sufficient to achieve minimaxity. An optimal sampling matrix typically
has similar spectrum as an i.i.d. Gaussian matrix, but a general restricted
isometric matrix does not necessarily exhibit similar spectrum.

We note, however, that many randomized schemes for generating a restricted
isometric matrix are natural candidates for generating minimax samplers.
As shown by our analysis, in order to obtain a desired sampling matrix
$\boldsymbol{M}$, we require $\boldsymbol{M}_{\boldsymbol{s}}$ to
yield similar spectrum over all $\boldsymbol{s}$. Many random matrices
satisfying RIP exhibit this property, and are hence minimax. 


\subsubsection{Necessary Sampling Rate}

It is well known that there exists a spectrum-blind sampling matrix
with $2k$ \emph{noiseless} measurements that admits perfect recovery
of any $k$-sparse signal. In the continuous-time counterpart, the
minimum sampling rate for perfect recovery in the absence of noise
is twice the spectrum occupancy \cite{MisEld2009}. Nevertheless,
this sampling rate does not allow zero capacity loss in our setting.
Since the channel output is contaminated by noise and thus has bandwidth
$W$, a spectrum-blind sampler is unable to suppress spectral contents
outside active subbands, and hence suffers from information rate loss
relative to Nyquist-rate capacity.

On the other hand, twice the spectrum occupancy does not have a threshold
effect in our setting, as illustrated in Figure \ref{fig:SuperLandauBinaryEntropyFunction}.
This arises from the fact that the transmitter can adapt its transmitted
signal to the instantaneous realization and sampling rate. For instance,
the spectral support of the transmitted signal may also shrink as
the sampling rate decreases, thus avoiding an infliction point on
the capacity curves.

\begin{comment}
a


\subsection{Larger Period $T_{q}$}

So far we have focused on the class of periodic sampling systems of
period $T_{q}=n/W$. A natural question arises as to whether employing
periodic sampling with a larger $T_{q}$ can decrease the minimax
capacity loss. 

Consider the class of periodic sampling systems with period $T_{q}=Nn/W$,
where $N$ is a positive integer. Then for a given state $\boldsymbol{s}$,
the sampled capacity becomes
\begin{equation}
C_{\boldsymbol{s}}^{\boldsymbol{Q}}={\displaystyle \int}_{0}^{\frac{W}{Nn}}\frac{1}{2}\log\det\left(\boldsymbol{I}_{m}+\frac{P}{W}\left(\boldsymbol{Q}(f)\boldsymbol{Q}^{*}(f)\right)^{-\frac{1}{2}}\boldsymbol{Q}_{\boldsymbol{s}}(f)\boldsymbol{H}_{\boldsymbol{s}}^{2}(f)\boldsymbol{Q}_{\boldsymbol{s}}^{*}(f)\left(\boldsymbol{Q}(f)\boldsymbol{Q}^{*}(f)\right)^{-\frac{1}{2}}\right)\mathrm{d}f,\label{eq:MatrixFormCsQ-1}
\end{equation}
where $\boldsymbol{Q}(\cdot)$ is the sampling coefficient function
dependent solely on the sampling system, which satisfies that for
any $f$, $\boldsymbol{Q}(f)$ consists of $m\times n$ block matrices
each of dimension $N\times N$. The matrix $\boldsymbol{Q}_{\boldsymbol{s}}(f)$
denotes the $mN\times kN$ matrices coming from the the block columns
at indices of $\boldsymbol{s}$. 

The following theorem characterizes a lower bound on the minimax capacity
loss. 

\begin{theorem}Consider the class of periodic sampling systems with
period $T_{q}=Nn/W$ for some integer $N>0$, and suppose that the
sampling coefficient functions $\boldsymbol{Q}(\cdot)$ are Riemann
integrable. Then the capacity loss satisfies
\begin{equation}
\inf_{\boldsymbol{Q}}\max_{\boldsymbol{s}\in{[n] \choose k}}L_{\boldsymbol{s}}^{\boldsymbol{Q}}\geq\frac{W}{2}\left\{ \beta\log\alpha-\frac{1}{\small\mathsf{SNR}_{\min}}\right\} .\label{eq:MinimaxGapGeneralConverse}
\end{equation}
\end{theorem}

\begin{proof}The above theorem is proved through the following useful
bounds, which is proved in Appendix \ref{eq:MinimaxGapLowerBoundGeneral}.

\begin{theorem}\label{theorem-Minimax-General-Upper-Bound}Consider
a matrix $\boldsymbol{B}$ that consists of $m\times n$ blocks each
of dimensions $N\times N$. Suppose that $\boldsymbol{B}\boldsymbol{B}^{*}=\boldsymbol{I}_{mN}$.
Then we have
\begin{align*}
\sum_{\boldsymbol{s}\in{[n] \choose k}}\left(\det\left(\epsilon\boldsymbol{I}_{kN}+\boldsymbol{B}_{\boldsymbol{s}}^{*}\boldsymbol{B}_{\boldsymbol{s}}\right)\right)^{\frac{1}{kN}} & \leq{n \choose k}\epsilon+\frac{m}{k}{n-1 \choose k-1}.
\end{align*}


Suppose that $\boldsymbol{B}_{i}$ $(1\leq i\leq p)$ are all matrices
that obey $\boldsymbol{B}_{i}\boldsymbol{B}_{i}^{*}=\boldsymbol{I}_{mN}$
and that contain $m\times n$ blocks each of dimension $N\times N$.
Then one has
\begin{align*}
\min_{\boldsymbol{s}\in{[n] \choose k}}\sum_{i=1}^{p}\frac{1}{npN}\log\left(\det\left(\epsilon\boldsymbol{I}_{kN}+\left(\boldsymbol{B}_{i}\right)_{\boldsymbol{s}}^{*}\left(\boldsymbol{B}_{i}\right)_{\boldsymbol{s}}\right)\right) & \leq-\beta\log\alpha+\epsilon.
\end{align*}
\end{theorem}

Note that $\boldsymbol{Q}^{\text{w}}(f)$ has orthonormal rows for
any $f$, and $\boldsymbol{Q}^{\text{w}}(f)$ is assumed to be Riemann
integrable. For any $\varepsilon>0$, we can find a sufficiently large
$p$ such that 
\[
{\displaystyle \int}_{0}^{W/nN}\log\det\left(\epsilon\boldsymbol{I}_{kN}+\boldsymbol{Q}_{\boldsymbol{s}}^{\text{w}*}(f)\boldsymbol{Q}_{\boldsymbol{s}}^{\text{w}}(f)\right)\mathrm{d}f\leq\varepsilon+\frac{W}{pnN}\sum_{i=1}^{p}\log\det\left(\epsilon\boldsymbol{I}_{kN}+\boldsymbol{Q}_{\boldsymbol{s}}^{\text{w}*}\left(\frac{iW}{pnN}\right)\boldsymbol{Q}_{\boldsymbol{s}}^{\text{w}}\left(\frac{iW}{pnN}\right)\right).
\]
Since $\varepsilon$ can be arbitrarily small, applying Theorem \ref{theorem-Evaluate-Det-IplusBB}
immediately yields that for any $\boldsymbol{Q}(\cdot)$:
\[
\min_{\boldsymbol{s}\in{[n] \choose k}}{\displaystyle \int}_{0}^{W/nN}\log\det\left(\epsilon\boldsymbol{I}_{kN}+\boldsymbol{Q}_{\boldsymbol{s}}^{\text{w}*}(f)\boldsymbol{Q}_{\boldsymbol{s}}^{\text{w}}(f)\right)\mathrm{d}f\leq W\left(\beta\log\frac{1}{\alpha}+\epsilon\right).
\]
This together with (\ref{eq:GsQNormalizedApproximationLandauSamplingLB})
leads to
\begin{align*}
\inf_{\boldsymbol{Q}}\max_{\boldsymbol{s}\in{[n] \choose k}}L_{\boldsymbol{s}}^{\boldsymbol{Q}} & \geq-\sup_{\boldsymbol{Q}}\min_{\boldsymbol{s}\in{[n] \choose k}}\frac{1}{2}{\displaystyle \int}_{0}^{W/nN}\log\det\left(\frac{1}{\small\mathsf{SNR}_{\min}}\boldsymbol{I}_{kN}+\boldsymbol{Q}_{\boldsymbol{s}}^{\text{w}*}(f)\boldsymbol{Q}_{\boldsymbol{s}}^{\text{w}}(f)\right)\mathrm{d}f\\
 & \geq\frac{W}{2}\left\{ \beta\log\alpha-\frac{1}{\small\mathsf{SNR}_{\min}}\right\} .
\end{align*}
\end{proof}

If we compare (\ref{eq:MinimaxGapGeneralConverse}) and (\ref{eq:MinimaxCapacityGapLowerBoundLargeN}),
the capacity loss per unit bandwidth due to limiting the period to
$n/W$ is upper bounded by the term 
\[
L_{\alpha,\beta}=\frac{1}{2}\left(\mathcal{H}\left(\beta\right)-\alpha\mathcal{H}\left(\frac{\beta}{\alpha}\right)-\beta\log\alpha\right)=-\frac{1}{2}\left(1-\beta\right)\log\left(1-\beta\right)+\frac{\alpha}{2}\left(1-\frac{\beta}{\alpha}\right)\log\left(1-\frac{\beta}{\alpha}\right).
\]
If $\beta\leq\alpha\ll1$, or $\beta\ll\alpha$, we have $\left|L_{\alpha,\beta}\right|\leq\frac{\beta}{2}$,
which is at most a portion of $\frac{\beta}{\mathcal{H}(\beta)}\approx\frac{1}{\log\frac{2}{\beta}}$
and can be quite small when $\beta\rightarrow0$. 

We caution, however, that the bound (\ref{eq:MinimaxGapLowerBoundGeneral})
is a loose lower bound with no achievability guarantee. In fact, this
bound can be quite loose when $n$ and $k$ are both large numbers.
No sampling schemes that can achieve this bound have been identified,
and we conjecture that the bound is over optimistic. 

We conjecture that when $n$, $m$, and $k$ are all large numbers,
the bound (\ref{eq:MinimaxCapacityGapLowerBoundLargeN}) is near-tight
except for some vanishingly small residual terms, i.e. sampling with
periodic modulation and low-pass filters is near minimax. In order
to solve the problem, one can study the following equivalent algebraic
problem: consider any matrix $\boldsymbol{B}$ that contains $m\times n$
blocks each of dimensions $N\times N$, then how to obtain a tight
estimate on the following value:
\[
\max_{\boldsymbol{B}:\boldsymbol{B}\boldsymbol{B}^{*}=\boldsymbol{I}_{mN}}\min_{\boldsymbol{s}\in{[n] \choose k}}\log\det\left(\epsilon\boldsymbol{I}_{kN}+\boldsymbol{B}_{\boldsymbol{s}}^{*}\boldsymbol{B}_{\boldsymbol{s}}\right).
\]
\end{comment}



\section{Conclusions\label{sec:Conclusion-Minimax}}

We have investigated optimal universal sampling design from a capacity
perspective. In order to evaluate the loss due to universal sub-Nyquist
sampling design, we introduced the notion of sampled capacity loss
relative to Nyquist-rate capacity, and characterize overall robustness
of the sampling design through the minimax capacity loss metric. Specifically,
we have determined the fundamental minimax limit on the sampled capacity
loss achievable by a class of channel-blind periodic sampling system.
This minimax limit turns out to be a constant that only depends on
the band sparsity ratio and undersampling factor, modulo a residual
term that vanishes in the SNR and the number of subbands. Our results
demonstrate that with exponentially high probability, random sampling
is minimax in terms of a universal sampler design. This highlights
the power of random sampling methods in the channel-blind design.
In addition, our results extend to discrete-time counterparts without
difficulty. We demonstrate that independent random sensing matrices
are minimax in discrete-time sparse vector channels. 

It remains to characterize the fundamental minimax capacity loss when
sampling is performed below the Landau rate, and to be seen whether
random sampling is still optimal in the sub-Landau-rate regime. It
would also be interesting to extend this framework to situations beyond
compound multiband channels, and our notion of sampled capacity loss
will be useful in evaluating the robustness for these scenarios. Our
framework and results may also be appropriate for other channels with
state where sparsity exists in other transform domains. In addition,
when it comes to multiple access channels or random access channels
\cite{ElGamalKim2012}, it is not clear how to find a channel-blind
sampler that is robust for the entire capacity region.

\begin{comment}
a


\section{Bayes Capacity Loss }

\textbf{\emph{(Please ignore this section (and all Bayes metric) in
this version -- will update it later)}}

Similar to the minimax sampling case, we can again cast the Bayesian
optimization problem to a simpler algebraic problem in the high-SNR
regime. In the current section, we consider Landau rate sampling.

Suppose that each index set ${\bf s}$ has prior probability $\mu({\bf s})$.
In the high-SNR regime, the average capacity loss is given by
\begin{align*}
G_{\mu}^{{\bf Q}}= & \sum_{{\bf s}\in\mathcal{S}}\mu({\bf s})\left\{ \frac{W}{2n}\log\det\left({\bf I}+\frac{P}{\beta W}{\bf H}_{[s]}^{2}\right)-\frac{W}{2n}\log\det\left({\bf I}+\frac{P}{\beta W}{\bf Q}_{{\bf s}}^{\dagger}{\bf H}_{{\bf s}}^{2}{\bf Q}_{{\bf s}}^{\dagger T}\right)\right\} \\
= & -\sum_{{\bf s}\in\mathcal{S}}\mu({\bf s})\frac{W}{2n}\log\det\left(\frac{\beta W}{P}{\bf H}_{{\bf s}}^{-2}+{\bf Q}_{{\bf s}}^{\dagger T}{\bf Q}_{{\bf s}}^{\dagger}\right)+O\left(\frac{1}{\text{SNR}_{\min}}\right)
\end{align*}
Hence, a lower bound is given by
\[
G_{{\bf s}}^{{\bf Q}}\geq-\frac{W}{2n}\log\left(\prod_{{\bf s}\in\mathcal{S}}\det\left[\frac{1}{\text{SNR}_{\min}}{\bf I}+{\bf Q}_{{\bf s}}^{\dagger T}{\bf Q}_{{\bf s}}^{\dagger}\right]^{\mu({\bf s})}\right)+O\left(\frac{1}{\text{SNR}_{\min}}\right).
\]
That said, the Bayesian optimization problem boils down to evaluating
\begin{equation}
\sup_{{\bf Q}}\log\left(\prod_{{\bf s}\in\mathcal{S}}\det\left[\frac{1}{\text{SNR}_{\min}}{\bf I}+{\bf Q}_{{\bf s}}^{\dagger T}{\bf Q}_{{\bf s}}^{\dagger}\right]^{\mu({\bf s})}\right).\label{eq:MetricBaysianSampling}
\end{equation}


The following theorem develops an upper bound on the metric (\ref{eq:MetricBaysianSampling}).

\begin{theorem}\label{theorem:BaysianUpperBound}Consider all $k\times n$
sampling matrices ${\bf Q}$ with $\beta=k/n$. If we define $\mu_{\min}:=\min\left\{ \mu({\bf s})\mid{\bf s}\in{[n] \choose k}\right\} $,
then the following inequality holds
\begin{equation}
\sup_{{\bf Q}}\frac{1}{n}\log\left(\prod_{{\bf s}\in\mathcal{S}}\det\left[\frac{1}{\mathrm{SNR}_{\min}}{\bf I}+{\bf Q}_{{\bf s}}^{\dagger T}{\bf Q}_{{\bf s}}^{\dagger}\right]^{\mu({\bf s})}\right)\leq-\beta\frac{\mathcal{H}\left(\left\{ \left.\frac{\sum_{{\bf s}:i\in{\bf s}}\mu({\bf s})}{k}\right|1\leq i\leq n\right\} \right)}{\log n}+\beta\log\alpha,\label{eq:MinimaxDetUpperBoundLargeN-BayesGeneral}
\end{equation}
and hence
\[
\inf_{{\bf Q}}\mathbb{E}_{\mu}\left(G_{{\bf s}}^{{\bf Q}}\right)\geq\frac{W}{2}\beta\frac{\mathcal{H}\left(\left\{ \left.\frac{\sum_{{\bf s}:i\in{\bf s}}\mu({\bf s})}{k}\right|1\leq i\leq n\right\} \right)}{\log n}-\frac{W}{2}\beta\log\alpha+O\left(\frac{1}{\mathrm{SNR}_{\min}}\right).
\]


Additionally, consider the class of sampling matrices $\mathcal{Q}_{\epsilon}^{\text{B}}$
with respect to a small $\epsilon>0$ such that if ${\bf Q}\in\mathcal{Q}_{\epsilon}^{\text{B}}$,
then
\begin{equation}
\sum_{{\bf s}\in{[n] \choose k}}\mu({\bf s})\log\det\left(\epsilon{\bf I}+{\bf Q}_{{\bf s}}^{\dagger T}{\bf Q}_{{\bf s}}^{\dagger}\right)\leq\sum_{{\bf s}\in{[n] \choose k}}\mu({\bf s})\log\det\left({\bf Q}_{{\bf s}}^{\dagger T}{\bf Q}_{{\bf s}}^{\dagger}\right)+o(n)\label{eq:ConditionLandauSamplingWellCondition-Bayes}
\end{equation}
 holds for all ${\bf s}\in{[n] \choose k}$. Then for any $\text{SNR}_{\min}\geq\epsilon^{-1}$,
we have
\begin{align}
\sup_{{\bf Q}\in\mathcal{Q}_{\epsilon}^{\text{B}}}\frac{1}{n}\log\left(\prod_{{\bf s}\in\mathcal{S}}\det\left[\frac{1}{\mathrm{SNR}_{\min}}{\bf I}+{\bf Q}_{{\bf s}}^{\dagger T}{\bf Q}_{{\bf s}}^{\dagger}\right]^{\mu({\bf s})}\right) & \leq-\beta\frac{\mathcal{H}\left(\left\{ \left.\frac{\sum_{{\bf s}:i\in{\bf s}}\mu({\bf s})}{k}\right|1\leq i\leq n\right\} \right)}{\log n}+\beta\log\alpha\nonumber \\
 & \quad+{n \choose k}\mu_{\min}\left(\alpha\mathcal{H}\left(\frac{\beta}{\alpha}\right)-\beta\log\alpha-\mathcal{H}\left(\beta\right)\right)+o(1).\label{eq:MinimaxDetUpperBoundLargeN-BayesTighter}
\end{align}
That said,
\begin{align}
\inf_{{\bf Q}\in\mathcal{Q}_{\epsilon}^{\text{B}}}\mathbb{E}_{\mu}\left(G_{{\bf s}}^{{\bf Q}}\right) & \geq\frac{W}{2}\beta\frac{\mathcal{H}\left(\left\{ \left.\frac{\sum_{{\bf s}\in{[n] \choose k}:i\in{\bf s}}\mu({\bf s})}{k}\right|1\leq i\leq n\right\} \right)}{\log n}-\frac{W}{2}\beta\log\alpha\nonumber \\
 & \quad\quad-\frac{W}{2}{n \choose k}\mu_{\min}\left(\alpha\mathcal{H}\left(\frac{\beta}{\alpha}\right)-\beta\log\alpha-\mathcal{H}(\beta)\right)+O\left(\frac{1}{\mathrm{SNR}_{\min}}\right).\label{eq:BayesCapacityGapLowerBound}
\end{align}
\end{theorem}

\begin{proof}See Appendix \ref{sec:Proof-of-Theorem-BayesianUpperBound}.\end{proof}

The first term in (\ref{eq:MinimaxDetUpperBoundLargeN-BayesGeneral})
arises by bounding the gain obtained by allocating the power to different
columns, while the second term arises by bounding the empirical mean
of all correlation matrices. In other words, the first term concerns
the magnitude of each column, while the second term concerns the angles
among different columns. 

If in addition ${\bf Q}$ is not too ill-conditioned as stated in
Condition (\ref{eq:ConditionLandauSamplingWellCondition-Bayes}),
then we can further tighter the bound via an additional term ${n \choose k}\mu_{\min}\left(\alpha\mathcal{H}\left(\frac{\beta}{\alpha}\right)-\beta\log\alpha-\mathcal{H}(\beta)\right)$.
Two scenarios are worth mentioning: 

(1) if $\kappa\mu_{\min}=\mu_{\max}:=\max_{s\in\mathcal{S}_{K,n}}\mu(s)$,
then $\mu_{\min}\geq\left[\kappa{n \choose k}\right]^{-1}$, and we
can further bound
\begin{align*}
\sup_{{\bf Q}}\frac{1}{n}\log\left(\prod_{{\bf s}\in\mathcal{S}}\det\left[\frac{1}{\text{SNR}_{\min}}{\bf I}+{\bf Q}_{{\bf s}}^{\dagger T}{\bf Q}_{{\bf s}}^{\dagger}\right]^{\mu({\bf s})}\right) & \leq-\beta\frac{\mathcal{H}\left(\left\{ \left.\frac{\sum_{{\bf s}\in{[n] \choose k}:i\in{\bf s}}\mu({\bf s})}{k}\right|1\leq i\leq n\right\} \right)}{\log n}\\
 & \quad\quad+\beta\log\alpha+\frac{1}{\kappa}\left(\alpha\mathcal{H}\left(\frac{\beta}{\alpha}\right)-\beta\log\alpha-\mathcal{H}(\beta)\right);
\end{align*}
If $\kappa=1$, that is, when $\mu({\bf s})$ are all equal, the bound
reduces to $\alpha\mathcal{H}\left(\frac{\beta}{\alpha}\right)-\mathcal{H}(\beta)$,
which coincides with the minimax bound (\ref{eq:MinimaxDetUpperBoundLargeN}).
That said, when all states are used equally likely, the Bayesian bound
coincides with the minimax bound.

(2) if $\mu_{\min}$ is not bounded away from zero, this term vanishes,
and the bound reduces to the general bound in (\ref{eq:MinimaxDetUpperBoundLargeN-BayesGeneral})

This upper bound is in general not tight since we do not know how
to simultaneously optimize the magnitudes of all columns and the angles
among them. We can, however, achieve a lower bound through random
sampling with appropriate weighting of magnitudes. This way we optimize
the magnitudes as bounded by the first term of (\ref{eq:MinimaxDetUpperBoundLargeN-BayesGeneral}),
and the loss due to suboptimal angle selection under Landau-rate sampling
is characterized in the following theorem.

\begin{theorem}\label{theorem-Bayes-Sampler-Achievability}Suppose
that the prior distribution of state ${\bf s}$ is $\mu({\bf s})$,
and define
\[
\forall j\text{ }(1\leq j\leq n),\quad\quad\hat{a}_{j}:=\sqrt{\sum_{{\bf s}:j\in{\bf s}}\mu({\bf s})}.
\]
Generate a $m\times n$ dimensional random ensemble ${\bf M}=\left(\xi_{ij}\right)_{1\leq i\leq k,1\leq j\leq n}$
such that $\xi_{ij}$ are jointly independent with zero mean and satisfy
${\bf Var}\left(\xi_{ij}\right)=\hat{a}_{j}^{2}$. Besides, each $\xi_{ij}/\hat{a}_{j}$
either are bounded  in magnitude by some constant $D$, or have logarithmic
Sobolev constant $c_{{\bf LS}}$. Assume that $\beta:=k/n$ is independent
of $\left(k,n\right)$. Under these assumptions, for any small $\epsilon>0$,
the following holds with probability exceeding $1-4\exp\left(-c_{9}\epsilon^{2}k^{2}/\hat{a}_{\max}^{2}\right)$:
\begin{align}
\frac{1}{n}\sum_{{\bf s}\in{[n] \choose k}}\mu({\bf s})\log\left(\det\left[\epsilon{\bf I}+{\bf M}_{{\bf s}}^{\dagger}{\bf M}_{{\bf s}}^{\dagger T}\right]\right) & =-\beta\frac{\mathcal{H}\left(\left\{ \left.\frac{\sum_{{\bf s}\in{[n] \choose k}:i\in{\bf s}}\mu({\bf s})}{k}\right|1\leq i\leq n\right\} \right)}{\log n}+\beta\log\beta\\
 & \quad-\mathcal{H}(\beta)-\beta\log\left(\sqrt[k]{S_{k}}\right)+O\left(\frac{\log k}{k}+\sqrt{\epsilon}\right),\label{eq:BayesLogDetAchievability}
\end{align}
where $S_{k}:=\frac{\sum_{{\bf s}\in{[n] \choose k}}\prod_{j:j\in{\bf s}}\hat{a}_{j}^{2}}{{n \choose k}}$
is the symmetric mean of degree $k$ in the $n$ variables $\left\{ \hat{a}_{i}^{2}\mid1\leq i\leq n\right\} $.
That said, the average capacity loss associated with ${\bf M}$ satisfies
\begin{equation}
\mathbb{E}_{\mu}G_{s}^{{\bf Q}}\leq\frac{W}{2}\left[\beta\frac{\mathcal{H}\left(\left\{ \left.\frac{\sum_{{\bf s}\in{[n] \choose k}:i\in{\bf s}}\mu({\bf s})}{k}\right|1\leq i\leq n\right\} \right)}{\log n}-\beta\log\beta+\mathcal{H}\left(\beta\right)+\beta\log\left(\sqrt[k]{S_{k}}\right)\right]+O\left(\frac{\log k}{k}+\frac{1}{\mathrm{SNR}_{\min}}+\sqrt{\epsilon}\right).\label{eq:BayesCapacityGapAchievability}
\end{equation}
with probability exceeding $1-8\exp\left(-c_{9}\epsilon^{2}k\log k/\hat{a}_{\max}^{2}\right)$.
\end{theorem}

\begin{proof}See Appendix \ref{sec:Proof-of-Theorem--Bayes-Sampler-Achievability}.\end{proof}

If we compare the achievability bound (\ref{eq:BayesCapacityGapAchievability})
and the converse bound (\ref{eq:BayesCapacityGapLowerBound}), we
can find that random sampling achieves a capacity that is within $\mathcal{H}(\beta)+\beta\log\left(\sqrt[k]{S_{k}}\right)$
bits per Hertz from optimality. When the channel is sparse (i.e. when
$\beta$ is small), the gap approaches 0, and hence it is near optimal
in the Bayesian sense. 

Although Gaussian ensembles are still near-optimal in minimizing the
average capacity loss, the random ensembles are no longer i.i.d. generated
-- the strength designated for each column needs to be weighted according
to the likelihood that the corresponding subband is included in the
state ${\bf s}$. This is intuitively meaningful since the more likely
a subband will be used, the more strength we should place for this
subband in order to preserve more information conveyed through this
subband. This implication holds under our assumptions that water-filling
power allocation is not adopted due to lack of CSI at the transmitter
side.
\end{comment}


\appendix


\section{Proof of Theorem \ref{theorem-Minimax-Final-water-filling}\label{sec:Proof-of-Theorem-Minimax-Final-water-filling}}

We would like to bound the gap between $C_{\boldsymbol{s}}^{\text{wf}}$
and $C_{\boldsymbol{s}}^{\text{eq}}$. In fact, the equation that
determines the water level implies that
\begin{align}
P & ={\displaystyle \int}_{0}^{W/n}\sum_{i=1}^{k}\left(\nu-\frac{1}{\left(\boldsymbol{H}_{\boldsymbol{s}}(f)\right)_{ii}^{2}}\right)^{+}\mathrm{d}f\geq{\displaystyle \int}_{0}^{W/n}\sum_{i=1}^{k}\left(\nu-\frac{1}{\left(\boldsymbol{H}_{\boldsymbol{s}}(f)\right)_{ii}^{2}}\right)\mathrm{d}f
\end{align}
which in turns yields
\[
\nu\leq\frac{P}{\beta W}+\frac{{\displaystyle \int}_{0}^{W/n}\sum_{i=1}^{k}\frac{1}{\left(\boldsymbol{H}_{\boldsymbol{s}}(f)\right)_{ii}^{2}}\mathrm{d}f}{\beta W}.
\]
With the above bound on the water level, the capacity can be bounded
above as
\begin{align*}
C_{\boldsymbol{s}}^{\mathrm{opt}} & ={\displaystyle \int}_{0}^{W/n}\frac{1}{2}\sum_{i=1}^{k}\log^{+}\left(\nu\left(\boldsymbol{H}_{\boldsymbol{s}}(f)\right)_{ii}^{2}\right)\mathrm{d}f\\
 & \leq{\displaystyle \int}_{0}^{W/n}\frac{1}{2}\sum_{i=1}^{k}\log^{+}\left(\frac{{\displaystyle \int}_{0}^{W/n}\sum_{j=1}^{k}\frac{\left(\boldsymbol{H}_{\boldsymbol{s}}(f)\right)_{ii}^{2}}{\left(\boldsymbol{H}_{\boldsymbol{s}}(f)\right)_{jj}^{2}}\mathrm{d}f}{\beta W}+\frac{P}{\beta W}\left(\boldsymbol{H}_{\boldsymbol{s}}(f)\right)_{ii}^{2}\right)\mathrm{d}f\\
 & \leq{\displaystyle \int}_{0}^{W/n}\frac{1}{2}\sum_{i=1}^{k}\log\left(A+\frac{P}{\beta W}\left(\boldsymbol{H}_{\boldsymbol{s}}(f)\right)_{ii}^{2}\right)\mathrm{d}f\\
 & ={\displaystyle \int}_{0}^{W/n}\log\det\left(A\boldsymbol{I}+\frac{P}{\beta W}\boldsymbol{H}_{\boldsymbol{s}}^{2}(f)\right)\mathrm{d}f,
\end{align*}
where $A:=\max_{\boldsymbol{s},i}\frac{{\displaystyle \int}_{0}^{W/n}\sum_{j=1}^{k}\frac{\left(\boldsymbol{H}_{\boldsymbol{s}}(f)\right)_{ii}^{2}}{\left(\boldsymbol{H}_{\boldsymbol{s}}(f)\right)_{jj}^{2}}\mathrm{d}f}{\beta W}$.
One can easily verify that $A\geq1$. Therefore, 
\begin{align*}
C_{\boldsymbol{s}}^{\mathrm{opt}}-C_{\boldsymbol{s}}^{P_{\text{eq}}} & \leq{\displaystyle \int}_{0}^{W/n}\log\det\left(A\boldsymbol{I}+\frac{P}{\beta W}\boldsymbol{H}_{\boldsymbol{s}}^{2}(f)\right)\mathrm{d}f-{\displaystyle \int}_{0}^{W/n}\log\det\left(\boldsymbol{I}+\frac{P}{\beta W}\boldsymbol{H}_{\boldsymbol{s}}^{2}(f)\right)\mathrm{d}f\\
 & \leq{\displaystyle \int}_{0}^{W/n}\sum_{i=1}^{k}\log\frac{A+\frac{P}{\beta W}\left(\boldsymbol{H}_{\boldsymbol{s}}^{2}(f)\right)_{ii}}{1+\frac{P}{\beta W}\left(\boldsymbol{H}_{\boldsymbol{s}}^{2}(f)\right)_{ii}}\mathrm{d}f\leq{\displaystyle \int}_{0}^{W/n}\sum_{i=1}^{k}\log\frac{A+\inf_{0\leq f\leq W}\frac{P}{\beta W}\frac{\left|H(f,\boldsymbol{s})\right|}{\sqrt{\mathcal{S}_{\eta}(f)}}}{1+\inf_{0\leq f\leq W}\frac{P}{\beta W}\frac{\left|H(f,\boldsymbol{s})\right|}{\sqrt{\mathcal{S}_{\eta}(f)}}}\mathrm{d}f\\
 & \leq\beta W\log\left(1+\frac{A-1}{1+\small\mathsf{SNR}_{\min}}\right)\\
 & \leq W\frac{\beta\left(A-1\right)}{1+\small\mathsf{SNR}_{\min}}.
\end{align*}
We also observe that
\begin{align*}
A & =\max_{\boldsymbol{s},i}\frac{{\displaystyle \int}_{0}^{W/n}\sum_{j=1}^{k}\frac{\left(\boldsymbol{H}_{\boldsymbol{s}}(f)\right)_{ii}^{2}}{\left(\boldsymbol{H}_{\boldsymbol{s}}(f)\right)_{jj}^{2}}\mathrm{d}f}{\beta W}\leq\frac{{\displaystyle \int}_{0}^{W/n}\sum_{j=1}^{k}\frac{\left(\boldsymbol{H}_{\boldsymbol{s}}(f)\right)_{ii}^{2}}{\inf_{f,\boldsymbol{s}}\frac{\left|H(f,\boldsymbol{s})\right|^{2}}{\mathcal{S}_{\eta}(f)}}\mathrm{d}f}{\beta W}\\
 & \leq\min\left\{ \frac{\max_{\boldsymbol{s}\in{[n] \choose k}}\int_{0}^{W}\frac{\left|H(f,\boldsymbol{s})\right|^{2}}{\mathcal{S}_{\eta}(f)}\mathrm{d}f}{\beta W\inf_{0\leq f\leq W,\boldsymbol{s}\in{[n] \choose k}}\frac{\left|H(f,\boldsymbol{s})\right|^{2}}{\mathcal{S}_{\eta}(f)}},\frac{\sup_{0\leq f\leq W,\boldsymbol{s}\in{[n] \choose k}}\frac{\left|H(f,\boldsymbol{s})\right|^{2}}{\mathcal{S}_{\eta}(f)}}{\inf_{0\leq f\leq W,\boldsymbol{s}\in{[n] \choose k}}\frac{\left|H(f,\boldsymbol{s})\right|^{2}}{\mathcal{S}_{\eta}(f)}}\right\} .
\end{align*}
Combining the above bounds and Theorem \ref{theorem-Minimax-Final}
completes the proof.


\section{Proof of Theorem \ref{theorem-Evaluate-Det-IplusBB}\label{sec:Proof-of-Theorem-Det-I-plus-BB}}

Before proving the results, we first state two facts. Consider any
$m\times m$ matrix $\boldsymbol{A}$, and list the eigenvalues of
$\boldsymbol{A}$ as $\lambda_{1},\cdots,\lambda_{m}$. Define the
characteristic polynomial of $\boldsymbol{A}$ as
\[
p_{\boldsymbol{A}}(t)=\det\left(t\boldsymbol{I}-\boldsymbol{A}\right)=t^{m}-S_{1}(\lambda_{1},\cdots,\lambda_{m})t^{m-1}+\cdots+(-1)^{m}S_{m}(\lambda_{1},\cdots,\lambda_{m}),
\]
where $S_{l}(\lambda_{1},\cdots,\lambda_{m})$ is the $l$th elementary
symmetric function of $\lambda_{1},\cdots,\lambda_{m}$ defined as
follows: 
\[
S_{l}(\lambda_{1},\cdots,\lambda_{m}):=\sum_{1\leq i_{1}<\cdots<i_{l}\leq m}\prod_{j=1}^{l}\lambda_{i_{j}}.
\]
We also define $E_{l}(\boldsymbol{A})$ as the sum of determinants
of all $l$-by-$l$ principal minors of $\boldsymbol{A}$. According
to \cite[Theorem 1.2.12]{HornJohnson}, $S_{l}(\lambda_{1},\cdots,\lambda_{m})=E_{l}(\boldsymbol{A})$
holds for all $1\leq l\leq m$. After a little manipulation we obtain
\begin{equation}
\det\left(t\boldsymbol{I}+\boldsymbol{A}\right)=t^{m}+E_{1}(\boldsymbol{A})t^{m-1}+\cdots+E_{m}(\boldsymbol{A}).\label{eq:DetIplusA_via_PrincipalMinor}
\end{equation}


Another fact we would like to stress is the entropy formula of binomial
coefficients. Specifically, for any $0<k<n$, one has \cite[Page 43]{Aoki1998Entropy}
\[
\frac{e^{n\mathcal{H}\left(\beta\right)}}{n+1}\leq{n \choose k}\leq e^{n\mathcal{H}\left(\beta\right)},
\]
and hence
\begin{equation}
\mathcal{H}\left(\beta\right)-\frac{\log(n+1)}{n}\leq\frac{1}{n}\log{n \choose k}\leq\mathcal{H}\left(\beta\right),\label{eq:EntropyApproximation}
\end{equation}
where $\mathcal{H}(x):=x\log\frac{1}{x}+(1-x)\log\frac{1}{1-x}$ denotes
the entropy function.

Now we are in a position to derive the proof for our main results. 

(1) Consider an $m\times n$ matrix $\boldsymbol{B}$ with orthonormal
rows ($m\geq k$), i.e. $\boldsymbol{B}\boldsymbol{B}^{*}=\boldsymbol{I}_{m}$.
$ $ Using this identity (\ref{eq:DetIplusA_via_PrincipalMinor}),
we can derive
\begin{align}
\sum_{\boldsymbol{s}\in{[n] \choose k}}\det\left(\epsilon\boldsymbol{I}_{m}+\boldsymbol{B}_{\boldsymbol{s}}\boldsymbol{B}_{\boldsymbol{s}}^{*}\right) & =\sum_{\boldsymbol{s}\in{[n] \choose k}}\left\{ \epsilon^{m}+\sum_{l=1}^{m}\epsilon^{m-l}E_{l}\left(\boldsymbol{B}_{\boldsymbol{s}}\boldsymbol{B}_{\boldsymbol{s}}^{*}\right)\right\} \nonumber \\
 & =\epsilon^{m}{n \choose k}+\sum_{l=1}^{k}\epsilon^{m-l}\sum_{\boldsymbol{s}\in{[n] \choose k}}E_{l}\left(\boldsymbol{B}_{\boldsymbol{s}}\boldsymbol{B}_{\boldsymbol{s}}^{*}\right),\label{eq:DecompositionPrincipalMinor}
\end{align}
where the last equality follows by observing that any $l$th order
($l>k$) minor of $\boldsymbol{B}_{\boldsymbol{s}}\boldsymbol{B}_{\boldsymbol{s}}^{*}$
is rank deficient, and hence $E_{l}\left(\boldsymbol{B}_{\boldsymbol{s}}\boldsymbol{B}_{\boldsymbol{s}}^{*}\right)=0$. 

Consider an index set $\boldsymbol{r}\in{[m] \choose l}$ with $l\leq k$,
and denote by $\left(\boldsymbol{B}_{\boldsymbol{s}}\boldsymbol{B}_{\boldsymbol{s}}^{*}\right)_{\boldsymbol{r}}$
the submatrix of $\boldsymbol{B}_{\boldsymbol{s}}\boldsymbol{B}_{\boldsymbol{s}}^{*}$
with rows and columns coming from the index set $\boldsymbol{r}$.
One can then verify that
\[
\det\left(\left(\boldsymbol{B}_{\boldsymbol{s}}\boldsymbol{B}_{\boldsymbol{s}}^{*}\right)_{\boldsymbol{r}}\right)=\det\left(\boldsymbol{B}_{\boldsymbol{r},\boldsymbol{s}}\boldsymbol{B}_{\boldsymbol{r},\boldsymbol{s}}^{*}\right)=\sum_{\tilde{\boldsymbol{r}}\in{\boldsymbol{s} \choose l}}\det\left(\boldsymbol{B}_{\boldsymbol{r},\tilde{\boldsymbol{r}}}\boldsymbol{B}_{\boldsymbol{r},\tilde{\boldsymbol{r}}}^{*}\right),
\]
where the last equality arises from the Cauchy-Binet formula (e.g.
\cite{Tao2012RMT}). Some algebraic manipulation yields that for any
$l\leq k$:
\begin{align}
\sum_{\boldsymbol{s}\in{[n] \choose k}}E_{l}\left(\boldsymbol{B}_{\boldsymbol{s}}\boldsymbol{B}_{\boldsymbol{s}}^{*}\right) & =\sum_{\boldsymbol{s}\in{[n] \choose k}}\sum_{\boldsymbol{r}\in{[m] \choose l}}\det\left(\left(\boldsymbol{B}_{\boldsymbol{s}}\boldsymbol{B}_{\boldsymbol{s}}^{*}\right)_{\boldsymbol{r}}\right)=\sum_{\boldsymbol{s}\in{[n] \choose k}}\sum_{\boldsymbol{r}\in{[m] \choose l}}\sum_{\tilde{\boldsymbol{r}}\in{\boldsymbol{s} \choose l}}\det\left(\boldsymbol{B}_{\boldsymbol{r},\tilde{\boldsymbol{r}}}\boldsymbol{B}_{\boldsymbol{r},\tilde{\boldsymbol{r}}}^{*}\right)\nonumber \\
 & =\sum_{\boldsymbol{r}\in{[m] \choose l}}\sum_{\tilde{\boldsymbol{r}}\in{[n] \choose l}}\sum_{\boldsymbol{s}:\tilde{\boldsymbol{r}}\subseteq\boldsymbol{s}}\det\left(\boldsymbol{B}_{\boldsymbol{r},\tilde{\boldsymbol{r}}}\boldsymbol{B}_{\boldsymbol{r},\tilde{\boldsymbol{r}}}^{*}\right)\nonumber \\
 & \overset{(a)}{=}\sum_{\boldsymbol{r}\in{[m] \choose l}}\sum_{\tilde{\boldsymbol{r}}\in{[n] \choose l}}{n-l \choose k-l}\det\left(\boldsymbol{B}_{\boldsymbol{r},\tilde{\boldsymbol{r}}}\boldsymbol{B}_{\boldsymbol{r},\tilde{\boldsymbol{r}}}^{*}\right)\nonumber \\
 & \overset{(b)}{=}\sum_{\boldsymbol{r}\in{[m] \choose l}}{n-l \choose k-l}={n-l \choose k-l}{m \choose l},\label{eq:lthOrderPrincipalMinor}
\end{align}
where (a) follows from the fact that the number of $k$-combinations
(out of $[n]$) containing $\tilde{\boldsymbol{r}}$ (an $l$-combination)
is ${n-l \choose k-l}$, and $(b)$ follows from the Cauchy-Binet
formula and the fact that $\boldsymbol{B}\boldsymbol{B}^{*}=\boldsymbol{I}_{m}$,
i.e. 
\[
\sum_{\tilde{\boldsymbol{r}}\in{[n] \choose l}}\det\left(\boldsymbol{B}_{\boldsymbol{r},\tilde{\boldsymbol{r}}}\boldsymbol{B}_{\boldsymbol{r},\tilde{\boldsymbol{r}}}^{*}\right)=\det\left(\boldsymbol{B}_{\boldsymbol{r},[n]}\boldsymbol{B}_{\boldsymbol{r},[n]}^{*}\right)=\det\left(\boldsymbol{I}_{l}\right)=1.
\]
Substituting (\ref{eq:lthOrderPrincipalMinor}) into (\ref{eq:DecompositionPrincipalMinor})
yields 
\begin{align*}
\sum_{\boldsymbol{s}\in{[n] \choose k}}\det\left(\epsilon\boldsymbol{I}_{m}+\boldsymbol{B}_{\boldsymbol{s}}\boldsymbol{B}_{\boldsymbol{s}}^{*}\right) & =\epsilon^{m}{n \choose k}+\sum_{l=1}^{k}\epsilon^{m-l}\sum_{\boldsymbol{s}\in{[n] \choose k}}E_{l}\left(\boldsymbol{B}_{\boldsymbol{s}}\boldsymbol{B}_{\boldsymbol{s}}^{*}\right)\\
 & =\sum_{l=0}^{k}{n-l \choose k-l}{m \choose l}\epsilon^{m-l}.
\end{align*}
By observing that $\boldsymbol{B}_{\boldsymbol{s}}$ is a tall $m\times k$
matrix, one has
\[
\det\left(\epsilon\boldsymbol{I}_{k}+\boldsymbol{B}_{\boldsymbol{s}}^{*}\boldsymbol{B}_{\boldsymbol{s}}\right)=\epsilon^{k-m}\det\left(\epsilon\boldsymbol{I}_{k}+\boldsymbol{B}_{\boldsymbol{s}}\boldsymbol{B}_{\boldsymbol{s}}^{*}\right)
\]
\[
\Rightarrow\quad\sum_{\boldsymbol{s}\in{[n] \choose k}}\det\left(\epsilon\boldsymbol{I}_{k}+\boldsymbol{B}_{\boldsymbol{s}}^{*}\boldsymbol{B}_{\boldsymbol{s}}\right)=\sum_{l=0}^{k}{n-l \choose k-l}{m \choose l}\epsilon^{k-l}.
\]


The above expression allows us to derive a crude bound as
\begin{align*}
{n \choose k}\min_{\boldsymbol{s}\in{[n] \choose k}}\det\left(\epsilon\boldsymbol{I}+\boldsymbol{B}_{\boldsymbol{s}}^{*}\boldsymbol{B}_{\boldsymbol{s}}\right) & \leq\sum_{\boldsymbol{s}\in{[n] \choose k}}\det\left(\epsilon\boldsymbol{I}+\boldsymbol{B}_{\boldsymbol{s}}^{*}\boldsymbol{B}_{\boldsymbol{s}}\right)=\sum_{l=0}^{k}{n-l \choose k-l}{m \choose l}\epsilon^{k-l}\\
 & \leq\sum_{l=0}^{k}{n \choose k-l}{m \choose l}\epsilon^{k-l}=\sum_{l=0}^{k}{n \choose l}{m \choose m-k+l}\epsilon^{l}\\
 & ={m \choose k}\sum_{l=0}^{k}{n \choose l}\frac{{m \choose m-k+l}}{{m \choose k}}\epsilon^{l}.
\end{align*}
Since the term ${m \choose m-k+l}/{m \choose k}$ can be bounded above
as
\[
\frac{{m \choose m-k+l}}{{m \choose k}}=\frac{\frac{m!}{\left(m-k+l\right)!\left(k-l\right)!}}{\frac{m!}{k!\left(m-k\right)!}}=\frac{k!}{\left(k-l\right)!\frac{\left(m-k+l\right)!}{\left(m-k\right)!}}=\frac{{k \choose l}}{{m-k+l \choose l}}\leq{k \choose l},
\]
we obtain
\begin{align}
{n \choose k}\min_{\boldsymbol{s}\in{[n] \choose k}}\det\left(\epsilon\boldsymbol{I}+\boldsymbol{B}_{\boldsymbol{s}}^{*}\boldsymbol{B}_{\boldsymbol{s}}\right) & \leq\sum_{\boldsymbol{s}\in{[n] \choose k}}\det\left(\epsilon\boldsymbol{I}+\boldsymbol{B}_{\boldsymbol{s}}^{*}\boldsymbol{B}_{\boldsymbol{s}}\right)\leq{m \choose k}\sum_{l=0}^{k}{n \choose l}\frac{{m \choose m-k+l}}{{m \choose k}}\epsilon^{l}\nonumber \\
 & \leq{m \choose k}\sum_{l=0}^{k}{n \choose l}{k \choose l}\epsilon^{l}\leq{m \choose k}\sum_{l=0}^{k}{n+k \choose 2l}\left(\sqrt{\epsilon}\right)^{2l}\nonumber \\
 & \leq{m \choose k}\sum_{i=0}^{n+k}{n+k \choose i}\left(\sqrt{\epsilon}\right)^{i}\nonumber \\
 & ={m \choose k}\left(1+\sqrt{\epsilon}\right)^{n+k},\label{eq:SumDetEpsilonBsBsUB}
\end{align}
where the last equality follows from the binomial theorem. 

(2) Since $\boldsymbol{B}_{i}$ has orthonormal rows, applying the
inequality of arithmetic and geometric means yields
\begin{align*}
\sum_{\boldsymbol{s}\in{[n] \choose k}}\left(\prod_{i=1}^{p}\det\left(\epsilon\boldsymbol{I}_{k}+\left(\boldsymbol{B}_{i}\right)_{\boldsymbol{s}}^{*}\left(\boldsymbol{B}_{i}\right)_{\boldsymbol{s}}\right)\right)^{\frac{1}{p}} & \leq\sum_{\boldsymbol{s}\in{[n] \choose k}}\frac{\sum_{i=1}^{p}\det\left(\epsilon\boldsymbol{I}_{k}+\left(\boldsymbol{B}_{i}\right)_{\boldsymbol{s}}^{*}\left(\boldsymbol{B}_{i}\right)_{\boldsymbol{s}}\right)}{p}\\
 & =\frac{1}{p}\sum_{i=1}^{p}\left\{ \sum_{\boldsymbol{s}\in{[n] \choose k}}\det\left(\epsilon\boldsymbol{I}_{k}+\left(\boldsymbol{B}_{i}\right)_{\boldsymbol{s}}^{*}\left(\boldsymbol{B}_{i}\right)_{\boldsymbol{s}}\right)\right\} \\
 & \leq{m \choose k}\left(1+\sqrt{\epsilon}\right)^{n+k},
\end{align*}
where the second inequality follows from (\ref{eq:SumDetEpsilonBsBsUB}).
Since $\boldsymbol{M}_{\alpha}$ has orthonormal rows, applying (\ref{eq:SumDetEpsilonBsBsUB})
yields
\begin{align}
{n \choose k}\min_{\boldsymbol{s}\in{[n] \choose k}}\left(\prod_{i=1}^{p}\det\left(\epsilon\boldsymbol{I}_{k}+\left(\boldsymbol{B}_{i}\right)_{\boldsymbol{s}}^{*}\left(\boldsymbol{B}_{i}\right)_{\boldsymbol{s}}\right)\right)^{\frac{1}{p}} & \leq\sum_{\boldsymbol{s}\in{[n] \choose k}}\left(\prod_{i=1}^{p}\det\left(\epsilon\boldsymbol{I}_{k}+\left(\boldsymbol{B}_{i}\right)_{\boldsymbol{s}}^{*}\left(\boldsymbol{B}_{i}\right)_{\boldsymbol{s}}\right)\right)^{\frac{1}{p}}\nonumber \\
 & \leq{m \choose k}\left(1+\sqrt{\epsilon}\right)^{n+k}.
\end{align}
Therefore,
\begin{align*}
\min_{\boldsymbol{s}\in{[n] \choose k}}\frac{1}{np}\sum_{i=1}^{p}\log\det\left(\epsilon\boldsymbol{I}_{k}+\left(\boldsymbol{B}_{i}\right)_{\boldsymbol{s}}^{*}\left(\boldsymbol{B}_{i}\right)_{\boldsymbol{s}}\right)= & \min_{\boldsymbol{s}\in{[n] \choose k}}\frac{1}{n}\log\left(\prod_{i=1}^{p}\det\left(\epsilon\boldsymbol{I}_{k}+\left(\boldsymbol{B}_{i}\right)_{\boldsymbol{s}}^{*}\left(\boldsymbol{B}_{i}\right)_{\boldsymbol{s}}\right)\right)^{\frac{1}{p}}\\
\leq & \frac{1}{n}\log\left(\frac{{m \choose k}}{{n \choose k}}\left(1+\sqrt{\epsilon}\right)^{n+k}\right)\\
= & \frac{1}{n}\log{m \choose k}-\frac{1}{n}\log{n \choose k}+\frac{n+k}{n}\log\left(1+\sqrt{\epsilon}\right)\\
\leq & \frac{1}{n}\log{m \choose k}-\frac{1}{n}\log{n \choose k}+2\sqrt{\epsilon}.
\end{align*}
When $\left(n,k,m\right)$ are all large numbers, the entropy approximation
(\ref{eq:EntropyApproximation}) allows us to approximate the above
bound as
\begin{align}
\min_{\boldsymbol{s}\in{[n] \choose k}}\frac{1}{np}\sum_{i=1}^{p}\log\det\left(\epsilon\boldsymbol{I}_{k}+\left(\boldsymbol{B}_{i}\right)_{\boldsymbol{s}}^{*}\left(\boldsymbol{B}_{i}\right)_{\boldsymbol{s}}\right) & \leq\frac{m}{n}\mathcal{H}\left(\frac{k}{m}\right)-\mathcal{H}\left(\frac{k}{n}\right)+\frac{\log\left(n+1\right)}{n}+2\sqrt{\epsilon}\nonumber \\
 & =\alpha\mathcal{H}\left(\frac{\beta}{\alpha}\right)-\mathcal{H}\left(\beta\right)+2\sqrt{\epsilon}+\frac{\log\left(n+1\right)}{n}.\label{eq:UBLogDetEpsilonBsBs_Entropy}
\end{align}


\begin{comment}
Denote by ${\bf Q}_{{\bf r},{\bf s}}^{\dagger}$ is the minor of ${\bf Q}^{\dagger}$
coming from the rows at indices from ${\bf r}$ and columns at indices
${\bf s}$. By this notation, our previous notation ${\bf Q}_{{\bf s}}^{\dagger}={\bf Q}_{[m],{\bf s}}^{\dagger}$. 

(1) For each ${\bf s}$, the Cauchy-Binet formula gives
\begin{equation}
\det\left({\bf Q}_{{\bf s}}^{\dagger T}{\bf Q}_{{\bf s}}^{\dagger}\right)=\sum_{{\bf r}\in{\left[m\right] \choose k}}\det\left({\bf Q}_{{\bf r},{\bf s}}^{\dagger}{\bf Q}_{{\bf r},{\bf s}}^{\dagger T}\right),\label{eq:CauchyBinetInequality}
\end{equation}
where ${\bf r}$ ranges over all $k$-element subsets of $[m]:=\left\{ 1,\cdots,m\right\} $.
Here, ${\bf Q}_{{\bf r},{\bf s}}^{\dagger}$ is the $k\times k$ minor
of ${\bf Q}_{{\bf s}}^{\dagger}$ coming from the rows of ${\bf Q}_{{\bf s}}^{\dagger}$
at indices from ${\bf r}$. Therefore,
\begin{align}
\sum_{{\bf s}\in{[n] \choose k}}\det\left({\bf Q}_{{\bf s}}^{\dagger T}{\bf Q}_{{\bf s}}^{\dagger}\right) & =\sum_{{\bf s}\in{[n] \choose k}}\sum_{{\bf r}\in{\left[m\right] \choose k}}\det\left({\bf Q}_{{\bf r},{\bf s}}^{\dagger}{\bf Q}_{{\bf r},{\bf s}}^{\dagger T}\right)=\sum_{{\bf r}\in{\left[m\right] \choose k}}\sum_{{\bf s}\in{[n] \choose k}}\det\left({\bf Q}_{{\bf r},{\bf s}}^{\dagger}{\bf Q}_{{\bf r},{\bf s}}^{\dagger T}\right)\nonumber \\
 & =\sum_{{\bf r}\in{\left[m\right] \choose k}}\det\left({\bf Q}_{{\bf r},[n]}^{\dagger}{\bf Q}_{{\bf r},[n]}^{\dagger T}\right)\label{eq:RearrangeCauchyBinet}\\
 & ={m \choose k}\label{eq:TotalCauchyBinet}
\end{align}
where (\ref{eq:RearrangeCauchyBinet}) and (\ref{eq:TotalCauchyBinet})
follows from the Cauchy-Binet formula 

\[
\sum_{{\bf s}\in{[n] \choose k}}\det\left({\bf Q}_{{\bf r},{\bf s}}^{\dagger}{\bf Q}_{{\bf r},{\bf s}}^{\dagger T}\right)=\det\left({\bf Q}_{{\bf r},[n]}^{\dagger}{\bf Q}_{{\bf r},[n]}^{\dagger T}\right)
\]
and the fact that ${\bf Q}^{\dagger}{\bf Q}^{\dagger T}={\bf I}$. 

By our assumption, for any $\text{SNR}_{\min}\geq\epsilon^{-1}$,we
have 
\[
\sum_{{\bf s}\in{[n] \choose k}}\det\left(\epsilon{\bf I}+{\bf Q}_{{\bf s}}^{\dagger T}{\bf Q}_{{\bf s}}^{\dagger}\right)\leq e^{n/\log n}\sum_{{\bf s}\in{[n] \choose k}}\det\left({\bf Q}_{{\bf s}}^{\dagger T}{\bf Q}_{{\bf s}}^{\dagger}\right)=e^{n/\log n}{m \choose k}
\]
Since there are in total ${n \choose k}$ different states ${\bf s}\in{[n] \choose k}$,
this immediately yields
\[
{n \choose k}\min_{{\bf s}\in{\left[n\right] \choose k}}\det\left(\frac{1}{\small\mathsf{SNR}_{\min}}{\bf I}+{\bf Q}_{{\bf s}}^{\dagger T}{\bf Q}_{{\bf s}}^{\dagger}\right)\leq e^{n/\log n}{m \choose k}
\]
\begin{equation}
\Longrightarrow\quad\min_{{\bf s}\in{\left[n\right] \choose k}}\log\det\left(\frac{1}{\text{SNR}_{\min}}{\bf I}+{\bf Q}_{{\bf s}}^{\dagger T}{\bf Q}_{{\bf s}}^{\dagger}\right)\leq-\log{n \choose k}+\log{m \choose k}+\frac{n}{\log n}\label{eq:UpperboundWithoutApprox}
\end{equation}
Since this upper bound holds for any ${\bf Q}$ of orthonormal rows,
we have established (\ref{eq:MinimaxDetUpperBound}). In particular,
when both $n$ and $k$ are very large, Stirling's formula gives
\begin{align*}
\frac{1}{n}\log{n \choose k} & =\frac{1}{n}\log\left\{ \frac{\left(1+o(1)\right)\sqrt{2\pi n}n^{n}e^{-n}}{\sqrt{2\pi(1-\alpha)n}\left((1-\beta)n\right)^{(1-\beta)n}e^{-(1-\beta)n}\sqrt{2\pi\beta n}(\beta n)^{\beta n}e^{-\beta n}}\right\} \\
 & =\frac{1}{n}\log\left\{ \frac{\left(1+o(1)\right)}{\sqrt{2\pi(1-\beta)\beta n}\left[(1-\beta)^{1-\beta}\beta^{\beta}\right]^{n}}\right\} \\
 & =\left(1-\beta\right)\log\frac{1}{1-\beta}+\beta\log\frac{1}{\beta}+o(1).
\end{align*}
Similarly, $\frac{1}{n}\log{m \choose k}=\alpha\mathcal{H}\left(\frac{\beta}{\alpha}\right)$,
where $\mathcal{H}(x):=x\log\frac{1}{x}+(1-x)\log\frac{1}{1-x}$ denotes
the binary entropy function. With these approximations, the upper
bound (\ref{eq:UpperboundWithoutApprox}) becomes
\begin{align*}
\min_{{\bf s}\in{\left[n\right] \choose k}}\log\det\left(\frac{1}{\small\mathsf{SNR}_{\min}}{\bf I}+{\bf Q}_{{\bf s}}^{\dagger T}{\bf Q}_{{\bf s}}^{\dagger}\right) & \leq\alpha\mathcal{H}\left(\frac{\beta}{\alpha}\right)-\mathcal{H}\left(\beta\right)+o(1)
\end{align*}
\end{comment}


\begin{comment}
a


\section{Proof of Theorem \ref{theorem-Minimax-General-Upper-Bound}\label{eq:MinimaxGapLowerBoundGeneral}}

Since $\boldsymbol{B}\boldsymbol{B}^{*}=\boldsymbol{I}_{mN}$, one
can derive
\[
\sum_{\boldsymbol{s}\in{[n] \choose k}}\text{tr}\left(\boldsymbol{B}_{\boldsymbol{s}}^{*}\boldsymbol{B}_{\boldsymbol{s}}\right)=\sum_{\boldsymbol{s}\in{[n] \choose k}}\text{tr}\left(\boldsymbol{B}_{\boldsymbol{s}}\boldsymbol{B}_{\boldsymbol{s}}^{*}\right)={n-1 \choose k-1}\text{tr}\left(\boldsymbol{B}\boldsymbol{B}^{*}\right)=mN{n-1 \choose k-1}.
\]
For each state $\boldsymbol{s}$, the determinant can be related to
the energy as follows
\begin{align*}
\det\left(\epsilon\boldsymbol{I}_{kN}+\boldsymbol{B}_{\boldsymbol{s}}^{*}\boldsymbol{B}_{\boldsymbol{s}}\right) & =\prod_{i=1}^{kN}\lambda_{i}\left(\epsilon\boldsymbol{I}_{kN}+\boldsymbol{B}_{\boldsymbol{s}}^{*}\boldsymbol{B}_{\boldsymbol{s}}\right)\leq\left(\frac{\sum_{i=1}^{kN}\lambda_{i}\left(\epsilon\boldsymbol{I}_{kN}+\boldsymbol{B}_{\boldsymbol{s}}^{*}\boldsymbol{B}_{\boldsymbol{s}}\right)}{kN}\right)^{kN}\\
 & =\left(\frac{\text{tr}\left(\epsilon\boldsymbol{I}_{kN}+\boldsymbol{B}_{\boldsymbol{s}}^{*}\boldsymbol{B}_{\boldsymbol{s}}\right)}{kN}\right)^{kN},
\end{align*}
where the inequality follows from the inequality of arithmetic and
geometric means. This is equivalent to 
\[
kN\left(\det\left(\epsilon\boldsymbol{I}_{kN}+\boldsymbol{B}_{\boldsymbol{s}}^{*}\boldsymbol{B}_{\boldsymbol{s}}\right)\right)^{\frac{1}{kN}}\leq\text{tr}\left(\epsilon\boldsymbol{I}_{kN}+\boldsymbol{B}_{\boldsymbol{s}}^{*}\boldsymbol{B}_{\boldsymbol{s}}\right)=kN\epsilon+\text{tr}\left(\boldsymbol{B}_{\boldsymbol{s}}^{*}\boldsymbol{B}_{\boldsymbol{s}}\right).
\]
Hence,
\begin{align*}
\sum_{\boldsymbol{s}\in{[n] \choose k}}\left(\det\left(\epsilon\boldsymbol{I}_{kN}+\boldsymbol{B}_{\boldsymbol{s}}^{*}\boldsymbol{B}_{\boldsymbol{s}}\right)\right)^{\frac{1}{kN}} & \leq\frac{1}{kN}\sum_{\boldsymbol{s}\in{[n] \choose k}}\left\{ kN\epsilon+\text{tr}\left(\boldsymbol{B}_{\boldsymbol{s}}^{*}\boldsymbol{B}_{\boldsymbol{s}}\right)\right\} \\
 & ={n \choose k}\epsilon+\frac{m}{k}{n-1 \choose k-1}.
\end{align*}


Besides, applying the inequality of arithmetic and geometric means
yields
\begin{align*}
\frac{1}{npN}\sum_{i=1}^{p}\log\left(\det\left(\epsilon\boldsymbol{I}_{kN}+\left(\boldsymbol{B}_{i}\right)_{\boldsymbol{s}}^{*}\left(\boldsymbol{B}_{i}\right)_{\boldsymbol{s}}\right)\right) & =\beta\log\left\{ \prod_{i=1}^{p}\left[\det\left(\epsilon\boldsymbol{I}_{kN}+\left(\boldsymbol{B}_{i}\right)_{\boldsymbol{s}}^{*}\left(\boldsymbol{B}_{i}\right)_{\boldsymbol{s}}\right)\right]^{\frac{1}{kN}}\right\} ^{\frac{1}{p}}\\
 & \leq\beta\log\left\{ \frac{1}{p}\sum_{i=1}^{p}\left[\det\left(\epsilon\boldsymbol{I}_{kN}+\left(\boldsymbol{B}_{i}\right)_{\boldsymbol{s}}^{*}\left(\boldsymbol{B}_{i}\right)_{\boldsymbol{s}}\right)\right]^{\frac{1}{kN}}\right\} .
\end{align*}
The Jensen's inequality then indicates that
\begin{align*}
\frac{1}{{n \choose k}}\sum_{\boldsymbol{s}\in{[n] \choose k}}\sum_{i=1}^{p}\frac{1}{npN}\log\left(\det\left(\epsilon\boldsymbol{I}_{kN}+\left(\boldsymbol{B}_{i}\right)_{\boldsymbol{s}}^{*}\left(\boldsymbol{B}_{i}\right)_{\boldsymbol{s}}\right)\right) & \leq\frac{1}{{n \choose k}}\sum_{\boldsymbol{s}\in{[n] \choose k}}\beta\log\left\{ \frac{1}{p}\sum_{i=1}^{p}\left[\det\left(\epsilon\boldsymbol{I}_{kN}+\left(\boldsymbol{B}_{i}\right)_{\boldsymbol{s}}^{*}\left(\boldsymbol{B}_{i}\right)_{\boldsymbol{s}}\right)\right]^{\frac{1}{kN}}\right\} \\
 & \leq\beta\log\left\{ \frac{1}{p}\frac{1}{{n \choose k}}\sum_{\boldsymbol{s}\in{[n] \choose k}}\sum_{i=1}^{p}\left[\det\left(\epsilon\boldsymbol{I}_{kN}+\left(\boldsymbol{B}_{i}\right)_{\boldsymbol{s}}^{*}\left(\boldsymbol{B}_{i}\right)_{\boldsymbol{s}}\right)\right]^{\frac{1}{kN}}\right\} \\
 & \overset{(a)}{\leq}\beta\log\left\{ \frac{1}{{n \choose k}}\left\{ {n \choose k}\epsilon+\frac{m}{k}{n-1 \choose k-1}\right\} \right\} \\
 & \leq\beta\log\left(\epsilon+\frac{1}{\alpha}\right)\leq\beta\log\frac{1}{\alpha}+\epsilon.
\end{align*}
This immediately yields
\begin{align*}
\min_{\boldsymbol{s}\in{[n] \choose k}}\sum_{i=1}^{p}\frac{1}{npN}\log\left(\det\left(\epsilon\boldsymbol{I}_{kN}+\left(\boldsymbol{B}_{i}\right)_{\boldsymbol{s}}^{*}\left(\boldsymbol{B}_{i}\right)_{\boldsymbol{s}}\right)\right) & \leq\frac{1}{{n \choose k}}\sum_{\boldsymbol{s}\in{[n] \choose k}}\sum_{i=1}^{p}\frac{1}{npN}\log\left(\det\left(\epsilon\boldsymbol{I}_{kN}+\left(\boldsymbol{B}_{i}\right)_{\boldsymbol{s}}^{*}\left(\boldsymbol{B}_{i}\right)_{\boldsymbol{s}}\right)\right)\\
 & \leq-\beta\log\alpha+\epsilon.
\end{align*}
\end{comment}



\section{Proof of Theorem \ref{theorem-Minimax-RMT-LandauRate}\label{sec:Proof-of-Theorem-Minimax-RMT-LandauRate}}

The quantity of interest can be bounded from below by
\begin{align*}
\log\det\left(\epsilon\boldsymbol{I}_{k}+\left(\boldsymbol{M}\boldsymbol{M}^{\top}\right)^{-1}\boldsymbol{M}_{\boldsymbol{s}}\boldsymbol{M}_{\boldsymbol{s}}^{\top}\right)= & \log\det\left(\epsilon\boldsymbol{M}\boldsymbol{M}^{\top}+\boldsymbol{M}_{\boldsymbol{s}}\boldsymbol{M}_{\boldsymbol{s}}^{\top}\right)-\log\det\left(\boldsymbol{M}\boldsymbol{M}^{\top}\right)\\
\geq & \log\det\left(\frac{\epsilon}{k}\sigma_{\min}\left(\boldsymbol{M}\boldsymbol{M}^{\top}\right)\boldsymbol{I}_{k}+\frac{1}{k}\boldsymbol{M}_{\boldsymbol{s}}\boldsymbol{M}_{\boldsymbol{s}}^{\top}\right)-\log\det\left(\frac{1}{k}\boldsymbol{M}\boldsymbol{M}^{\top}\right),
\end{align*}
which helps separate $\boldsymbol{M}_{\boldsymbol{s}}\boldsymbol{M}_{\boldsymbol{s}}^{\top}$
and $\boldsymbol{M}\boldsymbol{M}^{\top}$ if $\sigma_{\min}\left(\boldsymbol{M}\boldsymbol{M}^{\top}\right)$
is a constant bounded away from zero. The behavior of the least singular
value of a rectangular random matrix with independent sub-Gaussian
entries has been largely studied in the random matrix literature (e.g.
\cite[Theorem 3.1]{LitvakPajorRudelson2005} and \cite[Corollary 5.35]{Vershynin2012}),
which we cite as follows.

\begin{lem}\label{lemma-Concentration-of-Norm-M}Suppose that $m=(1-\delta)n$
for some absolute constant $\delta\in\left(0,1\right)$. Let $\boldsymbol{M}$
be an $m\times n$ real-valued random matrix whose entries are jointly
independent symmetric sub-Gaussian random variables with zero mean
and unit variance. Then there exist universal constants $C,c>0$ such
that
\begin{equation}
\sigma_{\min}\left(\boldsymbol{M}\boldsymbol{M}^{*}\right)>Cm\label{eq:sigma_min_sub-Gaussian}
\end{equation}
with probability at least $1-2\exp\left(-cn\right)$. 

In particular, if the entries of $\boldsymbol{M}$ are i.i.d. standard
Gaussian random variables, then for any constant $0<\xi<\sqrt{n}-\sqrt{m}$,
\[
\sigma_{\min}\left(\boldsymbol{M}\boldsymbol{M}^{*}\right)>\left(\sqrt{n}-\sqrt{m}-\xi\right)^{2}
\]
with probability at least%
\footnote{Note that this follows from \cite[Proposition 5.34 and Theorem 5.32]{Vershynin2012}
by observing that $\sigma_{\min}\left(\boldsymbol{M}\right)$ is a
1-Lipschitz function.%
} $1-\exp\left(-\xi^{2}/2\right)$.\end{lem}

Setting $m=k$, we can derive that with probability exceeding $1-2\exp\left(-cn\right)$,
\begin{align}
\log\det\left(\epsilon\boldsymbol{I}_{k}+\left(\boldsymbol{M}\boldsymbol{M}^{\top}\right)^{-1}\boldsymbol{M}_{\boldsymbol{s}}\boldsymbol{M}_{\boldsymbol{s}}^{\top}\right) & \geq\log\det\left(\epsilon C\boldsymbol{I}_{k}+\frac{1}{k}\boldsymbol{M}_{\boldsymbol{s}}\boldsymbol{M}_{\boldsymbol{s}}^{\top}\right)-\log\det\left(\frac{1}{k}\boldsymbol{M}\boldsymbol{M}^{\top}\right)\label{eq:AlgebraicQuantityLogDet-SubGaussian}
\end{align}
holds for general independent symmetric sub-Gaussian matrices. Also,
with probability at least $1-2e^{-2}$, 
\begin{align}
\log\det\left(\epsilon\boldsymbol{I}_{k}+\left(\boldsymbol{M}\boldsymbol{M}^{\top}\right)^{-1}\boldsymbol{M}_{\boldsymbol{s}}\boldsymbol{M}_{\boldsymbol{s}}^{\top}\right) & \geq\log\det\left(\epsilon\frac{\left(1-\sqrt{\beta}-\frac{2}{\sqrt{n}}\right)^{2}}{\beta}\boldsymbol{I}_{k}+\frac{1}{k}\boldsymbol{M}_{\boldsymbol{s}}\boldsymbol{M}_{\boldsymbol{s}}^{\top}\right)-\log\det\left(\frac{1}{k}\boldsymbol{M}\boldsymbol{M}^{\top}\right)\label{eq:AlgebraicQuantityLogDet-Gaussian}
\end{align}
holds for i.i.d. standard Gaussian matrices and any constant $\xi\in(0,1)$.

The next step is to quantify the term $\log\det\left(\epsilon\boldsymbol{I}+\frac{1}{k}\boldsymbol{M}_{\boldsymbol{s}}\boldsymbol{M}_{\boldsymbol{s}}^{\top}\right)$
for some small $0<\epsilon<1$. This has been characterized in Lemma
\ref{lem:MinLogdetMs_Landau}, indicating that 
\begin{equation}
\forall\boldsymbol{s}\in{[n] \choose k},\quad\log\det\left(\epsilon\boldsymbol{I}+\frac{1}{k}\boldsymbol{M}_{\boldsymbol{s}}\boldsymbol{M}_{\boldsymbol{s}}^{\top}\right)\geq-1-\mathcal{O}\left(\min\left\{ \frac{1}{\sqrt{n\epsilon}}\right\} ,\frac{\log\frac{1}{\epsilon}+\log n}{n^{1/4}}\right)\label{eq:LB_logdet_Landau}
\end{equation}
with exponentially high probability. In addition, since $\boldsymbol{M}$
satisfies the assumptions of Lemma \ref{lemma-Rectangular-Nonhermitian-LogDet}
with $\beta=\alpha$, simple manipulation gives
\begin{align}
\frac{1}{n}\log\det\left(\frac{1}{k}\boldsymbol{M}\boldsymbol{M}^{\top}\right) & =\frac{1}{n}\log\det\left(\frac{1}{n}\boldsymbol{M}\boldsymbol{M}^{\top}\right)+\frac{k}{n}\log\frac{n}{k}\nonumber \\
 & \leq(1-\beta)\log\frac{1}{1-\beta}-\beta+\beta\log\frac{1}{\beta}+\frac{1}{\sqrt{n}}\label{eq:logDetMMFinal}
\end{align}
with probability exceeding $1-C_{7}\exp\left(-c_{7}n\right)$. The
above results (\ref{eq:sigma_min_sub-Gaussian}), (\ref{eq:AlgebraicQuantityLogDet-SubGaussian}),
(\ref{eq:LB_logdet_Landau}) and (\ref{eq:logDetMMFinal}) taken collectively
yield the following: under the condition of Theorem \ref{theorem-Minimax-RMT-LandauRate},
one has 
\begin{align}
\forall\boldsymbol{s}:\quad\frac{1}{n}\log\det\left(\epsilon\boldsymbol{I}_{k}+\left(\boldsymbol{M}\boldsymbol{M}^{\top}\right)^{-1}\boldsymbol{M}_{\boldsymbol{s}}\boldsymbol{M}_{\boldsymbol{s}}^{\top}\right) & \geq-\mathcal{H}\left(\beta\right)+\frac{\log k}{2k}-\mathcal{O}\left(\frac{1}{\sqrt{n\epsilon}}\right)
\end{align}
with probability exceeding $1-C\exp\left(-cn\right)$ for some absolute
constants $c,C>0$. Combining this lower bound with the upper bound
developed in Theorem \ref{theorem-Evaluate-Det-IplusBB} (with $\alpha=\beta$)
concludes the proof.

\begin{comment}
When specialized to i.i.d. Gaussian ensembles, putting (\ref{eq:AlgebraicQuantityLogDet-Gaussian}),
(\ref{eq:Gaussian_Landau_Ms}), and (\ref{eq:logdetAA_UB_Gaussian})
together yields that for any $n>\max\left\{ \frac{4}{\left(1-\sqrt{\beta}\right)^{2}},7\right\} $:
\begin{align*}
 & \forall\boldsymbol{s}:\text{ }\frac{1}{n}\log\det\left(\epsilon\boldsymbol{I}_{k}+\left(\boldsymbol{M}\boldsymbol{M}^{\top}\right)^{-1}\boldsymbol{M}_{\boldsymbol{s}}\boldsymbol{M}_{\boldsymbol{s}}^{\top}\right)\\
 & \quad\quad\geq-\mathcal{H}\left(\beta\right)-\frac{\sqrt{2}\left(1-\sqrt{\beta}-\frac{2}{\sqrt{n}}\right)^{-1}}{\sqrt{k\epsilon}}-\frac{2\beta\left(1-\sqrt{\beta}-\frac{2}{\sqrt{n}}\right)^{-2}}{k\epsilon}+\frac{\log k}{2k}-\frac{2\log n}{n}-\frac{5\sqrt{\beta}}{\left(1-\sqrt{\beta}-\frac{2}{n}\right)}\frac{1}{n}\\
 & \quad\quad\geq-\mathcal{H}\left(\beta\right)-\frac{\sqrt{2}\left(1-\sqrt{\beta}-\frac{2}{\sqrt{n}}\right)^{-1}}{\sqrt{k\epsilon}}-\left(\frac{2}{\left(1-\sqrt{\beta}-\frac{2}{\sqrt{n}}\right)^{2}\epsilon}+\frac{5\sqrt{\beta}}{\left(1-\sqrt{\beta}-\frac{2}{n}\right)}\right)\frac{1}{n}-\frac{1.5\log n}{n}
\end{align*}
with probability at least $1-2n^{-2}-2e^{-2}>1/2$.
\end{comment}



\section{Proof of Theorem \ref{theorem-Minimax-RMT-SuperLandau} \label{sec:Proof-of-theorem-Minimax-RMT-SuperLandau}}

Our goal is to evaluate $\frac{1}{n}\log\det\left(\epsilon\boldsymbol{I}_{k}+\boldsymbol{M}_{\boldsymbol{s}}^{\top}\left(\boldsymbol{M}\boldsymbol{M}^{\top}\right)^{-1}\boldsymbol{M}_{\boldsymbol{s}}\right)$
for some small $\epsilon>0$. We first define two Wishart matrices
$\Xi_{\backslash\boldsymbol{s}}:=\frac{1}{n}\boldsymbol{M}\boldsymbol{M}^{\top}-\frac{1}{n}\boldsymbol{M}_{\boldsymbol{s}}\boldsymbol{M}_{\boldsymbol{s}}^{\top}$
and $ $$\Xi_{\boldsymbol{s}}:=\frac{1}{n}\boldsymbol{M}_{\boldsymbol{s}}\boldsymbol{M}_{\boldsymbol{s}}^{\top}$.
Apparently, $\Xi_{\boldsymbol{s}}\sim\mathcal{W}_{m}\left(k,\frac{1}{n}\boldsymbol{I}_{m}\right)$
and $\Xi_{\backslash\boldsymbol{s}}\sim\mathcal{W}_{m}\left(n-k,\frac{1}{n}\boldsymbol{I}_{m}\right)$.
When $1-\alpha>\beta$, i.e. $n-k>m$, $ $the Wishart matrix $\Xi_{\backslash\boldsymbol{s}}$
is invertible with probability 1. 

One difficulty in evaluating $\det\left(\epsilon\boldsymbol{I}_{k}+\boldsymbol{M}_{\boldsymbol{s}}^{\top}\left(\boldsymbol{M}\boldsymbol{M}^{\top}\right)^{-1}\boldsymbol{M}_{\boldsymbol{s}}\right)$
is that $\boldsymbol{M}_{\boldsymbol{s}}$ and $\boldsymbol{M}\boldsymbol{M}^{\top}$
are not independent. This motivates us to decouple them first as follows
\begin{align*}
\det\left(\epsilon\boldsymbol{I}_{k}+\boldsymbol{M}_{\boldsymbol{s}}^{\top}\left(\boldsymbol{M}\boldsymbol{M}^{\top}\right)^{-1}\boldsymbol{M}_{\boldsymbol{s}}\right) & =\epsilon^{k-m}\det\left(\epsilon\boldsymbol{I}_{m}+\left(\frac{1}{n}\boldsymbol{M}\boldsymbol{M}^{\top}\right)^{-1}\frac{1}{n}\boldsymbol{M}_{\boldsymbol{s}}\boldsymbol{M}_{\boldsymbol{s}}^{\top}\right)\\
 & =\epsilon^{k-m}\det\left(\epsilon\frac{1}{n}\boldsymbol{M}\boldsymbol{M}^{\top}+\frac{1}{n}\boldsymbol{M}_{\boldsymbol{s}}\boldsymbol{M}_{\boldsymbol{s}}^{\top}\right)\det\left(\frac{1}{n}\boldsymbol{M}\boldsymbol{M}^{\top}\right)^{-1}\\
 & =\epsilon^{k-m}\det\left(\epsilon\Xi_{\backslash\boldsymbol{s}}+\left(1+\epsilon\right)\Xi_{\boldsymbol{s}}\right)\det\left(\frac{1}{n}\boldsymbol{M}\boldsymbol{M}^{\top}\right)^{-1}\\
 & =\epsilon^{k-m}\det\left(\epsilon\boldsymbol{I}_{m}+\left(1+\epsilon\right)\Xi_{\boldsymbol{s}}\Xi_{\backslash\boldsymbol{s}}^{-1}\right)\det\left(\Xi_{\backslash\boldsymbol{s}}\right)\det\left(\frac{1}{n}\boldsymbol{M}\boldsymbol{M}^{\top}\right)^{-1}\\
 & =\det\left(\epsilon\boldsymbol{I}_{k}+\left(1+\epsilon\right)\frac{1}{n}\boldsymbol{M}_{\boldsymbol{s}}^{\top}\Xi_{\backslash\boldsymbol{s}}^{-1}\boldsymbol{M}_{\boldsymbol{s}}\right)\det\left(\Xi_{\backslash\boldsymbol{s}}\right)\det\left(\frac{1}{n}\boldsymbol{M}\boldsymbol{M}^{\top}\right)^{-1}
\end{align*}
or, equivalently,
\begin{align}
 & \frac{1}{n}\log\det\left(\epsilon\boldsymbol{I}_{k}+\boldsymbol{M}_{\boldsymbol{s}}^{\top}\left(\boldsymbol{M}\boldsymbol{M}^{\top}\right)^{-1}\boldsymbol{M}_{\boldsymbol{s}}\right)\nonumber \\
= & \frac{1}{n}\log\det\left(\epsilon\boldsymbol{I}_{k}+\left(1+\epsilon\right)\frac{1}{n}\boldsymbol{M}_{\boldsymbol{s}}^{\top}\Xi_{\backslash\boldsymbol{s}}^{-1}\boldsymbol{M}_{\boldsymbol{s}}\right)+\frac{1}{n}\log\det\left(\Xi_{\backslash\boldsymbol{s}}\right)-\frac{1}{n}\log\det\left(\frac{1}{n}\boldsymbol{M}\boldsymbol{M}^{\top}\right).\label{eq:logdet-identity-separate-New}
\end{align}
The point of developing this identity (\ref{eq:logdet-identity-separate-New})
is to decouple the left-hand side of (\ref{eq:logdet-identity-separate-New})
through 3 matrices $\boldsymbol{M}_{\boldsymbol{s}}^{\top}\Xi_{\backslash\boldsymbol{s}}^{-1}\boldsymbol{M}_{\boldsymbol{s}}$,
$\Xi_{\backslash\boldsymbol{s}}$ and $\boldsymbol{M}\boldsymbol{M}^{\top}$.
In particular, since $\boldsymbol{M}_{\boldsymbol{s}}$ and $\Xi_{\backslash\boldsymbol{s}}$
are jointly independent, we can examine the concentration of measure
for $\boldsymbol{M}_{\boldsymbol{s}}$ and $\Xi_{\backslash\boldsymbol{s}}$
separately when evaluating $\boldsymbol{M}_{\boldsymbol{s}}^{\top}\Xi_{\backslash\boldsymbol{s}}^{-1}\boldsymbol{M}_{\boldsymbol{s}}$. 

The second and third terms of (\ref{eq:logdet-identity-separate-New})
can be evaluated through Lemma \ref{lem:GaussianLogDet_LB}. Specifically,
Lemma \ref{lem:GaussianLogDet_LB} indicates that
\begin{equation}
\frac{1}{n}\log\det\left(\frac{1}{n}\boldsymbol{M}\boldsymbol{M}^{\top}\right)\leq-\left(1-\alpha\right)\log\left(1-\alpha\right)-\alpha+\mathcal{O}\left(\frac{1}{\sqrt{n}}\right)\label{eq:LogDetMMTSuperLandau}
\end{equation}
with probability at least $1-C_{6}\exp\left(-2n\right)$ for some
constant $C_{6}>0$, and that for all $\boldsymbol{s}\in{[n] \choose k}$,
\begin{align}
\frac{1}{n}\log\det\left(\Xi_{\backslash\boldsymbol{s}}\right) & =\frac{n-k}{n}\frac{1}{n-k}\log\det\left(\frac{n}{n-k}\Xi_{\backslash\boldsymbol{s}}\right)+\frac{1}{n}\log\det\left(\frac{n-k}{n}\boldsymbol{I}\right)\nonumber \\
 & \geq\left(1-\beta\right)\left\{ -\left(1-\frac{\alpha}{1-\beta}\right)\log\left(1-\frac{\alpha}{1-\beta}\right)-\frac{\alpha}{1-\beta}\right\} +\alpha\log\left(1-\beta\right)+\mathcal{O}\left(\frac{\log n}{n^{1/3}}\right)\nonumber \\
 & \geq-\left(1-\alpha-\beta\right)\log\left(1-\frac{\alpha}{1-\beta}\right)-\alpha+\alpha\log\left(1-\beta\right)+\mathcal{O}\left(\frac{\log n}{n^{1/3}}\right).\label{eq:LogDetMMTNotSSuperLandau}
\end{align}
hold simultaneously with probability exceeding $1-C_{9}\exp\left(-2n\right)$. 

Our main task then amounts to quantifying $\log\det\left(\epsilon\boldsymbol{I}_{k}+\boldsymbol{M}_{\boldsymbol{s}}^{\top}\Xi_{\backslash\boldsymbol{s}}^{-1}\boldsymbol{M}_{\boldsymbol{s}}\right)$,
which can be lower bounded via Lemma \ref{lemma-Expected-Log-Determinant-LambdaStatistics}.
This together with (\ref{eq:LogDetMMTSuperLandau}), (\ref{eq:LogDetMMTNotSSuperLandau})
and (\ref{eq:logdet-identity-separate-New}) yields that 
\begin{align*}
 & \frac{1}{n}\log\det\left(\epsilon\boldsymbol{I}_{k}+\boldsymbol{M}_{\boldsymbol{s}}^{\top}\left(\boldsymbol{M}\boldsymbol{M}^{\top}\right)^{-1}\boldsymbol{M}_{\boldsymbol{s}}\right)\\
 & \quad\geq-\left(\alpha-\beta\right)\log\left(\alpha-\beta\right)+\alpha\log\alpha+\left(1-\alpha-\beta\right)\log\left(1-\frac{\beta}{1-\alpha}\right)-\beta\log\left(1-\alpha\right)\\
 & \quad\quad-\left(1-\alpha-\beta\right)\log\left(1-\frac{\alpha}{1-\beta}\right)-\alpha+\alpha\log\left(1-\beta\right)+\left(1-\alpha\right)\log\left(1-\alpha\right)+\alpha-\mathcal{O}\left(\frac{\log n}{n^{1/3}}\right)\\
 & \quad=\alpha\mathcal{H}\left(\frac{\beta}{\alpha}\right)-\mathcal{H}\left(\beta\right)-\mathcal{O}\left(\frac{\log n}{n^{1/3}}\right)
\end{align*}
with probability exceeding $1-C_{9}\exp\left(-2n\right)$ for some
constants $C_{9}>0$. 

Since there are at most ${n \choose k}<e^{n}$ different states $\boldsymbol{s}$,
applying the union bound over all states completes the proof.

\begin{comment}

\section{Proof of Theorem \ref{theorem:BaysianUpperBound}\label{sec:Proof-of-Theorem-BayesianUpperBound}}

Suppose that Column $i$ has $\ell_{2}$ norm $\left\Vert {\bf Q}_{*i}^{\dagger}\right\Vert _{2}=a_{i}$.
Then for each state sequence ${\bf s}$, we define a correlation matrix
$\tilde{{\bf Q}}_{{\bf s}}=\left(\frac{\left\langle {\bf Q}_{*{\bf s}_{i}}^{\dagger},{\bf Q}_{*{\bf s}_{j}}^{\dagger}\right\rangle }{\left\Vert {\bf Q}_{*{\bf s}_{i}}^{\dagger}\right\Vert _{2}\left\Vert {\bf Q}_{*{\bf s}_{j}}^{\dagger}\right\Vert _{2}}\right)_{1\leq i,j\leq k}$,
which apparently satisfies
\[
\det\left[{\bf Q}_{{\bf s}}^{\dagger}{\bf Q}_{{\bf s}}^{\dagger T}\right]=\det\left[\tilde{{\bf Q}}_{{\bf s}}\tilde{{\bf Q}}_{{\bf s}}^{T}\right]\prod_{i\in{\bf s}}a_{{\bf s}}^{2},
\]
where the Hadamard inequality implies that 
\[
\det\left[\tilde{{\bf Q}}_{{\bf s}}\tilde{{\bf Q}}_{{\bf s}}^{T}\right]\leq\prod_{i\in{\bf s}}\left\Vert \tilde{{\bf Q}}_{*i}\right\Vert _{2}^{2}=1\quad\Rightarrow\quad\log\det\left[\tilde{{\bf Q}}_{{\bf s}}\tilde{{\bf Q}}_{{\bf s}}^{T}\right]\leq0.
\]
Hence, 
\begin{align}
 & \text{ }\frac{1}{n}\log\left(\prod_{{\bf s}\in{[n] \choose k}}\det\left[{\bf Q}_{{\bf s}}^{\dagger}{\bf Q}_{{\bf s}}^{\dagger T}\right]^{\mu({\bf s})}\right)\nonumber \\
= & \text{ }\frac{1}{n}\sum_{{\bf s}\in{[n] \choose k}}\left\{ \mu({\bf s})\sum_{i\in{\bf s}}\log a_{i}^{2}+\mu({\bf s})\log\det\left[\tilde{{\bf Q}}_{{\bf s}}\tilde{{\bf Q}}_{{\bf s}}^{T}\right]\right\} \nonumber \\
\leq & \text{ }\frac{1}{n}\sum_{{\bf s}\in{[n] \choose k}}\left\{ \mu({\bf s})\sum_{i\in{\bf s}}\log a_{i}^{2}+\mu_{\min}\log\det\left[\tilde{{\bf Q}}_{{\bf s}}\tilde{{\bf Q}}_{{\bf s}}^{T}\right]\right\} \nonumber \\
= & \sum_{i=1}^{n}\frac{2}{n}\left(\sum_{{\bf s}:i\in{\bf s}}\mu({\bf s})\right)\log a_{i}+\frac{\mu_{\min}}{n}\sum_{{\bf s}\in{[n] \choose k}}\log\det\left[\tilde{{\bf Q}}_{{\bf s}}\tilde{{\bf Q}}_{{\bf s}}^{T}\right].\label{eq:BayesianLogDetSplit}
\end{align}


We will first bound the first term of (\ref{eq:BayesianLogDetSplit}).
Since ${\bf Q}^{\dagger}$ has orthonormal rows, we can observe that

\[
\sum_{i=1}^{n}a_{i}^{2}=\sum_{i=1}^{m}\sum_{j=1}^{n}\left({\bf Q}_{ij}^{\dagger}\right)^{2}=m.
\]
 We note that $\max_{\left\{ a_{j}\right\} :\sum_{j=1}^{n}a_{j}^{2}\leq m}\sum_{i=1}^{n}2\left(\sum_{{\bf s}:i\in{\bf s}}\mu({\bf s})\right)\log a_{i}$
is a convex optimization problem with quadratic constraints. Define
the Lagrangian 
\[
\mathcal{L}\left(\left\{ a_{i}\right\} ,\lambda\right)=-\sum_{i=1}^{n}2\left(\sum_{{\bf s}:i\in{\bf s}}\mu({\bf s})\right)\log a_{i}+\lambda\left(\sum_{i=1}^{n}a_{i}^{2}-m\right).
\]
 Taking derivatives with respect to both $\lambda$ and $c_{i}$ yields
\[
\begin{cases}
-\frac{2}{a_{i}}\left(\sum_{{\bf s}:i\in{\bf s}}\mu({\bf s})\right)+2\lambda a_{i}=0,\\
\sum_{i=1}^{n}a_{i}^{2}-m=0.
\end{cases}
\]
 Hence
\[
a_{i}^{2}=\frac{1}{\lambda}\sum_{{\bf s}:i\in{\bf s}}\mu({\bf s}),
\]
 
\[
\Rightarrow\quad\lambda=\frac{1}{m}\sum_{i=1}^{n}\sum_{{\bf s}:i\in{\bf s}}\mu({\bf s})=\frac{1}{m}\sum_{{\bf s}\in{[n] \choose k}}\sum_{i:i\in{\bf s}}\mu({\bf s})=\frac{k}{m}\sum_{{\bf s}\in{[n] \choose k}}\mu({\bf s})=\frac{\beta}{\alpha}.
\]
 Therefore, the optimal $\left\{ \hat{a}_{j}\right\} $ satisfies
\[
\hat{a}_{j}^{2}=\frac{\alpha}{\beta}\sum_{{\bf s}:i\in{\bf s}}\mu({\bf s}),
\]
Thus, the first term of (\ref{eq:BayesianLogDetSplit}) can be bounded
above by
\begin{align*}
\frac{1}{n}\sum_{i=1}^{n}2\left(\sum_{{\bf s}:i\in{\bf s}}\mu({\bf s})\right)\log a_{i} & \leq\frac{1}{n}\sum_{i=1}^{n}\left(\sum_{{\bf s}:i\in{\bf s}}\mu({\bf s})\right)\left\{ \log\left(\sum_{{\bf s}:i\in{\bf s}}\mu(s)\right)+\log\frac{\alpha}{\beta}\right\} \\
 & =\frac{k}{n}\sum_{i=1}^{n}\frac{\sum_{{\bf s}:i\in{\bf s}}\mu({\bf s})}{k}\log\frac{\sum_{{\bf s}:i\in{\bf s}}\mu({\bf s})}{k}+\frac{1}{n}\sum_{i=1}^{n}\left(\sum_{s:i\in[s]}\mu(s)\right)\log\frac{\alpha}{\beta}k\\
 & =-\beta\mathcal{H}\left(\left\{ \left.\frac{\sum_{{\bf s}:i\in{\bf s}}\mu({\bf s})}{k}\right|1\leq i\leq n\right\} \right)+\beta\log m,
\end{align*}
where $\mathcal{H}(\cdot)$ denotes the entropy function.

The next step is to bound the second term of (\ref{eq:BayesianLogDetSplit}),
i.e. $\sum_{{\bf s}\in{[n] \choose k}}\log\det\left[\tilde{{\bf Q}}_{{\bf s}}\tilde{{\bf Q}}_{{\bf s}}^{T}\right]$.
If there is no further condition, then we can use the simple bound
that $\log\det\left[\tilde{{\bf Q}}_{{\bf s}}\tilde{{\bf Q}}_{{\bf s}}^{T}\right]\leq0$
by the Hadamard inequality, which completes the proof of (\ref{eq:MinimaxDetUpperBoundLargeN-BayesGeneral}). 

If we know additionally that

Define another matrix $\hat{{\bf Q}}$ such that
\[
\tilde{{\bf Q}}\tilde{{\bf Q}}^{T}=\left[\begin{array}{cccc}
b_{1}\\
 & b_{2}\\
 &  & \ddots\\
 &  &  & b_{m}
\end{array}\right]\hat{{\bf Q}}\hat{{\bf Q}}^{T}\left[\begin{array}{cccc}
b_{1}\\
 & b_{2}\\
 &  & \ddots\\
 &  &  & b_{m}
\end{array}\right],
\]
where $b_{i}:=\left\Vert \tilde{{\bf Q}}_{i*}\right\Vert _{2}$ is
the norm of the $i$th row of $\tilde{{\bf Q}}$. With this notation,
each row of $\tilde{{\bf Q}}$ has norm 1, and hence $\det\left(\hat{{\bf Q}}_{{\bf r},[n]}\hat{{\bf Q}}_{{\bf r},[n]}^{T}\right)\leq1$
by the Hadamard inequality. By using Cauchy-Binet formula, we can
now obtain
\begin{align*}
\sum_{{\bf s}\in{[n] \choose k}}\det\left(\tilde{{\bf Q}}_{{\bf s}}^{T}\tilde{{\bf Q}}_{{\bf s}}\right) & =\sum_{{\bf s}\in{[n] \choose k}}\sum_{{\bf r}\in{[m] \choose k}}\det\left(\tilde{{\bf Q}}_{{\bf r},{\bf s}}\tilde{{\bf Q}}_{{\bf r},{\bf s}}^{T}\right)=\sum_{{\bf r}\in{[m] \choose k}}\left[\sum_{{\bf s}\in{[n] \choose k}}\det\left(\tilde{{\bf Q}}_{{\bf r},{\bf s}}\tilde{{\bf Q}}_{{\bf r},{\bf s}}^{T}\right)\right]\\
 & =\sum_{{\bf r}\in{[m] \choose k}}\det\left(\tilde{{\bf Q}}_{{\bf r},[n]}\tilde{{\bf Q}}_{{\bf r},[n]}^{T}\right)=\sum_{{\bf r}\in{[m] \choose k}}\det\left(\hat{{\bf Q}}_{{\bf r},[n]}\hat{{\bf Q}}_{{\bf r},[n]}^{T}\right)\prod_{i:i\in{\bf r}}b_{i}^{2}\\
 & \leq\sum_{{\bf r}\in{[m] \choose k}}\prod_{i:i\in{\bf r}}b_{i}^{2}\\
 & \overset{(a)}{\leq}\text{ }{m \choose k}\left(\frac{\sum_{i=1}^{m}b_{i}^{2}}{m}\right)^{k}\\
 & \overset{(b)}{=}{m \choose k}\left(\frac{n}{m}\right)^{k},
\end{align*}
where the inequality (a) follows from the Maclaurin's inequality of
symmetric means. The equality (b) follows from the fact that $\sum_{i=1}^{n}b_{i}^{2}$
is the sum of squares of all entries of $\tilde{{\bf Q}}$ and that
each column of $\tilde{{\bf Q}}$ has $\ell_{2}$ norm 1, and hence
$\sum_{i=1}^{n}b_{i}^{2}=n$. Using the arithmetic-geometric mean
inequality, we can conclude
\begin{align*}
\frac{\mu_{\min}}{n}\sum_{{\bf s}\in{[n] \choose k}}\log\det\left(\epsilon{\bf I}+\tilde{{\bf Q}}_{{\bf s}}^{T}\tilde{{\bf Q}}_{{\bf s}}\right) & \leq\frac{\mu_{\min}}{n}\sum_{{\bf s}\in{[n] \choose k}}\log\det\left(\tilde{{\bf Q}}_{{\bf s}}^{T}\tilde{{\bf Q}}_{{\bf s}}\right)+o(1)\\
 & \leq\frac{{n \choose k}\mu_{\min}}{n}\log\frac{\sum_{{\bf s}\in{[n] \choose k}}\det\left(\tilde{{\bf Q}}_{{\bf s}}^{T}\tilde{{\bf Q}}_{{\bf s}}\right)}{{n \choose k}}+o(1)\leq\frac{{n \choose k}\mu_{\min}}{n}\log\frac{{m \choose k}\left(\frac{n}{m}\right)^{k}}{{n \choose k}}+o(1)\\
 & ={n \choose k}\mu_{\min}\left(\alpha\mathcal{H}\left(\frac{\beta}{\alpha}\right)-\beta\log\alpha-\mathcal{H}\left(\beta\right)\right)+o(1),
\end{align*}
which completes the proof of (\ref{eq:MinimaxDetUpperBoundLargeN-BayesTighter}).


\section{Proof of Theorem \ref{theorem-Bayes-Sampler-Achievability}\label{sec:Proof-of-Theorem--Bayes-Sampler-Achievability}}

Similar to the minimax case, we have 
\[
G_{{\bf s}}^{{\bf M}}\leq-\frac{W}{2n}\log\det\left(\frac{\lambda_{\min}\left(\frac{1}{k}{\bf M}{\bf M}^{T}\right)}{\text{SNR}_{\max}}{\bf I}+\frac{1}{k}{\bf M}_{{\bf s}}^{T}{\bf M}_{{\bf s}}\right)+\frac{W}{2n}\log\det\left(\frac{1}{k}{\bf M}{\bf M}^{T}\right)+O\left(\frac{1}{\text{SNR}_{\min}}\right).
\]
Hence, we need to evaluate $\log\det\left({\bf M}{\bf M}^{T}\right)$
and $\log\det\left(\epsilon{\bf I}+{\bf M}_{{\bf s}}^{T}{\bf M}_{{\bf s}}\right)$
for small $\epsilon>0$. 

Let us introduce another normalized random matrix ${\bf M}$ such
that $\tilde{{\bf M}}_{ij}:=\frac{1}{\hat{a}_{j}}{\bf M}_{ij}$. Apparently,
$\tilde{{\bf M}}$ is a random ensemble whose entries are jointly
independent with zero mean and unit variance. We can then write
\begin{align*}
\frac{1}{n}\log\det\left(\epsilon{\bf I}+{\bf M}_{{\bf s}}^{T}{\bf M}_{{\bf s}}\right) & \geq\frac{1}{n}\log\det\left(\frac{\epsilon}{\hat{a}_{\max}}{\bf I}+\frac{1}{k}\tilde{{\bf M}}_{{\bf s}}^{T}\tilde{{\bf M}}_{{\bf s}}\right)+\frac{1}{n}\log\prod_{i:i\in{\bf s}}\hat{a}_{i}^{2}.
\end{align*}
Therefore, we have 
\begin{align*}
\frac{1}{n}\sum_{{\bf s}\in{[n] \choose k}}\mu({\bf s})\log\det\left(\epsilon{\bf I}+\frac{1}{k}{\bf M}_{{\bf s}}^{T}{\bf M}_{{\bf s}}\right) & \geq\frac{1}{n}\sum_{{\bf s}\in{[n] \choose k}}\mu({\bf s})\log\det\left(\frac{\epsilon}{\hat{a}_{\max}}{\bf I}+\frac{1}{k}\tilde{{\bf M}}_{{\bf s}}^{T}\tilde{{\bf M}}_{{\bf s}}\right)+\frac{1}{n}\sum_{{\bf s}\in{[n] \choose k}}\mu({\bf s})\log\prod_{i:i\in{\bf s}}\hat{a}_{i}^{2}\\
 & =\frac{1}{n}\sum_{{\bf s}\in{[n] \choose k}}\mu({\bf s})\log\det\left(\frac{\epsilon}{\hat{a}_{\max}}{\bf I}+\frac{1}{k}\tilde{{\bf M}}_{{\bf s}}^{T}\tilde{{\bf M}}_{{\bf s}}\right)\\
 & \quad\quad-\beta\frac{\mathcal{H}\left(\left\{ \left.\frac{\sum_{{\bf s}:i\in{\bf s}}\mu({\bf s})}{k}\right|1\leq i\leq n\right\} \right)}{\log n}+\beta\log\beta,
\end{align*}
where the last equality follows using the same procedure as in the
proof of Theorem \ref{theorem:BaysianUpperBound}. 

Since $\hat{a}_{\max}=\Theta\left(1\right)$, we have shown in (\ref{eq:ConcentrationLogDetPlusEpsilon})
that for any small constant $\epsilon>0$, 
\begin{equation}
\frac{1}{k}\log\text{det}\left(\epsilon{\bf I}+\frac{1}{k}{\bf M}_{{\bf s}}^{T}{\bf M}_{{\bf s}}^{T}\right)\in\left[-1-O\left(\frac{\log k}{k}\right),-1+O\left(\frac{\log k}{k}\right)+2\sqrt{\epsilon}\right]
\end{equation}
holds with probability exceeding $1-4\exp\left(-\tilde{c}\epsilon^{2}k\log k\right)$
for some constant $\tilde{c}>0$. Hence, applying a union bound yields
\[
\frac{1}{n}\sum_{{\bf s}\in{[n] \choose k}}\mu({\bf s})\log\det\left(\frac{\epsilon}{\hat{a}_{\max}}{\bf I}+\frac{1}{k}\tilde{{\bf M}}_{{\bf s}}^{T}\tilde{{\bf M}}_{{\bf s}}\right)\in\left[-\beta-O\left(\frac{\log k}{k}\right),-\beta+O\left(\frac{\log k}{k}\right)+2\beta\sqrt{\frac{\epsilon}{\hat{a}_{\max}}}\right]
\]
with probability exceeding $1-3\exp\left(-\tilde{c}\epsilon^{2}k\log k\right)$. 

Now that we have shown the concentration of $\log\det\left(\frac{\epsilon}{k\hat{a}_{\max}}{\bf I}+\frac{1}{k}\tilde{{\bf M}}_{{\bf s}}^{T}\tilde{{\bf M}}_{{\bf s}}\right)$,
our remaining task is to evaluate $\log\det\left(\frac{1}{k}{\bf M}{\bf M}^{T}\right)$.
We know from \cite[Corollary 1.8]{GuionnetZeitouni2000} that
\[
\mathbb{P}\left\{ \left|\frac{1}{k}\log\det\left(\frac{1}{k}{\bf M}{\bf M}^{T}\right)-\frac{1}{k}\mathbb{E}\log\det\left(\frac{1}{k}{\bf M}{\bf M}^{T}\right)\right|>\delta\right\} \leq4\exp\left(-\frac{c_{9}\epsilon^{2}\delta^{2}}{\hat{a}_{\max}^{2}}k^{3}\right)
\]
for some constant $c_{9}>0$. Besides, the Cauchy-Binet formula yields
\begin{align*}
\mathbb{E}\det\left(\frac{1}{k}{\bf M}{\bf M}^{T}\right) & =\mathbb{E}\sum_{{\bf s}\in{[n] \choose k}}\det\left(\frac{1}{k}{\bf M}_{{\bf s}}{\bf M}_{{\bf s}}^{T}\right)\\
 & =\sum_{{\bf s}\in{[n] \choose k}}\mathbb{E}\det\left(\frac{1}{k}\tilde{{\bf M}}_{{\bf s}}\tilde{{\bf M}}_{{\bf s}}^{T}\right)\prod_{j:j\in{\bf s}}\hat{a}_{j}^{2}\\
 & =\frac{k!}{k^{k}}{n \choose k}\frac{\sum_{{\bf s}\in{[n] \choose k}}\prod_{j:j\in{\bf s}}\hat{a}_{j}^{2}}{{n \choose k}}.
\end{align*}
\[
\Rightarrow\quad\frac{1}{n}\mathbb{E}\log\det\left(\frac{1}{k}{\bf M}{\bf M}^{T}\right)=\frac{1}{n}\log\mathbb{E}\det\left(\frac{1}{k}{\bf M}{\bf M}^{T}\right)+o(1)=-\beta+\mathcal{H}\left(\beta\right)+\beta\log\left(\sqrt[k]{S_{k}}\right),
\]
where $S_{k}:=\frac{\sum_{{\bf s}\in{[n] \choose k}}\prod_{j:j\in{\bf s}}\hat{a}_{j}^{2}}{{n \choose k}}$
is the symmetric mean of degree $k$ in the $n$ variables $\left\{ \hat{a}_{i}^{2}\mid1\leq i\leq n\right\} $.
Therefore, 
\[
\frac{1}{n}\log\det\left(\frac{1}{k}{\bf M}{\bf M}^{T}\right)=-\beta+\mathcal{H}\left(\beta\right)+\beta\log\left(\sqrt[k]{S_{k}}\right)+o(1)
\]
 with probability exceeding $1-4\exp\left(-c_{9}\epsilon^{2}k^{2}/\hat{a}_{\max}^{2}\right)$. 

Combining all the above results yields
\begin{align*}
\frac{1}{n}\log\det\left(\frac{\lambda_{\min}\left(\frac{1}{k}{\bf M}{\bf M}^{T}\right)}{\text{SNR}_{\max}}{\bf I}+\frac{1}{k}{\bf M}_{{\bf s}}^{T}{\bf M}_{{\bf s}}\right)-\frac{1}{n}\log\det\left(\frac{1}{k}{\bf M}{\bf M}^{T}\right) & =-\beta\frac{\mathcal{H}\left(\left\{ \left.\frac{\sum_{{\bf s}:i\in{\bf s}}\mu({\bf s})}{k}\right|1\leq i\leq n\right\} \right)}{\log n}+\beta\log\beta\\
 & \quad-\mathcal{H}\left(\beta\right)-\beta\log\left(\sqrt[k]{S_{k}}\right)+O(\sqrt{\epsilon})+O\left(\frac{\log k}{k}\right)
\end{align*}
with probability exceeding $1-8\exp\left(-c_{9}\epsilon^{2}k\log k/\hat{a}_{\max}^{2}\right)$. 
\end{comment}


\begin{comment}
a

\begin{lem}\label{Lemma-Concentration-Upper-Bound}(1) Suppose that
$\zeta_{i}\text{ }(1\leq i\leq n)$ are independent random variables
with mean zero and variance 1, and that $\left\{ c_{i}\mid1\leq i\leq n\right\} $
is a set of constants independent of $\zeta_{i}$ and $n$, and is
bounded as $c_{\text{min}}\leq c_{i}\leq c_{\max}$. 

If $\zeta_{i}$ are bounded above  in magnitude by some constant $D$.
Then for any $\epsilon>0$ one has 
\[
\mathbb{P}\left(\frac{\sqrt{\sum_{i=1}^{n}c_{i}^{2}\zeta_{i}^{2}}}{\sqrt{\sum_{i=1}^{n}c_{i}^{2}}}\in\left[1-2\epsilon,1+2\epsilon\right]\right)\geq1-C\exp\left(-\frac{c\epsilon^{2}c_{\max}^{2}c_{\min}^{2}}{D^{2}}n\right)
\]
for some absolute constants $C,c>0$.

If $\zeta_{i}$ are Gaussian random variables, then 
\[
\mathbb{P}\left(\frac{\sqrt{\sum_{i=1}^{n}c_{i}^{2}\zeta_{i}^{2}}}{\sqrt{\sum_{i=1}^{n}c_{i}^{2}}}\in\left[1-2\epsilon,1+2\epsilon\right]\right)\geq1-2\exp\left(-\frac{\epsilon^{2}c_{\max}^{2}c_{\min}^{2}n}{2}\right).
\]


(2) Suppose that $\zeta_{ij}$ $(1\leq i\leq k,1\leq j\leq n)$ are
i.i.d. with zero mean and unit variance. Also, $ $$\zeta_{ij}$ are
either bounded  in magnitude by some constant $D$, or Gaussian distributed.
Let ${\bf W}=\left(\xi_{ij}\right)_{1\leq i\leq k,1\leq j\leq n}$
be a random matrix such that $\xi_{ij}=c_{j}\zeta_{ij}$. Then, 
\[
\frac{1}{n}\log\det\left({\bf W}{\bf W}^{*}\right)\leq\beta\log(1+2\epsilon)^{2}+\beta\log\left(\sum_{j=1}^{n}c_{j}^{2}\right)
\]
with probability exceeding $1-C\exp\left(-\tilde{c}\epsilon^{2}c_{\min}^{4}n\right)$
for some constants $\tilde{c},C>0$.

\end{lem}

\begin{proof}See Appendix \ref{sec:Proof-of-Lemma-Concentration-Upper-Bound}.\end{proof}
\end{comment}



\section{Proof of Lemma \ref{lemma-ConcentrationLogDetPlusConstant}\label{sec:Proof-of-Lemma-ConcentrationLogDetPlusConstant}}

Set $f(x)=\log\left(\epsilon+x\right)$. Observe that the Lipschitz
constant of $g\left(x\right):=f\left(x^{2}\right)=\log(\epsilon+x^{2})$
is upper bounded by
\[
\forall x\geq0,\quad\left|g'\left(x\right)\right|=\left|\frac{2x}{\epsilon+x^{2}}\right|=\left|\frac{2}{\frac{\epsilon}{x}+x}\right|\leq\frac{1}{\sqrt{\epsilon}}.
\]


When $\boldsymbol{A}_{ij}$'s are bounded in magnitude by $D$, define
the following\emph{ concave} function
\[
g_{\epsilon}\left(x\right):=\begin{cases}
\log\left(\epsilon+x^{2}\right), & \text{if }x\geq\sqrt{\epsilon},\\
\frac{1}{\sqrt{\epsilon}}\left(x-\epsilon\right)+\log\left(2\epsilon\right),\quad & \text{if }0\leq x<\sqrt{\epsilon},
\end{cases}
\]
whose Lipschitz constant is bounded by $\frac{1}{\sqrt{\epsilon}}$.
This function obeys the following interlacing bound 
\begin{equation}
\log\left(\frac{2}{e}\epsilon+x\right)\leq g_{\epsilon}\left(\sqrt{x}\right)\leq\log\left(\epsilon+x\right),\quad\forall x\geq0.\label{eq:sandwich}
\end{equation}
Define the function
\begin{equation}
f_{\epsilon}\left(\boldsymbol{A}\right):=\frac{1}{k}\sum_{i=1}^{k}g_{\epsilon}\left(\sqrt{\lambda_{i}\left(\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)}\right),\label{eq:DefnFepsilon}
\end{equation}
then it follows from (\ref{eq:sandwich}) that 
\begin{equation}
\frac{1}{k}\log\text{det}\left(\frac{2\epsilon}{e}\boldsymbol{I}+\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\leq f_{\epsilon}\left(\boldsymbol{A}\right)\leq\frac{1}{k}\log\text{det}\left(\epsilon\boldsymbol{I}+\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right).\label{eq:f_epsilon_defn}
\end{equation}
One can then apply \cite[Corollary 1.8(a)]{GuionnetZeitouni2000}
to derive that for any $\delta>\frac{8D\sqrt{\pi}}{\sqrt{\epsilon}}$,
\begin{align*}
f_{\epsilon}\left(\boldsymbol{A}\right) & \geq\mathbb{E}\left[f_{\epsilon}\left(\boldsymbol{A}\right)\right]-\frac{1}{k}\delta\\
 & \geq\mathbb{E}\left[\frac{1}{k}\log\text{det}\left(\frac{2\epsilon}{e}\boldsymbol{I}+\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]-\frac{1}{k}\delta
\end{align*}
with probability at least $1-4\exp\left(-\frac{\epsilon}{4D^{2}}\left(\delta-\frac{4D\sqrt{\pi}}{\sqrt{\epsilon}}\right)^{2}\right)$.
This together with (\ref{eq:f_epsilon_defn}) yields that for any
$\delta>\frac{8D\sqrt{\pi}}{\sqrt{\epsilon}}$,
\begin{align*}
\log\text{det}\left(\epsilon\boldsymbol{I}+\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\geq kf_{\epsilon}\left(\boldsymbol{A}\right) & \geq\mathbb{E}\left[\log\text{det}\left(\frac{2\epsilon}{e}\boldsymbol{I}+\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]-\delta
\end{align*}
with probability at least $1-4\exp\left(-\frac{\epsilon}{16D^{2}}\delta^{2}\right)$.
Similar argument indicates that for any $\delta>\frac{8D\sqrt{\pi}}{\sqrt{\frac{e\epsilon}{2}}}$,
\begin{align*}
\log\text{det}\left(\epsilon\boldsymbol{I}+\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\leq kf_{\frac{e}{2}\epsilon}\left(\boldsymbol{A}\right) & \leq\mathbb{E}\left[\log\text{det}\left(\frac{e\epsilon}{2}\boldsymbol{I}+\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]-\delta
\end{align*}
with probability at least $1-4\exp\left(-\frac{e\epsilon}{32D^{2}}\delta^{2}\right)$.

If $\boldsymbol{A}_{ij}$ satisfies the LSI with uniformly bounded
constant $c_{\text{LS}}$, then applying \cite[Corollary 1.8(b)]{GuionnetZeitouni2000}
leads to
\[
\left|\log\text{det}\left(\epsilon\boldsymbol{I}+\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)-\mathbb{E}\left[\log\text{det}\left(\epsilon\boldsymbol{I}+\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]\right|>\delta
\]
with probability at most $2\exp\left(-\frac{\epsilon\delta^{2}}{2c_{\text{LS}}}\right)$,
as claimed. 


\section{Proof of Lemma \ref{lemma-Expected-Log-Determinant-Plus-Constant}\label{sec:Proof-of-lemma-Expected-Log-Determinant-Plus-Constant}}

(1) By Jensen's inequality, 
\begin{align*}
\frac{1}{k}\mathbb{E}\left[\log\det\left(\epsilon\boldsymbol{I}+\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right] & \leq\frac{1}{k}\log\mathbb{E}\left[\det\left(\epsilon\boldsymbol{I}+\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]\\
 & \leq-1+\frac{1.5\log\left(ek\right)}{k}+2\sqrt{\epsilon}\log\frac{1}{\epsilon},
\end{align*}
where the last inequality follows from \cite[Lemma 3]{chen2014backing}.

\begin{comment}
The identity (\ref{eq:DetIplusA_via_PrincipalMinor}) gives
\[
\mathbb{E}\left[\det\left(\epsilon k\boldsymbol{I}+\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]=\left(\epsilon k\right)^{k}+\sum_{i=1}^{k}\left(\epsilon k\right)^{k-i}\mathbb{E}\left[E_{i}\left(\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right],
\]
where $E_{i}\left(\boldsymbol{A}\boldsymbol{A}^{\top}\right)$ denotes
the sum of determinants of all $i$-by-$i$ principal minors of $\boldsymbol{A}\boldsymbol{A}^{\top}$.
There is a well known fact that for any $\boldsymbol{G}=\left(\zeta_{ij}\right)_{1\leq i,j\leq l}$
with independent entries having mean zero and variance one, one has
$\mathbb{E}\left[\det\left(\boldsymbol{G}\boldsymbol{G}^{\top}\right)\right]=l!$.
In fact, if we denote by $\prod_{l}$ the permutation group of $l$
elements, then the Leibniz formula for the determinant gives
\[
\det\left(\boldsymbol{G}\right)=\sum_{\sigma\in\prod_{l}}\text{sgn}(\sigma)\prod_{i=1}^{k}\zeta_{i,\sigma(i)}.
\]
Since $\zeta_{ij}$ are jointly independent, we have
\begin{equation}
\mathbb{E}\left[\text{det}\left(\boldsymbol{G}\boldsymbol{G}^{\top}\right)\right]=\mathbb{E}\left[\left(\det\left(\boldsymbol{G}\right)\right)^{2}\right]=\sum_{\sigma\in\prod_{l}}\mathbb{E}\left[\prod_{i=1}^{l}\left|\zeta_{i,\sigma(i)}\right|^{2}\right]=\sum_{\sigma\in\prod_{l}}\prod_{i=1}^{l}\mathbb{E}\left[\left|\zeta_{i,\sigma(i)}\right|^{2}\right]=l!.\label{eq:ExpectedDetMSquareNew}
\end{equation}


Denote by $\left(\boldsymbol{A}\boldsymbol{A}^{\top}\right)_{\boldsymbol{s}}$
the principal minor of $\boldsymbol{A}\boldsymbol{A}^{\top}$ coming
from the index set $\boldsymbol{s}$, then $E_{i}\left(\boldsymbol{A}\boldsymbol{A}^{\top}\right)$
can be computed as
\begin{align*}
\mathbb{E}\left[E_{i}\left(\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right] & =\sum_{\boldsymbol{s}\in{[k] \choose i}}\mathbb{E}\left[\det\left(\left(\boldsymbol{A}\boldsymbol{A}^{\top}\right)_{\boldsymbol{s}}\right)\right]=\sum_{\boldsymbol{s}\in{[k] \choose i}}\mathbb{E}\left[\det\left(\boldsymbol{A}_{\boldsymbol{s},[n]}\boldsymbol{A}_{\boldsymbol{s},[n]}^{\top}\right)\right]\\
 & \overset{(a)}{=}\sum_{\boldsymbol{s}\in{[k] \choose i}}\sum_{\boldsymbol{r}\in{[k] \choose i}}\mathbb{E}\left[\det\left(\boldsymbol{A}_{\boldsymbol{s},\boldsymbol{r}}\boldsymbol{A}_{\boldsymbol{s},\boldsymbol{r}}^{\top}\right)\right]\\
 & \overset{(b)}{=}{k \choose i}{k \choose i}i!,
\end{align*}
where (a) follows from the Cauchy-Binet formula and (b) follows from
(\ref{eq:ExpectedDetMSquareNew}). Combining the above results with
simple algebraic manipulation yields
\begin{align}
\mathbb{E}\left[\det\left(\epsilon k\boldsymbol{I}+\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right] & =\left(\epsilon k\right)^{k}+\sum_{i=1}^{k}\left(\epsilon k\right)^{k-i}{k \choose i}\frac{k!}{\left(k-i\right)!}=k!\sum_{i=0}^{k}{k \choose k-i}\frac{\left(\epsilon k\right)^{k-i}}{\left(k-i\right)!}\nonumber \\
 & =k!\sum_{i=0}^{k}{k \choose i}\frac{\epsilon^{i}k^{i}}{i!}.\label{eq:ExpansionSumExpectedDetPlusConst}
\end{align}


In order to bound this sum, one way is to first identify the largest
term. Using the short hand notation $f_{\epsilon}(j):={k \choose j}\frac{k^{j}\epsilon^{j}}{j!}$
and $r_{\epsilon}(j):=\frac{f_{\epsilon}(j+1)}{f_{\epsilon}(j)}$,
we can obtain
\[
r_{\epsilon}(j)=\frac{f_{\epsilon}(j+1)}{f_{\epsilon}(j)}=\frac{\frac{k!}{\left(k-j-1\right)!\left(j+1\right)!}\frac{k^{j+1}\epsilon^{j+1}}{\left(j+1\right)!}}{\frac{k!}{\left(k-j\right)!j!}k^{j}\epsilon^{j}j!}=\frac{\left(k-j\right)k\epsilon}{\left(j+1\right)^{2}}.
\]
Apparently, $r_{\epsilon}(j)$ is a decreasing function of $j$. By
setting $r_{\epsilon}(j^{*})=1$, we can derive
\[
\frac{\left(k-j^{*}\right)k\epsilon}{\left(j^{*}+1\right)^{2}}=1\quad\Rightarrow\quad j^{*}=\frac{-k\epsilon-2+\sqrt{4k^{2}\epsilon-4+\left(k\epsilon+2\right)^{2}}}{2}
\]
which can be bounded by 
\[
k\left(\sqrt{\epsilon}-\epsilon\right)<\frac{-k\epsilon-2+\sqrt{4k^{2}\epsilon}}{2}<j^{*}<\frac{-k\epsilon-2+\sqrt{4k^{2}\epsilon}+\sqrt{\left(k\epsilon+2\right)^{2}}}{2}=k\sqrt{\epsilon}
\]
for large $k$ and small $\epsilon$. Suppose that $j^{*}=\tilde{\epsilon}k$
for some $\tilde{\epsilon}\in[\sqrt{\epsilon}-\epsilon,\sqrt{\epsilon}]$,
then by Stirling's formula
\begin{align*}
\frac{1}{k}\log f(j) & \leq\frac{1}{k}\log f\left(j^{*}\right)=\frac{1}{k}\log\left\{ {k \choose j^{*}}\frac{k^{j^{*}}\epsilon^{j^{*}}}{j^{*}!}\right\} \\
 & =-\tilde{\epsilon}\log\tilde{\epsilon}-\left(1-\tilde{\epsilon}\right)\log\left(1-\tilde{\epsilon}\right)+\tilde{\epsilon}\log k+\tilde{\epsilon}\log\epsilon-\tilde{\epsilon}\log\left(\tilde{\epsilon}k\right)+\tilde{\epsilon}+\mathcal{O}\left(\frac{\log k}{k}\right)\\
 & \leq2\sqrt{\epsilon}+\mathcal{O}\left(\frac{\log k}{k}\right)=2\sqrt{\epsilon}+\mathcal{O}\left(\frac{\log k}{k}\right)
\end{align*}
where the last equality follows from L' Hospital's rule. This together
with (\ref{eq:ExpansionSumExpectedDetPlusConst}) yields
\begin{align*}
\frac{1}{k}\log\mathbb{E}\left[\det\left(\epsilon k\boldsymbol{I}+\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right] & \leq\frac{1}{k}\log\left[k!\left(kf(j^{*})\right)\right]\\
 & =\frac{\log\left(k!\right)+\log k}{k}+2\sqrt{\epsilon}+\mathcal{O}\left(\frac{\log k}{k}\right)\\
 & =\log k-1+2\sqrt{\epsilon}+\mathcal{O}\left(\frac{\log k}{k}\right)
\end{align*}
Together with Jensen's inequality, one has
\begin{align*}
\frac{1}{k}\mathbb{E}\left[\log\det\left(\epsilon\boldsymbol{I}+\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right] & =-\log k+\frac{1}{k}\mathbb{E}\left[\log\det\left(\epsilon k\boldsymbol{I}+\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]\\
 & \leq-\log k+\frac{1}{k}\log\mathbb{E}\left[\det\left(\epsilon\boldsymbol{I}+\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]\\
 & =-1+2\sqrt{\epsilon}+\mathcal{O}\left(\frac{\log k}{k}\right).
\end{align*}
\end{comment}


(2) The lower bound follows from the concentration inequality. Define
\[
Y:=k\left(f_{\epsilon}\left(\boldsymbol{A}\right)-\mathbb{E}\left[f_{\epsilon}\left(\boldsymbol{A}\right)\right]\right),
\]
where $f_{\epsilon}\left(\boldsymbol{A}\right)$ is defined in (\ref{eq:DefnFepsilon}).
Similar to the proof of Lemma \ref{lemma-ConcentrationLogDetPlusConstant},
applying \cite[Corollary 1.8]{GuionnetZeitouni2000} indicates that

\[
\mathbb{P}\left(\left|Y\right|>\delta\right)\leq4\exp\left(-\tilde{c}\epsilon\delta^{2}\right)
\]
for some constant $\tilde{c}$. If we denote by $f_{Y}(\cdot)$ the
probability density function of $Y$, then
\begin{align*}
\mathbb{E}\left(e^{Y}\right) & \leq\mathbb{E}\left(e^{\left|Y\right|}\right)=\int_{0}^{\infty}e^{y}f_{\left|Y\right|}\left(y\right)\mathrm{d}y=-e^{y}\mathbb{P}\left(\left|Y\right|>y\right)\mid_{0}^{\infty}+\int_{0}^{\infty}e^{y}\mathbb{P}\left(\left|Y\right|>y\right)\mathrm{d}y\\
 & \leq1+\int_{0}^{\infty}4\exp\left(y-\tilde{c}\epsilon y^{2}\right)\mathrm{d}y\\
 & <1+4\sqrt{\frac{\pi}{\tilde{c}\epsilon}}\exp\left(\frac{1}{4\tilde{c}\epsilon}\right).
\end{align*}
Taking the logarithms of both sides and plugging in the expression
of $Y$ yields
\begin{align}
\log\left(\mathbb{E}\left[e^{kf_{\epsilon}\left(\boldsymbol{A}\right)}\right]\right) & \leq k\mathbb{E}\left[f_{\epsilon}\left(\boldsymbol{A}\right)\right]+\log\left[1+4\sqrt{\frac{\pi}{\tilde{c}\epsilon}}\exp\left(\frac{1}{4\tilde{c}\epsilon}\right)\right]\nonumber \\
 & \leq\mathbb{E}\left[\log\det\left(\epsilon\boldsymbol{I}+\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]+\log\left[1+4\sqrt{\frac{\pi}{\tilde{c}\epsilon}}\exp\left(\frac{1}{4\tilde{c}\epsilon}\right)\right].\label{eq:LowerBoundLogDetE_ELogDet}
\end{align}
Also, the inequality $kf_{\epsilon}\left(\boldsymbol{A}\right)\geq\log\det\left(\frac{2\epsilon}{e}\boldsymbol{I}+\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)$
allows us to derive
\begin{align}
\mathbb{E}\left[\log\det\left(\epsilon\boldsymbol{I}+\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right] & \geq\log\left(\mathbb{E}\left[e^{kf_{\epsilon}\left(\boldsymbol{A}\right)}\right]\right)-\log\left[1+4\sqrt{\frac{\pi}{\tilde{c}\epsilon}}\exp\left(\frac{1}{4\tilde{c}\epsilon}\right)\right]\nonumber \\
 & \geq\log\mathbb{E}\left[\det\left(\frac{2\epsilon}{e}\boldsymbol{I}+\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]-\log\left[1+4\sqrt{\frac{\pi}{\tilde{c}\epsilon}}\exp\left(\frac{1}{4\tilde{c}\epsilon}\right)\right].\label{eq:LBlogdet}
\end{align}


Furthermore, if we denote by $\prod_{k}$ the permutation group of
$k$ elements, then the Leibniz formula for the determinant gives
\[
\det\left(\boldsymbol{A}\right)=\sum_{\sigma\in\prod_{k}}\text{sgn}(\sigma)\prod_{i=1}^{k}\boldsymbol{A}_{i,\sigma(i)}.
\]
Taking advantage of the joint independence hypothesis yields
\[
\mathbb{E}\left[\text{det}\left(\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]=\mathbb{E}\left[\left(\det\left(\boldsymbol{A}\right)\right)^{2}\right]=\sum_{\sigma\in\prod_{k}}\mathbb{E}\left[\prod_{i=1}^{k}\left|\boldsymbol{A}_{i,\sigma(i)}\right|^{2}\right]=\sum_{\sigma\in\prod_{k}}\prod_{i=1}^{k}\mathbb{E}\left[\left|\boldsymbol{A}_{i,\sigma(i)}\right|^{2}\right]=k!
\]
This taken collectively with (\ref{eq:LBlogdet}) yields that
\begin{align*}
\frac{1}{k}\mathbb{E}\left[\log\det\left(\epsilon\boldsymbol{I}+\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{T}\right)\right] & \geq\frac{1}{k}\log\mathbb{E}\left[\det\left(\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]-\frac{1}{k}\log\left[1+4\sqrt{\frac{\pi}{\tilde{c}\epsilon}}\exp\left(\frac{1}{4\tilde{c}\epsilon}\right)\right]\\
 & =\frac{1}{k}\log\frac{k!}{k^{k}}-\frac{1}{k}\log\left[1+4\sqrt{\frac{\pi}{\tilde{c}\epsilon}}\exp\left(\frac{1}{4\tilde{c}\epsilon}\right)\right]\\
 & =-1+\frac{\log k}{2k}-\frac{1}{k}\log\left[1+4\sqrt{\frac{\pi}{\tilde{c}\epsilon}}\exp\left(\frac{1}{4\tilde{c}\epsilon}\right)\right],
\end{align*}
where the last inequality makes uses of the well-known Stirling-type
inequality
\[
k!\geq k^{k+\frac{1}{2}}e^{-k},\quad\Rightarrow\quad\frac{1}{k}\log\frac{k!}{k^{k}}\geq\frac{\log k}{2k}-1.
\]
This indicates that $\frac{1}{k}\mathbb{E}\left[\log\det\left(\epsilon\boldsymbol{I}+\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]\geq-1+\frac{\log k}{2k}-\mathcal{O}\left(\frac{1}{k\epsilon}\right).$

In particular, for i.i.d. Gaussian ensemble $\boldsymbol{A}_{ij}\sim\mathcal{N}\left(0,1\right)$,
one has $\tilde{c}=\frac{1}{2}$, and hence for any $\epsilon\leq0.8$:
\begin{align*}
\frac{1}{k}\mathbb{E}\left[\log\det\left(\epsilon\boldsymbol{I}+\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right] & \geq-1+\frac{\log k}{2k}-\frac{1}{k}\log\left[1+4\sqrt{\frac{\pi}{2\epsilon}}\exp\left(\frac{1}{2\epsilon}\right)\right]\\
 & \geq-1+\frac{\log k}{2k}-\frac{1}{k}\log\left[\exp\left(\frac{2}{\epsilon}\right)\right]\\
 & =-1+\frac{\log k}{2k}-\frac{2}{k\epsilon},
\end{align*}
where the second inequality uses the inequality that $1+4\sqrt{\frac{\pi}{2\epsilon}}\exp\left(\frac{1}{2\epsilon}\right)\leq\exp\left(\frac{2}{\epsilon}\right)$
for any $0<\epsilon\leq0.8$.

\begin{comment}
In particular, when $\boldsymbol{A}$ is drawn from i.i.d. Gaussian
ensemble, the above argument yields
\begin{align*}
\frac{1}{k}\mathbb{E}\log\det\left(\epsilon\boldsymbol{I}+\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{T}\right) & \geq-1+\frac{\log\epsilon}{k}-\frac{\sqrt{2\pi}}{k}-\frac{1}{8\epsilon^{2}k^{2}}.
\end{align*}
\end{comment}



\section{Proof of Lemma \ref{lemma:logdet_I_AA_LB}\label{sec:Proof-of-Lemma-logdet_I_AA_LB}}

We first attempt to estimate the number of eigenvalues of $\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}$
not exceeding $\epsilon$. To this end, we consider the following
function
\[
f_{2,\xi}\left(x\right):=\begin{cases}
-\sqrt{\frac{x}{\xi}}+2,\quad & \text{if }x\leq4\xi,\\
0 & \text{else},
\end{cases}
\]
as well as the convex function
\[
g_{2,\xi}\left(x\right):=f_{2,\xi}\left(x^{2}\right)=\begin{cases}
-\frac{x}{\sqrt{\xi}}+2,\quad & \text{if }x\leq2\sqrt{\xi},\\
0 & \text{else}.
\end{cases}
\]
Apparently, 
\begin{equation}
{\bf 1}_{\left(0,\xi\right)}\left(x\right)\leq f_{2,\xi}(x)\leq2\cdot{\bf 1}_{\left(0,4\xi\right)}(x),\quad\forall x\label{eq:f_2_epsilon_indicator}
\end{equation}
and the Lischiptz constant of $g_{2,\xi}\left(\cdot\right)$ is bounded
above by $\frac{1}{\sqrt{\xi}}$. 

By our assumptions, $\boldsymbol{A}_{ij}$'s are symmetric and sub-Gaussian.
If we generate the random matrix $\tilde{\boldsymbol{A}}$ such that
\[
\tilde{\boldsymbol{A}}_{ij}:=\boldsymbol{A}_{ij}{\bf 1}_{\left\{ \left|\boldsymbol{A}_{ij}\right|\leq k^{1/4}\right\} },
\]
then one can easily verify that 
\[
\mathbb{P}\left(\tilde{\boldsymbol{A}}=\boldsymbol{A}\right)=1-o\left(1\right),\quad\mathbb{E}\left[\tilde{\boldsymbol{A}}_{ij}\right]=0,\quad\mathbb{E}\left[\left|\tilde{\boldsymbol{A}}_{ij}\right|^{2}\right]=1-o\left(1\right).
\]
If we denote by
\[
N_{I}\left(\boldsymbol{A}\right):=\text{card}\left\{ i\mid\lambda_{i}\left(\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\in I\right\} 
\]
for any interval $I$, then it follows from the theory on local spectral
statistics \cite[Theorem 4.1]{Wang2012} that
\[
N_{\left(0,\xi\right)}\left(\tilde{\boldsymbol{A}}\right)\leq k{\displaystyle \int}_{0}^{\xi}\rho\left(x\right)\mathrm{d}x+\frac{1}{4}k\xi
\]
with probability at least $1-c_{5}/k^{3}$ for some constant $c_{5}>0$,
where $\rho(x)$ represents the Marchenko\textendash{}Pastur law 
\[
\rho(x):=\frac{1}{2\pi x}\sqrt{\left(4-x\right)x}\cdot{\bf 1}_{[0,1]}\left(x\right)\leq\frac{1}{\pi\sqrt{x}}.
\]
This immediately implies that
\begin{align}
\frac{1}{k}\mathbb{E}\left[N_{\left(0,\xi\right)}\left(\boldsymbol{A}\right)\right] & \leq\frac{c_{5}}{k^{3}}+\left(1-\frac{c_{5}}{k^{3}}\right)\left({\displaystyle \int}_{0}^{\xi}\rho\left(x\right)\mathrm{d}x+\frac{1}{4}\epsilon\right)\nonumber \\
 & \leq\frac{2}{\pi}\sqrt{\xi}+\frac{1}{4}\xi+\frac{c_{5}}{k^{3}}.\label{eq:UB_Nepsilon}
\end{align}
The concentration inequality \cite[Corollary 1.8]{GuionnetZeitouni2000}
ensures that for any $\delta=\Omega\left(\frac{1}{k\sqrt{\xi}}\right)$,
\begin{align*}
\frac{1}{k}N_{\left(0,\xi\right)}\left(\boldsymbol{A}\right) & \leq\text{ }\frac{1}{k}\sum_{i=1}^{k}f_{2,\xi}\left(\lambda_{i}\left(\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right)\leq\text{ }\frac{1}{k}\sum_{i=1}^{k}\mathbb{E}\left[f_{2,\xi}\left(\lambda_{i}\left(\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right)\right]+\delta
\end{align*}
with probability exceeding $1-4\exp\left(-\tilde{c}\xi\delta^{2}k^{2}\right)$
for some constant $\tilde{c}>0$. The inequality (\ref{eq:f_2_epsilon_indicator})
suggests that
\[
\frac{1}{k}\sum_{i=1}^{k}\mathbb{E}\left[f_{2,\xi}\left(\lambda_{i}\left(\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right)\right]\leq\frac{2}{k}\cdot\mathbb{E}\left[N_{\left(0,4\xi\right)}\left(\boldsymbol{A}\right)\right]\leq\frac{8}{\pi}\sqrt{\xi}+2\xi+\frac{2c_{5}}{k^{3}},
\]
and hence for any $\tau\geq\frac{1}{\sqrt{k}}$,
\begin{align*}
\frac{1}{k}N_{\left(0,\xi\right)}\left(\boldsymbol{A}\right) & \leq\text{ }\frac{8}{\pi}\sqrt{\xi}+2\xi+\frac{2c_{5}}{k^{3}}+\sqrt{\frac{\tau}{\tilde{c}k\xi}}
\end{align*}
with probability at least $1-4\exp\left(-\tau k\right)$. By setting
$\xi=\sqrt{\frac{\tau}{k}}$, one can derive that with probability
exceeding $1-4\exp\left(-\tau k\right)$,
\begin{align*}
\frac{1}{k}N_{\left(0,\xi\right)}\left(\boldsymbol{A}\right) & \leq\text{ }\frac{c_{11}\tau^{1/4}}{k^{1/4}},\quad\forall\frac{1}{k}\leq\tau<k
\end{align*}
holds for some universal constant $c_{11}>0$.

The above estimates on the eigenvalue concentration allow us to derive
\begin{align}
\frac{1}{k}\log\det\left(\epsilon\boldsymbol{I}+\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right) & \geq\frac{1}{k}\log\mathrm{det}^{\epsilon+\xi}\left(\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)-\frac{N_{\left(0,\xi\right)}\left(\boldsymbol{A}\right)}{k}\log\left(\frac{\epsilon+\xi}{\epsilon}\right)\nonumber \\
 & \geq\frac{1}{k}\log\mathrm{det}^{\xi}\left(\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)-\frac{c_{12}\tau^{1/4}}{k^{1/4}}\left(\log\frac{1}{\epsilon}+\log k\right)\label{eq:LogDet_temp_LB}
\end{align}
with exponential high probability, where $c_{12}>0$ is another constant.
Here, the function $\det^{\xi}\left(\cdot\right)$ is defined in (\ref{eq:Defn_Det_epsilon}).
Now it suffices to obtain a lower estimate on $\log\mathrm{det}^{\xi}\left(\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right)$.
Putting the bounds (\ref{eq:UB_Z}) and (\ref{eq:LowerBoundELogDetEpsilon})
together indicates that there exists a constant $c_{13}>0$ such that
\begin{align*}
\frac{1}{k}\log\mathrm{det}^{\xi}\left(\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right) & \geq-1+\frac{\log\left(2\pi k\right)}{2k}-\frac{c_{13}}{k\xi}-\frac{c_{13}\sqrt{\tau}}{\sqrt{k\xi}}\\
 & \geq-1+\frac{\log\left(2\pi k\right)}{2k}-\frac{c_{13}}{k^{1/2}\tau^{1/2}}-\frac{c_{13}\tau^{1/4}}{k^{1/4}}
\end{align*}
with probability exceeding $1-4\exp\left(-\tau k\right)$. This combined
with (\ref{eq:LogDet_temp_LB}) yields that for any $\frac{1}{\sqrt{k}}\leq\tau<k$,
\begin{align*}
\frac{1}{k}\log\det\left(\epsilon\boldsymbol{I}+\frac{1}{k}\boldsymbol{A}\boldsymbol{A}^{\top}\right) & \geq-1+\frac{\log\left(2\pi k\right)}{2k}-\frac{c_{14}\tau^{1/4}}{k^{1/4}}\left(\log\frac{1}{\epsilon}+\log k\right)
\end{align*}
with probability exceeding $1-4\exp\left(-\tau k\right)$, as claimed.


\section{Proof of Lemma \ref{lemma-Rectangular-Nonhermitian-LogDet}\label{sec:Proof-of-lemma-Rectangular-Nonhermitian-LogDet}}

We first develop the upper bound. The Cauchy-Binet formula indicates
that
\[
\mathbb{E}\left[\det\left(\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]=\sum_{\boldsymbol{s}\in{[n] \choose m}}\mathbb{E}\left[\det\left(\boldsymbol{A}_{\boldsymbol{s}}\boldsymbol{A}_{\boldsymbol{s}}^{\top}\right)\right],
\]
where $\boldsymbol{s}$ ranges over all $m$-combinations of $\{1,\cdots,n\}$,
and $\boldsymbol{A}_{\boldsymbol{s}}$ is the $m\times m$ minor of
$\boldsymbol{A}$ whose columns are the columns of $\boldsymbol{A}$
at indices from ${\bf s}$. It has been shown in (\ref{eq:ExpectedDetMSquareNew})
that for each jointly independent $m\times m$ ensemble $\boldsymbol{A}_{\boldsymbol{s}}$,
the determinant satisfies
\[
\mathbb{E}\left[\det\left(\boldsymbol{A}_{\boldsymbol{s}}\boldsymbol{A}_{\boldsymbol{s}}^{\top}\right)\right]=m!,
\]
which immediately leads to
\[
\mathbb{E}\left[\det\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]=\frac{1}{n^{m}}\sum_{\boldsymbol{s}\in{[n] \choose m}}\mathbb{E}\left[\det\left(\boldsymbol{A}_{\boldsymbol{s}}\boldsymbol{A}_{\boldsymbol{s}}^{\top}\right)\right]=\frac{m!}{n^{m}}{n \choose m}.
\]
Besides, using the well-known Stirling type inequality
\[
\sqrt{2\pi}m^{m+\frac{1}{2}}e^{-m}\leq m!\leq em^{m+\frac{1}{2}}e^{-m},
\]
one can obtain
\[
\log\left(m!\right)\leq\log\left(em^{m+\frac{1}{2}}e^{-m}\right)=\left(m+\frac{1}{2}\right)\log m-m+1
\]
and similarly
\[
\log\left(m!\right)\geq\left(m+\frac{1}{2}\right)\log m-m+\frac{1}{2}\log\left(2\pi\right).
\]
These further give rise to 
\begin{align}
\frac{1}{n}\log\mathbb{E}\left[\det\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right] & \leq-\frac{m}{n}\log n+\frac{\left(m+\frac{1}{2}\right)\log m}{n}-\frac{m}{n}+\frac{1}{n}+\mathcal{H}\left(\frac{m}{n}\right)\nonumber \\
 & =-\frac{m}{n}\log n+\frac{m\log m}{n}-\frac{m}{n}+\frac{2+\log m}{2n}+\mathcal{H}\left(\frac{m}{n}\right)\nonumber \\
 & =\left(1-\alpha\right)\log\frac{1}{1-\alpha}-\alpha+\frac{\log\left(e^{2}m\right)}{2n}\label{eq:UB_logEdet}
\end{align}
and, similarly, 
\begin{align}
\frac{1}{n}\log\mathbb{E}\left[\det\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right] & \geq\left(1-\alpha\right)\log\frac{1}{1-\alpha}-\alpha+\frac{\log\left(2\pi m\right)}{2n}.\label{eq:LB_logEdet}
\end{align}


Define
\[
f_{1,\epsilon}\left(x\right):=\begin{cases}
\frac{2}{\sqrt{\epsilon}}\left(\sqrt{x}-\sqrt{\epsilon}\right)+\log\epsilon,\quad & 0<x<\epsilon,\\
\log x, & x\geq\epsilon,
\end{cases}
\]
and
\begin{equation}
\text{det}^{\epsilon}\left(\boldsymbol{X}\right):=\prod_{i=1}^{m}e^{f_{1,\epsilon}\left(\lambda_{i}\left(\boldsymbol{X}\right)\right)}.\label{eq:Defn_Det_epsilon}
\end{equation}
One can easily justify that the Lischiptz constant of the function
\[
g_{1,\epsilon}\left(x\right):=f_{1,\epsilon}\left(x^{2}\right)=\begin{cases}
\frac{2}{\sqrt{\epsilon}}\left(x-\sqrt{\epsilon}\right)+\log\epsilon,\quad & 0<x<\sqrt{\epsilon},\\
2\log x & x\geq\sqrt{\epsilon},
\end{cases}
\]
is bounded above by $\frac{2}{\sqrt{\epsilon}}$, and that $g_{1,\epsilon}\left(x\right)$
is a \emph{concave} function. Also, 
\[
\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)=\text{det}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)
\]
holds on the event $\left\{ \lambda_{\min}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\geq\epsilon\right\} $.

Set
\begin{equation}
Z:=\log\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)-\mathbb{E}\left[\log\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right].\label{eq:DefnZ}
\end{equation}
Applying \cite[Corollary 1.8]{GuionnetZeitouni2000} then gives
\begin{equation}
\mathbb{P}\left(\left|Z\right|>\tau\right)\leq4\exp\left(-\frac{\tilde{c}\epsilon\tau^{2}}{\alpha}\right)\label{eq:UB_Z}
\end{equation}
for some constant $\tilde{c}$ (in particular, $\tilde{c}=\frac{1}{8}$
when $\boldsymbol{A}_{ij}\sim\mathcal{N}\left(0,1\right)$). This
allows us to derive 
\begin{align}
\mathbb{E}\left[e^{Z}\right] & \leq\mathbb{E}\left[e^{\left|Z\right|}\right]=-e^{z}\mathbb{P}\left(\left|Z\right|>z\right)\mid_{z=0}^{\infty}+\int_{0}^{\infty}e^{z}\mathbb{P}\left(\left|Z\right|>z\right)\mathrm{d}z\nonumber \\
 & \leq1+\int_{0}^{\infty}4\exp\left(z-\frac{\tilde{c}\epsilon z^{2}}{\alpha}\right)\mathrm{d}z\nonumber \\
 & <1+4\sqrt{\frac{\pi\alpha}{\tilde{c}\epsilon}}\exp\left(\frac{\alpha}{4\tilde{c}\epsilon}\right).\label{eq:UB_Ee_Z}
\end{align}
Taking the logarithm of both sides of (\ref{eq:UB_Ee_Z}) and plugging
in the expression of $Z$ yields
\begin{align}
\log\mathbb{E}\left[\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right] & \leq\mathbb{E}\left[\log\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]+\log\left[1+4\sqrt{\frac{\pi\alpha}{\tilde{c}\epsilon}}\exp\left(\frac{\alpha}{4\tilde{c}\epsilon}\right)\right],
\end{align}
leading to
\begin{align}
\frac{1}{n}\mathbb{E}\left[\log\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right] & \geq\frac{1}{n}\log\mathbb{E}\left[\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]-\mathcal{O}\left(\frac{1}{n\epsilon}\right)\geq\frac{1}{n}\log\mathbb{E}\left[\text{det}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]-\mathcal{O}\left(\frac{1}{n\epsilon}\right)\nonumber \\
 & =\left(1-\alpha\right)\log\frac{1}{1-\alpha}-\alpha+\frac{\log\left(2\pi m\right)}{2n}-\mathcal{O}\left(\frac{1}{n\epsilon}\right).\label{eq:LowerBoundELogDetEpsilon}
\end{align}
In the Gaussian case, this can be more explicitly expressed as: for
any $\epsilon\leq\alpha$:
\begin{align}
\frac{1}{n}\mathbb{E}\left[\log\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right] & \geq\frac{1}{n}\log\mathbb{E}\left[\text{det}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]-\frac{1}{n}\log\left[1+8\sqrt{\frac{2\pi\alpha}{\epsilon}}\exp\left(\frac{2\alpha}{\epsilon}\right)\right]\nonumber \\
 & \geq\frac{1}{n}\log\mathbb{E}\left[\text{det}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]-\frac{1}{n}\log\left[\exp\left(\frac{6\alpha}{\epsilon}\right)\right]\nonumber \\
 & \geq\left(1-\alpha\right)\log\frac{1}{1-\alpha}-\alpha+\frac{\log\left(2\pi m\right)}{2n}-\frac{6\alpha}{n\epsilon}.\label{eq:LowerBoundELogDetEpsilon-Gaussian}
\end{align}
where we have made use of the fact that $1+8\sqrt{\pi x}\exp(x)\leq\exp(3x)$
for any $x\geq2$.

On the other hand, in order to develop an upper bound on $\mathbb{E}\left[\log\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]$,
we set $\tau=\sqrt{\frac{\alpha\log n}{\tilde{c}\epsilon}}$ in the
inequality (\ref{eq:UB_Z}), which reveals that with probability at
least $1-4n^{-1}$, 
\[
\left|Z\right|=\left|\log\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)-\mathbb{E}\left[\log\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]\right|\leq\sqrt{\frac{\alpha\log n}{\tilde{c}\epsilon}}
\]
or, equivalently, 
\[
\frac{1}{e^{\sqrt{\frac{\alpha\log n}{\tilde{c}\epsilon}}}}e^{\mathbb{E}\left[\log\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]}\leq\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\leq e^{\sqrt{\frac{\alpha\log n}{\tilde{c}\epsilon}}}\cdot e^{\mathbb{E}\left[\log\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]}
\]
with probability exceeding $1-4n^{-1}$. In addition, the tail distribution
for $\sigma_{\min}\left(\boldsymbol{A}\right)$ satisfies \cite[Theorem 1.1]{RudelsonVershynin2009}
\[
\mathbb{P}\left\{ \sigma_{\min}\left(\frac{1}{\sqrt{n}}\boldsymbol{A}\right)\leq\tilde{\epsilon}\left(1-\sqrt{\alpha}\right)\right\} \leq\left(C\tilde{\epsilon}\right)^{(1-\alpha)n}+e^{-cn}
\]
for some constants $C,c>0$. For sufficiently small $\epsilon>0$,
one can simply write
\[
\mathbb{P}\left\{ \sigma_{\min}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\leq\epsilon\right\} \leq C_{\mathrm{m}}\exp\left(-c_{\mathrm{m}}n\right)
\]
for some constants $c_{\mathrm{m}},C_{\mathrm{m}}>0$, implying that
\begin{equation}
\mathbb{P}\left\{ \det\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)=\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right\} \geq1-C_{\mathrm{m}}\exp\left(-c_{\mathrm{m}}n\right).\label{eq:EqualityDetDetEpsilon}
\end{equation}


Hence, the union bound suggests that 
\[
\det\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)=\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\quad\text{and}\quad\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)>e^{-1}e^{\mathbb{E}\left[\log\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]}
\]
hold simultaneously with probability exceeding $1-\frac{\breve{c}}{n}$
for some constant $\breve{c}>0$. Consequently,
\[
\mathbb{E}\left[\det\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]\geq\left(1-\frac{\breve{c}}{n}\right)\frac{1}{e^{\sqrt{\frac{\alpha\log n}{\tilde{c}\epsilon}}}}e^{\mathbb{E}\left[\log\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]},
\]
indicating that for sufficiently large $n$, 
\begin{align*}
\frac{1}{n}\log\mathbb{E}\left[\det\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right] & \geq\frac{1}{n}\mathbb{E}\left[\log\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]+\frac{\log\left(1-\frac{\breve{c}}{n}\right)}{n}-\frac{\sqrt{\frac{\alpha\log n}{\tilde{c}\epsilon}}}{n}\\
 & \geq\frac{1}{n}\mathbb{E}\left[\log\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]-\mathcal{O}\left(\frac{\sqrt{\log n}}{\sqrt{\epsilon}n}\right)
\end{align*}
Equivalently, we have, for sufficiently large $n$ and sufficiently
small constant $\epsilon>0$, 
\begin{align}
\frac{1}{n}\mathbb{E}\left[\log\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right] & \leq\frac{1}{n}\log\mathbb{E}\left[\det\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]+\mathcal{O}\left(\frac{\sqrt{\log n}}{\sqrt{\epsilon}n}\right)\nonumber \\
 & \leq\left(1-\alpha\right)\log\frac{1}{1-\alpha}-\alpha+\frac{\log\left(e^{2}m\right)}{2n}+\mathcal{O}\left(\frac{\sqrt{\log n}}{\sqrt{\epsilon}n}\right),\label{eq:UpperBoundLogDetEpsilonAA}
\end{align}
where the last inequality follows from (\ref{eq:UB_logEdet}).

Now we are ready to develop a high-probability bound on $\frac{1}{n}\log\text{det}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)$.
By picking $\epsilon>0$ to be a sufficiently small \emph{constant},
one has
\begin{align*}
 & \mathbb{P}\left\{ \frac{1}{n}\log\text{det}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\geq\left(1-\alpha\right)\log\frac{1}{1-\alpha}-\alpha+\frac{1}{\sqrt{n}}\right\} \\
 & \quad\leq\text{ }\mathbb{P}\left\{ \frac{1}{n}\log\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\geq\left(1-\alpha\right)\log\frac{1}{1-\alpha}-\alpha+\frac{1}{\sqrt{n}}\right\} \\
 & \quad=\text{ }\mathbb{P}\left\{ \frac{1}{n}Z\geq\frac{1}{\sqrt{n}}-\mathcal{O}\left(\frac{\log n}{n\sqrt{\epsilon}}\right)\right\} \\
 & \quad\leq\text{ }C_{7}\exp\left(-c_{7}n\right)
\end{align*}
for some absolute constants $c_{7},C_{7}>0$. Similarly, for sufficiently
small \emph{constant} $\epsilon>0$,
\begin{align}
 & \mathbb{P}\left\{ \frac{1}{n}\log\text{det}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\leq\left(1-\alpha\right)\log\frac{1}{1-\alpha}-\alpha-\frac{1}{\sqrt{n}}\right\} \nonumber \\
 & \quad\leq\text{ }\mathbb{P}\left\{ \text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\neq\text{det}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right\} +\mathbb{P}\left\{ \frac{1}{n}\log\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\leq\left(1-\alpha\right)\log\frac{1}{1-\alpha}-\alpha-\frac{1}{\sqrt{n}}\right\} \nonumber \\
 & \quad\leq\text{ }C_{\mathrm{m}}\exp\left(-c_{\mathrm{m}}n\right)+\mathbb{P}\left\{ \frac{1}{n}Z\leq-\frac{1}{\sqrt{n}}+\mathcal{O}\left(\frac{\log n}{n}+\frac{1}{n\epsilon}\right)\right\} \label{eq:SplitBound}\\
 & \quad\leq\text{ }\tilde{C}_{7}\exp\left(-\tilde{c}_{7}n\right)\nonumber 
\end{align}
for some absolute constants $\tilde{c}_{7},\tilde{C}_{7}>0$. Here,
(\ref{eq:SplitBound}) is a consequence of (\ref{eq:EqualityDetDetEpsilon})
and (\ref{eq:LowerBoundELogDetEpsilon}). This establishes Part (1)
of the lemma.


\section{Proof of Lemma \ref{lem:GaussianLogDet_LB}\label{sec:Proof-of-Lemma-GaussianLogDet_LB}}

(1) In order to obtain a lower bound on $\log\text{det}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)$,
we first attempt to estimate the number of eigenvalues of $\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}$
that are smaller than some value $\epsilon>0$, i.e. $\sum_{i=1}^{n}{\bf 1}_{[0,\epsilon]}\left(\lambda_{i}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right)$.
Since the indicator function ${\bf 1}_{[0,\epsilon]}\left(\cdot\right)$
entails discontinuous points, we define an upper bound on ${\bf 1}_{[0,\epsilon]}(\cdot)$
as
\[
f_{2,\epsilon}(x):=\begin{cases}
1, & \text{if }0\leq x\leq\epsilon;\\
-x/\epsilon+2,\quad & \text{if }\epsilon<x\leq2\epsilon;\\
0, & \text{else}.
\end{cases}
\]
For any $\epsilon>0$, one can easily verify that%
\footnote{This follows from the inequality $\frac{\epsilon}{x}\geq1$ ($0\leq x\leq\epsilon$)
as well as the bound that $\frac{\epsilon}{x}+\frac{x}{\epsilon}-2\geq2\sqrt{\frac{\epsilon}{x}\cdot\frac{x}{\epsilon}}-2=0$.%
}
\[
f_{2,\epsilon}(x)\leq\frac{\epsilon}{x},\quad\forall x\geq0,
\]
and hence
\[
\sum_{i=1}^{n}f_{2,\epsilon}\left(\lambda_{i}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right)\leq\sum_{i=1}^{m}\frac{\epsilon}{\lambda_{i}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)}=\epsilon\text{tr}\left(\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)^{-1}\right).
\]
This further gives
\[
\mathbb{E}\left[\frac{1}{n}\sum_{i=1}^{n}f_{2,\epsilon}\left(\lambda_{i}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right)\right]\leq\frac{\epsilon}{n}\mathbb{E}\left[\text{tr}\left(\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)^{-1}\right)\right]=\frac{m\epsilon}{n-m-1}=\frac{\alpha}{1-\alpha-\frac{1}{n}}\epsilon,
\]
which follows from the property of inverse Wishart matrices (e.g.
\cite[Theorem 2.2.8]{Fujikoshi2010}).

\begin{comment}
e following crude bound
\begin{equation}
\sum_{i=1}^{n}{\bf 1}_{[0,\epsilon]}\left(\lambda_{i}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right)\leq\sum_{i=1}^{n}f_{2,\epsilon}\left(\lambda_{i}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right)<\sum_{i:\lambda_{i}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\leq2\epsilon}\log\frac{1}{\lambda_{i}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)}.\label{eq:UpperboundIndicatorGLog}
\end{equation}


Since $x\log\frac{1}{x}$ is an increasing function for $x<2\epsilon<e^{-1}$,
we have $\log\frac{1}{x}\leq\frac{2\epsilon\log\frac{1}{2\epsilon}}{x}$
and hence
\[
\sum_{i:\lambda_{i}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\leq2\epsilon}\log\frac{1}{\lambda_{i}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)}\leq\sum_{i:\lambda_{i}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\leq2\epsilon}\frac{2\epsilon\log\frac{1}{2\epsilon}}{\lambda_{i}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)}\leq2\epsilon\log\frac{1}{2\epsilon}\text{tr}\left(\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)^{-1}\right).
\]
\end{comment}


Clearly, the Lipschitz constant of the function 
\[
g_{2,\epsilon}(x):=f_{2,\epsilon}(x^{2})=\begin{cases}
1, & \text{if }0\leq x\leq\sqrt{\epsilon};\\
-x^{2}/\epsilon+2,\quad & \text{if }\sqrt{\epsilon}<x\leq\sqrt{2\epsilon};\\
0, & \text{else}.
\end{cases}
\]
 is bounded above by $\sqrt{8/\epsilon}$. Applying \cite[Corollary 1.8(b)]{GuionnetZeitouni2000}
then yields that for any $\delta>0$
\begin{align*}
 & \mathbb{P}\left(\frac{1}{n}\sum_{i=1}^{n}{\bf 1}_{[0,\epsilon]}\left(\lambda_{i}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right)>\frac{\alpha}{1-\alpha-\frac{1}{n}}\epsilon+\delta\right)\\
 & \quad\leq\text{ }\mathbb{P}\left(\frac{1}{n}\sum_{i=1}^{n}f_{2,\epsilon}\left(\lambda_{i}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right)>\frac{\alpha}{1-\alpha-\frac{1}{n}}\epsilon+\delta\right)\\
 & \quad\leq\text{ }\mathbb{P}\left(\frac{1}{n}\sum_{i=1}^{n}f_{2,\epsilon}\left(\lambda_{i}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right)>\mathbb{E}\left[\frac{1}{n}\sum_{i=1}^{n}f_{2,\epsilon}\left(\lambda_{i}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right)\right]+\delta\right)\\
 & \quad\leq\text{ }2\exp\left(-\frac{\epsilon}{16\alpha}\delta^{2}n^{2}\right).
\end{align*}
In other words, this immediately implies that for any $\tau>0$,
\begin{equation}
\frac{\text{card}\left\{ i\mid\lambda_{i}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)<\epsilon\right\} }{n}<\frac{\alpha}{1-\alpha-\frac{1}{n}}\epsilon+\frac{4\sqrt{\alpha\tau}}{\sqrt{n\epsilon}}\label{eq:SmallEvalueCardinalityBound}
\end{equation}
with probability exceeding $1-2\exp\left(-\tau n\right)$, as claimed.
By setting $\epsilon=n^{-1/3}$, one has
\begin{equation}
\frac{\text{card}\left\{ i\mid\lambda_{i}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)<\frac{1}{n^{1/3}}\right\} }{n}<\frac{\frac{\alpha}{1-\alpha-\frac{1}{n}}+4\sqrt{\alpha\tau}}{n^{1/3}}.\label{eq:CardinalityEpsilon}
\end{equation}
with probability at least $1-2\exp\left(-\tau n\right)$.

(2) Lemma \ref{lemma-Concentration-of-Norm-M} asserts that if $n>\frac{2}{1-\sqrt{\alpha}}$,
then
\[
\lambda_{\min}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\geq\left(1-\sqrt{\alpha}-\frac{2}{n}\right)^{2}
\]
with probability at least $1-\exp\left(-\frac{2}{n}\right)$. Conditional
on this event, we have
\[
\det\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)=\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)
\]
for the numerical value $\epsilon=\left(1-\sqrt{\alpha}-\frac{2}{n}\right)^{2}$.
Additionally, the bound (\ref{eq:UB_Z}) under Gaussian ensembles
can be explicitly written as
\begin{equation}
\mathbb{P}\left(\left|Z\right|>\tau\right)\leq2\exp\left(-\frac{\epsilon\tau^{2}}{8\alpha}\right),
\end{equation}
indicating that
\begin{equation}
\mathbb{P}\left\{ \left|\frac{1}{n}\log\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)-\mathbb{E}\left[\frac{1}{n}\log\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]\right|>\sqrt{\frac{4\alpha}{\epsilon n}}\right\} <2\exp\left(-\frac{n}{2}\right).\label{eq:GaussianDetAA}
\end{equation}
and
\begin{equation}
\mathbb{P}\left\{ \left|\frac{1}{n}\log\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)-\mathbb{E}\left[\frac{1}{n}\log\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]\right|>\frac{4\tau\sqrt{\frac{\alpha}{\epsilon}}}{n}\right\} <2\exp\left(-2\tau^{2}\right).\label{eq:GaussianDetAA-1}
\end{equation}
Putting these together yields that for any $n>\max\left\{ \frac{2}{1-\sqrt{\alpha}},6\right\} $,

\[
\det\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)=\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\quad\text{and}\quad\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)>\frac{1}{e^{\sqrt{\frac{4\alpha}{\epsilon n}}}}e^{\mathbb{E}\left[\log\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]}
\]
simultaneously hold with probability exceeding $1-2\exp\left(-\frac{n}{2}\right)-\exp\left(-\frac{2}{n}\right)>1/n$.
Since $\det\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)$
is non-negative, this gives
\[
\mathbb{E}\left[\det\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]\geq\frac{1}{n}\cdot\frac{1}{e^{\sqrt{\frac{4\alpha}{\epsilon n}}}}e^{\mathbb{E}\left[\log\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]},
\]
and thereby for any $n>\max\left\{ \frac{2}{1-\sqrt{\alpha}},7\right\} $,
\begin{align*}
\frac{1}{n}\mathbb{E}\left[\log\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right] & \leq\frac{1}{n}\log\mathbb{E}\left[\det\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]+\frac{\log n}{n}+\frac{\sqrt{\frac{4\alpha}{\epsilon}}}{n^{1.5}}\\
 & \leq\left(1-\alpha\right)\log\frac{1}{1-\alpha}-\alpha+\frac{\log\left(e^{2}m\right)+2\log n}{2n}+\frac{2\sqrt{\alpha}}{\left(1-\sqrt{\alpha}-\frac{2}{n}\right)}\frac{1}{n^{1.5}}\\
 & <\left(1-\alpha\right)\log\frac{1}{1-\alpha}-\alpha+\frac{2\log n}{n}+\frac{2\sqrt{\alpha}}{\left(1-\sqrt{\alpha}-\frac{2}{n}\right)}\frac{1}{n^{1.5}},
\end{align*}
where the second inequality follows from (\ref{eq:UB_logEdet}) and
the value $\epsilon=\left(1-\sqrt{\alpha}-\frac{2}{n}\right)^{2}$.

Putting this and (\ref{eq:GaussianDetAA-1}) together gives that for
any $n\geq4$,
\begin{align*}
\frac{1}{n}\log\det\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right) & \leq\frac{4\sqrt{\alpha}}{\left(1-\sqrt{\alpha}-\frac{2}{n}\right)}\frac{\tau}{n}+\left(1-\alpha\right)\log\frac{1}{1-\alpha}-\alpha+\frac{2\log n}{n}+\frac{2\sqrt{\alpha}}{\left(1-\sqrt{\alpha}-\frac{2}{n}\right)}\frac{1}{n^{1.5}}\\
 & <\left(1-\alpha\right)\log\frac{1}{1-\alpha}-\alpha+\frac{2\log n}{n}+\frac{4\sqrt{\alpha}}{\left(1-\sqrt{\alpha}-\frac{2}{n}\right)n}\left(\tau+\frac{1}{2\sqrt{n}}\right)\\
 & \leq\left(1-\alpha\right)\log\frac{1}{1-\alpha}-\alpha+\frac{2\log n}{n}+\frac{5\sqrt{\alpha}}{\left(1-\sqrt{\alpha}-\frac{2}{n}\right)n}\tau
\end{align*}
with probability exceeding $1-2\exp\left(-2\tau^{2}\right)$.

(3) On the other hand, the inequality (\ref{eq:CardinalityEpsilon})
indicates that for any $n>\left(\frac{\alpha}{1-\alpha-\frac{1}{n}}+4\sqrt{\alpha\tau}\right)^{3}$,
\[
\mathbb{P}\left\{ \lambda_{\max}\left(\boldsymbol{A}\boldsymbol{A}^{\top}\right)\leq\frac{1}{n^{1/3}}\right\} \leq\mathbb{P}\left\{ \frac{\text{card}\left\{ i\mid\lambda_{i}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)<\frac{1}{n^{1/3}}\right\} }{n}>\frac{\frac{\alpha}{1-\alpha-\frac{1}{n}}+4\sqrt{\alpha\tau}}{n^{1/3}}\right\} \leq2e^{-\tau n}.
\]
Also, it follows from \cite[Theorem 4.5]{ChenDongarra2005} that for
any constant $n\geq\frac{6.414}{1-\alpha}\cdot e^{\frac{\tau}{1-\alpha}}$,
\begin{align*}
\mathbb{P}\left\{ \frac{\lambda_{\max}\left(\boldsymbol{A}\boldsymbol{A}^{\top}\right)}{\lambda_{\min}\left(\boldsymbol{A}\boldsymbol{A}^{\top}\right)}>n^{2}\right\}  & \leq\text{}\mathbb{P}\left\{ \frac{\lambda_{\max}\left(\boldsymbol{A}\boldsymbol{A}^{\top}\right)}{\lambda_{\min}\left(\boldsymbol{A}\boldsymbol{A}^{\top}\right)}>\frac{n^{2}}{\left(n-m+1\right)^{2}}\cdot\left(n-m+1\right)^{2}\right\} \\
 & \leq\text{}\frac{1}{\sqrt{2\pi}}\left(\frac{6.414}{(1-\alpha)n}\right)^{\left(1-\alpha\right)n}\\
 & \leq\text{ }\frac{1}{\sqrt{2\pi}}e^{-\tau n}
\end{align*}
The above two bounds taken collectively imply that for any $n>\max\left\{ \frac{6.414}{1-\alpha}\cdot e^{\frac{\tau}{1-\alpha}},\left(\frac{\alpha}{1-\alpha-\frac{1}{n}}+4\sqrt{\alpha\tau}\right)^{3}\right\} $,
\begin{align*}
\mathbb{P}\left\{ \lambda_{\min}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)<\frac{1}{n^{7/3}}\right\}  & \leq\text{ }\mathbb{P}\left\{ \lambda_{\max}\left(\boldsymbol{A}\boldsymbol{A}^{\top}\right)\leq\frac{1}{n^{1/3}}\right\} +\mathbb{P}\left\{ \lambda_{\max}\left(\boldsymbol{A}\boldsymbol{A}^{\top}\right)>\frac{1}{n^{1/3}}\text{ and }\lambda_{\min}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)<\frac{1}{n^{7/3}}\right\} \\
 & \leq\text{ }\mathbb{P}\left\{ \lambda_{\max}\left(\boldsymbol{A}\boldsymbol{A}^{\top}\right)\leq\frac{1}{n^{1/3}}\right\} +\mathbb{P}\left\{ \frac{\lambda_{\max}\left(\boldsymbol{A}\boldsymbol{A}^{\top}\right)}{\lambda_{\min}\left(\boldsymbol{A}\boldsymbol{A}^{\top}\right)}>n^{2}\right\} \\
 & \leq\text{ }2e^{-\tau n}+\frac{1}{\sqrt{2\pi}}e^{-\tau n}<3e^{-\tau n}
\end{align*}
for any $n\geq2$. 

Consequently, when $\epsilon=n^{-1/3}$, 
\begin{align*}
\frac{1}{n}\log\det\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right) & \geq\frac{1}{n}\log\mathrm{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)-\sum_{i:\lambda_{i}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)<\frac{1}{n^{1/3}}}\left(\log\frac{1}{n^{1/3}}-\log\frac{1}{n^{7/3}}\right)\\
 & \geq\frac{1}{n}\log\mathrm{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)-\frac{\text{card}\left\{ i\mid\lambda_{i}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)<\frac{1}{n^{1/3}}\right\} }{n}\cdot\log\left(n^{2}\right)\\
 & >\frac{1}{n}\log\mathrm{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)-\left(\frac{2\alpha}{1-\alpha-\frac{1}{n}}+8\sqrt{\alpha\tau}\right)\cdot\frac{\log n}{n^{1/3}}
\end{align*}
with probability exceeding $1-3\exp\left(-\tau n\right)$. Making
use of (\ref{eq:GaussianDetAA-1}) yields that for $\epsilon=n^{-1/3}$,
\[
\mathbb{P}\left\{ \left|\frac{1}{n}\log\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)-\mathbb{E}\left[\frac{1}{n}\log\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]\right|>\frac{\sqrt{8\tau\alpha}}{n^{1/3}}\right\} <2\exp\left(-\tau n\right).
\]
Putting the above two bounds together implies that with probability
exceeding $1-5\exp(-\tau n)$, 
\begin{align*}
\frac{1}{n}\log\det\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right) & \geq\frac{1}{n}\log\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)-\left(\frac{2\alpha}{1-\alpha-\frac{1}{n}}+8\sqrt{\alpha\tau}\right)\frac{\log n}{n^{1/3}}\\
 & \geq\mathbb{E}\left[\frac{1}{n}\log\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]-\frac{\sqrt{8\tau\alpha}}{n^{1/3}}-\left(\frac{2\alpha}{1-\alpha-\frac{1}{n}}+8\sqrt{\alpha\tau}\right)\frac{\log n}{n^{1/3}}\\
 & >\mathbb{E}\left[\frac{1}{n}\log\text{det}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]-\left(\sqrt{8\tau\alpha}+\frac{2\alpha}{1-\alpha-\frac{1}{n}}+8\sqrt{\alpha\tau}\right)\cdot\frac{\log n}{n^{1/3}}\\
 & >\left(1-\alpha\right)\log\frac{1}{1-\alpha}-\alpha-\frac{6\alpha}{n^{2/3}}-\left(\frac{2\alpha}{1-\alpha-\frac{1}{n}}+11\sqrt{\alpha\tau}\right)\frac{\log n}{n^{1/3}},
\end{align*}
where the last inequality follows from (\ref{eq:LowerBoundELogDetEpsilon-Gaussian}).

\begin{comment}
for some constant $c_{5}>0$. 

Now that we have a tail upper bound for the condition number, we can
lower bound $\lambda_{\min}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)$
by developing a lower bound on $\lambda_{\max}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)$.
By setting $\epsilon=n^{-1/4}$ and $\delta=n^{-1/4}$ in (\ref{eq:SmallEvalueCardinalityBound})
and hence $\frac{\alpha}{1-\alpha}\epsilon\log\epsilon+\delta\ll1/2$,
we can derive a crude lower bound such that
\begin{align*}
\mathbb{P}\left(\lambda_{\max}\left(\frac{1}{n}{\bf A}\boldsymbol{A}^{\top}\right)<\frac{1}{n^{1/4}}\right) & \leq\mathbb{P}\left(\frac{\text{card}\left\{ i\mid\lambda_{i}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)<\frac{1}{n^{1/4}}\right\} }{n}>\frac{1}{2}\right)\\
 & \leq2\exp\left(-c_{4}m^{2}\right).
\end{align*}
This together with (\ref{eq:TailBoundConditionNumber}) yields
\begin{align}
 & \mathbb{P}\left(\lambda_{\min}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\leq\frac{1}{n^{4.25}}\right)\nonumber \\
 & \quad\leq\text{ }\mathbb{P}\left(\lambda_{\max}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)<\frac{1}{n^{1/4}}\right)+\mathbb{P}\left(\lambda_{\min}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\leq\frac{1}{n^{4.25}}\text{ and }\lambda_{\max}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)>\frac{1}{n^{1/4}}\right)\nonumber \\
 & \quad\leq\mathbb{P}\left(\lambda_{\max}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)<\frac{1}{n^{1/4}}\right)+\mathbb{P}\left(\frac{\lambda_{\max}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)}{\lambda_{\min}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)}>n^{4}\right)\nonumber \\
 & \quad\leq\exp\left(-\tilde{c}_{5}n\log n\right)\label{eq:LowerBoundLeastEigenValueAAT}
\end{align}
for some constant $\tilde{c}_{5}>0$. 

Set $\epsilon=n^{-1/2}$, $\delta=n^{-1/2}\log n$ from now on, which
indicates that
\begin{equation}
\frac{\text{card}\left\{ i:\lambda_{i}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)<\epsilon\right\} }{n}=O\left(\frac{\log n}{\sqrt{n}}\right)\label{eq:UpperboundSmallEvalue}
\end{equation}
with probability at least $1-\exp\left(-\hat{c}_{6}n\log^{2}n\right)$
for some constant $\hat{c}_{6}>0$. Note that
\begin{align*}
0\leq\frac{1}{n}\log\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)-\frac{1}{n}\log\text{det}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right) & =\frac{1}{n}\sum_{i:\lambda_{i}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)<\epsilon}\log\frac{\epsilon}{\lambda_{i}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)}\\
 & \leq\frac{\text{card}\left\{ i:\lambda_{i}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)<\epsilon\right\} }{n}\log\frac{\epsilon}{\lambda_{\min}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)}.
\end{align*}
This together with (\ref{eq:UpperboundSmallEvalue}) and (\ref{eq:LowerBoundLeastEigenValueAAT})
implies that
\begin{equation}
\frac{1}{n}\log\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)-\frac{1}{n}\log\text{det}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)=\mathcal{O}\left(\frac{\log^{2}n}{\sqrt{n}}\right)\label{eq:DifferenceLogDetEpsilon-LogDet}
\end{equation}
with probability exceeding $1-\exp\left(-\hat{c}_{7}n\log n\right)$
for some constant $\hat{c}_{7}>0$

Besides, Equation (\ref{eq:LowerBoundELogDetEpsilon}) gives a lower
bound on $\mathbb{E}\left[\frac{1}{n}\log\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\right]$.
Setting $\epsilon=n^{-1/2}$ and $\delta=n^{-1/2}\log n$ in Lemma
\ref{lemma-ConcentrationLogDetPlusConstant} leads to
\begin{equation}
\mathbb{P}\left(\frac{1}{n}\log\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)>\left(1-\alpha\right)\log\frac{1}{1-\alpha}-\alpha+\mathcal{O}\left(\frac{\log n+\log\epsilon}{n^{1/2}}\right)\right)\geq1-4\exp\left(-\tilde{c}n\log^{2}n\right).\label{eq:ConcentrationLogDetEpsilon}
\end{equation}
Combining (\ref{eq:DifferenceLogDetEpsilon-LogDet}) and (\ref{eq:ConcentrationLogDetEpsilon})
with a union bound yields
\[
\frac{1}{n}\log\text{det}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{\top}\right)\geq\left(1-\alpha\right)\log\frac{1}{1-\alpha}-\alpha+O\left(\frac{\log^{2}n}{n^{1/2}}\right)
\]
with probability exceeding $1-\exp\left(c_{9}n\log n\right)$ for
some constant $c_{9}>0$.
\end{comment}


\begin{comment}
Besides, since $ $$\frac{1}{n}\log\text{det}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{T}\right)\leq\frac{1}{n}\log\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{T}\right)$,
we have
\begin{align*}
\left|\frac{1}{n}\log\text{det}^{\epsilon}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{T}\right)-\frac{1}{n}\log\text{det}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{T}\right)\right| & =\left|\frac{1}{n}\sum_{i:\lambda_{i}\left(\frac{1}{n}{\bf A}{\bf A}^{T}\right)<\epsilon}\log\frac{\epsilon}{\lambda_{i}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{T}\right)}\right|\\
 & \leq\frac{\text{card}\left\{ i:\lambda_{i}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{T}\right)<\epsilon\right\} }{n}\left|\log\frac{\epsilon}{\lambda_{\min}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{T}\right)}\right|.
\end{align*}
With $\epsilon=n^{-1/4}$ and $\delta=n^{-1/4}$, this allows us to
derive
\[
\mathbb{P}\left\{ \frac{\text{card}\left\{ i:\lambda_{i}\left(\frac{1}{n}\boldsymbol{A}\boldsymbol{A}^{T}\right)<\epsilon\right\} }{n}\geq\frac{4\alpha}{1-\alpha}\frac{\left(1+0.25\log n\right)\log n}{n^{1/4}}\right\} 
\]
 $-\frac{\alpha}{4\left(1-\alpha\right)}n^{-0.25}\left(1+\log n\right)$
\end{comment}


\begin{comment}
We follow the truncation method introduced by Friedland \emph{et.
al. }\cite{FridlandRiderZeitouni2004,CostelloVu2009}. For any given
constant $\epsilon>0$, define a truncated logarithmic function as
\[
\log^{\epsilon}x:=\log\left(\max\left(x,\epsilon\right)\right),
\]
 then it can be observed that $\log^{\epsilon}x$ has Lipschitz constant
$1/\epsilon$. Define the associated truncated version of log-determinant
as
\[
\log\text{det}^{\epsilon}\left({\bf M}{\bf M}^{T}\right):=\sum_{i=1}^{k}2\log^{\epsilon}\left(\lambda_{i}\left({\bf M}{\bf M}^{T}\right)\right).
\]


If $\xi_{ij}$ is  bounded by $D$, then applying \cite[Corollary 1.8(a)]{GuionnetZeitouni2000}
leads to
\[
\mathbb{P}\left(\left|\frac{1}{k}\log\text{det}^{\epsilon}\left({\bf M}{\bf M}^{T}\right)-\mathbb{E}\left(\frac{1}{k}\log\text{det}^{\epsilon}\left({\bf M}{\bf M}^{T}\right)\right)\right|>2\delta\right)\leq4\exp\left(-\frac{\epsilon^{2}}{D^{2}}\left(\delta-\frac{2D\sqrt{\pi}}{\epsilon k}\right)^{2}k^{2}\right).
\]
Setting $\delta$ to be a positive constant independent of $k$, we
have for sufficiently large $k$ that
\[
\mathbb{P}\left(\left|\frac{1}{k}\log\text{det}^{\epsilon}\left({\bf M}{\bf M}^{T}\right)-\mathbb{E}\left(\frac{1}{k}\log\text{det}^{\epsilon}\left({\bf M}{\bf M}^{T}\right)\right)\right|>2\delta\right)\leq4\exp\left(-\frac{\epsilon^{2}\delta^{2}}{2D^{2}}k^{2}\right).
\]
If $\xi_{ij}$ satisfies the LSI with uniformly bounded constant $c_{\text{LS}}$,
then
\[
\mathbb{P}\left(\left|\frac{1}{k}\log\text{det}^{\epsilon}\left({\bf M}{\bf M}^{T}\right)-\mathbb{E}\left(\frac{1}{k}\log\text{det}^{\epsilon}\left({\bf M}{\bf M}^{T}\right)\right)\right|>2\delta\right)\leq2\exp\left(-\frac{2\epsilon^{2}\delta^{2}}{c_{\text{LS}}}k^{2}\right).
\]
Therefore, for both cases, there exists a constant $\tilde{c}>0$
such that
\begin{equation}
\mathbb{P}\left(\left|\frac{1}{k}\log\text{det}^{\epsilon}\left({\bf M}{\bf M}^{T}\right)-\mathbb{E}\left(\frac{1}{k}\log\text{det}^{\epsilon}\left({\bf M}{\bf M}^{T}\right)\right)\right|>2\delta\right)\leq4\exp\left(-\tilde{c}\epsilon^{2}\delta^{2}k^{2}\right).\label{eq:ConcentrationLogDetTruncated}
\end{equation}


The following two inequalities follows respectively from \cite[Equation (7)]{CostelloVu2009}
and \cite[Page 1566]{FridlandRiderZeitouni2004}: for any $\epsilon>0$,
one has

\[
\mathbb{E}\left(\text{det}^{\epsilon}\left({\bf M}{\bf M}^{T}\right)\right)\geq\mathbb{E}\left(\text{det}\left({\bf M}{\bf M}^{T}\right)\right)\geq\left(1+o(1)\right)\exp\left(O\left(k^{\frac{2}{3}}\log k\right)\right)\mathbb{E}\left(\text{det}^{\epsilon}\left({\bf M}{\bf M}^{T}\right)\right),
\]
\[
\mathbb{E}\left(\log\text{det}^{\epsilon}\left({\bf M}{\bf M}^{T}\right)\right)\leq\log\mathbb{E}\left(\text{det}^{\epsilon}\left({\bf M}{\bf M}^{T}\right)\right)\leq\mathbb{E}\left(\log\text{det}^{\epsilon}\left({\bf M}{\bf M}^{T}\right)\right)+\log\left(1+2\exp^{2b^{2}/\epsilon^{2}}\right)
\]
for some constant $b$. Simple manipulation yields
\begin{align}
\frac{1}{k}\mathbb{E}\left(\log\text{det}^{\epsilon}\left({\bf M}{\bf M}^{T}\right)\right) & =\frac{1}{k}\log\mathbb{E}\left(\text{det}^{\epsilon}\left({\bf M}{\bf M}^{T}\right)\right)+O\left(\frac{1}{k}\right)\nonumber \\
 & =\frac{1}{k}\log\mathbb{E}\left(\text{det}\left({\bf M}{\bf M}^{T}\right)\right)+O\left(\frac{\log k}{k^{1/3}}\right).\label{eq:ExpectedLogDetEpsilon}
\end{align}
Besides, the Leibniz formula for the determinant is given by 
\[
\text{det}\left({\bf M}\right)=\sum_{\sigma}\text{\text{sgn}}\left(\sigma\right)\prod_{i=1}^{k}\xi_{i,\sigma(i)},
\]
where the sum is over all permutations $\sigma$ of $\left\{ 1,2,\cdots,k\right\} $.
For any two different permutations $\sigma$ and $\tilde{\sigma}$,
we have $\prod_{i=1}^{k}\xi_{i,\sigma(i)}$ and $\prod_{i=1}^{k}\xi_{i,\tilde{\sigma}(i)}$
are independent by observing that $\xi_{ij}$ are jointly independent.
Therefore,
\begin{align}
\mathbb{E}\text{det}\left({\bf M}{\bf M}^{T}\right) & =\sum_{\sigma}\mathbb{E}\prod_{i=1}^{k}\left|\xi_{i,\sigma(i)}\right|^{2}=\sum_{\sigma}\prod_{i=1}^{k}\mathbb{E}\left|\xi_{i,\sigma(i)}\right|^{2}\nonumber \\
 & =k!\label{eq:ExpectedDetMSquare}
\end{align}
holds for any $k$. Combining (\ref{eq:ExpectedLogDetEpsilon}) and
(\ref{eq:ExpectedDetMSquare}) leads to
\begin{equation}
\frac{1}{k}\mathbb{E}\left(\log\text{det}^{\epsilon}\left({\bf M}{\bf M}^{T}\right)\right)=\frac{\log\left(k!\right)}{k}+O\left(\frac{\log k}{k^{1/3}}\right)=\log k-1+o(1).\label{eq:EstimateLogDetEpsilon}
\end{equation}
This together with (\ref{eq:ConcentrationLogDetTruncated}) yields
\begin{align*}
\mathbb{P}\left(\left|\frac{1}{k}\log\text{det}^{\epsilon}\left(\frac{1}{k}{\bf M}{\bf M}^{T}\right)+1\right|>3\delta\right) & \leq\mathbb{P}\left(\left|\frac{1}{k}\log\text{det}^{\epsilon}\left({\bf M}{\bf M}^{T}\right)-\log k+1-o(1)\right|>2\delta\right)\\
 & \leq4\exp\left(-\tilde{c}\epsilon^{2}\delta^{2}k^{2}\right).
\end{align*}
\end{comment}


\begin{comment}
a


\section{Proof of Lemma \ref{lemma-LogDetRandomCovariance}\label{sec:Proof-of-Lemma-LogDet-Random-Covariance}}

Suppose that ${\bf G}$ is the $k\times n$ dimensional i.i.d. Gaussian
random ensemble $\left(k=\beta n\right)$ such that ${\bf G}_{ij}$
has zero mean and unit variance, which clearly matches the moment
condition (\ref{eq:4thMomentCondition}). Our main tool for this lemma
is the four moment theorem for eigenvalues of random covariance matrices
proved by Tao, Vu and Wang\cite{TaoVu2012Covariance,Wang2012}. 

\begin{theorem}[Four Moment theorem \cite{Wang2012}]For sufficiently
small $c_{0}>0$ and sufficiently large $C_{0}>0$, the following
holds for every $0<\epsilon<1$ and $k\geq1$. Let ${\bf M}=\left(\zeta_{ij}\right)_{1\leq i\leq p,1\leq j\leq n}$
and $\tilde{{\bf M}}=\left(\tilde{\zeta}_{ij}\right)_{1\leq i\leq p,1\leq j\leq n}$
be matrix ensembles such that all entries are jointly independent,
have mean 0 and variance 1, and obey the moment condition $\sup_{i,j}\mathbb{E}\left|\zeta_{ij}\right|^{C_{0}}\leq C$
and $\sup_{i,j}\mathbb{E}\left|\tilde{\zeta}_{ij}\right|^{C_{0}}\leq C$.
Assume further that $\zeta_{ij}$ and $\tilde{\zeta}_{ij}$ match
to order $4$ and that $p/n\rightarrow\alpha$ for some $0<\alpha\leq1$.
Let $G$ be a smooth function obeying 
\begin{equation}
\forall0\leq j\leq5,\text{ }\forall\left(\lambda_{1},\cdots,\lambda_{p}\right)\in\mathbb{R}^{p}:\quad\left|\nabla^{j}G(\lambda_{1},\cdots,\lambda_{p})\right|\leq n^{c_{0}}.\label{eq:G_Derivative_Bound}
\end{equation}
Then
\[
\left|\mathbb{E}G\left(n\lambda_{1}\left({\bf M}{\bf M}^{T}\right),\cdots,n\lambda_{K}\left({\bf M}{\bf M}^{T}\right)\right)-\mathbb{E}G\left(n\lambda_{1}\left(\tilde{{\bf M}}\tilde{{\bf M}}^{T}\right),\cdots,n\lambda_{p}\left(\tilde{{\bf M}}\tilde{{\bf M}}^{T}\right)\right)\right|\leq\frac{1}{n^{c_{0}}}.
\]
\end{theorem}

Our intention is to define indicator functions related to the event
$\left\{ \frac{1}{n}\log\det{\bf M}{\bf M}^{T}\overset{n\rightarrow\infty}{\longrightarrow}\left(1-\alpha\right)\log\frac{1}{1-\alpha}-\alpha\right\} $
and examine its expectation. However, indicator functions are not
smooth functions, and the function $\sum_{i=1}^{k}\log\lambda_{i}$
is not uniformly bounded for all $\lambda_{i}$ as required by Condition
(\ref{eq:G_Derivative_Bound}). Therefore, our strategy is to first
bound $\lambda_{i}$ to some fixed range, and then use smooth bump
functions to approximate the indicator functions.

German \cite{German1980} and Silverstein \cite{Silverstein1985}
proved that as $n\rightarrow\infty$,
\[
\lambda_{1}\left({\bf M}{\bf M}^{T}\right)\rightarrow\left(1+\sqrt{\beta}\right)^{2}\quad\text{and}\quad\lambda_{\min}\left({\bf M}{\bf M}^{T}\right)\rightarrow\left(1-\sqrt{\beta}\right)^{2}
\]
almost surely. Define the following events $\mathcal{A}\left({\bf M}\right)=\left\{ \frac{\lambda_{1}\left({\bf M}{\bf M}^{T}\right)}{n}\leq2\left(1+\sqrt{\beta}\right)^{2}\right\} $,
$\mathcal{B}\left({\bf M}\right)=\left\{ \frac{\lambda_{\min}\left({\bf M}{\bf M}^{T}\right)}{n}\geq0.5\left(1+\sqrt{\beta}\right)^{2}\right\} $,
and $\mathcal{C}_{\epsilon}\left({\bf M}\right)=\left\{ \left|\frac{1}{n}\log\det\left(\frac{1}{n}{\bf M}{\bf M}^{T}\right)-(1-\alpha)\log\frac{1}{1-\alpha}+\alpha\right|\leq\epsilon\right\} $,
and introduce the following three indicator functions
\[
{\bf 1}_{\text{A}}(\lambda_{1})=\begin{cases}
1,\quad & \text{if }\frac{\lambda_{1}}{n}<2\left(1+\sqrt{\beta}\right)^{2};\\
0, & \text{otherwise.}
\end{cases}
\]
\[
{\bf 1}_{\text{B}}(\lambda_{\min})=\begin{cases}
1,\quad & \text{if }\frac{\lambda_{K}}{n}>\frac{1}{2}\left(1-\sqrt{\beta}\right)^{2};\\
0, & \text{otherwise.}
\end{cases}
\]
\[
{\bf 1}_{\text{C}}^{\epsilon}(x)=\begin{cases}
1,\quad & \text{if }\left|\frac{1}{n}x-(1-\alpha)\log\frac{1}{1-\alpha}+\alpha\right|\leq\epsilon;\\
0, & \text{otherwise.}
\end{cases}
\]
Define 
\begin{align*}
I_{\text{A},n}^{+} & \overset{\Delta}{=}\left[-\infty,2\left(1+\sqrt{\beta}\right)^{2}+n^{-c_{0}}\right],\\
I_{\text{A},n}^{-} & \overset{\Delta}{=}\left[-\infty,2\left(1+\sqrt{\beta}\right)^{2}\right],\\
I_{\text{B},n}^{+} & \overset{\Delta}{=}\left[\frac{1}{2}\left(1-\sqrt{\beta}\right)^{2}-n^{-c_{0}},\infty\right],\\
I_{\text{B},n}^{-} & \overset{\Delta}{=}\left[\frac{1}{2}\left(1-\sqrt{\beta}\right)^{2}+n^{-c_{0}},\infty\right],\\
I_{\text{C},n}^{+} & \overset{\Delta}{=}\left[\frac{K}{\alpha}\left((1-\alpha)\log\frac{1}{1-\alpha}+\alpha\right)\pm\left(\epsilon+n^{-c_{0}}\right)\right],\\
I_{\text{C},n}^{-} & \overset{\Delta}{=}\left[\frac{K}{\alpha}\left((1-\alpha)\log\frac{1}{1-\alpha}+\alpha\right)\pm\left(\epsilon-n^{-c_{0}}\right)\right].
\end{align*}
As mentioned in \cite{TaoVu2012Wigner}, we can then define a smooth
bump function $G_{\text{A},n}$ ($G_{\text{B},n}$, $G_{\text{C},n}$)
equal to one on $I_{\text{A},n}^{-}$ ($I_{\text{B},n}^{-}$, $I_{\text{C},n}^{-}$)
and vanishing outside of $I_{\text{A},n}^{+}$ ($I_{\text{B},n}^{+}$,
$I_{\text{C},n}^{+}$). By observing that 
\[
\mathbb{P}\left(\lambda_{1}\left(\frac{1}{n}{\bf M}{\bf M}^{T}\right)\leq2\left(1+\sqrt{\beta}\right)^{2}\right)\leq\mathbb{E}G_{\text{A},N}\leq\mathbb{P}\left(\lambda_{1}\left(\frac{1}{n}{\bf M}{\bf M}^{T}\right)\in I_{\text{A},N}^{+}\right),
\]
we can show that
\[
\left|\mathbb{P}\left(\lambda_{1}\left(\frac{1}{n}{\bf M}{\bf M}^{T}\right)\leq2\left(1+\sqrt{\beta}\right)^{2}\right)-\mathbb{P}\left(\lambda_{1}\left(\frac{1}{n}{\bf G}{\bf G}^{T}\right)\leq2\left(1+\sqrt{\beta}\right)^{2}+n^{-c_{0}}\right)\right|\leq2n^{-c_{0}}.
\]
Therefore, as $n\rightarrow\infty$, $\lambda_{1}\left(\frac{1}{n}{\bf M}{\bf M}^{T}\right)\leq2\left(1+\sqrt{\beta}\right)^{2}$
in probability. Similarly, $\lambda_{\min}\left(\frac{1}{n}{\bf M}{\bf M}^{T}\right)\geq0.5\left(1-\sqrt{\beta}\right)^{2}$
in probability as $n\rightarrow\infty$. Following the same spirit,
we can choose a smooth function 
\begin{align*}
G_{\text{0}}\left({\bf M}\right) & =G_{\text{0}}\left(N\lambda_{1}\left({\bf M}{\bf M}^{T}\right),\cdots,\lambda_{k}\left({\bf M}{\bf M}^{T}\right)\right)\\
 & \overset{\Delta}{=}G_{\text{A}}\left(\lambda_{1}\left(\frac{1}{n}{\bf M}{\bf M}^{T}\right)\right)G_{\text{B}}\left(\lambda_{k}\left(\frac{1}{n}{\bf M}{\bf M}^{T}\right)\right)G_{C}\left(\sum_{i=1}^{k}\log\left(\lambda_{i}\left(\frac{1}{n}{\bf M}{\bf M}^{T}\right)\right)\right)
\end{align*}
 such that $ $it satisfies the derivative bounds (\ref{eq:G_Derivative_Bound}).

Since we use the bump functions to approximate the indicator functions,
we can derive the following crude bound
\begin{align*}
\mathbb{P}\left(\mathcal{A}\left({\bf M}\right)\cap\mathcal{B}\left({\bf M}\right)\cap\mathcal{C}_{\epsilon}\left({\bf M}\right)\right) & \geq\mathbb{E}\left(G_{0}\left({\bf M}\right)\right)-\frac{3}{n^{c_{0}}}\geq\mathbb{E}\left(G_{0}\left({\bf G}\right)\right)-\frac{4}{n^{c_{0}}}\\
 & \geq\mathbb{P}\left(\mathcal{A}\left({\bf G}\right)\cap\mathcal{B}\left({\bf G}\right)\cap\mathcal{C}_{\epsilon}\left({\bf G}\right)\right)-\frac{7}{n^{c_{0}}}.
\end{align*}
Since $\mathbb{P}\left(\mathcal{A}\left({\bf G}\right)\cap\mathcal{B}\left({\bf G}\right)\cap\mathcal{C}_{\epsilon}\left({\bf G}\right)\right)\geq1-\mathbb{P}\left(\overline{\mathcal{A}\left({\bf G}\right)}\right)-\mathbb{P}\left(\overline{\mathcal{B}\left({\bf G}\right)}\right)-\mathbb{P}\left(\overline{\mathcal{C}_{\epsilon}\left({\bf G}\right)}\right)\rightarrow1$
as $n\rightarrow\infty$, we can immediately derive
\[
\lim_{N\rightarrow\infty}\mathbb{P}\left(\mathcal{A}\left({\bf M}\right)\cap\mathcal{B}\left({\bf M}\right)\cap\mathcal{C}_{\epsilon}\left({\bf M}\right)\right)=1.
\]
By observing that $\mathbb{P}\left(\mathcal{C}_{\epsilon}\left({\bf M}\right)\right)\geq\mathbb{P}\left(\mathcal{A}\cap\mathcal{B}\cap\mathcal{C}_{\epsilon}\right)$,
we conclude that 
\[
\lim_{N\rightarrow\infty}\mathbb{P}\left(\mathcal{C}_{\epsilon}\left({\bf M}\right)\right)=1
\]
holds for any $\epsilon>0$. Hence, as $n\rightarrow\infty$,

\[
\frac{1}{n}\log\det\left(\frac{1}{n}{\bf M}{\bf M}^{T}\right)\rightarrow(1-\alpha)\log\frac{1}{1-\alpha}-\alpha
\]
in probability.
\end{comment}



\section{Proof of Lemma \ref{lemma-Expected-Log-Determinant-LambdaStatistics}
\label{sec:Proof-of-lemma-Expected-Log-Determinant-LambdaStatistics}}

Suppose that the singular value decomposition of the real-valued $\boldsymbol{A}$
is given by $\boldsymbol{A}=\boldsymbol{U}_{\boldsymbol{A}}\left[\begin{array}{c}
\boldsymbol{\Sigma}_{\boldsymbol{A}}\\
{\bf 0}
\end{array}\right]\boldsymbol{V}_{\boldsymbol{A}}^{\top}$, where $\boldsymbol{\Sigma}_{\boldsymbol{A}}$ is a diagonal matrix
containing all $k$ singular values of $\boldsymbol{A}$. One can
then write 
\begin{align}
\log\det\left(\epsilon\boldsymbol{I}_{k}+\boldsymbol{A}^{\top}\boldsymbol{B}^{-1}\boldsymbol{A}\right) & =\log\det\left(\epsilon\boldsymbol{I}_{k}+\boldsymbol{V}_{\boldsymbol{A}}\left[\begin{array}{cc}
\boldsymbol{\Sigma}_{\boldsymbol{A}} & {\bf 0}\end{array}\right]\boldsymbol{U}_{\boldsymbol{A}}^{\top}\boldsymbol{B}^{-1}\boldsymbol{U}_{\boldsymbol{A}}\left[\begin{array}{c}
\boldsymbol{\Sigma}_{\boldsymbol{A}}\\
{\bf 0}
\end{array}\right]\boldsymbol{V}_{\boldsymbol{A}}^{\top}\right)\nonumber \\
 & =\log\det\left(\epsilon\boldsymbol{I}_{k}+\boldsymbol{\Sigma}_{\boldsymbol{A}}\left(\tilde{\boldsymbol{B}}^{-1}\right)_{[k]}\boldsymbol{\Sigma}_{\boldsymbol{A}}\right)\nonumber \\
 & \geq\log\det\left(\frac{1}{n}\boldsymbol{\Sigma}_{\boldsymbol{A}}^{2}\right)-\log\det\left\{ \frac{1}{n}\left(\tilde{\boldsymbol{B}}^{-1}\right)_{[k]}\right\} \label{eq:SeparationLogDetI_Plus_ABA}
\end{align}
where $\tilde{\boldsymbol{B}}=\boldsymbol{U}_{\boldsymbol{A}}^{\top}\boldsymbol{B}\boldsymbol{U}_{\boldsymbol{A}}\sim\mathcal{W}_{m}\left(n-k,\boldsymbol{U}_{\boldsymbol{A}}^{\top}\boldsymbol{U}_{\boldsymbol{A}}\right)=\mathcal{W}_{m}\left(n-k,\boldsymbol{I}_{m}\right)$
from the property of Wishart distribution. Here, $\left(\tilde{\boldsymbol{B}}^{-1}\right)_{[k]}$
denotes the leading $k\times k$ minor consisting of matrix elements
of $\tilde{\boldsymbol{B}}^{-1}$ in rows and columns from $1$ to
$k$, which is independent of $\boldsymbol{A}$ by Gaussianality. 

Note that $\frac{1}{n}\log\det\left(\frac{1}{n}\boldsymbol{\Sigma}_{\boldsymbol{A}}^{2}\right)=\frac{1}{n}\log\det\left(\frac{1}{n}\boldsymbol{A}^{\top}\boldsymbol{A}\right)$.
Then Lemma \ref{lem:GaussianLogDet_LB} implies that for any $\tau>\frac{1}{n^{1/3}}$,
\begin{align}
\frac{1}{n}\log\det\left(\frac{1}{n}\boldsymbol{\Sigma}_{\boldsymbol{A}}^{2}\right) & =\frac{1}{n}\log\det\left(\frac{1}{n}\boldsymbol{A}^{\top}\boldsymbol{A}\right)=\frac{m}{n}\frac{1}{m}\log\det\left(\frac{1}{m}\boldsymbol{A}^{\top}\boldsymbol{A}\right)+\frac{1}{n}\log\det\left(\frac{m}{n}\boldsymbol{I}_{k}\right)\nonumber \\
 & \geq\alpha\left(-\left(1-\frac{\beta}{\alpha}\right)\log\left(1-\frac{\beta}{\alpha}\right)-\frac{\beta}{\alpha}\right)+\beta\log\alpha-\frac{c_{8}\tau\log n}{n^{1/3}}\nonumber \\
 & =-\left(\alpha-\beta\right)\log\left(\frac{\alpha-\beta}{\alpha}\right)-\beta+\beta\log\alpha-\frac{c_{8}\tau\log n}{n^{1/3}}\nonumber \\
 & =-\left(\alpha-\beta\right)\log\left(\alpha-\beta\right)-\beta+\alpha\log\alpha-\frac{c_{8}\tau\log n}{n^{1/3}},\label{eq:ConcentrationLogDetAAT}
\end{align}
with probability exceeding $1-C_{8}\exp\left(-\tau^{2}n\right)$ for
some absolute constants $C_{8},c_{8}>0$. On the other hand, it is
well known (e.g. \cite[Theorem 2.3.3]{Fujikoshi2010}) that for a
Wishart matrix $\tilde{\boldsymbol{B}}\sim\mathcal{W}_{m}\left(n-k,\boldsymbol{I}_{m}\right)$,
$\left(\tilde{\boldsymbol{B}}^{-1}\right)_{[k]}^{-1}$ also follows
the Wishart distribution, that is, $\left(\tilde{\boldsymbol{B}}^{-1}\right)_{[k]}^{-1}\sim\mathcal{W}_{k}\left(n-m,\boldsymbol{I}_{k}\right)$.
Applying Lemma \ref{lem:GaussianLogDet_LB} again yields that
\begin{align}
 & \frac{1}{n}\log\det\left(\frac{1}{n}\left(\tilde{\boldsymbol{B}}^{-1}\right)_{[k]}^{-1}\right)=\frac{n-m}{n}\frac{1}{n-m}\log\det\left(\frac{1}{n-m}\left(\tilde{\boldsymbol{B}}^{-1}\right)_{[k]}^{-1}\right)+\frac{1}{n}\log\det\left(\frac{n-m}{n}\boldsymbol{I}_{k}\right)\nonumber \\
 & \quad\leq\left(1-\alpha\right)\left\{ -\left(1-\frac{\beta}{1-\alpha}\right)\log\left(1-\frac{\beta}{1-\alpha}\right)-\frac{\beta}{1-\alpha}\right\} +\beta\log\left(1-\alpha\right)+\frac{c_{9}\tau}{\sqrt{n}}\nonumber \\
 & \quad=-\left(1-\alpha-\beta\right)\log\left(1-\frac{\beta}{1-\alpha}\right)-\beta+\beta\log\left(1-\alpha\right)+\frac{c_{9}\tau}{\sqrt{n}}\label{eq:ConcentrationLogDetBMinor}
\end{align}
holds with probability exceeding $1-C_{9}\exp\left(-\tau^{2}n\right)$
for some universal constants $C_{9},c_{9}>0$.

Combining (\ref{eq:SeparationLogDetI_Plus_ABA}), (\ref{eq:ConcentrationLogDetAAT})
and (\ref{eq:SeparationLogDetI_Plus_ABA}) suggests that for any $\tau>\frac{1}{n^{1/3}}$,
\begin{align*}
\frac{1}{n}\log\det\left(\epsilon\boldsymbol{I}_{k}+\boldsymbol{A}^{\top}\boldsymbol{B}^{-1}\boldsymbol{A}\right) & \geq-\left(\alpha-\beta\right)\log\left(\alpha-\beta\right)+\alpha\log\alpha+\left(1-\alpha-\beta\right)\log\left(1-\frac{\beta}{1-\alpha}\right)\\
 & \quad\quad-\beta\log\left(1-\alpha\right)-\frac{c_{10}\tau\log n}{n^{1/3}}
\end{align*}
with probability exceeding $1-C_{10}\exp\left(-\tau^{2}n\right)$
for some constants $C_{10},c_{10}>0$.

\begin{comment}
a


\section{Proof of Lemma \ref{lemma-Concentration-Log-Det-As_W_As}\label{sec:Proof-of-lemma-Concentration-Log-Det-As_W_As}}

Note that $\Xi_{\backslash s}$ is a $m$-dimensional Wishart matrix
with $n-k$ degrees of freedom. Applying \cite[Lemma 4]{LozanoTuilinoVerdu2005}
or \cite[Theorem 2.12]{TulinoVerdu2004} yields
\begin{align}
\mathbb{E}\det\left({\bf A}_{s}^{T}\Xi_{\backslash s}^{-1}{\bf A}_{s}\right) & =\prod_{l=0}^{k-1}\frac{\Gamma\left(k+n-m-l-1\right)}{\Gamma\left(k+n-m-l\right)}\frac{\Gamma\left(m-l+1\right)}{\Gamma\left(m-l\right)}\nonumber \\
 & =\frac{m!\left(n-m-1\right)!}{\left(m-k\right)!\left(k+n-m-1\right)!}=\frac{{m \choose k}}{{k+n-m-1 \choose k}}\label{eq:ExpectedDet_A_s_W_A_s}
\end{align}
\begin{align*}
\mathbb{E}\det\left({\bf A}_{s}^{T}\Xi_{\backslash s}^{-1}{\bf A}_{s}\right)^{\frac{1}{2}} & =\prod_{l=0}^{k-1}\frac{\Gamma\left(k+n-m-l-\frac{1}{2}\right)}{\Gamma\left(k+n-m-l\right)}\frac{\Gamma\left(m-l+\frac{1}{2}\right)}{\Gamma\left(m-l\right)}
\end{align*}
Therefore, 
\begin{align}
\frac{\mathbb{E}\det\left({\bf A}_{s}^{T}\Xi_{\backslash s}^{-1}{\bf A}_{s}\right)}{\left(\mathbb{E}\det\left({\bf A}_{s}^{T}\Xi_{\backslash s}^{-1}{\bf A}_{s}\right)^{\frac{1}{2}}\right)^{2}} & =\prod_{l=0}^{k-1}\frac{\Gamma\left(k+n-m-l-1\right)\Gamma\left(k+n-m-l\right)}{\left[\Gamma\left(k+n-m-l-\frac{1}{2}\right)\right]^{2}}\frac{\Gamma\left(m-l\right)\Gamma\left(m-l+1\right)}{\left[\Gamma\left(m-l+\frac{1}{2}\right)\right]^{2}}\label{eq:RatioDet_vs_Det_half}
\end{align}
The asymptotic approximation $\frac{\Gamma(n+a)}{\Gamma(n+b)}=n^{a-b}\left(1+\frac{\left(a-b\right)\left(a+b-1\right)}{2n}+O(n^{-2})\right)$
\cite{ErdelyiTricomi1951} allows us to derive
\[
\frac{\Gamma(n)\Gamma(n+1)}{\left[\Gamma\left(n+\frac{1}{2}\right)\right]^{2}}=\frac{1}{n}\left(\frac{\Gamma(n+1)}{\Gamma(n+\frac{1}{2})}\right)^{2}=\frac{1}{n}\left(\sqrt{n}\left[1+\frac{1}{8n}+O\left(\frac{1}{n^{2}}\right)\right]\right)^{2}=1+\frac{1}{4n}+O\left(\frac{1}{n^{2}}\right),
\]
which, when plugged into (\ref{eq:RatioDet_vs_Det_half}), yields
\begin{align*}
\frac{\mathbb{E}\det\left({\bf A}_{s}^{T}\Xi_{\backslash s}^{-1}{\bf A}_{s}\right)}{\left(\mathbb{E}\det\left({\bf A}_{s}^{T}\Xi_{\backslash s}^{-1}{\bf A}_{s}\right)^{\frac{1}{2}}\right)^{2}} & =\prod_{l=0}^{k-1}\left(1+\frac{1}{4\left(k+n-m-l-1\right)}+\frac{1}{4\left(m-l-1\right)}+O\left(\frac{1}{n^{2}}\right)\right)\\
 & \leq\left(1+\frac{1}{4\left(n-m\right)}+\frac{1}{4\left(m-k\right)}+O\left(\frac{1}{n^{2}}\right)\right)^{k}<e
\end{align*}


Chebyshev inequality yields
\begin{align}
 & \text{ }\mathbb{P}\left(\frac{1}{n}\log\det\left({\bf A}_{s}^{T}\Xi_{\backslash s}^{-1}{\bf A}_{s}\right)\geq\frac{2}{n}\log\mathbb{E}\det\left({\bf A}_{s}^{T}\Xi_{\backslash s}^{-1}{\bf A}_{s}\right)^{\frac{1}{2}}+\frac{2\log\left(1+\sqrt{n}\right)}{n}\right)\nonumber \\
= & \text{ }\mathbb{P}\left(\det\left({\bf A}_{s}^{T}\Xi_{\backslash s}^{-1}{\bf A}_{s}\right)^{\frac{1}{2}}\geq(1+\sqrt{n})\mathbb{E}\det\left({\bf A}_{s}^{T}\Xi_{\backslash s}^{-1}{\bf A}_{s}\right)^{\frac{1}{2}}\right)\nonumber \\
\leq & \text{ }\frac{\text{{\bf Var}}\det\left({\bf A}_{s}^{T}\Xi_{\backslash s}^{-1}{\bf A}_{s}\right)^{\frac{1}{2}}}{n\left(\mathbb{E}\det\left({\bf A}_{s}^{T}\Xi_{\backslash s}^{-1}{\bf A}_{s}\right)^{\frac{1}{2}}\right)}<\frac{e}{n}\label{eq:LogDetUpperBound_A_s_W_A_s}
\end{align}


Similarly, \cite[Theorem 2.12]{TulinoVerdu2004} gives 
\[
\mathbb{E}\det\left({\bf A}_{s}^{T}\Xi_{\backslash s}^{-1}{\bf A}_{s}\right)^{-1}=\prod_{l=0}^{k-1}\frac{\Gamma\left(k+n-m-l+1\right)}{\Gamma\left(k+n-m-l\right)}\frac{\Gamma\left(m-l-1\right)}{\Gamma\left(m-l\right)},
\]
\begin{align*}
\mathbb{E}\det\left({\bf A}_{s}^{T}\Xi_{\backslash s}^{-1}{\bf A}_{s}\right)^{-\frac{1}{2}} & =\prod_{l=0}^{k-1}\frac{\Gamma\left(k+n-m-l+\frac{1}{2}\right)}{\Gamma\left(k+n-m-l\right)}\frac{\Gamma\left(m-l-\frac{1}{2}\right)}{\Gamma\left(m-l\right)},
\end{align*}
which allows us to bound 
\begin{align*}
\frac{\mathbb{E}\det\left({\bf A}_{s}^{T}\Xi_{\backslash s}^{-1}{\bf A}_{s}\right)^{-1}}{\left(\mathbb{E}\det\left({\bf A}_{s}^{T}\Xi_{\backslash s}^{-1}{\bf A}_{s}\right)^{-\frac{1}{2}}\right)^{2}} & =\prod_{l=0}^{k-1}\frac{\Gamma\left(k+n-m-l\right)\Gamma\left(k+n-m-l+1\right)}{\left[\Gamma\left(k+n-m-l+\frac{1}{2}\right)\right]^{2}}\frac{\Gamma\left(m-l\right)\Gamma\left(m-l-1\right)}{\left[\Gamma\left(m-l-\frac{1}{2}\right)\right]^{2}}\\
 & =\prod_{l=0}^{k-1}\left(1+\frac{1}{4\left(k+n-m-l\right)}+\frac{1}{4\left(m-l-1\right)}+O\left(\frac{1}{n^{2}}\right)\right)<e.
\end{align*}
By Chebyshev inequality we have
\begin{align}
 & \text{ }\mathbb{P}\left(\frac{1}{n}\log\det\left({\bf A}_{s}^{T}\Xi_{\backslash s}^{-1}{\bf A}_{s}\right)<-\frac{2}{n}\log\mathbb{E}\det\left({\bf A}_{s}^{T}\Xi_{\backslash s}^{-1}{\bf A}_{s}\right)^{-\frac{1}{2}}-\frac{2\log\left(1+\sqrt{n}\right)}{n}\right)\nonumber \\
= & \text{ }\mathbb{P}\left(\det\left({\bf A}_{s}^{T}\Xi_{\backslash s}^{-1}{\bf A}_{s}\right)^{-\frac{1}{2}}>(1+\sqrt{n})\mathbb{E}\det\left({\bf A}_{s}^{T}\Xi_{\backslash s}^{-1}{\bf A}_{s}\right)^{-\frac{1}{2}}\right)\nonumber \\
\leq & \text{ }\frac{\text{{\bf Var}}\det\left({\bf A}_{s}^{T}\Xi_{\backslash s}^{-1}{\bf A}_{s}\right)^{-\frac{1}{2}}}{n\left(\mathbb{E}\det\left({\bf A}_{s}^{T}\Xi_{\backslash s}^{-1}{\bf A}_{s}\right)^{-\frac{1}{2}}\right)^{2}}<\frac{e}{n}.\label{eq:LogDetLowerBound_A_s_W_A_s}
\end{align}
The next step is to evaluate $\mathbb{E}\det\left({\bf A}_{s}^{T}\Xi_{\backslash s}^{-1}{\bf A}_{s}\right)^{-\frac{1}{2}}$
and $\mathbb{E}\det\left({\bf A}_{s}^{T}\Xi_{\backslash s}^{-1}{\bf A}_{s}\right)^{\frac{1}{2}}$.
As we have derived above, 
\[
\sqrt{\mathbb{E}\det\left({\bf A}_{s}^{T}\Xi_{\backslash s}^{-1}{\bf A}_{s}\right)}\leq\mathbb{E}\det\left({\bf A}_{s}^{T}\Xi_{\backslash s}^{-1}{\bf A}_{s}\right)^{\frac{1}{2}}<\sqrt{e\mathbb{E}\det\left({\bf A}_{s}^{T}\Xi_{\backslash s}^{-1}{\bf A}_{s}\right)}.
\]
This along with (\ref{eq:ExpectedDet_A_s_W_A_s}) leads to
\begin{align}
\frac{1}{n}\log\mathbb{E}\det\left({\bf A}_{s}^{T}\Xi_{\backslash s}^{-1}{\bf A}_{s}\right)^{\frac{1}{2}} & =\frac{1}{2n}\log\frac{{m \choose k}}{{k+n-m-1 \choose k}}+O\left(\frac{1}{n}\right)\nonumber \\
 & =\frac{\alpha}{2}\log\left(\mathcal{H}\left(\frac{\beta}{\alpha}\right)\right)-\frac{1+\beta-\alpha}{2}\log\left(\mathcal{H}\left(\frac{\beta}{1+\beta-\alpha}\right)\right)+O\left(\frac{1}{n}\right).\label{eq:ExpectedDetAsWAs_half}
\end{align}
Similarly, we can derive
\begin{align}
-\frac{1}{n}\log\mathbb{E}\det\left({\bf A}_{s}^{T}\Xi_{\backslash s}^{-1}{\bf A}_{s}\right)^{-\frac{1}{2}} & =\frac{\alpha}{2}\log\left(\mathcal{H}\left(\frac{\beta}{\alpha}\right)\right)-\frac{1+\beta-\alpha}{2}\log\left(\mathcal{H}\left(\frac{\beta}{1+\beta-\alpha}\right)\right)+O\left(\frac{1}{n}\right).\label{eq:ExpectedDetAsWAs_minus_half}
\end{align}
Combining (\ref{eq:LogDetUpperBound_A_s_W_A_s}), (\ref{eq:LogDetLowerBound_A_s_W_A_s}),
(\ref{eq:ExpectedDetAsWAs_half}) and (\ref{eq:ExpectedDetAsWAs_minus_half})
yields: there is some constant $c>0$ such that
\[
\left|\frac{1}{n}\log\mathbb{E}\det\left({\bf A}_{s}^{T}\Xi_{\backslash s}^{-1}{\bf A}_{s}\right)-\alpha\log\left(\mathcal{H}\left(\frac{\beta}{\alpha}\right)\right)-\left(1+\beta-\alpha\right)\log\left(\mathcal{H}\left(\frac{\beta}{1+\beta-\alpha}\right)\right)\right|\leq\frac{c\log n}{n}
\]
with probability exceeding $1-2e/n$. 
\end{comment}


\begin{comment}
a


\section{Proof of Lemma @@}

Our goal is to evaluate
\begin{align*}
-G_{{\bf s}}^{{\bf Q}}\frac{2}{W} & \approx\frac{1}{n}\log\det\left(\epsilon{\bf I}+{\bf Q}_{{\bf s}}^{T}\left({\bf Q}{\bf Q}^{T}\right)^{-1}{\bf Q}_{{\bf s}}\right)\\
 & =\frac{1}{n}\log\left\{ \epsilon^{k-m}\det\left(\epsilon{\bf Q}{\bf Q}^{T}+{\bf Q}_{{\bf s}}{\bf Q}_{{\bf s}}^{T}\right)\det\left(\left({\bf Q}{\bf Q}^{T}\right)^{-1}\right)\right\} \\
 & =\frac{1}{n}\log\left\{ \epsilon^{k-m}\det\left(\epsilon{\bf Q}_{\backslash{\bf s}}{\bf Q}_{\backslash{\bf s}}^{T}+\left(1+\epsilon\right){\bf Q}_{{\bf s}}{\bf Q}_{{\bf s}}^{T}\right)\det\left(\left({\bf Q}{\bf Q}^{T}\right)^{-1}\right)\right\} \\
 & =\frac{1}{n}\log\left\{ \epsilon^{k-m}\det\left(\epsilon{\bf I}+\left(1+\epsilon\right)\left({\bf Q}_{\backslash{\bf s}}{\bf Q}_{\backslash{\bf s}}^{T}\right)^{-1}{\bf Q}_{{\bf s}}{\bf Q}_{{\bf s}}^{T}\right)\det\left({\bf Q}_{\backslash{\bf s}}{\bf Q}_{\backslash{\bf s}}^{T}\right)\det\left(\left({\bf Q}{\bf Q}^{T}\right)^{-1}\right)\right\} \\
 & =\frac{1}{n}\log\left\{ \det\left(\epsilon{\bf I}+\left(1+\epsilon\right){\bf Q}_{{\bf s}}^{T}\left({\bf Q}_{\backslash{\bf s}}{\bf Q}_{\backslash{\bf s}}^{T}\right)^{-1}{\bf Q}_{{\bf s}}\right)\det\left({\bf Q}_{\backslash{\bf s}}{\bf Q}_{\backslash{\bf s}}^{T}\right)\det\left(\left({\bf Q}{\bf Q}^{T}\right)^{-1}\right)\right\} \\
 & =\frac{1}{n}\log\det\left(\epsilon{\bf I}+\left(1+\epsilon\right)\frac{1}{n}{\bf Q}_{{\bf s}}^{T}\left(\frac{1}{n}{\bf Q}_{\backslash{\bf s}}{\bf Q}_{\backslash{\bf s}}^{T}\right)^{-1}{\bf Q}_{{\bf s}}\right)+\frac{1}{n}\log\det\left(\frac{1}{n}{\bf Q}_{\backslash{\bf s}}{\bf Q}_{\backslash{\bf s}}^{T}\right)-\frac{1}{n}\log\det\left(\frac{1}{n}{\bf Q}{\bf Q}^{T}\right)
\end{align*}
It is known that
\[
\frac{1}{n}\log\det\left(\frac{1}{n}{\bf Q}{\bf Q}^{T}\right)=\left(1-\alpha\right)\log\frac{1}{1-\alpha}-\alpha+O\left(\frac{\log n+\log\epsilon}{n}\right).
\]
\[
\frac{1}{n-k}\log\det\left(\frac{1}{n-k}{\bf Q}_{\backslash{\bf s}}{\bf Q}_{\backslash{\bf s}}^{T}\right)=\left(1-\frac{\alpha}{1-\beta}\right)\log\frac{1}{1-\frac{\alpha}{1-\beta}}-\frac{\alpha}{1-\beta}+O\left(\frac{\log n+\log\epsilon}{n}\right)
\]
\begin{align*}
\frac{1}{n}\log\det\left(\frac{1}{n}{\bf Q}_{\backslash{\bf s}}{\bf Q}_{\backslash{\bf s}}^{T}\right) & =\frac{n-k}{n}\left\{ \frac{1}{n-k}\log\det\left(\frac{1}{n-k}{\bf Q}_{\backslash{\bf s}}{\bf Q}_{\backslash{\bf s}}^{T}\right)\right\} +\frac{1}{n}\log\det\left(\frac{n-k}{n}{\bf I}\right)\\
 & =\left(1-\beta\right)\left\{ \left(1-\frac{\alpha}{1-\beta}\right)\log\frac{1}{1-\frac{\alpha}{1-\beta}}-\frac{\alpha}{1-\beta}\right\} +\alpha\log\left(1-\beta\right)\\
 & =\left(1-\alpha-\beta\right)\log\frac{1}{1-\frac{\alpha}{1-\beta}}-\alpha+\alpha\log\left(1-\beta\right)
\end{align*}
Hence, 
\[
\frac{1}{n}\log\det\left(\frac{1}{n}{\bf Q}_{\backslash{\bf s}}{\bf Q}_{\backslash{\bf s}}^{T}\right)-\frac{1}{n}\log\det\left(\frac{1}{n}{\bf Q}{\bf Q}^{T}\right)=\left(1-\alpha-\beta\right)\log\frac{1}{1-\frac{\alpha}{1-\beta}}+\alpha\log\left(1-\beta\right)-\left(1-\alpha\right)\log\left(1-\alpha\right)
\]
Next,
\[
\frac{\log\det\left(\epsilon{\bf I}+\left(1+\epsilon\right)\frac{1}{n}{\bf Q}_{{\bf s}}^{T}\left(\frac{1}{n}{\bf Q}_{\backslash{\bf s}}{\bf Q}_{\backslash{\bf s}}^{T}\right)^{-1}{\bf Q}_{{\bf s}}\right)}{\left(1+\epsilon\right)^{k}}\approx\log\det\left({\bf Q}_{{\bf s}}^{T}\left({\bf Q}_{\backslash{\bf s}}{\bf Q}_{\backslash{\bf s}}^{T}\right)^{-1}{\bf Q}_{{\bf s}}\right)
\]
where ${\bf Q}_{{\bf s}}$ and ${\bf Q}_{\backslash{\bf s}}$ are
jointly independent. Let $\Sigma_{\backslash{\bf s}}$ be the eigen
matrix of ${\bf Q}_{\backslash{\bf s}}{\bf Q}_{\backslash{\bf s}}^{T}$
and 
\begin{align*}
\mathbb{E}\det\left(\frac{1}{n}{\bf Q}_{{\bf s}}^{T}\left(\frac{1}{n}{\bf Q}_{\backslash{\bf s}}{\bf Q}_{\backslash{\bf s}}^{T}\right)^{-1}{\bf Q}_{{\bf s}}\right) & =\mathbb{E}\det\left({\bf Q}_{{\bf s}}^{T}\left({\bf Q}_{\backslash{\bf s}}{\bf Q}_{\backslash{\bf s}}^{T}\right)^{-1}{\bf Q}_{{\bf s}}\right)\\
 & =\frac{1}{n^{k}}\sum_{{\bf r}\in{[m] \choose k}}\mathbb{E}\det\left({\bf Q}_{{\bf r},{\bf s}}^{T}\left(\Sigma_{\backslash{\bf s}}^{-1}\right)_{{\bf r}}{\bf Q}_{{\bf r},{\bf s}}\right)\\
 & =\frac{k!}{n^{k}}\left\{ \sum_{{\bf r}\in{[m] \choose k}}\det\left(\Sigma_{\backslash{\bf s}}^{-1}\right)_{{\bf r}}\right\} \\
 & =\frac{k!}{n^{k}}\frac{\left\{ \sum_{\tilde{{\bf r}}\in{[m] \choose m-k}}\det\left(\Sigma_{\backslash{\bf s}}\right)_{\tilde{{\bf r}}}\right\} }{\det\left(\Sigma_{\backslash{\bf s}}\right)}\\
 & \geq\frac{k!}{n^{k}}\frac{{m \choose k}\left\{ \prod_{\tilde{{\bf r}}\in{[m] \choose m-k}}\det\left(\Sigma_{\backslash{\bf s}}\right)_{\tilde{{\bf r}}}\right\} ^{\frac{1}{{m \choose k}}}}{\det\left(\Sigma_{\backslash{\bf s}}\right)}\\
 & =\frac{k!}{n^{k}}\frac{{m \choose k}\left\{ \det\left(\Sigma_{\backslash{\bf s}}\right)\right\} ^{\frac{{m-1 \choose k-1}}{{m \choose k}}}}{\det\left(\Sigma_{\backslash{\bf s}}\right)}=\frac{k!}{n^{k}}{m \choose k}\det\left(\Sigma_{\backslash{\bf s}}\right)^{\beta-1}\\
 & =\frac{k!{m \choose k}}{n^{k}}
\end{align*}


The task now is to make sure that $\log\det\Sigma_{\backslash{\bf s}}\approx0$.
We know that for a given large integer $K$,
\begin{align*}
\frac{1}{n}\log\det\left(\frac{1}{K}{\bf Q}_{\backslash{\bf s}}{\bf Q}_{\backslash{\bf s}}^{T}\right) & =\frac{1}{n}\log\det\left(\frac{1}{n-k}{\bf Q}_{\backslash{\bf s}}{\bf Q}_{\backslash{\bf s}}^{T}\right)+\frac{m}{n}\log\frac{n-k}{K}\\
 & =\frac{n-k}{n}\frac{1}{n-k}\log\det\left(\frac{1}{n-k}{\bf Q}_{\backslash{\bf s}}{\bf Q}_{\backslash{\bf s}}^{T}\right)+\frac{m}{n}\log\frac{n-k}{K}\\
 & =\left(1-\beta\right)\left[-\left(1-\frac{\alpha}{1-\beta}\right)\log\left(1-\frac{\alpha}{1-\beta}\right)-\frac{\alpha}{1-\beta}\right]+\frac{m}{n}\log\frac{n-k}{K}\\
 & =-\left(1-\beta-\alpha\right)\log\left(1-\frac{\alpha}{1-\beta}\right)-\alpha+\alpha\log\frac{n-k}{K}\\
 & =0
\end{align*}
Then
\[
\left(\frac{1-\beta}{\alpha}-1\right)\log\left(1-\frac{\alpha}{1-\beta}\right)+1=\log\frac{n-k}{K}=\log\frac{n-k}{n}+\log\frac{n}{K}=\log\left(1-\beta\right)+\log\frac{n}{K}
\]
Therefore,
\begin{align*}
\frac{1}{n}\log\mathbb{E}\det\left(\frac{1}{K}{\bf Q}_{{\bf s}}^{T}\left(\frac{1}{K}{\bf Q}_{\backslash{\bf s}}{\bf Q}_{\backslash{\bf s}}^{T}\right)^{-1}{\bf Q}_{{\bf s}}\right) & \approx\frac{1}{n}\log\frac{k!{m \choose k}}{K^{k}}=\frac{1}{n}\log\frac{k!{m \choose k}}{n^{k}}+\frac{1}{n}\log\frac{n^{k}}{K^{k}}\\
 & =\frac{k\log k-k-k\log n+m\mathcal{H}\left(\frac{\beta}{\alpha}\right)}{n}+\beta\log\frac{n}{K}\\
 & =\beta\log\beta+\alpha\mathcal{H}\left(\frac{\beta}{\alpha}\right)-\beta+\beta\left(\frac{1-\beta}{\alpha}-1\right)\log\left(1-\frac{\alpha}{1-\beta}\right)+\beta-\beta\log\left(1-\beta\right)\\
 & =\beta\log\beta+\alpha\mathcal{H}\left(\frac{\beta}{\alpha}\right)+\frac{\beta}{\alpha}\left(1-\beta-\alpha\right)\log\left(1-\frac{\alpha}{1-\beta}\right)-\beta\log\left(1-\beta\right)
\end{align*}


Therefore,
\begin{align*}
 & \frac{1}{n}\log\det\left({\bf Q}_{{\bf s}}^{T}\left({\bf Q}_{\backslash{\bf s}}{\bf Q}_{\backslash{\bf s}}^{T}\right)^{-1}{\bf Q}_{{\bf s}}\right)+\frac{1}{n}\log\det\left(\frac{1}{n}{\bf Q}_{\backslash{\bf s}}{\bf Q}_{\backslash{\bf s}}^{T}\right)-\frac{1}{n}\log\det\left(\frac{1}{n}{\bf Q}{\bf Q}^{T}\right)\\
= & \beta\log\beta+\alpha\mathcal{H}\left(\frac{\beta}{\alpha}\right)+\frac{\beta}{\alpha}\left(1-\beta-\alpha\right)\log\left(1-\frac{\alpha}{1-\beta}\right)-\beta\log\left(1-\beta\right)+\\
 & \quad\quad\left(1-\alpha-\beta\right)\log\frac{1}{1-\frac{\alpha}{1-\beta}}+\alpha\log\left(1-\beta\right)-\left(1-\alpha\right)\log\left(1-\alpha\right)
\end{align*}
\begin{align*}
\mathbb{E}\det\left(\frac{1}{n}{\bf Q}_{{\bf s}}^{T}\left(\frac{1}{n}{\bf Q}_{\backslash{\bf s}}{\bf Q}_{\backslash{\bf s}}^{T}\right)^{-1}{\bf Q}_{{\bf s}}\right) & =\mathbb{E}\det\left({\bf Q}_{{\bf s}}^{T}\left({\bf Q}_{\backslash{\bf s}}{\bf Q}_{\backslash{\bf s}}^{T}\right)^{-1}{\bf Q}_{{\bf s}}\right)\\
 & =\frac{1}{n^{k}}\sum_{{\bf r}\in{[m] \choose k}}\mathbb{E}\det\left({\bf Q}_{{\bf r},{\bf s}}^{T}\left(\Sigma_{\backslash{\bf s}}^{-1}\right)_{{\bf r}}{\bf Q}_{{\bf r},{\bf s}}\right)\\
 & =\frac{k!}{n^{k}}\left\{ \sum_{{\bf r}\in{[m] \choose k}}\det\left(\Sigma_{\backslash{\bf s}}^{-1}\right)_{{\bf r}}\right\} \\
 & =\frac{k!}{n^{k}}\frac{\left\{ \sum_{\tilde{{\bf r}}\in{[m] \choose m-k}}\det\left(\Sigma_{\backslash{\bf s}}\right)_{\tilde{{\bf r}}}\right\} }{\det\left(\Sigma_{\backslash{\bf s}}\right)}\\
 & \geq\frac{k!}{n^{k}}\frac{{m \choose k}\left\{ \prod_{\tilde{{\bf r}}\in{[m] \choose m-k}}\det\left(\Sigma_{\backslash{\bf s}}\right)_{\tilde{{\bf r}}}\right\} ^{\frac{1}{{m \choose k}}}}{\det\left(\Sigma_{\backslash{\bf s}}\right)}\\
 & =\frac{k!}{n^{k}}\frac{{m \choose k}\left\{ \det\left(\Sigma_{\backslash{\bf s}}\right)\right\} ^{\frac{{m-1 \choose m-k-1}}{{m \choose k}}}}{\det\left(\Sigma_{\backslash{\bf s}}\right)}=\frac{k!}{n^{k}}{m \choose k}\frac{1}{\det\left(\frac{1}{n}\Sigma_{\backslash{\bf s}}\right)^{\frac{\beta}{\alpha}}}\\
 & =\frac{k!{m \choose k}}{n^{k}}\frac{1}{\det\left(\frac{1}{n}\Sigma_{\backslash{\bf s}}\right)^{\frac{\beta}{\alpha}}}
\end{align*}
Hence

\begin{align*}
\frac{1}{n}\log\mathbb{E}\det\left(\frac{1}{n}{\bf Q}_{{\bf s}}^{T}\left(\frac{1}{n}{\bf Q}_{\backslash{\bf s}}{\bf Q}_{\backslash{\bf s}}^{T}\right)^{-1}{\bf Q}_{{\bf s}}\right) & \geq\frac{k\log k-k-k\log n+m\mathcal{H}\left(\frac{k}{m}\right)}{n}-\frac{\beta}{\alpha}\frac{1}{n}\log\det\left(\frac{1}{n}\Sigma_{\backslash{\bf s}}\right)\\
 & =\beta\log\beta-\beta+\alpha\mathcal{H}\left(\frac{k}{m}\right)-\frac{\beta}{\alpha}\frac{1}{n}\log\det\left(\frac{1}{n}\Sigma_{\backslash{\bf s}}\right)
\end{align*}


Therefore, 

\begin{align*}
 & \frac{1}{n}\log\det\left({\bf Q}_{{\bf s}}^{T}\left({\bf Q}_{\backslash{\bf s}}{\bf Q}_{\backslash{\bf s}}^{T}\right)^{-1}{\bf Q}_{{\bf s}}\right)+\frac{1}{n}\log\det\left(\frac{1}{n}{\bf Q}_{\backslash{\bf s}}{\bf Q}_{\backslash{\bf s}}^{T}\right)-\frac{1}{n}\log\det\left(\frac{1}{n}{\bf Q}{\bf Q}^{T}\right)\\
\geq & \beta\log\beta-\beta+\alpha\mathcal{H}\left(\frac{\beta}{\alpha}\right)+\frac{\alpha-\beta}{\alpha}\frac{1}{n}\log\det\left(\frac{1}{n}\Sigma_{\backslash{\bf s}}\right)-\frac{1}{n}\log\det\left(\frac{1}{n}{\bf Q}{\bf Q}^{T}\right)\\
= & \beta\log\beta-\beta+\alpha\mathcal{H}\left(\frac{\beta}{\alpha}\right)+\frac{\alpha-\beta}{\alpha}\left\{ \left(1-\alpha-\beta\right)\log\frac{1}{1-\frac{\alpha}{1-\beta}}-\alpha+\alpha\log\left(1-\beta\right)\right\} -\left(1-\alpha\right)\log\left(1-\alpha\right)+\alpha\\
= & \beta\log\beta+\alpha\mathcal{H}\left(\frac{\beta}{\alpha}\right)+\frac{\alpha-\beta}{\alpha}\left\{ \left(1-\alpha-\beta\right)\log\frac{1}{1-\frac{\alpha}{1-\beta}}\right\} +\left(\alpha-\beta\right)\log\left(1-\beta\right)-\left(1-\alpha\right)\log\left(1-\alpha\right)\\
= & -\mathcal{H}(\beta)+\alpha\mathcal{H}\left(\frac{\beta}{\alpha}\right)+\frac{\alpha-\beta}{\alpha}\left\{ \left(1-\alpha-\beta\right)\log\frac{1-\beta}{1-\alpha-\beta}+\alpha\log\left(1-\beta\right)\right\} -\left(1-\alpha\right)\log\left(1-\alpha\right)\\
= & -\mathcal{H}(\beta)+\alpha\mathcal{H}\left(\frac{\beta}{\alpha}\right)+\frac{\alpha-\beta}{\alpha}\left\{ \left(1-\beta\right)\log\left(1-\beta\right)-\left(1-\alpha-\beta\right)\log\left(1-\alpha-\beta\right)\right\} -\left(1-\alpha\right)\log\left(1-\alpha\right)
\end{align*}
\end{comment}


\begin{comment}
@@@


\section{Proof of Lemma \ref{lemma-Trace-As_W_As}\label{sec:Proof-of-lemma-Trace-As_W_As}}

(1)\cite[Lemma 2.10]{TulinoVerdu2004} shows that
\[
\mathbb{E}\text{tr}\left({\bf W}^{-1}\right)=\frac{m}{n-m},\quad\mathbb{E}\text{tr}^{2}\left({\bf W}^{-1}\right)=\frac{m}{n-m}\left(\frac{m-1}{n-m+1}+\frac{n}{\left(n-m\right)^{2}-1}\right).
\]
Hence, if $n-m\gg1$, then there exists some constant $c>0$ such
that 
\begin{align*}
{\bf Var}\left(\text{tr}\left({\bf W}^{-1}\right)\right) & =\mathbb{E}\text{tr}^{2}\left({\bf W}^{-1}\right)-\left(\mathbb{E}\text{tr}\left({\bf W}^{-1}\right)\right)^{2}\leq\frac{c}{n-m}.
\end{align*}
Therefore,
\[
\mathbb{P}\left(\left|\frac{\text{tr}\left({\bf W}^{-1}\right)-\frac{m}{n-m}}{\frac{m}{n-m}}\right|\geq\epsilon\right)\leq\frac{{\bf Var}\left(\text{tr}\left({\bf W}^{-1}\right)\right)}{\left(\mathbb{E}\text{tr}\left({\bf W}^{-1}\right)\right)^{2}}=\frac{c}{\epsilon^{2}\left(n-m\right)}\left(\frac{n-m}{m}\right)^{2}.
\]
which completes the proof of Part (1).

(2) Note that $\text{tr}\left({\bf A}^{T}{\bf B}{\bf {\bf A}}\right)=\text{tr}\left({\bf B}{\bf A}{\bf A}^{T}\right)$,
where ${\bf A}{\bf A}^{T}\sim\mathcal{W}_{m}(k,{\bf I}_{m})$. \cite[Theorem 2.2.6]{Fujikoshi2010}
implies that
\[
\mathbb{E}\text{tr}\left({\bf A}^{T}{\bf B}{\bf {\bf A}}\right)=k\text{tr}{\bf B},\quad\mathbb{E}\left[\left(\text{tr}\left({\bf A}^{T}{\bf B}{\bf {\bf A}}\right)\right)^{2}\right]=k\text{tr}{\bf B}+k\text{tr}\left({\bf B}^{2}\right)+k^{2}\left(\text{tr}{\bf B}\right)^{2},
\]
which immediately yields
\[
{\bf Var}\left(\text{tr}\left({\bf B}{\bf W}\right)\right)=\mathbb{E}\left[\left(\text{tr}\left({\bf B}{\bf W}\right)\right)^{2}\right]-\left(\mathbb{E}\text{tr}\left({\bf B}{\bf W}\right)\right)^{2}=k\left(\text{tr}{\bf B}+\text{tr}\left({\bf B}^{2}\right)\right).
\]
For any given $\epsilon>0$, applying the Chebyshev inequality yields
\begin{align*}
\mathbb{P}\left(\left|\frac{\text{tr}\left({\bf B}{\bf W}\right)-k\text{tr}{\bf B}}{k\text{tr}{\bf B}}\right|>\epsilon\right) & \leq\frac{{\bf Var}\left(\text{tr}\left({\bf B}{\bf W}\right)\right)}{\epsilon^{2}\left(\mathbb{E}\text{tr}\left({\bf B}{\bf W}\right)\right)^{2}}=\frac{\text{tr}{\bf B}+\text{tr}\left({\bf B}^{2}\right)}{\epsilon^{2}\left(\text{tr}{\bf B}\right)^{2}}\frac{1}{k}\\
 & \leq\frac{\text{tr}{\bf B}+\left(\text{tr}{\bf B}\right)^{2}}{\epsilon^{2}\left(\text{tr}{\bf B}\right)^{2}}\frac{1}{k}=\frac{1}{\epsilon^{2}}\left(1+\frac{1}{\text{tr}{\bf B}}\right)\frac{1}{k}.
\end{align*}


(3) When $\text{tr}\left({\bf W}^{-1}\right)\in\left[\left(1-\epsilon\right)\frac{\alpha}{1-\alpha},\left(1+\epsilon\right)\frac{\alpha}{1-\alpha}\right]$,
we have $1+\frac{1}{\text{tr}{\bf B}}\leq1+\frac{1-\alpha}{\left(1-\epsilon\right)\alpha}$.
By combining results (1) and (2), we can express
\begin{align*}
 & \text{ }\mathbb{P}\left(\left|\frac{\text{tr}\left({\bf A}^{T}{\bf W}^{-1}{\bf A}\right)}{k\frac{\alpha}{1-\alpha}}\right|\in\left[\left(1-\epsilon\right)^{2},\left(1+\epsilon\right)^{2}\right]\right)\\
\leq\text{ } & \mathbb{P}\left(\text{ }\frac{\text{tr}\left({\bf W}^{-1}\right)}{\frac{\alpha}{1-\alpha}}\notin\left[1-\epsilon,1+\epsilon\right]\right)+\mathbb{P}\left(\left|\frac{\text{tr}\left({\bf A}^{T}{\bf W}^{-1}{\bf A}\right)}{k\frac{\alpha}{1-\alpha}}\right|\in\left[\left(1-\epsilon\right)^{2},\left(1+\epsilon\right)^{2}\right]\left|\text{ }\frac{\text{tr}\left({\bf W}^{-1}\right)}{\frac{\alpha}{1-\alpha}}\in\left[1-\epsilon,1+\epsilon\right]\right.\right)\\
\leq\text{ } & \frac{c_{1}}{\epsilon^{2}}\frac{1-\alpha}{\alpha^{2}}\frac{1}{n}+\frac{1}{\epsilon^{2}}\left(1+\frac{1-\alpha}{\left(1-\epsilon\right)\alpha}\right)\frac{1}{k},
\end{align*}
which completes the proof.
\end{comment}


\begin{comment}
a


\section{Proof of Lemma \ref{Lemma-Concentration-Upper-Bound}\label{sec:Proof-of-Lemma-Concentration-Upper-Bound}}

(1) Since $\zeta_{i}$ are of zero mean and unit variance, we have
\[
\mathbb{E}\left(\sum_{i=1}^{n}c_{i}^{2}\zeta_{i}^{2}\right)=\sum_{i=1}^{n}c_{i}^{2}.
\]
Therefore,
\begin{align*}
\text{Var}\left(\sum_{i=1}^{n}c_{i}^{2}\zeta_{i}^{2}\right) & =\mathbb{E}\left(\sum_{i=1}^{n}c_{i}^{2}\left(\zeta_{i}^{2}-1\right)\right)^{2}=\sum_{i=1}^{n}c_{i}^{4}\mathbb{E}\left(\zeta_{i}^{2}-1\right)^{2}\leq\sum_{i=1}^{n}c_{i}^{4}\left(\mathbb{E}\zeta_{i}^{4}-1\right)\\
 & \leq D\sum_{i=1}^{n}c_{i}^{4}.
\end{align*}
By Chebyshev inequality,
\begin{align*}
\mathbb{P}\left(\left|\sum_{i=1}^{n}c_{i}^{2}\zeta_{i}^{2}-\mathbb{E}\left(\sum_{i=1}^{n}c_{i}^{2}\zeta_{i}^{2}\right)\right|\geq\epsilon\mathbb{E}\left(\sum_{i=1}^{n}c_{i}^{2}\zeta_{i}^{2}\right)\right) & \leq\frac{\text{Var}\left(\sum_{i=1}^{n}c_{i}^{2}\zeta_{i}^{2}\right)}{\epsilon^{2}\left[\mathbb{E}\left(\sum_{i=1}^{n}c_{i}^{2}\zeta_{i}^{2}\right)\right]^{2}}.\\
 & \leq\frac{\sum_{i=1}^{n}c_{i}^{4}}{\epsilon^{2}\left(\sum_{i=1}^{n}c_{i}^{2}\right)^{2}}\leq\frac{c_{\text{max}}^{4}}{\epsilon^{2}c_{\min}^{2}}\frac{1}{N}.
\end{align*}
For sufficiently large $N$, the above probability vanishes, which
implies that 
\[
\text{median}\left(\sum_{i=1}^{n}c_{i}^{2}\zeta_{i}^{2}\right)\in\left[\left(1-\epsilon\right)\mathbb{E}\left(\sum_{i=1}^{n}c_{i}^{2}\zeta_{i}^{2}\right),\left(1+\epsilon\right)\mathbb{E}\left(\sum_{i=1}^{n}c_{i}^{2}\zeta_{i}^{2}\right)\right],
\]
\[
\text{i.e., }\text{median}\left(\sqrt{\sum_{i=1}^{n}c_{i}^{2}\zeta_{i}^{2}}\right)\in\left[\left(1-\epsilon\right)\sqrt{\sum_{i=1}^{n}c_{i}^{2}},\left(1+\epsilon\right)\sqrt{\sum_{i=1}^{n}c_{i}^{2}}\right].
\]


Define a $ $1-Lipschitz function $F(\cdot)$ (i.e. $\left|F({\bf x})-F({\bf y})\right|\leq\left\Vert {\bf x}-{\bf y}\right\Vert _{2}$
for all ${\bf x},{\bf y}\in\mathbb{R}^{n}$) as 
\[
F(\zeta_{1},\cdots,\zeta_{n})=\frac{\sqrt{\sum_{j=1}^{n}\left|c_{j}\zeta_{ij}\right|^{2}}}{c_{\max}}.
\]
 If $\zeta_{i}$ is bounded by some constant $D$, then Talagrand
concentration inequality $ $\cite[Theorem 2.1.13]{Tao2012RMT} yields
\begin{align*}
\mathbb{P}\left(\frac{\left|\sqrt{\sum_{i=1}^{n}c_{i}^{2}\zeta_{i}^{2}}-\sqrt{\sum_{i=1}^{n}c_{i}^{2}}\right|}{\sqrt{\sum_{i=1}^{n}c_{i}^{2}}}\geq2\epsilon\right) & \leq\mathbb{P}\left(\frac{\left|\sqrt{\sum_{i=1}^{n}c_{i}^{2}\zeta_{i}^{2}}-\text{median}\left(\sqrt{\sum_{i=1}^{n}c_{i}^{2}\zeta_{i}^{2}}\right)\right|}{\sqrt{\sum_{i=1}^{n}c_{i}^{2}}}\geq\epsilon\right)\\
 & \leq\mathbb{P}\left(\left|F(\zeta_{1},\cdots,\zeta_{n})-\text{median}\left(F(\zeta_{1},\cdots,\zeta_{n})\right)\right|\geq\epsilon c_{\max}\sqrt{\sum_{i=1}^{n}c_{i}^{2}}\right)\\
 & \leq\mathbb{P}\left(\left|F\left(\zeta_{1},\cdots,\zeta_{n}\right)-\text{median}F\left(\zeta_{1},\cdots,\zeta_{n}\right)\right|\geq\epsilon c_{\max}c_{\min}\sqrt{n}\right)\\
 & \leq C\exp\left(-\frac{c\epsilon^{2}c_{\max}^{2}c_{\min}^{2}}{D^{2}}n\right).
\end{align*}
If $\zeta_{i}$ is Gaussian random variable, then applying similar
steps with the standard Gaussian concentration inequality \cite[Theorem 3.4]{MassartPicard2007}
yields
\begin{align*}
\mathbb{P}\left(\frac{\left|\sqrt{\sum_{i=1}^{n}c_{i}^{2}\zeta_{i}^{2}}-\sqrt{\sum_{i=1}^{n}c_{i}^{2}}\right|}{\sqrt{\sum_{i=1}^{n}c_{i}^{2}}}\geq2\epsilon\right) & \leq\mathbb{P}\left(\left|F(\zeta_{1},\cdots,\zeta_{n})-\text{median}\left(F(\zeta_{1},\cdots,\zeta_{n})\right)\right|\geq\epsilon c_{\max}c_{\min}\sqrt{n}\right)\\
 & \leq2\exp\left(-\frac{\epsilon^{2}c_{\max}^{2}c_{\min}^{2}n}{2}\right).
\end{align*}


(2) Using the union bound and the result shown in Part (1) yields
\begin{align*}
 & \mathbb{P}\left(\left\Vert {\bf W}\right\Vert _{\text{F}}^{2}\notin\left[(1-2\epsilon)^{2}k\sum_{j=1}^{n}c_{j}^{2},(1+2\epsilon)^{2}k\sum_{j=1}^{n}c_{j}^{2}\right]\right)\\
\leq & \sum_{j=1}^{n}\mathbb{P}\left(\left\Vert {\bf W}_{*j}\right\Vert _{2}\notin\left[(1-2\epsilon)\sqrt{kc_{j}^{2}},(1+2\epsilon)\sqrt{kc_{j}^{2}}\right]\right)\\
\leq & \text{ }C\exp\left(-\tilde{c}c_{j}^{4}\epsilon^{2}n\right)
\end{align*}
for some constants $C,\tilde{c}>0$. Therefore, one has 
\begin{align*}
\frac{1}{n}\log\det\left({\bf W}{\bf W}^{*}\right) & =\frac{1}{n}\log\prod_{i=1}^{k}\lambda_{i}\left({\bf W}{\bf W}^{*}\right)\leq\frac{1}{n}\log\left(\frac{\sum_{i=1}^{k}\lambda_{i}\left({\bf W}{\bf W}^{*}\right)}{k}\right)^{k}\\
 & =\frac{1}{n}\log\left(\frac{\left\Vert {\bf W}\right\Vert _{\text{F}}^{2}}{k}\right)^{k}\leq\beta\log\frac{(1+2\epsilon)^{2}k\sum_{j=1}^{n}c_{j}^{2}}{k}\\
 & =\beta\log(1+2\epsilon)^{2}+\beta\log\left(\sum_{j=1}^{n}c_{j}^{2}\right)
\end{align*}
with probability exceeding $1-C\exp\left(-\tilde{c}c_{j}^{4}\epsilon^{2}n\right)$.
\end{comment}


\begin{comment}
Let $a_{k}\overset{\Delta}{=}\left[\det\left(C_{[k]}C_{[k]}^{T}\right)\right]^{1/k}$,
then 
\[
a_{k}=\left[\det\left(C_{[k]}C_{[k]}^{T}\right)\right]^{1/k}\leq\frac{1}{k}\left\Vert C_{[k]}\right\Vert _{\text{F}}^{2}
\]
 Therefore, our goal is to maximize 
\[
\prod_{[k]}a_{k}^{kp_{k}}
\]
 or equivalently
\[
\sum_{[k]}kp_{k}\log a_{k}
\]
 This can be upper bounded by 
\begin{align*}
\max\quad & \sum_{[k]}kp_{k}\log a_{k}\\
\text{subject to}\quad & \sum_{[k]}a_{k}\leq1
\end{align*}
 Since this is a convex program, the Lagrangian is
\[
L\left(\left\{ a_{k}\right\} ,\lambda\right)=\sum_{[k]}kp_{k}\log a_{k}+\lambda\left(1-\sum_{[k]}a_{k}\right)
\]
 By taking derivatives with respect to $a_{k}$, we have
\[
\frac{kp_{k}}{a_{k}}-\lambda=0
\]
 which immediately yields
\[
\frac{a_{1}}{p_{1}}=\frac{a_{2}}{p_{2}}=\cdots=\frac{a_{k}}{p_{k}}=\cdots
\]
 
\[
\Longleftrightarrow\quad a_{1}=p_{1},\quad a_{2}=p_{2},\quad,\cdots
\]
 This yields that 
\[
\det\left(C_{[k]}C_{[k]}^{T}\right)=p_{1}^{k}
\]
 
\end{comment}


\bibliographystyle{IEEEtran} \bibliographystyle{IEEEtran} \bibliographystyle{IEEEtran}
\bibliographystyle{IEEEtran}
\bibliography{bibfileMinimax}

\end{document}

MinimaxSampling.bbl
% Generated by IEEEtran.bst, version: 1.13 (2008/09/30)
\begin{thebibliography}{10}
\providecommand{\url}[1]{#1}
\csname url@samestyle\endcsname
\providecommand{\newblock}{\relax}
\providecommand{\bibinfo}[2]{#2}
\providecommand{\BIBentrySTDinterwordspacing}{\spaceskip=0pt\relax}
\providecommand{\BIBentryALTinterwordstretchfactor}{4}
\providecommand{\BIBentryALTinterwordspacing}{\spaceskip=\fontdimen2\font plus
\BIBentryALTinterwordstretchfactor\fontdimen3\font minus
  \fontdimen4\font\relax}
\providecommand{\BIBforeignlanguage}[2]{{%
\expandafter\ifx\csname l@#1\endcsname\relax
\typeout{** WARNING: IEEEtran.bst: No hyphenation pattern has been}%
\typeout{** loaded for the language `#1'. Using the pattern for}%
\typeout{** the default language instead.}%
\else
\language=\csname l@#1\endcsname
\fi
#2}}
\providecommand{\BIBdecl}{\relax}
\BIBdecl

\bibitem{ChenEldarGoldsmith2012}
Y.~Chen, Y.~C. Eldar, and A.~J. Goldsmith, ``Channel capacity under
  sub-{N}yquist nonuniform sampling,'' \emph{IEEE Transactions on Information
  Theory}, vol.~60, no.~8, August 2014.

\bibitem{ButSte1992}
P.~L. Butzer and R.~L. Stens, ``Sampling theory for not necessarily
  band-limited functions: A historical overview,'' \emph{SIAM Review}, vol.~34,
  no.~1, pp. 40--53, 1992.

\bibitem{MisEld2011}
M.~Mishali and Y.~C. Eldar, ``Sub-{N}yquist sampling: Bridging theory and
  practice,'' \emph{IEEE Signal Processing Magazine}, vol.~28, no.~6, pp.
  98--124, Nov. 2011.

\bibitem{Landau1967}
H.~Landau, ``Necessary density conditions for sampling and interpolation of
  certain entire functions,'' \emph{Acta Mathematica}, vol. 117, pp. 37--52,
  1967.

\bibitem{HerleyWong1999}
C.~Herley and P.~W. Wong, ``Minimum rate sampling and reconstruction of signals
  with arbitrary frequency support,'' \emph{IEEE Transactions on Information
  Theory}, vol.~45, no.~5, pp. 1555 --1564, Jul. 1999.

\bibitem{VenkataramaniBresler2001}
R.~Venkataramani and Y.~Bresler, ``Optimal sub-{N}yquist nonuniform sampling
  and reconstruction for multiband signals,'' \emph{IEEE Transactions on Signal
  Processing}, vol.~49, no.~10, pp. 2301 --2313, Oct. 2001.

\bibitem{CandRomTao06}
E.~Candes, J.~Romberg, and T.~Tao, ``Robust uncertainty principles: exact
  signal reconstruction from highly incomplete frequency information,''
  \emph{IEEE Transactions on Information Theory}, vol.~52, no.~2, pp. 489--509,
  Feb. 2006.

\bibitem{Don2006}
D.~L. Donoho, ``Compressed sensing,'' \emph{IEEE Transactions on Information
  Theory}, vol.~52, no.~4, pp. 1289 --1306, April 2006.

\bibitem{eldar2012compressed}
Y.~C. Eldar and G.~Kutyniok, \emph{Compressed Sensing: Theory and
  Applications}.\hskip 1em plus 0.5em minus 0.4em\relax Cambridge University
  Press Cambridge, 2012, vol.~95.

\bibitem{MisEld2010Theory2Practice}
M.~Mishali and Y.~C. Eldar, ``From theory to practice: Sub-{N}yquist sampling
  of sparse wideband analog signals,'' \emph{IEEE Journal of Selected Topics in
  Signal Processing}, vol.~4, no.~2, pp. 375 --391, Apr. 2010.

\bibitem{BluDraVetMarCou2008}
T.~Blu, P.-L. Dragotti, M.~Vetterli, P.~Marziliano, and L.~Coulot, ``Sparse
  sampling of signal innovations,'' \emph{IEEE Signal Processing Magazine},
  vol.~25, no.~2, pp. 31 --40, Mar. 2008.

\bibitem{GedTurEld2010}
K.~Gedalyahu, R.~Tur, and Y.~Eldar, ``Multichannel sampling of pulse streams at
  the rate of innovation,'' \emph{IEEE Transactions on Signal Processing},
  vol.~59, no.~4, pp. 1491 --1504, April 2011.

\bibitem{Med2000}
M.~Medard, ``The effect upon channel capacity in wireless communications of
  perfect and imperfect knowledge of the channel,'' \emph{IEEE Trans. on Info
  Theory}, vol.~46, no.~3, pp. 933 --946, May. 2000.

\bibitem{MedGal2002}
M.~Medard and R.~G. Gallager, ``Bandwidth scaling for fading multipath
  channels,'' \emph{IEEE Transactions on Information Theory}, vol.~48, no.~4,
  pp. 840 --852, Apr. 2002.

\bibitem{Bello1963}
P.~Bello, ``Characterization of randomly time-variant linear channels,''
  \emph{IEEE Transactions on Communications Systems}, vol.~11, no.~4, pp. 360
  --393, Dec. 1963.

\bibitem{BiglieriProakisShamai1998}
E.~Biglieri, J.~Proakis, and S.~Shamai, ``Fading channels:
  information-theoretic and communications aspects,'' \emph{IEEE Transactions
  on Information Theory}, vol.~44, no.~6, pp. 2619 --2692, Oct 1998.

\bibitem{ForUng1998}
G.~D. Forney and G.~Ungerboeck, ``Modulation and coding for linear {G}aussian
  channels,'' \emph{IEEE Transactions on Information Theory}, vol.~44, no.~6,
  pp. 2384 --2415, Oct 1998.

\bibitem{Sha1994}
S.~Shamai, ``Information rates by oversampling the sign of a bandlimited
  process,'' \emph{IEEE Transactions on Information Theory}, vol.~40, no.~4,
  pp. 1230 --1236, Jul. 1994.

\bibitem{KochLap2010}
\BIBentryALTinterwordspacing
T.~Koch and A.~Lapidoth, ``Increased capacity per unit-cost by oversampling,''
  Sep. 2010. [Online]. Available: \url{http://arxiv.org/abs/1008.5393}
\BIBentrySTDinterwordspacing

\bibitem{PelegShamai}
M.~Peleg and S.~Shamai, ``On sparse sensing of coded signals at sub-{L}andau
  sampling rates,'' \emph{IEEE 27th Convention of Electrical Electronics
  Engineers in Israel (IEEEI)}, pp. 1 --5, Nov. 2012.

\bibitem{ChenGolEld2010}
Y.~Chen, Y.~C. Eldar, and A.~J. Goldsmith, ``Shannon meets {N}yquist:
  {C}apacity of sampled {G}aussian channels,'' \emph{IEEE Transactions on
  Information Theory}, vol.~59, no.~8, August 2013.

\bibitem{Papoulis1977}
A.~Papoulis, ``Generalized sampling expansion,'' \emph{IEEE Transactions on
  Circuits and Systems}, vol.~24, no.~11, pp. 652 -- 654, Nov 1977.

\bibitem{ElGamalKim2012}
A.~E. Gamal and Y.~H. Kim, \emph{Network Information theory}.\hskip 1em plus
  0.5em minus 0.4em\relax Cambridge University Press, 2011.

\bibitem{BlackwellBriemanThomasian1959}
D.~Blackwell, L.~Breiman, and A.~J. Thomasian, ``The capacity of a class of
  channels,'' \emph{The Annals of Mathematical Statistics}, vol.~30, no.~4, pp.
  1229--1241, 1959.

\bibitem{EffrosGoldsmithLiang2010}
M.~Effros, A.~Goldsmith, and Y.~Liang, ``Generalizing capacity: New definitions
  and capacity theorems for composite channels,'' \emph{IEEE Transactions on
  Information Theory}, vol.~56, no.~7, pp. 3069 --3087, July 2010.

\bibitem{DonohoJavanmardMontanari2012}
\BIBentryALTinterwordspacing
D.~Donoho, A.~Javanmard, and A.~Montanari, ``Information-theoretically optimal
  compressed sensing via spatial coupling and approximate message passing,''
  Dec. 2011. [Online]. Available: \url{http://arxiv.org/abs/1112.0708}
\BIBentrySTDinterwordspacing

\bibitem{WuVerdu2012}
Y.~Wu and S.~Verdu, ``Optimal phase transitions in compressed sensing,''
  \emph{IEEE Transactions on Information Theory}, vol.~58, no.~10, pp. 6241
  --6263, Oct. 2012.

\bibitem{Vershynin2012}
R.~Vershynin, ``Introduction to the non-asymptotic analysis of random
  matrices,'' \emph{Compressed Sensing, Theory and Applications}, pp. 210 --
  268, 2012.

\bibitem{GuionnetZeitouni2000}
A.~Guionnet and O.~Zeitouni, ``Concentration of the spectral measure for large
  matrices,'' \emph{Electronic Communications in Probability}, no.~5, pp. 119
  --136, 2000.

\bibitem{ghosh2010fundamentals}
A.~Ghosh, J.~Zhang, R.~Muhamed, and J.~G. Andrews, \emph{Fundamentals of
  {LTE}}.\hskip 1em plus 0.5em minus 0.4em\relax Pearson Education, 2010.

\bibitem{LTEWhitePaper}
``{LTE} in a nutshell: White paper on physical layer,'' 2010.

\bibitem{LehmannCasella1998}
E.~L. Lehmann and G.~Casella, \emph{Theory of Point Estimation}.\hskip 1em plus
  0.5em minus 0.4em\relax New York: Springer, 1998.

\bibitem{raginsky2012concentration}
M.~Raginsky and I.~Sason, ``Concentration of measure inequalities in
  information theory, communications and coding,'' \emph{arXiv preprint
  arXiv:1212.4663}, 2012.

\bibitem{MassartPicard2007}
P.~Massart and J.~Picard, \emph{Concentration Inequalities and Model
  Selection}, ser. Lecture Notes in Mathematics.\hskip 1em plus 0.5em minus
  0.4em\relax Springer, 2007.

\bibitem{TaoVu2012Nonhermitian}
\BIBentryALTinterwordspacing
T.~Tao and V.~Vu, ``Random matrices: Universality of local spectral statistics
  of non-{H}ermitian matrices,'' June 2012. [Online]. Available:
  \url{arxiv.org/pdf/1206.1893.pdf}
\BIBentrySTDinterwordspacing

\bibitem{TaoVu2012Covariance}
------, ``Random covariance matrices: Universality of local statistics of
  eigenvalues,'' \emph{The Annals of Probability}, vol.~40, no.~3, pp. 1285 --
  1315, 2012.

\bibitem{CanTao2005Decoding}
E.~Candes and T.~Tao, ``Decoding by linear programming,'' \emph{IEEE
  Transactions on Information Theory}, vol.~51, no.~12, pp. 4203--4215, 2005.

\bibitem{Tao2012RMT}
T.~Tao, \emph{Topics in Random Matrix Theory}, ser. Graduate Studies in
  Mathematics.\hskip 1em plus 0.5em minus 0.4em\relax Providence, Rhode Island:
  American Mathematical Society, 2012.

\bibitem{MisEld2009}
M.~Mishali and Y.~Eldar, ``Blind multiband signal reconstruction: Compressed
  sensing for analog signals,'' \emph{IEEE Transactions on Signal Processing},
  vol.~57, no.~3, pp. 993 --1009, March 2009.

\bibitem{HornJohnson}
R.~A. Horn and C.~R. Johnson, \emph{Matrix analysis}.\hskip 1em plus 0.5em
  minus 0.4em\relax Cambridge University Press, 1985.

\bibitem{Aoki1998Entropy}
M.~Aoki, \emph{New Approaches to Macroeconomic Modeling}.\hskip 1em plus 0.5em
  minus 0.4em\relax Cambridge University Press, 1998.

\bibitem{LitvakPajorRudelson2005}
A.~Litvak, A.~Pajor, M.~Rudelson, and N.~Tomczak-Jaegermann, ``Smallest
  singular value of random matrices and geometry of random polytopes,''
  \emph{Advances in Mathematics}, vol. 195, no.~2, pp. 491 -- 523, 2005.

\bibitem{chen2014backing}
Y.~Chen, A.~J. Goldsmith, and Y.~C. Eldar, ``Backing off from infinity: Tight
  performance bounds for large random vector channels,'' \emph{arXiv preprint
  arXiv:1312.2574}, 2014.

\bibitem{Wang2012}
K.~WANG, ``Random covariance matrices: Universality of local statistics of
  eigenvalues up to the edge,'' \emph{Random Matrices: Theory and
  Applications}, vol.~1, no.~1, p. 1150005, 2012.

\bibitem{RudelsonVershynin2009}
M.~Rudelson and R.~Vershynin, ``Smallest singular value of a random rectangular
  matrix,'' \emph{Communications on Pure and Applied Mathematics}, vol.~62,
  no.~12, pp. 1707--1739, 2009.

\bibitem{Fujikoshi2010}
Y.~Fujikoshi, V.~V. Ulyanov, and R.~Shimizu, \emph{Multivariate Statistics:
  High-dimensional and Large-sample Approximations}, ser. Wiley Series in
  Probability and Statistics.\hskip 1em plus 0.5em minus 0.4em\relax Hoboken,
  New Jersey: Wiley, 2010.

\bibitem{ChenDongarra2005}
Z.~Chen and J.~J. Dongarra, ``Condition numbers of {G}aussian random
  matrices,'' \emph{SIAM Journal on Matrix Analysis and Applications}, vol.~27,
  no.~3, pp. 603--620, July 2005.

\end{thebibliography}
