  This paper studies the Gaussian approximation of high-dimensional and
non-degenerate U-statistics of order two under the supremum norm. We propose a
two-step Gaussian approximation procedure that does not impose structural
assumptions on the data distribution. Specifically, subject to mild moment
conditions on the kernel, we establish the explicit rate of convergence that
decays polynomially in sample size for a high-dimensional scaling limit, where
the dimension can be much larger than the sample size. We also supplement a
practical Gaussian wild bootstrap method to approximate the quantiles of the
maxima of centered U-statistics and prove its asymptotic validity. The wild
bootstrap is demonstrated on statistical applications for high-dimensional
non-Gaussian data including: (i) principled and data-dependent tuning parameter
selection for regularized estimation of the covariance matrix and its related
functionals; (ii) simultaneous inference for the covariance and rank
correlation matrices. In particular, for the thresholded covariance matrix
estimator with the bootstrap selected tuning parameter, we show that the
Gaussian-like convergence rates can be achieved for heavy-tailed data, which
are less conservative than those obtained by the Bonferroni technique that
ignores the dependency in the underlying data distribution. In addition, we
also show that even for subgaussian distributions, error bounds of the
bootstrapped thresholded covariance matrix estimator can be much tighter than
those of the minimax estimator with a universal threshold.
