  Over the last few years, debiased estimators have been proposed in order to
establish rigorous confidence intervals for high-dimensional problems in
machine learning and data science. The core argument is that the error of these
estimators with respect to the ground truth can be expressed as a Gaussian
variable plus a remainder term that vanishes as long as the dimension of the
problem is sufficiently high. Thus, uncertainty quantification (UQ) can be
performed exploiting the Gaussian model. Empirically, however, the remainder
term cannot be neglected in many realistic situations of moderately-sized
dimensions, in particular in certain structured measurement scenarios such as
Magnetic Resonance Imaging (MRI). This, in turn, can downgrade the advantage of
the UQ methods as compared to non-UQ approaches such as the standard LASSO. In
this paper, we present a method to improve the debiased estimator by sampling
without replacement. Our approach leverages recent results of ours on the
structure of the random nature of certain sampling schemes showing how a
transition between sampling with and without replacement can lead to a weighted
reconstruction scheme with improved performance for the standard LASSO. In this
paper, we illustrate how this reweighted sampling idea can also improve the
debiased estimator and, consequently, provide a better method for UQ in Fourier
imaging.
