  Given any domain $X\subseteq \mathbb{R}^d$ and a probability measure $\rho$
on $X$, we study the problem of approximating in $L^2(X,\rho)$ a given function
$u:X\to\mathbb{R}$, using its noiseless pointwise evaluations at random
samples. For any given linear space $V\subset L^2(X,\rho)$ with dimension $n$,
previous works have shown that stable and optimally converging Weighted
Least-Squares (WLS) estimators can be constructed using $m$ random samples
distributed according to an auxiliary probability measure $\mu$ that depends on
$V$, with $m$ being linearly proportional to $n$ up to a logarithmic term. As a
first contribution, we present novel results on the stability and accuracy of
WLS estimators with a given approximation space, using random samples that are
more structured than those used in the previous analysis. As a second
contribution, we study approximation by WLS estimators in the adaptive setting.
For any sequence of nested spaces $(V_k)_{k} \subset L^2(X,\rho)$, we show that
a sequence of WLS estimators of $u$, one for each space $V_k$, can be
sequentially constructed such that: i) the estimators remain provably stable
with high probability and optimally converging in expectation, simultaneously
for all iterations from one to $k$, and ii) the overall number of samples
necessary to construct all the first $k$ estimators remains linearly
proportional to the dimension of $V_k$. We propose two sampling algorithms that
achieve this goal. The first one is a purely random algorithm that recycles
most of the samples from the previous iterations. The second algorithm recycles
all the samples from all the previous iterations. Such an achievement is made
possible by crucially exploiting the structure of the random samples. Finally
we develop numerical methods for the adaptive approximation of functions in
high dimension.
