  The Markov Chain Monte Carlo (MCMC) methods are popular when considering
sampling from a high-dimensional random variable $\mathbf{x}$ with possibly
unnormalised probability density $p$ and observed data $\mathbf{d}$. However,
MCMC requires evaluating the posterior distribution $p(\mathbf{x}|\mathbf{d})$
of the proposed candidate $\mathbf{x}$ at each iteration when constructing the
acceptance rate. This is costly when such evaluations are intractable. In this
paper, we introduce a new non-Markovian sampling algorithm called Moving Target
Monte Carlo (MTMC). The acceptance rate at $n$-th iteration is constructed
using an iteratively updated approximation of the posterior distribution
$a_n(\mathbf{x})$ instead of $p(\mathbf{x}|\mathbf{d})$. The true value of the
posterior $p(\mathbf{x}|\mathbf{d})$ is only calculated if the candidate
$\mathbf{x}$ is accepted. The approximation $a_n$ utilises these evaluations
and converges to $p$ as $n \rightarrow \infty$. A proof of convergence and
estimation of convergence rate in different situations are given.
