  We develop a Bayesian inference method for diffusions observed discretely and
with noise, which is free of discretisation bias. Unlike existing unbiased
inference methods, our method does not rely on exact simulation techniques.
Instead, our method uses standard time-discretised approximations of
diffusions, such as the Euler--Maruyama scheme. Our approach is based on
particle marginal Metropolis--Hastings, a particle filter, randomised
multilevel Monte Carlo, and importance sampling type correction of approximate
Markov chain Monte Carlo. The resulting estimator leads to inference without a
bias from the time-discretisation as the number of Markov chain iterations
increases. We give convergence results and recommend allocations for algorithm
inputs. Our method admits a straightforward parallelisation, and can be
computationally efficient. The user-friendly approach is illustrated on three
examples, where the underlying diffusion is an Ornstein--Uhlenbeck process, a
geometric Brownian motion, and a 2d non-reversible Langevin equation.
