  One of the fundamental assumptions in stochastic control of continuous time
processes is that the dynamics of the underlying (diffusion) process is known.
This is, however, usually obviously not fulfilled in practice. On the other
hand, over the last decades, a rich theory for nonparametric estimation of the
drift (and volatility) for continuous time processes has been developed. The
aim of this paper is bringing together techniques from stochastic control with
methods from statistics for stochastic processes to find a way to both learn
the dynamics of the underlying process and control in a reasonable way at the
same time. More precisely, we study a long-term average impulse control
problem, a stochastic version of the classical Faustmann timber harvesting
problem. One of the problems that immediately arises is an
exploration-exploitation dilemma as is well known for problems in machine
learning. We propose a way to deal with this issue by combining exploration and
exploitation periods in a suitable way. Our main finding is that this
construction can be based on the rates of convergence of estimators for the
invariant density. Using this, we obtain that the average cumulated regret is
of uniform order $O({T^{-1/3}})$.
