  We consider solutions of the linear heat equation in $\mathbb{R}^N$ with
isolated singularities. It is assumed that the position of a singular point
depends on time and is H\"older continuous with the exponent $\alpha \in
(0,1)$. We show that any isolated singularity is removable if it is weaker than
a certain order depending on $\alpha$. We also show the optimality of the
removability condition by showing the existence of a solution with a
nonremovable singularity. These results are applied to the case where the
singular point behaves like a fractional Brownian motion with the Hurst
exponent $H \in (0,1/2] $. It turns out that $H=1/N$ is critical.
