  We propose a $k^{\rm th}$-order unfitted finite element method ($2\le k\le
4$) to solve the moving interface problem of the Oseen equations. Thorough
error estimates for the discrete solutions are presented by considering errors
from interface-tracking, time integration, and spatial discretization. In
literatures on time-dependent Stokes interface problems, error estimates for
the discrete pressure are usually sub-optimal, namely, $(k-1)^{\rm th}$-order,
under the $L^2$-norm. We have obtained a $(k-1)^{\rm th}$-order error estimate
for the discrete pressure under the $H^1$-norm. Numerical experiments for a
severely deforming interface show that optimal convergence orders are obtained
for $k = 3$ and $4$.
