  Recent one-shot video tuning methods, which fine-tune the network on a
specific video based on pre-trained text-to-image models (e.g., Stable
Diffusion), are popular in the community because of the flexibility. However,
these methods often produce videos marred by incoherence and inconsistency. To
address these limitations, this paper introduces a simple yet effective noise
constraint across video frames. This constraint aims to regulate noise
predictions across their temporal neighbors, resulting in smooth latents. It
can be simply included as a loss term during the training phase. By applying
the loss to existing one-shot video tuning methods, we significantly improve
the overall consistency and smoothness of the generated videos. Furthermore, we
argue that current video evaluation metrics inadequately capture smoothness. To
address this, we introduce a novel metric that considers detailed features and
their temporal dynamics. Experimental results validate the effectiveness of our
approach in producing smoother videos on various one-shot video tuning
baselines. The source codes and video demos are available at
\href{https://github.com/SPengLiang/SmoothVideo}{https://github.com/SPengLiang/SmoothVideo}.
