  Semantic segmentation and semantic image synthesis are two representative
tasks in visual perception and generation. While existing methods consider them
as two distinct tasks, we propose a unified diffusion-based framework (SemFlow)
and model them as a pair of reverse problems. Specifically, motivated by
rectified flow theory, we train an ordinary differential equation (ODE) model
to transport between the distributions of real images and semantic masks. As
the training object is symmetric, samples belonging to the two distributions,
images and semantic masks, can be effortlessly transferred reversibly. For
semantic segmentation, our approach solves the contradiction between the
randomness of diffusion outputs and the uniqueness of segmentation results. For
image synthesis, we propose a finite perturbation approach to enhance the
diversity of generated results without changing the semantic categories.
Experiments show that our SemFlow achieves competitive results on semantic
segmentation and semantic image synthesis tasks. We hope this simple framework
will motivate people to rethink the unification of low-level and high-level
vision. Project page: https://github.com/wang-chaoyang/SemFlow.
