  Generative flow networks (GFlowNets), as an emerging technique, can be used
as an alternative to reinforcement learning for exploratory control tasks.
GFlowNet aims to generate distribution proportional to the rewards over
terminating states, and to sample different candidates in an active learning
fashion. GFlowNets need to form a DAG and compute the flow matching loss by
traversing the inflows and outflows of each node in the trajectory. No
experiments have yet concluded that GFlowNets can be used to handle continuous
tasks. In this paper, we propose generative continuous flow networks
(CFlowNets) that can be applied to continuous control tasks. First, we present
the theoretical formulation of CFlowNets. Then, a training framework for
CFlowNets is proposed, including the action selection process, the flow
approximation algorithm, and the continuous flow matching loss function.
Afterward, we theoretically prove the error bound of the flow approximation.
The error decreases rapidly as the number of flow samples increases. Finally,
experimental results on continuous control tasks demonstrate the performance
advantages of CFlowNets compared to many reinforcement learning methods,
especially regarding exploration ability.
