  Generative flow networks (GFlowNets) are a method for learning a stochastic
policy for generating compositional objects, such as graphs or strings, from a
given unnormalized density by sequences of actions, where many possible action
sequences may lead to the same object. We find previously proposed learning
objectives for GFlowNets, flow matching and detailed balance, which are
analogous to temporal difference learning, to be prone to inefficient credit
propagation across long action sequences. We thus propose a new learning
objective for GFlowNets, trajectory balance, as a more efficient alternative to
previously used objectives. We prove that any global minimizer of the
trajectory balance objective can define a policy that samples exactly from the
target distribution. In experiments on four distinct domains, we empirically
demonstrate the benefits of the trajectory balance objective for GFlowNet
convergence, diversity of generated samples, and robustness to long action
sequences and large action spaces.
