  The Generative Flow Network is a probabilistic framework where an agent
learns a stochastic policy for object generation, such that the probability of
generating an object is proportional to a given reward function. Its
effectiveness has been shown in discovering high-quality and diverse solutions,
compared to reward-maximizing reinforcement learning-based methods.
Nonetheless, GFlowNets only learn from rewards of the terminal states, which
can limit its applicability. Indeed, intermediate rewards play a critical role
in learning, for example from intrinsic motivation to provide intermediate
feedback even in particularly challenging sparse reward tasks. Inspired by
this, we propose Generative Augmented Flow Networks (GAFlowNets), a novel
learning framework to incorporate intermediate rewards into GFlowNets. We
specify intermediate rewards by intrinsic motivation to tackle the exploration
problem in sparse reward environments. GAFlowNets can leverage edge-based and
state-based intrinsic rewards in a joint way to improve exploration. Based on
extensive experiments on the GridWorld task, we demonstrate the effectiveness
and efficiency of GAFlowNet in terms of convergence, performance, and diversity
of solutions. We further show that GAFlowNet is scalable to a more complex and
large-scale molecule generation domain, where it achieves consistent and
significant performance improvement.
