  In this work we present a new method of black-box optimization and constraint
satisfaction. Existing algorithms that have attempted to solve this problem are
unable to consider multiple modes, and are not able to adapt to changes in
environment dynamics. To address these issues, we developed a modified
Cross-Entropy Method (CEM) that uses a masked auto-regressive neural network
for modeling uniform distributions over the solution space. We train the model
using maximum entropy policy gradient methods from Reinforcement Learning. Our
algorithm is able to express complicated solution spaces, thus allowing it to
track a variety of different solution regions. We empirically compare our
algorithm with variations of CEM, including one with a Gaussian prior with
fixed variance, and demonstrate better performance in terms of: number of
diverse solutions, better mode discovery in multi-modal problems, and better
sample efficiency in certain cases.
