  Reinforcement Learning, a machine learning framework for training an
autonomous agent based on rewards, has shown outstanding results in various
domains. However, it is known that learning a good policy is difficult in a
domain where rewards are rare. We propose a method, optimistic proximal policy
optimization (OPPO) to alleviate this difficulty. OPPO considers the
uncertainty of the estimated total return and optimistically evaluates the
policy based on that amount. We show that OPPO outperforms the existing methods
in a tabular task.
