  Optical multi-layer thin films are widely used in optical and energy
applications requiring photonic designs. Engineers often design such structures
based on their physical intuition. However, solely relying on human experts can
be time-consuming and may lead to sub-optimal designs, especially when the
design space is large. In this work, we frame the multi-layer optical design
task as a sequence generation problem. A deep sequence generation network is
proposed for efficiently generating optical layer sequences. We train the deep
sequence generation network with proximal policy optimization to generate
multi-layer structures with desired properties. The proposed method is applied
to two energy applications. Our algorithm successfully discovered
high-performance designs, outperforming structures designed by human experts in
task 1, and a state-of-the-art memetic algorithm in task 2.
