  Two-dimensional nanomaterials, such as graphene, have been extensively
studied because of their outstanding physical properties. Structure and
geometry optimization of nanopores on such materials is beneficial for their
performances in real-world engineering applications, like water desalination.
However, the optimization process often involves very large number of
experiments or simulations which are expensive and time-consuming. In this
work, we propose a graphene nanopore optimization framework via the combination
of deep reinforcement learning (DRL) and convolutional neural network (CNN) for
efficient water desalination. The DRL agent controls the growth of nanopore by
determining the atom to be removed at each timestep, while the CNN predicts the
performance of nanoporus graphene for water desalination: the water flux and
ion rejection at a certain external pressure. With the synchronous feedback
from CNN-accelerated desalination performance prediction, our DRL agent can
optimize the nanoporous graphene efficiently in an online manner. Molecular
dynamics (MD) simulations on promising DRL-designed graphene nanopores show
that they have higher water flux while maintaining rival ion rejection rate
compared to the normal circular nanopores. Semi-oval shape with rough edges
geometry of DRL-designed pores is found to be the key factor for their high
water desalination performance. Ultimately, this study shows that DRL can be a
powerful tool for material design.
