  We present a simple and effective method for representing periodic functions
and enforcing exactly the periodic boundary conditions for solving differential
equations with deep neural networks (DNN). The method stems from some simple
properties about function compositions involving periodic functions. It
essentially composes a DNN-represented arbitrary function with a set of
independent periodic functions with adjustable (training) parameters. We
distinguish two types of periodic conditions: those imposing the periodicity
requirement on the function and all its derivatives (to infinite order), and
those imposing periodicity on the function and its derivatives up to a finite
order $k$ ($k\geqslant 0$). The former will be referred to as $C^{\infty}$
periodic conditions, and the latter $C^{k}$ periodic conditions. We define
operations that constitute a $C^{\infty}$ periodic layer and a $C^k$ periodic
layer (for any $k\geqslant 0$). A deep neural network with a $C^{\infty}$ (or
$C^k$) periodic layer incorporated as the second layer automatically and
exactly satisfies the $C^{\infty}$ (or $C^k$) periodic conditions. We present
extensive numerical experiments on ordinary and partial differential equations
with $C^{\infty}$ and $C^k$ periodic boundary conditions to verify and
demonstrate that the proposed method indeed enforces exactly, to the machine
accuracy, the periodicity for the DNN solution and its derivatives.
