  The development of a reliable subgrid-scale (SGS) model for large-eddy
simulation (LES) is of great importance for many scientific and engineering
applications. Recently, deep learning approaches have been tested for this
purpose using high-fidelity data such as direct numerical simulation (DNS) in a
supervised learning process. However, such data are generally not available in
practice. Deep reinforcement learning (DRL) using only limited target
statistics can be an alternative algorithm in which the training and testing of
the model are conducted in the same LES environment. The DRL of turbulence
modeling remains challenging owing to its chaotic nature, high dimensionality
of the action space, and large computational cost. In the present study, we
propose a physics-constrained DRL framework that can develop a deep neural
network (DNN)-based SGS model for the LES of turbulent channel flow. The DRL
models that produce the SGS stress were trained based on the local gradient of
the filtered velocities. The developed SGS model automatically satisfies the
reflectional invariance and wall boundary conditions without an extra training
process so that DRL can quickly find the optimal policy. Furthermore, direct
accumulation of reward, spatially and temporally correlated exploration, and
the pre-training process are applied for the efficient and effective learning.
In various environments, our DRL could discover SGS models that produce the
viscous and Reynolds stress statistics perfectly consistent with the filtered
DNS. By comparing various statistics obtained by the trained models and
conventional SGS models, we present a possible interpretation of better
performance of the DRL model.
