  The class of deep deterministic off-policy algorithms is effectively applied
to solve challenging continuous control problems. Current approaches commonly
utilize random noise as an exploration method, which has several drawbacks,
including the need for manual adjustment for a given task and the absence of
exploratory calibration during the training process. We address these
challenges by proposing a novel guided exploration method that uses an ensemble
of Monte Carlo Critics for calculating exploratory action correction. The
proposed method enhances the traditional exploration scheme by dynamically
adjusting exploration. Subsequently, we present a novel algorithm that
leverages the proposed exploratory module for both policy and critic
modification. The presented algorithm demonstrates superior performance
compared to modern reinforcement learning algorithms across a variety of
problems in the DMControl suite.
