  This paper investigates the automatic exploration problem under the unknown
environment, which is the key point of applying the robotic system to some
social tasks. The solution to this problem via stacking decision rules is
impossible to cover various environments and sensor properties. Learning based
control methods are adaptive for these scenarios. However, these methods are
damaged by low learning efficiency and awkward transferability from simulation
to reality. In this paper, we construct a general exploration framework via
decomposing the exploration process into the decision, planning, and mapping
modules, which increases the modularity of the robotic system. Based on this
framework, we propose a deep reinforcement learning based decision algorithm
which uses a deep neural network to learning exploration strategy from the
partial map. The results show that this proposed algorithm has better learning
efficiency and adaptability for unknown environments. In addition, we conduct
the experiments on the physical robot, and the results suggest that the learned
policy can be well transfered from simulation to the real robot.
