  Animals are known to make efficient probabilistic inferences based on
uncertain and noisy information from the outside world. Although it is known
that generic neural networks can perform near-optimal point estimation by
probabilistic population codes which has been proposed as a neural basis for
encoding of probability distribution, the mechanisms of sampling-based
inference has not been clarified. In this study, we trained two types of
artificial neural networks: feedforward neural networks (FFNNs) and recurrent
neural networks (RNNs) to perform sampling based probabilistic inference. Then,
we analyzed and compared the mechanisms of sampling in the RNN with those in
the FFNN. As a result, it was found that sampling in RNN is performed by a
mechanism that efficiently utilizes the properties of dynamical systems, unlike
FFNN. It was also found that sampling in RNNs acts as an inductive bias,
enabling more accurate estimation than in MAP estimation. These results will
provide important implications for the discussion of the relationship between
dynamical systems and information processing in neural networks.
