  A Pseudo-Random Number Generator (PRNG) is any algorithm generating a
sequence of numbers approximating properties of random numbers. These numbers
are widely employed in mid-level cryptography and in software applications.
Test suites are used to evaluate PRNGs quality by checking statistical
properties of the generated sequences. These sequences are commonly represented
bit by bit. This paper proposes a Reinforcement Learning (RL) approach to the
task of generating PRNGs from scratch by learning a policy to solve a partially
observable Markov Decision Process (MDP), where the full state is the period of
the generated sequence and the observation at each time step is the last
sequence of bits appended to such state. We use a Long-Short Term Memory (LSTM)
architecture to model the temporal relationship between observations at
different time steps, by tasking the LSTM memory with the extraction of
significant features of the hidden portion of the MDP's states. We show that
modeling a PRNG with a partially observable MDP and a LSTM architecture largely
improves the results of the fully observable feedforward RL approach introduced
in previous work.
