  In this paper, we model nested polar code construction as a Markov decision
process (MDP), and tackle it with advanced reinforcement learning (RL)
techniques. First, an MDP environment with state, action, and reward is defined
in the context of polar coding. Specifically, a state represents the
construction of an $(N,K)$ polar code, an action specifies its reduction to an
$(N,K-1)$ subcode, and reward is the decoding performance. A neural network
architecture consisting of both policy and value networks is proposed to
generate actions based on the observed states, aiming at maximizing the overall
rewards. A loss function is defined to trade off between exploitation and
exploration. To further improve learning efficiency and quality, an `integrated
learning' paradigm is proposed. It first employs a genetic algorithm to
generate a population of (sub-)optimal polar codes for each $(N,K)$, and then
uses them as prior knowledge to refine the policy in RL. Such a paradigm is
shown to accelerate the training process, and converge at better performances.
Simulation results show that the proposed learning-based polar constructions
achieve comparable, or even better, performances than the state of the art
under successive cancellation list (SCL) decoders. Last but not least, this is
achieved without exploiting any expert knowledge from polar coding theory in
the learning algorithms.
