  Deep reinforcement learning (DRL) has demonstrated its potential in solving
complex manufacturing decision-making problems, especially in a context where
the system learns over time with actual operation in the absence of training
data. One interesting and challenging application for such methods is the
assembly sequence planning (ASP) problem. In this paper, we propose an approach
to the implementation of DRL methods in ASP. The proposed approach introduces
in the RL environment parametric actions to improve training time and sample
efficiency and uses two different reward signals: (1) user's preferences and
(2) total assembly time duration. The user's preferences signal addresses the
difficulties and non-ergonomic properties of the assembly faced by the human
and the total assembly time signal enforces the optimization of the assembly.
Three of the most powerful deep RL methods were studied, Advantage Actor-Critic
(A2C), Deep Q-Learning (DQN), and Rainbow, in two different scenarios: a
stochastic and a deterministic one. Finally, the performance of the DRL
algorithms was compared to tabular Q-Learnings performance. After 10,000
episodes, the system achieved near optimal behaviour for the algorithms tabular
Q-Learning, A2C, and Rainbow. Though, for more complex scenarios, the algorithm
tabular Q-Learning is expected to underperform in comparison to the other 2
algorithms. The results support the potential for the application of deep
reinforcement learning in assembly sequence planning problems with human
interaction.
