  The IEEE Very Small Size Soccer (VSSS) is a robot soccer competition in which
two teams of three small robots play against each other. Traditionally, a
deterministic coach agent will choose the most suitable strategy and formation
for each adversary's strategy. Therefore, the role of a coach is of great
importance to the game. In this sense, this paper proposes an end-to-end
approach for the coaching task based on Reinforcement Learning (RL). The
proposed system processes the information during the simulated matches to learn
an optimal policy that chooses the current formation, depending on the opponent
and game conditions. We trained two RL policies against three different teams
(balanced, offensive, and heavily offensive) in a simulated environment. Our
results were assessed against one of the top teams of the VSSS league, showing
promising results after achieving a win/loss ratio of approximately 2.0.
