  This paper presents the concepts of Artificial Intelligence,
Multi-Agent-Systems, Coordination, Intelligent Robotics and Deep Reinforcement
Learning. Emphasis is given on and how AI and DRL, may be efficiently used to
create efficient robot skills and coordinated robotic teams, capable of
performing very complex actions and tasks, such as playing a game of soccer.
The paper also presents the concept of robotic soccer and the vision and
structure of the RoboCup initiative with emphasis on the Humanoid Simulation 3D
league and the new challenges this competition, poses. The final topics
presented at the paper are based on the research developed/coordinated by the
author throughout the last 22 years in the context of the FCPortugal project.
The paper presents a short description of the coordination methodologies
developed, such as: Strategy, Tactics, Formations, Setplays, and Coaching
Languages and the use of Machine Learning to optimize the use of this concepts.
The topics presented also include novel stochastic search algorithms for black
box optimization and their use in the optimization of omnidirectional walking
skills, robotic multi-agent learning and the creation of a humanoid kick with
controlled distance. Finally, new applications using variations of the Proximal
Policy Optimization algorithm and advanced modelling for robot and multi-robot
learning are briefly explained with emphasis for our new humanoid sprinting and
running skills and an amazing humanoid robot soccer dribbling skill. FCPortugal
project enabled us to publish more than 100 papers and win several competitions
in different leagues and many scientific awards at RoboCup. In total, our team
won more than 40 awards in international competitions including a clear victory
at the Simulation 3D League at RoboCup 2022 competition, scoring 84 goals and
conceding only 2.
