Disruptive innovations in RoboCup 2D Soccer Simulation League: from Cyberoos'98 to Gliders2016
Mikhail Prokopenko$^1$
Peter Wang$^{2
We review disruptive innovations introduced in the RoboCup 2D Soccer Simulation League over the twenty years since its inception, and trace the progress of our champion team (Gliders). We conjecture that the League has been developing as an ecosystem shaped by diverse approaches taken by participating teams, increasing in its overall complexity. A common feature is that different champion teams succeeded in finding a way to decompose the enormous search-space of possible single- and multi-agent behaviours, by automating the exploration of the problem space with various techniques which accelerated the software development efforts. These methods included interactive debugging, machine learning, automated planning, and opponent modelling. The winning approach developed by Gliders is centred on human-based evolutionary computation which optimised several components such as an action-dependent evaluation function, dynamic tactics with Voronoi diagrams, information dynamics, and bio-inspired collective behaviour.
Gliders2016-revised.tex
%%%%%%%%%%%%%%%%%%%% author.tex %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% sample root file for your "contribution" to a proceedings volume
%
% Use this file as a template for your own input.
%
%%%%%%%%%%%%%%%% Springer %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%


\documentclass{svproc}
%
% RECOMMENDED %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%

% to typeset URLs, URIs, and DOIs
\usepackage{url}
\def\UrlFont{\rmfamily}

\setlength{\textheight}{20.2cm}

\begin{document}
\mainmatter              % start of a contribution
%
\title{Disruptive innovations in RoboCup 2D Soccer Simulation League: from Cyberoos'98 to Gliders2016}
%
\titlerunning{From Cyberoos'98 to Gliders2016}  % abbreviated title (for running head)
%                                     also used for the TOC unless
%                                     \toctitle is used
%
\author{Mikhail Prokopenko$^1$ \and Peter Wang$^{2}$}
%
\authorrunning{Mikhail Prokopenko and Peter Wang} % abbreviated author list (for running head)
%
%%%% list of authors for the TOC (use if author list has to be modified)
\tocauthor{Mikhail Prokopenko, Peter Wang}
%
\institute{Complex Systems Research Group, Faculty of Engineering and IT\\ 
			The University of Sydney, NSW 2006, Australia\\
\email{mikhail.prokopenko@sydney.edu.au}\\
\and
Data Mining, CSIRO Data61\\ 
PO Box 76, Epping, NSW 1710, Australia}

\maketitle              % typeset the title of the contribution

\begin{abstract}
We review  disruptive innovations introduced in the RoboCup 2D Soccer Simulation League over the twenty years since its inception, and trace  the progress of our champion team (Gliders). We conjecture that the League has been developing as an ecosystem shaped by diverse approaches taken by participating teams, increasing in its overall complexity. A common feature is that different champion teams succeeded in finding a way to decompose the enormous search-space of possible single- and multi-agent behaviours, by automating the exploration of the problem space with various techniques which accelerated the software development efforts.  These methods included interactive debugging, machine learning, automated planning, and opponent modelling.  The winning approach developed by Gliders is centred on human-based evolutionary computation which optimised several components such as an action-dependent evaluation function, dynamic tactics with Voronoi diagrams, information dynamics, and bio-inspired collective behaviour. 
 
\end{abstract}
%
\section{Introduction}
\begin{quote}
{\em \small \hspace*{25mm} Agent Smith:   ``You can't win, it's pointless to keep fighting!\\ \hspace*{45mm} Why, Mr. Anderson? Why do you persist?''\\
\hspace*{25mm}  Neo:  ``Because I choose to.''} \vspace{-2mm} \begin{flushright}{\small The Matrix Revolutions.} \end{flushright}
\end{quote}

The first official RoboCup was held in 1997, proposing a new benchmark for Artificial Intelligence (AI) and robotics. Incidentally, another classical AI challenge was successfully met in May 1997 when IBM Deep Blue defeated the human world champion in chess.  By design, RoboCup and chess  differ in a few key elements: environment (static vs dynamic), state change (turn-taking vs real-time), information accessibility (complete vs incomplete), sensor readings (symbolic vs non-symbolic), and control (central vs distributed) \cite{Asada99}.  These differences are emphasised in the RoboCup 2D Soccer Simulation League \cite{Kitano97}, which quickly gained prominence, becoming one of the largest RoboCup leagues. 

In this league, two teams of 12 fully autonomous software programs (called ``agents'') play soccer in a two-dimensional virtual soccer stadium (11 player agents and 1 coach agent in each team), with no remote control. Each player agent receives relative and noisy input from its virtual sensors (visual, acoustic and physical) and may perform some basic actions in order to influence its environment, e.g., running, turning and kicking the ball. The coach agent  receives perfect input but can communicate with the player agents only infrequently and through a fairly limited channel.  The ability to simulate soccer matches without physical robots abstracts away low-level issues such as image processing and motor breakages, allowing teams to focus on the development of complex team behaviours and strategies for a larger number of autonomous agents \cite{rcsymp2014,RAM15}. 

A simulated game lasts just over 10 minutes on average, and is played over a small network of computer workstations which execute the code in parallel.
Each simulation step takes merely a tenth of a second, during which the entire sensory-motor cycle takes place within an agent: starting with receiving new sensory inputs from the simulator, proceeding to updating the internal memory, to evaluating possible choices, to sending the chosen action back to the simulator. The main challenge for each agent is to derive the best possible action to execute at any specific time, while facing unexpected  actions of the opposing agents.

Over 20 years, the RoboCup community has developed the open-source 2D simulator and visualisation software which currently, with various packaged utilities and basic agent libraries, contains nearly a million lines of code. During this period, the League and the participating teams have  undergone several  transitions each of which eventually expanded the level of agents' intelligence and their behavioral complexity.  In this paper we attempt to trace not only the ten-year long progress of our own team from its first implementation (Cyberoos; participated between 1998 and 2003) to the RoboCup-2016 champion team (Gliders; competed first in 2012), but also put this trace in the context of the twenty-year long evolution of the Sim2D League itself. 

The conjecture we put forward is that the League has been developing as an ecosystem with an increasing complexity shaped by different approaches taken by participating teams. Furthermore, this evolving ecosystem has experienced a series of salient transitions leading to  emergence of qualitatively new properties in the intelligence exhibited by the agents. By a transition we do not mean a mere extension of some simulated capabilities, such as the introduction of goalkeepers, heterogeneous player types, or a coach language.  Instead, we associate a transition with a specific methodological advance which played the role of a \emph{disruptive innovation}, with wide-spread consequences affecting the entire ``ecosystem'', for example, a release of standard libraries, and so on. 
We use the term ``disruptive innovation'' in a broad sense to indicate an innovation that creates a new ecosystem (by analogy with a new market or value network), eventually disrupting an existing system, displacing established structures and relationships.


\section{A simulated world}

	
The foundation supporting the evolution of the League is undoubtedly the construction of the soccer server itself, providing a centralised world model with several key features, enhanced over the following years:
\begin{itemize}
\item distributed  client/server  system  running  on  a  network and producing fragmented,  localised  and  imprecise  (noisy  and latent)  information  about  the  environment  (virtual soccer field) \cite{Noda2003,haker2001method};
\item concurrent  communication  with  a  number of  autonomous agents \cite{Stone99};
\item heterogeneous  sensory  data  (visual,  auditory,  kinetic) without a global  vision, and limited  range  of basic commands/effectors  (turn,  kick,  dash, $\ldots$)  \cite{ATAL2000};
\item asynchronous  perception-action  activity and limited  window of opportunity  to  perform  an  action \cite{Butler2001};
\item autonomous decision-making under  constraints  enforced  by  teamwork (collaboration)  and opponent  (competition) \cite{IAAI2000};
\item conflicts  between reactivity  and deliberation \cite{Reis2001}.
\end{itemize}
The only restriction that was imposed from the outset is  that participants should ``never use central control mechanisms to control a team of agents'' \cite{NodaSMAK98}.

A crucial feature making this simulated world an evolving ``ecosystem'' is the availability of binaries (and sometimes the source code) of participating teams, contained within an online team repository. The repository is updated after each annual RoboCup competition, allowing the participants to improve their teams with respect to the top teams of the previous championships. These improvements diversify the teams' functionality and explore the immense search-space of possible behaviours in the quest for optimal solutions. This process results in a co-evolution of the teams, raising the overall competition level. 

 
\vspace{-2mm}
\section{Partial automation of development efforts}

``AT Humboldt'' from Humboldt University, Germany became the first champion of the League at RoboCup-1997 (Nagoya, Japan). The team used a combination of reactive and planning systems, successfully deploying its agents within the simulated world.  

The following couple of years passed under the domination of ``CMUnited'' team from Carnegie Mellon University (USA) which took the championship in 1998 (Paris, France) and 1999 (Stockholm, Sweden). One of the key reasons for this success was the development of several  tools partially automating the overall effort, such as an offline agent training module, and layered disclosure: a technique for disclosing to a human designer  the specific detailed reasons for an agent actions (in run-time or retroactively).  Layered disclosure made  it  possible  to inspect  the  details  of  an  individual  player's decision-making process at any point \cite{LNAI99-simulator}, becoming in our view the first disruptive innovation in the League.   Together with the offline agent training module, it clearly exemplified the power of automation in accelerating  the development effort --- precisely because it enabled the design effort to reach into a larger part of the search-space by encoding more diverse behaviours.

It is important to point out that there were other novelties introduced by CMUnited-98 and CMUnited-99, such as ``single-channel, low-bandwidth communication'',  ``predictive, locally optimal skills (PLOS)'', ``strategic positioning using attraction and repulsion (SPAR)'', etc. \cite{LNAI99-simulator}, but we believe that it is the partial automation of the software development that became the \emph{disruptive} innovation. It has led to a wide-spread adoption of several debugging, visualising, log-playing, log-analysing, and machine learning tools.  

	
	
\section{Configurational space}

A number of new teams in 2000 utilised the code base of the 1999 champions, CMUnited-99: it provided code for interaction with the soccer server, skills, strategies, and debugging tools in a variety of programming languages  \cite{StoneABFKLSTW00}.
The champion of RoboCup-2000 held in Melbourne, Australia, ``FC Portugal'' from University of Aveiro and University of Porto, extended this code base with a systematic approach to describing team strategy, the concepts of tactics, formations and player types, as well as the situation based strategic positioning, the dynamic positioning and role exchange mechanisms \cite{ReisL00,Reis2001}.

The generic innovation underlying these mechanisms comprised the ability to configure diverse single- and multi-agent behaviours. The range of these behaviours span from active (ball possession) to strategic (ball recovery), from formations to tactics, and from individual skills to team strategies.  Such diversity resulted in a considerable configurational flexibility displayed by the winning team, significantly increasing the software development productivity, and more importantly, expanding the extent of the available behavioural search-space.  

Not surprisingly, the expansion brought about by the larger configurational capacity was further exploited by the introduction of a standard coach language \cite{P-008-M65} enabling high-level coaching with explicit definition of formations, situations, player types and time periods, and resulting in a high-level coordination of team behaviour.  In other words, a disruptive innovation again was delivered by a method which allowed to access deeper regions of the available search-space.

Team ``TsinghuAeolus'' from Tsinghua University, China, which won the next two championships (RoboCup-2001 in Seattle, USA, and RoboCup-2002 in Fukuoka), focussed specifically on increasing the agents' adaptability via a novel online advice-taking mechanism \cite{Jinyi2004}. The configurational space was extended by a task-decomposition mechanism that assigned different parts of the task to different agents. 

A major boost to the League was provided by the partial release of the source code of the next champion, team ``UvA Trilearn'' from University of Amsterdam, The Netherlands, which won RoboCup-2003 in Padua, Italy \cite{Kok03robocup}.  This release resulted in a standardisation of many low-level behaviours and world model, effectively ``locking in'' the configurational space attained by that time, and motivating several teams to switch their code base to UvA Trilearn base.


\section{Cyberoos: 1998 -- 2003}	

At this stage we take a brief look at our first team, Cyberoos, which participated in RoboCup competitions between 1998 and 2003. The Cyberoos'98 team took
$3^{rd}$ place in the 1998 Pacific Rim RoboCup competition \cite{prokopenko1998designing}, while Cyberoos'2000 were $4^{th}$ in the Open European RoboCup-2000 \cite{Butler2001}. Despite these regional successes, the team's best result at the world stage was a shared $9^{th}$ place which Cyberoos repeatedly took at the RoboCup competitions in 2000, 2001, 2002 and 2003, never reaching the quarter-finals \cite{Prokopenko2001,ProkopenkoWH01,Prokopenko02,Prokopenko03}. In hindsight, the main reason for this lack of progress was an oversight of the main tendency driving the innovations in the League: the exploration of the search-space due to the automation of the development efforts and the standardisation of the configurational space. 

Instead, the approach taken by Cyberoos focussed on self-organisation of emergent behaviour within a purely reactive agent architecture \cite{ProkopenkoWH01}.  Only during the later years the Cyberoos architecture diversified, and included semi-automated methods that quantified the team performance in generic information-theoretic terms  \cite{Prokopenko02,Prokopenko03}. This approach focussed on measuring the behavioural and belief dynamics in multi-agent systems, offering a possibility to evolve the team behaviour,  optimised under a universal objective function, within the framework of information-driven self-organisation \cite{nehaniv2005evolutionary,prokopenko06-alife,sab06}. However, this framework has started to take a functional shape only a few years later, after the time when the Cyberoos team effort stopped in 2003. 

\vspace{-3mm}
\section{Search-space decomposition}
\vspace{-1mm}

The next decade of RoboCup championships witnessed an intense competition between three teams: ``Brainstormers'' from University of Osnabr{\"u}ck, Germany,  ``WrightEagle'' from University of Science and Technology of China, and ``HELIOS'' from Fukuoka University and Osaka Prefecture University, Japan.   Brainstormers became champions three times: in 2005 (Osaka, Japan), 2007 (Atlanta, USA), and 2008 (Suzhou, China);  WrightEagle came first an incredible six times: in 2006 (Bremen, Germany), 2009 (Graz, Austria), 2011 (Istanbul, Turkey), 2013 (Eindhoven, The Netherlands), 2014 (Joao Pessoa, Brazil) and 2015 (Hefei, China); and HELIOS succeeded twice: in 2010 (Singapore) and 2012 (Mexico City, Mexico). 

\vspace{-2mm}
\subsection{Machine learning}

Brainstormers' effort focussed on reinforcement learning methods aiming at a universal machine learning system, where the agents learn to generate the appropriate behaviors to satisfy the most general objective of ``winning the match''. Unfortunately, as has been acknowledged \cite{gabel2008brainstormers}, ``even from very optimistic complexity estimations it becomes obvious, that in the soccer simulation domain, both conventional solution methods and also advanced today's reinforcement learning techniques come to their limit --– there are more than $(108 \times 50)^{23}$ different states and more than $(1000)^{300}$ different policies per agent per half time''.  

The high dimensionality of the search space motivated Brainstormers to use a multilayer perceptron neural network \cite{gabel2008brainstormers}: a feedforward artificial neural network which utilises a supervised learning technique called backpropagation for training the network. Rather than developing a universal learning system, Brainstormers succeeded in decomposing the problem into a number of individual behaviours (e.g., NeuroKick, NeuroIntercept, NeuroHassle) and tactics (e.g., NeuroAttack2vs2, NeuroAttack3vs4, NeuroAttack7vs8), learned with supervised learning techniques.  

Recently, there has been some renewed interest in backpropagation networks due to the successes of deep learning.  In our view, the potential of reinforcement learning methods in RoboCup has not yet been fully realised, and deep learning may yet to become a disruptive innovation for the Simulation league.


\subsection{Automated planning}

WrightEagle team addressed the challenges of (i) high dimensionality of the search space and (ii) the limited computation time available in each decision cycle, by using Markov Decision Processes (MDPs). The developed framework decomposes a given MDP into a set of sub-MDPs arranged over a hierarchical structure, and includes heuristics approximating online planning techniques \cite{zhang2013decision}. WrightEagle approach abandoned ``the pursuit of absolute accuracy'' and divided the continuous soccer field into the discrete space, further subdividing it into the players' control areas according to geometric reachability.  The resultant structure enables automated planning, accelerating the search process and extending the search depth \cite{zhang2013decision}.

\vspace{-2mm}
\subsection{Opponent modelling}

``HELIOS'' team \cite{Akiyama2008,Akiyama2010} followed a similar path, targeting a decomposition of the problem space in developing an unsupervised learning method based on Constrained Delaunay Triangulation (CDT) \cite{chew1989constrained}. A Delaunay triangulation for a set $P$ of points in a plane is a triangulation $\cal{D}(P)$ such that no point in $P$ is inside the circumcircle of any triangle in $\cal{D}(P)$ (in CDT the circumcircle of some triangles  contains other triangles' vertices).  The method divides the soccer field into a set of triangles, which provide an input plane region for Neural Gas (NG) and Growing Neural Gas (GNG) methods. Specifically, the set $P_b$ of $N$ points represents specifically chosen positions of the ball on the field, while sets $P_i$ describe the sets of coordinates of each player $1 \le i \le 11$, so that there is a bijective correspondence between $P_b$ and each of $P_i$. Moreover, when the ball takes any position within a triangle of ${\cal{D}}(P_b)$, each player's position is computed in a congruent way within ${\cal{D}}(P_i)$.   During offline experiments or even during a game, the behaviour of the opponent, for example,  the players' motion, directions of the passes, and the overall team formations, can be mapped, analysed and categorised \cite{Akiyama2008,Akiyama2010}.


It is evident that the main reason behind the recurrent successes of all three champion approaches is a dynamic decomposition of the problem space  and its subsequent efficient exploration. This innovation goes beyond a simple standardisation of  low-level behaviours within a rich but static configurational space, by employing automated learning and planning methods in a dynamic search.  


\section{Standardisation of ``hardware''}

An influential disruptive innovation arrived in 2010, when HELIOS team released a major update of their well-developed code base \cite{agent2d}:
\begin{itemize}
\item \emph{librcsc-4.0.0}: a base library for the RoboCup Soccer Simulator (RCSS);
\item \emph{agent2d-3.0.0}: a base source code for a team;
\item \emph{soccerwindow2-5.0.0}: a viewer and a visual debugger program for RCSS;
\item \emph{fedit2-2.0.0}: a team formation editor for \emph{agent2d}.
\end{itemize}
This resulted in nearly 80\% of the League's teams switching their code  base to agent2d over the next few years.  One may think of this phenomenon as a standardisation of the simulated hardware, freeing the effort to improving the higher-level tactical behaviours. 
 


\section{Gliders (2012 -- 2016): Fusing human innovation and artificial evolution}

We turn our attention to our champion team which won  RoboCup-2016 (Leipzig, Germany):  Gliders \cite{POWH12,gliders2013tdp,gliders2014tdp,gliders2015tdp,gliders2016tdp}.  Gliders2012 and Gliders2013 reached the semi-finals of RoboCup in 2012 and 2013;  Gliders2014 were runner-ups in 2014; Gliders2015 finished third in RoboCup-2015, and Gliders2016 (a joint effort of the University of Sydney and CSIRO) became world champions in 2016. 

RoboCup-2016 competition  included 18 teams from 9 countries: Australia, Brazil, China, Egypt,
Germany, Iran, Japan, Portugal and Romania. Gliders2016 played 23 games during several rounds, winning 19 times, losing twice and drawing twice, with the total score of 62:13, or 2.70 : 0.57 on average. In the two-game semi-final round, Gliders2016 defeated team CSU\_Yunlu from Central South University (China),  winning both games with the same score 2:1.  The single-game final against team HELIOS2016 (Japan) went into the extra time, and ended with Gliders2016 winning 2:1.  The third place was taken by team Ri-one from Ritsumeikan University (Japan).

The 2016 competition also included an evaluation round, where all 18 participating teams played one game each against the champion of RoboCup-2015, team WrightEagle (China).  Only two teams, the eventual finalists Gliders2016 and HELIOS2016, managed to win against the previous year champion, with Gliders defeating WrightEagle 1:0, and HELIOS producing the top score 2:1. 

The Gliders team code is written in C++ using \emph{agent2d-3.1.1} \cite{agent2d}, and fragments of source code of team MarliK released in 2012 \cite{marlik}. 

In order to optimise the code, the Gliders development effort over the last five years involved \emph{human-based evolutionary computation} (HBEC): a set of evolutionary computation techniques that rely on human innovation \cite{kosorukoff2001human,Cheng2004}. 

In general, evolutionary algorithms search a large space of possible solutions that together form a population. Each solution is a ``genotype'': a complex data structure representing the entire team behaviour encoded through a set of ``design points''. A design point can be as simple as a single parameter  (e.g., risk tolerance  in making a pass), or as complicated as a multi-agent tactical behaviour (e.g., a conditional statement describing the situation when a defender moves forward to produce an offside trap). 

Some design points are easy to vary. For instance, a formation defined via Delaunay Triangulations ${\cal{D}}(P_b)$ and ${\cal{D}}(P_i)$, $1 \le i \le 11$, is an ordered list of coordinates, and varying and recombining such a list can be relatively easily automated.  Other design points have an internal structure and are harder to permute.  For example, a conditional statement describing a tactic has a condition and an action, encoded by numerous parameters such as positional coordinates, state information, and action details. Once such a statement (a design point) is created by human designers, its encoding can be used by evolutionary algorithms. However, the inception of the tactic needs creative innovation in the first place, justifying the hybrid HBEC approach.      

The HBEC solutions representing team behaviours are evaluated with respect to their fitness, implemented as the average team performance, estimated over thousands of games for each generation played against a specific opponent. Some solutions are retained and recombined (i.e, the members of the population live) and some are removed (i.e., die) through selection.  Importantly, the evolutionary process is carried out within different landscapes (one per known opponent), and typically results in different solutions evolved to outperform specific opponents. In order to maintain coherence of the resultant code, each design point is implemented with a logical mask switching the corresponding part of the genotype on and off for specific opponents (determined by their team names). This is loosely analogous to epigenetic programming \cite{Tanev2008}.     

The approach is aimed at constantly improving performance from one artificial ``generation" to another, with team designers innovating and recombining behaviours while the fitness landscape and the mutations are for the most part automated. The performance of Gliders was evaluated on several supercomputer clusters, executing on some days tens of thousands of the experimental runs with different behaviour versions. It would be a fair estimate that the number of such trials is approaching 10 million.   The overall search-space explored by the HBEC includes variations in both Gliders behaviour and opponent modelling. The approach incorporates disruptive innovations of the past years, including the standardisation of simulated ``hardware'' and several effective search-space decompositions.
  
Specific variations included (i) action-dependent evaluation function, (ii) dynamic tactics with Voronoi diagrams, (iii) information dynamics, and (iv) bio-inspired collective behaviour.

The approach introduced in Gliders2012 \cite{POWH12} retained the advantages of a single evaluation metric (implemented in \emph{agent2d}  \cite{agent2d}), but diversified the evaluation by considering multiple points as desirable states.  These desirable states for action-dependent evaluation are computed using Voronoi diagrams which underlie many tactical schemes of Gliders. 

Starting from 2013, Gliders utilised information dynamics \cite{liz10e,Wang12,ay-robot,liz10d,cliff2013towards,liz08b} for tactical analysis and opponent modelling.  This analysis involves computation of information transfer and storage, relating the information transfer  to responsiveness of the players,  and the information storage within a team to the team's rigidity and lack of tactical richness.

The constraints on mobility, identified by the information dynamics, were investigated and partially overcome with bio-inspired  collective behaviour \cite{gliders2015tdp}.  Gliders2015 utilise several elements of swarm behavior, attempting to keep each player's position as close as possible to that suggested by a specific tactical scheme, while incorporating slight variations in order to maximise the chances of receiving the pass and/or shooting at the opponent's goal.   This behaviour increased the degree of coherent mobility: on the one hand, the players are constantly refining their positions in response to opponent players, but on the other hand, the repositioning is not erratic and the players move in coordinated ways.    

These directions were unified within a single development and evaluation framework which allowed to explore the search-space in two ways: translating human expertise into new behaviours and tactics, and exhaustively recombining them with an artificial evolution, leveraging the power of modern supercomputing. This fusion, we believe, produced a disruptive innovation on its own, providing the winning edge for Gliders.

  
\section{Conclusion}			
\vspace{-1mm}

In this paper we reviewed  disruptive innovations which affected advancement of the RoboCup 2D Soccer Simulation  League over the twenty years since its inception, and placed the progress of our champion team in this context. It is important to realise that the neither of these processes has been linear, and many ideas have been developing  along a spiral-shaped trajectory, resurfacing over the years in a different implementation. For example, the utility of evolutionary computation supported by supercomputing has been suggested as early as 1997, when a simulated team was developed with the agents whose high-level decision making
behaviors had been entirely evolved using genetic programming \cite{Luke98geneticprogramming}. Yet the complexity of the domain proved to be too challenging for this approach to gain a widespread adoption at that time.  

Without an exception, all the winning approaches combined elements of some automation (debugging, machine learning, planning, opponent modelling, evolutionary computation) with human-based innovation in terms of a decomposition of the search-space, providing various configurations, templates and structures. Is there still a way toward a fully automated solution, when the agents learn or evolve to play a competitive game without a detailed guidance from human designers, but rather by trying to satisfy a universal objective  (``win a game'')?  


On the one hand, the ability to run a massive number of simulated games on supercomputing clusters producing replicable results will only strengthen in time \cite{RAM15}, and so may lend some hope in meeting this challenge positively. On the other hand, the enormous size and dimensionality of the search-space  would defy any unstructured exploration strategy.  A methodology successfully resolving this dilemma may not only provide an ultimate disruptive innovation  in the League, but also provide a major breakthrough in the general AI research. 

	
\vspace{-3mm}
\section*{Acknowledgments}
\vspace{-2mm}

Several people contributed to Cyberoos and Gliders development over the years. Marc Butler, Thomas Howard and Ryszard Kowalczyk made exceptionally valuable  contributions to Cyberoos' effort during 1998 -- 2002 \cite{prokopenko1998designing,Butler2001,Prokopenko2001,ProkopenkoWH01}. We are grateful to Gliders team members Oliver Obst, particularly for establishing the tournament infrastructure supporting the team's performance evaluation on CSIRO Accelerator Cluster (Bragg), and Victor Jauregui, for several important insights on soccer tactics used in Gliders2016 \cite{gliders2016tdp}.  We thank David Budden for developing a new self-localisation method introduced in Gliders2013 \cite{budden2013particle,gliders2013tdp} as well as contributing to the analysis of competition formats \cite{RAM15}, and Oliver Cliff for developing a new communication scheme adopted by Gliders from 2014 \cite{gliders2014tdp}.  The overall effort has also benefited from the study quantifying tactical interaction networks, carried out in collaboration with Oliver Cliff, Joseph T. Lizier, X. Rosalind Wang and Oliver Obst \cite{cliff2013towards}. We are thankful to Ivan Duong, Edward Moore and Jason Held for their contribution to Gliders2012 \cite{POWH12}.  Gliders team logo was created by Matthew Chadwick. 



%
% ---- Bibliography ----
%

\bibliographystyle{splncs}

\begin{thebibliography}{10}

\bibitem{Asada99}
Asada, M., Kitano, H., Noda, I., Veloso, M.:
\newblock {R}obo{C}up: {T}oday and tomorrow -- {W}hat we have have learned.
\newblock Artificial Intelligence \textbf{110} (1999)  193--214

\bibitem{Kitano97}
Kitano, H., Tambe, M., Stone, P., Veloso, M.M., Coradeschi, S., Osawa, E.,
  Matsubara, H., Noda, I., Asada, M.:
\newblock {The RoboCup Synthetic Agent Challenge 97}.
\newblock In: RoboCup-97: Robot Soccer World Cup I.
\newblock Springer, London (1998)  62--73

\bibitem{rcsymp2014}
Budden, D., Wang, P., Obst, O., Prokopenko, M.:
\newblock Simulation leagues: Analysis of competition formats.
\newblock In Bianchi, R.A.C., Akin, H.L., Ramamoorthy, S., Sugiura, K., eds.:
  RoboCup 2014: Robot Soccer World Cup XVIII. Volume 8992 of Lecture Notes in
  Computer Science., Springer (2014)  183--194

\bibitem{RAM15}
Budden, D.M., Wang, P., Obst, O., Prokopenko, M.:
\newblock Robocup simulation leagues: Enabling replicable and robust
  investigation of complex robotic systems.
\newblock IEEE Robotics and Automation Magazine \textbf{22}(3) (2015)  140--146

\bibitem{Noda2003}
Noda, I., Stone, P.:
\newblock {The RoboCup Soccer Server and CMUnited Clients: Implemented
  Infrastructure for MAS Research}.
\newblock Autonomous Agents and Multi-Agent Systems \textbf{7}(1--2)
  (July--September 2003)  101--120

\bibitem{haker2001method}
Haker, M., Meyer, A., Polani, D., Martinetz, T.:
\newblock A method for incorporation of new evidence to improve world state
  estimation.
\newblock In Birk, A., Coradeschi, S., Tadokoro, S., eds.: RoboCup 2001: Robot
  Soccer World Cup {V}. Volume 2377 of Lecture Notes in Computer Science.,
  Springer (2002)  362--367

\bibitem{Stone99}
Stone, P., Veloso, M.:
\newblock Task decomposition, dynamic role assignment, and low-bandwidth
  communication for real-time strategic teamwork.
\newblock Artificial Intelligence \textbf{110}(2) (1999)  241--273

\bibitem{ATAL2000}
Riley, P., Stone, P., Veloso, M.:
\newblock Layered disclosure: Revealing agents' internals.
\newblock In Castelfranchi, C., Lesperance, Y., eds.: Intelligent Agents VII.
  Agent Theories, Architectures, and Languages -- 7th International Workshop,
  ATAL-2000, Boston, MA, USA, July 7--9, 2000. Lecture Notes in Artificial
  Intelligence.
\newblock Springer (2001)

\bibitem{Butler2001}
Butler, M., Prokopenko, M., Howard, T.:
\newblock Flexible synchronisation within {RoboCup} environment: A comparative
  analysis.
\newblock In Stone, P., Balch, T., Kraetzschmar, G., eds.: RoboCup 2000: Robot
  Soccer World Cup IV. Volume 2019 of Lecture Notes in Artificial Intelligence.
\newblock Springer, Berlin, Heidelberg (2001)  119--128

\bibitem{IAAI2000}
Stone, P., Riley, P., Veloso, M.:
\newblock Defining and using ideal teammate and opponent models.
\newblock In: Proceedings of the Twelfth Annual Conference on Innovative
  Applications of Artificial Intelligence. (2000)

\bibitem{Reis2001}
Reis, L.P., Lau, N., Oliveira, E.:
\newblock Situation based strategic positioning for coordinating a team of
  homogeneous agents.
\newblock In: Balancing Reactivity and Social Deliberation in Multi-Agent
  Systems, From RoboCup to Real-World Applications.
\newblock Springer, London, UK (2001)  175--197

\bibitem{NodaSMAK98}
Noda, I., Suzuki, S., Matsubara, H., Asada, M., Kitano, H.:
\newblock Robocup-97: The first robot world cup soccer games and conferences.
\newblock AI Magazine \textbf{19}(3) (1998)  49--59

\bibitem{LNAI99-simulator}
Stone, P., Riley, P., Veloso, M.:
\newblock The {CMU}nited-99 champion simulator team.
\newblock In Veloso, M., Pagello, E., Kitano, H., eds.: {R}obo{C}up-99: Robot
  Soccer World Cup {III}. Volume 1856 of Lecture Notes in Artificial
  Intelligence.
\newblock Springer (2000)  35--48

\bibitem{StoneABFKLSTW00}
Stone, P., Asada, M., Balch, T.R., Fujita, M., Kraetzschmar, G.K., Lund, H.H.,
  Scerri, P., Tadokoro, S., Wyeth, G.:
\newblock {Overview of RoboCup-2000}.
\newblock In Stone, P., Balch, T.R., Kraetzschmar, G.K., eds.: RoboCup 2000:
  Robot Soccer World Cup {IV}. Volume 2019 of Lecture Notes in Computer
  Science., Springer (2000)  1--28

\bibitem{ReisL00}
Reis, L.P., Lau, N.:
\newblock {FC Portugal Team Description: RoboCup 2000 Simulation League
  Champion}.
\newblock In Stone, P., Balch, T.R., Kraetzschmar, G.K., eds.: RoboCup 2000:
  Robot Soccer World Cup IV. Volume 2019 of Lecture Notes in Computer Science.,
  Springer (2001)  29--40

\bibitem{P-008-M65}
Reis, L., Lau, N.:
\newblock {COACH UNILANG - A Standard Language for Coaching a (Robo)Soccer
  Team}.
\newblock In Birk, A., Coradeschi, S., Tadokoro, S., eds.: {Robocup 2001: Robot
  Soccer World Cup V}. Volume {2377} of {Lecture Notes in Computer Science}.,
  Springer ({2001})  {183--192}

\bibitem{Jinyi2004}
Jinyi, Y., Ni, L., Fan, Y., Yunpeng, C., Zengqi, S.:
\newblock {Technical Solutions of TsinghuAeolus for Robotic Soccer}.
\newblock In Polani, D., Browning, B., Bonarini, A., Yoshida, K., eds.: RoboCup
  2003: Robot Soccer World Cup VII, Springer (2004)  205--213

\bibitem{Kok03robocup}
Kok, J.R., Vlassis, N., Groen, F.:
\newblock {U}v{A} {T}rilearn 2003 team description.
\newblock In Polani, D., Browning, B., Bonarini, A., Yoshida, K., eds.:
  Proceedings CD RoboCup 2003, Padua, Italy, Springer (July 2003)

\bibitem{prokopenko1998designing}
Prokopenko, M., Kowalczyk, R., Lee, M., Wong, W.Y.:
\newblock Designing and modelling situated agents systematically: CyberoosҠ98.
\newblock In: Proceedings of the PRICAI-98 Workshop on RoboCup. (1998)  75--89

\bibitem{Prokopenko2001}
Prokopenko, M., Butler, M., Howard, T.:
\newblock On emergence of scalable tactical and strategic behaviour.
\newblock In Stone, P., Balch, T., Kraetzschmar, G., eds.: RoboCup 2000: Robot
  Soccer World Cup IV. Volume 2019 of Lecture Notes in Artificial Intelligence.
\newblock Springer, Berlin, Heidelberg (2001)  357--366

\bibitem{ProkopenkoWH01}
Prokopenko, M., Wang, P., Howard, T.:
\newblock {Cyberoos'2001: ``Deep Behaviour Projection'' Agent Architecture}.
\newblock In Birk, A., Coradeschi, S., Tadokoro, S., eds.: RoboCup 2001: Robot
  Soccer World Cup {V}. Volume 2377 of Lecture Notes in Computer Science.,
  Springer (2002)  507--510

\bibitem{Prokopenko02}
Prokopenko, M., Wang, P.:
\newblock Relating the entropy of joint beliefs to multi-agent coordination.
\newblock In Kaminka, G.A., Lima, P.U., Rojas, R., eds.: RoboCup 2002: Robot
  Soccer World Cup VI. Volume 2752 of Lecture Notes in Computer Science.,
  Springer (2003)  367--374

\bibitem{Prokopenko03}
Prokopenko, M., Wang, P.:
\newblock Evaluating team performance at the edge of chaos.
\newblock In Polani, D., Browning, B., Bonarini, A., Yoshida, K., eds.: RoboCup
  2003: Robot Soccer World Cup VII. Volume 3020 of Lecture Notes in Computer
  Science., Springer (2004)  89--101

\bibitem{nehaniv2005evolutionary}
Nehaniv, C., Polani, D., Olsson, L., Klyubin, A.:
\newblock Evolutionary information-theoretic foundations of sensory ecology:
  Channels of organism-specific meaningful information.
\newblock Modeling Biology: Structures, Behaviour, Evolution (2005)  9--11

\bibitem{prokopenko06-alife}
Prokopenko, M., Gerasimov, V., Tanev, I.:
\newblock Measuring spatiotemporal coordination in a modular robotic system.
\newblock In Rocha, L., Yaeger, L., Bedau, M., Floreano, D., Goldstone, R.,
  Vespignani, A., eds.: Artificial Life X: Proceedings of The 10th
  International Conference on the Simulation and Synthesis of Living Systems,
  Bloomington IN, USA (2006)  185--191

\bibitem{sab06}
Prokopenko, M., Gerasimov, V., Tanev, I.:
\newblock Evolving spatiotemporal coordination in a modular robotic system.
\newblock In Nolfi, S., Baldassarre, G., Calabretta, R., Hallam, J.C.T.,
  Marocco, D., Meyer, J.A., Miglino, O., Parisi, D., eds.: From Animals to
  Animats 9: 9th International Conference on the Simulation of Adaptive
  Behavior (SAB 2006), Rome, Italy, September 25-29 2006. Volume 4095 of
  Lecture notes in computer science. (2006)  558--569

\bibitem{gabel2008brainstormers}
Riedmiller, M., Gabel, T., Trost, F., Schwegmann, T.:
\newblock {Brainstormers 2D -- Team Description 2008}.
\newblock In: RoboCup 2008: Robot Soccer World Cup XII; CD. (2008)

\bibitem{zhang2013decision}
Zhang, H., Chen, X.:
\newblock {The decision-making framework of WrightEagle, the RoboCup 2013
  soccer simulation 2D league champion team}.
\newblock In: Robot Soccer World Cup, Springer (2013)  114--124

\bibitem{Akiyama2008}
Akiyama, H., Noda, I.:
\newblock Multi-agent positioning mechanism in the dynamic environment.
\newblock In Visser, U., Ribeiro, F., Ohashi, T., Dellaert, F., eds.: {RoboCup
  2007: Robot Soccer World Cup XI}.
\newblock Springer, Berlin, Heidelberg (2008)  377--384

\bibitem{Akiyama2010}
Akiyama, H., Shimora, H.:
\newblock Helios2010 team description.
\newblock In: RoboCup 2010: Robot Soccer World Cup XIV; CD. (2010)

\bibitem{chew1989constrained}
Chew, L.P.:
\newblock {Constrained Delaunay Triangulations}.
\newblock Algorithmica \textbf{4}(1-4) (1989)  97--108

\bibitem{agent2d}
Akiyama, H.:
\newblock {Agent2D Base Code}.
\newblock \url{http://www.rctools.sourceforge.jp} (2010)

\bibitem{POWH12}
Prokopenko, M., Obst, O., Wang, P., Held, J.:
\newblock {Gliders2012: Tactics with action-dependent evaluation functions}.
\newblock In: RoboCup 2012 Symposium and Competitions: Team Description Papers,
  Mexico City, Mexico, June 2012. (2012)

\bibitem{gliders2013tdp}
Prokopenko, M., Obst, O., Wang, P., Budden, D., Cliff, O.:
\newblock {Gliders2013: Tactical analysis with information dynamics}.
\newblock In: RoboCup 2013 Symposium and Competitions: Team Description Papers,
  Eindhoven, The Netherlands, June 2013. (2013)

\bibitem{gliders2014tdp}
Prokopenko, M., Obst, O., Wang, P.:
\newblock {Gliders2014: Dynamic Tactics with Voronoi Diagrams}.
\newblock In: RoboCup 2014 Symposium and Competitions: Team Description Papers,
  Joao Pessoa, Brazil, July 2014. (2014)

\bibitem{gliders2015tdp}
Prokopenko, M., Wang, P., Obst, O.:
\newblock {Gliders2015: Opponent avoidance with bio-inspired flocking
  behaviour}.
\newblock In: RoboCup 2015 Symposium and Competitions: Team Description Papers,
  Hefei, China, July 2015. (2015)

\bibitem{gliders2016tdp}
Prokopenko, M., Wang, P., Obst, O., Jaurgeui, V.:
\newblock {Gliders2016: Integrating multi-agent approaches to tactical
  diversity}.
\newblock In: RoboCup 2016 Symposium and Competitions: Team Description Papers,
  Leipzig, Germany, July 2016. (2016)

\bibitem{marlik}
Tavafi, A., Nozari, N., Vatani, R., Yousefi, M.R., Rahmatinia, S., Pirdir, P.:
\newblock {MarliK 2012 Soccer 2D Simulation Team Description Paper}.
\newblock In: RoboCup 2012 Symposium and Competitions: Team Description Papers,
  Mexico City, Mexico. (2012)

\bibitem{kosorukoff2001human}
Kosorukoff, A.:
\newblock Human based genetic algorithm.
\newblock In: Systems, Man, and Cybernetics, 2001 IEEE International Conference
  on. Volume~5., IEEE (2001)  3464--3469

\bibitem{Cheng2004}
Cheng, C.D., Kosorukoff, A.:
\newblock Interactive one-max problem allows to compare the performance of
  interactive and human-based genetic algorithms.
\newblock In Deb, K., ed.: Genetic and Evolutionary Computation -- GECCO 2004:
  Genetic and Evolutionary Computation Conference, Seattle, USA, June 26-30,
  2004.
\newblock Springer (2004)  983--993

\bibitem{Tanev2008}
Tanev, I., Yuta, K.:
\newblock Epigenetic programming: Genetic programming incorporating epigenetic
  learning through modification of histones.
\newblock Information Sciences \textbf{178}(23) (2008)  4469--4481

\bibitem{liz10e}
Lizier, J.T., Prokopenko, M., Zomaya, A.Y.:
\newblock Information modification and particle collisions in distributed
  computation.
\newblock Chaos \textbf{20}(3) (2010)  037109

\bibitem{Wang12}
Wang, X.R., Miller, J.M., Lizier, J.T., Prokopenko, M., Rossi, L.F.:
\newblock Quantifying and tracing information cascades in swarms.
\newblock PLoS ONE \textbf{7}(7) (2012)  e40084

\bibitem{ay-robot}
Ay, N., Bernigau, H., Der, R., Prokopenko, M.:
\newblock Information-driven self-organization: the dynamical system approach
  to autonomous robot behavior.
\newblock Theory in Biosciences \textbf{131} (2012)  161--179

\bibitem{liz10d}
Lizier, J.T., Prokopenko, M., Zomaya, A.Y.:
\newblock Coherent information structure in complex computation.
\newblock Theory in Biosciences \textbf{131} (2012)  193--203

\bibitem{cliff2013towards}
Cliff, O., Lizier, J., Wang, R., Wang, P., Obst, O., Prokopenko, M.:
\newblock Towards quantifying interaction networks in a football match.
\newblock In Behnke, S., Veloso, M., Visser, A., Xiong, R., eds.: RoboCup 2013:
  Robot Soccer World Cup XVII, Springer (2013)  1--12

\bibitem{liz08b}
Lizier, J.T., Prokopenko, M., Zomaya, A.Y.:
\newblock A framework for the local information dynamics of distributed
  computation in complex systems.
\newblock In Prokopenko, M., ed.: {Guided Self-Organization: Inception}.
\newblock Springer (2014)  115--158

\bibitem{Luke98geneticprogramming}
Luke, S.:
\newblock {Genetic Programming Produced Competitive Soccer Softbot Teams for
  RoboCup97}.
\newblock In Koza, J.R., Banzhaf, W., Chellapilla, K., Deb, K., Dorigo, M.,
  Fogel, D.B., Garzon, M.H., Goldberg, D.E., Iba, H., Riolo, R.L., eds.:
  Proceedings of the 3rd Annual Genetic Programming Conf., Morgan Kaufmann
  (1998)  214--222

\bibitem{budden2013particle}
Budden, D., Prokopenko, M.:
\newblock Improved particle filtering for pseudo-uniform belief distributions
  in robot localisation.
\newblock In Behnke, S., Veloso, M.M., Visser, A., Xiong, R., eds.: RoboCup
  2013: Robot Soccer World Cup XVII. Volume 8371 of Lecture Notes in Artificial
  Intelligence., Springer (2013)

\end{thebibliography}



\end{document}

1612.00947.extracted_bib
{Asada99 Asada, M., Kitano, H., Noda, I., Veloso, M.: \newblock {Robo{Cup: {Today and tomorrow -- {What we have have learned. \newblock Artificial Intelligence 110 (1999) 193--214
1612.00947.extracted_bib
{Kitano97 Kitano, H., Tambe, M., Stone, P., Veloso, M.M., Coradeschi, S., Osawa, E., Matsubara, H., Noda, I., Asada, M.: \newblock {The RoboCup Synthetic Agent Challenge 97. \newblock In: RoboCup-97: Robot Soccer World Cup I. \newblock Springer, London (1998) 62--73
1612.00947.extracted_bib
{rcsymp2014 Budden, D., Wang, P., Obst, O., Prokopenko, M.: \newblock Simulation leagues: Analysis of competition formats. \newblock In Bianchi, R.A.C., Akin, H.L., Ramamoorthy, S., Sugiura, K., eds.: RoboCup 2014: Robot Soccer World Cup XVIII. Volume 8992 of Lecture Notes in Computer Science., Springer (2014) 183--194
1612.00947.extracted_bib
{RAM15 Budden, D.M., Wang, P., Obst, O., Prokopenko, M.: \newblock Robocup simulation leagues: Enabling replicable and robust investigation of complex robotic systems. \newblock IEEE Robotics and Automation Magazine 22(3) (2015) 140--146
1612.00947.extracted_bib
{Noda2003 Noda, I., Stone, P.: \newblock {The RoboCup Soccer Server and CMUnited Clients: Implemented Infrastructure for MAS Research. \newblock Autonomous Agents and Multi-Agent Systems 7(1--2) (July--September 2003) 101--120
1612.00947.extracted_bib
{haker2001method Haker, M., Meyer, A., Polani, D., Martinetz, T.: \newblock A method for incorporation of new evidence to improve world state estimation. \newblock In Birk, A., Coradeschi, S., Tadokoro, S., eds.: RoboCup 2001: Robot Soccer World Cup {V. Volume 2377 of Lecture Notes in Computer Science., Springer (2002) 362--367
1612.00947.extracted_bib
{Stone99 Stone, P., Veloso, M.: \newblock Task decomposition, dynamic role assignment, and low-bandwidth communication for real-time strategic teamwork. \newblock Artificial Intelligence 110(2) (1999) 241--273
1612.00947.extracted_bib
{ATAL2000 Riley, P., Stone, P., Veloso, M.: \newblock Layered disclosure: Revealing agents' internals. \newblock In Castelfranchi, C., Lesperance, Y., eds.: Intelligent Agents VII. Agent Theories, Architectures, and Languages -- 7th International Workshop, ATAL-2000, Boston, MA, USA, July 7--9, 2000. Lecture Notes in Artificial Intelligence. \newblock Springer (2001)
1612.00947.extracted_bib
{Butler2001 Butler, M., Prokopenko, M., Howard, T.: \newblock Flexible synchronisation within {RoboCup environment: A comparative analysis. \newblock In Stone, P., Balch, T., Kraetzschmar, G., eds.: RoboCup 2000: Robot Soccer World Cup IV. Volume 2019 of Lecture Notes in Artificial Intelligence. \newblock Springer, Berlin, Heidelberg (2001) 119--128
1612.00947.extracted_bib
{IAAI2000 Stone, P., Riley, P., Veloso, M.: \newblock Defining and using ideal teammate and opponent models. \newblock In: Proceedings of the Twelfth Annual Conference on Innovative Applications of Artificial Intelligence. (2000)
1612.00947.extracted_bib
{Reis2001 Reis, L.P., Lau, N., Oliveira, E.: \newblock Situation based strategic positioning for coordinating a team of homogeneous agents. \newblock In: Balancing Reactivity and Social Deliberation in Multi-Agent Systems, From RoboCup to Real-World Applications. \newblock Springer, London, UK (2001) 175--197
1612.00947.extracted_bib
{NodaSMAK98 Noda, I., Suzuki, S., Matsubara, H., Asada, M., Kitano, H.: \newblock Robocup-97: The first robot world cup soccer games and conferences. \newblock AI Magazine 19(3) (1998) 49--59
1612.00947.extracted_bib
{LNAI99-simulator Stone, P., Riley, P., Veloso, M.: \newblock The {CMUnited-99 champion simulator team. \newblock In Veloso, M., Pagello, E., Kitano, H., eds.: {Robo{Cup-99: Robot Soccer World Cup {III. Volume 1856 of Lecture Notes in Artificial Intelligence. \newblock Springer (2000) 35--48
1612.00947.extracted_bib
{StoneABFKLSTW00 Stone, P., Asada, M., Balch, T.R., Fujita, M., Kraetzschmar, G.K., Lund, H.H., Scerri, P., Tadokoro, S., Wyeth, G.: \newblock {Overview of RoboCup-2000. \newblock In Stone, P., Balch, T.R., Kraetzschmar, G.K., eds.: RoboCup 2000: Robot Soccer World Cup {IV. Volume 2019 of Lecture Notes in Computer Science., Springer (2000) 1--28
1612.00947.extracted_bib
{ReisL00 Reis, L.P., Lau, N.: \newblock {FC Portugal Team Description: RoboCup 2000 Simulation League Champion. \newblock In Stone, P., Balch, T.R., Kraetzschmar, G.K., eds.: RoboCup 2000: Robot Soccer World Cup IV. Volume 2019 of Lecture Notes in Computer Science., Springer (2001) 29--40
1612.00947.extracted_bib
{P-008-M65 Reis, L., Lau, N.: \newblock {COACH UNILANG - A Standard Language for Coaching a (Robo)Soccer Team. \newblock In Birk, A., Coradeschi, S., Tadokoro, S., eds.: {Robocup 2001: Robot Soccer World Cup V. Volume {2377 of {Lecture Notes in Computer Science., Springer ({2001) {183--192
1612.00947.extracted_bib
{Jinyi2004 Jinyi, Y., Ni, L., Fan, Y., Yunpeng, C., Zengqi, S.: \newblock {Technical Solutions of TsinghuAeolus for Robotic Soccer. \newblock In Polani, D., Browning, B., Bonarini, A., Yoshida, K., eds.: RoboCup 2003: Robot Soccer World Cup VII, Springer (2004) 205--213
1612.00947.extracted_bib
{Kok03robocup Kok, J.R., Vlassis, N., Groen, F.: \newblock {Uv{A {Trilearn 2003 team description. \newblock In Polani, D., Browning, B., Bonarini, A., Yoshida, K., eds.: Proceedings CD RoboCup 2003, Padua, Italy, Springer (July 2003)
1612.00947.extracted_bib
{prokopenko1998designing Prokopenko, M., Kowalczyk, R., Lee, M., Wong, W.Y.: \newblock Designing and modelling situated agents systematically: CyberoosҠ98. \newblock In: Proceedings of the PRICAI-98 Workshop on RoboCup. (1998) 75--89
1612.00947.extracted_bib
{Prokopenko2001 Prokopenko, M., Butler, M., Howard, T.: \newblock On emergence of scalable tactical and strategic behaviour. \newblock In Stone, P., Balch, T., Kraetzschmar, G., eds.: RoboCup 2000: Robot Soccer World Cup IV. Volume 2019 of Lecture Notes in Artificial Intelligence. \newblock Springer, Berlin, Heidelberg (2001) 357--366
1612.00947.extracted_bib
{ProkopenkoWH01 Prokopenko, M., Wang, P., Howard, T.: \newblock {Cyberoos'2001: ``Deep Behaviour Projection'' Agent Architecture. \newblock In Birk, A., Coradeschi, S., Tadokoro, S., eds.: RoboCup 2001: Robot Soccer World Cup {V. Volume 2377 of Lecture Notes in Computer Science., Springer (2002) 507--510
1612.00947.extracted_bib
{Prokopenko02 Prokopenko, M., Wang, P.: \newblock Relating the entropy of joint beliefs to multi-agent coordination. \newblock In Kaminka, G.A., Lima, P.U., Rojas, R., eds.: RoboCup 2002: Robot Soccer World Cup VI. Volume 2752 of Lecture Notes in Computer Science., Springer (2003) 367--374
1612.00947.extracted_bib
{Prokopenko03 Prokopenko, M., Wang, P.: \newblock Evaluating team performance at the edge of chaos. \newblock In Polani, D., Browning, B., Bonarini, A., Yoshida, K., eds.: RoboCup 2003: Robot Soccer World Cup VII. Volume 3020 of Lecture Notes in Computer Science., Springer (2004) 89--101
1612.00947.extracted_bib
{nehaniv2005evolutionary Nehaniv, C., Polani, D., Olsson, L., Klyubin, A.: \newblock Evolutionary information-theoretic foundations of sensory ecology: Channels of organism-specific meaningful information. \newblock Modeling Biology: Structures, Behaviour, Evolution (2005) 9--11
1612.00947.extracted_bib
{prokopenko06-alife Prokopenko, M., Gerasimov, V., Tanev, I.: \newblock Measuring spatiotemporal coordination in a modular robotic system. \newblock In Rocha, L., Yaeger, L., Bedau, M., Floreano, D., Goldstone, R., Vespignani, A., eds.: Artificial Life X: Proceedings of The 10th International Conference on the Simulation and Synthesis of Living Systems, Bloomington IN, USA (2006) 185--191
1612.00947.extracted_bib
{sab06 Prokopenko, M., Gerasimov, V., Tanev, I.: \newblock Evolving spatiotemporal coordination in a modular robotic system. \newblock In Nolfi, S., Baldassarre, G., Calabretta, R., Hallam, J.C.T., Marocco, D., Meyer, J.A., Miglino, O., Parisi, D., eds.: From Animals to Animats 9: 9th International Conference on the Simulation of Adaptive Behavior (SAB 2006), Rome, Italy, September 25-29 2006. Volume 4095 of Lecture notes in computer science. (2006) 558--569
1612.00947.extracted_bib
{gabel2008brainstormers Riedmiller, M., Gabel, T., Trost, F., Schwegmann, T.: \newblock {Brainstormers 2D -- Team Description 2008. \newblock In: RoboCup 2008: Robot Soccer World Cup XII; CD. (2008)
1612.00947.extracted_bib
{zhang2013decision Zhang, H., Chen, X.: \newblock {The decision-making framework of WrightEagle, the RoboCup 2013 soccer simulation 2D league champion team. \newblock In: Robot Soccer World Cup, Springer (2013) 114--124
1612.00947.extracted_bib
{Akiyama2008 Akiyama, H., Noda, I.: \newblock Multi-agent positioning mechanism in the dynamic environment. \newblock In Visser, U., Ribeiro, F., Ohashi, T., Dellaert, F., eds.: {RoboCup 2007: Robot Soccer World Cup XI. \newblock Springer, Berlin, Heidelberg (2008) 377--384
1612.00947.extracted_bib
{Akiyama2010 Akiyama, H., Shimora, H.: \newblock Helios2010 team description. \newblock In: RoboCup 2010: Robot Soccer World Cup XIV; CD. (2010)
1612.00947.extracted_bib
{chew1989constrained Chew, L.P.: \newblock {Constrained Delaunay Triangulations. \newblock Algorithmica 4(1-4) (1989) 97--108
1612.00947.extracted_bib
{agent2d Akiyama, H.: \newblock {Agent2D Base Code. \newblock \url{http://www.rctools.sourceforge.jp (2010)
1612.00947.extracted_bib
{POWH12 Prokopenko, M., Obst, O., Wang, P., Held, J.: \newblock {Gliders2012: Tactics with action-dependent evaluation functions. \newblock In: RoboCup 2012 Symposium and Competitions: Team Description Papers, Mexico City, Mexico, June 2012. (2012)
1612.00947.extracted_bib
{gliders2013tdp Prokopenko, M., Obst, O., Wang, P., Budden, D., Cliff, O.: \newblock {Gliders2013: Tactical analysis with information dynamics. \newblock In: RoboCup 2013 Symposium and Competitions: Team Description Papers, Eindhoven, The Netherlands, June 2013. (2013)
1612.00947.extracted_bib
{gliders2014tdp Prokopenko, M., Obst, O., Wang, P.: \newblock {Gliders2014: Dynamic Tactics with Voronoi Diagrams. \newblock In: RoboCup 2014 Symposium and Competitions: Team Description Papers, Joao Pessoa, Brazil, July 2014. (2014)
1612.00947.extracted_bib
{gliders2015tdp Prokopenko, M., Wang, P., Obst, O.: \newblock {Gliders2015: Opponent avoidance with bio-inspired flocking behaviour. \newblock In: RoboCup 2015 Symposium and Competitions: Team Description Papers, Hefei, China, July 2015. (2015)
1612.00947.extracted_bib
{gliders2016tdp Prokopenko, M., Wang, P., Obst, O., Jaurgeui, V.: \newblock {Gliders2016: Integrating multi-agent approaches to tactical diversity. \newblock In: RoboCup 2016 Symposium and Competitions: Team Description Papers, Leipzig, Germany, July 2016. (2016)
1612.00947.extracted_bib
{marlik Tavafi, A., Nozari, N., Vatani, R., Yousefi, M.R., Rahmatinia, S., Pirdir, P.: \newblock {MarliK 2012 Soccer 2D Simulation Team Description Paper. \newblock In: RoboCup 2012 Symposium and Competitions: Team Description Papers, Mexico City, Mexico. (2012)
1612.00947.extracted_bib
{kosorukoff2001human Kosorukoff, A.: \newblock Human based genetic algorithm. \newblock In: Systems, Man, and Cybernetics, 2001 IEEE International Conference on. Volume~5., IEEE (2001) 3464--3469
1612.00947.extracted_bib
{Cheng2004 Cheng, C.D., Kosorukoff, A.: \newblock Interactive one-max problem allows to compare the performance of interactive and human-based genetic algorithms. \newblock In Deb, K., ed.: Genetic and Evolutionary Computation -- GECCO 2004: Genetic and Evolutionary Computation Conference, Seattle, USA, June 26-30, 2004. \newblock Springer (2004) 983--993
1612.00947.extracted_bib
{Tanev2008 Tanev, I., Yuta, K.: \newblock Epigenetic programming: Genetic programming incorporating epigenetic learning through modification of histones. \newblock Information Sciences 178(23) (2008) 4469--4481
1612.00947.extracted_bib
{liz10e Lizier, J.T., Prokopenko, M., Zomaya, A.Y.: \newblock Information modification and particle collisions in distributed computation. \newblock Chaos 20(3) (2010) 037109
1612.00947.extracted_bib
{Wang12 Wang, X.R., Miller, J.M., Lizier, J.T., Prokopenko, M., Rossi, L.F.: \newblock Quantifying and tracing information cascades in swarms. \newblock PLoS ONE 7(7) (2012) e40084
1612.00947.extracted_bib
{ay-robot Ay, N., Bernigau, H., Der, R., Prokopenko, M.: \newblock Information-driven self-organization: the dynamical system approach to autonomous robot behavior. \newblock Theory in Biosciences 131 (2012) 161--179
1612.00947.extracted_bib
{liz10d Lizier, J.T., Prokopenko, M., Zomaya, A.Y.: \newblock Coherent information structure in complex computation. \newblock Theory in Biosciences 131 (2012) 193--203
1612.00947.extracted_bib
{cliff2013towards Cliff, O., Lizier, J., Wang, R., Wang, P., Obst, O., Prokopenko, M.: \newblock Towards quantifying interaction networks in a football match. \newblock In Behnke, S., Veloso, M., Visser, A., Xiong, R., eds.: RoboCup 2013: Robot Soccer World Cup XVII, Springer (2013) 1--12
1612.00947.extracted_bib
{liz08b Lizier, J.T., Prokopenko, M., Zomaya, A.Y.: \newblock A framework for the local information dynamics of distributed computation in complex systems. \newblock In Prokopenko, M., ed.: {Guided Self-Organization: Inception. \newblock Springer (2014) 115--158
1612.00947.extracted_bib
{Luke98geneticprogramming Luke, S.: \newblock {Genetic Programming Produced Competitive Soccer Softbot Teams for RoboCup97. \newblock In Koza, J.R., Banzhaf, W., Chellapilla, K., Deb, K., Dorigo, M., Fogel, D.B., Garzon, M.H., Goldberg, D.E., Iba, H., Riolo, R.L., eds.: Proceedings of the 3rd Annual Genetic Programming Conf., Morgan Kaufmann (1998) 214--222
1612.00947.extracted_bib
{budden2013particle Budden, D., Prokopenko, M.: \newblock Improved particle filtering for pseudo-uniform belief distributions in robot localisation. \newblock In Behnke, S., Veloso, M.M., Visser, A., Xiong, R., eds.: RoboCup 2013: Robot Soccer World Cup XVII. Volume 8371 of Lecture Notes in Artificial Intelligence., Springer (2013)