  We discover the mechanism for the transition from self-segregation (into
opposing groups) to clustering (towards cautious behaviors) in the evolutionary
minority game (EMG). The mechanism is illustrated with a statistical mechanics
analysis of a simplified EMG involving three groups of agents: two groups of
opposing agents and one group of cautious agents. Two key factors affect the
population distribution of the agents. One is the market impact (the
self-interaction), which has been identified previously. The other is the
market inefficiency due to the short-time imbalance in the number of agents
using opposite strategies. Large market impact favors "extreme" players who
choose fixed strategies, while large market inefficiency favors cautious
players. The phase transition depends on the number of agents ($N$), the
reward-to-fine ratio ($R$), as well as the wealth reduction threshold ($d$) for
switching strategy. When the rate for switching strategy is large, there is
strong clustering of cautious agents. On the other hand, when $N$ is small, the
market impact becomes large, and the extreme behavior is favored.
