  AI systems may be better thought of as peers than as tools. This paper
explores applications of augmented collective intelligence (ACI) beneficial to
collaborative ideation. Design considerations are offered for an experiment
that evaluates the performance of hybrid human- AI collectives. The
investigation described combines humans and large language models (LLMs) to
ideate on increasingly complex topics. A promising real-time collection tool
called Polis is examined to facilitate ACI, including case studies from citizen
engagement projects in Taiwan and Bowling Green, Kentucky. The authors discuss
three challenges to consider when designing an ACI experiment: topic selection,
participant selection, and evaluation of results. The paper concludes that
researchers should address these challenges to conduct empirical studies of ACI
in collaborative ideation.
