  Developments in the field of Artificial Intelligence (AI), and particularly
large language models (LLMs), have created a 'perfect storm' for observing
'sparks' of Artificial General Intelligence (AGI) that are spurious. Like
simpler models, LLMs distill meaningful representations in their latent
embeddings that have been shown to correlate with external variables.
Nonetheless, the correlation of such representations has often been linked to
human-like intelligence in the latter but not the former. We probe models of
varying complexity including random projections, matrix decompositions, deep
autoencoders and transformers: all of them successfully distill information
that can be used to predict latent or external variables and yet none of them
have previously been linked to AGI. We argue and empirically demonstrate that
the finding of meaningful patterns in latent spaces of models cannot be seen as
evidence in favor of AGI. Additionally, we review literature from the social
sciences that shows that humans are prone to seek such patterns and
anthropomorphize. We conclude that both the methodological setup and common
public image of AI are ideal for the misinterpretation that correlations
between model representations and some variables of interest are 'caused' by
the model's understanding of underlying 'ground truth' relationships. We,
therefore, call for the academic community to exercise extra caution, and to be
keenly aware of principles of academic integrity, in interpreting and
communicating about AI research outcomes.
