  As AI systems appear to exhibit ever-increasing capability and generality,
assessing their true potential and safety becomes paramount. This paper
contends that the prevalent evaluation methods for these systems are
fundamentally inadequate, heightening the risks and potential hazards
associated with AI. I argue that a reformation is required in the way we
evaluate AI systems and that we should look towards cognitive sciences for
inspiration in our approaches, which have a longstanding tradition of assessing
general intelligence across diverse species. We will identify some of the
difficulties that need to be overcome when applying cognitively-inspired
approaches to general-purpose AI systems and also analyse the emerging area of
"Evals". The paper concludes by identifying promising research pathways that
could refine AI evaluation, advancing it towards a rigorous scientific domain
that contributes to the development of safe AI systems.
