  Working memory (WM), a fundamental cognitive process facilitating the
temporary storage, integration, manipulation, and retrieval of information,
plays a vital role in reasoning and decision-making tasks. Robust benchmark
datasets that capture the multifaceted nature of WM are crucial for the
effective development and evaluation of AI WM models. Here, we introduce a
comprehensive Working Memory (WorM) benchmark dataset for this purpose. WorM
comprises 10 tasks and a total of 1 million trials, assessing 4
functionalities, 3 domains, and 11 behavioral and neural characteristics of WM.
We jointly trained and tested state-of-the-art recurrent neural networks and
transformers on all these tasks. We also include human behavioral benchmarks as
an upper bound for comparison. Our results suggest that AI models replicate
some characteristics of WM in the brain, most notably primacy and recency
effects, and neural clusters and correlates specialized for different domains
and functionalities of WM. In the experiments, we also reveal some limitations
in existing models to approximate human behavior. This dataset serves as a
valuable resource for communities in cognitive psychology, neuroscience, and
AI, offering a standardized framework to compare and enhance WM models,
investigate WM's neural underpinnings, and develop WM models with human-like
capabilities. Our source code and data are available at
https://github.com/ZhangLab-DeepNeuroCogLab/WorM.
