I mention to Ponce that XDREAM’s images are really unsettling, as if they’ve been plucked from some deep recess of the uncanny valley. “Yes!” he laughs. “They are!” He thinks they’re so good at stimulating monkey visual neurons that they’re also tickling our cells in a way that makes us feel uncomfortable. If one could use XDREAM on human neurons, “would we find similar images or different, and what would we think of them?” he asks. “At the moment, that’s not something anyone can do. But it makes me wonder.”
Livingstone also wonders whether XDREAM’s disquieting output hints at why so many mythical creatures are exaggerated versions of familiar things. Visual neurons, it seems, like exaggeration: In previous studies, her team showed that face-selective cells will respond more strongly to caricatures than to actual faces. “I think that gargoyles and leprechauns, these archetypes that people imagine … there’s a basis in our brains for them,” she says.
Beyond being weird, the most striking thing about XDREAM’s images is that they’re mostly unrecognizable. The team probed 46 neurons across six monkeys, and a few face-like motifs aside, most of the resulting images were messes of color, texture, and shape, which didn’t fit into obvious buckets. “It is striking that cells that were thought to code for simple objects or object parts may in fact code for much more complex visual stimuli,” says Leyla Isik, a neuroscientist at Johns Hopkins University. “Some may find it unsatisfying that the generated images cannot be described easily in terms of semantic categories. This ‘limitation,’ however, may just be a reality of the complex nature of the primate visual cortex.”
Through these experiments, researchers are learning more not just about the brain itself, but also about how to simulate it. Many neuroscientists have developed artificial neural networks that can analyze images and recognize objects, ostensibly by doing something close to what the brain’s actual visual centers do. But how close?
To find out, Pouya Bashivan at MIT used one such neural network to create images that should, theoretically, stimulate an actual brain in particular ways. His colleagues, Kohitij Kar and James DiCarlo, then showed these synthetic images to monkeys to see whether they worked as predicted.
The results were encouraging, if mixed. The neural network succeeded in fashioning images that would stimulate specific neurons more strongly than natural photos. But it wasn’t as good at another task: exciting one neuron while suppressing all its neighbors. This varied scorecard suggests that the network isn’t yet capturing everything there is to capture about the visual system.
Read: The AI-art gold rush is here
Still, it’s capturing something. Bashivan’s team focused on a region that supposedly responds to simple curves, but the images that his network churned out included grids, lattices, and cinnamon-roll swirls. Much like XDREAM’s hallucinogenic not-quite-faces, these complex images suggest that our understanding of how the brain sees the world is too basic. “If we only go by the intuitions of human researchers, we might get it wrong,” says Bashivan. “We’ll do better if we have models that contain all the knowledge in the field.”
“As biologists, many of us are still skeptical that current neural networks are similar enough to the brain to model it reliably,” Ponce says. But like Bashivan, he thinks that such models are the way forward, and studies such as these will help improve them. “Both approaches are about understanding a black box: the brain,” he says. “Both methods are necessary.”