

The semantic network that the researchers compiled and reported in Nature in 2016 - another patchwork map, a mosaic of meaning that tiled large swaths of the cortex - was “a really new thing” at this scale and dimensionality, Gallant said. They had people listen to hours of podcast recordings, and then assessed how their brains responded to the hundreds of concepts they’d heard in those stories. He and his team decided to try something similar, only this time using language instead of visual stimuli. But it also turned up hundreds of other selective patches for the first time: regions that responded to images of animals, family members, indoor scenes, outdoor scenes, people in motion, and more. It confirmed the identity of some regions of the visual cortex that were already known from earlier research, such as areas selectively responsive to faces or places. They found remarkably consistent patterns in all the participants - patterns that formed a generalized map of visual meaning. Then, segmenting the data into records for roughly pea-size volumes of brain tissue called voxels, they analyzed the scans to determine where hundreds of objects and actions were represented across the cortex. Huth, Gallant and their colleagues had participants watch hours of silent videos while inside fMRI scanners. One of Gallant’s graduate students at the time, Alex Huth, used the Gallant lab’s cutting-edge techniques to analyze where the brain might encode different kinds of visual information. Because fMRI machines can measure changes in blood flow and electrical activity in the brain, neuroscientists often use them to study which parts of the cortex respond to different stimuli. In 2012, Jack Gallant, a computational and cognitive neuroscientist at the University of California, Berkeley, had spent the better part of a decade developing functional MRI (magnetic resonance imaging) tools and models for studying the human visual system. The new work on semantics was completely independent of the work on memory - the two unfolding at around the same time but on opposite sides of the United States. That insight might help to explain why memory is so often such an imperfect record of the past - and could provide a better understanding of what it really means to remember something. Instead, it is more like a reconstruction of the original experience, based on its semantic content. The finding, published last October in Nature Neuroscience, suggests that in many cases, a memory isn’t a facsimile of past perceptions that gets replayed. When they compared that map to one they made showing where the brain represents categories of visual information, they observed meaningful differences between the patterns.Īnd those differences looked exactly like the ones reported in the studies on vision and memory. A team of neuroscientists created a semantic map of the brain that showed in remarkable detail which areas of the cortex respond to linguistic information about a wide range of concepts, from faces and places to social relationships and weather phenomena.
