Model-based analysis of brain activity reveals the hierarchy of language in 305 subjects

10/12/2021
by   Charlotte Caucheteux, et al.
0

A popular approach to decompose the neural bases of language consists in correlating, across individuals, the brain responses to different stimuli (e.g. regular speech versus scrambled words, sentences, or paragraphs). Although successful, this `model-free' approach necessitates the acquisition of a large and costly set of neuroimaging data. Here, we show that a model-based approach can reach equivalent results within subjects exposed to natural stimuli. We capitalize on the recently-discovered similarities between deep language models and the human brain to compute the mapping between i) the brain responses to regular speech and ii) the activations of deep language models elicited by modified stimuli (e.g. scrambled words, sentences, or paragraphs). Our model-based approach successfully replicates the seminal study of Lerner et al. (2011), which revealed the hierarchy of language areas by comparing the functional-magnetic resonance imaging (fMRI) of seven subjects listening to 7min of both regular and scrambled narratives. We further extend and precise these results to the brain signals of 305 individuals listening to 4.1 hours of narrated stories. Overall, this study paves the way for efficient and flexible analyses of the brain bases of language.

READ FULL TEXT

page 3

page 9

research
06/03/2022

Toward a realistic model of speech processing in the brain with self-supervised learning

Several deep neural networks have recently been shown to generate activa...
research
03/02/2021

Decomposing lexical and compositional syntax and semantics with deep language models

The activations of language transformers like GPT2 have been shown to li...
research
02/25/2021

Inductive biases, pretraining and fine-tuning jointly account for brain responses to speech

Our ability to comprehend speech remains, to date, unrivaled by deep lea...
research
07/07/2022

Neural Language Models are not Born Equal to Fit Brain Data, but Training Helps

Neural Language Models (NLMs) have made tremendous advances during the l...
research
05/19/2017

Deep adversarial neural decoding

Here, we present a novel approach to solve the problem of reconstructing...
research
09/10/2020

Brain2Word: Decoding Brain Activity for Language Generation

Brain decoding, understood as the process of mapping brain activities to...
research
06/27/2019

Relating Simple Sentence Representations in Deep Neural Networks and the Brain

What is the relationship between sentence representations learned by dee...

Please sign up or login with your details

Forgot password? Click here to reset