Abstract
The human species seems to be endowed with the ability to rapidly discover the complex embedded structures present in the environment. This ability may be at the origin of the species-specific capacities for music, mathematics and natural language. We here study how the human brain stores sequences of events that exhibit varying levels of regularity and sequence information. I will present the results from the behavioural and brain imaging studies we ran to determine how visual sequences of spatial positions and auditory sequences made of two sounds are encoded. Results suggest that the sequences are internally compressed using an abstract, language-like code that captures their regularities. For visual sequences, behavior suggests that participants quickly discover the most compact description of each sequence provided by the postulated language, and that they use elementary geometrical rules to predict the next items. fMRI results show that brain activity in dorsal inferior prefrontal cortex correlates with the complexity provided by the language, namely the sequences’ minimal description length (MDL), while the right dorsolateral prefrontal cortex encodes the presence of embedded structures. To access the temporal unfolding of the successive brain representations involved in sequence encoding, we performed a MEG study. Using multivariate decoders, each successive location could be decoded from brain signals, and upcoming locations were anticipated prior to their actual onset. Crucially, sequences with lower MDL led to lower error rates and to increased anticipations. Furthermore, neural codes specific to the numerical and geometrical primitives of the postulated language could be detected, both in isolation and within the sequences. Does the postulated Language-of-Thought generalise to other types of sequences ? To answer this question, we exposed participants to a hierarchy of binary sound sequences of variable complexity, whose minimal description required transition probabilities, chunking, or nested structures. Occasional deviant sounds probed the participants’ knowledge of the sequence. We predicted that task difficulty and brain activity would be proportional to the MDL in our formal language. Furthermore, activity should increase with MDL for learned sequences, and decrease with MDL for deviants. These predictions were upheld in both fMRI and MEG, indicating that sequence predictions are highly dependent on sequence structure and become weaker and delayed as complexity increases. The proposed language recruited bilateral superior temporal, precentral, anterior intraparietal and cerebellar cortices. We propose that these areas collectively encode regular sequences as repetitions with variations and their recursive composition into nested structures. In conclusion, we note that for both visual spatial and binary auditory sequences, the regions that were involved in sequence encoding overlapped extensively with a localizer for mathematical calculation, and much less with spoken or written language processing.
Topic Areas : Syntax and Combinatorial Semantics, Computational Approaches