In the paper entitled “A modality independent proto-organization of human multisensory areas”, we show that the superior temporal cortex and neighboring areas retain the ability to represent sensory correspondences across modalities, despite the complete lack of prior auditory or visual experience. Given this finding, we argue that the superior temporal cortex must be innately provided with a functional “scaffolding” enabling a shared neural representation of coherent auditory and visual sensory events. The relevance of this observation is that the demonstration of a preserved functional topography in the superior temporal cortex for basic audiovisual processing provides additional support to the theory that large portions of the human neocortex may possess a predetermined modality-independent architecture that sustains subsequent, experience-dependent functional specialization. In turn, this finding has important implications both for dissecting the “nature vs. nurture” contribution to human brain growth and functioning and for the development of novel educational and rehabilitative strategies in individuals who lack the sensory ability(ies).
The superior temporal cortex is acknowledged as being a pivotal hub for the processing of audiovisual information. Combining signals across different sensory modalities is essential for an efficient interaction with the external world. To do so, the brain must detect the information conveyed by different sensory inputs and couple coherent events in space and time. Although some evidence of basic multisensory computations appears to be present in newborns, the extent to which audiovisual experience is necessary for the superior temporal cortex to develop and become able to detect shared features between the two sensory streams remains unknown. The current study intended to address this issue by assessing the impact of sensory experience on the development of brain computations related to the processing of real-world events. We reasoned that adult individuals who specifically lack visual or auditory information due to congenital sensory loss represent an optimal model to determine to what extent sensory experience is a necessary prerequisite for developing brain functional properties.
In the study, we measured the similarity of brain responses between two groups of sensory-deprived (SD, with congenital lack of either sight or hearing) adults and two samples of typically-developed (TD) individuals, exposed to the auditory and/or visual versions of the same long-lasting narrative. Brain response was assessed using functional Magnetic Resonance Imaging (fMRI) while the subjects were watching and/or listening to the same, edited version of Walt Disney’s action movie “101 Dalmatians”. We adopted a methodological approach, called Intersubject Correlation (ISC) analysis, based on the concept that brain regions do “synchronize” across individuals when processing the same stimulus features. The figure below illustrates the experimental paradigm and the analytical approach. We reasoned that any evidence of synchronization within the superior temporal cortex across conditions and experimental groups would indicate that this region sustains shared representations of visual and auditory stimulus properties even when information from one of the two sensory channels is missing since birth. Additionally, we thoroughly detailed the sensory events (low-level perceptual and high-level semantic) driving the synchronization and characterized its temporal dynamics by depicting the chronotopic organization of multisensory regions.
Results demonstrated that the congenital absence of visual or auditory experience does not affect the ability of the superior temporal cortex to represent unimodal auditory and visual streams of the same perceptual events. This finding indicates that the functional architecture of the superior temporal cortex, based on the extraction of common basic features from auditory and visual signals, emerges despite the lack of audiovisual inputs since birth and thus irrespectively of postnatal audiovisual sensory experiences. This observation favors the hypothesis that the human superior temporal cortex is endowed with a functional scaffolding to process basic perceptual features that define sensory correspondences across audition and vision and also carries significant implications for developing innovative educational strategies in sensory-deprived individuals.
- Pietrini P, Furey ML, Ricciardi E, Gobbini MI, Wu H-WC, Cohen L, Guazzelli M, Haxby JV (2004). Supramodal category-related representations of objects in the human ventral visual pathway. Proc Natl Acad Sci (USA), 101:5658-5663.
- Beauchamp, M. S., Argall, B. D., Bodurka, J., Duyn, J. H., & Martin, A. (2004). Unraveling multisensory integration: patchy organization within human STS multisensory cortex. Nat Neurosci, 7(11), 1190-1192.
- Hasson, U., Nir, Y., Levy, I., Fuhrmann, G., & Malach, R. (2004). Intersubject synchronization of cortical activity during natural vision. Science, 303(5664), 1634-1640.
- Ricciardi, E., Bottari, D., Ptito, M., Röder, B., & Pietrini, P. (2020). The sensory-deprived brain as a unique tool to understand brain development and function. Neurosci Biobehav Revs, 108, 78-82.