Unimodal versus bimodal learning and processing of audio-visual stimuli in spiking neural networks

Authors: Raphaël Bergoin,Alessandro Torcini,Gustavo Deco,Mathias Quoy,Mario Senden,Gorka Zamora-López

Presentation type: Poster at SNUFA 2026 online workshop (4-5 Nov 2026)

Abstract

Multisensory integration is a fundamental feature of cognition, enabling the brain to combine information from different sensory modalities into coherent representations. Despite its importance, the mechanisms underlying this process remain incompletely understood and continue to pose a challenge for biological and artificial systems. In this study, we investigate multisensory learning using a biologically inspired spiking neural network (SNN) architecture previously shown to self-organize and maintain modular structures. The network is trained on visual and auditory representations of digits, and its ability to encode, recognize, and generate sensory stimuli is evaluated in both unimodal and bimodal settings. Our results show that bimodal learning promotes the emergence of cross-modal connectivity and more selective multisensory representations than unimodal learning. Specifically, the network exhibits (i) a functional specialization of inhibitory interneuron subtypes that shape the attractor landscape, (ii) the emergence of synergistic hubs supporting multisensory integration, and (iii) a competitive decision-making mechanism driven by hub imbalance. Beyond bottom-up recognition, the proposed architecture also supports top-down generation of learned sensory representations. These findings highlight how biologically grounded network dynamics can give rise to key computational principles underlying multisensory processing.

Are you the author and do you want to add some related links to this page? If so, check out our guide to adding related links and videos.