How does heterogeneity influence RNN’s performance and criticality

Authors: Keren Gao, Guozhang Chen

Presentation type: Poster at SNUFA 2024 online workshop (5-6 Nov 2024)

Abstract

Accumulating studies have shed light on the importance of neural heterogeneity on network performance, such as robustness. Additionally, there has been ample evidence that brains operate at a critical state, and neural networks’ performance can often be optimized at criticality. So we want to investigate whether the underlying mechanism of heterogeneity improving network performance is that heterogeneity would push the network towards criticality. So far, we have implemented a RNN based on reservoir computing with adjustable heterogeneity in membrane time constant and looked into how the order parameters implying criticality, such as eigenvalue spectrum radius, Lyapunov exponent and branching ratio, change with the degree of heterogeneity.