Brains, Universal Approximators? Only if Super-Turing—And They Are

Authors: Emmett Redd

Presentation type: Poster at SNUFA 2024 online workshop (5-6 Nov 2024)

Abstract

Some claim that brains follow the hard Church-Turing Thesis. This theoretically limits brains to the integer computation space. Universal Approximation depends on the real number space since there are always gaps between the rational numbers that a Turing machine can compute. In the 1990s, a researcher proved three theorems of recurrent neural networks that were super-Turing. One is physically possible via rational weights and stochastic signals. Its operation gives indirect access to the real number space. Spiking neural networks also have stochastic signals and rational weights (resulting from quantized neurotransmitter packets and charges). Noise affects both ion currents and spike timing. The noise size in both signal types is approximately one part in ten thousand. In exploration of the theorem, a computer study trained neural networks with two (logistic and Hénon) chaotic maps. The neural networks’ outputs had various sized uniform random numbers added. The sums were precision limited (for shortened completion time) and recurred. At small and large random number size, the computations did not reliably mimic chaos. In the logistic map, the size optimized to the accepted positive Lyapunov exponent at one part in thirty thousand—Hénon map, one part in five hundred. The study’s results have a range containing the biological neuron noise sizes, even though it used uniform random numbers in contrast to spiking networks having Gaussian noise. The study matching the Lyapunov exponents showed that the noise-enhanced, limited-precision recurrent neural networks were consistent with super-Turing operation. Biological neural networks have similar features as the physically realizable theorem and have found an optimum noise size to achieve super-Turing operation. Nonetheless, the Universal Approximation, achieved within a spiking network via stochastic resonance, collapses to rational approximation upon output.