Optimizing Neural Networks on a Brain-Inspired Computer

Many computational properties are maximized when the dynamics of a network are at a "critical point", a state where systems can quickly change their overall characteristics in fundamental ways, transitioning e.g. between order and chaos or stability and instability. Therefore, the critical state is widely assumed to be optimal for any computation in recurrent neural networks, which are used in many AI applications.

Researchers from the HBP partner Heidelberg University and the Max-Planck-Institute for Dynamics and Self-Organization challenged this assumption by testing the performance of a spiking recurrent neural network on a set of tasks with varying complexity at - and away from critical dynamics. They instantiated the network on a prototype of the analog neuromorphic BrainScaleS-2 system. BrainScaleS is a state-of-the-art brain-inspired computing system with synaptic plasticity implemented directly on the chip. It is one of two neuromorphic systems currently under development within the European Human Brain Project.

First, the researchers showed that the distance to criticality can be easily adjusted in the chip by changing the input strength, and then demonstrated a clear relation between criticality and task-performance. The assumption that criticality is beneficial for every task was not confirmed: whereas the information-theoretic measures all showed that network capacity was maximal at criticality, only the complex, memory intensive tasks profited from it, while simple tasks actually suffered. The study thus provides a more precise understanding of how the collective network state should be tuned to different task requirements for optimal performance.

Mechanistically, the optimal working point for each task can be set very easily under homeostatic plasticity by adapting the mean input strength. The theory behind this mechanism was developed very recently at the Max Planck Institute. "Putting it to work on neuromorphic hardware shows that these plasticity rules are very capable in tuning network dynamics to varying distances from criticality", says senior author Viola Priesemann, group leader at MPIDS. Thereby tasks of varying complexity can be solved optimally within that space.

The finding may also explain why biological neural networks operate not necessarily at criticality, but in the dynamically rich vicinity of a critical point, where they can tune their computation properties to task requirements. Furthermore, it establishes neuromorphic hardware as a fast and scalable avenue to explore the impact of biological plasticity rules on neural computation and network dynamics.

"As a next step, we now study and characterize the impact of the spiking network's working point on classifying artificial and real-world spoken words", says first author Benjamin Cramer of Heidelberg University.

Benjamin Cramer, David Stöckel, Markus Kreft, Michael Wibral, Johannes Schemmel, Karlheinz Meier, Viola Priesemann.
Control of criticality and computation in spiking neuromorphic networks with plasticity.
Nature Communications, 2020. doi: 10.1038/s41467-020-16548-3.

Most Popular Now

IBM Watson Health Recognizes Top-Perform…

IBM (NYSE: IBM) Watson Health® announced its 2020 Fortune/IBM Watson Health 100 Top Hospitals list and 15 Top Health Systems award winners, naming the top-performing hospitals and health systems in...

Proposed Framework for Integrating Chatb…

While the technology for developing artificial intelligence-powered chatbots has existed for some time, a new viewpoint piece in JAMA lays out the clinical, ethical, and legal aspects that must be...

New Machine Learning Method Allows Hospi…

To answer medical questions that can be applied to a wide patient population, machine learning models rely on large, diverse datasets from a variety of institutions. However, health systems and...

Online Tools can Improve Autism Diagnosi…

Online tools and assessments can help speed up diagnosis of autism spectrum disorder (ASD), the first comprehensive survey of research in the field has concluded. The survey showed that using internet-based...

World's Smallest Imaging Device has Hear…

A team of researchers led by the University of Adelaide and University of Stuttgart has used 3D micro-printing to develop the world's smallest, flexible scope for looking inside blood vessels...

Optimizing Neural Networks on a Brain-In…

Many computational properties are maximized when the dynamics of a network are at a "critical point", a state where systems can quickly change their overall characteristics in fundamental ways, transitioning...

Clinical-Grade Wearables Offer Continuou…

Although it might be tempting to rely on your fitness tracker to catch early signs of COVID-19, Northwestern University researchers caution that consumer wearables are not sophisticated enough to monitor...

NHS Lanarkshire to Advance Scotland's Pa…

An early warning system that helps busy doctors, nurses and other healthcare professionals to quickly identify when patients are at risk of deterioration is to be deployed across hospitals and...

'Drawn-on-Skin' Electronics offer Breakt…

A team of researchers led by Cunjiang Yu, Bill D. Cook Associate Professor of Mechanical Engineering at the University of Houston, has developed a new form of electronics known as...

Philips Launches Pre-Hospital Wireless M…

Royal Philips (NYSE: PHG, AEX: PHIA), a global leader in health technology, today announced the launch of its remote monitoring and defibrillator solution (Tempus ALS) for pre-hospital settings in the...

Siilo Raises £8.6 Million to Enable Secu…

Siilo, the secure collaboration app for healthcare professionals, has raised a £8.6 million (€9.5m) in Series A funding. European digital health VC Heal Capital is at the forefront of the...