Optimizing Neural Networks on a Brain-Inspired Computer

Many computational properties are maximized when the dynamics of a network are at a "critical point", a state where systems can quickly change their overall characteristics in fundamental ways, transitioning e.g. between order and chaos or stability and instability. Therefore, the critical state is widely assumed to be optimal for any computation in recurrent neural networks, which are used in many AI applications.

Researchers from the HBP partner Heidelberg University and the Max-Planck-Institute for Dynamics and Self-Organization challenged this assumption by testing the performance of a spiking recurrent neural network on a set of tasks with varying complexity at - and away from critical dynamics. They instantiated the network on a prototype of the analog neuromorphic BrainScaleS-2 system. BrainScaleS is a state-of-the-art brain-inspired computing system with synaptic plasticity implemented directly on the chip. It is one of two neuromorphic systems currently under development within the European Human Brain Project.

First, the researchers showed that the distance to criticality can be easily adjusted in the chip by changing the input strength, and then demonstrated a clear relation between criticality and task-performance. The assumption that criticality is beneficial for every task was not confirmed: whereas the information-theoretic measures all showed that network capacity was maximal at criticality, only the complex, memory intensive tasks profited from it, while simple tasks actually suffered. The study thus provides a more precise understanding of how the collective network state should be tuned to different task requirements for optimal performance.

Mechanistically, the optimal working point for each task can be set very easily under homeostatic plasticity by adapting the mean input strength. The theory behind this mechanism was developed very recently at the Max Planck Institute. "Putting it to work on neuromorphic hardware shows that these plasticity rules are very capable in tuning network dynamics to varying distances from criticality", says senior author Viola Priesemann, group leader at MPIDS. Thereby tasks of varying complexity can be solved optimally within that space.

The finding may also explain why biological neural networks operate not necessarily at criticality, but in the dynamically rich vicinity of a critical point, where they can tune their computation properties to task requirements. Furthermore, it establishes neuromorphic hardware as a fast and scalable avenue to explore the impact of biological plasticity rules on neural computation and network dynamics.

"As a next step, we now study and characterize the impact of the spiking network's working point on classifying artificial and real-world spoken words", says first author Benjamin Cramer of Heidelberg University.

Benjamin Cramer, David Stöckel, Markus Kreft, Michael Wibral, Johannes Schemmel, Karlheinz Meier, Viola Priesemann.
Control of criticality and computation in spiking neuromorphic networks with plasticity.
Nature Communications, 2020. doi: 10.1038/s41467-020-16548-3.

Most Popular Now

Hull University Teaching Hospitals NHS T…

Hull University Teaching Hospitals NHS Trust has deployed DXC Technology's (NYSE:DXC) cloud-based Clinical Aide mobile application, which both improves secure access to electronic patient medical records and increases patient and...

Central Hospital in Finland Automates Sy…

This November Central Ostrobothnia Central Hospital in Finland has started symptom tracking and remote monitoring of corona virus-infected patients with the Buddy Healthcare mobile application and clinical monitoring platform. The...

Ultrapotent COVID-19 Vaccine Candidate D…

An innovative nanoparticle vaccine candidate for the pandemic coronavirus produces virus-neutralizing antibodies in mice at levels ten-times greater than is seen in people who have recovered from COVID-19 infections. Designed...

Interactive virtual reality emerges as a…

Bristol scientists have demonstrated a new virtual reality [VR] technique which should help in developing drugs against the SARS-CoV-2 virus - and enable researchers to share models and collaborate in...

More than 1,000 Exhibitors have Already …

16 - 19 November 2020, Düsseldorf, Germany. This year, the world's largest and leading medical trade fair, MEDICA, and the number one international event for the medical technology manufacturing supplier market...

Softphone App to Save NHS up to £20 Mill…

GPs and patients are to benefit from a new softphone app that will eliminate almost all the "hidden cost" of calls for remote consultations and is forecast to save the...

DMEA Newcomer Award: The Search is on fo…

13 - 15 April 2021, Berlin, Germany. More efficient, digital and patient-friendly! Tomorrow's healthcare system promises great things. And to ensure this becomes reality a fresh impetus and creative ideas are...

App to Help NHS Prevent Deadly Hospital …

Compass app to help NHS trusts prevent hospital-acquired acute kidney injury and pneumonia - both of which cause harm to COVID-19 patients and can be fatal. Hospitals could alleviate pressure...

Leeds Deploys Imaging Tech for Advanced …

A multi-million pound initiative that is digitising, connecting and applying artificial intelligence to NHS pathology services in the North of England has taken an important step, after Leeds Teaching Hospitals...

Survey of COVID-19 Research Provides Fre…

Researchers at Karolinska Institutet in Sweden have explored all COVID-19 research published during the initial phase of the pandemic. The results, which were achieved by using a machine learning-based approach...

COVID-19 High Performance Computing Cons…

The COVID-19 High Performance Computing (HPC) Consortium, a unique public-private effort to make supercomputing power available to researchers working on projects related to COVID-19, announced that it has entered into...