Optimizing Neural Networks on a Brain-Inspired Computer

Many computational properties are maximized when the dynamics of a network are at a "critical point", a state where systems can quickly change their overall characteristics in fundamental ways, transitioning e.g. between order and chaos or stability and instability. Therefore, the critical state is widely assumed to be optimal for any computation in recurrent neural networks, which are used in many AI applications.

Researchers from the HBP partner Heidelberg University and the Max-Planck-Institute for Dynamics and Self-Organization challenged this assumption by testing the performance of a spiking recurrent neural network on a set of tasks with varying complexity at - and away from critical dynamics. They instantiated the network on a prototype of the analog neuromorphic BrainScaleS-2 system. BrainScaleS is a state-of-the-art brain-inspired computing system with synaptic plasticity implemented directly on the chip. It is one of two neuromorphic systems currently under development within the European Human Brain Project.

First, the researchers showed that the distance to criticality can be easily adjusted in the chip by changing the input strength, and then demonstrated a clear relation between criticality and task-performance. The assumption that criticality is beneficial for every task was not confirmed: whereas the information-theoretic measures all showed that network capacity was maximal at criticality, only the complex, memory intensive tasks profited from it, while simple tasks actually suffered. The study thus provides a more precise understanding of how the collective network state should be tuned to different task requirements for optimal performance.

Mechanistically, the optimal working point for each task can be set very easily under homeostatic plasticity by adapting the mean input strength. The theory behind this mechanism was developed very recently at the Max Planck Institute. "Putting it to work on neuromorphic hardware shows that these plasticity rules are very capable in tuning network dynamics to varying distances from criticality", says senior author Viola Priesemann, group leader at MPIDS. Thereby tasks of varying complexity can be solved optimally within that space.

The finding may also explain why biological neural networks operate not necessarily at criticality, but in the dynamically rich vicinity of a critical point, where they can tune their computation properties to task requirements. Furthermore, it establishes neuromorphic hardware as a fast and scalable avenue to explore the impact of biological plasticity rules on neural computation and network dynamics.

"As a next step, we now study and characterize the impact of the spiking network's working point on classifying artificial and real-world spoken words", says first author Benjamin Cramer of Heidelberg University.

Benjamin Cramer, David Stöckel, Markus Kreft, Michael Wibral, Johannes Schemmel, Karlheinz Meier, Viola Priesemann.
Control of criticality and computation in spiking neuromorphic networks with plasticity.
Nature Communications, 2020. doi: 10.1038/s41467-020-16548-3.

Most Popular Now

73,000 Scientists Collaborate over New C…

More than 73,000 users collaborate on new online platform set up by the European Open Science Cloud Initiative, where scientists share COVID-19 data and accelerate our understanding of the virus...

Bayer and Informed Data Systems Inc. (On…

Bayer and Informed Data Systems Inc. (One Drop), a US-based digital health company, today announced that they have entered into an agreement to jointly develop digital health products for multiple...

C3-Cloud: the Digital Coordinated Care P…

Typically, when a patient is receiving care from GPs and Hospitals, these are normally uncoordinated and the patient is often presented with conflicting advice, or clinicians are required to assess...

Philips Launches New Innovations to Help…

Royal Philips (NYSE: PHG, AEX: PHIA), a global leader in health technology, today presented new approaches to health and healthcare, and the shift to health-at-home, at its virtual consumer health...

Philips to Expand its Image-Guided Thera…

Royal Philips (NYSE: PHG, AEX: PHIA), a global leader in health technology, today announced that it has signed an agreement to acquire Intact Vascular, Inc., a U.S.-based developer of medical...

Contact Tracing Apps Unlikely to Contain…

Contract tracing apps used to reduce the spread of COVID-19 are unlikely to be effective without proper uptake and support from concurrent control measures, finds a new study by UCL...

Innovation Improves Care Delivery Around…

Allscripts Healthcare Solutions (NASDAQ: MDRX) was recently awarded five separate patents over the course of just eight days, reflecting the company's commitment to delivering unique solutions to its clients and...

Alcidion Appointed to NHSX Clinical Comm…

NHSX has awarded smart health informatics provider Alcidion a place on the Clinical Communications Procurement Framework, a new procurement vehicle intended to help the NHS phase out pagers by the...

Internet Searches for Anxiety Attacks Ta…

Many health experts are concerned that the COVID-19 pandemic could be having widespread effects on people's mental health, but assessing these concerns is difficult without data. "Traditional public health surveillance lacks...

Allscripts and Microsoft Extend Strategi…

Allscripts and Microsoft Corp. announced the extension of their long-standing strategic alliance to enable the expanded development and delivery of cloud-based health IT solutions. The five-year extension will support Allscripts...

Newly Merged CCG Expands Tech Support to…

Norfolk and Waveney Clinical Commissioning Group to support prescribers to improve the quality and safety of prescribing through deploying prescribing decision technology FDB OptimiseRx. A newly merged clinical commissioning group...

Anonymized Cell Phone Location Data can …

In March 2020, federal officials declared the COVID-19 outbreak a national emergency. Around the same time, most states implemented stay-at-home advisories - to different degrees and at different times. Publicly...