The research was published in the March-May 2017 issue of the IBM Journal of Research and Development. According to the University of California San Diego it lays the groundwork to develop realistic “biomimetic neuroprosthetics” -- brain implants that replicate brain circuits and their function -- that one day could replace lost or damaged brain cells or tissue from tumors, stroke, or other diseases.
“In patients with motor paralysis, the biomimetic neuroprosthetic could be used to replace the deteriorated motor cortex where it could interact directly with healthy brain pre-motor regions, and send commands and receive feedback via the spinal cord to a prosthetic arm,” said W.W. Lytton, a professor of physiology and pharmacology at State University of New York (SUNY) Downstate Medical Center in Brooklyn, N.Y., and the study’s principal investigator.
This scenario was portrayed in the IBM paper ‘Evolutionary algorithm optimization of biological learning parameters in a biomimetic neuroprosthesis’. The increasing complexity of the virtual arm, which included many realistic biomechanical processes, and the more challenging dynamics of the neural system, called for more sophisticated methods and highly parallel computing in a system such as Comet to tackle thousands of model possibilities, says Amit Majumdar, director of the Data Enabled Scientific Computing division at SDSC, principal investigator of the NSG, and co-author of the IBM Journal paper. “Combining these computational advantages can be an effective approach to build even more realistic biomimetic neuroprostheses for future clinical applications,” he added.
Researchers now recognize that controlling even more realistic and complex systems, including prosthetic limbs, involving larger numbers of bones, joints and muscles. This requires computer models that more closely resemble real brain circuitry.
“We argue that for the model to respond in a biophysiologically realistic manner to ongoing dynamic inputs from the real brain, it needs to reproduce as closely as possible the structure and function or actual cortical cells and microcircuits,” said Salvador Dura-Bernal, a research assistant professor in physiology and pharmacology with Downstate, and the paper’s first author.
In this case, reward signal is based on the ability of the computer model to control how close a virtual hand comes to a target. If the hand got close to the target, synapses generating that movement were rewarded; if the hand was further away, those synapses were punished.
Identifying the best reinforcement learning model required the identification of an optimal set of characteristics or parameters; among others, these include learning and exploratory movement rates, duration of training, and motor command threshold measured in spikes.
“Only the fittest individuals remain,” said Dura-Bernal, “those models that are able to learn better, survive and propagate their genes.” Applied to this system, each individual gene represents a model with a particular set of learning parameters. The fitness of the individual is the ability of the model to learn to control the virtual arm based on real brain signals.
“We emphasize the concept of covering multiple scales, from the molecular, through the cellular, up to the network level,” said Dura-Bernal. “This will be instrumental in understanding and treating brain disorders such as epilepsy, schizophrenia, Parkinson’s, motor paralysis, depression, or amnesia.”
“In patients with motor paralysis, the biomimetic neuroprosthetic could be used to replace the deteriorated motor cortex where it could interact directly with healthy brain pre-motor regions, and send commands and receive feedback via the spinal cord to a prosthetic arm,” said W.W. Lytton, a professor of physiology and pharmacology at State University of New York (SUNY) Downstate Medical Center in Brooklyn, N.Y., and the study’s principal investigator.
This scenario was portrayed in the IBM paper ‘Evolutionary algorithm optimization of biological learning parameters in a biomimetic neuroprosthesis’. The increasing complexity of the virtual arm, which included many realistic biomechanical processes, and the more challenging dynamics of the neural system, called for more sophisticated methods and highly parallel computing in a system such as Comet to tackle thousands of model possibilities, says Amit Majumdar, director of the Data Enabled Scientific Computing division at SDSC, principal investigator of the NSG, and co-author of the IBM Journal paper. “Combining these computational advantages can be an effective approach to build even more realistic biomimetic neuroprostheses for future clinical applications,” he added.
Fusing computational and biological principles
Researchers have been trying for over a decade to fuse computational and biological principles to create realistic computer models that would form the basis for silicon-based neural circuits or implants that would replace damaged brain tissue. In this emerging field, a primary goal has been the decoding of electrical signals recorded from the brain to move, for example, a prosthetic arm. In scenarios once considered science fiction, techniques that encode neural signals from a prosthetic virtual arm to the brain are now allowing users to feel what they are touching.Researchers now recognize that controlling even more realistic and complex systems, including prosthetic limbs, involving larger numbers of bones, joints and muscles. This requires computer models that more closely resemble real brain circuitry.
More realistic artificial neural network
To get closer to this goal, the researchers in the now published study relied on several concepts inspired by biology to create a more realistic artificial neural network that allows the motor cortex to learn to direct a virtual arm – consisting of eight bones, seven joints and 14 muscle branches – to a specified target. The biomimetic model in question involved more than 8,000 spiking neurons and about 500,000 synaptic connections. The main component consisted of primary motor cortex microcircuits based on brain activity mapping, connected to a circuitry model of the spinal cord and the virtual arm.“We argue that for the model to respond in a biophysiologically realistic manner to ongoing dynamic inputs from the real brain, it needs to reproduce as closely as possible the structure and function or actual cortical cells and microcircuits,” said Salvador Dura-Bernal, a research assistant professor in physiology and pharmacology with Downstate, and the paper’s first author.
Reinforcement learning
The researchers trained their model with spike-timing dependent plasticity (STDP) and reinforcement learning, believed to be the basis for memory and learning in mammalian brains. Briefly, the process refers to the ability of synaptic connections to become stronger based on when they are activated in relation to each other, meshed with a system of biochemical rewards or punishments that are tied to correct or incorrect decisions.In this case, reward signal is based on the ability of the computer model to control how close a virtual hand comes to a target. If the hand got close to the target, synapses generating that movement were rewarded; if the hand was further away, those synapses were punished.
Identifying the best reinforcement learning model required the identification of an optimal set of characteristics or parameters; among others, these include learning and exploratory movement rates, duration of training, and motor command threshold measured in spikes.
“Evolutionary” Algorithm
To isolate reinforcement learning parameters that yielded the best control over a virtual arm, the researchers turned to “evolutionary algorithms.” The methodology follows the principles of biological evolution, where a population of individuals, each representing a set of genes or parameters, evolves over generations until one of them reaches a desired fitness level. With every generation, individuals are evaluated and selected for reproduction, produce new offspring by crossing their genes and applying random mutations, and subsequently are replaced by fitter offspring.“Only the fittest individuals remain,” said Dura-Bernal, “those models that are able to learn better, survive and propagate their genes.” Applied to this system, each individual gene represents a model with a particular set of learning parameters. The fitness of the individual is the ability of the model to learn to control the virtual arm based on real brain signals.
Future studies
For their study, the researchers evolved a population of 60 individuals (models with different learning parameters) over 1,000 generations, where at every generation a new set of characteristics was evaluated to measure its fitness. Future studies will focus on developing even more realistic models of the primary motor cortex microcircuits to help understand and decipher the neural code – how information is encoded and transmitted in the brain.“We emphasize the concept of covering multiple scales, from the molecular, through the cellular, up to the network level,” said Dura-Bernal. “This will be instrumental in understanding and treating brain disorders such as epilepsy, schizophrenia, Parkinson’s, motor paralysis, depression, or amnesia.”
