Author: Alexander Vandesompele

Reservoir computing with spiking populations

Passively compliant robots are robots with passive compliant parts, for instance springs or soft body parts. They can be cheaper, safer and more versatile than traditional stiff robotics. Since the compliance introduces non-linearities that are not easy to model analytically, we need to monitor the body with sensors and use machine learning to interpret those sensors.

Reservoir computing allows to train non-linear dynamical systems using only simple machine learning techniques. The unit of our reservoir is a population of spiking neurons. During training the desired motor commands are gradually taught to the closed loop system (with gradual FORCE learning), as illustrated in the figure below. The only weights that need to be learned are those to the readouts.


The neurorobotics platform provides a convenient interface between the robot model and a spiking ‘brain’. After training, we have a closed loop system consisting of only the body and its ‘brain’. The body sensors drive the ‘brain’ activity which in turn drives the actuators. The motor commands are ‘embedded’ into the dynamics of this system. The animation below shows the resulting trained closed loop gait controller. Spike trains are shown for all neurons in one population.


If desired, the system can be trained with an extra input to the reservoir (in addition to sensor inputs). This extra input can be coupled with different motor commands (for instance different gaits, or different frequency of the same gait). After training, this extra input can be set by an external actor (be it another ‘brain’ region or just a human) to control the system in real time:



Reservoir computing for generic motor signal generation

Cyclic movements, for instance in locomotion, can be driven by cyclic neural activity, so called Central pattern generators (CPGs). CPGs have been observed at the spinal cord level and even in neural networks isolated from the brain and from sensorimotor feedback. The speed of CPG controlled locomotion, including shift of gait type, can be controlled by simple high level signals, such as tonic electrical stimulation of the brain stem. At the spinal cord level, sensorimotor feedback is integrated to fine tune the motor signals to the environment.

To integrate higher level commands with sensor/body feedback for motor signal generation, we are developing a control system based on reservoir computing (see figure below). The reservoir consists of populations of spiking neurons that are randomly connected. Inputs to the reservoir are on the one hand a generic periodic signal (modeling the high level command), and on the other hand sensor/body feedback from the robotic body that is to be controlled. The reservoir computing paradigm allows for straightforward extraction of desired motor signals from the resulting reservoir activity.


In a future blog post the physical and virtual robotic platform to conduct these experiments will be presented.