Bio-inspiration and modularity make robotic locomotion adaptable in the NRP

Current robotic control strategies are mainly based on trajectory plans that adjust the movements based on the next desired state. These control policies do not efficiently perform where the dimensionality of the control problem increases or disturbances are perturbing the system from the external environment. These issues are critical in locomotion tasks and the need for different control methods arises. To make autonomous robots able to move in a real and dynamic environment, the research has focused on biologically inspired controller, such as neuro-controller.

The interaction among different bio-inspired motion controllers whose communication represents a simplified model of the neural locomotion control in vertebrates is possible in the Neurorobotics Platform.


The presented solution combines classical control strategies with reservoir computing and spiking neural networks (Reservoir computing with spiking populations by Alex Vandesompele) to obtain a scalable and adaptable controller by taking advantage of the different learning properties of neural networks. To reflect the scalability of the controller, the experiments are performed on the simulated modular Fable Robot. In the experiment, it is built in a quadruped configuration so that the control architecture is composed of 4 cerebellar microcircuits, called Unit Learning Machines (ulm).

The use of a spiking neural network with reservoir computing as a trajectory planner (Central Pattern Generator, cpg) allows the learning of complex periodic trajectories for the movements of the robotic modules, whose frequency modulation is possible by just changing the frequency of the input signal to the network. Not optimally tuned PIDs give to the robot early stability during the first part of the simulation and provide a torque command for each module. Thus, a cerebellar network composed of 4 micro complexes computes and provides corrective effort contributions based on the inverse dynamics model of each robotic modules.

In the video below, it is possible to appreciate the locomotion improvements of the robot in an experimental simulation. The recording shows the simulation around second 100-130 when the position error is decreasing and stabilizing. The brain visualizer shown the spiking activity of the input population of the Central Pattern Generator (the higher groups) which are reflected in the blinking of one population of the reservoir (lower group). The spike train window (on the left) shows the periodicity of the spike trains which generate the trajectories for the modules (starting from the bottom, the activities of the input populations and one reservoir population are displayed).


The feed-forward cerebellar effort contribution decreases the mean of the position error of 0.3 radiant and its variance of 0.01 radiant compared to the case when just the effort command from the cpg is provided to the robot (the plot concerning the behavior of the second module is shown below). Moreover, the trend of the error is decreasing along the simulation time and the distance covered by the robot with the cerebellar-like neural network contribution is 9.48 m while the cpg controller contributes to have the robot walk for 1.39 m.

The modular configuration of the Fable Robot makes easier to test the control strategy for different configurations of the robot and patterns of locomotion, having the cerebellar-like neural network compensate the error after a short learning phase, since it has previously learned the internal model of the module.

This work was done in collaboration between DTU, Ghent and SSSA teams.



A new integration experiment for the visual system of the NRP

We implemented a new experiment to demonstrate that the NRP is able to run many models together, as a single visual system. Here, a retina model, a deep neural network for saliency computation, a spiking cortical model for early-stage visual segmentation and an echo state network for saccade generation collaborate, despite the differences in their structures. The NRP provides a common framework where models can talk to each other easily.

In the video below, the robot has to keep track of a stimulus that moves on a screen. The saliency model computes where the stimulus lies (most salient region in the visual field), and whenever it is not in the fovea, it delivers a signal to the saccade model, so that an eye movement is generated towards the visual stimulus. When the stimulus reaches the fovea, the second task is to segregate the target (small tilted bars) from the flanking square (impairs target detection). For this purpose, when a saccade is generated, the segmentation model triggers saccade inhibition and then sends local signals that initiate a spreading segmentation process that lies in the networks dynamics, attempting to segregate the target from the flanker. These local signals are sent, using the saliency output as a 2D probability density distribution, so that segmentation is only triggered around regions that are worthy of attention. Only when the segmentation is successful, the target is detected. During the whole experiment, the retina model delivers adaptation to the lighting of the scene. Thanks to it, the cortical representation of the stimulus is stable and segmentation is possible even in low-lighting conditions.

The NRP demonstrates its ability to run large-scale, collaborative simulations. Being able to run a whole visual system gives many novel opportunities to vision research, especially to explain global effects in human vision.

Reservoir computing with spiking populations

Passively compliant robots are robots with passive compliant parts, for instance springs or soft body parts. They can be cheaper, safer and more versatile than traditional stiff robotics. Since the compliance introduces non-linearities that are not easy to model analytically, we need to monitor the body with sensors and use machine learning to interpret those sensors.

Reservoir computing allows to train non-linear dynamical systems using only simple machine learning techniques. The unit of our reservoir is a population of spiking neurons. During training the desired motor commands are gradually taught to the closed loop system (with gradual FORCE learning), as illustrated in the figure below. The only weights that need to be learned are those to the readouts.


The neurorobotics platform provides a convenient interface between the robot model and a spiking ‘brain’. After training, we have a closed loop system consisting of only the body and its ‘brain’. The body sensors drive the ‘brain’ activity which in turn drives the actuators. The motor commands are ‘embedded’ into the dynamics of this system. The animation below shows the resulting trained closed loop gait controller. Spike trains are shown for all neurons in one population.


If desired, the system can be trained with an extra input to the reservoir (in addition to sensor inputs). This extra input can be coupled with different motor commands (for instance different gaits, or different frequency of the same gait). After training, this extra input can be set by an external actor (be it another ‘brain’ region or just a human) to control the system in real time:


New mechanisms in the Laminart model

The Laminart model is a spiking cortical network for early-stage visual segmentation. Using network dynamics simulated in NEST, it is able to parse its visual input into several perceptual groups. It is the largest simulated network available on the Neurorobotics Platform. It explains very well many behavioural results about visual crowding (a behavioural paradigm in which human observers try to identify a target being disrupted by nearby flanking shapes, the target being “crowded” when identification performance is impaired by the flankers – see fig. 1).

Fig. 1. a) Crowding in real life. If you look at the bull’s eye, the kid on the left is easily identifiable. However, the one on the right is harder to identify, because the nearby elements have similar features (orange color, human shape). c) Crowding in behavioural experiments. The visual stimuli on the x-axis are presented in the periphery of the visual field of human observers. The task is to identify the direction of the offset of the target (small tilted bars). Flanking squares try to decrease performance. What is plotted on the y-axis is the target offset at which observers give 75% of correct answers (low values indicate good performance). When the target is alone (dashed line), performance is very good. When only one square flanks the target, performance decreases dramatically. However, when more squares are added, the task becomes easier and easier. All classical models of crowding fail at explaining the latter condition (uncrowding), because they all predict that more flankers induce more interferences.

The model explains the results by stating that the target is crowded when it is considered as in the same perceptual group as the flanker, allowing interferences. When the flankers form a group on their own, they “frame” the target, increasing performance (less interference happens between elements that are in different perceptual groups). This idea explains the behavioural results very well. More than that, the model offers neural mechanisms for perceptual grouping and segmentation. However, these mechanisms work properly only with very simple flanking shapes (squares, rectangles, etc.). As soon as the shapes are slightly more complex (more orientations, various scales, etc.), the mechanisms that are necessary to produce the results do not work properly. This is a problem, because uncrowding happens for any flanking shape (not only squares). Here, we introduce new ways of implementing the key mechanisms of the model.

The first mechanism groups boundaries together (see fig. 2). This is linked to illusory contours computation. Activity spreads to connect edges that are well aligned. The new idea that was implemented here is that the illusory contours can now spread at different scales, according to the configuration of the stimulus. For example, if you are reading, you will tune your illusory contours mechanism, so that horizontal grouping happens between letters of the same word, but not between words, using the fact that letters of a same word are closer than letters from neighbouring words. We used the same idea for our crowding paradigms, where different grouping scales lead to different crowding results.

V2  V2

Laminart grouping
Fig. 2. Top: illustration of how the multi-scale grouping mechanism works (V2, layer 2/3 activity). Left: if tuned for low distances, no illusory contour will spread from the flankers to the target. Right: if tuned for long distances along the horizontal direction, illusory contours easily spreads through long distances, to link all the flanking octagons. Bottom: dynamics of the grouping mechanism. Activity naturally spreads in V2. However, spreading also activates interneurons that activate spreading control. In this state, nothing spreads. However, stimulus onset triggers a damping signal whose duration is specific for the orientation of spreading. It is highly dependent on the stimulus shape. For example, in the top-left stimulus, the closed surfaces impair spreading around the target. In the top-right stimulus, the horizontal alignement of the flankers makes that the damping signal is quite long along the horizontal direction, allowing horizontal boundaries to spread across flankers. When the damping signal stops, the illusory contours are already stabilised and do not spread back.

The second mechanism parses subsets of the image that are linked by boundary grouping to different segmentation layers (different regions of the network’s activity – see fig. 3). In our case, this allows the model to block interferences between parts of the image that belong to different groups. The new mechanism is better, because the former one required tonic activity of many neurons in the segmentation network, which was biologically implausible. Now it is only driven by input-related activity and by a brief segmentation signal.


Laminart segmentation
Fig. 3. Top: Illustration of the mechanisms of segmentation. After the segmentation signal is sent, activity spreads from V2 segmentation layer 0 (SL0) to V2 SL1-2 (after competition, all the flanker-related activity ends up in SL2). Activity can spread throughout all flankers thanks to the illusory contours between them. This is only possible with the new grouping mechanism. Bottom: dynamics of the segmentation mechanism. The interneuron layer is the one that does all the spreading. It also triggers its own control by spreading. The control layer is being inhibited by activity in SL1, allowing the interneurons to shut down activity in SL0, disinhibiting activity in SL1. Because all the dishinibition relies on V2 activity, it will only spread along connected boundaries. You can see in the figure on the top that the control layer acts as a sheath to make sure that the interneuron activity only spreads along V2 activity.

In the future, the new mechanisms will be generalised to more orientations (ongoing work), and the new version of the model is going to be integrated on the NRP. Also, top-down influence on how damping signals distribute across orientations will be investigated.

Going beyond conventional AI

European Robotics Forum 2018 in Tampere

The Neurorobotics Platform developed in the SP10  keeps improving its usability and reliability, and is looking to expand its user base. If the feedback obtained from the audience at the European Robotics Forum (900 registered guests, all roboticists from research and industry) is anything to go by, the NRP is in prime position to fill the need expressed by this community for an interdisciplinary simulation platform than connects neuroscience, AI and robotics.

Indeed, during our Workshop at the ERF and the various discussions that ensued, we were able to speak with a large number of researchers and company representatives from different backgrounds and activities. The overwhelming majority has clearly caught on the potential advantages of using the NRP, especially with standard AI tools such as TensorFlow. Furthermore, we found they were open to considering the ability of the NRP to establish brain-derived intelligent controllers that go beyond conventional AI. Finally, compliant robotics based on the up-and-coming technology of elastic elements that can make robots safe by design is an active area of research where ERF participants also saw potential for the NRP (OpenSim, custom robot designer, etc.).

We are thus looking forward to collaborating with our new contacts in the industry, and to improving the platform even further for their benefit.


(Benedikt Feldotto (TUM) walking the audience through the NRP’s many features)


Towards using the NRP with motor actuated robots

The Neurorobotics Platform Development and Research teams are joining their forces to give the users the possibility to interface their own motor actuated robots with the NRP.
This new intriguing feature will help all the roboticists to deploy their neural controllers on real robots, without the need of any external software.
A new working-mode for the NRP has been created by the SSSA team in Pisa, supporting NEST as the neural interface and ROS for the robotic interface.

The iCub humanoid robot doesn’t have a native support for ROS, but instead it uses YARP (Yet Another Robotic Platform) as a middleware.
This makes it necessary for the NRP to support YARP in order to interact with the hardware iCub platform. Such integration is also being currently developed by the SSSA team.
While not being able to send commands to the iCub yet, this extension allows already to read from its sensors, as for instance the eye cameras.

You can see in the picture below one of our team members waving at the iCub seen from the NRP web frontend.


A further mode is being developed by the KIT team in Karlsruhe, to support the SpiNNaker neuromorphic platform and thus allow whole experiments to go completely on hardware, leaving only the communication and synchronization to the NRP closed loop engine.

In the next months, the NRP will support the execution of transfer function at a fixed rate, in real time, thus allowing the implementation of controllers with the well-estabilished fixed rate approach, widely used in the robotic community.

Handling experiment-specific python packages in the NRP

In this blog post I share my method to handle experiment-specific python packages. Some of  my experiments require TensorFlow v1.6, some others need Keras – which itself requires a prior version of TensorFlow – how to handle all this on your locally installed NRP?

My method relies on the package virtualenvwrapper, which allows you to keep your python virtualenv in a single place.

pip install virtualenvwrapper --user

Additionally, I have a custom config which adds a virtualenv to the $PYTHONPATH when I activate it. Copy the postactivate and postdeactivate scripts to $WORKON_HOME – the configuration folder of virtualenvwrapper.

Now, let’s say you have an NRP experiment with custom python packages listed in a requirements.txt. Create a virtualenv for this experiment and install the experiment-specific packages:

mkvirtualenv --system-site-packages my_venv
pip install -r requirements.txt

To access your experiment-specific packages from within the NRP, simply start the NRP from the same terminal, where the virtualenv is activated:

workon my_venv

That’s it!