Category: Neurorobotics

CDP4 at the HBP Summit: integrating deep models for visual saliency in the NRP

Back in the beginning of 2017, we had a great NRP Hackathon @FZI in Karlsruhe, where Alexander Kroner (SP4) presented his deep learning model for computing visual saliency.

We now presented this integration at the Human Brain Summit 2017 in Glasgow as a collaboration in CDP4 – visuo-motor integration. During this presentation we also shown how to integrate any deep learning models in the Neurorobotics Platform, as was already presented in the Young Researcher Event by Kenny Sharma.

We will continue this collaboration with SP4 by connecting the saliency model to eye movements and memory modules.



Optimising compliant robot locomotion using the HBP Neurorobotics platform

If we want robots to become a part of our everyday life, future robot platforms will have to be safe and much cheaper than most useful robots are now. Safety can be obtained by making robots compliant using passive elements (springs, soft elastic materials). Unfortunately, accurate mechanical (dynamic/kinematic) models of such robots are not available and in addition, especially when cheaper materials are used, their dynamical properties drift over time because of wear.

Therefore, cheap robots with passive compliance need adaptive control that is as robust as possible to mechanical and morphological variations. Adaptation training on each physical robot will still be necessary, but this should converge as quickly as possible.

The Tigrillo quadruped robot will be used to investigate neural closed loop motor control for locomotion to address these issues. In particular, we want to investigate how the NRP simulation framework can be used to develop such robust neural control.

As a first step, we implemented a parameterised Tigrillo simulation model generator. Using a simple script, a Gazebo simulation model with given body dimensions, mass distributions and spring constants can be generated to be simulated in the NRP. We then implemented evolutionary optimisation (CMA-ES) in the NRP’s Virtual coach to find efficient motor control patterns, which then generated with spiking population networks using a reservoir computing approach. Finally, these control patterns were transferred to the physical robot’s SpiNNaker board and the resulting gaits were compared to the simulation results.

These steps are illustrated in the video below.

Next steps are:

  • to tune the parameter ranges of  the Tigrillo generator to those that are realistic for the real robot;
  • to implement sensors on the physical robot and calibrate equivalent simulated sensors;
  • to use our setup to obtain the desired robust closed loop control and validate both qualitatively and quantitatively on the physical robot.

Many thanks to Gabriel Urbain, Alexander Vandesompele, Brecht Willems and prof. Francis wyffels for their input.


OpenSim support in the Neurorobotics platform

A key area of research of the Neurorobotics Platform (NRP) is the in-silico study of sensormotor skills and locomotion of biological systems. To simulate the physical environment and system embodiments, the NRP uses the Gazebo robotics simulator.

To perform biologically significant experiments, Gazebo has however been lacking an important feature until now: The ability to model and simulate musco-skeletal kinematics.

Therefore researchers had to rely on ad-hoc implementations calculating effective joint torques for the system at hand, wich is time consuming, error prone and cumbersome.

The physics plugin we implemented provides OpenSim as an additional physics engine alongside the physics engines already supported by Gazebo (ODE, Bullet, SimBody and DART). OpenSim is using SimBody as its underlying framework, thus featuring a stable and accurate mechanical simulation. The OpenSim plugin supports many of SimBody’s kinematic constraint types and implements collision detection support for sphere, plane and triangle mesh shapes along with corresponding contact forces (as exposed by OpenSim’s API).

However, first and foremost it treats physiological models of muscles as first class citizens alongside rigid bodies and kinematic joints. OpenSim is shipped with a number of predefined muscle-tendon actuators. Currently, users of our plugin can use OpenSim’s native XML configuration file format to specify the structure and properties of muscle-tendon systems, which are created on top of Gazebo models specified in Gazebo’s own file format (SDF).

A ROS-based messaging interface provides accessors for excitations and other biophysical parameters allowing to control musco-skeletal systems from external applications such as the Neurorobotics platform.

As demonstration of the capabilities of our physics plugin, we augmented a simple four-legged walker with a set of eight muscles (one synergist-antagonist pair per leg).

The problem we address in this demo is the reinforcement learning task of deriving a controller that excites the muscles in a pattern such that the walker is driven forward. Our setup consists of a Python application (remote-controlling Gazebo via the ROS-based messaging interface for the OpenSim plugin) performing the high-level optimization procedure and running a neural network (NN) controller.

We employ a simple genetic optimization procedure based on Python’s DEAP package to find parameters of the NN that maximize the score the walker obtains in individual trial runs.

The walker is rewarded for moving forward and penalized for unwanted motion behaviour (e. g. ground contacts of the walker’s body, moving off-center).

During a trial run, the physics simulation is stepped in small time increments, and during each iteration the NN is fed with various state variables. The NN’s output is comprised of excitation levels for the muscles. For simplicity we stuck to well-known artificial neural networks, implemented via the Tensorflow package.

We also experimented with fully dynamic grasping simulation using SimBody’s collision detection system and contact force implementations. Although the simulation setup for the grasping tests only comprised a simple two-jaw gripper and a cubic shape (consisting of a triangle mesh shape), the SimBody engine as used in our plugin was able to maintain a stable grasp using fully dynamic contact forces, tackling a problem that is notoriously difficult to solve with other physics engines.

Another application using the OpenSim plugin for Gazebo features a simplified muscle model of a mouse’s foreleg actuated by a neuronal controller modelled according the spinal cord of a real mouse. The details of this experimental setup will be covered in a separate blog post.

The OpenSim plugin does not support all of the features implemented with other engines in Gazebo. For instance, some joint types are not implemented yet. Also, some features unique to OpenSim (like inverse dynamics simulation) are not yet available in the current implementation.

To simplify the design of kinematic models with muscle systems and custom acutator models, it is planned to provide researchers and users of the NRP with a consistent, simple way to specify muscles via a graphical interface using the NRP‘s Robot Designer application.

A one-day workshop during the last Performance Show in Ghent

Last week, we had the chance to organize the first edition of a SP10 Performance Show in the city of Ghent, Belgium. This two-days meeting between all the partners involved in the HBP Neurorobotics subproject (SP10) was an opportunity to discuss the latest progress of each research groups and ensure a convergence of views and efforts for the next events, researches and developments.


SP10 Performance Show September 2017
A discussion during the SP10 Performance Show


On the second day, we divided our work into two tracks. Whereas the Main Track dealt with administrative and research activities, the Secondary Track was organized as a workshop on the theme Thinking the NRP of the Future. It was formatted as short one-day hackaton where everyone started by summarizing one or several iconic research advances that had been done in the last year in his field, which helped us grouping into 4 different work teams :

  • Reinforcement Learning with the NRP
  • Integrating worms brains and soft bodies in the NRP
  • Real-time interaction between real and simulated robots in the NRP
  • Helping research on visuomotor learning with the child using simulations in the NRP


SP10 Performance Show September 2017
On Tuesday, a work group is brainstorming about integrating worms in the NRP


Each of those teams brainstormed to imagine and design an experiment that could help research to move forward and a list of requirements in term of developments it would need to be achieved. After lunch, the results of this brainstorm were presented to everyone to get feedback and comments before we started working on designing a first prototype in the NRP and coding some useful models that we would need in further work. To be continued then…

A neuro-biomechanical model that highlights the ability of spinal sensorimotor circuits to generate oscillatory locomotor outputs

The goal of this project is to uncover the functional role of proprioceptive sensorimotor circuits in motor control, and to understand how their recruitment through electrical stimulation can elicit treadmill locomotion in the absence of brain inputs. This understanding is pivotal for the translation of experimental spinal cord stimulation therapies into a viable clinical application.

To this aim, we developed a closed loop neuromusculoskeletal model that encompass a spiking neural network of the muscle spindle pathway of two antagonist muscles, a musculoskeletal model of the mouse hindlimb, and a model of epidural electrical stimulation (Figure 1). The network includes alpha motoneurons, Ia inhibitory interneurons, group II excitatory interneurons, and group Ia and group II afferent fibers. The number of cells, the connectivity, and the firing behavior of alpha motor neurons was tuned according to experimental values found in literature. The effect of epidural electrical stimulation was integrated in the neuronal network by modelling every stimulation pulse as a supra threshold synaptic input in all the cells recruited by the stimulation. An experimentally validated FEM model of the lumbar rat spinal cord was used to compute the percentage of fibers recruited by the stimulation.

Closed loop simulations were performed by using the firing rates of the motoneurons populations as a signal to control the muscles activity of the musculoskeletal model, while using the muscles length information coming from the musculoskeletal model to estimate the firing rates of the neural network afferent fibers. In particular, the firing rates of Ia and II afferent fibers were estimated using an experimentally derived muscles spindle model.

The preliminary results show that muscle spindle feedback circuits alone can produce alternated movements typical of locomotion, when biomechanics and gravity are considered.

Current work is being performed in order to expand the modeled muscle spindle circuitry to control all the main hindlimb muscles together. To this purpose, the developed network will be used as a template for every couple of antagonist muscles and heteronymous connections across the different joints will be implemented. With this complete model of the hindlimb muscle spindle circuitry we will be able to assess whether this single sensorimotor pathway is sufficient to produce treadmill locomotion in combination with EES, or whether other spinal neural networks are necessarily involved.


Figure 1 : Closed loop simulation framework of Spinal Cord model and rodent hind limb to study epidural electrical stimulation

  • Emanuele Formento (PhD, TNE & G-Lab, EPFL)
  • Shravan Tata Ramalingasetty (PhD, BioRob, EPFL)

Successful NRP User Workshop

Date: 24.07.2017
Venue: FZI, Karlsruhe, Germany

Thanks to all of the 17 participants for making this workshop a great time.

Last week, we held a successful Neurorobotics Platform (NRP) User Workshop in FZI, Karlsruhe.  We welcomed 17 attendants over three days, coming from various sub-projects (such as Martin Pearson, SP3) and HBP outsiders (Carmen Peláez-Moreno and  Francisco José Valverde Albacete). We focused on hands-on sessions so that users got comfortable using the NRP themselves.


Thanks to our live boot image with the NRP pre-installed, even users who did not follow the local installation steps beforehand could run the platform locally in no time. During the first day, we provided a tutorial experiment, exclusively developed for the event, which walked the users through the many features of the NRP. This tutorial experiment is inspired from the baby playing ping pong video, which is here simulated with an iCub robot. This tutorial experiment will soon get released with the official build of the platform.



On the second and third days, more freedom was given to the users so that they could implement their own experiments. We had short hands-on sessions on the Robot Designer as well as Virtual Coach, for offline optimization and analysis. Many new experiments were successfully integrated into the platform: the Miro robot from Consequential Robotics,  a snake-like robot moving with Central Patterns Generators (CPG), revival of the Lauron experiment, …


Screenshot from 2017-09-08 14-29-33_crop

We received great feedback from the users. We are looking forward for the organization of the next NRP User Workshop!


Simulating tendon driven robots

According to the concept of embodiment, a brain needs to be connected to a body interacting with the world for biological learning to happen, developing biomimetic robots is crucial to fully understand human intelligence. Here, a tendon driven approach can model muscle behavior in terms of flexibility, compliance and contractive force.

While this concept is clearly beneficial for research, it is very difficult to accurately model in simulation. In contrast to classical robots with motors applying torques in the joints, the simulation needs to apply forces  along wrapped ropes mimicking tendons and muscles. The artificial muscles developped in the Myorobotics [1] project include mechanical parts for flexiblity and force as well as electrical control in different operating modes as seen in Figure 1. To close the reality gap all physical properties need to be considered in modelling.


Figure 1: Myorobotics muscle unit (from [2])

We implemented a plugin for Gazebo that finally allows us to simulate the Myorobotics muscle setup. The plugin models tendon kinematics as well as mechanical and electrical properties of the technical actuator. The calculated forces derived from control commands can now be applied directly to a robot simulated in Gazebo. This brings it one step closer to being integrated into the NRP, allowing us to equip muscle units to arbitrary robot morphologies.

Ultimately, this will enable us to compare simulated biological muscles simulated by OpenSim to the technical muscle of Myorobotics modelled with this plugin. Eventually, this will help to build better biomimetic muscle units behaving just like their biological counterparts.



[2] C. Richter, S. Jentzsch, R. Hostettler, J. A. Garrido, E. Ros, A. Knoll, F. Röhrbein, P. van der Smagt, and J. Conradt, “Scalability in neural control”, IEEE ROBOTICS &AUTOMATIONMAGAZINE, vol. 1070, no. 9932/16, 2016.



Alexander Kuhn, Benedikt Feldotto (TU München)