Category: Uncategorized

Going beyond conventional AI

European Robotics Forum 2018 in Tampere

The Neurorobotics Platform developed in the SP10  keeps improving its usability and reliability, and is looking to expand its user base. If the feedback obtained from the audience at the European Robotics Forum (900 registered guests, all roboticists from research and industry) is anything to go by, the NRP is in prime position to fill the need expressed by this community for an interdisciplinary simulation platform than connects neuroscience, AI and robotics.

Indeed, during our Workshop at the ERF and the various discussions that ensued, we were able to speak with a large number of researchers and company representatives from different backgrounds and activities. The overwhelming majority has clearly caught on the potential advantages of using the NRP, especially with standard AI tools such as TensorFlow. Furthermore, we found they were open to considering the ability of the NRP to establish brain-derived intelligent controllers that go beyond conventional AI. Finally, compliant robotics based on the up-and-coming technology of elastic elements that can make robots safe by design is an active area of research where ERF participants also saw potential for the NRP (OpenSim, custom robot designer, etc.).

We are thus looking forward to collaborating with our new contacts in the industry, and to improving the platform even further for their benefit.


(Benedikt Feldotto (TUM) walking the audience through the NRP’s many features)



Fable robot simulator

Fable is a 2 DoF modular robot arm that is being used by the group of DTU in order to develop the task of “Self-Adaptation in Modular Robotics”.

Thanks to the modularity provided by Fable, it is feasible to combine several modules together in order to create different robotic configurations increasing the complexity of the system. In this way, one is able to work on manipulation tasks as well as in locomotion tasks just by plugging a few modules together to form an arm, a worm, a spider,…

In the process to make the Fable robot as accessible as possible to the community, here at DTU we have been working on the implementation of the Fable v2.0 simulator.

We have created 3 different configurations:

A simple robotic arm, 2 DoF (1 Fable module)


A worm-like robot, 4 DoF (2 Fable module)


A quadruped-like robot, 8 DoF (4 Fable module)


This robot model has not been included to the NRP yet, but soon will be available for users. We will keep you updated.




Sensory driven hind-limb mouse locomotion model

In the paper on hind-limb locomotion of a cat in simulation [\textit{reference}], the authors studied the importance two main sensory feedbacks important swing-stance phase switching and which of the particular feedbacks are more important than the other for stable locomotion. In this preliminary work we set-up similar rules to produce locomotion in the mouse model developed in the Neuro-Robotics Platform(NRP). This work will be used to study the role of sensory feedback in locomotion and its integration with feed-forward components such as the Central Pattern Generator’s(CPG’s).In the paper on hind-limb locomotion of a cat in simulation [1], the authors studied the importance two main sensory feedbacks important swing-stance phase switching and which of the particular feedbacks are more important than the other for stable locomotion. In this preliminary work we set-up similar rules to produce locomotion in the mouse model developed in the Neuro-Robotics Platform(NRP). This work will be used to study the role of sensory feedback in locomotion and its integration with feed-forward components such as the Central Pattern Generator’s(CPG’s).

Bio-mechanical model :
We use the Neuro-Robotics platform (NRP) to develop the simulation model and its environment. The rigid body model of the mouse available in NRP was obtained from a high resolution 3D scan of a real mouse. Relationship between the segments are established via joints. For the purpose of this experiment only hind-limbs are actuated. Thus the current model has in total eight actuated joints, four in each hind-limb. Muscles are modeled as hill type muscles with passive and active dynamics. Muscle morphometry and related parameters were obtained from [2]. Each of the actuated joint consisted of at least one pair of antagonist muscle. Some joints also bi-articular muscles. In total the model consists of sixteen muscles. Proprioceptive feedback from muscles and rigid body and tactile information close the loop between the different components of locomotion.

This slideshow requires JavaScript.

Reflex controller :
The idea here is to break the motion of hind limb locomotion into four phases, namely (i) swing (ii) touch-down (iii) stance (iv) lift-off. Proprioceptive feedback and joint angles dictate the reflex conditions under which the phase transitions from one to another. Figure shows the four phases and their sequence of transition. For the hind limbs to change from one phase to another we optimize the muscle activation patterns as a function of proprioceptive feedback and joint angle. This ensures a smooth transition between one phase to another when a necessary condition is met.

Discussions :
With the bio-mechanical model of mouse in NRP and reflex control law we are able to reproduce stable hind-limb gait patterns that are purely sensory driven. The next steps to taken in the experiment are :

  1. Convert reflex laws into neuron based reflex loops
  2. Extend the reflex model for quadruped locomotion
  3. Add a CPG layer to interface with the reflex loops

References :

  1. O. Ekeberg and K. Pearson, “Computer simulation of stepping in the hind legs of the cat: an examination of mechanisms regulating the stance-to-swing transition.” Journal of neurophysiology, vol. 94, no. 6, pp. 4256–68, dec 2005.
  2. J. P. Charles, O. Cappellari, A. J. Spence, J. R. Hutchinson, and D. J. Wells, “Musculoskeletal geometry, muscle architecture and functional specialisations of the mouse hindlimb,” PLoS ONE, vol. 11, no. 4, pp. 1–21, 2016.

Preliminary neural recordings with the M-Platform

Post 4-Fig 1
(FIG 1) The new robotic platform to have an access to the brain cortex and to record neural signal.

The M-Platform, a robotic device for motor rehabilitation after stroke in mice, has been upgraded to allow recording of neural activity during the pulling task (FIG 1). Now the platform provides the unique possibility to integrate kinetic and kinematic data with electrophysiological recordings in awake mice during a voluntary forelimb retraction task.

Post 4-Fig 2
(FIG 2) The interface of  OmniPlex D System (Plexon, USA), the system used to perform acute-electrophysiological recordings.

The new device was tested on four healthy mice: an array of 16 channels linear probe (ATLAS, USA) was inserted into the Rostral Forelimb Area (RFA) at 850 µm of depth. Signals were recorded by OmniPlex D System (Plexon, USA) at a frequency of 40 kHz (FIG 2). The analysis of the data was performed offline. We obtained promising results both for the low frequency activity, i.e. Local Field Potential (LFP), and for the high frequency activity, i.e. Multiunit Activity (MUA) and spike sorting. In particular in FIG. 3 a correspondence between the LFP and the force peak is evident; however we are planning to increase the number of recorded animals to generalize our results.

Post 4-Fig 3
(FIG 3) On the top the mean of the LFP recordings in different channels aligned on the onset; at the bottom the mean of corresponding force peaks.

This success paves the way for investigation of neuroplastic events after a cortical damages, i.e. stroke. Moreover the possibility to record spiking activity in the Caudal Forelimb Area (CFA) during the task in healthy animals allows to study firing rate in different channels and find patterns to correlate neural activity and movement of the forelimb.

SP10 + SP6 + CerebNEST New collaboration

Last month, during the last HBP summit, SP10 was able to start working on potential new collaborations with other subprojects and partnering projects in order to keep focus on the main goal of the Neurorobotics platform and the Human Brain Project. Not only for the current phase of the project (SGA 1), but also for the coming years of research.

We are really happy to say that a few days ago, the DTU Neurorobotics team came to an agreement with the SP6 (University of Pavia) and the HBP Partnering project CerebNEST (Politecnico di Milano) in order to integrate to SpiNNaker their cerebellum model (Antonietti et al., 2016 IEEE TBME) that has been already implemented in NEST.



Having a cerebellar model working in real-time in a neuromorphic platform is going to provide the possibility to analyze the performance of the model with different physical robotics platforms such as the modular robot Fable.

We will keep you updated along the process!

Practical lab course on the Neurorobotics Platform @KIT

This semester, for the first time, the Neurorobotics Platform will be used as a teaching tool for students interested in embodied artificial intelligence.

The lab course started last week for KIT students, offered by FZI in Karlsruhe. Previously, instead of this practical class, we were offering a seminar were students would make literature research on Neurorobotics and learning. For the seminars, we had around 10 students registering per semester, but this year for the practical lab course, more than 20 students registered, most of them in master degree.



The initial meeting took place last week. The students were splits in seven groups of three. Their first task, familiarize themselves with the NRP and PyNN by solving the tutorial baseball experiment and provided python notebook exercises. All groups were given USB sticks with live boot for them to easily install the NRP, and also access to an online version. Throughout the semester, students will learn about Neurorobotics and the platform by designing challenges and solve them.

Organizers: Camilo Vasquez Tieck, Jacques Kaiser, Martin Schulze, Lea Steffen

Mouse modeling for robotics and neuroscience…

… or why we are building a zoo of artificial mice.

Neurorobotics is about connecting simulated brains to virtual and physical robot bodies. Differently from other approaches in robotics or machine learning, the focus is on high biological plausibility, i.e. a neurorobotic system is designed to capture and predict the quantitative behavior of its biological counterpart as closely as possible. However, what is exactly meant by “close” depends on the granularity of the brain model. Clearly, simple neural networks with only a few neurons can be studied on an equally simple robot. In case of the Braitenberg vehicle experiment on the Neurorobotics Platform, a mobile robot platform with four wheels and a camera is perfectly sufficient. By contrast, brain simulations that are comprised of millions of neurons require realistic body models to simulate and reproduce data from neuroscience as accurately as possible. In this context, standard robots are no longer a viable choice. Neurorobotics is therefore not only about connecting a robot body to a brain but also about the design, simulation, and construction of that body.

The brain models developed in the Human Brain Project are among the most complex and realistic ones ever built and therefore it is only logical that they require the most realistic body models ever built. But how does the perfect body model look like? The answer is both simple and tricky: Since most of the data in neuroscience is obtained from rodents, particularly mice, the perfect choice for the body model is to simulate a mouse body. The tricky part is to determine the level of detail that is necessary to provide meaningful embodiment for the brain models. We are therefore currently designing and building a zoo of different mouse models, each of which serves a specific purpose.

The maximum level of biological detail can only be achieved in simulation. For this reason, we are developing a virtual mouse body that not only looks like a real mouse but that also has the same biomechanical properties. Every bone of the skeleton was modeled individually based on bones of real mice. Combined with the musculoskeletal simulation that will soon be available in the Neurorobotics Platform, the skeleton will enable realistic biomechanical simulations.

Rendering of the completed mouse skeleton

The latest version of the virtual mouse got a soft skin that is fitted to the skeleton. Together with the recently added simulation of the fur, our mouse is almost indistinguishable from its biological colleagues!

Rendering of the mouse model with skin and fur

Unlike simulation, the real world imposes many constraints on the types of robots that can be built. However, having a physical counterpart to our virtual mouse is beneficial for many reasons. It not only enables direct interaction with the robot but is in particular also a first step to applying results from neurorobotics research in real-world applications. Our first prototype of the mouse robot was built with a focus on small size and biomimetic leg design for robust locomotion. Upcoming releases will not only feature improved mechanics but in particular also include more sensors. Follow our blog to see how our mouse is slowly growing up!

Completed initial prototype of the mouse robot

Many thanks to Matthias Clostermann, Eva Siehmann, and Peer Lucas for their contributions!

Florian Walter, Technical University of Munich
October 13, 2017