HBP at the 5th EU-Japan Workshop on Neurorobotics and Cognitive Systems

On 19-20 June, representatives of the HBP Neurorobotics group attended the 5th edition of the EU-Japan workshop on Neurorobotics and Cognitive Systems in Tokyo. This yearly, jointly organized event is the mark of the ongoing successful collaboration between EU and Japan, bringing together renowned scientists to discuss the most cutting edge research ideas at the intersection of robotics, neurosciences and computer sciences.

Group picture in Tokyo — *From the left, Prof. Rüdiger Dillmann, Prof. Satoshi Oota, and Prof. Alois Knoll. © Photography courtesy of Dr F. Morin.*

As always, the workshop proved a fertile ground for productive and engaging discussions. The keynote lectures of Prof. Yasuo Kuniyoshi (University of Tokyo / RIKEN), “VR-based Neuro-Rehabilitation for Alleviating Phantom-Limb Pain”, and of Andreas Rowald (EPFL), “Targeted neurotechnology restores locomotion in humans with spinal cord injury” were particularly stimulating for the audience.

Audience at one of the lectures — *Audience attending the keynote lecture of Prof. Yasuo Kuniyoshi, © Photography courtesy of Dr F. Morin.*

Another highlight of this year’s event was the update on the RIKEN-AIST joint project focusing on the development of a solution to address the worldwide issue of a rapidly ageing society. The project aims to leverage both robotics and biology in order to design an “endoskeletal robot suit” (StillSuit), the purpose of which is to jointly stimulate both the physical and cognitive abilities of ageing individuals throughout their daily activities. The project intends to establish a holistic view of the human body as a biological system; of particular interest for HBP, it is now considering using the Neurorobotics Platform as a simulation tool to integrate multimodal data (genetic, electrophysiological, biomechanical, etc.).

Later this year, in Munich, HBP partners working on Neurorobotics will host a follow-up workshop where joint efforts with RIKEN & AIST will be further planned and discussed.

NRP multi-stakeholder workshop in Munich

On 23-24 January, neuroscientists, computer scientists, roboticists and developers met for a two-day workshop in the beautiful city of Munich for a multi-stakeholder workshop hosted by HBP partner TUM. The aim of this workshop was to strengthen and extend our understanding of the needs of current and prospective users of the NRP, so that the software development process of the NRP could be better aligned with those needs.

Dr Porr discussing his feedback with the workshop participants

The workshop was organised and coordinated by Dr Yannick Morel and Victoria Neumann, and gathered 15 participants, both HBP members and external users. They exposed their expectations (and sometimes frustrations) in lively discussions over the two days of the meeting, providing invaluable insight into the value proposition of the NRP. In particular, we would like to thank Dr Bernd Porr, from the University of Glasgow, for joining us and contributing with both feedback and ideas. A big thanks also to our dev team who actively contributed to an engaging and productive conversation throughout the workshop!

HBP exhibition at the German Bundestag

Running until the 19th of December, the HBP exhibition in the German Bundestag, in Berlin, represents a unique opportunity to discover the latest achievements of the project while enjoying fascinating images and videos of state of the art brain research. The work and ambitions of the 131 HBP partners is presented here, at the Paul-Löbe-Haus, Eingang West, in Konrad-Adenauer-Straße 1, Berlin-Mitte. Hurry up!

EBRAINS community building

On 9 and 10 of December, 2019, HBP partners met in Brussels, at the Royal Flemish Academy of Belgium for Science and the Arts, to start a critical dialogue with international industry representatives and European patient organisations. The goal of this and many similar activities that will follow, is to build a sustained stakeholder engagement to make sure society will benefit from the services provided by the EBRAINS infrastructure.

The event, organised by the Danish Board of Technology Foundation, included a day of presentations and a workshop day during which participants discussed what the features of a future, inclusive EBRAINS community should be.

Prof. Jan G. Bjaalie and Dr Timo Dicksheid gave introductory talks about the EBRAINS infrastructure, while the Neurorobotics service in the EBRAINS was presented by Dr Francesca Cavallaro.

HBP Curriculum workshops

In the past September the HBP Neurorobotics team has been involved in and actively contributed to two HBP curriculum workshops.

For the series ICT for non-specialists, the workshop Spiking neural networks – applications to computing, algorithmic and robotics, was organised by the TUM Chair of Robotics, Artificial Intelligence and Embedded Systems, and took place in Munich, Germany, on 18 September, 2019. Dr Fabrice Morin, SP10 manager, gave the introductory talk The basics of spiking neurons: Biological facts, models and computational properties, and 28 participants from across Europe attended this full day that ended with the open lecture of Prof. Robert Riener, ETH Zurich, on Bionic exo-skeletons.

On 26-27 September, a curriculum workshop for the series Research ethics and societal impact was hosted by Graz University of Technology, Austria. Gender issues, research ethics and Responsible Research and Innovation have been the main topics of this workshop titled Neuroscience, robotics, AI and medical informatics: New insights with diversity & ethics. PhD candidate Benedikt Feldotto, TUM, presented the talk Robot stereotypes – What you see is not what you get, providing insights on the main workshop theme from the robotics perspective.

From visual perception to motor control

On June 6-7, the European Institute for Theoretical Neuroscience hosted a workshop on visuo-motor integration to present the current stage of the HBP Co-Design Project #4, a project which aims to develop a visuo-motor integration neural network model based on multi-level human neuroscience data. In the dedicated Neurorobotics session, two talks by fellow colleagues Benedikt Feldotto, TUM, and Lorenzo Vannucci, SSSA, highlighted the model implementation capabilities of the HBP Neurorobitcs Platform, NRP. The workshop agenda can be accessed here.

Neurorobotics at the AWS-Summit in Berlin

Hall 1 was full for the “Build Your First Robot with AWS RoboMaker” session at the AWS-Summit in Berlin.
Together with Amazon we presented a introduction into robotics and simulation. We also demonstrated how robotics can be enhanced by including neuroscience using the Neurorobotics Plattform and what physical robotics can learn from nature by showcasing the NRP Mouse.

AWS-Sumit_Berlin_2019

HTM Embodiment – A case study on how embodying a brain in a body unveils the full power of neural computation

As Neuroroboticists in the Human Brain Project we claim that a body interacting in the real world is necessary to understand the process learning and cognition in the brain. Additionally, we claim that a body is not just an addon to let a brain do something in the world, but the body itself executes a part of computation (morphological computation, see for example the concept of a passive dynamic walker) and even further a neural network can only demonstrate its full capabilities if its linked within the sensory-motor space. These conceptual ideas gain more and more popularity, even though most AI applications in robotics still only see the body as a data generation engine, or in opposite direction as an execution engine of the network output.

We conducted a case study with the Hierarchical Temporal Memory¹ implementation to demonstrate how embodiment unveils new capabilities of a neural network in closed loop of a sensory-motor space. The neocortex inspired artificial neural network already demonstrates very good results for on data prediction, classification and find application in language processing scenarios. However, it still lacks breakthrough applications in robotic control scenarios.

The experiments where conducted with a Myororobotics² biomimetic robot arm: 3D printed ball and socket as well as a hinge joint imitate the skeleton of the upper human arm. Tendons, that are controlled by electric motors and routed via springs, imitate muscle activation to enable the compliant characteristics musculoskeletal bodies integrate naturally. The HTM is a very good fit for robotic applications as learning is done continuously and online.

Research was done in three stages, from an HTM application for prediction towards closing the sensory-motor loop for conditioned control:

Proprioceptive motion prediction

We demonstrate that the HTM can well predict motions of the robotic arm, that where generated by a human hand-shake like interaction. The HTM can deal with natural human motion variations and predicts arm positions well after only few iterations.

Neural motion storage and recall for robot control

Once the HTM has learned a periodic motion, that means trajectory points are stored in neural synaptic connections, it can be recalled for motion control. In this base experiment a sinewave motion is learned, and afterwards the trajectory prediction output controls the motion. Hereby, the same motion that was learned can be continued for several iterations, and also parts of the motions a recalled occasionally.

Classical Conditioning of motion primitives

The spatial and temporal associative capabilities of the HTM enable the HTM for Classical Conditioning. According to the figure above, we demonstrated subsequent stimuli and arm liftings to the robotic arm. The HTM learned the stimuli to behavior relationship as well as the motion trajectory itself. After learning, same stimuli let the arm execute the same learned motion by neural recall. Also, variances of stimuli in contrast to the learned ones recall similar motions, e.g. a weaker stimulus triggers a weaker motion execution.

While the first experiment is a special application of HTM predictions, the second one is a first control example Jeff Hawkins already suggested that prediction outputs are similar to motor control commands. The third experiment combines both results. Here, the neural network is rather a client of the real world situation. The HTM reveals a behavior according to Classical Conditioning that would not have been observed just sticking with the principle of applying AI output to robotics, but letting robotics “control” the AI neural network. Ultimately, we see our results a first step towards agents that are able to learn from their environment and executing behaviors autonomously triggered by environmental stimuli. Due to the important role of Classical Conditioning in every mammal, we also hope to better understand the role of unsupervised learning for behavior learning.

We appreciate to see our research results published in the “Bioinspiration and Biomimetics” journal and hope to foster the discussions on the importance of AI embodiment for truly intelligent robot development. Check out the paper results at https://doi.org/10.1088/1748-3190/aae1c2

Hierarchical Temporal Memory: https://numenta.org/
Myorobotics: https://myorobotics.readthedocs.io/en/master/

NRP @ The 28th Annual Conference of Japanese Neural Network Society

The organizers brought some additional chairs as the room was filled with people interested in the Neurorobotics Platform. We presented our simulation platform at the 28th Annual Conference of Japanese Neural Network Society (JNNS) at Okinawa Institute of Science and Technology (OIST). After a short introduction of Neurorobotics research we did a live demonstration and showcased our latest experiments. The platform gained a lot of interest and we could already talk about specific research experiments that we will integrate in the NRP in collaboration with institutes from Japan in the next weeks.

During the poster session we presented our two accepted papers “Multisensory Integration in the HBP Neurorobotics Platform” and “A Control Hierarchy Inpspired by the Spinal Cord to Exploit Self-Organizing Motion Primitives for Purposeful Trajectory Generation”. Looking forward to the last day of interesting presentations about enhancements in Deep Learning, Spiking Neural Networks and Neurorobotics.

Human Brain Project Summit 2018

The Neurorobotics group is currently attending the annual Human Brain Project Summit, this year taking place in Maastricht, and well there is quite a lot to report!

Starting on Monday we organized a joint booth with the division of Cognitive Architectures at the Open Day. People from all ages, research and industry as well as quite diverse disciplines had a look at the latest features of our Neurorobotics Platform, interacted with our biologically inspired robots and could even use Virtual Reality glasses to look and feel through a robots eyes. The Open Day was a great success as even more people attended compared to our last year booth representation.

On Tuesday Mariya Gabriel, the European Commissioner in charge of Digital Economy and Society, opened the HBP Summit and we got the opportunity to present our latest research results at our robots booth. Apparently, she really liked the ongoing development as not only she stressed the Human Brain Project being one of the research priorities for the next EU budget, but also the robots booth got featured on her twitter account!

Strengthening economy by pushing #AI #HPC & #Digital #Skills is on the top of our #research priorities in next #MFF #EUbudget. @HumanBrainProj is a wonderful example and we do our outmost to continue the support in #HorizonEurope. #HBPsummit2018 @FETFlagships pic.twitter.com/r1oJl1oCFG

— Mariya Gabriel (@GabrielMariya) October 16, 2018

We now are gathering to exchange the latest results, discuss further collaboration opportunities within the Human Brain Project and outline the next research steps in talks, parallel sessions and workshpos. Afterwards, Thursday and Friday, we will conclude the week with an SP10 Neurorobotics meeting.

Neurorobotics in the Human Brain Project at IROS Conference in Madrid

Last week we presented the Human Brain Project and in particular the Neurorobotics Subproject to the world wide robotics community at the International Conference on Intelligent Robots and Systems (IROS) in Madrid. During the exhibition our Human Brain Project booth presented facts and figures about the project and we answered any question from the robotics community. As a major, the Neurorobotics Platform was advertised on our posters but as well on our video screen and to be directly tested accessing our servers on a laptop. We could engage a lot of interested researchers and companies. Potential users of our platform but also multiple collaboration perspectives have been discussed and will be refined in the ongoing weeks.

Our booth was located right next to other European robotics projects. In particular we could also exchange and strengthen our collaboration with our booth neighbors such as the EDEN2020 to develop “An Enhanced Ecosystem for Neurosurgery in 2020”. Also funded by the Horizon 2020 European Funding Programme, intersections in terms of brain health and robotics are key elements for exchange and support across the european project landscape.

Bio-inspiration and modularity make robotic locomotion adaptable in the NRP

Current robotic control strategies are mainly based on trajectory plans that adjust the movements based on the next desired state. These control policies do not efficiently perform where the dimensionality of the control problem increases or disturbances are perturbing the system from the external environment. These issues are critical in locomotion tasks and the need for different control methods arises. To make autonomous robots able to move in a real and dynamic environment, the research has focused on biologically inspired controller, such as neuro-controller.

The interaction among different bio-inspired motion controllers whose communication represents a simplified model of the neural locomotion control in vertebrates is possible in the Neurorobotics Platform.

archi_eli

The presented solution combines classical control strategies with reservoir computing and spiking neural networks (Reservoir computing with spiking populations by Alex Vandesompele) to obtain a scalable and adaptable controller by taking advantage of the different learning properties of neural networks. To reflect the scalability of the controller, the experiments are performed on the simulated modular Fable Robot. In the experiment, it is built in a quadruped configuration so that the control architecture is composed of 4 cerebellar microcircuits, called Unit Learning Machines (ulm).

The use of a spiking neural network with reservoir computing as a trajectory planner (Central Pattern Generator, cpg) allows the learning of complex periodic trajectories for the movements of the robotic modules, whose frequency modulation is possible by just changing the frequency of the input signal to the network. Not optimally tuned PIDs give to the robot early stability during the first part of the simulation and provide a torque command for each module. Thus, a cerebellar network composed of 4 micro complexes computes and provides corrective effort contributions based on the inverse dynamics model of each robotic modules.

In the video below, it is possible to appreciate the locomotion improvements of the robot in an experimental simulation. The recording shows the simulation around second 100-130 when the position error is decreasing and stabilizing. The brain visualizer shown the spiking activity of the input population of the Central Pattern Generator (the higher groups) which are reflected in the blinking of one population of the reservoir (lower group). The spike train window (on the left) shows the periodicity of the spike trains which generate the trajectories for the modules (starting from the bottom, the activities of the input populations and one reservoir population are displayed).

video_4mod_fable_v1

The feed-forward cerebellar effort contribution decreases the mean of the position error of 0.3 radiant and its variance of 0.01 radiant compared to the case when just the effort command from the cpg is provided to the robot (the plot concerning the behavior of the second module is shown below). Moreover, the trend of the error is decreasing along the simulation time and the distance covered by the robot with the cerebellar-like neural network contribution is 9.48 m while the cpg controller contributes to have the robot walk for 1.39 m.

The modular configuration of the Fable Robot makes easier to test the control strategy for different configurations of the robot and patterns of locomotion, having the cerebellar-like neural network compensate the error after a short learning phase, since it has previously learned the internal model of the module.

This work was done in collaboration between DTU, Ghent and SSSA teams.

A new integration experiment for the visual system of the NRP

We implemented a new experiment to demonstrate that the NRP is able to run many models together, as a single visual system. Here, a retina model, a deep neural network for saliency computation, a spiking cortical model for early-stage visual segmentation and an echo state network for saccade generation collaborate, despite the differences in their structures. The NRP provides a common framework where models can talk to each other easily.

In the video below, the robot has to keep track of a stimulus that moves on a screen. The saliency model computes where the stimulus lies (most salient region in the visual field), and whenever it is not in the fovea, it delivers a signal to the saccade model, so that an eye movement is generated towards the visual stimulus. When the stimulus reaches the fovea, the second task is to segregate the target (small tilted bars) from the flanking square (impairs target detection). For this purpose, when a saccade is generated, the segmentation model triggers saccade inhibition and then sends local signals that initiate a spreading segmentation process that lies in the networks dynamics, attempting to segregate the target from the flanker. These local signals are sent, using the saliency output as a 2D probability density distribution, so that segmentation is only triggered around regions that are worthy of attention. Only when the segmentation is successful, the target is detected. During the whole experiment, the retina model delivers adaptation to the lighting of the scene. Thanks to it, the cortical representation of the stimulus is stable and segmentation is possible even in low-lighting conditions.

The NRP demonstrates its ability to run large-scale, collaborative simulations. Being able to run a whole visual system gives many novel opportunities to vision research, especially to explain global effects in human vision.

Reservoir computing with spiking populations

Passively compliant robots are robots with passive compliant parts, for instance springs or soft body parts. They can be cheaper, safer and more versatile than traditional stiff robotics. Since the compliance introduces non-linearities that are not easy to model analytically, we need to monitor the body with sensors and use machine learning to interpret those sensors.

Reservoir computing allows to train non-linear dynamical systems using only simple machine learning techniques. The unit of our reservoir is a population of spiking neurons. During training the desired motor commands are gradually taught to the closed loop system (with gradual FORCE learning), as illustrated in the figure below. The only weights that need to be learned are those to the readouts.

Model_overview_v2

The neurorobotics platform provides a convenient interface between the robot model and a spiking ‘brain’. After training, we have a closed loop system consisting of only the body and its ‘brain’. The body sensors drive the ‘brain’ activity which in turn drives the actuators. The motor commands are ‘embedded’ into the dynamics of this system. The animation below shows the resulting trained closed loop gait controller. Spike trains are shown for all neurons in one population.

CL_walkingGait_spikes

If desired, the system can be trained with an extra input to the reservoir (in addition to sensor inputs). This extra input can be coupled with different motor commands (for instance different gaits, or different frequency of the same gait). After training, this extra input can be set by an external actor (be it another ‘brain’ region or just a human) to control the system in real time:

CL_GaitTransition_spikes

New mechanisms in the Laminart model

The Laminart model is a spiking cortical network for early-stage visual segmentation. Using network dynamics simulated in NEST, it is able to parse its visual input into several perceptual groups. It is the largest simulated network available on the Neurorobotics Platform. It explains very well many behavioural results about visual crowding (a behavioural paradigm in which human observers try to identify a target being disrupted by nearby flanking shapes, the target being “crowded” when identification performance is impaired by the flankers – see fig. 1).

Fig. 1. a) Crowding in real life. If you look at the bull’s eye, the kid on the left is easily identifiable. However, the one on the right is harder to identify, because the nearby elements have similar features (orange color, human shape). c) Crowding in behavioural experiments. The visual stimuli on the x-axis are presented in the periphery of the visual field of human observers. The task is to identify the direction of the offset of the target (small tilted bars). Flanking squares try to decrease performance. What is plotted on the y-axis is the target offset at which observers give 75% of correct answers (low values indicate good performance). When the target is alone (dashed line), performance is very good. When only one square flanks the target, performance decreases dramatically. However, when more squares are added, the task becomes easier and easier. All classical models of crowding fail at explaining the latter condition (uncrowding), because they all predict that more flankers induce more interferences.

The model explains the results by stating that the target is crowded when it is considered as in the same perceptual group as the flanker, allowing interferences. When the flankers form a group on their own, they “frame” the target, increasing performance (less interference happens between elements that are in different perceptual groups). This idea explains the behavioural results very well. More than that, the model offers neural mechanisms for perceptual grouping and segmentation. However, these mechanisms work properly only with very simple flanking shapes (squares, rectangles, etc.). As soon as the shapes are slightly more complex (more orientations, various scales, etc.), the mechanisms that are necessary to produce the results do not work properly. This is a problem, because uncrowding happens for any flanking shape (not only squares). Here, we introduce new ways of implementing the key mechanisms of the model.

The first mechanism groups boundaries together (see fig. 2). This is linked to illusory contours computation. Activity spreads to connect edges that are well aligned. The new idea that was implemented here is that the illusory contours can now spread at different scales, according to the configuration of the stimulus. For example, if you are reading, you will tune your illusory contours mechanism, so that horizontal grouping happens between letters of the same word, but not between words, using the fact that letters of a same word are closer than letters from neighbouring words. We used the same idea for our crowding paradigms, where different grouping scales lead to different crowding results.

Laminart grouping — Fig. 2. Top: illustration of how the multi-scale grouping mechanism works (V2, layer 2/3 activity). Left: if tuned for low distances, no illusory contour will spread from the flankers to the target. Right: if tuned for long distances along the horizontal direction, illusory contours easily spreads through long distances, to link all the flanking octagons. Bottom: dynamics of the grouping mechanism. Activity naturally spreads in V2. However, spreading also activates interneurons that activate spreading control. In this state, nothing spreads. However, stimulus onset triggers a damping signal whose duration is specific for the orientation of spreading. It is highly dependent on the stimulus shape. For example, in the top-left stimulus, the closed surfaces impair spreading around the target. In the top-right stimulus, the horizontal alignement of the flankers makes that the damping signal is quite long along the horizontal direction, allowing horizontal boundaries to spread across flankers. When the damping signal stops, the illusory contours are already stabilised and do not spread back.

The second mechanism parses subsets of the image that are linked by boundary grouping to different segmentation layers (different regions of the network’s activity – see fig. 3). In our case, this allows the model to block interferences between parts of the image that belong to different groups. The new mechanism is better, because the former one required tonic activity of many neurons in the segmentation network, which was biologically implausible. Now it is only driven by input-related activity and by a brief segmentation signal.

segmentation1

Laminart segmentation — Fig. 3. Top: Illustration of the mechanisms of segmentation. After the segmentation signal is sent, activity spreads from V2 segmentation layer 0 (SL0) to V2 SL1-2 (after competition, all the flanker-related activity ends up in SL2). Activity can spread throughout all flankers thanks to the illusory contours between them. This is only possible with the new grouping mechanism. Bottom: dynamics of the segmentation mechanism. The interneuron layer is the one that does all the spreading. It also triggers its own control by spreading. The control layer is being inhibited by activity in SL1, allowing the interneurons to shut down activity in SL0, disinhibiting activity in SL1. Because all the dishinibition relies on V2 activity, it will only spread along connected boundaries. You can see in the figure on the top that the control layer acts as a sheath to make sure that the interneuron activity only spreads along V2 activity.

In the future, the new mechanisms will be generalised to more orientations (ongoing work), and the new version of the model is going to be integrated on the NRP. Also, top-down influence on how damping signals distribute across orientations will be investigated.

Going beyond conventional AI

European Robotics Forum 2018 in Tampere

The Neurorobotics Platform developed in the SP10 keeps improving its usability and reliability, and is looking to expand its user base. If the feedback obtained from the audience at the European Robotics Forum (900 registered guests, all roboticists from research and industry) is anything to go by, the NRP is in prime position to fill the need expressed by this community for an interdisciplinary simulation platform than connects neuroscience, AI and robotics.

Indeed, during our Workshop at the ERF and the various discussions that ensued, we were able to speak with a large number of researchers and company representatives from different backgrounds and activities. The overwhelming majority has clearly caught on the potential advantages of using the NRP, especially with standard AI tools such as TensorFlow. Furthermore, we found they were open to considering the ability of the NRP to establish brain-derived intelligent controllers that go beyond conventional AI. Finally, compliant robotics based on the up-and-coming technology of elastic elements that can make robots safe by design is an active area of research where ERF participants also saw potential for the NRP (OpenSim, custom robot designer, etc.).

We are thus looking forward to collaborating with our new contacts in the industry, and to improving the platform even further for their benefit.

(Benedikt Feldotto (TUM) walking the audience through the NRP’s many features)

Towards using the NRP with motor actuated robots

The Neurorobotics Platform Development and Research teams are joining their forces to give the users the possibility to interface their own motor actuated robots with the NRP.
This new intriguing feature will help all the roboticists to deploy their neural controllers on real robots, without the need of any external software.
A new working-mode for the NRP has been created by the SSSA team in Pisa, supporting NEST as the neural interface and ROS for the robotic interface.

The iCub humanoid robot doesn’t have a native support for ROS, but instead it uses YARP (Yet Another Robotic Platform) as a middleware.
This makes it necessary for the NRP to support YARP in order to interact with the hardware iCub platform. Such integration is also being currently developed by the SSSA team.
While not being able to send commands to the iCub yet, this extension allows already to read from its sensors, as for instance the eye cameras.

You can see in the picture below one of our team members waving at the iCub seen from the NRP web frontend.

real_icub_nrp

A further mode is being developed by the KIT team in Karlsruhe, to support the SpiNNaker neuromorphic platform and thus allow whole experiments to go completely on hardware, leaving only the communication and synchronization to the NRP closed loop engine.

In the next months, the NRP will support the execution of transfer function at a fixed rate, in real time, thus allowing the implementation of controllers with the well-estabilished fixed rate approach, widely used in the robotic community.

Handling experiment-specific python packages in the NRP

In this blog post I share my method to handle experiment-specific python packages. Some of my experiments require TensorFlow v1.6, some others need Keras – which itself requires a prior version of TensorFlow – how to handle all this on your locally installed NRP?

My method relies on the package virtualenvwrapper, which allows you to keep your python virtualenv in a single place.

pip install virtualenvwrapper --user

Additionally, I have a custom config which adds a virtualenv to the $PYTHONPATH when I activate it. Copy the postactivate and postdeactivate scripts to $WORKON_HOME – the configuration folder of virtualenvwrapper.

Now, let’s say you have an NRP experiment with custom python packages listed in a requirements.txt. Create a virtualenv for this experiment and install the experiment-specific packages:

mkvirtualenv --system-site-packages my_venv
pip install -r requirements.txt

To access your experiment-specific packages from within the NRP, simply start the NRP from the same terminal, where the virtualenv is activated:

workon my_venv
cle-start

That’s it!

Real Robot Control with the Neurorobotics Platform

Goals

Thanks to its architecture, the NRP should be well suited for directly controlling a real robotic platform with spiking neural networks. Indeed the closed-loop mechanism of the NRP software but also the use of ROS as a middleware enables developments in this direction.

A first motivation for such a project is to outsource the heavy computation load of simulating spiking neural networks on embedded hardware to a fixed server, which can itself interface with neuromorphic hardware like SpiNNaker if required. Consequently, it helps to reach real-time performance on small and low-energy robotic platforms where neuronal computation would have been impossible otherwise. A second motivation is the possibility to partially train the neural network in the NRP, to avoid mechanical and electrical wear of the physical robot. This, however, requires the transferability of neural control from simulation to the real robot after the training; this challenging field is more known as transfer learning and requires a minimum level of accuracy in the simulation models and equations.

Methodology

Our work is focused on real-time locomotion of a compliant quadruped robot using CPGs. To outsource the controller to the NRP as discussed above, we have designed both a robot and its 3D clone in simulation. In this setup, they both have four actuators (one for each “body-to-leg” joint) and four sensors (one for each unactuated “knee” joint). The motor position follows a simple open-loop CPG signal with the same amplitude and phase for each leg, such that the robot will alternate between standing-up and sitting-down periodically for fifty seconds. During this experiment, the sensor values are merely recorded, and not used to regulate the control signal. Given the structure of the kinematic chain with springs and dampers in the knee joints, the system can be explicitly described with a Lagrangian equation. The latter is a function of the physical parameters of the robot which can only be evaluated with a large range of uncertainty as we work with laser-cut or 3D-printed parts assembled with a non-quantified amount of slack. However, the parameters of the simulation model can be set roughly and then optimized to maximize the similarity between the sensors signals output from the hardware and the physics engine. In this setup, we use CMA-ES for this job.

Screenshot from 2018-02-13 19-18-41

Results

After optimization, we can proceed to a qualitative visual validation using different controllers. To this goal, the NRP is installed locally on a machine connected to the same network as the robot. ROS is configured on the robot and in the NRP to enable streaming of actuation and sensing topics between the different machines. A proper calibration of the robot sensors and actuators is also needed to provide meaningful results. A NRP experiment embedding the new optimized model robot is created and used to pilot the robot and the simulated physics. In the current progress, the process seemed to give encouraging results regarding the accuracy and reliability of the simulation, but further tuning is still necessary. An illustration is presented in the following animated figure. The small delay observed between the image on the screen and the robot can be due to the NRP visual rendering in the browser or to the motor PID values.

The aim of pre-training is to exclude actuation patterns that lead to instability of the physical robot (stumbling, falling) and to tune the controller into a good regime. As an interesting fact, our first experiments indicate that there is a good correlation between failure in the simulator and failures in real observations.

Structuring your local NRP experiment – some tips

Within the context of CDP4, we created a NRP experiment showcasing some functional models from SP1/4:

A trained deep network to compute bottom-up saliency
A saccade generation model

Since these models are generic, we want to package them so that they can easily be reused in other experiment, such as the WP10.2 strategic experiment. In this post, we quickly explain the structure of the CDP4 experiment on how modularity is achieved.

We decided to implement the functional modules from SP1/SP4 as ROS packages. Therefore, these modules can be used within the NRP (in the GazeboRosPackages folder), but also independently without the NRP, in any other catkin workspace. This has the advantage that the saliency model can be fed webcam images, and easily mounted on a real robot.

The main difference compared to implementing them as transfer function is synchronicity. When the user runs the saliency model on is CPU, processing a single camera image takes around 3 seconds. If the saliency model was implemented as a transfer function, the simulation would pause until the saliency output is ready. This causes the experiment to run slower but conserves reproducability. On the other hand, implemented as a ROS-node, the simulation does not wait for the saliency network to process an image, so the simulation runs faster.

The saliency model is a pre-trained deep network running on TensorFlow. The weights and topology of the network are saved in data files, loaded during the execution. Since these files are heavy and not interesting to version-control, we uploaded them on our owncloud, where they are automatically downloaded by the saliency model if not present. This also makes it simple for our collaborators in SP1/4 to provide us with new pre-trained weights/topology.

The CDP4 experiment itself has its own repo and is very lean, as it relies on these reusable modules. Additionally, an install script is provided to download the required modules in the GazeboRosPackages.

The topic of installing TensorFlow or other python libraries required by the CDP4 experiment, so that they do not collide with other experiment-specific libraries, will be covered in another blog post.

Implementing cerebellar learning rules for NEST simulator

The cerebellum is a relatively small center in the nervous system that accounts around half of the existing neurons. As we previously documented, the researches from the University of Granada are taking advantage of the NeuroRobotics Platform (NRP) in order to prove how cerebellar plasticity may contribute to vestibule-ocular reflex (VOR) adaptation.

Implementing neurorobotic experiments often requires some multidisciplinary efforts as:

Establishing a neuroscience-relevant working hypothesis.
Implementing an avatar or robot simulator to perform the task.
Developing the brain model with the indicated level of detail.
Transforming brain activity -spikes- into signals that can be used by the robot and viceversa.

The NRP provides useful tools in order to facilitate most of these steps. However, the definition of complex brain models might requires the implementation of neuron and synapsis models for the brain simulation platform (NEST in our particular case). The cerebellar models that we are including involves plasticity at two different synaptic sites: the parallel fibers (PF) and the mossy fibers (MF, targeting the vestibular nuclei neurons).

Synaptic_plasticity

Although we will go deeper into the equations (see the reference above for further details) each parallel fiber synapsis will be depressed (LTD) when a presynaptic spike occurs closely to the occurrence of a complex spike of the target Purkinje cell (PC, see figure). Similarly, the plasticity at the mossy fiber/vestibular nuclei (VN) synapsis will be driven by the inhibitory activity coming from the Purkinje neurons.

These learning rules have been previously implemented for EDLUT simulator and used for complex manipulation tasks in [1]. The neuron and synapsis models have been released in GitHub and also as part of the NRP source code. This work in the framework of the HBP will allow researchers to demonstrate the role that plasticity at the parallel fibers and mossy fibers play in vestibule-occular reflex movements.

[1] Luque, N. R., Garrido, J. A., Naveros, F., Carrillo, R. R., D’Angelo, E., & Ros, E. (2016). Distributed cerebellar motor learning: a spike-timing-dependent plasticity model. Frontiers in computational neuroscience, 10.

Using the NRP, a saliency computation model drives visual segmentation in the Laminart model

Recently, a cortical model for visual grouping and segmentation (the Laminart model) has been integrated to the NRP. From there, the goal was to build a whole visual system on the NRP, connecting many models for different functions of vision (retina processing, saliency computation, saccades generation, predictive coding, …) in a single virtual experiment, including the Laminart as a model for early visual processing. While this process is on-the-go (see here), some scientifically relevant progress already arose from the premises of this implementation. This is what is going to be explained right now.

The Laminart is one of the only models being able to satisfactorily explain how crowding occurs in the visual system. Crowding is a visual phenomenon that happens when perception of a target deteriorates in the presence of nearby elements. Crowding occurs in real life (for example when driving in the street, see fig. 1a) and is widely studied in many psychophysical experiments (see fig. 1b). Crowding happens ubiquitously in the visual system and must thus be accounted for by any complete model of the visual system.

While crowding was for a long time believed to be driven by local interactions in the brain (e.g. decremental feed-forward pooling of different receptive fields along the hierarchy of the visual system, jumbling the target’s visual features with the one from nearby elements), it recently appeared that adding remote contextual elements can still modulate crowding (see fig. 1c). The entire visual configuration is eligible to determine what happens at the very tiny scale of the target!

dfds — Fig. 1: a) Crowding in real life. If you look at the bull’s eye, the kid on the right will be easily identifiable. However, the one on the left will be harder to identify, because the nearby elements have similar features (yellow color, human shape). b) Crowding in psychophysical experiments. Top: the goal is to identify the letter in the center, while looking at the fixation cross. Neighbouring letters make the task more difficult, especially if they are very close to the target. Center and bottom: the goal here is to identify the offset of the target (small tilted lines). Again, the neighbouring lines make the task more difficult. c) The task is the same as before (visual stimuli on the x-axis are presented in the periphery of the visual field and observer must report the offset of the target). This time, squares try to decrease performance. What is plotted on the y-axis is the target offset at which observers give 75% of correct answers (low values indicate good performance). When the target is alone (dashed line), performance is very good. When only one square flanks the target, performance decreases dramatically. However, when more squares are added, the task becomes easier and easier.

To account for this exciting phenomenon (named uncrowding), Francis et al. (2017) proposed a model that parses the visual stimulus in several groups, using low-level, cortical dynamics (arising from a biologically plausible and laminarly structured network of spiking neurons, with fixed connectivity). Crucially, the Laminart is a 2-stage model in which the input image is segmented in different groups before any decremental interaction can happen between the target and nearby elements. In other words: how elements are grouped in the visual field determines how crowding occurs, making the latter a simple and behaviourally measurable phenomenon that unambiguously describes a central feature of human vision (grouping). In fig. 1c (right), the 7 squares form a group that frames the target, instead of interfering with it, hence enhancing performance. In the Laminart model, the 7 squares are grouped together by illusory contours and are segmented out, leaving a privileged access to the target left alone. However, in order to work, the Laminart model needs to start the segmentation spreading process somewhere (see fig. 2).

Grouping — Fig. 2: Dynamics of the layer 2/3 of the area V2 of the Laminart model, for two different stimuli. The red/green lines correspond to the activity of the neurons that detect a vertical/horizontal contrast contour. The three different columns for each stimulus correspond to three segmentation layers, where the visual stimulus is parsed in different groups. The blue circles are spreading signals that start the segmentation process (one for each segmentation layer that is not SL0). Left: the flanker is close to the target. It is thus hard for the spreading signals to segment the flanking square from the target. Right: the flankers extend further and are linked by illusory contours. It is more easy for the the signals to segment them from the target. Thus, this condition produces less crowding than the other.

Up to now, the model was sending ad-hoc top-down signals, lacking an explicit process to generate them. Here, using the NRP, we could easily connect it to a model for saliency computation that was just integrated to the platform. Feeding the Laminart, the saliency computation model delivers its output as a bottom-up influence to where segmentation signals should arise. On the NRP, we created the stimuli appearing in the experimental results shown in fig. 1c, and presented them to the iCub robot. In this experiment, each time a segmentation signal is sent, its location is sampled from the saliency computation map, linking both models in an elegant manner. Notably, when only 1 squares flanks the target, the saliency map is merely a big blob around the target, whereas when 7 squares flank the target, the saliency map is more peaky around the outer squares (see fig. 3). Consequently, the more squares there are, the more probable it is that the segmentation signals succeed in creating 2 groups from the flankers and the target, releasing the target from crowding. This fits very well with the results of figure 1c. The next step for this project is to reproduce the results quantitatively on the NRP.

151626720039374009 (1)

To sum up, building a visual system on the NRP, we could easily make the connection between a saliency computation model and our Laminart model. This connection greatly enhanced the latter model and gives it the opportunity to explain very well how uncrowding occurs in human vision and the low-level mechanisms of visual grouping. In the future, we will run psychophysical experiments in our lab, where it is possible to disentangle top-down from bottom-up influence on uncrowding, seeing whether a strong influence of saliency computation on visual grouping makes any sense.

Fable robot simulator

Fable is a 2 DoF modular robot arm that is being used by the group of DTU in order to develop the task of “Self-Adaptation in Modular Robotics”.

Thanks to the modularity provided by Fable, it is feasible to combine several modules together in order to create different robotic configurations increasing the complexity of the system. In this way, one is able to work on manipulation tasks as well as in locomotion tasks just by plugging a few modules together to form an arm, a worm, a spider,…

In the process to make the Fable robot as accessible as possible to the community, here at DTU we have been working on the implementation of the Fable v2.0 simulator.

We have created 3 different configurations:

A simple robotic arm, 2 DoF (1 Fable module)

fable1

A worm-like robot, 4 DoF (2 Fable module)

fable2

A quadruped-like robot, 8 DoF (4 Fable module)

fable3

This robot model has not been included to the NRP yet, but soon will be available for users. We will keep you updated.

Sensory driven hind-limb mouse locomotion model

In the paper on hind-limb locomotion of a cat in simulation [\textit{reference}], the authors studied the importance two main sensory feedbacks important swing-stance phase switching and which of the particular feedbacks are more important than the other for stable locomotion. In this preliminary work we set-up similar rules to produce locomotion in the mouse model developed in the Neuro-Robotics Platform(NRP). This work will be used to study the role of sensory feedback in locomotion and its integration with feed-forward components such as the Central Pattern Generator’s(CPG’s).In the paper on hind-limb locomotion of a cat in simulation [1], the authors studied the importance two main sensory feedbacks important swing-stance phase switching and which of the particular feedbacks are more important than the other for stable locomotion. In this preliminary work we set-up similar rules to produce locomotion in the mouse model developed in the Neuro-Robotics Platform(NRP). This work will be used to study the role of sensory feedback in locomotion and its integration with feed-forward components such as the Central Pattern Generator’s(CPG’s).

Bio-mechanical model :
We use the Neuro-Robotics platform (NRP) to develop the simulation model and its environment. The rigid body model of the mouse available in NRP was obtained from a high resolution 3D scan of a real mouse. Relationship between the segments are established via joints. For the purpose of this experiment only hind-limbs are actuated. Thus the current model has in total eight actuated joints, four in each hind-limb. Muscles are modeled as hill type muscles with passive and active dynamics. Muscle morphometry and related parameters were obtained from [2]. Each of the actuated joint consisted of at least one pair of antagonist muscle. Some joints also bi-articular muscles. In total the model consists of sixteen muscles. Proprioceptive feedback from muscles and rigid body and tactile information close the loop between the different components of locomotion.

This slideshow requires JavaScript.

Reflex controller :
The idea here is to break the motion of hind limb locomotion into four phases, namely (i) swing (ii) touch-down (iii) stance (iv) lift-off. Proprioceptive feedback and joint angles dictate the reflex conditions under which the phase transitions from one to another. Figure shows the four phases and their sequence of transition. For the hind limbs to change from one phase to another we optimize the muscle activation patterns as a function of proprioceptive feedback and joint angle. This ensures a smooth transition between one phase to another when a necessary condition is met.

Discussions :
With the bio-mechanical model of mouse in NRP and reflex control law we are able to reproduce stable hind-limb gait patterns that are purely sensory driven. The next steps to taken in the experiment are :

Convert reflex laws into neuron based reflex loops
Extend the reflex model for quadruped locomotion
Add a CPG layer to interface with the reflex loops

References :

O. Ekeberg and K. Pearson, “Computer simulation of stepping in the hind legs of the cat: an examination of mechanisms regulating the stance-to-swing transition.” Journal of neurophysiology, vol. 94, no. 6, pp. 4256–68, dec 2005.
J. P. Charles, O. Cappellari, A. J. Spence, J. R. Hutchinson, and D. J. Wells, “Musculoskeletal geometry, muscle architecture and functional specialisations of the mouse hindlimb,” PLoS ONE, vol. 11, no. 4, pp. 1–21, 2016.

Neurorobotics Platform (NRP) User Workshop.

The workshop to introduce Neurorobotics Platform (NRP) was held on the SSSA with the participation of M.Sc. and Ph.D. students. During the workshop, two instructors from the development and research teams provided introductory information on Human Brain Project and, specifically, SP-10 Neurorobotics Platform features including open source technologies used in the NRP (e.g., ROS and Gazebo), development cycles and graphical user interface for the first time users. After the introduction, the users installed the NRP by either following instruction from the HBP Neurorobotics repository or via the bootable flash disks in order to install the NRP for a hands-on session.

The users followed the instructions from tutorial_baseball_exercise to create an experiment as a first demo and to get familiarity with the NRP concepts such as transfer functions, Brain-Body interface, closed-loop engine, to mention a few. This session ended with successfully solving the tutorial requirements with the assistance of the instructors. In the last part of the workshop, the participants discussed to integrate their own on-going project to the NRP. One of the participants expresses his ideas on integration Cerebellar model to the NRP:

My objective is to study the computational characteristics of the cerebellum, responsible for precise motor control in biological agents. Currently, a rate based model of the cerebellum has been implemented to produce accurate saccades in the primate type oculomotor system. My plan is to convert this model into a full spike based cerebellar model in the NEST simulator and apply this control model on the iCub gazebo. The NRP is definitely poised to provide me with this functionality.

Another participant expressed his plan to integrate a continuum robot, I-SUPPORT, to the NRP:

My on-going works with the NRP to create an I-SUPPORT robot model using an OpenSim muscle model to simulate the behavior of the McKibben’s present in the robotic arm.

The last project idea:

The experiments on invariant object recognition and multi-modal object representation by integrating the Hierarchical Temporal Memory, many (deep) layered networks and Spiking Neural Networks to the NRP.

The workshop closed with the evaluation of each session and discussions on the requirements for the proposed projects.

Posted by: Murat Kirtay (SSSA)

Preliminary neural recordings with the M-Platform

Post 4-Fig 1 — **(FIG 1)** The new robotic platform to have an access to the brain cortex and to record neural signal.

The M-Platform, a robotic device for motor rehabilitation after stroke in mice, has been upgraded to allow recording of neural activity during the pulling task (FIG 1). Now the platform provides the unique possibility to integrate kinetic and kinematic data with electrophysiological recordings in awake mice during a voluntary forelimb retraction task.

Post 4-Fig 2 — **(FIG 2)** The interface of OmniPlex D System (Plexon, USA), the system used to perform acute-electrophysiological recordings.

The new device was tested on four healthy mice: an array of 16 channels linear probe (ATLAS, USA) was inserted into the Rostral Forelimb Area (RFA) at 850 µm of depth. Signals were recorded by OmniPlex D System (Plexon, USA) at a frequency of 40 kHz (FIG 2). The analysis of the data was performed offline. We obtained promising results both for the low frequency activity, i.e. Local Field Potential (LFP), and for the high frequency activity, i.e. Multiunit Activity (MUA) and spike sorting. In particular in FIG. 3 a correspondence between the LFP and the force peak is evident; however we are planning to increase the number of recorded animals to generalize our results.

Post 4-Fig 3 — **(FIG 3)** On the top the mean of the LFP recordings in different channels aligned on the onset; at the bottom the mean of corresponding force peaks.

This success paves the way for investigation of neuroplastic events after a cortical damages, i.e. stroke. Moreover the possibility to record spiking activity in the Caudal Forelimb Area (CFA) during the task in healthy animals allows to study firing rate in different channels and find patterns to correlate neural activity and movement of the forelimb.

SP10 + SP6 + CerebNEST New collaboration

Last month, during the last HBP summit, SP10 was able to start working on potential new collaborations with other subprojects and partnering projects in order to keep focus on the main goal of the Neurorobotics platform and the Human Brain Project. Not only for the current phase of the project (SGA 1), but also for the coming years of research.

We are really happy to say that a few days ago, the DTU Neurorobotics team came to an agreement with the SP6 (University of Pavia) and the HBP Partnering project CerebNEST (Politecnico di Milano) in order to integrate to SpiNNaker their cerebellum model (Antonietti et al., 2016 IEEE TBME) that has been already implemented in NEST.

blog2

Having a cerebellar model working in real-time in a neuromorphic platform is going to provide the possibility to analyze the performance of the model with different physical robotics platforms such as the modular robot Fable.

We will keep you updated along the process!

Build a fully functional visual system on the NRP

A collaboration arises from the conjoint goals of CDP4 (a co-designed project within HBP whose goal is to merge several models of the ventral and dorsal streams of the visual system into a complete model of visuo-motor integration) and and WP10.2 (a subpart of the Neurorobotics sub-project – SP10 – that integrates many models for early visual processing and motor control on the NRP). The idea is to import everything that was done in CDP4 in an already existent experiment of the NRP that already connected a model for early visual processing (visual segmentation – ventral stream) to a retina model (see here).

By connecting many models for different functions of the dorsal and the ventral stream on the NRP, this experiment will build the basis of a complete functional model of vision that can be used by any virtual NRP experiment that would require a visual system (motor-control task, decision making based on visual cues, etc.). The first step of the project is to prove that the NRP provides an efficient tool to connect various models. Indeed, different models evolve on very different framework and can potentially be very incompatible. The NRP will thus provide a unique compatibility framework, to connect models easily. The current goal of the experiment is merely to make a proof of concept and thus a very simplified version of a visual system will be built (see image below, and here, if you have access).

WP10-2_CDP4_Experiment (1)

The functions of the visual system will be connected in a modular way, so that it is possible to compare the behaviour of different models for a single function of the visual system, once embedded in a full visual system, and so that any neuroscientist can meaningfully integrate all global accounts of visual perception into his/her model, once incorporated into the NRP experiment. For example, our Laminart model (spiking model of early visual processing for visual segmentation – Francis 2017 [1]), presented here, needs to send spreading signal locally, to initiate parsing of visual information into several segmentation layers. For now, these signals are sent by hand. To gain generality, the model would need bottom-up influence on where these signals are sent (or top-down). It would thus be very interesting for us to send these signals according to the output of a saliency computation model. The Laminart model could then, for example, form a non-retinotopic representation of a moving object by constantly sending signals around saliency peaks computed by the saliency model of CDP4.

Citations:

Francis, G., Manassi, M., Herzog, M. H. (2017). Neural Dynamics of Grouping and Segmentation Explain Properties of Visual Crowding, Psychological Review.

Cerebellar Adaptation in Vestibule-Ocular Reflex Task

Embodiment allows biologically plausible brain models to be tested in realistic environments, receiving similar feedback as it happens in real life or behavioural experimental set-ups. By adding dynamic synapses researchers can observe the effect that behavioural adaptation plays in network state evolution and vice versa. The NeuroRobotics Platform (NRP) notably boosts the embodiment of brain models into challenging tasks, allowing the neuroscientists to skip the technical issues of implementing the simulation of the scene.

One of the nervous centres that has traditionally received more attention in neuroscience is the cerebellum. It has recurrently shown to play a critical role in learning of tasks involving temporally precise movements, and its influence in eye movement control has received frequent experimental support. Although studies from cerebellum-related patients evidence that the cerebellum is also involved in complex tasks, such as limb coordination and manipulation tasks, eye movement control involves a neural circuitry that is simpler and deeply known. However, there still remain many open questions in how the cerebellum manages to control eye movement with such an astonishing accuracy.

Researchers from the University of Granada aim to study the cerebellar role under an “embodied cognition” scenario in which the cerebellum is responsible for solving and facilitating the body interaction with the environment. To that aim, they have set a behavioural task, the vestibule-ocular reflex (VOR), a neural structure facilitating the neural interaction, the cerebellar model, and a front-end human body, the humanoid iCub robot. VOR_Cerebellum_UGR_2

In particular, two particular hypotheses are to be tested with the proposed model: (i) the VOR phase adaptation due to parallel fibre (one of the main plastic synapsis in the cerebellar cortex) plasticity [1], and (ii) the learning consolidation and gain adaptation in VOR experiments thanks to the deep cerebellar nuclei synaptic plasticity [2].

They have modelled the neural basis of VOR control to provide a mechanistic understanding of the cerebellar functionality, which plays a key role in VOR adaptation. On the one hand, this modelling work aims at cross-linking data on VOR at behavioural and neural level. Through the simulation of VOR control impairments, we will examine possible consequences on the vestibular system processing capabilities of the VOR model. This approach may provide hints, or novel hypothesis, to better interpreting experimental data gathered in VOR testing. VOR_Cerebellum_UGR_1

[1] Clopath, C., Badura, A., De Zeeuw, C. I., & Brunel, N. (2014). A cerebellar learning model of vestibulo-ocular reflex adaptation in wild-type and mutant mice. Journal of Neuroscience, 34(21), 7203-7215.

[2] Luque, N. R., Garrido, J. A., Naveros, F., Carrillo, R. R., D’Angelo, E., & Ros, E. (2016). Distributed cerebellar motor learning: a spike-timing-dependent plasticity model. Frontiers in computational neuroscience, 10.

Jesús A. Garrido, Francisco Naveros, Niceto R. Luque and Eduardo Ros. University of Granada.

Practical lab course on the Neurorobotics Platform @KIT

This semester, for the first time, the Neurorobotics Platform will be used as a teaching tool for students interested in embodied artificial intelligence.

The lab course started last week for KIT students, offered by FZI in Karlsruhe. Previously, instead of this practical class, we were offering a seminar were students would make literature research on Neurorobotics and learning. For the seminars, we had around 10 students registering per semester, but this year for the practical lab course, more than 20 students registered, most of them in master degree.

The initial meeting took place last week. The students were splits in seven groups of three. Their first task, familiarize themselves with the NRP and PyNN by solving the tutorial baseball experiment and provided python notebook exercises. All groups were given USB sticks with live boot for them to easily install the NRP, and also access to an online version. Throughout the semester, students will learn about Neurorobotics and the platform by designing challenges and solve them.

Organizers: Camilo Vasquez Tieck, Jacques Kaiser, Martin Schulze, Lea Steffen

Self-Adaptation in Modular Robots at the HBP Summit.

During the last few days at the annual Human Brain Project summit, we had the chance to show to the public some of our experiments.

All these experiments are based on the same concept; a biomimetic control architecture based on the modularity of the cerebellar circuit. Everything integrated by means of machine learning and a spiking cerebellum model which allows the system to adapt and manage changes in its dynamics.

Here it is shown one of the two experiments used at the demo of the first day of the summit. In the “Icub ball balancing” experiment (implemented on the NRP), the Icub robot is able to learn in real time and control the system fulfilling the task for up to 4 joints. The scalability of the system allows to change the number of actuated joints showing the modular and robust aspect of the control architecture.

icub_exp

In the second experiment we were able to test the same control architecture on the real modular robot Fable by Shape Robotics. This time the spiking cerebellar model was implemented using the neuromorphic platform SpiNNaker.

modular_spinn

CDP4 at the HBP Summit: integrating deep models for visual saliency in the NRP

Back in the beginning of 2017, we had a great NRP Hackathon @FZI in Karlsruhe, where Alexander Kroner (SP4) presented his deep learning model for computing visual saliency.

We now presented this integration at the Human Brain Summit 2017 in Glasgow as a collaboration in CDP4 – visuo-motor integration. During this presentation we also shown how to integrate any deep learning models in the Neurorobotics Platform, as was already presented in the Young Researcher Event by Kenny Sharma.

We will continue this collaboration with SP4 by connecting the saliency model to eye movements and memory modules.

deep-dive-cdp4 nrp-saliency

A quadruped robot with traditional computation hardware as a step for a SpiNNaker version

In this post, we describe the components and the architecture of the Tigrillo robot, a compliant quadruped platform controlled with a Raspberry Pi to achieve early research on CPGs and transfer learning. In order to situate the technical description that follows in a scientific context, it may be useful to explain the research methodology that is used:

Optimisation of a parametric CPG controller using the NRP and the VirtualCoach
Transfer and validation on the Tigrillo quadruped platform
Collection and analysis of sensors feedback ont the robot and in the NRP to design and improve a robust closed-loop system
Implementation of the CPGs using NEST on the NRP
Transfer and validation on our quadruped robot embedding SpiNNaker hardware
Comparaison between simulations and the real platforms and extraction of knowledge to iterate on step 1.

The Tigrillo robot enables step 2 by providing a robot to validate the accuracy an general behavior in the NRP simulations.

Mechanical details:

The design process of Tigrillo platform have been guided considering three main features for the robot: compliance, cheapness, versatility. The compliance is a key element in this research as it is believed to add efficiency and robustness to locomotion, like what we can see in biology. However, it also challenges classical control techniques as the dynamics of the robot is now governed by equations with a higher complexity level. On the current platform, the compliance is mainly ensure by using springs in the legs knee instead of actuating them.

Electrical and Software architecture:

Sensors and Actuators: 4 Dynamixel RX-24F servomotors, an IMU (Inertial Measurement Unit), various force and flexion sensors in the feet and legs
Power supply: A DC step-up voltage convertor connected to a 3 cells LiPo battery to supply the boards and motors with a regulated voltage and a stalk current that can rise to 10A when the legs are pushing together and the motors have to deliver a high torque.
Control Board: A OpenCM board (based on an Atmel ARM Cortex-M3 microprocessor) that reads the analog sensor values at a constant frequency and send the position or velocity commands to the servomotors using the protocol standard defined by Dynamixel.
Computation board: A Raspberry Pi with Ubuntu Mate 16.04 that implements a CPG controller included in the same Python software stack that the one used in the NRP and thus easily switch from simulation to trials and validation in the real world.

tigrillo_electrical_schema

The software repository also includes board documentation on the top of the python code used for control and simulation.

Mouse modeling for robotics and neuroscience…

… or why we are building a zoo of artificial mice.

Neurorobotics is about connecting simulated brains to virtual and physical robot bodies. Differently from other approaches in robotics or machine learning, the focus is on high biological plausibility, i.e. a neurorobotic system is designed to capture and predict the quantitative behavior of its biological counterpart as closely as possible. However, what is exactly meant by “close” depends on the granularity of the brain model. Clearly, simple neural networks with only a few neurons can be studied on an equally simple robot. In case of the Braitenberg vehicle experiment on the Neurorobotics Platform, a mobile robot platform with four wheels and a camera is perfectly sufficient. By contrast, brain simulations that are comprised of millions of neurons require realistic body models to simulate and reproduce data from neuroscience as accurately as possible. In this context, standard robots are no longer a viable choice. Neurorobotics is therefore not only about connecting a robot body to a brain but also about the design, simulation, and construction of that body.

The brain models developed in the Human Brain Project are among the most complex and realistic ones ever built and therefore it is only logical that they require the most realistic body models ever built. But how does the perfect body model look like? The answer is both simple and tricky: Since most of the data in neuroscience is obtained from rodents, particularly mice, the perfect choice for the body model is to simulate a mouse body. The tricky part is to determine the level of detail that is necessary to provide meaningful embodiment for the brain models. We are therefore currently designing and building a zoo of different mouse models, each of which serves a specific purpose.

The maximum level of biological detail can only be achieved in simulation. For this reason, we are developing a virtual mouse body that not only looks like a real mouse but that also has the same biomechanical properties. Every bone of the skeleton was modeled individually based on bones of real mice. Combined with the musculoskeletal simulation that will soon be available in the Neurorobotics Platform, the skeleton will enable realistic biomechanical simulations.

mouse_skeleton — Rendering of the completed mouse skeleton

The latest version of the virtual mouse got a soft skin that is fitted to the skeleton. Together with the recently added simulation of the fur, our mouse is almost indistinguishable from its biological colleagues!

mouse_fur — Rendering of the mouse model with skin and fur

Unlike simulation, the real world imposes many constraints on the types of robots that can be built. However, having a physical counterpart to our virtual mouse is beneficial for many reasons. It not only enables direct interaction with the robot but is in particular also a first step to applying results from neurorobotics research in real-world applications. Our first prototype of the mouse robot was built with a focus on small size and biomimetic leg design for robust locomotion. Upcoming releases will not only feature improved mechanics but in particular also include more sensors. Follow our blog to see how our mouse is slowly growing up!

mouse_robot — Completed initial prototype of the mouse robot

Many thanks to Matthias Clostermann, Eva Siehmann, and Peer Lucas for their contributions!

Florian Walter, Technical University of Munich
October 13, 2017

Customized design of musculoskeletal robots with the Robot Designer

In a recent blogpost we introduced the integration of muscle simulations in the Neurorobotics Platform, technically integrating the musculoskeletal simulator OpenSim into the robotic simulator Gazebo. This will enable researchers to conduct experiments with biologically validated muscle actuation. A variety of body models can be studied, either highly biomimetic or of rather technical nature. These studies becomes even more important considering the concept of embodiment, a brain is always embedded in the body and hereby the morphology gets a crucial role in any behavior learning task. For neurorobotics researchers the investigation of this direct coupling of the brain to a morphology in terms of the skeleton structure, body shape, joint assembly as well as muscle attachment points, will give rise to multiple experimental opportunities.

To foster morphological experiments in the Neurorobotics Platform, a fast and user-friendly way for adaptation of the skeleton and muscles is required. Hence, we enhanced our Blender Robot Designer plugin for interactive muscle definition. After creating of a robot by defining the kinematic structure and geometry characteristics, one can now define muscle attachments and paths in a graphical way. As demonstrated in the figure 1 with the mouse skeleton of our CDP1 mouse model, you can select an arbitrary number of pathpoints on the robot model itself. As pathpoints get listed in the user interface you can delete, change the order or refine the location of every point at any time. Afterwards muscle characteristics such as the muscle force and fiber length can be adapted and you can choose between different muscle types provided by OpenSim from biological measurements. Defined muscles get directly exported with the robot model as an additional file in the .osim format and hereby are ready for use in the Neurorobotics platform.

Figure 1: Graphical definition of muscles on a validated mouse skeleton in the Robot Designer

With the introduced muscle definition tool we hope to help researchers tackle arising questions from both a morphological and embodiment perspective: What is the effect of variation of muscle paths and characteristics on the agent’s behavior? How can a brain learn to act with a complex musculoskeletal body model and how does the musculoskeletal structure enhance learning of body motions?

For a quick start with the Robot Designer have a look at our documentation.

Benedikt Feldotto

Technical University of Munich

Optimising compliant robot locomotion using the HBP Neurorobotics platform

If we want robots to become a part of our everyday life, future robot platforms will have to be safe and much cheaper than most useful robots are now. Safety can be obtained by making robots compliant using passive elements (springs, soft elastic materials). Unfortunately, accurate mechanical (dynamic/kinematic) models of such robots are not available and in addition, especially when cheaper materials are used, their dynamical properties drift over time because of wear.

Therefore, cheap robots with passive compliance need adaptive control that is as robust as possible to mechanical and morphological variations. Adaptation training on each physical robot will still be necessary, but this should converge as quickly as possible.

The Tigrillo quadruped robot will be used to investigate neural closed loop motor control for locomotion to address these issues. In particular, we want to investigate how the NRP simulation framework can be used to develop such robust neural control.

As a first step, we implemented a parameterised Tigrillo simulation model generator. Using a simple script, a Gazebo simulation model with given body dimensions, mass distributions and spring constants can be generated to be simulated in the NRP. We then implemented evolutionary optimisation (CMA-ES) in the NRP’s Virtual coach to find efficient motor control patterns, which then generated with spiking population networks using a reservoir computing approach. Finally, these control patterns were transferred to the physical robot’s SpiNNaker board and the resulting gaits were compared to the simulation results.

These steps are illustrated in the video below.

Next steps are:

to tune the parameter ranges of the Tigrillo generator to those that are realistic for the real robot;
to implement sensors on the physical robot and calibrate equivalent simulated sensors;
to use our setup to obtain the desired robust closed loop control and validate both qualitatively and quantitatively on the physical robot.

Many thanks to Gabriel Urbain, Alexander Vandesompele, Brecht Willems and prof. Francis wyffels for their input.

How we simplify your neurons ?

Reproducing complex behaviors of a musculoskeletal model such as rodent locomotion, requires the creation of a controller able to process high bandwidth of sensory input and compute the corresponding motor response.
This usually entails creating large scale neural networks which in turn result in high computational costs. To solve this issue, mathematical simplification methods are needed to capture the essential properties of these networks.

One of the most crucial steps in mouse brain reconstruction is the reduction of detailed neuronal morphologies to point neurons. This is however not trivial, as these morphologies are not only needed to determine the connectivity between neurons by providing contact points, but also by allowing the computation of the propagation of the current through your cell.
This requires however the computation of the potential of every dendritic and axonal sub-sections.

A new model is thus needed that us computationally lighter but generic enough to capture all possible dynamics observed in detailed models.
Recent work by Christian Pozzorini et al. [1] tried to address this issue by creating a General Integrate and Fire (or GIF) point neuron model. This was done by optimizing neuronal parameters by using activities, and input currents.
The GIF model captures more dynamics of biological neurons than the classical Integrate and Fire (or IaF) model, such as stochasticity of spiking or spike-triggered current. However, it still cannot reproduce all dendritic dynamics observed in detailed models.

Simplification_Pozzorini

As a result, Rössert and al. [2] created an algorithm to reduce the synaptic and dendritic processes, by creating cluster of receptors. Each receptor receives multiple currents and treats them using linear filtering. This point neuron model is therefore not only one of the most biologically accurate that exists, but is also faster than a detailed counterpart. This is crucial for large scale simulations.

Simplification_Rossert

Simplification of neuron models is a way to extract the base dynamics of your neurons to simulate only what is needed. It is also an important indicator of the information that get lost in the process. It will be therefore a required step in our project in order to simulate the whole mouse brain and indeed, we will use these models in our project of closed-loop simulation with the rodent body.

[1] Pozzorini, C., Mensi, S., Hagens, O., Naud, R., Koch, C., & Gerstner, W. (2015). Automated High-Throughput Characterization of Single Neurons by Means of Simplified Spiking Models. PLOS Computational Biology PLoS Comput Biol, 11(6).

[2] Rössert, C., Pozzorini, C., Chindemi, G., Davison, A. P., Eroe, C., King, J., … Muller, E. (2016). Automated point-neuron simplification of data-driven microcircuit models.

OpenSim support in the Neurorobotics platform

A key area of research of the Neurorobotics Platform (NRP) is the in-silico study of sensormotor skills and locomotion of biological systems. To simulate the physical environment and system embodiments, the NRP uses the Gazebo robotics simulator.

To perform biologically significant experiments, Gazebo has however been lacking an important feature until now: The ability to model and simulate musco-skeletal kinematics.

Therefore researchers had to rely on ad-hoc implementations calculating effective joint torques for the system at hand, wich is time consuming, error prone and cumbersome.

The physics plugin we implemented provides OpenSim as an additional physics engine alongside the physics engines already supported by Gazebo (ODE, Bullet, SimBody and DART). OpenSim is using SimBody as its underlying framework, thus featuring a stable and accurate mechanical simulation. The OpenSim plugin supports many of SimBody’s kinematic constraint types and implements collision detection support for sphere, plane and triangle mesh shapes along with corresponding contact forces (as exposed by OpenSim’s API).

However, first and foremost it treats physiological models of muscles as first class citizens alongside rigid bodies and kinematic joints. OpenSim is shipped with a number of predefined muscle-tendon actuators. Currently, users of our plugin can use OpenSim’s native XML configuration file format to specify the structure and properties of muscle-tendon systems, which are created on top of Gazebo models specified in Gazebo’s own file format (SDF).

A ROS-based messaging interface provides accessors for excitations and other biophysical parameters allowing to control musco-skeletal systems from external applications such as the Neurorobotics platform.

As demonstration of the capabilities of our physics plugin, we augmented a simple four-legged walker with a set of eight muscles (one synergist-antagonist pair per leg).

The problem we address in this demo is the reinforcement learning task of deriving a controller that excites the muscles in a pattern such that the walker is driven forward. Our setup consists of a Python application (remote-controlling Gazebo via the ROS-based messaging interface for the OpenSim plugin) performing the high-level optimization procedure and running a neural network (NN) controller.

We employ a simple genetic optimization procedure based on Python’s DEAP package to find parameters of the NN that maximize the score the walker obtains in individual trial runs.

The walker is rewarded for moving forward and penalized for unwanted motion behaviour (e. g. ground contacts of the walker’s body, moving off-center).

During a trial run, the physics simulation is stepped in small time increments, and during each iteration the NN is fed with various state variables. The NN’s output is comprised of excitation levels for the muscles. For simplicity we stuck to well-known artificial neural networks, implemented via the Tensorflow package.

We also experimented with fully dynamic grasping simulation using SimBody’s collision detection system and contact force implementations. Although the simulation setup for the grasping tests only comprised a simple two-jaw gripper and a cubic shape (consisting of a triangle mesh shape), the SimBody engine as used in our plugin was able to maintain a stable grasp using fully dynamic contact forces, tackling a problem that is notoriously difficult to solve with other physics engines.

Another application using the OpenSim plugin for Gazebo features a simplified muscle model of a mouse’s foreleg actuated by a neuronal controller modelled according the spinal cord of a real mouse. The details of this experimental setup will be covered in a separate blog post.

The OpenSim plugin does not support all of the features implemented with other engines in Gazebo. For instance, some joint types are not implemented yet. Also, some features unique to OpenSim (like inverse dynamics simulation) are not yet available in the current implementation.

To simplify the design of kinematic models with muscle systems and custom acutator models, it is planned to provide researchers and users of the NRP with a consistent, simple way to specify muscles via a graphical interface using the NRP‘s Robot Designer application.

A one-day workshop during the last Performance Show in Ghent

Last week, we had the chance to organize the first edition of a SP10 Performance Show in the city of Ghent, Belgium. This two-days meeting between all the partners involved in the HBP Neurorobotics subproject (SP10) was an opportunity to discuss the latest progress of each research groups and ensure a convergence of views and efforts for the next events, researches and developments.

SP10 Performance Show September 2017 — A discussion during the SP10 Performance Show

On the second day, we divided our work into two tracks. Whereas the Main Track dealt with administrative and research activities, the Secondary Track was organized as a workshop on the theme Thinking the NRP of the Future. It was formatted as short one-day hackaton where everyone started by summarizing one or several iconic research advances that had been done in the last year in his field, which helped us grouping into 4 different work teams :

Reinforcement Learning with the NRP
Integrating worms brains and soft bodies in the NRP
Real-time interaction between real and simulated robots in the NRP
Helping research on visuomotor learning with the child using simulations in the NRP

Each of those teams brainstormed to imagine and design an experiment that could help research to move forward and a list of requirements in term of developments it would need to be achieved. After lunch, the results of this brainstorm were presented to everyone to get feedback and comments before we started working on designing a first prototype in the NRP and coding some useful models that we would need in further work. To be continued then…

First validation of the virtual M-Platform

The virtual model of the robotic platform has to accurately reproduce movements of the slide according to the applied force at different values friction force levels (FIG 1). The friction levels, that in the M-Platform are modulated with an actuated system, are reproduced on the virtual model regulating the friction coefficient of the slide. This study has been carried out as a joint work with the Prof. LASCHI’s group (SSSA, member of SP10).

FIG 1bis blog — **(FIG 1)** M-Platform on the Gazebo Simulator

We tested a pool of animals performing the pulling task on the real M-Platform in different conditions (i.e. increasing friction force levels to be overcome in order to perform the task). The animals performed a force through their forelimb trying to pull a slide back until a resting position. These real force signals have been used as inputs to the simulator to evaluate if the output monitored variables (i.e. the variation of position of the slide following the application of the force) could be comparable between real and simulated environment. Reasonable results for single pulling movements have been observed, whereas same synchronicity and trend but less reproducibility have been seen for multiple movements (FIG 2).

We think that these results are due to the difficulties to model the inertial force of the linear slide acting on the real M-Platform, one-two order of magnitude lower than the friction force and the force performed by the animal. Indeed for high force peaks (resulting into single movements), the animals are able to complete the entire pulling movement (10 mm) and this is properly simulated in the NRP. However when the force peaks are lower in amplitude, in the real experiment the inertial force allows longer movements than the simulated ones. These latter are generated by the simulated model by means of the application of the force overcoming the friction level. Thus, whenever the force goes down this threshold, suddenly the movement is stopped not describing the real movement of the slide. Although the variation in position is different, the synchronicity of the movements and its trend continue being the same.

FIG 2 bis blog — **(FIG 2)** Two examples of the comparison between real and simulated experiments

In Figure 2 on the left a single force peak (red curve) overcoming the friction value (0.4N) is recorded during a real pulling task performed by a mouse on the M-Platform. The resulting variation of position is shown on the bottom left panel (red curve). The same real force has been used as input force acting on the handle-joint of the simulated M-Platform. This over-friction-threshold force (computed force, blue curve) can generate a simulated movement in the NRP model, as shown in the blue line on the bottom left panel, similar to the real position curve. On the right panels, multiple force peaks (red curve) overcoming the friction value (0.4N) are recorded during a real pulling task performed by a mouse on the M-Platform. Same procedure as previously described has been followed. In this case the trend is similar between real and simulated positions and the synchronicity between force peaks and movements is still present.

Development of an interface board to connect neuromorphic hardware with real world and simulated robots

Neuromorphic computing systems (e.g. SpiNNaker) allow real-time closed-loop robot control in simulation and real-world robots and fascilitate the use of neuromorphic sensors such as silicon retinae and silicon chochleae, because of the spiking nature of these systems.

To connect such neuromorphic hardware to the NRP, an interface board was developed to allow the communication of spikes between SpiNNaker and neuromorphic sensors and actuators.

The current systems allows for up to 500.000 events to be processed per second on five UART ports simultaneously, which is significantly faster compared to the Ethernet interface SpiNNaker provides with about 20.000 events per second.

The second iteration of the interface board will integrate communication via UART, SPI, CAN, and high-speed USB, while also increasing the throughput by using a more advanced microprocessor and optimised CPLD programming (Fig. 1).

IOBoard_2ndIt — Fig. 1: A prototype of the second iteration of the interface board currently in development.

Showcases of the interface board include connecting SpiNNaker to a 2 DOF MyoRobotics system (Fig. 2), a modular framework for the development of compliant musculoskeletal robots. The results of this experiment are published in [1], but research is still ongoing.

MyoRobotAndSpiNNaker_edited — Fig. 2: Example setup of the interface board with SpiNNaker and MyoRobotics 2 DoF arm [1].

Future work will include more showcases of real-world robots driven by neuromorphic hardware as well as the integration of these robots into the NRP, to make these robots accessible to a broader user base and to provide an infrastructure to enable researchers to test their control algorithms on real robots.

[1] RICHTER, Christoph, et al. Musculoskeletal robots: scalability in neural control. IEEE Robotics & Automation Magazine, 2016, 23. Jg., Nr. 4, S. 128-137.

Connecting the Laminart model to a retina modelling framework on the NRP

After having integrated a cortical model for visual segmentation to the NRP, we (Laboratory of Psychophysics, EPFL) connected it to a framework for retina modelling that was already integrated to the NRP. Early August, collaborating with SSSA, we could design a virtual experiment where the iCub robot performs a visual segmentation task, using both retinal and cortical model (see next figure).

AAA

The iCub robot performs a visual segmentation task, using the Laminart model. The goal is to detect the target (small tilted lines). This is only possible if the nearby flankers are segmented by the model. The retina model delivers its output to the Laminart model. This experiment was done to check the compatibility between both models. The scientific goals for this connection is to use the retina to deliver gain controlled input to the Laminart, to gain insights about how color information can be used by the Laminart model to create perceptual groups and to see how retinal magnification can have an influence on grouping. Left windows: output of the Laminart model (top: V2 activity – bottom: V4 activity) ; the model parses visual information into different perceptual groups, thanks to grouping mechanisms. Center windows: output of the retina model (ON- and OFF-centered ganglion cells activity). Right window: output of the brain visualiser, displaying all the neurons of the Laminart model (here: approximately 500’000 IaF neurons).

The future plan for this virtual experiment is to use the connection between the retinal and the cortical model to extend the predictions of the Laminart model to more general cases. For now, the model is the only one to explain many behavioural data about visual crowding (Francis, 2017 [1]), using grouping mechanisms, and it will be very interesting to see how color information is used by the visual system to group elements together. We will use data about crowding and color (Manassi, 2012 [2]) to validate the connection.

More in general, the NRP can be used to give a realistic framework to any model. For example, we tried to see how the Laminart model behaves in realistic conditions by adding some feedback, making the robot move its eyes towards the target when it is detected. The outcome was that the segmentation was not stable, if the bottom-up input was shifted by an eye movement. Using this, we designed a mechanism explaining how vision can generate non-retinotopic representation of objects (see next figure).

output2 (Converted)

The robot moves its eyes towards the target when detected. Each eye movement triggers new segmentation signals whose location adapt to the amplitude and the direction of the eye movements (low neuronal cost). Using this simple mechanism, the model can generate a non-retinotopic representation of the perceptual groups.

Citations:

Francis, G., Manassi, M., Herzog, M. H. (2017). Neural Dynamics of Grouping and Segmentation Explain Properties of Visual Crowding, Psychological Review.
Manassi, M., Sayim, B., & Herzog, M. H. (2012). Grouping, pooling, and when bigger is better in visual crowding. Journal of Vision, 12(10), 13-13.

A neuro-biomechanical model that highlights the ability of spinal sensorimotor circuits to generate oscillatory locomotor outputs

The goal of this project is to uncover the functional role of proprioceptive sensorimotor circuits in motor control, and to understand how their recruitment through electrical stimulation can elicit treadmill locomotion in the absence of brain inputs. This understanding is pivotal for the translation of experimental spinal cord stimulation therapies into a viable clinical application.

To this aim, we developed a closed loop neuromusculoskeletal model that encompass a spiking neural network of the muscle spindle pathway of two antagonist muscles, a musculoskeletal model of the mouse hindlimb, and a model of epidural electrical stimulation (Figure 1). The network includes alpha motoneurons, Ia inhibitory interneurons, group II excitatory interneurons, and group Ia and group II afferent fibers. The number of cells, the connectivity, and the firing behavior of alpha motor neurons was tuned according to experimental values found in literature. The effect of epidural electrical stimulation was integrated in the neuronal network by modelling every stimulation pulse as a supra threshold synaptic input in all the cells recruited by the stimulation. An experimentally validated FEM model of the lumbar rat spinal cord was used to compute the percentage of fibers recruited by the stimulation.

Closed loop simulations were performed by using the firing rates of the motoneurons populations as a signal to control the muscles activity of the musculoskeletal model, while using the muscles length information coming from the musculoskeletal model to estimate the firing rates of the neural network afferent fibers. In particular, the firing rates of Ia and II afferent fibers were estimated using an experimentally derived muscles spindle model.

The preliminary results show that muscle spindle feedback circuits alone can produce alternated movements typical of locomotion, when biomechanics and gravity are considered.

Current work is being performed in order to expand the modeled muscle spindle circuitry to control all the main hindlimb muscles together. To this purpose, the developed network will be used as a template for every couple of antagonist muscles and heteronymous connections across the different joints will be implemented. With this complete model of the hindlimb muscle spindle circuitry we will be able to assess whether this single sensorimotor pathway is sufficient to produce treadmill locomotion in combination with EES, or whether other spinal neural networks are necessarily involved.

HBP_Blog1

Figure 1 : Closed loop simulation framework of Spinal Cord model and rodent hind limb to study epidural electrical stimulation

Emanuele Formento (PhD, TNE & G-Lab, EPFL)
Shravan Tata Ramalingasetty (PhD, BioRob, EPFL)

Successful NRP User Workshop

Date: 24.07.2017
Venue: FZI, Karlsruhe, Germany

Thanks to all of the 17 participants for making this workshop a great time.

Last week, we held a successful Neurorobotics Platform (NRP) User Workshop in FZI, Karlsruhe. We welcomed 17 attendants over three days, coming from various sub-projects (such as Martin Pearson, SP3) and HBP outsiders (Carmen Peláez-Moreno and Francisco José Valverde Albacete). We focused on hands-on sessions so that users got comfortable using the NRP themselves.

Thanks to our live boot image with the NRP pre-installed, even users who did not follow the local installation steps beforehand could run the platform locally in no time. During the first day, we provided a tutorial experiment, exclusively developed for the event, which walked the users through the many features of the NRP. This tutorial experiment is inspired from the baby playing ping pong video, which is here simulated with an iCub robot. This tutorial experiment will soon get released with the official build of the platform.

On the second and third days, more freedom was given to the users so that they could implement their own experiments. We had short hands-on sessions on the Robot Designer as well as Virtual Coach, for offline optimization and analysis. Many new experiments were successfully integrated into the platform: the Miro robot from Consequential Robotics, a snake-like robot moving with Central Patterns Generators (CPG), revival of the Lauron experiment, …

Screenshot from 2017-09-08 14-29-33_crop

We received great feedback from the users. We are looking forward for the organization of the next NRP User Workshop!

A primary test of the upgraded M-Platform

The M-Platform is a robotic device for mice that mimics a human robot device for upper limb stroke rehabilitation (the “arm-guide”) [1]. This platform allows head-fixed mice to carry out intensive and highly repeatable exercises with the forelimb, specifically repeated sessions of forelimb retraction [2]. The new upgrade of the M-Platform is the design of a component providing a variable level of friction to the slide (FIG 1).

FIG 1 blog — **(FIG 1)** The new component of the M-Platform used to finely control the static friction acting on the slide movement. It is composed of felt pad contacting the slide (2) moved by a screw connected to a servo motor (1), controlled by a microcontroller. The working area of the animal (3) is not obstructed by the new component.

To test the upgraded M-Platform, an experimental protocol has been designed. The experimental group consists of mice performing a two-weeks training with high friction (0.5N) which are compared to a control group (training with a lower friction (0.3N)). We measured also isometric force during the pulling performance. First results have shown higher isometric force and better performance for high-trained animals compared to controls (FIG 2).

FIG 2 blog — **(FIG 2)** Preliminary results after a protocol to evaluate the effect of the friction in the pulling task (trials). The protocol consists of a 2 weeks of training, 10 trials/day per 4 days, for two groups of animals. A slight change of the training condition can modify the strength performed by healthy animals.

This experiment has been designed not to use injured (e.g. stroke) animals but healthy ones. In this way a possible translation to the complete NRP model (comprising also the point neuron model simulating motor cortex and biomechanical model) should be more feasible.

Bibliography

[1] Reinkensmeyer DJ, Kahn LE, Averbuch M, McKenna-Cole A, Schmit BD, Rymer WZ (2000). Understanding and treating arm movement impairment after chronic brain injury: progress with the ARM guide. J Rehabil Res Dev 37; 653-662.

[2] Spalletti C, Lai S, Mainardi M, Panarese A, Ghionzoli A, Alia C, Gianfranceschi L, Chisari C, Micera S, Caleo M (2014). A robotic system for quantitative assessment and poststroke training of forelimb retraction in mice. Neurorehabil Neural Repair 28: 188-196.

Simulating tendon driven robots

According to the concept of embodiment, a brain needs to be connected to a body interacting with the world for biological learning to happen, developing biomimetic robots is crucial to fully understand human intelligence. Here, a tendon driven approach can model muscle behavior in terms of flexibility, compliance and contractive force.

While this concept is clearly beneficial for research, it is very difficult to accurately model in simulation. In contrast to classical robots with motors applying torques in the joints, the simulation needs to apply forces along wrapped ropes mimicking tendons and muscles. The artificial muscles developped in the Myorobotics [1] project include mechanical parts for flexiblity and force as well as electrical control in different operating modes as seen in Figure 1. To close the reality gap all physical properties need to be considered in modelling.

5917_682_547

Figure 1: Myorobotics muscle unit (from [2])

We implemented a plugin for Gazebo that finally allows us to simulate the Myorobotics muscle setup. The plugin models tendon kinematics as well as mechanical and electrical properties of the technical actuator. The calculated forces derived from control commands can now be applied directly to a robot simulated in Gazebo. This brings it one step closer to being integrated into the NRP, allowing us to equip muscle units to arbitrary robot morphologies.

Ultimately, this will enable us to compare simulated biological muscles simulated by OpenSim to the technical muscle of Myorobotics modelled with this plugin. Eventually, this will help to build better biomimetic muscle units behaving just like their biological counterparts.

Bibliography:

[1] http://www.myorobotics.eu

[2] C. Richter, S. Jentzsch, R. Hostettler, J. A. Garrido, E. Ros, A. Knoll, F. Röhrbein, P. van der Smagt, and J. Conradt, “Scalability in neural control”, IEEE ROBOTICS &AUTOMATIONMAGAZINE, vol. 1070, no. 9932/16, 2016.

Alexander Kuhn, Benedikt Feldotto (TU München)

Paper accepted: Towards Grasping with Spiking Neural Networks for Anthropomorphic Robot Hands

We got a paper about grasping with spiking neural networks accepted for ICANN 2017!

The complete architecture is shown in the figure. The hand network (left ) receives the proprioception of all fingers and a grasp type signal to generate fingertip targets. Each finger network (middle) receives its proprioception and fingertip target to generate motor commands.

Screen Shot 2017-07-10 at 11.00.48

Abstract:

Representation and execution of movement in biology is an active field of research relevant to neurorobotics. Humans can remember grasp motions and modify them during execution based on the shape and the intended interaction with objects. We present a hierarchical spiking neural network with a biologically inspired architecture for representing different grasp motions. We demonstrate the ability of our network to learn from human demonstration using synaptic plasticity on two different exemplary grasp types (pinch and cylinder). We evaluate the performance of the network in simulation and on a real anthropomorphic robotic hand. The network exposes the ability of learning finger coordination and synergies between joints that can be used for grasping.

Keywords:

grasp motion representation, spiking networks, neurorobotics, motor primitives.

[1] J. C. Vasquez Tieck, H. Donat, J. Kaiser, I. Peric, S. Ulbrich, A. Roennau, Z. Marius, and R. Dillmann, “Towards Grasping with Spiking Neural Networks for Anthropomorphic Robot Hands,” ICANN, 2017.

Registration now open: Neurorobotics Platform User Workshop

The Neurorobotics User Workshop will take place at FZI, Karlsruhe, from the 24th to the 26th of July. This workshop is a great opportunity for new users to integrate with the community and make progress with their own experiments.

The event is free of charge but registration is mandatory.
Please register for the workshop here.

Mid-Term Vision for HBP

This vision is for a seven-year time horizon: it is to be achieved by the end of the regular funding period of the HBP, i.e., by the time the HBP enters the status of a European Research Infrastructure. So, by 2023, we expect our current research in “Future Computing and Robotics” to have produced a number of unique, tangible results in the form of “products” and a number of ground breaking “base technologies and methods” that will significantly facilitate and accelerate future research in the European Infrastructure in a diverse range of fields.

In conjunction with future computing, HBP’s robotics research plays multiple, significant roles in the HBP:

(Closed Loop Studies): it links the real world with the “virtual world of simulation” by connecting physical sensors (e.g., cameras) in the real world to a simulated brain. This brain controls a body which, in turn, can impact and alter the real world environment. Robotics, therefore, provides the potential to perform realistic “closed-loop-studies”: perception – cognition – action. This will establish a whole new field of robot design: virtual prototyping of robots that can then be readily built as real machines and function like the simulated ones. This will not only speed up robot development by orders of magnitude, it will also dramatically improve the testing and verification of their behaviour within a wide variety of circumstances.
(Brain-Derived Products): it links brain research to information technology by using scientific results (e.g., data, and models of behaviour) obtained in brain research and refining it to a readiness level where it can be used by commercial companies and easily taken up and rapidly turned into new categories of products, e.g., using specialized neuromorphic hardware, also currently being developed by HBP. This will allow novel control technologies that achieve robustness and adaptivity far beyond todays algorithmic controls… and ones that actually rival biologic systems.
(Virtualised Brain Research): it links information technology to brain research by designing new tools for brain researchers, with which they can design experiments and then carry them out in simulation. For example, one can study a completely simulated animal’s navigation or sensorimotor skills as it operates in a completely simulated environment (e.g., a maze or a straight or sinusoidal vertical path), and the signals of the simulated brain will be recorded in real-time for immediate analysis. These same principles can be applied to humans and humanoid avatars, allowing bold and fruitful research on degenerative brain diseases, for example.

We envision that the unique integration of the above three paths will lead to widespread mutually beneficial fertilization and research acceleration through the two-way inspiration of the involved disciplines. The vehicle for bi-directional translation (brain science « robotics) is the HBP’s neurorobotics platform.

At this point, we can see the following vision taking shape: we have taken the first steps towards the design of a virtual mouse. This animal, which only exists in a computer, has eyes, whiskers, skin, a brain, and a body with bones and muscles that function exactly like its natural counterpart. Clearly, all of these elements are still far from being perfect, i.e., from exhibiting behaviour and function corresponding to the original creature. However, the more brain scientists learn about these functions and the more data become available, the more we can integrate said results into the virtual mouse, and the faster we can improve the “mouse fidelity”. In parallel, we will apply the same principles to the simulation of human embodiment. The possibilities are endless.

Using the virtual mouse (or humans, or any other animals) in the future, brain scientists can not only copy traditional design experiments into the computer and study the results immediately, they can also modify the mouse any way they want, e.g., introduce lesions into the brain or cut muscles and study the impact it has. Moreover, they can place as many electrodes or other sensors in the body as they want. But perhaps the most astounding benefits of these new possibilities are that scientists can perform experiments that are very, complex – if not impossible to perform in the real world. This includes very long-term studies with permanent recordings (and these can be done 10,000 times faster than in real-time!), animal swarms with parallel recordings, and plasticity and learning effects over many years.

On the technology side, we can envision a number of brain-derived base technologies that result from our work. One straightforward example is robot-based prostheses that have myo-electric interfaces and which can not only be developed in simulation, but which can be tailor-made or personalized to the properties of one specific person – because every single aspect can be simulated. This is a rather simple example; the disruptive products will most likely involve a complex artificial brain running on neuromorphic hardware and capable of super-fast learning, which, for the first time, would make highly intelligent household robots possible that can adapt their behaviour to various tasks.

Substantial progress towards both a comprehensive understanding of the brain and technologies that are derived from the brain’s working principles can only be made by advancing theory and methodology at the system level. While the fields of artificial intelligence and machine learning in particular have recently gained unprecedented momentum that is primarily driven by the success of big data and deep neural networks, the resulting tools, models, and methods are still highly domain-specific. With the ubiquitous availability of cheap storage, massive processing power, and large-scale datasets, the actual challenge no longer lies in the design of a system that performs a specific task, but in the integration of the wealth of different narrow-scoped models from machine learning and neuroscience channelled into a coherent cognitive model. The platform infrastructure of HBP enables the design and implementation of such a model by integrating different tools, methods and theories in a common research environment. For the very first time, different brain theories, neural network architectures and learning algorithms can be directly comparable to both each other and to experimental ground truth. In this context, neurorobotics serves as a central “workbench” for the study of integrated cognitive models in real-world tasks and as a prototyping tool that enables the seamless transfer of these models into new products and services.

To achieve these goals, we need to reinforce the “input side”, i.e., brain scientists need to talk to roboticists much more intensively than they have done up to now. Then, really new concepts can emerge. One particularly attractive concept could be the automatic generation of models from data: data driven model generation. This would make it possible to use every new data collection to improve the virtual models with a minimum of human intervention and hence keep the virtual robot permanently and synergetically coupled to developments in brain science. Of central importance is the permanent adjustment and calibration of these data models with the corresponding cognitive brain system, which in itself is a complex and long-term endeavour. This goal can only be achieved on the basis of a very close interaction between theorists, data/computer scientists and engineers – and as such, could be a perfect example of a synergistic transdisciplinary cooperation that can only be performed in a European Research Infrastructure.

Collaboration between scientists and developers towards integration work in the NRP

Visual-motor coordination is a key research field for understanding our brain and for developing new brain-like technologies.

To address the development and evaluation of bio-inspired control architectures based on cerebellar features, SP10 scientists and developers are collaborating in the implementation of several experiments in the Neurorobotics Platform.

Ismael Baira Ojeda from the Technical University of Denmark (DTU) visited the Scuola Superiore Sant’Anna (Pisa, Italy) to integrate the Adaptive Feedback Error Learning architecture [1] into the Neurorobotics Platform using the iCub humanoid robot. This control architecture uses a combination of Machine Learning techniques and cerebellar-like microcircuits in order to give an optimized input space [2], a fast learning and accuracy for the motor control of robots. In the experiment, the iCub was commanded to balance a ball towards the center of a board, which the iCub held in its hand.

The experiment was later refined and finished during the Install Party hosted by Fortiss (April 2017).

Next, the AFEL architecture could be scaled up and combined with vision and motor control breakthroughs within the different SPs.

Thanks to all the scientists and developers for your support, especially Lorenzo Vannucci, Alessandro Ambrosano and Kenny Sharma!

iCub ball balancing — The prototype experiment running on the Neurorobotics Platform.

References:

[1] Tolu, S., Vanegas, M., Luque, N. R., Garrido, J. A., & Ros, E. (2012). Bio-inspired adaptive feedback error learning architecture for motor control. Biological Cybernetics, 1-16.

[2] Vijayakumar, S., D’souza, A., & Schaal, S. (2005). Incremental online learning in high dimensions. Neural Computation, 17(12), 2602-2634.

Short-term visual prediction – published

Short-term visual prediction is important both in biology and robotics. It allows us to anticipate upcoming states of the environment and therefore plan more efficiently.

In collaboration with Prof. Maass group (IGI, TU Graz, SP9) we proposed a biologically inspired functional model. This model is based on liquid state machines and can learn to predict visual stimuli from address events provided by a Dynamic Vision Sensor (DVS).

Fig_1_rescaled

We validated this model on various experiments both with simulated and real DVS. The results were accepted for publication in [1]. We are now currently working on using those short-term visual predictions to control robots.

[1] “Scaling up liquid state machines to predict over address events from dynamic vision sensors”, Jacques Kaiser, Rainer Stal, Anand Subramoney et al., Special issue in Bioinspiration & Biomimetics, 2017.

Sensory models for the simulated mouse in the NRP

A biologically inspired translation model for proprioceptive sensory information was developed. The translation is achieved implementing a computational model of neural activity of type Ia and type II sensory fibers connected to muscle spindles. The model also includes activity of both static and dynamic gamma-motoneurons, that provide fusimotor activation capable of regulating the sensitivity of the proprioceptive feedback, through the contraction of specific intrafusal fibers (Proske, 1997¹).

The proposed model is an extension of a state-of-the art computational models of muscle spindle activity (Mileusnic, 2006²). The model developed by Mileusnic and colleagues, albeit complete and validated against neuroscientific data, was completely rate based, thus it was modified in order to be integrated in a spiking neural network simulation. In particular, a spike integration technique was employed to compute fusimotor activation and the generated rate was used to generate spike trains.

The proprioceptive model is implemented on NEST (code available here), in order to provide an easy integration inside the NRP, and on SpiNNaker, for supporting real-time robotic applications. The proposed component can be coupled to both biomechanical models, like musculo-skeletal systems, and common robotic platforms (via suitable conversions from encoder values to simulated muscle length). In particular, this model will be used, as part of CDP1, to provide sensory feedback from the virtual mouse body.

Results of this work have been published in this article:

Vannucci, Lorenzo, Egidio Falotico, and Cecilia Laschi. “Proprioceptive Feedback through a Neuromorphic Muscle Spindle Model.” Frontiers in Neuroscience 11 (2017): 341.

1 Proske, U. (1997). The mammalian muscle spindle. Physiology, 12(1), 37-42.

2 Mileusnic, M. P., Brown, I. E., Lan, N., & Loeb, G. E. (2006). Mathematical models of proprioceptors. I. Control and transduction in the muscle spindle. Journal of neurophysiology, 96(4), 1772-1788.

The virtual M-Platform

Preclinical animal studies can offer a significant contribution to gain knowledge about brain function and neuroplastic mechanisms (i.e. the structural and functional changes of the neurons following inner or external stimuli). For example, an external stimulus as a cortical infarct (i.e. stroke) can produce a cascade of similar neural changes both in a human and animal (i.e. monkeys, rodents etc) brains. And even further stimuli such as input provided during a rehabilitative training can have this impact. The possibility to exploiting the neural plasticity, addressing the treatments in combination with technological advanced methods (e.g. robot-based therapy) is one goal that the HBP is pursuing.

The Neurorobotics Platform is fully part of this picture and is providing an environment that will be an important benchmark for these studies. Two labs from the Scuola Superiore Sant’Anna, in Pisa, are tightly working to develop a virtual model of a experiment carried on in a real neuroscientific environment. The core of this set up is the M-Platform (Spalletti and Lai et al. 2013), a device able to train mice to perform a retraction-pulling task with their forelimb (Figure 1A). During last months, the device has been characterized and upgraded to improve its repeatability (Figure 1B). Meanwhile, a first example of the virtual M-Platform (Figure 1C) has been developed.

nrp_mplatfom — Figure: The real M-Platform (A); the CAD design of the main component of the M-Platform, i.e. actuation and sensing, (B) and its virtual model in the NRP (C)

The main components of the M-Platform (i.e. linear actuator, linear slide, handle) have been converted in a suitable format for the Gazebo simulator. Properties of the model such as link weights, joint limits and frictions have been adjusted according to the real characteristics of the slide. The actuator was connected to a PID controller whose parameters have been tuned to reproduce the behavior of the real motor.

A simple experiment has thus been designed in the NRP (currently installed on a local machine), for testing the behavior of the obtained model. The experiment includes a 100 neurons brain model, divided in two populations of 90 and 10 neurons respectively. In this closed loop experiment, the first neuron population spikes randomly, and the spike rate of the population is converted to a force value picked out of a predefined range, compatible with the range of forces possibly performable by the mouse through its forelimb.

The computed force values are continuously applied to the handle and can move the slide until the starting position. Once there, the second neural population, wired to suppress the first population spike rate when active, is triggered, so there’s no more force acting on the slide. The motor pushes the slide until the maximum extension position and it then comes back to its starting position, letting the loop start again (see video).

Functional components for control and behavioural models

Gaze stabilization experiment

In this work, we focused on reflexes used by humans for gaze stabilization. A model of gaze stabilization, based on the coordination of the vestibulo-collic reflex (VCR) and vestibulo-ocular reflex (VOR) has been designed and implemented on humanoid robots. The model, inspired on neuroscientific cerebellar theories, is provided with learning and adaptation capabilities based on internal models.

In a first phase, we designed experiments to assess the model’s response to disturbances, validating the model both with the NRP and with a real humanoid robot (SABIAN). In this phase, we mounted the SABIAN head on an oscillating platform (shown below) able to rotate along the pitch axis, in order to produce a disturbance.

The oscillating platform. In (a) the SABIAN head mounted on the platform, with its inertial reference frame is shown. The transmission of motion from the DC motor to the oscillating platform is depicted in (b).

In a second phase, we carried out experiments for testing the gaze stabilization capability of the model, during a locomotion task. We gathered human data of torso displacement while walking and running. The data has been used to animate a virtual iCub while the gaze stabilization model was active.

Balancing experiment

Using the same principles of the gaze stabilization experiment, we carried out a balancing experiment for a simulated iCub. In this experiment, the simulated iCub is holding up a red tray with a green ball on top. The goal of the experiment is to control the robot’s roll and pitch joints for the wrist, in order to keep the ball in the center of the tray. The control model for the wrist joints is provided with learning and adaptation capabilities based on internal models.

Visual segmentation experiment

A cortical model for visual segmentation (Laminart) has been built with the aim of integrating it in the neurorobotics platform. The goal is to see how the model behaves in a realistic visual environment. A second goal is to connect it to another model for the retina.

The model consists of a biologically plausible network containing hundreds of thousands of neurons and several millions connections embedded in about 50 cortical layers. It is built functionnaly in order to link objects that are likely to group together with illusory contours, and to segment disctinct perceptual groups in separate segmentation layers.

Up to now, the Laminart model has been successfully integrated in the NRP and first expriments are being built to check the behaviour of the model and discover what has to be added to it to ensure it can coherently segment objects from each other in a realistic environment. Besides, the Laminart model is almost connected to the retina model.

In the future, the model will be connected to other models for saliency detection, learning, predictive coding, decision making, on the NRP, to create a closed loop experiment. It will also take into account some experimental data about texture segmentation and contour integration.

Visual perception experiment

In this work, we evaluated the construction of neural models for visual perception. The validation scenario chosen for the models is an end-to-end controller capable of lane following for an self-driving vehicle. We developed a visual encoder from camera images to spikes inspired by the silicon retina (i.e., the DVS Dynamic Vision Sensor). The veichle controller embeds a wheel decoder based on a virtual agonist antagonist muscle model.

visual-perception-jacques-1

Grasping experiment

During the first 12 month of SGA1, we investigated methods for representing and executing grasping motions with spiking neural networks that can be simulated in the NEST simulator and therefore, the Neurorobotics Platform. For grasping in particular, humans can remember motions and modify them while executing based on the shape and the interaction with objects. We developed a spiking neural network with a biologically inspired architecture to perform different grasping motions, that first learns with plasticity from human demonstration in simulation and then is used to control a humanoid robotic hand. The network is made with two types of associative networks trained independently: One represents single fingers and learns joint synergies as motion primitives; and another represents the hand and coordinates multiple finger networks to execute a specific grasp. Both receive the joint states as proprioception using population encoding, and the finger networks also receives tactile feedback to inhibit the output neurons and stop the motion if a contact with an object is detected.

grasping-camilo-1

grasping-camilo-4

Multimodal sensory representation for invariant object recognition

This functional component integrates multisensory information -namely tactile, visual and auditory- to form an object representation. Although we firstly target invariant object recognition problem using the only visual information, the component is capable of combining other sensory modalities. The model is based on computational phases of the Hierarchical Temporal Memory which is inspired by operating principles of the mammalian neocortex. The model was adapted and modified to extract a multimodal sensory representation of an object. The representation can be interpreted as a cortical representation of perceived inputs. To test the model, we perform object recognition in COIL-20 and COIL-100 datasets in which consist of 20 and 100 different objects (see Figure 1). In details, each object rotated 5 degrees on a turntable and object image was captured by the camera (see Figure2). In addition to image acquisition steps, a number post-processing procedures such as background elimination and size normalization were performed on the images.

multimodal-murat-1

Figure 1 Selected images from different categories.

multimodal-murat-2

Figure 2 A duck object under various rotational transformations.

To obtain object representations, the standard image processing algorithms were performed to binarize and downsize available images in datasets. Then, the model was fed with the processed image data to generate sparsely distributed representation of the perceived images. A sample processed image and cortical representation of the same visual pattern are illustrated in Figure 3 and Figure 4, respectively. Note that, the representation of an object with different sensory inputs can be achieved by same procedure and concatenating the obtained representations for each modality.

Figure 3 A processed visual pattern. Figure 4 Cortical representation of a visual pattern

After obtaining representation for all images, we perform recognition operations by grouping the datasets into two categories which are memory representation (or training set) and unseen object patterns (or test set). The representation similarity metric defined as the number of same active cortical columns (the same active bits in the same location) between existing and unseen patterns. The recognition accuracies are shown in Table below. and were derived via splitting training and testing dataset by 10% to 90% and each time incremented by 10.

Training percent	COIL-20	COIL-100
10	90.4	89.0
20	94.3	91.2
30	96.9	94.9
40	97.2	95.6
50	98.3	96.5
60	98.2	97.0
70	98.4	97.3
80	98.6	97.0
90	98.7	96.8

The obtained results indicate that the modal performs well with single modality. Our ongoing studies focus on integrating multiple sensory information (e.g. tactile) to represent multimodal representation to achieve a grasping task.

Morphological Properties of Mass-Spring-Damper Networks for Optimal Locomotion Learning

Robotic Embodiment

The combination of brain inspired AI and robotics is in the core of our work in the Human Brain Project. AI is a vague concept that originated from computer sciences many decades ago and encompasses all algorithms that mimic some cognitive functions of the human species. They are increaslingly based on methods that learn automatically from big datasets.

However, applying those methods to control robots is not as straightforward as it could seem. Unlike computer software, robots generally evolve in noisy and continuously changing environments but on the other hand, their mechanical complexity can be seen as an asset to simplify the control. This is studied through the fields of embodiment and morphological computation. Extreme examples have shown that mechanical structures could provide very natural behavior with no controller at all.

The Passive Walker experiment from T. McGeer is a powerful demonstration emphazing the importance of the body design versus the controller complexity to obtain robust and natural locomotion gaits.

Towards a Formalization of the Concept

Some recent investigations have tried to formalize the relation between the dynamical complexity of a mechanical system and its capability to require simple control. To this goal, a simple yet efficient tool consists in simulating structures composed of masses connected with actuated damper-spring links.

To extend this research, we developed a basic simulator of mass-spring-damper (MSD) networks and optimized a naive locomotion controller to provide them with efficient gaits in term of traveled distance and dissipated power. Three experiments have been done in open-loop to determinate the influence of the size of a structure (estimated though the number of nodes), the compliance (inverse of the spring stiffness) and the saturation at high powers.

This video presents several simulation renditions. The different locomotion processes displayed are learned through optimization in open-loop control.

In the second part of this work, the capacity of realizing closed-loop control in a very simple way requiring very few numerical computations has then been demonstrated.

The principal components in the closed-loop learning pipeline consist in a readout layer which is trained at each time step and a signal mixer that gradually integrates the feedback in the actuation signal.

Our Contribution

A full discussion about the results is accessible directly in this article under Creative Common license.

This work has been realized at Ghent Uuniversity together with Jonas Degrave, Francis wyffels, Joni Dambre and Benonie Carette. It is mainly mainly academic and provides a methodology to optimize a controller for locomotion and indications on what we can expect from its complexity to be able to realize this experiment. In the future, this knowledge will be used to conduct similar experiments on quadruped robots both in the real world and in simulation using the Neuro-Robotic Platform (NRP) developed in HBP.

Integrating Nengo into the NRP?

On 11th March we had the honor of welcoming Terrence Stewart from the University of Waterloo (http://compneuro.uwaterloo.ca/people/terrence-c-stewart.html) at the Technical University of Munich. During these two days, he first gave a fascinating presentation on Nengo and neural engineering in general.
This was followed by extensive discussions with our developers to investigate a possible integration of Nengo into our platform after it had been installed on his laptop. To this extent, we discussed what overlaps already exist and identified missing parts to make this integration happen.
This yields the opportunity for our NRP to offer additional spiking neuron simulators aside from NEST.
This collaboration would be benefitial for both sides, with us offereing a platform to interface Nengo with Roboy or other muscle based simulations.

20170411_144539 20170411_164538

SP9 Quarterly in-person meeting

We are closely collaborating with SP9 (Neuromorphic hardware) to support big networks in real time. On the 20th and 21st of March 2017, we participated in the SP9 Quaterly in-person meeting to present the Neurorobotics Platform and our integration of SpiNNaker.

SP9 During the meeting, we identified MUSIC as a a single interface between our platform and both supercomputers from SP7 as well as SpiNNaker. We also pointed out the features we were missing in MUSIC to keep the Neurorobotics platform interactive, most importantly dynamical ports and reset.

We also presented some complex learning rules we are working on to help SP9 identify user requirements for SpiNNaker 2 design. We were surprised to learn that one of the most complicated learning rule we are working on – SPORE derived by David Kappel in Prof. Maass group – is also used as a benchmark for SpiNNaker 2 by Prof. Mayr. This reward-based learning rule can be used to train arbitrary recurrent network of spiking neurons. Confident that it will play an important role in SGA2, we sent our master student Michael Hoff from FZI, Karlsruhe to TU Graz to use this rule in a robotic setup.

Reservoir computing for generic motor signal generation

Cyclic movements, for instance in locomotion, can be driven by cyclic neural activity, so called Central pattern generators (CPGs). CPGs have been observed at the spinal cord level and even in neural networks isolated from the brain and from sensorimotor feedback. The speed of CPG controlled locomotion, including shift of gait type, can be controlled by simple high level signals, such as tonic electrical stimulation of the brain stem. At the spinal cord level, sensorimotor feedback is integrated to fine tune the motor signals to the environment.

To integrate higher level commands with sensor/body feedback for motor signal generation, we are developing a control system based on reservoir computing (see figure below). The reservoir consists of populations of spiking neurons that are randomly connected. Inputs to the reservoir are on the one hand a generic periodic signal (modeling the high level command), and on the other hand sensor/body feedback from the robotic body that is to be controlled. The reservoir computing paradigm allows for straightforward extraction of desired motor signals from the resulting reservoir activity.

blogPostFigure

In a future blog post the physical and virtual robotic platform to conduct these experiments will be presented.

Static Validation & Verification for Neurorobotics Experiments: Aims and Scope

Validation and Verification (V&V) techniques have been widely used to make sure that simulation results accurately predict reality. As the NRP is a simulation platform to simulate a neural network embodied by a robot, the primary problem of validating that simulation results can be transferred into reality is an intrinsic problem. In fact, validating that a given neural network connected to a robot in a specific way produces the desired results is the very purpose of the NRP.

However, as of today, these validation tasks are performed entirely dynamic, i.e. through actually simulating the experiments. In this series of blog posts, we investigate how this dynamic validation can be supported by static validation and verification activities.

For this, we see the following advantages:

The neuroscientist gets an early feedback on their experiment. Because a static validation or verification is independent of a concrete simulation, the analsis can be performed before the code is actually simulated. This aid the design of neurorobotics experiments escpecially without a running simulation. For the NRP where the editors are currently only available within a running simulation, this means we could validate for example Transfer Functions before they are actually applied to the simulation.
The simulation platform uses resources more efficiently as no simulation resources are aquired for experiments that cannot be run. As of today, this advantage is not significant, as users may only change an experiment within a simulation unless they are willing to edit the plain XML models, but in the future, this is an important goal.

Static V&V argue on all possible execution paths of an experiment. However, the ability of a neural network to learn and adapt to new situations, but also the complexity of the interactions of a robot with its environment make it infeasible to argue based on single execution paths. Therefore, static V&V techniques are mostly restricted to arguments on all execution paths, in particular errorneous parts.

Therefore, the aim of static V&V in the context of neurorobotics must be to find neurobotics experiments that are errorneous for all errorneous executions, i.e. experiments that include flaws such we know that the experiment is not going to work, regardless of the exact behavior of the neural network.

For a successful validation and verification, we need to look at the three main artifacts in a neurorobotics simulation:

The neural network
The robot
The Transfer Functions that connect the latter.

These parts will be looked at in more detail in future blog post on this subject.

Real Saccades for Virtual Robots

Vision is a central theme of research in both robotics and neuroscience. Yet, even though the requirements faced by robots and humans that need to perceive their environments are quite similar (high resolution, low latency, wide field of view etc.), technical vision systems are fundamentally different from the human visual system. One particular reason for these differences are the special properties of the human eye.

A special characteristic of human vision are saccadic eye movements:

“Saccade refers to a rapid jerk-like movement of the eyeball which subserves vision by redirecting the visual axis to a new location.”

From John Findlay and Robin Walker (2012) Human saccadic eye movements. Scholarpedia, 7(7):5095.

Clearly, compared to a camera that is statically mounted next to a robot, human vision strongly relies on active actuation of the eyes. Importantly, the perspective on the scene changes after every eye movement which in turn influences the following saccades. Investigating computational models for saccadic eye movements therefore calls for a neurorobotics approach which directly captures this closed-loop interdependence between changing visual input and saccade generation.

To address the challenge of developing and evaluating realistic models of saccadic eye movements, SP10 is closely collaborating with other partners from the Human Brain Project in Co-Design Project 4 on Visuo-Motor Integration. In this project Rainer Goebel and Marion Senden from Maastricht University are currently developing a neural model for saccade generation with nest.

Last week, Mario visited us in Munich to integrate a first working version of the model together with SP10-colleague Florian Walter into the Neurorobotics Platform. After an introductory talk by Mario on visuo-motor integration, we directly started developing a new experiment for the Neurorobotics Platform that controls the eyes of our virtual iCub robot based on the output of the saccade model. This experiment, for the first time, interfaces a nest model comprised of analog non-spiking neurons with the Neurorobotics Platform. In future releases of the platform, these new neuron types will give both neuroscientists and roboticists even more freedom in defining their brain models.

In the next step, the saccade generation model will be connected to a salience map model to study saccades in complex visual scenes that are simulated on the Neurorobotics Platform.

Many thanks to the prompt support from the SP10 development team, especially Kenny Sharma!

Saccade-Experiment — The prototype experiment running on the Neurorobotics Platform.

Florian Walter (Technical University of Munich) and Mario Senden (Maastricht University) in front of the integrated prototype experiment on saccade generation.

Videnskab.dk came to interview DTU Center for Playware!

by Ismael Baira Ojeda ¦ Research assistant at DTU – Center for Playware.

The research of the DTU Center for Playware and the Human Brain Project does not go unnoticed in Denmark.

robot_henrik_2_1

Professor Henrik Hautop Lund is in charge of Denmark’s contribution to a major EU project to map the human brain. His group of researchers including Silvia Tolu and Ismael Baira Ojeda at the DTU Center for Playware are developing cerebellar-like models that together with machine learning algorithms are controlling and teaching modular robots how to move (Photo: Henrik Hautop Lund, DTU Electrical Engineering).

Following, a translation of the article:

Artificial brains to provide innovative brain-like technologies.

Approximately 100 research groups collaborate within The Human Brain Project, working at different topics regarding neuroscientific and robotics research.

“Our role involves robotics research, that is to create models of the brain to be put into a simulation of a physical body. We must not only create a complete artificial brain but also implement the interaction between ‘nerve signals’ and movement, “explains Henrik Hautop Lund, head of the Danish contribution to the project.

DTU researchers implement cerebellar-like models using the neuromorphic SpiNNaker platform. Those models are linked via radio to the robot modules achieving the motor control and learning of the desired trajectory.

How is it done?

The artificial brain is implemented on simulations or in neuromorphic hardware. The brain-like model sends signals to a radio transmitter that transmits them to the robot. When the radio signal is received by the robot, the robot reads them and then traces out the movement defined by the code. Source: Ismael Baira Ojeda.

Click here to watch our short demo! – Video edited by Videnskab.dk

This interaction between brain models and robot actuators might make possible the development of more flexible prosthesis in the future that may have a greater human-like movement, explains Henrik Hautop Lund.

“We may eventually create robots that are more compliant and that can adapt better to new or uncertain environments while achieving smooth movements. ” comments Ismael Baira Ojeda.

At the same time, Henrik Lund Hautop thinks that in the future we will be able to enjoy household robots that can better adapt to different households and needs.

“It is not good that a robot has stiff and precise movements that could possibly damage a person if it is to be part of a household or collaborate with humans.” says Ismael Baira Ojeda.

Click here if you feel like reading the original Videnskab’s article in danish.

Gazebo DVS plugin – towards a sensor library

On the NRP, we already support any sensor included by Gazebo. Mostly, they consist of classical robotic sensors such as laser scanner and camera.

However, Gazebo does not include recent biologically inspired sensor, neither does it include neuroscience’s models of organic sensors. Those type of sensors are important for the NRP. To keep the workflow identical for classical robotic sensors and newly developed sensors, we decided to implement the later as gazebo plugins. Essentially, our sensor library will consist of a list of gazebo plugins simulating various biologically inspired sensors.

So far, we implemented a simulation of the Dynamic Vision Sensor (DVS) which is open-source and available on our SP10 github. In the coming month, we will also adapt our implementation of COREM, retina simulation framework [1,2] and wrap it in a Gazebo plugin.

DVS_generic_image_viewer

[1] Martínez-Cañada, P., Morillas, C., Pino, B., Ros, E., & Pelayo, F. (2016). A Computational Framework for Realistic Retina Modeling. International Journal of Neural Systems, 26(07), 1650030.

[2] Ambrosano A. et al. (2016). Retina Color-Opponency Based Pursuit Implemented Through Spiking Neural Networks in the Neurorobotics Platform. Biomimetic and Biohybrid Systems. Living Machines 2016.

NRP User Hackathon @ Karlsruhe

At the end of the ramp-up phase, we realized that the NeuroRobotics Platform (NRP) lacked users despite increasing maturity. To resolve this, in SGA-1 we splitted the core team in two: a development and a research team. The former would continue developing the NRP, while the latter would become driving users. This split became particulary interesting with the rise of new potential users such as joining SP10 partners as well as CDP co-workers.

To engage those potential users with the NRP, we organized the first NRP User Hackathon in FZI, Karlsruhe from the 15th to the 17th of February 2017. During those three days, two NRP and robotic experts (Jacques Kaiser, FZI, & Alessandro Ambrosano, SSSA) helped neuroscientists Alban Bornet (EPFL, joining SP-10) and Alexander Kroner (Maastricht University, CDP4 partner) integrating their models to the NRP. With various knowledge backgrounds, the small committe pair-promming setup allowed everyone to learn from the others.

For Alban’s visual segmentation model implemented in PyNN+NEST, the NRP integration brought light to interesting performances on real scenes, while we would interact with the environment by moving cubes around. His model was the most complex neural model which has ever been run within the NRP, consisting of more than 50 000 neurons and 300 000 synapses for a 20×20 input image size and simple settings. To speed up the run-time of the model, we also ported it to SpiNNaker in batch processing mode.

hackathon-team

For Alexander, we were able to connect his bottom-up visual attention model implemented with Keras+Theano (deep learning frameworks) to ROS, and consequently to the NRP. This gave us some insights into how we would implement an upcoming NRP feature: running arbitrary user code. In this instance, we could wrap his model in a rosnode converting input images to saliency map images. We could imagine a spiking network model taking those saliency maps as input and performing saccadic eye movements.

saliency-combined

Both Alban and Alexander adopted the NeuroRobotics Platform and will spread the word in their respective labs. After the success of this hackathon, it is likely that we will soon organize more to grow the user base of the NRP organically.

The Closed Loop Engine Architecture Explained

In order to simulate arbitrary neurorobotics experiments incorporating a coupled simulation of a neural network and a robot as its physical counterpart, it is required to abstract from the technical details.

Specification in Python

For the specification of a closed loop in the Neurorobotics Platform, we have chosen the Python language, as Python seems very popular among neuroscientists and is generally easy to learn. The specification of a closed loop is divided into Transfer Functions, which can be specified in a Python internal DSL called PyTF. PyTF essentially defines a set of decorators to specify how the parameters of a regular Python function should be mapped either to the neural network or to robot sensors or control channels. Transfer Functions in PyTF look as follows:

import hbp_nrp_cle.tf_framework as nrp
from geometry_msgs.msg import Vector3, Twist
@nrp.MapSpikeSink("left_wheel_neuron",nrp.brain.actors[0], nrp.leaky_integrator_alpha)
@nrp.MapSpikeSink("right_wheel_neuron",nrp.brain.actors[1], nrp.leaky_integrator_alpha)
@nrp.Neuron2Robot(Topic(’/husky/cmd_vel’, Twist))
def wheel_transmit(t, left_wheel_neuron, right_wheel_neuron):
   linear = Vector3(20 * min(left_wheel_neuron.voltage,
                             right_wheel_neuron.voltage), 0, 0)
   angular = Vector3(0, 0, 100 *
                     (right_wheel_neuron.voltage - left_wheel_neuron.voltage))
   return Twist(linear=linear, angular=angular)

Here, the decorators of the Python function describe how the parameters of the underlying Python function should be mapped to Robot control and neural network information. Each decorator specifies the parameter that is mapped and how this parameter is mapped. The first parameter must be named t and must not be mapped. Instead, it is automatically filled with the simulation time.

Runtime Architecture

From this specification, the CLE deducts a runtime architecture of the Transfer Function. For the Transfer Function above, this creates the Transfer Function component WheelTransmit in the diagram below. The CLE then creates the necessary components to connect each required interface of the Transfer Function component with a respective implementation.

runtimearchitecture

Not shown in the diagram, the CLE also deducts a specification to which neurons the leaky integrator components should be connected to. Furthermore, the choice of the concrete component type is

A Transfer Function generally only implements an open loop, thus either forwards information from the neuronal network to the robot or the other way round. To establish a closed loop, other Transfer Functions in the opposite direction are required, as depicted in the lower part of the diagram above.

Static Architecture

To assemble the runtime architecture from a given specification, the CLE uses a static architecture to dispatch which components should be created for a given interface based on the chosen neural and world simulator. A diagram of this architecture is shown below.

staticclearchitecture

For each world and neural simulator, the CLE distinguishes between components managing the control flow and components managing the data flow. The concrete implementation is encapsulated behind one of four interfaces, making the simulators used by the CLE easy to exchange. This separation also makes it possible to reuse data flow implementations for multiple simulators. For example, multiple world simulators use ROS to communicate with the robot.

Continue reading “The Closed Loop Engine Architecture Explained” →

New frontiers Article Explains the Technology Powering the HBP Neurorobotics Platform

frontiers

After our recent Science Supplement article on Neurorobotics in the Human Brain Project, our paper “Connecting artificial brains to robots in a comprehensive simulation framework: the Neurorobotics Platform” now got accepted for publication by frontiers in Neurorobotics. Which of the papers should you read? Definitely both! The Science Supplement article states the key concepts of neurorobotics and outlines how they are reflected in the HBP Neurorobotics Workflow. The new frontiers paper explains how this workflow is actually implemented in the Neurorobotics Platform:

“Combined efforts in the fields of neuroscience, computer science and biology allowed to design biologically realistic models of the brain based on spiking neural networks. For a proper validation of these models, an embodiment in a dynamic and rich sensory environment, where the model is exposed to a realistic sensory-motor task, is needed. Due to the complexity of these brain models that, at the current stage, cannot deal with real-time constraints, it is not possible to embed them into a real world task. Rather, the embodiment has to be simulated as well. While adequate tools exist to simulate either complex neural networks or robots and their environments, there is so far no tool that allows to easily establish a communication between brain and body models. The Neurorobotics Platform is a new web-based environment that aims to filling this gap by offering scientists and technology developers a software infrastructure allowing them to connect brain models to detailed simulations of robot bodies and environments and to use the resulting neurorobotic systems for in-silico experimentation.

In order to simplify the workflow and reduce the level of the required programming skills, the platform provides editors for the specification of experimental sequences and conditions, environments, robots, and brain-body connectors. In addition to that, a variety of existing robots and environments are provided. This work presents the architecture of the first release of the Neurorobotics Platform developed in subproject 10 “Neurorobotics” of the Human Brain Project (HBP). At the current state, the Neurorobotics Platform allows researchers to design and run basic experiments in neurorobotics using simulated robots and simulated environments linked to simplified versions of brain models. We illustrate the capabilities of the platform with three example experiments: a Braitenberg task implemented on a mobile robot, a sensory-motor learning task based on a robotic controller and a visual tracking embedding a retina model on the iCub humanoid robot. These use-cases allow to assess the applicability of the Neurorobotics Platform for robotic tasks as well as in neuroscientific experiments.”

Can’t wait to start reading the paper? A preprint is availalble for free open access download on frontiers.

Publication in a Supplement to Science on Brain-Inspired Intelligent Robotics

The article “Neurorobotics: A strategic pillar of the Human Brain Project” was released in a Science Supplement on “Brain-inspired intelligent robotics: The intersection of robotics and neuroscience”, explaining the importance of our subproject and its research.

To give you an overview, you can find the first section below:

“Neurorobotics is an emerging science that studies the interaction of brain, body, and environment in closed perception–action loops where a robot’s actions affect its future sensory input. At the core of this field are robots controlled by simulated nervous systems that model the structure and function of biological brains at varying levels of detail (1). In a typical neurorobotics experiment, a robot or agent will perceive its current environment through a set of sensors that will transmit their signals to a simulated brain. The brain model may then produce signals that will cause the robot to move, thereby changing the agent’s perception of the environment. Observing how the robot then interacts with its environment and how the robot’s actions influence its future sensory input allows scientists to study how brain and body have to work together to produce the appropriate response to a given stimulus. Thus, neurorobotics links robotics and neuroscience, enabling a seamless exchange of knowledge between these two disciplines. Here, we provide an introduction to neurorobotics and report on the current state of development of the European Union–funded Human Brain Project’s (HBP’s) Neurorobotics Platform (2, 3). HBP is Europe’s biggest project in information communication technologies (ICT) to date (www.humanbrainproject.eu) and is one of two large-scale, long-term flagship research initiatives selected by the European Commission to promote disruptive scientific advance in future key technologies. It will have a duration of 10 years and deliver six open ICT platforms for future research in neuroscience, medicine, and computing, aimed at unifying the understanding of the human brain and translating this knowledge into commercial products.”

Read the entire paper here on page 25:
http://www.sciencemag.org/sites/default/files/custom-publishing/documents/Brain-inspired-robotics-supplement_final.pdf?_ga=1.158217660.785230381.1481986150

(image source: http://www.sciencemag.org/sites/all/themes/science/images/facebook-share.jpg)