Author: sssaneurorobotics

Towards using the NRP with motor actuated robots

The Neurorobotics Platform Development and Research teams are joining their forces to give the users the possibility to interface their own motor actuated robots with the NRP.
This new intriguing feature will help all the roboticists to deploy their neural controllers on real robots, without the need of any external software.
A new working-mode for the NRP has been created by the SSSA team in Pisa, supporting NEST as the neural interface and ROS for the robotic interface.

The iCub humanoid robot doesn’t have a native support for ROS, but instead it uses YARP (Yet Another Robotic Platform) as a middleware.
This makes it necessary for the NRP to support YARP in order to interact with the hardware iCub platform. Such integration is also being currently developed by the SSSA team.
While not being able to send commands to the iCub yet, this extension allows already to read from its sensors, as for instance the eye cameras.

You can see in the picture below one of our team members waving at the iCub seen from the NRP web frontend.


A further mode is being developed by the KIT team in Karlsruhe, to support the SpiNNaker neuromorphic platform and thus allow whole experiments to go completely on hardware, leaving only the communication and synchronization to the NRP closed loop engine.

In the next months, the NRP will support the execution of transfer function at a fixed rate, in real time, thus allowing the implementation of controllers with the well-estabilished fixed rate approach, widely used in the robotic community.


Neurorobotics Platform (NRP) User Workshop.

The workshop to introduce Neurorobotics Platform (NRP) was held on the SSSA with the ​participation of M.Sc. and Ph.D. students. During the workshop, two instructors from the development and research teams provided introductory information on Human Brain Project and, specifically, SP-10 Neurorobotics Platform features including open source technologies used in the NRP (e.g., ROS and Gazebo), development cycles and graphical user interface for the first time users. After the introduction, the users installed the NRP by either following instruction from the HBP Neurorobotics repository or via the bootable flash disks in order to install the NRP for a hands-on session.

The users followed the instructions from tutorial_baseball_exercise to create an experiment as a first demo and to get familiarity with the NRP concepts such as transfer functions, Brain-Body interface, closed-loop engine, to mention a few. This session ended with successfully solving the tutorial requirements with the assistance of the instructors. In the last part of the workshop, the participants discussed to integrate their own on-going project to the NRP. One of the participants expresses his ideas on integration Cerebellar model to the NRP:

My objective is to study the computational characteristics of the cerebellum, responsible for precise motor control in biological agents. Currently, a rate based model of the cerebellum has been implemented to produce accurate saccades in the primate type oculomotor system. My plan is to convert this model into a full spike based cerebellar model in the NEST simulator and apply this control model on the iCub gazebo. The NRP is definitely poised to provide me with this functionality.

Another participant expressed his plan to integrate a continuum robot, I-SUPPORT, to the NRP:

My on-going works with the NRP to create an I-SUPPORT robot model using an OpenSim muscle model to simulate the behavior of the McKibben’s present in the robotic arm.

The last project idea:

The experiments on invariant object recognition and multi-modal object representation by integrating the Hierarchical Temporal Memory, many (deep) layered networks and Spiking Neural Networks to the NRP.

The workshop closed with the evaluation of each session and discussions on the requirements for the proposed projects.

Posted by: Murat Kirtay (SSSA)

Sensory models for the simulated mouse in the NRP

A biologically inspired translation model for proprioceptive sensory information was developed. The translation is achieved implementing a computational model of neural activity of type Ia and type II sensory fibers connected to muscle spindles. The model also includes activity of both static and dynamic gamma-motoneurons, that provide fusimotor activation capable of regulating the sensitivity of the proprioceptive feedback, through the contraction of specific intrafusal fibers (Proske, 19971).

Figure 1 Intrafusal fibers

The proposed model is an extension of a state-of-the art computational models of muscle spindle activity (Mileusnic, 20062). The model developed by Mileusnic and colleagues, albeit complete and validated against neuroscientific data, was completely rate based, thus it was modified in order to be integrated in a spiking neural network simulation. In particular, a spike integration technique was employed to compute fusimotor activation and the generated rate was used to generate spike trains.

The proprioceptive model is implemented on NEST (code available here), in order to provide an easy integration inside the NRP, and on SpiNNaker, for supporting real-time robotic applications. The proposed component can be coupled to both biomechanical models, like musculo-skeletal systems, and common robotic platforms (via suitable conversions from encoder values to simulated muscle length). In particular, this model will be used, as part of CDP1, to provide sensory feedback from the virtual mouse body.

Results of this work have been published in this article:

Vannucci, Lorenzo, Egidio Falotico, and Cecilia Laschi. “Proprioceptive Feedback through a Neuromorphic Muscle Spindle Model.” Frontiers in Neuroscience 11 (2017): 341.

1 Proske, U. (1997). The mammalian muscle spindle. Physiology, 12(1), 37-42.

2 Mileusnic, M. P., Brown, I. E., Lan, N., & Loeb, G. E. (2006). Mathematical models of proprioceptors. I. Control and transduction in the muscle spindle. Journal of neurophysiology, 96(4), 1772-1788.

The virtual M-Platform

Preclinical animal studies can offer a significant contribution to gain knowledge about brain function and neuroplastic mechanisms (i.e. the structural and functional changes of the neurons following inner or external stimuli). For example, an external stimulus as a cortical infarct (i.e. stroke) can produce a cascade of similar neural changes both in a human and animal (i.e. monkeys, rodents etc) brains. And even further stimuli such as input provided during a rehabilitative training can have this impact. The possibility to exploiting the neural plasticity, addressing the treatments in combination with technological advanced methods (e.g. robot-based therapy) is one goal that the HBP is pursuing.

The Neurorobotics Platform is fully part of this picture and is providing an environment that will be an important benchmark for these studies. Two labs from the Scuola Superiore Sant’Anna, in Pisa, are tightly working to develop a virtual model of a experiment carried on in a real neuroscientific environment. The core of this set up is the M-Platform (Spalletti and Lai et al. 2013), a device able to train mice to perform a retraction-pulling task with their forelimb (Figure 1A). During last months, the device has been characterized and upgraded to improve its repeatability (Figure 1B). Meanwhile, a first example of the virtual M-Platform (Figure 1C) has been developed.

Figure: The real M-Platform (A); the CAD design of the main component of the M-Platform, i.e. actuation and sensing, (B) and its virtual model in the NRP (C)

The  main components of the M-Platform (i.e. linear actuator, linear slide, handle) have been converted in a suitable format for the Gazebo simulator. Properties of the model such as link weights, joint limits and frictions have been adjusted according to the real characteristics of the slide. The actuator was connected to a PID controller whose parameters have been tuned to reproduce the behavior of the real motor.

A simple experiment has thus been designed in the NRP (currently installed on a local machine), for testing the behavior of the obtained model. The experiment includes a 100 neurons brain model, divided in two populations of 90 and 10 neurons respectively. In this closed loop experiment, the first neuron population spikes randomly, and the spike rate of the population is converted to a force value picked out of a predefined range, compatible with the range of forces possibly performable by the mouse through its forelimb.

The computed force values are continuously applied to the handle and can move the slide until the starting position. Once there, the second neural population, wired to suppress the first population spike rate when active, is triggered, so there’s no more force acting on the slide. The motor pushes the slide until the maximum extension position and it then comes back to its starting position, letting the loop start again (see video).