Category: Robotics

Bio-inspiration and modularity make robotic locomotion adaptable in the NRP

Current robotic control strategies are mainly based on trajectory plans that adjust the movements based on the next desired state. These control policies do not efficiently perform where the dimensionality of the control problem increases or disturbances are perturbing the system from the external environment. These issues are critical in locomotion tasks and the need for different control methods arises. To make autonomous robots able to move in a real and dynamic environment, the research has focused on biologically inspired controller, such as neuro-controller.

The interaction among different bio-inspired motion controllers whose communication represents a simplified model of the neural locomotion control in vertebrates is possible in the Neurorobotics Platform.

archi_eli

The presented solution combines classical control strategies with reservoir computing and spiking neural networks (Reservoir computing with spiking populations by Alex Vandesompele) to obtain a scalable and adaptable controller by taking advantage of the different learning properties of neural networks. To reflect the scalability of the controller, the experiments are performed on the simulated modular Fable Robot. In the experiment, it is built in a quadruped configuration so that the control architecture is composed of 4 cerebellar microcircuits, called Unit Learning Machines (ulm).

The use of a spiking neural network with reservoir computing as a trajectory planner (Central Pattern Generator, cpg) allows the learning of complex periodic trajectories for the movements of the robotic modules, whose frequency modulation is possible by just changing the frequency of the input signal to the network. Not optimally tuned PIDs give to the robot early stability during the first part of the simulation and provide a torque command for each module. Thus, a cerebellar network composed of 4 micro complexes computes and provides corrective effort contributions based on the inverse dynamics model of each robotic modules.

In the video below, it is possible to appreciate the locomotion improvements of the robot in an experimental simulation. The recording shows the simulation around second 100-130 when the position error is decreasing and stabilizing. The brain visualizer shown the spiking activity of the input population of the Central Pattern Generator (the higher groups) which are reflected in the blinking of one population of the reservoir (lower group). The spike train window (on the left) shows the periodicity of the spike trains which generate the trajectories for the modules (starting from the bottom, the activities of the input populations and one reservoir population are displayed).

video_4mod_fable_v1

The feed-forward cerebellar effort contribution decreases the mean of the position error of 0.3 radiant and its variance of 0.01 radiant compared to the case when just the effort command from the cpg is provided to the robot (the plot concerning the behavior of the second module is shown below). Moreover, the trend of the error is decreasing along the simulation time and the distance covered by the robot with the cerebellar-like neural network contribution is 9.48 m while the cpg controller contributes to have the robot walk for 1.39 m.

The modular configuration of the Fable Robot makes easier to test the control strategy for different configurations of the robot and patterns of locomotion, having the cerebellar-like neural network compensate the error after a short learning phase, since it has previously learned the internal model of the module.

This work was done in collaboration between DTU, Ghent and SSSA teams.

 

Advertisements

Towards using the NRP with motor actuated robots

The Neurorobotics Platform Development and Research teams are joining their forces to give the users the possibility to interface their own motor actuated robots with the NRP.
This new intriguing feature will help all the roboticists to deploy their neural controllers on real robots, without the need of any external software.
A new working-mode for the NRP has been created by the SSSA team in Pisa, supporting NEST as the neural interface and ROS for the robotic interface.

The iCub humanoid robot doesn’t have a native support for ROS, but instead it uses YARP (Yet Another Robotic Platform) as a middleware.
This makes it necessary for the NRP to support YARP in order to interact with the hardware iCub platform. Such integration is also being currently developed by the SSSA team.
While not being able to send commands to the iCub yet, this extension allows already to read from its sensors, as for instance the eye cameras.

You can see in the picture below one of our team members waving at the iCub seen from the NRP web frontend.

real_icub_nrp

A further mode is being developed by the KIT team in Karlsruhe, to support the SpiNNaker neuromorphic platform and thus allow whole experiments to go completely on hardware, leaving only the communication and synchronization to the NRP closed loop engine.

In the next months, the NRP will support the execution of transfer function at a fixed rate, in real time, thus allowing the implementation of controllers with the well-estabilished fixed rate approach, widely used in the robotic community.

Real Robot Control with the Neurorobotics Platform

Goals

Thanks to its architecture, the NRP should be well suited for  directly controlling a real robotic platform with spiking neural networks. Indeed the closed-loop mechanism of the NRP software but also the use of ROS as a middleware enables developments in this direction.

A first motivation for such a project is to outsource the heavy computation load of simulating spiking neural networks on embedded hardware to a fixed server, which can itself interface with neuromorphic hardware like SpiNNaker if required. Consequently, it helps to reach real-time performance on small and low-energy robotic platforms where neuronal computation would have been impossible otherwise. A second motivation is the possibility to partially train the neural network in the NRP, to avoid mechanical and electrical wear of the physical robot. This, however, requires the transferability of neural control  from simulation to the real robot after the training; this challenging field is more known as transfer learning and requires a minimum level of accuracy in the simulation models and equations.

Methodology

Our work is focused on real-time locomotion of a compliant quadruped robot using CPGs. To outsource the controller to the NRP as discussed above, we have designed both a robot and its 3D clone in simulation. In this setup, they both have four actuators (one for each “body-to-leg” joint) and four sensors (one for each unactuated “knee” joint). The motor position follows a simple open-loop CPG signal with the same amplitude and phase for each leg, such that the robot will alternate between standing-up and sitting-down periodically for fifty seconds. During this experiment, the sensor values are merely recorded,  and not used to regulate the control signal. Given the structure of the kinematic chain with springs and dampers in the knee joints, the system can be explicitly described with a Lagrangian equation. The latter is a function of the physical parameters of the robot which can only be evaluated with a large range of uncertainty as we work with laser-cut or 3D-printed parts assembled with a non-quantified amount of slack. However, the parameters of the simulation model can be set roughly and then optimized to maximize the similarity between the sensors signals output from the hardware and the physics engine. In this setup, we use CMA-ES for this job.

Screenshot from 2018-02-13 19-18-41

Results

After optimization, we can proceed to a qualitative visual validation using different controllers. To this goal, the NRP is installed locally on a machine connected to the same network as the robot. ROS is configured on the robot and in the NRP to enable streaming of actuation and sensing topics between the different machines. A proper calibration of the robot sensors and actuators is also needed to provide meaningful results. A NRP experiment embedding the new optimized model robot is created and used to pilot the robot and the simulated physics. In the current progress, the process seemed to give encouraging results regarding the accuracy and reliability of the simulation, but further tuning is still necessary. An illustration is presented in the following animated figure. The small delay observed between the image on the screen and the robot can be due to the NRP visual rendering in the browser or to the motor PID values.

control_nrp.gif

The aim of pre-training is to exclude actuation patterns that lead to instability of the physical robot (stumbling, falling) and to tune the controller into a good regime. As an interesting fact, our first experiments indicate that there is a good correlation between  failure in the simulator and failures in real observations.

fall_nrp.gif

Structuring your local NRP experiment – some tips

Structuring your local NRP experiment – some tips

Within the context of CDP4, we created a NRP experiment showcasing some functional models from SP1/4:

  • A trained deep network to compute bottom-up saliency
  • A saccade generation model

Since these models are generic, we want to package them so that they can easily be reused in other experiment, such as the WP10.2 strategic experiment. In this post, we quickly explain the structure of the CDP4 experiment on how modularity is achieved.

We decided to implement the functional modules from SP1/SP4 as ROS packages. Therefore, these modules can be used within the NRP (in the GazeboRosPackages folder), but also independently without the NRP, in any other catkin workspace. This has the advantage that the saliency model can be fed webcam images, and easily mounted on a real robot.

The main difference compared to implementing them as transfer function is synchronicity. When the user runs the saliency model on is CPU, processing a single camera image takes around 3 seconds. If the saliency model was implemented as a transfer function, the simulation would pause until the saliency output is ready. This causes the experiment to run slower but conserves reproducability. On the other hand, implemented as a ROS-node, the simulation does not wait for the saliency network to process an image, so the simulation runs faster.

The saliency model is a pre-trained deep network running on TensorFlow. The weights and topology of the network are saved in data files, loaded during the execution. Since these files are heavy and not interesting to version-control, we uploaded them on our owncloud, where they are automatically downloaded by the saliency model if not present. This also makes it simple for our collaborators in SP1/4 to provide us with new pre-trained weights/topology.

The CDP4 experiment itself has its own repo and is very lean, as it relies on these reusable modules. Additionally, an install script is provided to download the required modules in the GazeboRosPackages.

The topic of installing TensorFlow or other python libraries required by the CDP4 experiment, so that they do not collide with other experiment-specific libraries, will be covered in another blog post.

 

Self-Adaptation in Modular Robots at the HBP Summit.

During the last few days at the annual Human Brain Project summit, we had the chance to show to the public some of our experiments.

 

All these experiments are based on the same concept; a biomimetic control architecture based on the modularity of the cerebellar circuit. Everything integrated by means of machine learning and a spiking cerebellum model which allows the system to adapt and manage changes in its dynamics.

Here it is shown one of the two experiments used at the demo of the first day of the summit. In the “Icub ball balancing” experiment (implemented on the NRP), the Icub robot is able to learn in real time and control the system fulfilling the task for up to 4 joints. The scalability of the system allows to change the number of actuated joints showing the modular and robust aspect of the control architecture.

icub_exp

 

In the second experiment we were able to test the same control architecture on the real modular robot Fable by Shape Robotics. This time the spiking cerebellar model was implemented using the neuromorphic platform SpiNNaker.

modular_spinn

CDP4 at the HBP Summit: integrating deep models for visual saliency in the NRP

Back in the beginning of 2017, we had a great NRP Hackathon @FZI in Karlsruhe, where Alexander Kroner (SP4) presented his deep learning model for computing visual saliency.

We now presented this integration at the Human Brain Summit 2017 in Glasgow as a collaboration in CDP4 – visuo-motor integration. During this presentation we also shown how to integrate any deep learning models in the Neurorobotics Platform, as was already presented in the Young Researcher Event by Kenny Sharma.

We will continue this collaboration with SP4 by connecting the saliency model to eye movements and memory modules.

deep-dive-cdp4nrp-saliency

A quadruped robot with traditional computation hardware as a step for a SpiNNaker version

In this post, we describe the components and the architecture of the Tigrillo robot, a compliant quadruped platform controlled with a Raspberry Pi to achieve early research on CPGs and transfer learning. In order to situate the technical description that follows in a scientific context, it may be useful to explain the research methodology that is used:

  1. Optimisation of a parametric CPG controller using the NRP and the VirtualCoach
  2.  Transfer and validation on the Tigrillo quadruped platform
  3. Collection and analysis of sensors feedback ont the robot and in the NRP to design and improve a robust closed-loop system
  4.  Implementation of the CPGs using NEST on the NRP
  5. Transfer and validation on our quadruped robot embedding SpiNNaker hardware
  6. Comparaison between simulations and the real platforms and extraction of knowledge to iterate on step 1.

The Tigrillo robot enables step 2 by providing a robot to validate the accuracy an general behavior in the NRP simulations.

Mechanical details:

The design process of Tigrillo platform have been guided considering three main features for the robot: compliance, cheapness, versatility. The compliance is a key element in this research as it is believed to add efficiency and robustness to locomotion, like what we can see in biology. However, it also challenges classical control techniques as the dynamics of the robot is now governed by equations with a higher complexity level. On the current platform, the compliance is mainly ensure by using springs in the legs knee instead of actuating them.

Electrical and Software architecture:

  • Sensors and Actuators: 4 Dynamixel RX-24F servomotors, an IMU (Inertial Measurement Unit), various force and flexion sensors in the feet and legs
  • Power supply: A DC step-up voltage convertor connected to a 3 cells LiPo battery to supply the boards and motors with a regulated voltage and a stalk current that can rise to 10A when the legs are pushing together and the motors have to deliver a high torque.
  • Control Board: A OpenCM board (based on an Atmel ARM Cortex-M3 microprocessor) that reads the analog sensor values at a constant frequency and send the position or velocity commands to the servomotors using the protocol standard defined by Dynamixel.
  • Computation board: A Raspberry Pi with Ubuntu Mate 16.04 that implements a CPG controller included  in the same Python software stack that the one used in the NRP and thus easily switch from simulation to trials and validation in the  real world.

tigrillo_electrical_schema

The software repository also includes board documentation on the top of the python code used for control and simulation.