Short-term visual prediction – published

Short-term visual prediction is important both in biology and robotics. It allows us to anticipate upcoming states of the environment and therefore plan more efficiently.

In collaboration with Prof. Maass group (IGI, TU Graz, SP9) we proposed a biologically inspired functional model. This model is based on liquid state machines and can learn to predict visual stimuli from address events provided by a Dynamic Vision Sensor (DVS).

Fig_1_rescaled

We validated this model on various experiments both with simulated and real DVS. The results were accepted for publication in [1]. We are now currently working on using those short-term visual predictions to control robots.

[1] “Scaling up liquid state machines to predict over address events from dynamic vision sensors”, Jacques Kaiser, Rainer Stal, Anand Subramoney et al., Special issue in Bioinspiration & Biomimetics, 2017.

Advertisements

Sensory models for the simulated mouse in the NRP

A biologically inspired translation model for proprioceptive sensory information was developed. The translation is achieved implementing a computational model of neural activity of type Ia and type II sensory fibers connected to muscle spindles. The model also includes activity of both static and dynamic gamma-motoneurons, that provide fusimotor activation capable of regulating the sensitivity of the proprioceptive feedback, through the contraction of specific intrafusal fibers (Proske, 19971).

spindle
Figure 1 Intrafusal fibers

The proposed model is an extension of a state-of-the art computational models of muscle spindle activity (Mileusnic, 20062). The model developed by Mileusnic and colleagues, albeit complete and validated against neuroscientific data, was completely rate based, thus it was modified in order to be integrated in a spiking neural network simulation. In particular, a spike integration technique was employed to compute fusimotor activation and the generated rate was used to generate spike trains.

The proprioceptive model is implemented on NEST, in order to provide an easy integration inside the NRP, and on SpiNNaker, for supporting real-time robotic applications. The proposed component can be coupled to both biomechanical models, like musculo-skeletal systems, and common robotic platforms (via suitable conversions from encoder values to simulated muscle length). In particular, this model will be used, as part of CDP1, to provide sensory feedback from the virtual mouse body.

1 Proske, U. (1997). The mammalian muscle spindle. Physiology, 12(1), 37-42.

2 Mileusnic, M. P., Brown, I. E., Lan, N., & Loeb, G. E. (2006). Mathematical models of proprioceptors. I. Control and transduction in the muscle spindle. Journal of neurophysiology, 96(4), 1772-1788.

The virtual M-Platform

Preclinical animal studies can offer a significant contribution to gain knowledge about brain function and neuroplastic mechanisms (i.e. the structural and functional changes of the neurons following inner or external stimuli). For example, an external stimulus as a cortical infarct (i.e. stroke) can produce a cascade of similar neural changes both in a human and animal (i.e. monkeys, rodents etc) brains. And even further stimuli such as input provided during a rehabilitative training can have this impact. The possibility to exploiting the neural plasticity, addressing the treatments in combination with technological advanced methods (e.g. robot-based therapy) is one goal that the HBP is pursuing.

The Neurorobotics Platform is fully part of this picture and is providing an environment that will be an important benchmark for these studies. Two labs from the Scuola Superiore Sant’Anna, in Pisa, are tightly working to develop a virtual model of a experiment carried on in a real neuroscientific environment. The core of this set up is the M-Platform (Spalletti and Lai et al. 2013), a device able to train mice to perform a retraction-pulling task with their forelimb (Figure 1A). During last months, the device has been characterized and upgraded to improve its repeatability (Figure 1B). Meanwhile, a first example of the virtual M-Platform (Figure 1C) has been developed.

nrp_mplatfom
Figure: The real M-Platform (A); the CAD design of the main component of the M-Platform, i.e. actuation and sensing, (B) and its virtual model in the NRP (C)

The  main components of the M-Platform (i.e. linear actuator, linear slide, handle) have been converted in a suitable format for the Gazebo simulator. Properties of the model such as link weights, joint limits and frictions have been adjusted according to the real characteristics of the slide. The actuator was connected to a PID controller whose parameters have been tuned to reproduce the behavior of the real motor.

A simple experiment has thus been designed in the NRP (currently installed on a local machine), for testing the behavior of the obtained model. The experiment includes a 100 neurons brain model, divided in two populations of 90 and 10 neurons respectively. In this closed loop experiment, the first neuron population spikes randomly, and the spike rate of the population is converted to a force value picked out of a predefined range, compatible with the range of forces possibly performable by the mouse through its forelimb.

The computed force values are continuously applied to the handle and can move the slide until the starting position. Once there, the second neural population, wired to suppress the first population spike rate when active, is triggered, so there’s no more force acting on the slide. The motor pushes the slide until the maximum extension position and it then comes back to its starting position, letting the loop start again (see video).

Functional components for control and behavioural models

Gaze stabilization experiment

In this work, we focused on reflexes used by humans for gaze stabilization. A model of gaze stabilization, based on the coordination of the vestibulo-collic reflex (VCR) and vestibulo-ocular reflex (VOR) has been designed and implemented on humanoid robots. The model, inspired on neuroscientific cerebellar theories, is provided with learning and adaptation capabilities based on internal models.

In a first phase, we designed experiments to assess the model’s response to disturbances, validating the model both with the NRP and with a real humanoid robot (SABIAN). In this phase, we mounted the SABIAN head on an oscillating platform (shown below) able to rotate along the pitch axis, in order to produce a disturbance.

The oscillating platform. In (a) the SABIAN head mounted on the platform, with its inertial reference frame is shown. The transmission of motion from the DC motor to the oscillating platform is depicted in (b).

In a second phase, we carried out experiments for testing the gaze stabilization capability of the model, during a locomotion task. We gathered human data of torso displacement while walking and running. The data has been used to animate a virtual iCub while the gaze stabilization model was active.

Balancing experiment

Using the same principles of the gaze stabilization experiment, we carried out a balancing experiment for a simulated iCub. In this experiment, the simulated iCub is holding up a red tray with a green ball on top. The goal of the experiment is to control the robot’s roll and pitch joints for the wrist, in order to keep the ball in the center of the tray. The control model for the wrist joints is provided with learning and adaptation capabilities based on internal models.

Visual segmentation experiment

A cortical model for visual segmentation (Laminart) has been built with the aim of integrating it in the neurorobotics platform. The goal is to see how the model behaves in a realistic visual environment. A second goal is to connect it to another model for the retina.
The model consists of a biologically plausible network containing hundreds of thousands of neurons and several millions connections embedded in about 50 cortical layers. It is built functionnaly in order to link objects that are likely to group together with illusory contours, and to segment disctinct perceptual groups in separate segmentation layers.
Up to now, the Laminart model has been successfully integrated in the NRP and first expriments are being built to check the behaviour of the model and discover what has to be added to it to ensure it can coherently segment objects from each other in a realistic environment. Besides, the Laminart model is almost connected to the retina model.
In the future, the model will be connected to other models for saliency detection, learning, predictive coding, decision making, on the NRP, to create a closed loop experiment. It will also take into account some experimental data about texture segmentation and contour integration.
segmentation

Visual perception experiment

In this work, we evaluated the construction of neural models for visual perception. The validation scenario chosen for the models is an end-to-end controller capable of lane following for an self-driving vehicle. We developed a visual encoder from camera images to spikes inspired by the silicon retina (i.e., the DVS Dynamic Vision Sensor). The veichle controller embeds a wheel decoder based on a virtual agonist antagonist muscle model.

visual-perception-jacques-1

Grasping experiment

During the first 12 month of SGA1, we investigated methods for representing and executing grasping motions with spiking neural networks that can be simulated in the NEST simulator and therefore, the Neurorobotics Platform. For grasping in particular, humans can remember motions and modify them while executing based on the shape and the interaction with objects. We developed a spiking neural network with a biologically inspired architecture to perform different grasping motions, that first learns with plasticity from human demonstration in simulation and then is used to control a humanoid robotic hand. The network is made with two types of associative networks trained independently: One represents single fingers and learns joint synergies as motion primitives; and another represents the hand and coordinates multiple finger networks to execute a specific grasp. Both receive the joint states as proprioception using population encoding, and the finger networks also receives tactile feedback to inhibit the output neurons and stop the motion if a contact with an object is detected.

grasping-camilo-1

grasping-camilo-4

Multimodal sensory representation for invariant object recognition

This functional component integrates multisensory information -namely tactile, visual and auditory- to form an object representation. Although we firstly target invariant object recognition problem using the only visual information, the component is capable of combining other sensory modalities. The model is based on computational phases of the Hierarchical Temporal Memory which is inspired by operating principles of the mammalian neocortex. The model was adapted and modified to extract a multimodal sensory representation of an object. The representation can be interpreted as a cortical representation of perceived inputs. To test the model, we perform object recognition in COIL-20 and COIL-100 datasets in which consist of 20 and 100 different objects (see Figure 1). In details, each object rotated 5 degrees on a turntable and object image was captured by the camera (see Figure2). In addition to image acquisition steps, a number post-processing procedures such as background elimination and size normalization were performed on the images.

multimodal-murat-1

Figure 1 Selected images from different categories.

multimodal-murat-2

Figure 2 A duck object under various rotational transformations.

To obtain object representations, the standard image processing algorithms were performed to binarize and downsize available images in datasets. Then, the model was fed with the processed image data to generate sparsely distributed representation of the perceived images. A sample processed image and cortical representation of the same visual pattern are illustrated in Figure 3 and Figure 4, respectively. Note that, the representation of an object with different sensory inputs can be achieved by same procedure and concatenating the obtained representations for each modality.

Figure 3 A processed visual pattern.                            Figure 4 Cortical representation of a visual pattern

After obtaining representation for all images, we perform recognition operations by grouping the datasets into two categories which are memory representation (or training set) and unseen object patterns (or test set). The representation similarity metric defined as the number of same active cortical columns (the same active bits in the same location) between existing and unseen patterns. The recognition accuracies are shown in Table below. and were derived via splitting training and testing dataset by 10% to 90% and each time incremented by 10.

Training percent

COIL-20

COIL-100

10

90.4

89.0

20

94.3

91.2

30

96.9

94.9

40

97.2

95.6

50

98.3

96.5

60

98.2

97.0

70

98.4

97.3

80

98.6

97.0

90

98.7

96.8

The obtained results indicate that the modal performs well with single modality. Our ongoing studies focus on integrating multiple sensory information (e.g. tactile) to represent multimodal representation to achieve a grasping task.

Morphological Properties of Mass-Spring-Damper Networks for Optimal Locomotion Learning

Morphological Properties of Mass-Spring-Damper Networks for Optimal Locomotion Learning

Robotic Embodiment

The combination of brain inspired AI and robotics is in the core of our work in the Human Brain Project. AI is a vague concept that originated from computer sciences many decades ago and encompasses all algorithms that mimic some cognitive functions of the human species. They are increaslingly based on methods that learn automatically from big datasets.

However, applying those methods to control robots is not as straightforward as it could seem. Unlike computer software, robots generally evolve in noisy and continuously changing environments but on the other hand, their mechanical complexity can be seen as an asset to simplify the control. This is studied through the fields of embodiment and morphological computation. Extreme examples have shown that mechanical structures could provide very natural behavior with no controller at all.

The Passive Walker experiment from T. McGeer is a powerful demonstration emphazing the importance of the body design versus the controller complexity to obtain robust and natural locomotion gaits.

Towards a Formalization of the Concept

Some recent investigations have tried to formalize the relation between the dynamical complexity of a mechanical system and its capability to require simple control. To this goal, a simple yet efficient tool consists in simulating structures composed of masses connected with actuated damper-spring links.

To extend this research, we developed a basic simulator of mass-spring-damper (MSD) networks and optimized a naive locomotion controller to provide them with efficient gaits in term of traveled distance and dissipated power. Three experiments have been done in open-loop to determinate the influence of the size of a structure (estimated though the number of nodes), the compliance (inverse of the spring stiffness) and the saturation at high powers.

This video presents several simulation renditions. The different locomotion processes displayed are learned through optimization in open-loop control.

In the second part of this work, the capacity of realizing closed-loop control in a very simple way requiring very few numerical computations has then been demonstrated.

clThe principal components in the closed-loop learning pipeline consist in a readout layer which is trained at each time step and a signal mixer that gradually integrates the feedback in the actuation signal.

Our Contribution

A full discussion about the results is accessible directly in this article under Creative Common license.

This work has been realized at Ghent Uuniversity together with Jonas Degrave, Francis wyffels, Joni Dambre and Benonie Carette. It is mainly mainly academic and provides a methodology to optimize a controller for locomotion and indications on what we can expect from its complexity to be able to realize this experiment. In the future, this knowledge will be used to conduct similar experiments on quadruped robots both in the real world and in simulation using the Neuro-Robotic Platform (NRP) developed in HBP.

Integrating Nengo into the NRP?

On 11th March we had the honor of welcoming Terrence Stewart from the University of Waterloo (http://compneuro.uwaterloo.ca/people/terrence-c-stewart.html) at the Technical University of Munich. During these two days, he first gave a fascinating presentation on Nengo and neural engineering in general.
This was followed by extensive discussions with our developers to investigate a possible integration of Nengo into our platform after it had been installed on his laptop. To this extent, we discussed what overlaps already exist and identified missing parts to make this integration happen.
This yields the opportunity for our NRP to offer additional spiking neuron simulators aside from NEST.
This collaboration would be benefitial for both sides, with us offereing a platform to interface Nengo with Roboy or other muscle based simulations.

20170411_14453920170411_164538

SP9 Quarterly in-person meeting

We are closely collaborating with SP9 (Neuromorphic hardware) to support big networks in real time. On the 20th and 21st of March 2017, we participated in the SP9 Quaterly in-person meeting to present the Neurorobotics Platform and our integration of SpiNNaker.

SP9During the meeting, we identified MUSIC as a a single interface between our platform and both supercomputers from SP7 as well as SpiNNaker. We also pointed out the features we were missing in MUSIC to keep the Neurorobotics platform interactive, most importantly dynamical ports and reset.

We also presented some complex learning rules we are working on to help SP9 identify user requirements for SpiNNaker 2 design. We were surprised to learn that one of the most complicated learning rule we are working on – SPORE derived by David Kappel in Prof. Maass group – is also used as a benchmark for SpiNNaker 2 by Prof. Mayr. This reward-based learning rule can be used to train arbitrary recurrent network of spiking neurons. Confident that it will play an important role in SGA2, we sent our master student Michael Hoff from FZI, Karlsruhe to TU Graz to use this rule in a robotic setup.