Validation and Verification (V&V) techniques have been widely used to make sure that simulation results accurately predict reality. As the NRP is a simulation platform to simulate a neural network embodied by a robot, the primary problem of validating that simulation results can be transferred into reality is an intrinsic problem. In fact, validating that a given neural network connected to a robot in a specific way produces the desired results is the very purpose of the NRP.
However, as of today, these validation tasks are performed entirely dynamic, i.e. through actually simulating the experiments. In this series of blog posts, we investigate how this dynamic validation can be supported by static validation and verification activities.
For this, we see the following advantages:
- The neuroscientist gets an early feedback on their experiment. Because a static validation or verification is independent of a concrete simulation, the analsis can be performed before the code is actually simulated. This aid the design of neurorobotics experiments escpecially without a running simulation. For the NRP where the editors are currently only available within a running simulation, this means we could validate for example Transfer Functions before they are actually applied to the simulation.
- The simulation platform uses resources more efficiently as no simulation resources are aquired for experiments that cannot be run. As of today, this advantage is not significant, as users may only change an experiment within a simulation unless they are willing to edit the plain XML models, but in the future, this is an important goal.
Static V&V argue on all possible execution paths of an experiment. However, the ability of a neural network to learn and adapt to new situations, but also the complexity of the interactions of a robot with its environment make it infeasible to argue based on single execution paths. Therefore, static V&V techniques are mostly restricted to arguments on all execution paths, in particular errorneous parts.
Therefore, the aim of static V&V in the context of neurorobotics must be to find neurobotics experiments that are errorneous for all errorneous executions, i.e. experiments that include flaws such we know that the experiment is not going to work, regardless of the exact behavior of the neural network.
For a successful validation and verification, we need to look at the three main artifacts in a neurorobotics simulation:
- The neural network
- The robot
- The Transfer Functions that connect the latter.
These parts will be looked at in more detail in future blog post on this subject.