Event Based Velocity Prediction Snn

Neuromorphic computing uses very-large-scale integration (VLSI) systems with the goal of replicating neurobiological structures and signal conductance mechanisms. Neuromorphic processors can run spiking neural networks (SNNs) that mimic how biological neurons function, particularly by emulating the emission of electrical spikes. A key benefit of using SNNs and neuromorphic technology is the ability to optimize the size, weight, and power consumed in a system. SNNs can be trained and employed in various robotic and computer vision applications; we attempt to use event-based to create a novel approach in order to the predict velocity of objects moving in frame. Data generated in this work is recorded and simulated as event camera data using ESIM. Vicon motion tracking data provides the ground truth position and time values, from which the velocity is calculated. The SNNs developed in this work regress the velocity vector, consisting of the x, y, and z-components, while using the event data, or the list of events associated with each velocity measurement, as the input features. With the use of the novel dataset created, three SNN models were trained and then the model that minimized the loss function the most was further validated by omitting a subset of data used in the original training. The average loss, in terms of RMSE, on the test set after using the trained model on the omitted subset of data was 0.000386. Through this work, it is shown that it is possible to train an SNN on event data in order to predict the velocity of an object in view. (Spring 2022 MS Computer Science Thesis - North Carolina State University)