Semantic Scene Filtering for Event Cameras in Long-Term Outdoor Monitoring Scenarios

Abstract

Event cameras are biologically inspired devices. They are fundamentally different from conventional frame-based sensors in that they directly transmit an (x, y, t) output stream of asynchronously and independently detected changes in brightness. For the development of monitoring systems, scenario-based long-term experiments are much more representative than day-to-day experiments. However, unconstrained “real-world” factors pose processing challenges.

To perform a semantic scene filtering on the output stream of an event camera in such an outdoor monitoring scenario, this paper describes a multi-stage processing chain. The goal is to identify and store only those segments that contain events that were triggered by a specific set of objects of interest. The main idea of the proposed processing pipeline is to pre-process the data stream using different filters to identify Patches-Of-Interest (PoIs). These PoIs, natively represented as space-time event clouds, are further processed by PointNet++, a 3D-based semantic segmentation network. An evaluation was performed on about 89 h of real-world outdoor sensor data, achieving a semantic filtering with a false negative rate of ${\approx }3.8\%$ and a true positive rate of ${\approx }96.2\%$.

This work was supported by the European Regional Development Fund under grant number EFRE-0801082 as part of the project “plsm” (https://plsm-project.com/).