Object Anchoring for Autonomous Robots Using the Spatio-Temporal-Semantic Environment Representation SEEREP

Abstract

For single-plant specific weed regulation, robotic systems and agricultural machinery in general have to collect a large amount of temporal and spatial high-resolution sensor data. SEEREP, the Spatio-Temporal-Semantic Environment Representation, can be used to structure and manage such data more efficiently. SEEREP deals with the spatial, temporal and semantic modalities of data simultaneously and provides an efficient query interface for all three modalities that can be combined for high-level analyses. It supports popular robotic sensor data such as images and point clouds, as well as sensor and robot coordinate frames changing over time. This query interface enables high-level reasoning systems as well as other data analysis methods to handle partially unstructured environments that change over time, as for example agricultural environments. But the current methodology of SEEREP cannot store the result of the analysis methods regarding specific objects instances in the world. Especially the results of the anchoring problem which searches for a connection between symbolic and sub-symbolic data cannot be represented nor queried. Thus, we propose a further development of the SEEREP methodology in this paper: For a given object, we link the existing semantic labels in different datasets to a unique common instance, thereby enabling queries for datasets showing this object instance and with this enabling the efficient provision of datasets for object-centric analysis algorithms. Additionally, the results of those algorithms can be stored linked to the instance either by adding facts in a triple-store like manner or by adding further data linked to the instance, like a point, representing the position of the instance. We show the benefits of our anchoring approach in an agricultural setting with the use-case of single-plant specific weed regulation.

The DFKI Niedersachsen (DFKI NI) is sponsored by the Ministry of Science and Culture of Lower Saxony and the VolkswagenStiftung. This work is supported by the Federal Ministry for the Environment, Nature Conservation, Nuclear Safety and Consumer Protection (BMUV) within the CognitiveWeeding project (grant number: 67KI21001B).