Unsupervised semantic clustering and localization for mobile robotics tasks

Abstract

Due to its vast applicability, the semantic interpretation of regions or entities increasingly attracts the attention of scholars within the robotics community. The paper at hand introduces a novel unsupervised technique to semantically identify the position of an autonomous agent in unknown environments. When the robot explores a certain path for the first time, community detection is achieved through graph-based segmentation. This allows the agent to semantically define its surroundings in future traverses even if the environment’s lighting conditions are changed. The proposed semantic clustering technique exploits the Louvain community detection algorithm, which constitutes a novel and efficient method for identifying groups of measurements with consistent similarity. The produced communities are combined with metric information, as provided by the robot’s odometry through a hierarchical agglomerative clustering method. The suggested algorithm is evaluated in indoors and outdoors datasets creating topological maps capable of assisting semantic localization. We demonstrate that the system categorizes the places correctly when the robot revisits an environment despite the possible lighting variation.

Introduction

Contemporary research in robotics endows modern autonomous systems with the ability to semantically recognize and segment regions/entities. This affords them with increased flexibility and the ability to interpret and interact with the environment on a higher level. Apart from places, robots are capable of recognizing objects and classify them in semantic areas. Overall, semantic interpretation is considered an active and fundamental research field, which relies mostly on computer vision methods for place recognition [1].

Localization (operated either on metric or topological maps) stands for a critical ability incorporated into nowadays robots, yet semantics, which can constitute an essential foundation for this capacity, is still underdeveloped. Moreover, the successful communication between humans and robots can be established only through the ability of the second to sense and classify their own surroundings by precisely recalling spatial memories [2]. Metric maps are mainly used for small-scale spaces [3] and they are organized in a geometric manner, while the relative conceptual remains hidden. The reveal of this hidden information can be achieved by reorganizing it on the basis of a topological map [4]. This is directly related with topology, the branch of mathematics studying characteristics invariant in continuous deformations [5], [6]. A scene’s description based on topological maps retains information regarding the semantic region it belongs. These graphs are a fundamental element of a semantic map, enabling abstraction to metric maps [3].

In order to construct a topological map, the similarity between the acquired camera measurements needs to be computed. Zhang et al. [7] identified the mechanisms for quantifying such similarities, as to the degree of abstracted information from the respective images, into the following categories. Image-based approaches rely on pixel differences between consecutive images to determine changes in a scene. Techniques based on local-features detect and try to associate various key-points, so as to measure the similarity between different frames [8], [9]. Furthermore, in histogram-based approaches, images are compared by means of features’ statistics [3]. Lastly, a typical approach for simplifying the information regarding a place is to address the problem with a Bag-of-Words (BoW) representation. The method of BoW describes the input measurements as a quantized set of local features, thus reducing the searching space to gain efficiency [10]. However, a scene contains randomly scattered features, thus a meaningful way to describe it is via histograms of visual words (visual word vectors) [11].

This paper addresses the problem of semantically mapping an environment. With the term semantic, we refer to the signs and the things to which they refer [1]. Thus, within the scope of this work, the term semantic mapping is used to indicate the identification and the recoding of visual signs and symbols that contain meaningful information. Our goal is to produce an unsupervised system to segment the robot’s trajectory into different semantic regions. Then, each time the robot passes through the same area, it gets self-localized within one of the computed semantic divisions by exclusively using visual information. This essentially means that a specific label for each trajectory segment is not necessarily required to achieve semantic localization. On the contrary, our method is able to identify distinct regions that preserve semantic consistency, however, a meaningful name can be assigned to each one of them with minimum effort after the map has being completed. We make use of the BoW image representation as a means of uniformly describing the captured images and shaping semantic clusters based on their similarities. The semantic interpretation of the robot’s path is achieved by applying the Louvain Community Detection Algorithm (LCDA) [12]. Semantically important properties of a particular environment are identified with great tolerance over various lighting conditions. Extending our previous work [13], we improved the development of the places’ semantic representation by introducing an additional hierarchical agglomerative clustering method to incorporate the information of a metric map. Using only LCDA, many snapshots of the same semantic region are redundantly grouped into different clusters due to dissimilarities occurring by observing the same scene from arbitrary orientations. This is a common problem for any appearance-based method that utilizes monocular and unidirectional cameras since the view of the environment changes significantly when different content is observed [8], [14], [15]. By means of geometrical information, such over-segmented areas can be consolidated, thus improving the quality of the map. A its final stage the map contains both semantic and metric information (topological map).

The rest of the paper is structured as follows. In Section 2, we discuss representative related work in the field of semantic mapping construction. Section 3 contains a detailed explanation of our approach. In Section 4, we present the results of our experiments, while in the last section, we draw conclusions and present suggestions for future work.