Personal Semantic Indexation of Images Using Textual Annotations

Abstract

This paper presents an automatic indexation module integrated in online photos management software. Semantic descriptors are generated from textual annotations associated to personal photos. Users naturally annotate their pictures with natural language comments in order to personally describe the main elements of the pictures. Form those personal descriptions, we extract semantic descriptors which are used to organize users’ pictures. Our main goal is to retrieve people and places directly or indirectly cited in textual annotations. The descriptors extraction stage is based on a deep linguistic analysis of the textual annotations, which offers a first disambiguation of the possible interpretations and allows for complex descriptors identification (i.e. paraphrases). Paraphrases are then resolved using semantic knowledge sources: a geographical thesaurus and a personal knowledge base of the users’ relationships with people. The goal of our system is to automatically integrate new pictures in the user’s context accordingly to extracted descriptors. The context that we consider is mainly composed of the current user’s taxonomy of descriptors. Thus, our system builds or completes automatically a taxonomy of descriptors which is personalized and relevant for one user.