DocNER: A Deep Learning System for Named Entity Recognition in Handwritten Document Images

Abstract

The extraction of relevant information from handwritten document images is still a challenging task. In this paper, we introduce a framework for named entities recognition from a collection of real handwritten marriage records. For this purpose, we perform an extensive evaluation of two different information extraction approaches to locate and recognize named entities from unstructured handwritten documents. The first one is based on an end-to-end neural network model that jointly performs transcription and semantic annotation of handwritten document images. While the second relies on two stages, the first one focuses on Handwritten Text Recognition (HTR) to transcribe documents into electronic texts, while the second seeks to identify semantic named entities using a state-of-the-art Natural Language Processing (NLP) model. This work is evaluated on a collection of real French handwritten marriage records, discussing the advantages and drawbacks of the explored approaches. The achieved results show the efficiency of the proposed framework even though it does not use any post-processing stage. Additional experiments are conducted using the Esposalles database to compare our methods to the participated systems in the ICDAR 2017 Information Extraction competition.