Moment-based local binary patterns: A novel descriptor for invariant pattern recognition applications

Abstract

A novel descriptor able to improve the classification capabilities of a typical pattern recognition system is proposed in this paper. The introduced descriptor is derived by incorporating two efficient region descriptors, namely image moments and local binary patterns (LBP), commonly used in pattern recognition applications, in the last decades. The main idea behind this novel feature extraction methodology is the need of improved recognition capabilities, a goal achieved by the combinative use of these descriptors. This collaboration aims to make use of the major advantages each one presents, by simultaneously complementing each other, in order to elevate their weak points. In this way, the useful properties of the moments and moment invariants regarding their robustness to the noise presence, their global information coding mechanism and their invariant behaviour under scaling, translation and rotation conditions, along with the local nature of the LBP, are combined in a single concrete methodology. As a result a novel descriptor invariant to common geometric transformations of the described object, capable to encode its local characteristics, is formed and its classification capabilities are investigated through massive experimental scenarios. The experiments have shown the superiority of the introduced descriptor over the moment invariants, the LBP operator and other well-known from the literature descriptors such as HOG, HOG-LBP and LBP-HF.

Introduction

A crucial part of a modern intelligent imaging system, which learns from its environment and interacts with it, is the pattern recognition procedure. In general, a pattern recognition process employs four stages: (1) image acquisition, (2) image pre-processing (denoising, filtering, etc.), (3) feature extraction and finally (4) classification. The third step is probably the most complicated and it affects the overall performance of the system. A feature extraction method (FEM) can be termed successful if the resulted features (descriptors) describe uniquely the processed object in a scene. The more successful a FEM is, the more efficient the classification is.

The discrimination power of a descriptor is measured by its ability to capture the particular characteristics of the described pattern, which distinguish it among similar or totally different objects. A difficult pattern recognition task consists of objects being quite similar and differing slightly. In these cases the descriptors need to have strong local nature in order to encode the information that distinguishes them. A considerable performance evaluation of some well-known local descriptors has been performed in [1], with very useful and constructive conclusions.

Among the most widely used local descriptors, the Local Binary Patterns (LBP) operator proposed by Ojala et al. [2], has attracted the attention of the scientists and motivated them to extent its applicability to many disciplines. The LBP operator is initially introduced as texture descriptor but it has been applied, after some modifications, to face recognition [3], [4], facial expression [5], pedestrian detection [6], etc. Although there is a specific LBP version being rotation invariant [7], [8], its application in traditional pattern recognition tasks where rotated, translated and scaled objects have to be recognized, is not suggested.

Another popular FEM that is used to generate discriminative feature sets is the computation of image moments. Moments have been used successfully in many classification applications [9], [10], [11], [12] and their ability to describe an object fully makes them a powerful tool in computer vision applications, like object recognition in robotic applications and object characterization in visual inspection based quality control systems. However, since image moments are region descriptors they provide global information of an object. Although moments of higher orders capture the object’s details, their local behaviour is quite poor.

In this work a novel FEM that constructs rotation, translation, scale invariant descriptors of local nature, is introduced. The proposed feature extraction methodology aims to utilize the local behaviour of the LBP and the invariance nature of the orthogonal image moments. The derived descriptors called moment based local binary patterns (Mb-LBP) seem to be effective pattern features, which improve the classification ability of the traditional moment invariants and extend the performance of the original LBP in more general pattern recognition problems.

In order to investigate the behaviour of the proposed descriptors, an exhaustive experimental plan has been arranged, consisting of several pattern recognition tasks (face recognition, facial expression recognition, texture recognition, object recognition) by using several benchmark datasets from the literature.

To summarize, the contribution of this work lies on the construction of a novel LBP-like descriptor which is making use of the invariant properties the moments and moment invariants have. This is achieved by using a novel RST (rotation, scale, translation) invariant image representation called image momentgram, which is constructed by applying a straightforward procedure. This new image representation depends on the type of the used moment family and therefore can take several forms, by incorporating different image information. This flexibility helps on finding the most appropriate moment family that better describes the problem at hand. The utility of the momentgram is to combine the main advantages of the moments and moment invariants with those of the LBP method, aiming to construct an invariant local descriptor. Based on this RST image representation the LBP operator can be applied on, in order to extract invariant feature vectors for pattern recognition applications.

The Mb-LBP descriptor shows a global behaviour that comes from the nature of the moment functions to describe the image’s content in several components (“bands” - moment orders). On the other hand, the local behaviour of the Mb-LBP descriptor is based on the local information of the constructed momentgram and not that of the original image. This is the reason why the Mb-LBP descriptor is stable under rotation/scaling/translation, since these transformations are filtered by the construction of the momentgram, while the distinctive patterns’ information is captured through the LBP histograms of the momentgrams.

The paper is organized as follows: Section 2 describes the most popular moment families and the corresponding moment invariants. The basic theory of LBP operator is presented in Section 3, while the proposed local descriptor along with its definitional principles is discussed in Section 4. An extensive experimental study regarding the classification performance of the proposed descriptor, in comparison with the moments, moment invariants feature vectors, the conventional LBP and other popular descriptors, takes place in Section 5. Finally, the main conclusions are summarized and discussed in Section 6.