Moment-based local binary patterns: A novel descriptor for invariant pattern recognition applications
Introduction
A crucial part of a modern intelligent imaging system, which learns from its environment and interacts with it, is the pattern recognition procedure. In general, a pattern recognition process employs four stages: (1) image acquisition, (2) image pre-processing (denoising, filtering, etc.), (3) feature extraction and finally (4) classification. The third step is probably the most complicated and it affects the overall performance of the system. A feature extraction method (FEM) can be termed successful if the resulted features (descriptors) describe uniquely the processed object in a scene. The more successful a FEM is, the more efficient the classification is.
The discrimination power of a descriptor is measured by its ability to capture the particular characteristics of the described pattern, which distinguish it among similar or totally different objects. A difficult pattern recognition task consists of objects being quite similar and differing slightly. In these cases the descriptors need to have strong local nature in order to encode the information that distinguishes them. A considerable performance evaluation of some well-known local descriptors has been performed in [1], with very useful and constructive conclusions.
Among the most widely used local descriptors, the Local Binary Patterns (LBP) operator proposed by Ojala et al. [2], has attracted the attention of the scientists and motivated them to extent its applicability to many disciplines. The LBP operator is initially introduced as texture descriptor but it has been applied, after some modifications, to face recognition [3], [4], facial expression [5], pedestrian detection [6], etc. Although there is a specific LBP version being rotation invariant [7], [8], its application in traditional pattern recognition tasks where rotated, translated and scaled objects have to be recognized, is not suggested.
Another popular FEM that is used to generate discriminative feature sets is the computation of image moments. Moments have been used successfully in many classification applications [9], [10], [11], [12] and their ability to describe an object fully makes them a powerful tool in computer vision applications, like object recognition in robotic applications and object characterization in visual inspection based quality control systems. However, since image moments are region descriptors they provide global information of an object. Although moments of higher orders capture the object’s details, their local behaviour is quite poor.
In this work a novel FEM that constructs rotation, translation, scale invariant descriptors of local nature, is introduced. The proposed feature extraction methodology aims to utilize the local behaviour of the LBP and the invariance nature of the orthogonal image moments. The derived descriptors called moment based local binary patterns (Mb-LBP) seem to be effective pattern features, which improve the classification ability of the traditional moment invariants and extend the performance of the original LBP in more general pattern recognition problems.
In order to investigate the behaviour of the proposed descriptors, an exhaustive experimental plan has been arranged, consisting of several pattern recognition tasks (face recognition, facial expression recognition, texture recognition, object recognition) by using several benchmark datasets from the literature.
To summarize, the contribution of this work lies on the construction of a novel LBP-like descriptor which is making use of the invariant properties the moments and moment invariants have. This is achieved by using a novel RST (rotation, scale, translation) invariant image representation called image momentgram, which is constructed by applying a straightforward procedure. This new image representation depends on the type of the used moment family and therefore can take several forms, by incorporating different image information. This flexibility helps on finding the most appropriate moment family that better describes the problem at hand. The utility of the momentgram is to combine the main advantages of the moments and moment invariants with those of the LBP method, aiming to construct an invariant local descriptor. Based on this RST image representation the LBP operator can be applied on, in order to extract invariant feature vectors for pattern recognition applications.
The Mb-LBP descriptor shows a global behaviour that comes from the nature of the moment functions to describe the image’s content in several components (“bands” - moment orders). On the other hand, the local behaviour of the Mb-LBP descriptor is based on the local information of the constructed momentgram and not that of the original image. This is the reason why the Mb-LBP descriptor is stable under rotation/scaling/translation, since these transformations are filtered by the construction of the momentgram, while the distinctive patterns’ information is captured through the LBP histograms of the momentgrams.
The paper is organized as follows: Section 2 describes the most popular moment families and the corresponding moment invariants. The basic theory of LBP operator is presented in Section 3, while the proposed local descriptor along with its definitional principles is discussed in Section 4. An extensive experimental study regarding the classification performance of the proposed descriptor, in comparison with the moments, moment invariants feature vectors, the conventional LBP and other popular descriptors, takes place in Section 5. Finally, the main conclusions are summarized and discussed in Section 6.
Section snippets
Moments and moment invariants
Image moments have attracted the attention of the engineering scientific community for several decades, as a powerful tool to describe the content of an image. They have been used in many research fields of the engineering life, such as pattern recognition [9], [10], computer vision [11], [12] and image processing [13], [14] with significant results. The first introduction of image moments for classification purpose was performed by Hu [9], by introducing the concept of moment invariants for
Local binary patterns (LBP)
Local binary patterns (LBP) operator has been introduced by Ojala et al. [2], for texture analysis purposes. LBP is a powerful illumination invariant local descriptor, which a binary code that describes the local texture pattern is constructed by thresholding a neighbourhood of a centred pixel. If the gray value of each neighbourhood’s pixel is greater than the value of the centred one, this pixel takes the value 1 in a thresholding fashion; otherwise it takes the 0 value. In this way a
Moment-based local binary patterns (Mb-LBP)
The application of the LBP operators, defined in the previous sections, on the original image pattern, has a significant limitation regarding the invariance properties of the resulted descriptors. While translation and rotation invariances are achieved through the construction procedure of the local binary patterns, less attention has been paid to the scaling invariance. This deficiency of the previously defined LBP operators makes them inappropriate to more general pattern recognition
Experimental study
In order to investigate the classification performance and behaviour of the proposed descriptor, a set of appropriate experiments has been arranged. For the experimental purposes, specific software has been developed in C++ language that implements the momentgrams construction. Moreover, the MATLAB implementations for the LBP [30] and the built-in k-NN classifier [31] are used, while all experiments are executed in a Pentium 3.3 GHz PC with 2 GB RAM.
Conclusion
A new methodology for the generation of high discriminative feature vectors was proposed in the previous sections. The introduced descriptor consists of two main processing modules, the momentgram construction and the application of the LBP operator on it. This two-step technique is making use of the main advantages of the image expansion on its moments or moment invariants and the LBP local descriptor. As a result an enhanced LBP histogram, which is invariant under common geometric
Dr. George A. Papakostas received the diploma of Electrical and Computer in Engineering in 1999 and the M.Sc. and Ph.D. Degree in Electrical and Computer Engineering (topic in Feature Extraction and Pattern Recognition) in 2002 and 2007, respectively, from Democritus University of Thrace (DUTH), Greece. He is the author of 50 publications in international scientific journals and conferences. His research interests are focused in the field of computational intelligence, pattern recognition,
References (43)
- et al.
A comparative study of texture measures with classification based on feature distributions
Pattern Recognit.
(1996) - et al.
A novel distribution-based feature for rapid object detection
Neurocomputing
(2011) - et al.
Affective interaction based on person-independent facial expression space
Neurocomputing
(2008) - et al.
Pyramid binary pattern features for real-time pedestrian detection from infrared videos
Neurocomputing
(2011) - et al.
Rotation-invariant texture classification using feature distributions
Pattern Recognit.
(2000) - et al.
Binary SIPPER plankton image classification using random subspace
Neurocomputing
(2010) - et al.
Accurate and speedy computation of image legendre moments for computer vision applications
Image Vision Comput.
(2010) - et al.
A new class of Zernike moments for computer vision applications
Inf. Sci.
(2007) - et al.
Trademark image retrieval using synthetic features for describing global shape and interior structure
Pattern Recognit.
(2009) - et al.
A comprehensive review of current local features for computer vision
Neurocomputing
(2008)
Projective invariants of co-moments of 2D images
Pattern Recognit.
Computation strategies of orthogonal image moments: a comparative study
Appl. Math. Comput.
Translation and scale invariants of Tchebichef moments
Pattern Recognit.
On the computational aspects of Zernike moments
Image Vision Comput.
Efficient and accurate computation of geometric moments on gray-scale images
Pattern Recognit.
A unified methodology for efficient computation of discrete orthogonal image moments
Inf. Sci.
Novel moment invariants for improved classification performance in computer vision applications
Pattern Recognit.
An introduction to ROC analysis
Pattern Recognit. Lett.
A performance evaluation of local descriptors
IEEE Trans. Pattern Anal. Mach. Intell.
Face description with local binary patterns: application to face recognition
IEEE Trans. Pattern Anal. Mach. Intell.
Multiresolution gray-scale and rotation invariant texture classification with local binary patterns
IEEE Trans. Pattern Anal. Mach. Intell.
Cited by (53)
Local Neighborhood Intensity Pattern–A new texture feature descriptor for image retrieval
2018, Expert Systems with ApplicationsCitation Excerpt :Again, a combined application of LBP along with the above mentioned Gabor filter gave better result (Tlig, Sayadi, & Fnaiech, 2012). Besides these, the concept of the moment was applied for feature extraction in (Papakostas, Koulouriotis, Karakasis, & Tourassis, 2013). Again, in Directional Local Extrema Pattern (Murala, Maheshwari, & Balasubramanian, 2012a), the feature was extracted by the edge information.
A new hybrid stereovision-based distance-estimation approach for mobile robot platforms
2018, Computers and Electrical EngineeringAn efficient and robust approach for biomedical image retrieval using Zernike moments
2018, Biomedical Signal Processing and ControlA content-based approach to medical image retrieval
2022, Research Anthology on Improving Medical Imaging Techniques for Analysis and InterventionDifferential Invariant Signatures for Planar Lie Group Transformations with Application to Images
2022, Journal of Lie TheoryMoment Transform-Based Compressive Sensing in Image Processing
2022, Communications in Computer and Information Science
Dr. George A. Papakostas received the diploma of Electrical and Computer in Engineering in 1999 and the M.Sc. and Ph.D. Degree in Electrical and Computer Engineering (topic in Feature Extraction and Pattern Recognition) in 2002 and 2007, respectively, from Democritus University of Thrace (DUTH), Greece. He is the author of 50 publications in international scientific journals and conferences. His research interests are focused in the field of computational intelligence, pattern recognition, computer vision, neural networks, feature extraction, optimization, signal and image processing.
Dr. Dimitrios E. Koulouriotis, Associate Professor. Dr. Koulouriotis received his Ph.D. in Intelligent Systems from Department of Production and Management Engineering, Technical University of Crete, Greece, in 2001. He is currently in the Department of Production and Management Engineering, Democritus University of Thrace, as an Associate Professor. His research interests include computational intelligence, machine learning and pattern recognition and their applications in vision, image and signal processing.
Karakassis Evaggelos, received the diploma in Production Management Engineering. He is currently pursuing the Ph.D. degree in the Department of Production Management Engineering, Democritius University of Thrace, Greece.His research interest includes robotics, image processing, computational intelligence.
Dr. Vasileios D. Tourassis, Professor. Dr. Tourassis received his undergraduate degree from the Aristotle University of Thessaloniki, Greece in 1980 and his M.S. and Ph.D. degrees from Carnegie-Mellon University, USA in 1982 and 1985, respectively, all in Electrical Engineering. Before joining DUTH, Dr. Tourassis was a tenured professor at the University of Rochester, USA. His current research interests include robotics and signal processing. Dr. Tourassis is a Senior Member of the Institute of Electrical and Electronic Engineers (IEEE), a Senior Member of the Society of Manufacturing Engineers (SME) and Editorial Board member of the Journal of Intelligent and Robotic Systems.