Evolutionary multi-objective visual cortex for object classification in natural images

Highlights

• We proposed a new methodology for image description.

• We present a multi-objective approach for brain programming.

• We match the state-of-the-art in classifying GRAZ-01 and outperform it in GRAZ-02.

Abstract

In recent years computer vision systems have used the human visual system as inspiration for solving different tasks such as object detection and classification. Computational models as the artificial visual cortex (AVC) have shown promising results in solving such problems. Thus, this paper proposes a new methodology for creating an image descriptor vector for classification, and at the same time, finding the objects’ location within the image. Also, this work implements the brain programming paradigm from a multi-objective perspective in order to improve the performance in the object classification task. This methodology is implemented for training the proposed model in order to classify the images from the GRAZ-01 and GRAZ-02 databases. The solutions found in this research match, and in some cases outperform, other techniques of the state-of-the-art for classifying the aforementioned databases.

Introduction

Numerous natural systems (brains, immune systems and societies) and artificial systems (parallel and distributed computing, artificial neural networks and evolutionary programs) are generally characterized by behaviors that emerge from non-trivial interactions between a large number of components often based on hierarchical structures [1]. The complexity of understanding and designing such systems while approaching difficult tasks resides in finding the best interactions. In this way, some research communities have focus their efforts in analyzing and creating such systems.

Holland describes a complex adaptive system as the integration of several interdependent entities, that collaborate to solve a given task, and are able to adapt to environmental changes or variations among the parts [2]. The elements that compose such systems are often called agents [3]. A well known example of an adaptive system for classification is the simple pattern recognition device presented in Holland's seminal work [4]. In such example, the complexity resides in the high amount of possible configurations of an array of binary sensors of size a × b, that is 2^ab, and the system's necessity in finding the right configuration for recognizing a given pattern. Today there exists examples of this kind of systems that focus on solving computer vision problems.

Sight is one of the most important senses for human beings, since it contributes approximately 70% of the information received by the brain. This information helps in the decision-making process performed during the interactions with the environment. Several scientific communities have focus their research in understanding the organization of the brain with the goal of emulating it. There are several computational models [5], [6], [7], [8], [9], [10], [11], [12], [13], [14] inspired in the hierarchical structure of the human visual system, its neurophysiological characteristics and neuropsychological theories such as: the feature integration theory [15], the biased competition theory [16], the recognition-by-components paradigm [17], the simple and complex cells model [18] and the two path cortical model [19]. These systems approach different visual tasks as object recognition, detection and classification.

Nowadays there are several evolutionary systems that focus in solving the aforementioned visual tasks. For example, Olague and Trujillo describe a Multi-objective Genetic Programming (MOGP) system for synthesizing interest point detectors for view-based object recognition [20]. Shao et al. proposed a feature learning system for classifying the objects in the Caltech-101 database. They evolve a set of two-dimensional operators using MOGP for creating what they call a “near-optimal” image descriptor [21]. Similarly, Al-Sahaf et al. present two GP based methods, the One-shot GP and Compound-GP systems, that aim to evolve a program for the task of binary classification of texture images [22].

A computational model of particular interest for this work is known as the artificial visual cortex (AVC) [23], which in turn is derived from the HMAX model [9], since it approaches the object recognition problem based on the human visual cortex. This model proposed by Olague et al. shows great performance at solving the absence/presence problem for object recognition. The AVC is based primarily on two models: a psychological model called feature integration theory and a neurophysiological model called the two pathway cortical model.

The first theory states that the visual attention task in human beings is performed in two stages. The first one is called the pre-attentive stage, where visual information is processed in parallel over different feature dimensions that compose the scene: shape, color, orientation, spacial frequency, brightness and motion direction. The second stage, called focal attention, integrates the extracted features from the previous stage in order to highlight a region of the scene. Thus, visual attention is the capability of a creature, living or artificial, to focus an object of interest on a visual environment [24]. Visual attention can be formally defined as “the process that establishes a relationship between the different properties in the scene, perceived through the visual system, with the objective of finding the best aspect for solving the task at hand” [25].

The second theory is the two pathway cortical paradigm. This neurophysiological model states that there are two information routes within the visual cortex, the dorsal and ventral streams. Both subsystems receive the same visual information as input, nevertheless they differ in the information transformations performed at each of them [27]. The dorsal stream is mainly related to the spacial detection of objects and visual attention [16], [26]. Additionally, the ventral stream is linked to object recognition and shape representation [19].

The natural visual system generally performs the detection process as part of solving the classification task. However, there is no evidence on which process has greater impact at the moment of processing the visual information. In this work, we focus primarily on the classification task; nevertheless, the idea is to optimize the model using a multi-objective perspective by emulating both processes of the natural system, as a strategy for obtaining better classification results.

The main contribution of this work is the study of object classification from a multi-objective perspective, based on the integration of the single objective approaches developed in [23], [30], while extending the preliminary results published in [31]. In this article the system's implementation is detailed following the SPEA2 algorithm.

This methodology has been implemented for classifying the Person class from the GRAZ-02 database [31], which is part of the European network PASCAL's Visual Objects Classes challenge. Moreover, in this work we optimize the system for classifying other four classes: Bike and Person from GRAZ-01, and Bike and Cars from GRAZ-02. We opted for these five classes since they are used as a testing standard for image classification [29], [11], [14], [13], and also because each evolutionary run requires a considerable computational time as will be explained in Section 3.

As part of the contributions, a cross-validation process is presented in this work, since GP has been criticized for over-training, and such test shows the performances of the methodology that we use to validate the results of the learning strategy that will be explained in the following section. In addition, a frequency of use analysis of the results allows us to observe how often the functions and terminals are applied in the proposed model while processing the GRAZ database.

The remainder of this paper is organized as follows. Section 2 details the stages of our approach using a multi-objective evolutionary framework, where we describe the AVCMO model focusing on the proposed methodology for building the image descriptor and the brain programming algorithm under a multi-objective approach. Then, Section 3 provides the performance of the AVCMO model for image classification of GRAZ-01 and GRAZ-02 classes. Finally, the conclusions for this work are given in Section 4.