Global salient information maximization for saliency detection

Abstract

In this paper, a new method for saliency detection is proposed. Based on the defined features of the salient object, we solve the problem of saliency detection from three aspects. Firstly, from the view of global information, we partition the image into two clusters, namely, salient component and background component by employing Principal Component Analysis (PCA) and k-means clustering. Secondly, the maximal salient information is applied to find the position of saliency and eliminate the noise. Thirdly, we enhance the saliency for the salient regions while weaken the background regions. Finally, the saliency map is obtained based on these aspects. Experimental results show that the proposed method achieves better results than the state of the art methods. And this method can be applied for graph based salient object segmentation.

Introduction

Humans and other animals cannot pay attention to more than one or very few items simultaneously. They only could facilitate learning and survival by enabling organisms to focus their limited perceptual and cognitive resources on the most pertinent subset of the available sensory data. This subset can be called as saliency of the sensory data which is the state or quality by which it stands out relative to its neighbors [1], such as an object, a person, a pixel. Usually, the saliency of the sensory data represents the most valuable information from a large amount of the sensory data in the fields of computer vision, design, graphics, and human computer interaction. For computer vision, substantial progresses [2], [3] have been made in the psychophysics of visual attention and many computational models [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24] of visual attention have been proposed in the early years. Visual attention is useful for some applications, such as complex scenes understanding [4], [6], [7], [8], [9], [10], [11], object detection [20], tracking [17], [19], retargeting [21], and recognition [24]. The salient object detection is one of the most popular applications of visual attention.

Most models of visual attention for saliency detection are biologically inspired and based on a bottom-up computational model [25]. They are usually used to detect the approximate position of the salient region or object, which can be classified as four categories of models of visual attention.

The first is about methods with graph representation. In the early report [26], random walks are employed on image lattice to compute the visual saliency. Harel et al. [18] extend this method by proposing a better dissimilarity measure to model the transition probability between two vertices. Wang et al. [27] utilize fully-connected graph structure to simulate the cortical neuron connection, and yield a new visual saliency measure called Site Entropy Rate.

The second category is about the methods based on information maximization. In these methods, the information is considered as the driving force behind attentive sampling, the visual saliency is measured using the rarity of feature. Bruce et al. [17] employ the self-information of sparse features to measure saliency. Mancas et al. [22] describe a rarity-based visual attention model to provide an approximation of human perception by visualizing its gradual discovery of the visual environment. It is based on the theory of self-information. Hou et al. [19] introduce the incremental coding length (ICL) to allocate different amount of energy to features according to their rarity under the assumption that salient feature can offer entropy gain. However, the obtained salient information is usually local information which has its limits to detect saliency regions accurately. And the obtained saliency map always represents the approximate position of the salient regions or objects, while lot of salient information is ignored.

The third category is about the methods for the center-surround mechanism. These methods model the center-surround mechanism of primary visual cortical cells. A biological-plausible visual saliency model [20] is proposed, which implies that visual perception relies on a linear measure of similarity on intensity, color, and orientation. However, Gao et al. [28], [29] find that this is in conflict with the well known properties of high level human judgments of similarity. Therefore, Gao et al. propose a discriminant center-surround hypothesis using mutual information, and it can provide optimal solutions for many other saliency problems for computer vision.

The fourth category is about the methods based on machine learning. Liu et al. [30] combine pixel-based saliency measurements in a CRF and derives a binary segmentation separating the object from the background. Alexe et al. [31] design an objectness measure under a Bayesian framework and explicitly training it to distinguish windows containing an object from background windows.

Recently, a new type of method [21] is proposed for context-aware saliency detection. This method aims at detecting the image regions which represent the scene. It is different from previous definitions whose goal is to either identify fixation points or detect the domain object. Cheng et al. [13] propose a histogram-based contrast method (HC) to measure saliency. HC-maps assign pixel-wise saliency values based simply on color separation from all other image pixels to produce full resolution saliency maps. They also incorporate spatial relations to produce region-based contrast (RC) maps to improve the HC-maps.

From the above analysis, we can see that most of the methods focus on obtaining the saliency map which is made up of parts of the salient region or object. The context-aware method [21] can obtain the saliency map which describes the whole salient region or object. However, it also introduces a lot of false alarms. Therefore, the challenge of saliency detection is to acquire the salient region or object as accurate as possible, not only approximate position of the saliency. Solving this problem is one of the most important goals for scene understanding, which involves interpreting the whole image by recognizing all the objects of interest within an image and their spatial extent or shape.

In this paper, we propose a novel method to saliency detection. Firstly, a definition about the features of the salient objects is introduced. Secondly, we present the method of global salient information maximization which can obtain the salient information from three aspects. (1) Detect global salient component by PCA based method; (2) extract maximal salient information to eliminate the noise of saliency detection; (3) enhance the salient information. Finally, the optimal saliency map is obtained.

This paper is organized into six sections. The next section introduces features of the salient object used in our paper. Section 3 describes the proposed theory that finds the salient objects. Section 4 provides some experimental results of the proposed approach, and comparison with 11 previous works about saliency detection is also performed in qualitative and quantitative measures. In Section 5, we apply the proposed saliency detection method to graph based image segmentation. Finally, Section 6 draws the conclusion of this paper.