Focusing intermediate pixels loss for salient object segmentation

Abstract

To improve the network performance of salient object segmentation, many researchers modified the loss functions and set weights to pixel losses. However, these loss functions paid less attention to intermediate pixels of which the predicted probabilities lie in the intermediate region between correct and incorrect classification. To solve this problem, focusing intermediate pixels loss is proposed. Firstly, foreground and background are divided into correct and incorrect classified sets respectively to discover intermediate pixels which are difficult to determine the category. Secondly, the intermediate pixels are paid more attention according to the predicted probability. Finally, misclassified pixels are strengthened dynamically with the order of training epochs. The proposed method can 1) make the model focus on intermediate pixels that have more uncertainty; 2) solve the vanishing gradient problem of Focal Loss for well-classified pixels. Experiment results on six public datasets and two different type of network structures show that the proposed method performs better than other state-of-the-art weighted loss functions and the average F_β is increased by about 2.7% compared with typical cross entropy.

Appendix

When calculating the gradient of loss function, we found that the gradient we calculated (Fig. 5b) is inconsistent with Dual Focal Loss (DFL) (Fig. 1b) [11] in background. The two images are shown below.

After research, we found that Hossain et al. [11] misunderstood cross entropy loss formula. The binary cross entropy loss is calculated as follows.

$${\mathrm{L}}_{\mathrm{CE}}=-{\sum}_{\mathrm{i}=1}^{\mathrm{T}}\left[{\mathrm{y}}_{\mathrm{i}}\log {\mathrm{p}}_{\mathrm{i}}+\left(1-{\mathrm{y}}_{\mathrm{i}}\right)\log \left(1-{\mathrm{p}}_{\mathrm{i}}\right)\right]$$

In foreground, y_i is 1. Cross entropy loss of foreground pixels is as follows.

$${\mathrm{L}}_{\mathrm{F}}^{\mathrm{i}}=-\log {\mathrm{p}}_{\mathrm{i}}$$

In background, y_i is 0. Cross entropy loss of background pixels is as follows.

$${\mathrm{L}}_{\mathrm{B}}^{\mathrm{i}}=-\log \left(1-{\mathrm{p}}_{\mathrm{i}}\right)$$

While Hossain et al. [11] calculated cross entropy loss of foreground and background in the same way.

$${\mathrm{L}}_{\mathrm{F}}^{\mathrm{i}}={\mathrm{L}}_{\mathrm{B}}^{\mathrm{i}}=-\log {\mathrm{p}}_{\mathrm{i}}$$

To explore the reason for the author’s error, we think it may be affected by Focal Loss (FL) [20]. On page 3 of FL, for the convenience of writing, cross entropy is written as follows.

$$\mathrm{CE}\left(p,y\right)=\mathrm{CE}\left({p}_{\mathrm{t}}\right)=-\log \left({p}_{\mathrm{t}}\right)$$

Focal loss is written as follows.

$$\mathrm{FL}\left({p}_{\mathrm{t}}\right)=-{\left(1-{p}_{\mathrm{t}}\right)}^{\gamma }=-\log \left({p}_{\mathrm{t}}\right)$$

However, p_t is not simply expressed as pixel predicted probability, it is distinguished to foreground and background. Hossain et al. [11] may directly regard p_t as predicted probability of the pixel. The screenshot of original focal loss is as follows.

Focal Loss

The Focal Loss is designed to address the one-stage object detection scenario in which there is an extreme imbalance between foreground and background classes during trainig (e.g., 1:1000). We introduce the focal loss starting from the cross entropy (CE) loss for binary classification¹:

$$\mathrm{CE}\left(p,y\right)=\left\{\begin{array}{c}-\log (p)\ \mathrm{if}\ y=1\\ {}-\log \left(1-p\right)\mathrm{otherwie}\end{array}\right.$$

(18)

In the above y ∈ {±}1 specifies the ground-truth class and p ∈ [0, 1] is the model’s estimated probability for the class with label y = 1. For notational convience, we define p_t:

$${p}_{\mathrm{t}}=\left\{\begin{array}{c}p\kern0.5em \mathrm{if}\ y=1\\ {}1-p\kern0.5em \mathrm{otherwise}\end{array}\right.$$

(19)

and rewrite CE (p, y) = CE(p_t) =  − log(p_t).