Spatio-Colour Asplünd’s Metric and Logarithmic Image Processing for Colour Images (LIPC)

Abstract

Asplünd’s metric, which is useful for pattern matching, consists in a double-sided probing, i.e. the over-graph and the sub-graph of a function are probed jointly. This paper extends the Asplünd’s metric we previously defined for colour and multivariate images using a marginal approach (i.e. component by component) to the first spatio-colour Asplünd’s metric based on the vectorial colour LIP model (LIPC). LIPC is a non-linear model with operations between colour images which are consistent with the human visual system. The defined colour metric is insensitive to lighting variations and a variant which is robust to noise is used for colour pattern matching.

1 Introduction

The Asplünd’s metric initially defined for binary shapes [1, 4] has been extended to grey-scale images by Jourlin et al. [6, 7] and to colour and multivariate images in the LIP framework by Noyel and Jourlin [13]. It consists in probing a function by two homothetic template functions, i.e. the probes which are computed by the LIP multiplication.

The Logarithmic Image Processing (LIP) model initially defined for grey level images by Jourlin and Pinoli [8, 9] is perfectly suited for images acquired by transmitted light (i.e. when the observed object is located between the source and the sensor) and by reflected light because of its consistency with the Human Vision [3]. The necessity to analyse together the channels of the colour images (i.e. by a vectorial analysis) has led to the introduction of the Logarithmic Image Processing for Colour images (LIPC) by Jourlin et al. [5].

The LIP Asplünd’s metric was defined in [13] in a marginal way (i.e. channel by channel). In this paper, our contribution is to extend this metric by using the spatio-colour properties [11, 12] of the colour LIPC framework.

After some prerequisites about the colour LIPC model and about the marginal LIP Asplünd’s metric, we will define a spatio-colour Asplünd’s metric in the LIPC framework. Then we will perform spatio-colour pattern matching which is robust to noise. Examples will illustrate the definitions.

2 Prerequisites

2.1 LIPC Model

A colour image $\mathbf {f}$, defined on a domain $D \subset \mathbb {R}^{N}$, with values in $\mathcal {T}^3 = [0,M[^3$, $M \in \mathbb R$, is written:

$$\begin{aligned} \mathbf {f} : \left\{ \begin{array}{cc} D \rightarrow &{} \mathcal {T}^3 = [0,M[^3\\ x \rightarrow &{} \mathbf {f}(x) = \left( f_R(x) , f_G(x) , f_B(x) \right) \\ \end{array}\right. \end{aligned}$$

(1)

$f_R$, $f_G$, $f_B$ are the red, green and blue channels (i.e. components) of $\mathbf {f}$, $\mathbf {f}(x)$ is a vector-pixel and x is the spatial coordinate of the vector-pixel. The real value M is equal to $2^8=256$ for 8 bits images. Given P the number of pixels, the matrix $\mathbf {F}$ of $E \rightarrow \mathcal {T}$, $E = 3 \times P$, associated to the image $\mathbf {f}$ is written:

$$\begin{aligned} \mathbf {F}= \left[ \begin{array}{cccc} f_R(x_1) &{} f_R(x_2) &{} ... &{} f_R(x_P)\\ f_G(x_1) &{} f_G(x_2) &{} ... &{} f_G(x_P)\\ f_B(x_1) &{} f_B(x_2) &{} ... &{} f_B(x_P)\\ \end{array}\right] \end{aligned}$$

(2)

To make the comments easier, the word “image” designates both the matrix $\mathbf {F}$ and the image $\mathbf {f}$. The image space for 24-bits images $\mathbf {F}$ is written $\mathcal {I}^3$.

A colour image is a particular case of a multivariate image defined as $\mathbf {f}_{\lambda } : D \rightarrow \mathcal {T}^L$, where $L \in \mathbb {N}$ is the number of channels [11, 12].

As for the grey-level LIP, the colour LIPC framework is based on colour transmittance [5]. It is valid for transmitted and reflected images [3]. It models the human perceptual system approach by taking into account: (i) the sensitivity of the human eye in the visible domain characterised by colour matching functions of Stiles and Burch [14] and (ii) the spectral distribution of light with the D65 illuminant [15].

In the LIPC framework, the transmittance of the sum of two images is equal to the product of their transmittances $\mathbf {T}_{\mathbf {F}}$ and $\mathbf {T}_{\mathbf {G}}$: . The symbol of the LIPC addition is and $*$ represents the element-wise multiplication [5]. The addition of two images $\mathbf {F}, \mathbf {G}\in \mathcal {I}^3$ is:

(3)

$\mathbf {\acute{K}}$ and $\mathbf {\acute{U}}$ are real matrices of size $3 \times 3$ corresponding to the LIPC mixing model¹. From the LIPC addition, a multiplication by a scalar $\alpha \in \mathbb R$ has been defined:

(5)

The space is the positive cone of a vector space with robust mathematical properties.

Physical interpretation [5]: the LIPC addition corresponds to the superposition of two semi-transparent layers. A LIPC multiplication by a scalar $\alpha \in \, ]0,1[$ brightens the result by suppressing layers, while a scalar $\alpha \in \, ]1,+\infty [$ darkens the result by superimposing $\alpha$ times the image on itself.

2.2 Marginal Asplünd’s Metric for Colour and Multivariate Images

In [13], an Asplünd’s metric between colour images was defined with the LIP model by using a marginal approach (i.e. channel by channel) [11, 12].

Definition 1

The Asplünd’s metric (with LIP multiplication) between two colour images $\mathbf {f}$ and $\mathbf {g}$ on a region $Z \subset D$ is

(6)

with and .

In particular, by the property of the distance .

3 Asplünd’s Metric Defined in the Logarithmic Image Processing Colour (LIPC) Framework

Given two colours $C_1=(r_1,g_1,b_1)$, $C_2=(r_2,g_2,b_2) \in \mathcal {T}^3$, as we are only looking for lower and upper bounds, a marginal order [2] is used: $C_1 \ge C_2 \Leftrightarrow \{r_1\ge r_2$ and $g_1\ge g_2$ and $b_1\ge b_2 \}$.

Definition 2

Given two colours $C_1$, $C_2 \in \mathcal {T}^3$, their Asplünd’s distance (with LIPC multiplication) is equal to:

(7)

and .

Strictly speaking, is a metric if the colours $C_n$ are replaced by their equivalence classes .

Comment: in Eq. 7 contrary to the Asplünd’s distance (with LIP multiplication) defined in [13] (Eq. 6), we have $\lambda \le \mu$ because, by definition of the colour LIPC model the scales are inverted as compared to the grey LIP model [5].

Colour metrics (with LIPC multiplication) between two colour images $\mathbf {f}$ and $\mathbf {g}$ may be defined as the sum ($d_1$ metric) or the supremum ($d_{\infty }$) of on the region of interest $Z \subset D$ of cardinal $\#Z$

(8)

The Asplünd’s metric can be extended to colour functions.

Definition 3

The colour Asplünd’s metric (with LIPC multiplication) between two colour images $\mathbf {f}$ and $\mathbf {g}$ on a region $Z \subset D$ is

(9)

and .

In Fig. 1, the Asplünd’s metric has been computed between the colour probe $\mathbf {g}$ and the colour function $\mathbf {f}$ on their definition domain D.

Fig. 1.

Computation of the Asplünd’s distance between two colour functions . Each colour channel is represented by a line of the same colour. (Colour figure online)

Comment: the lower (resp. upper) bound (resp. ) may not be equal to any point of the function $\mathbf {f}$ but strictly less (or greater) than the function. Indeed, one can demonstrate that the following assertion is verified: “given ”.

The metric can be adapted to local processing with a colour template image (i.e. the probe) $\mathbf {t}$ defined on a spatial support $D_t \subset D$. For each point $x \in D$, the distance is computed on the neighbourhood $D_t(x)$ centred in x where $\mathbf {f}_{\left| D_t(x)\right. }$ is the restriction of $\mathbf {f}$ to $D_t(x)$.

Definition 4

Given a colour image $\mathbf {f}$ defined on D with values in $\mathcal {T}^3$, $\left( \mathcal {T}^{3}\right) ^{D}$, a colour probe $\mathbf {t}$ defined on $D_t$ with values in $\mathcal {T}^{3}$, $\left( \mathcal {T}^{3}\right) ^{D_{t}}$, and $D_t(x)$ the neighbourhood $D_t$ centred in $x \in D$, the map of Asplünd’s distances (with ) is:

(10)

In Fig. 2, the map of Asplünd’s distances is computed between a colour function and a colour probe. The minima of the map corresponds to the location of a pattern which is similar to the probe.

Fig. 2.

(c) Map of the Asplünd’s distances between a colour function and a probe. (a) and (b) Each colour channel is represented by a line of the same colour. (Colour figure online)

Asplünd’s distance is sensitive to noise because the probe lays on regional extrema that may be caused by noise (Fig. 1). In [7, 13], definitions of Asplünd’s distance with a tolerance on the extrema have been introduced. In this paper, we extend this definition for colour images with LIPC model.

To reduce the sensitivity of Asplünd’s distance to the noise, the “Measure metric” or “M-metric” has been defined in the context of “Measure Theory”. The image being digitized, the number of pixels of D is finite and the “measure” of a subset of D is linked to the cardinal of this subset, e.g. the percentage P of its elements with respect to D. We are looking for a subset $D'$ of D, such that $\mathbf {f}_{\left| D'\right. }$ and $\mathbf {g}_{\left| D'\right. }$ are neighbours for Asplünd’s metric and the complementary set $D\setminus D'$ of $D'$ into D is of small size when compared to D. This last condition is written as: $P(D \setminus D') = \frac{\#(D \setminus D')}{\#D} \le p$, where p is an acceptable percentage and $\#D$ is the number of elements in D.

Given $\epsilon$ a small positive real number, the neighbourhood of function $\mathbf {f}$ is

(11)

The closest points of the probe to the function are discarded as in [5, 13].

Definition 5

Given two constant vector-pixels $\mathbf {c}_{\mu }, \mathbf {c}_{\lambda } \in \mathcal {T}^3$, a percentage p of points to be discarded. The colour Asplünd’s metric (with LIPC multiplication) with tolerance between two colour images $\mathbf {f}$ and $\mathbf {g}$ on a region $Z \subset D$ is

(12)

and and $\mathbf {c}_{\lambda }$ are increased such as a percentage p of points is discarded.

In Fig. 3, a tolerance of $p = 20\%$ is used to discard two points. The Asplünd’s distance decreases from 0.43 to 0.21.

A map of Asplünd’s distances (with ) can now be defined.

Definition 6

  Given a colour image $\mathbf {f}$ of $\left( \mathcal {T}^{3}\right) ^{D}$, a colour probe $\mathbf {t}$ of $\left( \mathcal {T}^{3}\right) ^{D_{t}}$ and a tolerance $p\in [0,1]$, the map of Asplünd’s distances with a tolerance is:

(13)

$D_t(x)$ is the neighbourhood $D_t$ centred in $x \in D$.

Fig. 3.

Colour Asplünd’s distance with a tolerance of $p = 20\%$. ($\mu$, $\lambda$) are the scalars multiplying the probe without tolerance. ($\mu '$, $\lambda '$) are the scalars multiplying the probe with tolerance. and (Colour figure online)

Fig. 4.

Maps of Asplünd’s distances without tolerance and with . $\tilde{\mathbf {f}}$ image with a white noise ($\sigma ^2=2.6$, spatial density 1%). (f) Correlation map. (Colour figure online)

4 Examples

In Fig. 4, we look for the bricks of a wall, similar to a colour probe. A blue brick has been added to the wall. In the image without noise $\mathbf {f}$, the regional minima of the map (dark points in Fig. 4b) correspond to the centre of the bricks similar to the probe (according to the Asplünd’s distance). The white rectangle corresponds to the maxima of the distances between the blue brick and the probe. Therefore, the distance is sensitive to colour (i.e. the hue). In the image with noise $\tilde{\mathbf {f}}$, the map without tolerance is more sensitive to noise (Fig. 4e) than the map with tolerance (Fig. 4c). Indeed, the minima are preserved into the map with tolerance (Fig. 4c) compared to the map without (Fig. 4e). The minima can be extracted using mathematical morphology [10, 16]. Importantly, all the maps of Asplünd’s distances are insensitive to the vertical lighting drift. Moreover, a correlation map is useless to find the location of the bricks (Fig. 4f).

In Fig. 5, two images of the same scene, a bright image $\mathbf {f}$ and a dark image $\tilde{\mathbf {f}}$, are acquired with two different exposure times. The probe $\mathbf {t}$ is extracted in the bright image and used to compute the map of Asplünd’s distance in the darker image. By finding the minima of the map, all the balls are detected and their contours are added to the image in Fig. 5(b). One can notice that the Asplünd’s distance is very robust to the lighting variations.

Fig. 5.

Detection of coloured balls on a dark image $\tilde{\mathbf {f}}$ with a probe $\mathbf {t}$ extracted in the bright image $\mathbf {f}$. (a) The border of the probe $\mathbf {t}$ is coloured in white. (Colour figure online)

5 Conclusion and Perspectives

A new spatio-colour Asplünd’s distance based on colour LIPC model has been defined. It is a true colour (i.e. vectorial) metric based on a colour model consistent with the human visual system. It is also consistent with the previous properties given in [7, 13]. An extension of this metric robust to noise has been presented and illustrated on pattern recognition examples. This double-sided probing distance is efficient for colour pattern matching and performs better than traditional correlation methods. In future work, we will evaluate in details the properties of this colour distance on practical applications (e.g. in medical, remote sensing or industrial images). We will compare it to the marginal colour Asplünd’s distance and we will study the links between Asplünd’s probing and mathematical morphology.