A model for the gray-intensity distribution of historical handwritten documents and its application for binarization

Abstract

In this article, our goal is to describe mathematically and experimentally the gray-intensity distributions of the fore- and background of handwritten historical documents. We propose a local pixel model to explain the observed asymmetrical gray-intensity histograms of the fore- and background. Our pixel model states that, locally, the gray-intensity histogram is the mixture of gray-intensity distributions of three pixel classes. Following our model, we empirically describe the smoothness of the background for different types of images. We show that our model has potential application in binarization. Assuming that the parameters of the gray-intensity distributions are correctly estimated, we show that thresholding methods based on mixtures of lognormal distributions outperform thresholding methods based on mixtures of normal distributions. Our model is supported with experimental tests that are conducted with extracted images from DIBCO 2009 and H-DIBCO 2010 benchmarks. We also report results for all four DIBCO benchmarks.

Appendices

Appendix 1: A Frontier pixel convergence

Let $X\sim N(\mu _1,\sigma _2),\,Y\sim N(\mu _2,\sigma _2)$ and $U\sim \hbox {Unif}(0,1)$, all independent. In this section we prove that if $W$ is defined as $W:=uX+(1-u)Y$, then

• 1. If $U$ is a degenerated random variable with value $u$, we have that $W\sim N(u\mu _1+(1-u)\mu _2, \sqrt{u^2\sigma _1^2\!+\!(1\!-\!u)^2\sigma _2^2})$.

• 2. If $\sigma _1=\sigma _2=\sigma$ then $W$ has lighter tails than a random variable that is normally distributed with standard deviation $\sqrt{3}\sigma$.

• 3. As $\sigma _1,\sigma _2 \rightarrow 0$ we have $W$ tends to be a random variable that is uniform distributed.

Proof of 1: From the properties of the moment generating functions (MGF’s), we have:

$$\begin{aligned} M_{W}(t)=M_{X}(ut)M_{Y}([1-u]t), \end{aligned}$$

(15)

where $M_W(t):=E(\exp \{tW\})$.

Since the MGF of a normal random variable with parameters $(\mu , \sigma )$ is

$$\begin{aligned} \exp \left( \mu t+\frac{\sigma ^2t^2}{2}\right) \!, \end{aligned}$$

(16)

then (15) is equal to

$$\begin{aligned} M_{W}(t)&= \exp \left( \mu _1 tu+\frac{\sigma _1^2t^2u^2}{2}\right) \nonumber \\&\times \exp \left( \mu _2 t(1-u)+\frac{\sigma _2^2t^2(1-u)^2}{2}\right) \end{aligned}$$

(17)

$$\begin{aligned}&= \exp \left( t\left( u\mu _1+(1-u)\mu _2\right) \right. \nonumber \\&\left. +\frac{t^2}{2} \left( u^2\sigma _1^2+(1-u)^2\sigma _2^2\right) \right) \!, \end{aligned}$$

(18)

that corresponds to a random variable that is normally distributed with the specified parameters.

Proof of 2: From the properties of MGF’s, we have:

$$\begin{aligned} M_{W|u}(t)=M_{X|u}(ut)M_{Y|u}([1-u]t), \end{aligned}$$

(19)

where the function $M_{W|u}(t)$ denotes the moment generating function of $W$ with respect to the conditional density $f_{X,Y|U}(x,y|u)$.

Then the conditional MGF in (19) is equal to

$$\begin{aligned} M_{W|u}(t)&= \exp \left( t\left( u\mu _1+(1-u)\mu _2\right) \right. \nonumber \\&\left. + \frac{t^2\sigma ^2}{2}\left( u^2+(1-u)^2\right) \right) \!. \end{aligned}$$

(20)

Without loss of generality assume that $\mu _1>\mu _2$. To obtain unconditional MGF of $W$ we integrate the last expression over the $U$’s domain as following:

$$\begin{aligned} M_{W}(t)&= \int \limits _0^1 M_{W|u}(t)\hbox {d}u \end{aligned}$$

(21)

$$\begin{aligned}&= \int \limits _{0}^{1} c \cdot \exp \left( tu(\mu _1-\mu _2)+t^2\sigma ^2(u(u-1))\right) \hbox {d}u \nonumber \\ \end{aligned}$$

(22)

$$\begin{aligned}&= c \cdot \int \limits _{0}^{1} \exp \left( tu(\mu _1-\mu _2)+t^2\sigma ^2(u(u-1))\right) \hbox {d}u, \nonumber \\ \end{aligned}$$

(23)

where

$$\begin{aligned} c = \exp \left( t\mu _1+\frac{t^2\sigma ^2}{2}\right) \!. \end{aligned}$$

(24)

If $t>0$, then (23) is smaller than or equal to

$$\begin{aligned}&\le \exp \left( t\mu _1+\frac{t^2\sigma ^2}{2}\right) \exp \left( t(\mu _1-\mu _2)+\frac{t^2\sigma ^2}{4}\right) \end{aligned}$$

(25)

$$\begin{aligned}&= \exp \left( t\mu _1+\frac{t^23\sigma ^2}{2}\right) \!. \end{aligned}$$

(26)

Similarly, if $t<0$ then (23) is smaller than or equal to

$$\begin{aligned} \exp \left( t\mu _2+\frac{t^23\sigma ^2}{2}\right) \!. \end{aligned}$$

(27)

Since in both cases, the moment generating function is dominated by a MGF of a random variable with Normal distribution and variance $3\sigma ^2$, the conclusion follows.

Proof of 3: Based on (20) we have that

$$\begin{aligned} M_{W|u}(t)\rightarrow \exp \left( \mu _1tu+\mu _2t(1-u)\right) \!, \end{aligned}$$

(28)

as $\sigma _1,\sigma _2 \rightarrow 0$.

To obtain the MGF of $W$ we integrate $M_{W|u}(t)$ as

$$\begin{aligned} M_{W}(t)&= \int \limits _0^1\exp \left( \mu _1tu+\mu _2t(1-u) \right) \hbox {d}u\end{aligned}$$

(29)

$$\begin{aligned}&= \exp \left( \mu _2t\right) \int \limits _0^1\exp \left( u(\mu _1t-\mu _2t)\right) \hbox {d}u\end{aligned}$$

(30)

$$\begin{aligned}&= \frac{\exp \left( \mu _1t\right) -\exp \left( \mu _2t\right) }{t(\mu _1-\mu _2)}, \end{aligned}$$

(31)

that is the MGF of an uniform distributed random variables with lower and upper limits equal to $\min \{\mu _1,\mu _2\}$ and $\max \{\mu _1,\mu _2\}$, respectively.

Appendix 2: Quasi-thresholding methods

To simplify our notation, the subindexes $f,\,b,\,if,\,of,\,ib$, and $ob$ abbreviate the foreground, background, inner foreground, outer foreground, inner background, and outer background sets, respectively. Furthermore, we also simplify our notation of the means and variances of gray intensities of a set $\mathcal{A }$ by

$$\begin{aligned} \hat{\mu }_{\mathcal{A }}&= \frac{1}{| \mathcal{A } |} \sum _{\varvec{p} \in \mathcal{A } } I(\varvec{p} )\quad \text { and} \end{aligned}$$

(32)

$$\begin{aligned} \hat{\sigma }^{2}_{\mathcal{A }}&= \frac{1}{| \mathcal{A } |} \sum _{\varvec{p} \in } \left[ I(\varvec{p} ) - \hat{\mu }_{\mathcal{A }} \right] ^{2}. \end{aligned}$$

(33)

1.1 Quasi-threshold $LI$

The mixture $LI$ models the gray-intensity histogram as the mixture of two distributions: Lognormal for the foreground and inverted lognormal for the background. Formally, its threshold is defined by Bayes rule as the value $x$ that satisfies:

$$\begin{aligned} w_{f}\lambda (x;\tilde{\mu }_{f},\tilde{\sigma }_{f}) = w_{b}\tilde{\lambda }(x;c_{b},\tilde{\tilde{\mu }}_{b},\tilde{\sigma }_{b}) \end{aligned}$$

(34)

such that $\hat{\mu }_{f} < x < \hat{\mu }_{b}$, where

$$\begin{aligned} w_{f} = \frac{ | \mathcal{F }|}{| \mathcal{P } |}, \quad w_{b} = \frac{ | \mathcal{B }|}{| \mathcal{P } |}, \end{aligned}$$

(35)

$\lambda (x;\tilde{\mu }_{f},\tilde{\sigma }_{f})$ and $\tilde{\lambda }(x;c_{b},\tilde{\tilde{\mu }}_{b},\tilde{\sigma }_{b})$ denote the probability distribution functions of the lognormal and inverted lognormal distributions. These functions are given by:

• (1) Lognormal:

  $$\begin{aligned} \lambda (x;\tilde{\mu }_{f},\tilde{\sigma }_{f}) = \frac{1}{x\tilde{\sigma }_{f}\sqrt{2\pi }} \exp \left( -\frac{ (\ln (x) - \tilde{\mu }_{f} )^{2} }{2\tilde{\sigma }^{2}_{f}} \right) \!,\nonumber \\ \end{aligned}$$

  (36)

  where

  $$\begin{aligned} \tilde{\mu }_{f}&= \ln (\hat{\mu }_{f}) - \frac{1}{2}\ln \left( 1 + \frac{\hat{\sigma }^{2}_{f}}{\hat{\mu }^{2}_{f}} \right) \quad \text { and } \end{aligned}$$

  (37)
  $$\begin{aligned} \tilde{\sigma }^{2}_{f}&= \frac{1}{2}\ln \left( 1 + \frac{\hat{\sigma }^{2}_{f}}{\hat{\mu }^{2}_{f}} \right) \!. \end{aligned}$$

  (38)

• (2) Inverted lognormal:

  $$\begin{aligned} \tilde{\lambda }(x;c_{b},\tilde{\tilde{\mu }}_{b},\tilde{\sigma }_{b}) = \lambda (c_{b} - x;\tilde{\tilde{\mu }}_{b},\tilde{\sigma }_{b}), \end{aligned}$$

  (39)

  where $\tilde{\sigma }_{b}$ is computed in an analogous manner as $\tilde{\sigma }_{f}$,

  $$\begin{aligned} \tilde{\tilde{\mu }}_{b}&= \ln (c_{b} - \hat{\mu }_{f}) - \frac{1}{2}\ln \left( 1 + \frac{\hat{\sigma }^{2}_{f}}{[c_{b} - \hat{\mu }_{f}]^{2}} \right) \!,\quad \text { and } \nonumber \\ \end{aligned}$$

  (40)
  $$\begin{aligned} c_{b}&= \underset{ \varvec{p} \in \mathcal{B } }{\max } \, \left( I(\varvec{p} ) \right) + 1. \end{aligned}$$

  (41)

1.2 Quasi-thresholding $NLIN$

The mixture $NLIN$ models the gray-intensity histogram as the mixture of the gray-intensity distributions of the inner foreground, outer foreground, inner background, and outer background. Such sets are estimated as:

$$\begin{aligned} \hat{\mathcal{F }}^{*}&= \mathcal{F }\setminus \mathcal{E }, \quad \hat{\mathcal{F }}^{\circ }= \mathcal{F }\cap \mathcal{E }, \quad \hat{\mathcal{B }}^{*}= \mathcal{B }\setminus \mathcal{E }, \quad \text {and} \nonumber \\&\hat{\mathcal{B }}^{\circ }= \mathcal{B }\cap \mathcal{E }, \end{aligned}$$

(42)

where $\mathcal{E }$ denotes the set of 8-edge pixels:

$$\begin{aligned} \mathcal{E }= \left\{ \varvec{p} \in \mathcal{P } \, \text {such that} \, \mathcal{F }_{1}(\varvec{p} ) \not = \emptyset \, \text {and} \, \mathcal{B }_{1}(\varvec{p} ) \not = \emptyset \right\} \end{aligned}$$

(43)

Once the frontier pixels are estimated, the gray-intensity distribution of the foreground is modeled as the mixture of a normal distribution (corresponding to the inner foreground) and a lognormal distribution (corresponding to the outer foreground). On the other hand, the gray-intensity distribution of the background is modeled as the mixture of a normal distribution (corresponding to the inner background) and an inverted lognormal distribution (corresponding to the outer background).

Formally, the threshold of $NLIN$ is defined by Bayes rule as the value $x$ that satisfies:

$$\begin{aligned} M_{f}(x) = M_{b}(x) \end{aligned}$$

(44)

such that $\hat{\mu }_{f} < x < \hat{\mu }_{b}$, where

$$\begin{aligned} M_{f}(x)&= \hat{w}_{if}\phi (x;\hat{\mu }_{if},\hat{\sigma }_{if})\nonumber \\&+ \hat{w}_{of}\lambda (x;\tilde{\mu }_{of},\tilde{\sigma }_{of}),\end{aligned}$$

(45)

$$\begin{aligned} M_{b}(x)&= \hat{w}_{ob}\tilde{\lambda }(x;c_{ob}, \tilde{\tilde{\mu }}_{ob}, \tilde{\sigma }_{ob})\nonumber \\&+ \hat{w}_{ib}\phi (x;\hat{\mu }_{ib},\hat{\sigma }_{ib}),\end{aligned}$$

(46)

$$\begin{aligned} \hat{w}_{if}&= \frac{|\hat{\mathcal{F }}^{*}|}{|\mathcal{P } |}, \quad \hat{w}_{of} = \frac{|\hat{\mathcal{F }}^{\circ }|}{|\mathcal{P } |}, \quad \hat{w}_{ob} = \frac{|\hat{\mathcal{B }}^{\circ }|}{|\mathcal{P } |}, \quad \text {and} \nonumber \\&\hat{w}_{ib} = \frac{|\hat{\mathcal{B }}^{*}|}{|\mathcal{P } |}. \end{aligned}$$

(47)

The functions $\lambda (x;\tilde{\mu }_{of},\tilde{\sigma }_{of})$ and $\tilde{\lambda }(x;c_{ob},\tilde{\tilde{\mu }}_{ob},\tilde{\sigma }_{ob})$ are defined in a similar manner as in the section “Quasi-threshold $LI$” of Appendix; $\phi (x;\hat{\mu }_{if},\hat{\sigma }_{if})$ denotes the probability density function of a normal distribution given by:

$$\begin{aligned} \phi (x;\hat{\mu }_{if},\hat{\sigma }_{if}) = \frac{1}{\hat{\sigma }_{if}\sqrt{2\pi }} \exp \left( -\frac{ (x - \hat{\mu }_{if} )^{2} }{2\hat{\sigma }^{2}_{if}} \right) . \end{aligned}$$

(48)

In similar manner, $\phi (x;\hat{\mu }_{ib},\hat{\sigma }_{ib})$ is defined.

1.3 Quasi-thresholding methods based on normal distributions

We implemented two mixtures based on normal distributions: $NN$ and $NNNN$. The former mixes two normal distributions to approximate the gray-intensity distribution, while the latter mixes four normal distributions. Their parameters are estimated in similar manner as in the previous subsections.

The threshold of $NN$ is defined by Bayes rule as the value $x$ that satisfies:

$$\begin{aligned} w_{f}\phi (x;\hat{\mu }_{f},\hat{\sigma }_{f}) = w_{b}\phi (x;\hat{\mu }_{b},\hat{\sigma }_{b}) \end{aligned}$$

(49)

such that $\hat{\mu }_{f} < x < \hat{\mu }_{b}$. Likewise, the threshold of $NNNN$ is defined as:

$$\begin{aligned} M_{f}(x) = M_{b}(x) \end{aligned}$$

(50)

such that $\hat{\mu }_{f} < x < \hat{\mu }_{b}$, where

$$\begin{aligned} M_{f}(x) = \hat{w}_{if}\phi (x;\hat{\mu }_{if},\hat{\sigma }_{if}) + \hat{w}_{of}\phi (x;\hat{\mu }_{of},\hat{\sigma }_{of}) \end{aligned}$$

(51)

and

$$\begin{aligned} M_{b}(x) = \hat{w}_{ob}\phi (x;\hat{\mu }_{ob},\hat{\sigma }_{ob}) + \hat{w}_{ib}\phi (x;\hat{\mu }_{ib},\hat{\sigma }_{ib}). \end{aligned}$$

(52)