Real-time image enhancement with efficient dynamic programming

Abstract

Image enhancement is a problem of fundamental importance in the area of low level image processing. The goal of image enhancement is to significantly improve the visual effects of images or to obtain the fine details that are invisible in degraded images. In this paper, a new accurate image enhancement algorithm is developed to efficiently perform image enhancement with a dynamic programming approach. Specifically, an objective function is developed for the mappings between an original image and its enhanced versions to evaluate the effectiveness of enhancement. The objective function is then optimized by a dynamic programming algorithm to achieve the optimal enhancement effect. It is also shown that the computation efficiency of this dynamic programming algorithm can be significantly improved when certain conditions are satisfied. Testing results show that this new algorithm can efficiently generate images with significantly improved effectiveness of enhancement and is thus potentially useful for real-time applications. An implementation of the algorithm in MATLAB is freely available at the link: https://github.com/yinglei2020/YingleiSong.

Appendices

Appendix 1. The Proof of Lemma 1

We prove the lemma by contradiction. If there exists integers q₁and v₁that satisfy1 < q₁ ≤ h, 1 ≤ v₁ < l and T(q₁, v₁ + 1) < T(q₁, v₁)holds. Letw₁ = T(q₁, v₁)andw₂ = T(q₁, v₁ + 1), then w₂ < w₁ holds. Since f is a monotonously increasing convex function, F_kis also a monotonously increasing convex function.

From the definition of w₁,w₂and the fact that w₂ < w₁, it is clear that the following inequality holds for w₁andw₂.

$$ R\left({q}_1-1,{w}_2\right)+{F}_{q_1}\left({e}_{v_1}-{e}_{w_2}\right)\le R\left({q}_1-1,{w}_1\right)+{F}_{q_1}\left({e}_{v_1}-{e}_{w_1}\right) $$

(A.1)

The above inequality implies the following inequality.

$$ R\left({q}_1-1,{w}_1\right)-R\left({q}_1-1,{w}_2\right)\ge {F}_{q_1}\left({e}_{v_1}-{e}_{w_2}\right)-{F}_{q_1}\left({e}_{v_1}-{e}_{w_1}\right) $$

(A.2)

On the other hand, since ${F}_{q_1}$is a monotonously increasing convex function, the following inequality must hold for ${F}_{q_1}$.

$$ {F}_{q_1}\left({e}_{v_1}-{e}_{w_2}\right)-{F}_{q_1}\left({e}_{v_1}-{e}_{w_1}\right)>{F}_{q_1}\left({e}_{v_1+1}-{e}_{w_2}\right)-{F}_{q_1}\left({e}_{v_1+1}-{e}_{w_1}\right) $$

(A.3)

From the inequalities in Eqs. A.2 and A.3, the following inequality can be obtained.

$$ R\left({q}_1-1,{w}_1\right)-R\left({q}_1-1,{w}_2\right)>{F}_{q_1}\left({e}_{v_1+1}-{e}_{w_2}\right)-{F}_{q_1}\left({e}_{v_1+1}-{e}_{w_1}\right) $$

(A.4)

The following inequality can be obtained from the inequality in Eq. A.4.

$$ R\left({q}_1-1,{w}_1\right)+{F}_{q_1}\left({e}_{v_1+1}-{e}_{w_1}\right)>R\left({q}_1-1,{w}_2\right)+{F}_{q_1}\left({e}_{v_1+1}-{e}_{w_2}\right) $$

(A.5)

It is clear that the inequality in Eq. A.5 contradicts with the fact that w₂ maximizes the right hand side in Eq. 6 for pair (q, v₁ + 1). The Lemma immediately follows from the contradiction.

Appendix 2. The Proof of Lemma 2

We use the principle of induction to prove the Lemma. In the case where q = 2, from Eq. 6, the following equation holds for R(q, v).

$$ R\left(q,v\right)={F}_2\left({e}_v-{e}_1\right) $$

(B.1)

Since fis a monotonously increasing convex function, F₂ is also a monotonously increasing convex function. The following inequality thus holds due to the fact that e_v + 1 − e_v = e_v − e_v − 1.

$$ {F}_2\left({e}_v-{e}_1\right)-F\left({e}_{v-1}-{e}_1\right)>{F}_2\left({e}_{v+1}-{e}_1\right)-{F}_2\left({e}_v-{e}_1\right) $$

(B.2)

The lemma thus holds in the case where q = 2.

We then assume that the Lemma holds for q = k₁, wherek₁is an integer that satisfies2 ≤ k₁ < h, we need to show that the Lemma also holds when q = k₁ + 1. We use w₁, w₂and w₃to denote the integers that can maximize the right hand side of Eq. 6 for pairs (k₁ + 1, v − 1), (k₁ + 1, v) and (k₁ + 1, v + 1) respectively. From Lemma 1,w₁ ≤ w₂ ≤ w₃must hold. It is thus clear that there are two possible cases on the relationship between w₁ and w₃. Specifically, either w₁ = w₃ or w₁ < w₃.

If w₁ = w₃ is the case, it is clear that w₁ = w₂ and w₂ = w₃ both hold. From Eq. 6, the following equations hold.

$$ R\left({k}_1+1,y\right)-R\left({k}_1+1,y-1\right)={F}_{k_1+1}\left(e{}_y-{e}_{w_1}\right)-{F}_{k_1+1}\left({e}_{y-1}-{e}_{w_1}\right) $$

(B.3)

$$ R\left({k}_1+1,y+1\right)-R\left({k}_1+1,y\right)={F}_{k_1+1}\left({e}_{y+1}-{e}_{w_1}\right)-{F}_{k_1+1}\left({e}_y-{e}_{w_1}\right) $$

(B.4)

Since F_q is a monotonously increasing convex function and e_y + 1 − e_y = e_y − e_y − 1, the following inequality holds.

$$ {F}_{k_1+1}\left({e}_y-{e}_{w_1}\right)-{F}_{k_1+1}\left({e}_{y-1}-{e}_{w_1}\right)>{F}_{k_1+1}\left({e}_{y+1}-{e}_{w_1}\right)-{F}_{k_1+1}\left({e}_y-{e}_{w_1}\right) $$

(B.5)

The Lemma thus holds in this case.

On the other hand, if w₁ < w₃ is the case, we can obtain the following equations and inequalities from Eq. 6.

$$ R\left({k}_1+1,y-1\right)=R\left({k}_1,{w}_1\right)+{F}_q\left({e}_{y-1}-{e}_{w_1}\right) $$

(B.6)

$$ R\left({k}_1+1,y\right)\ge R\left({k}_1,{w}_1+1\right)+{F}_q\left({e}_y-{e}_{w_1+1}\right) $$

(B.7)

$$ R\left({k}_1+1,y\right)\ge R\left({k}_1,{w}_3-1\right)+{F}_q\left({e}_y-{e}_{w_3-1}\right) $$

(B.8)

$$ R\left({k}_1+1,y+1\right)=R\left({k}_1,{w}_3\right)+{F}_q\left({e}_{y+1}-{e}_{w_3}\right) $$

(B.9)

Since ${e}_{y-1}-{e}_{w_1}={e}_y-{e}_{w_1+1}$ and ${e}_y-{e}_{w_3-1}={e}_{y+1}-{e}_{w_3}$both hold, Eqs. B.6, B.7, B.8, and B.9 lead to the following inequalities.

$$ R\left({k}_1+1,y\right)-R\left({k}_1+1,y-1\right)\ge R\left({k}_1,{w}_1+1\right)-R\left({k}_1,{w}_1\right) $$

(B.10)

$$ R\left({k}_1+1,y+1\right)-R\left({k}_1+1,y\right)\le R\left({k}_1,{w}_3\right)-R\left({k}_1,{w}_3-1\right) $$

(B.11)

Since w₁ < w₃, we have w₁ + 1 ≤ w₃. From the assumption that the Lemma holds when q = k₁, the following inequality holds.

$$ R\left({k}_1,{w}_1+1\right)-R\left({k}_1,{w}_1\right)\ge R\left({k}_1,{w}_3\right)-R\left({k}_1,{w}_3-1\right) $$

(B.12)

From the inequalities in Eqs. B.9, B.10, and B.11, the following inequality holds.

$$ R\left({k}_1+1,y\right)-R\left({k}_1+1,y-1\right)\ge R\left({k}_1+1,y+1\right)-R\left({k}_1+1,y\right) $$

(B.13)

The Lemma thus holds when q = k₁ + 1. The lemma thus follows from the principle of induction.

Appendix 3. The Proof of Theorem 1

From Lemma 1, since T(q, y) ≥ T(q, y − 1) holds for each pair (q, y), where 1 < y ≤ l, the search for the integer that maximizes the right hand side of Eq. 6 to compute R(q, y)can start with T(q, y − 1).

On the other hand, we assume the search finds the first location w such that T(q, y − 1) ≤ w < y and the following inequality holds.

$$ R\left(q-1,w\right)+{F}_q\left({e}_y-{e}_w\right)>R\left(q-1,w+1\right)+{F}_q\left({e}_y-{e}_{w+1}\right) $$

(C.1)

We show that w is the integer that maximizes the right hand side of Eq. 6 for R(q, y). In other words, w = T(q, y) and the search can stop at w and the value of R(q, y)can be computed based onw. To show this, we show that for any integer v that satisfiesw ≤ v < y, the following inequality holds.

$$ R\left(q-1,v\right)+{F}_q\left({e}_y-{e}_v\right)>R\left(q-1,v+1\right)+{F}_q\left({e}_y-{e}_{v+1}\right) $$

(C.2)

We show this fact by the principle of induction. When v = w, the inequality in Eq. C.2 holds from Eq. C.1. We thus assume that Eq. C.2 holds for v = k₂, where w ≤ k₂ < y − 1. The following inequality thus holds.

$$ R\left(q-1,{k}_2+1\right)-R\left(q-1,{k}_2\right)<{F}_q\left({e}_y-{e}_{k_2}\right)-F\left({e}_y-{e}_{k_2+1}\right) $$

(C.3)

From Lemma 2, the following inequality holds.

$$ R\left(q-1,{k}_2+1\right)-R\left(q-1,{k}_2\right)\ge R\left(q-1,{k}_2+2\right)-R\left(q-1,{k}_2+1\right) $$

(C.4)

Since F_q is a monotonously increasing convex function and ${e}_{k_2+1}-{e}_{k_2}={e}_{k_2+2}-{e}_{k_2+1}$, the following inequality holds.

$$ {F}_q\left({e}_y-{e}_{k_2+1}\right)-{F}_q\left({e}_y-{e}_{k_2+2}\right)>{F}_q\left({e}_y-{e}_{k_2}\right)-{F}_q\left({e}_y-{e}_{k_2+1}\right) $$

(C.5)

Equations C.3, C.4, and C.5 lead to the following inequality

$$ R\left(q-1,{k}_2+2\right)-R\left(q-1,{k}_2+1\right)<{F}_q\left({e}_y-{e}_{k_2+1}\right)-{F}_q\left({e}_y-{e}_{k_2+2}\right) $$

(C.6)

The inequality in Eq. C.6 implies the following inequality.

$$ R\left(q-1,{k}_2+1\right)+{F}_q\left({e}_y-{e}_{k_2+1}\right)>R\left(q-1,{k}_2+2\right)+{F}_q\left({e}_y-{e}_{k_2+1}\right) $$

(C.7)

The inequality in Eq. C.7 suggests that Eq. C.2 also holds when v = k₂ + 1. Equation C.2 thus holds for any integer v that satisfiesw ≤ v < y. This fact implies that and the search can stop at w and w = T(q, y). The theorem thus follows.