Multidimensional gain control in image representation and processing in vision

Abstract

A generic model of automatic gain control (AGC) is proposed as a general framework for multidimensional automatic contrast sensitivity adjustment in vision, as well as in other sensory modalities. We show that a generic feedback AGC mechanism, incorporating a nonlinear synaptic interaction into the feedback loop of a neural network, can enhance and emphasize important image attributes, such as curvature, size, depth, convexity/concavity and more, similar to its role in the adjustment of photoreceptors and retinal network sensitivity over the extremely high dynamic range of environmental light intensities, while enhancing the contrast. We further propose that visual illusions, well established by psychophysical experiments, are a by-product of the multidimensional AGC. This hypothesis is supported by simulations implementing AGC, which reproduce psychophysical data regarding size contrast effects known as the Ebbinghaus illusion, and depth contrast effects. Processing of curvature by an AGC network illustrates that it is an important mechanism of image structure pre-emphasis, which thereby enhances saliency. It is argued that the generic neural network of AGC constitutes a universal, parsimonious, unified mechanism of neurobiological automatic contrast sensitivity control. This mechanism/model can account for a wide range of physiological and psychophysical phenomena, such as visual illusions and contour completion, in cases of occlusion, by a basic neural network. Likewise, and as important, biologically motivated AGC provides attractive new means for the development of intelligent computer vision systems.

Appendices

Appendix 1: Uniqueness of the AGC solution

We prove (based on Shefer 1979) that if a solution of the AGC model exists, it is unique, assuming $s(x)>0$. In fact, it is sufficient to prove uniqueness for the feedback signal, $f(x)$, derived from (3) and (4):

$$\begin{aligned} f(x)=\int \!\!s(x')\left[ \alpha -f(x')\right] W(x-x')dx'. \end{aligned}$$

(31)

We prove that if two bounded solutions of (31) exist, $f_{1} (x)$ and $f_{2} (x)$, they must be identical. To this end, we define the difference between the two assumed-to-exist solutions:

$$\begin{aligned} b(x){\mathop {\Delta }\limits _{=}} f_{2} (x)-f_{1} (x). \end{aligned}$$

(32)

$f_{1} (x)$ and $f_{2} (x)$ are bounded, and therefore, $b(x)$ is also bounded. We define its maximum value:

$$\begin{aligned} M_{b} {\mathop {\Delta }\limits _{=}} \mathop {\max }\limits _{x} \left| b(x)\right| . \end{aligned}$$

(33)

Substituting Eq. (31) in Eq. (32) we get:

$$\begin{aligned} b(x)&= \int \!\!s(x')\left[ \alpha -f_{2} (x')\right] W(x-x')dx'\nonumber \\&-\int \!\!s(x')\left[ \alpha -f_{1} (x')\right] W(x-x')dx' \nonumber \\&= \int \!\!s(x')\left[ -f_{2} (x')+f_{1} (x')\right] W(x-x')dx' \nonumber \\&= -\int \!\!s(x')b(x')W(x-x')dx'. \end{aligned}$$

(34)

Assuming that $s(x)$ is bounded, we define its maximum value:

$$\begin{aligned} M_{s} {\mathop {\Delta }\limits _{=}} \mathop {\max }\limits _{x} \left| s(x)\right| . \end{aligned}$$

(35)

From all the above we get:

$$\begin{aligned} M_{b}&= \mathop {\max }\limits _{x} \left| \int \!\!s(x')b(x')W(x-x')dx' \right| \nonumber \\&\le \mathop {\max }\limits _{x}\int \!\left| s(x')\right| \left| b(x')\right| \left| W(x-x')\right| dx'. \end{aligned}$$

(36)

Substituting Eqs. (33) and (35) in Eq. (36) we get:

$$\begin{aligned} M_{b}&\le \mathop {\max }\limits _{x} \int M_{s} M_{b} \left| W(x-x')\right| dx'\nonumber \\&= M_{s} M_{b} \mathop {\max }\limits _{x} \int \left| W(x-x')\right| dx'. \end{aligned}$$

(37)

Substituting Eq. (6) in Eq. (37) we get:

$$\begin{aligned} M_{b} \le M_{s} M_{b} \mathop {\max }\limits _{x} S_{W} =M_{s} M_{b} S_{W} \end{aligned}$$

(38)

and thus

$$\begin{aligned} M_{b} \le M_{s} M_{b} S_{W}. \end{aligned}$$

(39)

For $M_{s} S_{W} <1$, Eq. (39) is valid only if $M_{b} =0$, which implies that there is a unique solution for Eq. (31).

The meaning of $M_{s} S_{W} <1$ is the condition for unique solution given $s(x) \ge 0 \; \forall \, x$ given in (35), i.e., ${\mathop {\max }\limits _{x}}\{s(x)\} <1/S_W$.

If the assumption of $s(x)>0$ is not valid, and $s(x)$ assumes also negative values (as is the case for curvature values), the above proof is still valid, by taking the absolute value of the feedback: $f(x)\rightarrow \left| f(x)\right|$, in which case Eq. (31) becomes

$$\begin{aligned} f(x)=\int \!\!s(x')\left[ \alpha -\left| f(x')\right| \right] W(x-x')dx' , \end{aligned}$$

(40)

and the rest of the proof is unaffected.

Appendix 2: The curve-construction algorithm

The basic principle underlying drawing of a curve determined by its curvature information is based on Eq. (28) which approximates the curve by an arc of radius $\mathrm{R}$ (Fig. 23). This is a good approximation under the following assumptions.

Fig. 23

Curve-construction tool: variables’ definitions

1.1 Assumptions

Curvature, per definition, is a quadratic term. As such, the forward problem (calculating the curvature vector of a curve) is a well-posed problem and can be dealt with easily. On the other hand, the backward problem, which is drawing a curved line from its curvature information only, is an ill-posed problem, rendering it impossible to solve without making some assumptions. Further, the filtering process is a nonlinear necessitating additional assumptions. These assumptions are:

• 1. The curved line is discrete.

• 2. The line is smooth enough.

• 3. The starting point of the line is known and is not affected by filtering.

• 4. The tangent to the curve at the first point of the curve is known and does not change by the filtering.

• 5. The length of the original curve between each sequential point is known and does not change by filtering. Therefore, curve length is constant.

• 6. Curvature information is known (or given).

• 7. Curvature values are positive for counter clockwise (CCW) curve and negative for clockwise (CW) curve as defined in Fig. 24.

• 8. Centered coordinates are implemented, i.e., the point (0, 0) coincides with the center of the image.

Fig. 24

Definition of signed curvature definition: The curvature is defined as positive if the unit tangent rotates counter clockwise, and negative if it rotates clockwise

1.2 The algorithm

The algorithm is, as mentioned, based on approximating curve segments by circular segments. This is done by calculating for each point on the curve the center point of a circle that matches the point’s curvature and position. Then, an arc (part of a circle) is drawn in the same length of the original curve segment. Figure 23 shows this idea and the algorithm variables.

Given the curvature vector, the starting point and its tangent orientation, and the arcs lengths, one can draw the curved line following these steps (see Fig. 25 for summary):

Fig. 25

A flowchart of the curvature drawing-tool algorithm

• 1. Calculate the correct center point “CC” (see Fig. 23) is the center point of a circle that match the curvature at CP according to Eq. (28). This point must satisfy two equations, as follows:

  • Radius connecting CC and CP is perpendicular to the tangent at CP,

    $$\begin{aligned} \overrightarrow{\mathrm{CC} - \mathrm{CP}} \cdot \overrightarrow{\mathrm{cl'} (\mathrm{CP})} = 0 , \end{aligned}$$

    (41)

    where $\mathrm{cl'} (\mathrm{CP})$ is the derivative of the curved line $\mathrm{cl}$ at $\mathrm{CP}$.

  • Curvature can be represented locally by a circle with radius length equal to $1/\kappa$:

    $$\begin{aligned} (\mathrm{CC}_x - \mathrm{CP}_x)^2 + (\mathrm{CC}_y - \mathrm{CP}_y)^2 = (1/\kappa )^2 = R^2 \end{aligned}$$

    (42)

    The above equations have two solutions for center points. One possible solution fits the positive curvature, and the other one fits the negative curvature. The correct point is thus chosen according to the curvature sign.

  For negative curvatures (as the one in Fig. 23), the correct point is to the right of the tangent vector and to the left for positive curvatures.

• 2. Calculating the next-point position (NP) Calculate the next point according to an arc that has the following characteristic:

  • Starts from the current point. Has radius equal to $1/\kappa$.

  • Is an arc belonging to a circle with central point CC, calculated in the previous step.

  • Has the same length as the original segment. (This is equivalent to saying that to the arc corresponds an angle $\theta$).

• 3. Calculating the next-point tangent

  • Calculate $\theta$ according to the following equation:

    $$\begin{aligned} \theta = l/R = l\cdot \kappa , \end{aligned}$$

    where $l$ is the length of the arc connecting CP and NP.

  • Calculate the angular orientation of the current tangent.

  • Calculate the orientation of the tangent at NP with reference to the one at CP:

    $$\begin{aligned} \sphericalangle \mathrm{cl'} (\mathrm{NP}) = \sphericalangle \mathrm{cl'} (\mathrm{CP}) + \theta \end{aligned}$$

  • For (curvature $< 0$): $\sphericalangle \mathrm{cl'} (\mathrm{NP}) = \sphericalangle \mathrm{cl'} (\mathrm{CP}) - \theta$

  • Calculate $(X_1 , Y_1)$ according to the point of intercept of the current tangent and the subsequent point tanget.

  • Calculate the next-point tangent by subtracting $(X_1 , Y_1)$ from NP.

Note:

• 1. In case the curvature is equal to 0:

  • Next-point tangent is equal to current tangent.

  • Next point is calculated by taking the original segment length along the direction of current tangent.

• 2. Calculating $(X_1,Y_1)$ is necessary (although the next-point slope is known) because of the fact that the slope orientation is not known (it is the output of an arctan function). As noted above, the orientation of the tangent is important in determining the correct CP.