A border-ownership model based on computational electromagnetism

Abstract

The mathematical relation between a vector electric field and its corresponding scalar potential field is useful to formulate computational problems of lower/middle-order visual processing, specifically related to the assignment of borders to the side of the object: so-called border ownership (BO). BO coding is a key process for extracting the objects from the background, allowing one to organize a cluttered scene. We propose that the problem is solvable simultaneously by application of a theorem of electromagnetism, i.e., “conservative vector fields have zero rotation, or “curl.” We hypothesize that (i) the BO signal is definable as a vector electric field with arrowheads pointing to the inner side of perceived objects, and (ii) its corresponding scalar field carries information related to perceived order in depth of occluding/occluded objects. A simple model was developed based on this computational theory. Model results qualitatively agree with object-side selectivity of BO-coding neurons, and with perceptions of object order. The model update rule can be reproduced as a plausible neural network that presents new interpretations of existing physiological results. Results of this study also suggest that T-junction detectors are unnecessary to calculate depth order.

Introduction

Humans can perceive objects and their relative order from a cluttered scene filled with objects that are mutually occlusive. In the real world, when one object overlaps another, the border one sees between the two objects is attributed to the occluding object. Assigning borders to the perceived object is believed to be a key process of perception; it facilitates distinction of objects from the background (Nakayama, Shimojo, & Silverman, 1989). This process is called border ownership (BO) coding. One can consider an ambiguous figure that can be perceived as either a vase or two faces (Fig. 1 a). In the perception of a vase, the border is assigned to the inner white area, which is perceived as closer to the viewer (Fig. 1 b). Similarly, in the perception of two faces, the border is assigned to the outer black area, which is perceived as closer to the viewer (Fig. 1 c). This example illustrates how BO coding and the perception of depth order are closely related.

Neurophysiological experiments using simple shaded stimuli on areas V1, V2, and V4 in the visual cortex of monkeys revealed the existence of neurons which respond more strongly to an edge when the “owner” of the edge is located on a specific side (Zhou, Friedman, & von der Heydt, 2000). Spatial selectivity for the owner’s location was found to emerge in neurons of area V2 in the visual cortex. Border ownership (BO) is believed to be coded by calculating the difference in response of these sets of neurons with opposite object-side preference towards a stimulus.

Various models have been proposed to code BO as neural networks in area V2 Craft et al. (2007), Li (2005), Sakai et al. (2012). The model explained by Li (2005) requires more than 20 free parameters to function well. Craft and his colleagues suggest hypothetical “grouping cells”, of which the receptive fields have an annular distribution (Craft et al., 2007). The model produced by Sakai et al. relies on randomly generated feedforward neural connections (Sakai et al., 2012). These studies reproduce the responses of BO-related neurons, but they are fundamentally flawed in their approach to elucidating visual systems from computational viewpoints. It is quite difficult to elucidate what sort of mathematical problem those models are intended to solve.

No mathematical formulation of the BO problem has been proposed to date. However, many problems of early vision can be formulated using regularization theory (Poggio, Torre, & Koch, 1985). Neural network models to solve these problems are drawn from the theory. Similarly, if the BO coding problem is expressed in a mathematically well-defined fashion, then one might be able to deduce a neural network model for the mathematical problem, and in turn understand the characteristics of the neural connections involved. Consequently, we might also, for example, reduce the number of parameters in Li’s model.

A fundamental question must be considered: what is the visual system trying to achieve through BO coding? As shown in Fig. 1, BO coding enables the visual cortex to calculate a rough reproduction of the outer world information, which includes depth order in addition to observable two-dimensional (2D) information such as surface brightness and edges. A further study of neurons in V2 revealed the existence of neurons that respond selectively to the edges of objects with actual stereoscopic depth information (Qiu & von der Heydt, 2005). Some of these neurons show selectivity to both 2D figures (owners) as well as three-dimensional (3D) figures placed on a certain side of their receptive field. This result reinforces the idea that area V2 might be involved in the process of a rough reproduction of a 3D image, and that BO coding and depth coding are interdependent processes.

As described in this paper, we attempt to overcome the flaws of current studies by formulating BO coding as a well-defined mathematical problem and subsequently solving a middle-level visual task: estimation of the depth order of objects from 2D images. We hope that formulation and solutions related to BO coding and order estimation presented in this paper will serve as the basis for a comprehensive model that can aid in the elucidation of low/middle-level visual processing. The remainder of this paper is organized as follows: Section 2 presents discussion of the key ideas underpinning our theory and proposes a neural network model for BO coding. Section 3 presents results of numerical simulations of our model. Section 4 emphasizes some important findings derived from simulation results, including the deduced neural connections. Section 5 is the conclusion.