Alexander White
ABSTRACT
Contextual grouping mechanisms in early visual cortex are thought to aid in perception of ambiguous textures, including partially obscured targets under real-world high dynamic range (HDR) luminance. Yet, deep neural networks struggle with naturalistic obscuration and illumination while requiring millions of neurons and power-hungry GPUs for processing. We hypothesized that contextual grouping mechanisms for edge and luminance processing may aid in localization of targets under natural obscuration and illumination. To address this issue, we developed a novel small (< 10,000 neurons) spiking neural network (SNN) that uses spike time correlations to leverage the combined luminance and orientation similarity of nearby image regions for image pre-processing, to support downstream deep neural network (DNN) target localization. The network has leaky integrate-and-fire neurons with current based (CuBa) synapses and is simulated using the Nengo LOIHI API, with potential application via Intel's LOIHI neuromorphic hardware. We collected 89 HDR images of a target dummy in a heavily wooded environment under varying occlusion and illumination. We used SNN preprocessing to adjust local image contrast based on the grouping mechanism, followed by a DNN classifier (Detectron2) to localize the target. Results show that a small SNN for image preprocessing can aid image segmentation and localization of occluded targets, marking an initial step towards more efficient and accurate target recognition under natural illumination and occlusion.
- Hafner, D., Demetz, O. and Weickert, J. Simultaneous HDR and optic flow computation. IEEE, City, 2014.Google Scholar
Digital Library
- Lazar, A. A., Ukani, N. H. and Zhou, Y. Sparse identification of contrast gain control in the fruit fly photoreceptor and amacrine cell layer. The Journal of Mathematical Neuroscience, 10, 1 (2020), 1--35.Google ScholarCross Ref
- Albright, T. D. and Stoner, G. R. Contextual influences on visual processing. Annual review of neuroscience, 25, 1 (2002), 339--379.Google Scholar
- Gilbert, C., Ito, M., Kapadia, M. and Westheimer, G. Interactions between attention, context and learning in primary visual cortex. Vision research, 40, 10--12 (2000), 1217--1226.Google Scholar
- Adelson, E. H. Lightness Perception and Lightness Illusions. MIT Press, City, 2000.Google Scholar
- Chen, C.-C. and Tyler, C. W. Lateral modulation of contrast discrimination: Flanker orientation effects. Journal of Vision, 2, 6 (2002), 8--8.Google ScholarCross Ref
- Cass, J. R. and Spehar, B. Dynamics of collinear contrast facilitation are consistent with long-range horizontal striate transmission. Vision research, 45, 21 (2005), 2728--2739.Google Scholar
- Hung, C. P., Callahan-Flintoft, C., Fedele, P. D., Fluitt, K. F., Odoemene, O., Walker, A. J., Harrison, A. V., Vaughan, B. D., Jaswa, M. S. and Wei, M. Abrupt darkening under high dynamic range (HDR) luminance invokes facilitation for high-contrast targets and grouping by luminance similarity. Journal of Vision, 20, 7 (2020), 9--9.Google ScholarCross Ref
- Hung, C. P., Ramsden, B. M. and Roe, A. W. A functional circuitry for edge-induced brightness perception. Nature neuroscience, 10, 9 (2007), 1185--1190.Google Scholar
- Zemach, I. K. and Rudd, M. E. Effects of surround articulation on lightness depend on the spatial arrangement of the articulated region. JOSA A, 24, 7 (2007), 1830--1841.Google ScholarCross Ref
- Allred, S. R. and Brainard, D. H. A Bayesian model of lightness perception that incorporates spatial variation in the illumination. Journal of Vision, 13, 7 (2013), 18--18.Google ScholarCross Ref
- Allred, S. R., Radonjić, A., Gilchrist, A. L. and Brainard, D. H. Lightness perception in high dynamic range images: Local and remote luminance effects. Journal of vision, 12, 2 (2012), 7--7.Google Scholar
- Radonjić, A., Allred, S. R., Gilchrist, A. L. and Brainard, D. H. The dynamic range of human lightness perception. Current Biology, 21, 22 (2011), 1931--1936.Google ScholarCross Ref
- White, A. J., Hung, C. P. and Lo, C.-C. HDR luminance normalization via contextual facilitation in highly recurrent spiking neural networks. SPIE, City, 2022.Google ScholarCross Ref
- Wang, X.-J. Decision making in recurrent neuronal circuits. Neuron, 60, 2 (2008), 215--234.Google ScholarCross Ref
- Kogo, N., Kern, F. B., Nowotny, T., van Ee, R., van Wezel, R. and Aihara, T. Dynamics of a mutual inhibition circuit between pyramidal neurons compared to human perceptual competition. Journal of Neuroscience, 41, 6 (2021), 1251--1264.Google ScholarCross Ref
- Koyama, M. and Pujala, A. Mutual inhibition of lateral inhibition: a network motif for an elementary computation in the brain. Current opinion in neurobiology, 49 (2018), 69--74.Google Scholar
- Lin, Y.-S., Chen, C.-C. and Greenlee, M. W. The role of lateral modulation in orientation-specific adaptation effect. Journal of Vision, 22, 2 (2022), 13--13.Google ScholarCross Ref
- Liu, B., White, A. J. and Lo, C.-C. Augmenting Flexibility: Mutual Inhibition Between Inhibitory Neurons Expands Functional Diversity. bioRxiv (2020), 2020.2011.2008.371179.Google Scholar
- Wang, X.-J. and Yang, G. R. A disinhibitory circuit motif and flexible information routing in the brain. Current opinion in neurobiology, 49 (2018), 75--83.Google Scholar
- Cavanaugh, J. R., Bair, W. and Movshon, J. A. Nature and interaction of signals from the receptive field center and surround in macaque V1 neurons. Journal of neurophysiology, 88, 5 (2002), 2530--2546.Google Scholar
- Bekolay, T., Bergstra, J., Hunsberger, E., DeWolf, T., Stewart, T. C., Rasmussen, D., Choo, X., Voelker, A. R. and Eliasmith, C. Nengo: a Python tool for building large-scale functional brain models. Frontiers in neuroinformatics, 7 (2014), 48.Google Scholar
- Davies, M., Srinivasa, N., Lin, T.-H., Chinya, G., Cao, Y., Choday, S. H., Dimou, G., Joshi, P., Imam, N. and Jain, S. Loihi: A neuromorphic manycore processor with on-chip learning. Ieee Micro, 38, 1 (2018), 82--99.Google ScholarCross Ref
- Golomb, D. and Rinzel, J. Dynamics of globally coupled inhibitory neurons with heterogeneity. Physical review E, 48, 6 (1993), 4810.Google Scholar
- Wolfe, J. M. and Utochkin, I. S. What is a preattentive feature? Current opinion in psychology, 29 (2019), 19--26.Google Scholar
- Chu, C. C., Chien, P. F. and Hung, C. P. Tuning dissimilarity explains short distance decline of spontaneous spike correlation in macaque V1. Vision Research, 96 (2014), 113--132.Google ScholarCross Ref
- Li, S., Liu, N., Zhang, X., McLaughlin, D. W., Zhou, D. and Cai, D. Dendritic computations captured by an effective point neuron model. Proceedings of the National Academy of Sciences, 116, 30 (2019), 15244--15252.Google Scholar
- Zucker, S. W. Early orientation selection: Tangent fields and the dimensionality of their support. Computer Vision, Graphics, and Image Processing, 32, 1 (1985), 74--103.Google Scholar
- Prins, N. and Kingdom, F. A. Applying the model-comparison approach to test specific research hypotheses in psychophysical research using the Palamedes toolbox. Frontiers in psychology, 9 (2018), 1250.Google Scholar
- Yeh, C.-I., Xing, D. and Shapley, R. M. "Black" responses dominate macaque primary visual cortex v1. Journal of Neuroscience, 29, 38 (2009), 11753--11760.Google ScholarCross Ref
- Chin, A. L., Lin, C. Y., Fu, T. F., Dickson, B. J. and Chiang, A. S. Diversity and wiring variability of visual local neurons in the Drosophila medulla M6 stratum. Journal of Comparative Neurology, 522, 17 (2014), 3795--3816.Google Scholar
- Gonzalez, R. C. and Woods, R. E. Thresholding. Digital image processing (2002), 595--611.Google ScholarDigital Library
Index Terms
- Neuromorphic luminance-edge contextual preprocessing of naturally obscured targets
Recommendations
An analog astrocyte---neuron interaction circuit for neuromorphic applications
Recent neurophysiologic findings have shown that astrocytes (the most abundant type of glial cells) are active partners in neural information processing and regulate the synaptic transmission dynamically. Motivated by these findings, in the present ...
Neuromorphic Synapses for Artificial Dendrites
We describe neuromorphic, variable-weight synapses on artificial dendrites that facilitate experimentation with correlative adaptation rules. Attention is focused on those aspects of biological synaptic function that could affect a neuromorphic network‘s ...
Implementation of an elaborated neuromorphic model of a biological photoreceptor
We describe here an elaborated neuromorphic model based on the photoreceptors of flies and realised in both software simulation and hardware using discrete circuit components. The design of the model is based on optimisations and further elaborations to ...
Comments