Skip to main content
Springer Nature Link
Log in

Border Ownership, Category Selectivity and Beyond

  • Conference paper
  • First Online:
Advances in Visual Computing (ISVC 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13599))

Included in the following conference series:

Abstract

Object segmentation is a fundamental problem for both biological and machine vision systems. Recent advances in deep learning have allowed significant progress in terms of the ability of machine vision systems to carry out object segmentation, but this work has ignored a key piece of information that the biological vision system uses for segmentation: border ownership, or the determination for a given edge of which side the object is that owns it. Here we present a method for determining border ownership using a deep neural network model. Additionally, the model learns selectivity for object categories, suggesting a potential relationship between border ownership information and object category-selectivity. Our model may serve as a basic building block for machine vision systems that aim to reproduce the robustness of biological vision systems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Kirillov, A., He, K., Girshick, R., Rother, C., Dollár, P.: Panoptic Segmentation. arXiv:1801.00868v3 [cs.CV] (10 Apr. 2019). https://www.cityscapes-dataset.com/

  2. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. Communications of the ACM 60(6). https://doi.org/10.1145/3065386

  3. Dedieu, A., Rikhye, RV., Gredilla, M.L., George, D.: Learning attention-controllable border-ownership for objectness inference and binding. bioRxiv, https://doi.org/10.1101/2020.12.31.424926

  4. Buades, A., Grompone von Gioi, R., Navarro, J.: Joint Contours, Corner and T-Junction Detection: An Approach Inspired by the Mammal Visual System. Journal of Mathematical Imaging and Vision 60(3), 341–354 (2017). https://doi.org/10.1007/s10851-017-0763-z

    Article  MathSciNet  MATH  Google Scholar 

  5. Vaswani, A., et al.: Attention is All You Need. arXiv:1706.03762v5 [cs.CL] (6 Dec. 2017)

  6. Lee, C-H., Liu, Z., Wu, L., Luo, P.: MaskGAN: Towards Diverse and Interactive Facial Image Manipulation. arXiv:1907.11922 [cs.CV] (1 Apr. 2020). https://github.com/switchablenorms/CelebAMask-HQ

  7. Szegedy, C., et al.: Going deeper with convolutions. arXiv:1409.4842v1 [cs.CV]

  8. George, D., et al.: A generative vision model that trains with high data efficiency and breaks text-based CAPTCHAs. Science (2017). https://doi.org/10.1126/science.aag2612

  9. llg, E., Mayer, N., Saikia, T., Keuper, M., Dosovitskiy, A., Brox, T.: FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks. arXiv:1612.01925 [cs.CV] (6 Dec. 2016). https://github.com/NVIDIA/flownet2-pytorch

  10. Craft, E., Schütze, H., Niebur, E., von der Heydt, R.: A neural model of figure-ground organization. J Neurophysiol 97, 4310–4326 (2007). https://doi.org/10.1152/jn.00203.2007

    Article  Google Scholar 

  11. Qiu, F.T., Sugihara, T., von der Heydt, R.: Figure-ground mechanisms provide structure for selective attention. Nat Neurosci. 10(11), 1492-1499 (November 2007). https://doi.org/10.1038/nn1989

  12. Girard, P., Hupe, J.M., Bullier, J.: Feedforward and Feedback Connections Between Areas V1 and V2 of the Monkey Have Similar Rapid Conduction Velocities. J. Neurophysiol 85, 1328–1331

    Google Scholar 

  13. Ko, H.-K., von der Heydt, R.: Figure-ground organization in the visual cortex: does meaning matter? Articles in Press. J Neurophysiology (4 Oct 2017). https://doi.org/10.1152/jn.00131.2017

  14. Jehee, J.F.M., Lamme, V.A.F., Roelfsema, P.R.: Boundary assignment in a recurrent network architecture. Vision Research 47(9), 1153–1165. https://doi.org/10.1016/j.visres.2006.12.018

  15. He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask RCNN. arXiv:1703.06870v3 [cs.CV] (24 Jan 2018)

  16. He, K., Zhang, X., Ren, S., Sun, J.: Deep Residual Learning for Image Recognition. arXiv:1512.03385v1 [cs.CV] (10 Dec. 2015)

  17. Wrenninge, M., Unger, J.: Synscapes: A Photorealistic Synthetic Dataset for Street Scene Parsing. arXiv:1810.08705 [cs.CV] (19 Oct. 2018). https://7dlabs.com/Synscapes-overview

  18. Minervini, M., Fischbach, A., Scharr, H., Tsaftaris, S.A.: Finely-granted annotated datasets for image-based plant phenotyping. Pattern Recognition Letters. https://doi.org/10.1016/j.patrec.2015.10.013; https://www.plant-phenotyping.org/datasets-home

  19. Aubry, M., Maturana, D., Efros, A.A., Russell, B.C., Sivic, J.: Seeing 3D chairs: exemplar part-based 2D-3D alignment using a large dataset of CAD models. FlyingChair dataset: https://lmb.informatik.uni-freiburg.de/resources/datasets/GenerateChairs.en.html

  20. Ronneberger, O., Fischer, P., Brox, T.: U-Net: Convolutional Networks for Biomedical Image Segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    Chapter  Google Scholar 

  21. Sundberg, P., Brox, T., Maire, M., Arbelaez, P., Malik, J.: Occlusion boundary detection and figure/ground assignment from optic flow. CVPR 2011, 2233–2240 (2011). https://doi.org/10.1109/CVPR.2011.5995364

    Article  Google Scholar 

  22. Pinheiro, P.O., Lin, T.-Y., Collobert, R., Dollár, P.: Learning to Refine Object Segments. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 75–91. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_5

    Chapter  Google Scholar 

  23. Bao, P., Tsao, D.Y.: Representation of multiple objects in macaque category-selective areas. Nature Communications 9, 1774 (2018). https://doi.org/10.1038/s41467-018-04126-7

  24. Bao, P., She, L., McGill, M., Tsao, D.Y.: A map of object space in primate inferotemporal cortex. Nature Article, published 17 June 2020. https://doi.org/10.1038/s41586-020-2350-5

  25. von der Heydt, R.: Figure-ground organization and the emergence of proto-objects in the visual cortex. Front. Phychol. 6, 1695. https://doi.org/10.3389/fpsyg.2015.01695

  26. von der Heydt, R., Zhang, N.R.: Figure and ground: how the visual cortex integrates local cues for global organization. J Neurophysiol 120, 3085–3098 (2018). published 25 July 2018. https://doi.org/10.1152/jn00125.2018

  27. Zhu, S.D., Zhang, L.A., von der Heydt, R.: Searching for object pointers in the visual cortex. JNP Journal of Neurophysiology (11 May 2020). https://doi.org/10.1152/jn.00112.2020

  28. Wu, Z., et al.: 3D ShapeNets: A Deep Representation for Volumetric Shapes. arXiv:1406.5670v3 [cs.CV] (15 Apr. 2015). https://modelnet.cs.princeton.edu/

  29. Chen, T., Cheng, X., Tsao, T.: Supplementary Materials for: Border-ownership, Category-selectivity, and Beyond. http://opticarraytech.com/Supplementary-Border-Ownership-Category-Selectivity-and-Beyond.v4.2.pdf

Download references

Acknowledgment

We thank Albert Tsao very much for his careful review and editorial suggestions that greatly improved the manuscript. The inventions described here are protected by US patent 11,282,293.

Author information

Authors and Affiliations

  1. OpticArray Technologies, Inc, 1917 Dundee Road, Rockville, MD, 20850, USA

    Tianlong Chen, Xuemei Cheng & Thomas Tsao

Authors
  1. Tianlong Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xuemei Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thomas Tsao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tianlong Chen .

Editor information

Editors and Affiliations

  1. University of Nevada, Reno, NV, USA

    George Bebis

  2. University of Illinois Urbana-Champaign, Urbana, IL, USA

    Bo Li

  3. National University of Singapore, Singapore, Singapore

    Angela Yao

  4. Microsoft Research Asia, Beijing, China

    Yang Liu

  5. University of Missouri, Columbia, MO, USA

    Ye Duan

  6. City University of Hong Kong, Kowloon, Hong Kong

    Manfred Lau

  7. Idaho National Laboratory, Idaho Falls, ID, USA

    Rajiv Khadka

  8. Salesforce, Seattle, WA, USA

    Ana Crisan

  9. Tufts University, Medford, MA, USA

    Remco Chang

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Chen, T., Cheng, X., Tsao, T. (2022). Border Ownership, Category Selectivity and Beyond. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2022. Lecture Notes in Computer Science, vol 13599. Springer, Cham. https://doi.org/10.1007/978-3-031-20716-7_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-20716-7_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-20715-0

  • Online ISBN: 978-3-031-20716-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics