Skip to main content
SpringerLink
Log in

Polarized Optical-Flow Gyroscope

  • Conference paper
  • First Online:
Computer Vision – ECCV 2020 (ECCV 2020)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12361))

Included in the following conference series:

Abstract

We merge by generalization two principles of passive optical sensing of motion. One is common spatially resolved imaging, where motion induces temporal readout changes at high-contrast spatial features, as used in traditional optical-flow. The other is the polarization compass, where axial rotation induces temporal readout changes due to the change of incoming polarization angle, relative to the camera frame. The latter has traditionally been modeled for uniform objects. This merger generalizes the brightness constancy assumption and optical-flow, to handle polarization. It also generalizes the polarization compass concept to handle arbitrarily textured objects. This way, scene regions having partial polarization contribute to motion estimation, irrespective of their texture and non-uniformity. As an application, we derive and demonstrate passive sensing of differential ego-rotation around the camera optical axis.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The maps of each polarization variable are presented in the Supplementary Material.

References

  1. Technology Image Sensor: Polarization Products Sony Semiconductor Solutions Group. https://www.sony-semicon.co.jp/e/products/IS/polarization/technology.html. Accessed 16 July 2020

  2. Agrawal, P., Ratnoo, A., Ghose, D.: Inverse optical flow based guidance for UAV navigation through urban canyons. Aerosp. Sci. Technol. 68, 163–178 (2017)

    Google Scholar 

  3. Barta, A., Horváth, G.: Underwater binocular imaging of aerial objects versus the position of eyes relative to the flat water surface. JOSA A 20(12), 2370 (2003)

    Google Scholar 

  4. Ben-Ezra, M.: Segmentation with invisible keying signal. In: CVPR, pp. 32–37. IEEE (2000)

    Google Scholar 

  5. Berezhnyy, I., Dogariu, A.: Time-resolved Mueller matrix imaging polarimetry. Opt. Express 12(19), 4635–4649 (2004)

    Google Scholar 

  6. Bomzon, Z., Biener, G., Kleiner, V., Hasman, E.: Spatial Fourier-transform polarimetry using space-variant subwavelength metal-stripe polarizers. Opt. Lett. 26(21), 1711–1713 (2001)

    Google Scholar 

  7. Brines, M.L.: Dynamic patterns of skylight polarization as clock and compass. J. Theor. Biol. 86(3), 507–512 (1980)

    Google Scholar 

  8. Brines, M.L., Gould, J.L.: Skylight polarization patterns and animal orientation. J. Exp. Biol. 96(1), 69–91 (1982)

    Google Scholar 

  9. Carey, S.J., Barr, D.R., Dudek, P.: Low power high-performance smart camera system based on SCAMP vision sensor. J. Syst. Architect. 59(10), 889–899 (2013)

    Google Scholar 

  10. Chahl, J.S., Srinivasan, M.V., Zhang, S.W.: Landing strategies in honeybees and applications to uninhabited airborne vehicles. Int. J. Rob. Res. 23(2), 101–110 (2004)

    Google Scholar 

  11. Collin, C., Pattanaik, S., LiKamWa, P., Bouatouch, K.: Computation of polarized subsurface BRDF for rendering. In: Proceedings of Graphics Interface, pp. 201–208. Canadian Information Processing Society (2014)

    Google Scholar 

  12. Courroux, S., Chevobbe, S., Darouich, M., Paindavoine, M.: Use of wavelet for image processing in smart cameras with low hardware resources. J. Syst. Architect. 59(10), 826–832 (2013)

    Google Scholar 

  13. Cronin, T.W.: Polarization vision and its role in biological signaling. Integr. Comp. Biol. 43(4), 549–558 (2003)

    Google Scholar 

  14. Dahlberg, A.R., Pust, N.J., Shaw, J.A.: Effects of surface reflectance on skylight polarization measurements at the Mauna Loa Observatory. Opt. Express 19(17), 16008–16021 (2011)

    Google Scholar 

  15. Diamant, Y., Schechner, Y.Y.: Overcoming visual reverberations. In: CVPR, pp. 1–8. IEEE (2008)

    Google Scholar 

  16. Diner, D.J., et al.: First results from a dual photoelastic-modulator-based polarimetric camera. Appl. Opt. 49(15), 2929–2946 (2010)

    Google Scholar 

  17. Dupeyroux, J., Diperi, J., Boyron, M., Viollet, S., Serres, J.: A novel insect-inspired optical compass sensor for a hexapod walking robot. In: IROS, pp. 3439–3445. IEEE/RSJ (2017)

    Google Scholar 

  18. Dupeyroux, J., Serres, J., Viollet, S.: A hexapod walking robot mimicking navigation strategies of desert ants cataglyphis. In: Vouloutsi, V., et al. (eds.) Living Machines 2018. LNCS (LNAI), vol. 10928, pp. 145–156. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-95972-6_16

    Chapter  Google Scholar 

  19. Dupeyroux, J., Viollet, S., Serres, J.R.: An ant-inspired celestial compass applied to autonomous outdoor robot navigation. Rob. Auton. Syst. 117, 40–56 (2019)

    Google Scholar 

  20. Evangelista, C., Kraft, P., Dacke, M., Labhart, T., Srinivasan, M.: Honeybee navigation: critically examining the role of the polarization compass. Philos. Trans. Roy. Soc. B Biol. Sci. 369(1636), 20130037 (2014)

    Google Scholar 

  21. Fan, C., Hu, X., Lian, J., Zhang, L., He, X.: Design and calibration of a novel camera-based bio-inspired polarization navigation sensor. IEEE Sens. J. 16(10), 3640–3648 (2016)

    Google Scholar 

  22. Farid, H., Adelson, E.H.: Separating reflections from images by use of independent component analysis. JOSA A 16(9), 2136–2145 (1999)

    Google Scholar 

  23. Fernandes, J., Postula, A., Srinivasan, M., Thurrowgood, S.: Insect inspired vision for micro aerial vehicle navigation. In: ACRA, pp. 1–8. ARAA (2011)

    Google Scholar 

  24. Foster, J.J., Sharkey, C.R., Gaworska, A.V., Roberts, N.W., Whitney, H.M., Partridge, J.C.: Bumblebees learn polarization patterns. Curr. Biol. 24(12), 1415–1420 (2014)

    Google Scholar 

  25. Garcia, M., Edmiston, C., Marinov, R., Vail, A., Gruev, V.: Bio-inspired color-polarization imager for real-time in situ imaging. Optica 4(10), 1263 (2017)

    Google Scholar 

  26. Gill, P.R., Kellam, M., Tringali, J., Vogelsang, T., Erickson, E., Stork, D.G.: Computational diffractive imager with low-power image change detection. In: Computational Optical Sensing and Imaging, pp. CM3E-2. OSA (2015)

    Google Scholar 

  27. Goddard, S.M., Forward, R.B.: The role of the underwater polarized light pattern, in sun compass navigation of the grass shrimp, Palaemonetes Vulgaris. J. Compa. Physiol. A Neuroethol. Sens. Neural Behavi. Physiol. 169(4), 479–491 (1991)

    Google Scholar 

  28. Gruev, V., Van der Spiegel, J., Engheta, N.: Dual-tier thin film polymer polarization imaging sensor. Opt. Express 18(18), 19292–19303 (2010)

    Google Scholar 

  29. Guan, L., Liu, S., Qi Li, S., Lin, W., Yuan Zhai, L., Kui Chu, J.: Study on polarized optical flow algorithm for imaging bionic polarization navigation micro sensor. Optoelectron. Lett. 14(3), 220–225 (2018)

    Google Scholar 

  30. Guan, L., et al.: Study on displacement estimation in low illumination environment through polarized contrast-enhanced optical flow method for polarization navigation applications. Optik 210, 164513 (2020)

    Google Scholar 

  31. Guarnera, G.C., Peers, P., Debevec, P., Ghosh, A.: Estimating surface normals from spherical stokes reflectance fields. In: Fusiello, A., Murino, V., Cucchiara, R. (eds.) ECCV 2012. LNCS, vol. 7584, pp. 340–349. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33868-7_34

    Chapter  Google Scholar 

  32. Guo, Q., et al.: Compact single-shot metalens depth sensors inspired by eyes of jumping spiders. Proc. Natl. Acad. Sci. 116(46), 22959–22965 (2019)

    Google Scholar 

  33. Gupta, M., Narasimhan, S.G., Schechner, Y.Y.: On controlling light transport in poor visibility environments. In: CVPR, pp. 1–8. IEEE (2008)

    Google Scholar 

  34. Gutierrez, D., Narasimhan, S.G., Jensen, H.W., Jarosz, W.: Scattering. In: ACM Siggraph Asia Courses, p. 18 (2008)

    Google Scholar 

  35. Hooper, B.A., Baxter, B., Piotrowski, C., Williams, J., Dugan, J.: An airborne imaging multispectral polarimeter. In: Oceans, vol. 2, p. 7. IEEE/MTS (2009)

    Google Scholar 

  36. Horisaki, R., Choi, K., Hahn, J., Tanida, J., Brady, D.J.: Generalized sampling using a compound-eye imaging system for multi-dimensional object acquisition. Opt. Express 18(18), 19367–19378 (2010)

    Google Scholar 

  37. Horstmeyer, R., Euliss, G., Athale, R., Levoy, M.: Flexible multimodal camera using a light field architecture. In: ICCP, pp. 1–8. IEEE (2009)

    Google Scholar 

  38. Horváth, G.: Polarized Light and Polarization Vision in Animal Sciences, vol. 2. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-642-54718-8

    Book  Google Scholar 

  39. Ihrke, I., Kutulakos, K.N., Lensch, H.P., Magnor, M., Heidrich, W.: State of the art in transparent and specular object reconstruction. In: Eurographics STAR - State of the Art Report (2008)

    Google Scholar 

  40. Joos, F., Buenzli, E., Schmid, H.M., Thalmann, C.: Reduction of polarimetric data using Mueller calculus applied to Nasmyth instruments. In: Observatory Operations: Strategies, Processes, and Systems II, vol. 7016, p. 70161I. SPIE (2008)

    Google Scholar 

  41. Kadambi, A., Taamazyan, V., Shi, B., Raskar, R.: Polarized 3D: high-quality depth sensing with polarization cues. In: ICCV, pp. 3370–3378. IEEE (2015)

    Google Scholar 

  42. Kadambi, A., Taamazyan, V., Shi, B., Raskar, R.: Depth sensing using geometrically constrained polarization normals. Int. J. Comput. Vision 125(1–3), 34–51 (2017)

    MathSciNet  Google Scholar 

  43. Kaftory, R., Schechner, Y.Y., Zeevi, Y.Y.: Variational distance-dependent image restoration. In: CVPR, pp. 1–8. IEEE (2007)

    Google Scholar 

  44. Kattawar, G.W., Adams, C.N.: Stokes vector calculations of the submarine light field in an atmosphere-ocean with scattering according to a Rayleigh phase matrix: effect of interface refractive index on radiance and polarization. Limnol. Oceanogr. 34(8), 1453–1472 (1989)

    Google Scholar 

  45. Kong, N., Tai, Y.W., Shin, J.S.: A physically-based approach to reflection separation: from physical modeling to constrained optimization. IEEE Trans. Pattern Anal. Mach. Intell. 36(2), 209–221 (2013)

    Google Scholar 

  46. Lanman, D., Wetzstein, G., Hirsch, M., Heidrich, W., Raskar, R.: Polarization fields: dynamic light field display using multi-layer LCDs. ACM Trans. Graph. 30, 186 (2011)

    Google Scholar 

  47. Lerner, A., Sabbah, S., Erlick, C., Shashar, N.: Navigation by light polarization in clear and turbid waters. Philos. Trans. Roy. Soc. B Biol. Sci. 366(1565), 671–679 (2011)

    Google Scholar 

  48. Levis, A., Schechner, Y.Y., Davis, A.B., Loveridge, J.: Multi-view polarimetric scattering cloud tomography and retrieval of droplet size. arXiv preprint arXiv:2005.11423 (2020)

  49. Maeda, T., Kadambi, A., Schechner, Y.Y., Raskar, R.: Dynamic heterodyne interferometry. In: ICCP, pp. 1–11. IEEE (2018)

    Google Scholar 

  50. McGuire, K., De Croon, G., De Wagter, C., Tuyls, K., Kappen, H.: Efficient optical flow and stereo vision for velocity estimation and obstacle avoidance on an autonomous pocket drone. IEEE Rob. Autom. Lett. 2(2), 1070–1076 (2017)

    Google Scholar 

  51. Miyazaki, D., Saito, M., Sato, Y., Ikeuchi, K.: Determining surface orientations of transparent objects based on polarization degrees in visible and infrared wavelengths. JOSA A 19(4), 687–694 (2002)

    Google Scholar 

  52. Mohan, A., Woo, G., Hiura, S., Smithwick, Q., Raskar, R.: Bokode: imperceptible visual tags for camera based interaction from a distance. ACM Trans. Graph. 28, 98 (2009)

    Google Scholar 

  53. Monjur, M., Spinoulas, L., Gill, P.R., Stork, D.G.: Panchromatic diffraction gratings for miniature computationally efficient visual-bar-position sensing. Sens. Transducers 194(11), 127 (2015)

    Google Scholar 

  54. Mujat, M., Baleine, E., Dogariu, A.: Interferometric imaging polarimeter. JOSA A 21(11), 2244–2249 (2004)

    Google Scholar 

  55. Pijnacker Hordijk, B.J., Scheper, K.Y., De Croon, G.C.: Vertical landing for micro air vehicles using event-based optical-flow. J. Field Rob. 35(1), 69–90 (2018)

    Google Scholar 

  56. Reddy, D., Veeraraghavan, A., Chellappa, R.: P2C2: programmable pixel compressive camera for high speed imaging. In: CVPR, pp. 329–336. IEEE (2011)

    Google Scholar 

  57. Schechner, Y.Y.: Inversion by P4: polarization-picture post-processing. Philos. Trans. Roy. Soc. B Biol. Sci. 366(1565), 638–648 (2011)

    Google Scholar 

  58. Schechner, Y.Y.: Self-calibrating imaging polarimetry. In: ICCP, pp. 1–10. IEEE (2015)

    Google Scholar 

  59. Schechner, Y.Y., Diner, D.J., Martonchik, J.V.: Spaceborne underwater imaging. In: ICCP, pp. 1–8. IEEE (2011)

    Google Scholar 

  60. Schechner, Y.Y., Nayar, S.K.: Generalized mosaicing: polarization panorama. IEEE Trans. Pattern Anal. Mach. Intell. 27(4), 631–636 (2005)

    Google Scholar 

  61. Schilling, K., Schechner, Y., Koren, I.: Cloudct-computed tomography of clouds by a small satellite formation. In: Proceedings of the 12th IAA Symposium on Small Satellites for Earth Observation (2019)

    Google Scholar 

  62. Shantaiya, S., Verma, K., Mehta, K.: Multiple object tracking using Kalman filter and optical flow. Eur. J. Adv. Eng. Technol. 2(2), 34–39 (2015)

    Google Scholar 

  63. Shashar, N.: Transmission of linearly polarized light in seawater: implications for polarization signaling. J. Exp. Biol. 207(20), 3619–3628 (2004)

    Google Scholar 

  64. Shashar, N., et al.: Underwater linear polarization: physical limitations to biological functions. Philos. Trans. Roy. Soc. B Biol. Sci. 366(1565), 649–654 (2011)

    Google Scholar 

  65. Shashar, N., Rutledge, P., Cronin, T.: Polarization vision in cuttlefish in a concealed communication channel? J. Exp. Biol. 199(9), 2077–2084 (1996)

    Google Scholar 

  66. Shashar, N., Hagan, R., Boal, J.G., Hanlon, R.T.: Cuttlefish use polarization sensitivity in predation on silvery fish. Vis. Res. 40, 71–75 (2000)

    Google Scholar 

  67. Shashar, N., Milbury, C., Hanlon, R.: Polarization vision in cephalopods: neuroanatomical and behavioral features that illustrate aspects of form and function. Mar. Freshw. Behav. Physiol. 35, 57–68 (2002)

    Google Scholar 

  68. Shen, C., et al.: Optical flow sensor/INS/magnetometer integrated navigation system for MAV in GPS-denied environment. J. Sens. 2016, 1–10 (2016)

    Google Scholar 

  69. Smith, F., Stewart, D.: Robot and insect navigation by polarized skylight. In: BIOSIGNALS, pp. 183–188 (2014)

    Google Scholar 

  70. Song, X., Seneviratne, L.D., Althoefer, K.: A Kalman filter-integrated optical flow method for velocity sensing of mobile robots. IEEE/ASME Trans. Mechatron. 16(3), 551–563 (2010)

    Google Scholar 

  71. Srinivasan, M.V.: Honeybees as a model for the study of visually guided flight, navigation, and biologically inspired robotics. Physiol. Rev. 91(2), 413–460 (2011)

    Google Scholar 

  72. Stork, D.G.: Optical elements as computational devices for low-power sensing and imaging. In: Imaging Systems and Applications, pp. ITu4E-4. OSA (2017)

    Google Scholar 

  73. Stork, D.G., Gill, P.R.: Special-purpose optics to reduce power dissipation in computational sensing and imaging systems. In: Sensors, pp. 1–3. IEEE (2017)

    Google Scholar 

  74. Tanaka, K., Mukaigawa, Y., Matsushita, Y., Yagi, Y.: Descattering of transmissive observation using parallel high-frequency illumination. In: ICCP, pp. 1–8. IEEE (2013)

    Google Scholar 

  75. Tessens, L., Morbée, M., Philips, W., Kleihorst, R., Aghajan, H.: Efficient approximate foreground detection for low-resource devices. In: International Conference on Distributed Smart Cameras, pp. 1–8. ACM/IEEE (2009)

    Google Scholar 

  76. Thurrowgood, S., Moore, R.J., Soccol, D., Knight, M., Srinivasan, M.V.: A biologically inspired, vision-based guidance system for automatic landing of a fixed-wing aircraft. J. Field Rob. 31(4), 699–727 (2014)

    Google Scholar 

  77. Treeaporn, V., Ashok, A., Neifeld, M.A.: Increased field of view through optical multiplexing. Opt. Express 18(21), 22432–22445 (2010)

    Google Scholar 

  78. Treibitz, T., Schechner, Y.Y.: Polarization: Beneficial for visibility enhancement? In: CVPR, pp. 525–532. IEEE (2009)

    Google Scholar 

  79. Tyo, J.S., Goldstein, D.L., Chenault, D.B., Shaw, J.A.: Review of passive imaging polarimetry for remote sensing applications. Appl. Opt. 45(22), 5453–5469 (2006)

    Google Scholar 

  80. Umeyama, S., Godin, G.: Separation of diffuse and specular components of surface reflection by use of polarization and statistical analysis of images. IEEE Trans. Pattern Anal. Mach. Intell. 5, 639–647 (2004)

    Google Scholar 

  81. Wang, D., Liang, H., Zhu, H., Zhang, S.: A bionic camera-based polarization navigation sensor. Sensors 14(7), 13006–13023 (2014)

    Google Scholar 

  82. Wehner, R.D.: Polarized-light navigation by insects. Sci. Am. 235(1), 106–115 (1976)

    Google Scholar 

  83. Xu, Z., Yue, D.K., Shen, L., Voss, K.J.: Patterns and statistics of in-water polarization under conditions of linear and nonlinear ocean surface waves. J. Geophys. Res. Oceans 116(12), 1–14 (2011)

    Google Scholar 

  84. Yuan, W., Dana, K., Varga, M., Ashok, A., Gruteser, M., Mandayam, N.: Computer vision methods for visual MIMO optical system. In: CVPR Workshops, pp. 37–43. IEEE (2011)

    Google Scholar 

  85. Zallat, J., Torzynski, M., Lallement, A.: Double-pass self-spectral-calibration of a polarization state analyzer. Opt. Lett. 37(3), 401–403 (2012)

    Google Scholar 

  86. Zhang, L., Hancock, E.R., Atkinson, G.A.: Reflection component separation using statistical analysis and polarisation. In: Vitrià, J., Sanches, J.M., Hernández, M. (eds.) IbPRIA 2011. LNCS, vol. 6669, pp. 476–483. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-21257-4_59

    Chapter  Google Scholar 

Download references

Acknowledgements

We thank M. Sheinin, A. Levis, A. Vainiger, T. Loeub, V. Holodovsky, M. Fisher, Y. Gat, and O. Elezra for fruitful discussions. We thank I. Czerninski, O. Shubi, D. Yegudin, and I. Talmon for technical support. Yoav Schechner is the Mark and Diane Seiden Chair in Science at the Technion. He is a Landau Fellow - supported by the Taub Foundation. His work was conducted in the Ollendorff Minerva Center. Minvera is funded through the BMBF. This work is supported by the Israel Science Foundation (ISF fund 542/16).

Author information

Authors and Affiliations

  1. Viterbi Faculty of Electrical Engineering, Technion - Israel Institute of Technology, 32000, Haifa, Israel

    Masada Tzabari & Yoav Y. Schechner

Authors
  1. Masada Tzabari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yoav Y. Schechner
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Masada Tzabari .

Editor information

Editors and Affiliations

  1. University of Oxford, Oxford, UK

    Andrea Vedaldi

  2. Graz University of Technology, Graz, Austria

    Horst Bischof

  3. University of Freiburg, Freiburg im Breisgau, Germany

    Thomas Brox

  4. University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

    Jan-Michael Frahm

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 1102 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Tzabari, M., Schechner, Y.Y. (2020). Polarized Optical-Flow Gyroscope. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, JM. (eds) Computer Vision – ECCV 2020. ECCV 2020. Lecture Notes in Computer Science(), vol 12361. Springer, Cham. https://doi.org/10.1007/978-3-030-58517-4_22

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-58517-4_22

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-58516-7

  • Online ISBN: 978-3-030-58517-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation