Abstract
Transparency detection is a hard problem, as suggested by animals and humans flying or running into glass. However, humans seem to be able to learn and improve on the task with experience, begging the question, whether computers are able to do so too. Making a computer learn and understand transparency would be beneficial for moving agents, such as robots or autonomous vehicles. Our contributions are threefold: First, we conducted a perception study to obtain insights about human transparency detection methods, when borders of transparent objects are not visible. Second, based on our study insights we created a novel synthetic dataset called DISTOPIA, which focuses on the warping properties of transparent objects, placed in a variety of natural scenes and contains over 140 000 high resolution images. Third, we modified and trained a deep neural network classification model with an attention module to detect transparency through warping. Our results show that a neural network trained on synthetic data depicting only distortion effects can solve the transparency detection problem and surpasses human performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
von Hermann, H.: Allgemeine Encyklopädie der Physik / 9 Handbuch der physiologischen Optik, volume 9. Leopold Voss. https://doi.org/10.3931/E-RARA-21259
Beatrix Tudor-Hart. Beiträge zur psychologie der gestalt. (1). https://doi.org/10.1007/bf00492012
Metelli, F.: An algebraic development of the theory of perceptual transparency. (1). https://doi.org/10.1080/00140137008931118
Metelli, F.: The perception of transparency. (4). https://doi.org/10.1038/scientificamerican0474-90
Adelson, E.H., Anandan, P.: Ordinal characteristics of transparency
Watanabe, T., Cavanagh, P.: Transparent surfaces defined by implicit x junctions. (16). https://doi.org/10.1016/0042-6989(93)90111-9
Anderson, B.L., Julesz, B.: A theoretical analysis of illusory contour formation in stereopsis. (4). https://doi.org/10.1037/0033-295x.102.4.705
Watanabe, T., Cavanagh, P.: Texture laciness: the texture equivalent of transparency? (3). https://doi.org/10.1068/p250293
D’Zmura, M., Colantoni, P., Knoblauch, K., Laget, B.: Color transparency. (4). https://doi.org/10.1068/p260471
Tavel, M.: What determines whether a substance is transparent? for instance, why is silicon transparent when it is glass but not when it is sand or a computer chip. http://www.scientificamerican.com/article/what-determines-whether-a/
Faul, F., Ekroll, V.: Psychophysical model of chromatic perceptual transparency based on substractive color mixture. (6). https://doi.org/10.1364/josaa.19.001084
Fulvio, J.M., Singh, M., Maloney, L.T.: Combining achromatic and chromatic cues to transparency. (8). https://doi.org/10.1167/6.8.1
Thilak, V., Voelz, D.G., Creusere, C.D.: Polarization-based index of refraction and reflection angle estimation for remote sensing applications. (30). https://doi.org/10.1364/ao.46.007527
Bex, P.J.: (in) sensitivity to spatial distortion in natural scenes. (2). https://doi.org/10.1167/10.2.23
Klank, U., Carton, D., Beetz, M.: Transparent object detection and reconstruction on a mobile platform. In: 2011 IEEE International Conference on Robotics and Automation, pp. 5971–5978. IEEE. https://doi.org/10.1109/icra.2011.5979793
Sayim, B., Cavanagh, P.: The art of transparency. (7). https://doi.org/10.1068/i0459aap
von Gioi, R.G., Jakubowicz, J., Morel, J.-M., Randall, G.: Lsd: a line segment detector. https://doi.org/10.5201/ipol.2012.gjmr-lsd
Alt, N., Rives, P., Steinbach, E.: Reconstruction of transparent objects in unstructured scenes with a depth camera. In: 2013 IEEE International Conference on Image Processing, pp. 4131–4135. IEEE. https://doi.org/10.1109/icip.2013.6738851
Koffka, K.: Principles Of Gestalt Psychology. Routledge. https://doi.org/10.4324/9781315009292
Lysenkov, I., Rabaud, V.: Pose estimation of rigid transparent objects in transparent clutter. In: 2013 IEEE International Conference on Robotics and Automation, pp. 162–169. IEEE. https://doi.org/10.1109/icra.2013.6630571
Maeno, K., Nagahara, H., Shimada, A., Taniguchi, R.-I.: Light field distortion feature for transparent object recognition. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2786–2793. IEEE. https://doi.org/10.1109/cvpr.2013.359
Schlüter, N., Faul, F.: Are optical distortions used as a cue for material properties of thick transparent objects? 14(14), 2. https://doi.org/10.1167/14.14.2
Han, K., Wong, K.-Y.K., Liu, M.: A fixed viewpoint approach for dense reconstruction of transparent objects. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4001–4008. IEEE. https://doi.org/10.1109/cvpr.2015.7299026
Kawabe, T., Maruya, K., Nishida, S.: Perceptual transparency from image deformation. (33). https://doi.org/10.1073/pnas.1500913112
Phillips, C.J., Lecce, M., Daniilidis, K.: Seeing glassware: from edge detection to pose estimation and shape recovery. In Robotics: Science and Systems XII, vol. 3, p. 3. Michigan, USA, Robotics: Science and Systems Foundation. https://doi.org/10.15607/rss.2016.xii.021
Qian, Y., Gong, M., Yang, Y.-H.: 3d reconstruction of transparent objects with position-normal consistency. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4369–4377. IEEE. https://doi.org/10.1109/cvpr.2016.473
Park, J., Woo, S., Lee, J.-Y., Kweon, I.S.: Bam: Bottleneck attention module. In: British Machine Vision Conference. arXiv. 1048550/ARXIV.1807.06514
Schlüter, N., Faul, F.: Visual shape perception in the case of transparent objects. (4). https://doi.org/10.1167/19.4.24
Tsai, D., Dansereau, D.G., Peynot, T., Corke, P.: Distinguishing refracted features using light field cameras with application to structure from motion. (2). https://doi.org/10.1109/lra.2018.2884765
Mei, H., et al.: Don’t hit me! glass detection in real-world scenes. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3687–3696. IEEE. https://doi.org/10.1109/cvpr42600.2020.00374
Sajjan, S., et al.: Clear grasp: 3d shape estimation of transparent objects for manipulation. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 3634–3642. IEEE. DOI: https://doi.org/10.1109/icra40945.2020.9197518
Xie, E., Wang, W., Wang, W., Ding, M., Shen, C., Luo, P.: Segmenting transparent objects in the wild. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12358, pp. 696–711. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58601-0_41
Gu, G., Ko, B., Go, S., Lee, S.-H., Lee, J., Shin, M.: Towards light-weight and real-time line segment detection. https://doi.org/10.48550/ARXIV.2106.00186
He, H., et al.: Enhanced boundary learning for glass-like object segmentation. In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 15859–15868. IEEE. https://doi.org/10.1109/iccv48922.2021.01556
Karras, T., et al.: Alias-free generative adversarial networks. In: Proceedings NeurIPS. arXiv. 1048550/ARXIV.2106.12423
Zhu, L., et al.: Rgb-d local implicit function for depth completion of transparent objects. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4649–4658. IEEE. https://doi.org/10.1109/cvpr46437.2021.00462
Fang, H., Fang, H.-S., Xu, S., Lu, C.: Transcg: a large-scale real-world dataset for transparent object depth completion and a grasping baseline. (3). https://doi.org/10.1109/lra.2022.3183256
Mei, H., et al.: Glass segmentation using intensity and spectral polarization cues. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12622–12631. IEEE. https://doi.org/10.1109/cvpr52688.2022.01229
Wirth, T., Jamili, A., von Buelow, M., Knauthe, V., Guthe, S.: Fitness of general-purpose monocular depth estimation architectures for transparent structures. In: Pelechano, N., Vanderhaeghe, D., (eds.) Eurographics 2022 - Short Papers. The Eurographics Association. https://doi.org/10.2312/EGS.20221020
Yang, J., Li, C., Dai, X., Yuan, L., Gao, J.: Focal modulation networks. https://doi.org/10.48550/ARXIV.2203.11926
Yu, L., et al.: Progressive glass segmentation. https://doi.org/10.1109/tip.2022.3162709
Pavlovia. https://pavlovia.org/
Poly haven. https://polyhaven.com/
Acknowledgements
Part of the research in this paper was funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) project number 407 714 161. We thank Frank Jäkel for his generous support and the anonymous reviewers whose comments helped improve this manuscript.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Knauthe, V. et al. (2023). Distortion-Based Transparency Detection Using Deep Learning on a Novel Synthetic Image Dataset. In: Gade, R., Felsberg, M., Kämäräinen, JK. (eds) Image Analysis. SCIA 2023. Lecture Notes in Computer Science, vol 13885. Springer, Cham. https://doi.org/10.1007/978-3-031-31435-3_17
Download citation
DOI: https://doi.org/10.1007/978-3-031-31435-3_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-31434-6
Online ISBN: 978-3-031-31435-3
eBook Packages: Computer ScienceComputer Science (R0)
