skip to main content
research-article

Interpretable Latent Space for Meteorological Out-of-Distribution Detection via Weak Supervision

Published: 15 May 2024 Publication History

Abstract

Deep neural networks (DNNs) are effective tools for learning-enabled cyber-physical systems (CPSs) that handle high-dimensional image data. However, DNNs may make incorrect decisions when presented with inputs outside the distribution of their training data. These inputs can compromise the safety of CPSs. So, it becomes crucial to detect inputs as out-of-distribution (OOD) and interpret the reasons for their classification as OOD. In this study, we propose an interpretable learning method to detect OOD caused by meteorological features like darkness, lightness, and rain.
To achieve this, we employ a variational autoencoder (VAE) to map high-dimensional image data to a lower-dimensional latent space. We then focus on a specific latent dimension and encourage it to classify different intensities of a particular meteorological feature in a monotonically increasing manner. This is accomplished by incorporating two additional terms into the VAE’s loss function: a classification loss and a positional loss. During training, we optimize the utilization of label information for classification. Remarkably, our results demonstrate that using only 25% of the training data labels is sufficient to train a single pre-selected latent dimension to classify different intensities of a specific meteorological feature. We evaluate the proposed method on two distinct datasets, CARLA and Duckietown, employing two different rain-generation methods. We show that our approach outperforms existing approaches by at least 15 in the F1 score and precision when trained and tested on CARLA dataset.

References

[1]
A. M. A. Abdulghani and G. G. Menekşe Dalveren. 2022. Moving object detection in video under different weather conditions using YOLO and faster R-CNN algorithms. European Journal of Science and Technology. (33) (2022), 40–54.
[2]
Alexander Amini, Wilko Schwarting, Guy Rosman, Brandon Araki, Sertac Karaman, and Daniela Rus. 2018. Variational autoencoder for end-to-end control of autonomous driving with novelty detection and training de-biasing. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 568–575.
[3]
Yoshua Bengio, Aaron Courville, and Pascal Vincent. 2013. Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 8 (2013), 1798–1828.
[4]
Terrance E. Boult, Steve Cruz, Akshay Raj Dhamija, Manuel Gunther, James Henrydoss, and Walter J. Scheirer. 2019. Learning and the unknown: Surveying steps toward open world recognition. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 9801–9807.
[5]
Feiyang Cai and Xenofon Koutsoukos. 2020. Real-time out-of-distribution detection in learning-enabled cyber-physical systems. In 2020 ACM/IEEE 11th International Conference on Cyber-Physical Systems (ICCPS). IEEE, 174–183.
[6]
Feiyang Cai, Ali I. Ozdagli, and Xenofon Koutsoukos. 2021. Detection of dataset shifts in learning-enabled cyber-physical systems using variational autoencoder for regression. In 2021 4th IEEE International Conference on Industrial Cyber-Physical Systems (ICPS). IEEE, 104–111.
[7]
Feiyang Cai, Ali I. Ozdagli, and Xenofon Koutsoukos. 2022. Variational autoencoder for classification and regression for out-of-distribution detection in learning-enabled cyber-physical systems. Applied Artificial Intelligence 36, 1 (2022), 2131056.
[8]
Feiyang Cai, Ali I. Ozdagli, Nicholas Potteiger, and Xenofon Koutsoukos. 2021. Inductive conformal out-of-distribution detection based on adversarial autoencoders. In 2021 IEEE International Conference on Omni-Layer Intelligent Systems (COINS). IEEE, 1–6.
[9]
Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. 2016. The cityscapes dataset for semantic urban scene understanding. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[10]
Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun. 2017. CARLA: An open urban driving simulator. In Proceedings of the 1st Annual Conference on Robot Learning. 1–16.
[11]
Yeli Feng, Daniel Jun Xian Ng, and Arvind Easwaran. 2021. Improving variational autoencoder based out-of-distribution detection for embedded real-time applications. ACM Transactions on Embedded Computing Systems (TECS) 20, 5s (2021), 1–26.
[12]
Irina Higgins, Loic Matthey, Arka Pal, Christopher P. Burgess, Xavier Glorot, Matthew M. Botvinick, Shakir Mohamed, and Alexander Lerchner. 2017. beta-vae: Learning basic visual concepts with a constrained variational framework. ICLR (Poster) 3 (2017).
[13]
San Jose. 2021. How to add rain to a photo with photoshop. https://www.photoshopessentials.com/photo-effects/photoshop-weather-effects-rain/2023 June 1.
[14]
Hans-Michael Kaltenbach. 2011. A Concise Guide to Statistics. Springer Science & Business Media.
[15]
Leonard Kaufman. 1990. Partitioning around medoids (program PAM). Finding Groups in Data 344 (1990), 68–125.
[16]
Hyunjik Kim and Andriy Mnih. 2018. Disentangling by factorising. In International Conference on Machine Learning. PMLR, 2649–2658.
[17]
Diederik P. Kingma and Max Welling. 2013. Auto-encoding variational Bayes. arXiv preprint arXiv:1312.6114 (2013).
[18]
Diederik P. Kingma and Max Welling. 2019. An introduction to variational autoencoders. Foundations and Trends® in Machine Learning 12, 4 (2019), 307–392.
[19]
Puneet Kohli and Anjali Chadha. 2020. Enabling pedestrian safety using computer vision techniques: A case study of the 2018 Uber Inc. self-driving car crash. In Future of Information and Communication Conference. Springer, 261–279.
[20]
Damon Lavrinc. 2014. This is how bad self-driving cars suck in rain. Technology Report (2014).
[21]
Francesco Locatello, Stefan Bauer, Mario Lucic, Gunnar Raetsch, Sylvain Gelly, Bernhard Schölkopf, and Olivier Bachem. 2019. Challenging common assumptions in the unsupervised learning of disentangled representations. In International Conference on Machine Learning. PMLR, 4114–4124.
[22]
Francesco Locatello, Michael Tschannen, Stefan Bauer, Gunnar Rätsch, Bernhard Schölkopf, and Olivier Bachem. 2019. Disentangling factors of variation using few labels. arXiv preprint arXiv:1905.01258 (2019).
[23]
Graziano Mita, Maurizio Filippone, and Pietro Michiardi. 2021. An identifiable double VAE for disentangled representations. In International Conference on Machine Learning. PMLR, 7769–7779.
[24]
Raymond T. Ng and Jiawei Han. 2002. CLARANS: A method for clustering objects for spatial data mining. IEEE Transactions on Knowledge and Data Engineering 14, 5 (2002), 1003–1016.
[25]
Liam Paull, Jacopo Tani, Heejin Ahn, Javier Alonso-Mora, Luca Carlone, Michal Cap, Yu Fan Chen, Changhyun Choi, Jeff Dusek, Yajun Fang, Daniel Hoehener, Shih-Yuan Liu, Michael Novitzky, Igor Franzoni Okuyama, Jason Pazis, Guy Rosman, Valerio Varricchio, Hsueh-Cheng Wang, Dmitry Yershov, Hang Zhao, Michael Benjamin, Christopher Carr, Maria Zuber, Sertac Karaman, Emilio Frazzoli, Domitilla Del Vecchio, Daniela Rus, Jonathan How, John Leonard, and Andrea Censi. 2017. Duckietown: An open, inexpensive and flexible platform for autonomy education and research. In 2017 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 1497–1504.
[26]
Shreyas Ramakrishna. 2022. Efficient Out-of-Distribution Detection Using Latent Space of B-VAE for Cyber-Physical Systems. https://github.com/scope-lab-vu/Beta-VAE-OOD-Detector
[27]
Shreyas Ramakrishna, Zahra Rahiminasab, Gabor Karsai, Arvind Easwaran, and Abhishek Dubey. 2022. Efficient out-of-distribution detection using latent space of β-VAE for cyber-physical systems. ACM Transactions on Cyber-Physical Systems (TCPS) 6, 2 (2022), 1–34.
[28]
Lukas Ruff, Jacob R. Kauffmann, Robert A. Vandermeulen, Grégoire Montavon, Wojciech Samek, Marius Kloft, Thomas G. Dietterich, and Klaus-Robert Müller. 2021. A unifying review of deep and shallow anomaly detection. Proc. IEEE 109, 5 (2021), 756–795.
[29]
OpenCV team. 2023. Color Conversions. OpenCV, Palo Alto, CA, USA.
[30]
Yuchi Tian, Kexin Pei, Suman Jana, and Baishakhi Ray. 2018. DeepTest: Automated testing of deep-neural-network-driven autonomous cars. In Proceedings of the 40th International Conference on Software Engineering. 303–314.
[31]
Maxime Tremblay, Shirsendu Sukanta Halder, Raoul De Charette, and Jean-François Lalonde. 2021. Rain rendering for evaluating and improving robustness to bad weather. International Journal of Computer Vision 129 (2021), 341–360.
[32]
Bill Vlasic and Neal E. Boudette. 2016. Self-driving Tesla was involved in fatal crash, US says. New York Times 302016 (2016).
[33]
Chien-Yao Wang, Alexey Bochkovskiy, and Hong-Yuan Mark Liao. 2022. YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. arXiv preprint arXiv:2207.02696 (2022).
[34]
Jingkang Yang, Kaiyang Zhou, Yixuan Li, and Ziwei Liu. 2021. Generalized out-of-distribution detection: A survey. arXiv preprint arXiv:2110.11334 (2021).
[35]
Yahan Yang, Ramneet Kaur, Souradeep Dutta, and Insup Lee. 2022. Interpretable detection of distribution shifts in learning enabled cyber-physical systems. In 2022 ACM/IEEE 13th International Conference on Cyber-Physical Systems (ICCPS). IEEE, 225–235.
[36]
Hang Yin, Feifei Zheng, Huan-Feng Duan, Dragan Savic, and Zoran Kapelan. 2023. Estimating rainfall intensity using an image-based deep learning model. Engineering 21 (2023), 162–174.
[37]
Michael Yuhas, Yeli Feng, Daniel Jun Xian Ng, Zahra Rahiminasab, and Arvind Easwaran. 2021. Embedded out-of-distribution detection on an autonomous robot platform. In Proceedings of the Workshop on Design Automation for CPS and IoT. 13–18.
[38]
Michael Yuhas, Daniel Jun Xian Ng, and Arvind Easwaran. 2022. Design methodology for deep out-of-distribution detectors in real-time cyber-physical systems. In 2022 IEEE 28th International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA). 180–185. DOI:

Index Terms

  1. Interpretable Latent Space for Meteorological Out-of-Distribution Detection via Weak Supervision

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Cyber-Physical Systems
    ACM Transactions on Cyber-Physical Systems  Volume 8, Issue 2
    April 2024
    342 pages
    EISSN:2378-9638
    DOI:10.1145/3613588
    • Editor:
    • Chenyang Lu
    Issue’s Table of Contents

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Journal Family

    Publication History

    Published: 15 May 2024
    Online AM: 07 March 2024
    Accepted: 29 February 2024
    Revised: 10 January 2024
    Received: 16 July 2023
    Published in TCPS Volume 8, Issue 2

    Check for updates

    Author Tags

    1. Safety critical system
    2. rain detection
    3. lightness detection
    4. representation learning
    5. autonomous vehicle

    Qualifiers

    • Research-article

    Funding Sources

    • MoE, Singapore, Tier-2

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 183
      Total Downloads
    • Downloads (Last 12 months)183
    • Downloads (Last 6 weeks)15
    Reflects downloads up to 16 Feb 2025

    Other Metrics

    Citations

    View Options

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    Full Text

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media