Abstract
State-of-the-art instance segmentation techniques currently provide a bounding box, class, mask, and scores for each instance. What they do not provide is an epistemic uncertainty estimate of these predictions. With our approach, we want to identify corner cases by considering the epistemic uncertainty. Corner cases are data/situations that are underrepresented or not covered in our data set. Our work is based on Mask R-CNN. We estimate the epistemic uncertainty by extending the architecture with Monte-Carlo dropout layers. By repeatedly executing the forward pass, we create a large number of predictions per instance. Afterward, we cluster the predictions of an instance based on the bounding box coordinates. It becomes possible to determine the epistemic position uncertainty for the bounding boxes and the classifier’s epistemic class uncertainty. For the epistemic uncertainty regarding the bounding box position and the class assignment, we provide a criterion for detecting corner cases utilizing the model’s epistemic uncertainty.
F. Heidecker and A. Hannan—Contributed equally to this work.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Audi AG: Driving Dataset (2019). https://www.a2d2.audi/a2d2/en.html. Accessed 12 Apr 2020
Blei, D., Jordan, M.: Variational inference for dirichlet process mixtures. J. Bayesian Anal. 1, 121–144 (2006)
Blundell, C., Cornebise, J., Kavukcuoglu, K., Wierstra, D.: Weight Uncertainty in Neural Network. In: Proceedings of the 32nd ICML, vol. 37, pp. 1613–1622. PMLR, Lille, France (2015)
Choi, S., Lee, K., Lim, S., Oh, S.: Uncertainty-aware learning from demonstration using mixture density networks with sampling-free variance modeling. In: 2018 IEEE ICRA, pp. 6915–6922. IEEE, Brisbane, QLD, Australia (2018)
DeVries, T., Taylor, G.W.: Learning Confidence for Out-of-Distribution Detection in Neural Networks arxiv:1802.04865v1 (2018)
Dietterich, T.G.: Ensemble methods in machine learning. In: Kittler, J., Roli, F. (eds.) MCS 2000. LNCS, vol. 1857, pp. 1–15. Springer, Heidelberg (2000). https://doi.org/10.1007/3-540-45014-9_1
Gal, Y.: Uncertainty in Deep Learning. Ph.D. thesis, University of Cambridge (2016)
Gal, Y., Ghahramani, Z.: Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning. In: Proceedings of The 33rd ICML, vol. 48, pp. 1050–1059. JMLR.org, New York (2016)
He, K., Gkioxari, G., Dollar, P., Girshick, R.: Mask R-CNN. In: 2017 IEEE ICCV, pp. 2980–2988. IEEE, Venice, Italy (2017)
He, K., Zhang, X., Ren, S., Sun, J.: Deep Residual Learning for Image Recognition. In: 2016 IEEE CVPR, pp. 770–778. IEEE, Las Vegas, NV, USA (2016)
Hendrycks, D., Gimpel, K.: A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks arxiv:1610.02136v3 (2017)
Hüllermeier, E., Waegeman, W.: Aleatoric and Epistemic Uncertainty in Machine Learning: A Tutorial Introduction (2019)
Ilg, E., Cicek, O., Galesso, S., Klein, A., Makansi, O., Hutter, F., Brox, T.: Uncertainty estimates and multi-hypotheses networks for optical flow. In: ECCV, pp. 652–667. Munich, Germany (2018)
Lakshminarayanan, B., Pritzel, A., Blundell, C.: Simple and Scalable Predictive Uncertainty Estimation Using Deep Ensembles. In: Proceedings of the 31st NIPS, pp. 6405–6416. Curran Associates Inc, Red Hook, NY, USA (2017)
Liang, S., Li, Y., Srikant, R.: Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks arxiv:1706.02690v4 (2018)
Liu, J.Z., Paisley, J., Kioumourtzoglou, M.A., Coull, B.: Accurate uncertainty estimation and decomposition in ensemble learning. In: Proceedings of the 33rd NIPS, pp. 8952–8963. Curran Associates Inc, Vancouver, Canada (2019)
Malinin, A., Gales, M.: Predictive Uncertainty Estimation via Prior Networks. In: Proceedings of the 32nd NIPS, pp. 7047–7058. Curran Associates Inc, Red Hook, NY, USA (2018)
Pinheiro, P.O., Collobert, R., Dollár, P.: Learning to Segment Object Candidates. In: Proceedings of the 28th NIPS, vol. 2, pp. 1990–1998. MIT Press, Cambridge, MA, USA (2015)
Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You Only Look Once: Unified, Real-Time Object Detection. In: 2016 IEEE CVPR, pp. 779–788. IEEE, Las Vegas, NV, USA (2016)
Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE TPAMI 39(6), 1137–1149 (2017)
Shalev, G., Adi, Y., Keshet, J.: Out-of-Distribution Detection using Multiple Semantic Label Representations. In: Proceedings of the 32nd NIPS, pp. 7375–7385. Curran Associates Inc. (2018)
Shelhamer, E., Long, J., Darrell, T.: Fully convolutional networks for semantic segmentation. IEEE TPAMI 39(4), 640–651 (2017)
Sieman, R.: Strange Off-Road Dirt Bikes & Vehicles (2013), https://www.off-road.com/dirtbike/feature/strange-offroad-dirt-bikes-vehicles-53605.html?printable Accessed 12 Apr 2020
Uijlings, J.R.R., van de Sande, K.E.A., Gevers, T., Smeulders, A.W.M.: Selective search for object recognition. IJCV 104(2), 154–171 (2013)
Yuan, Y., Chen, X., Wang, J.: Object-Contextual Representations for Semantic Segmentation arxiv:1909.11065v2 (2019)
Acknowledgment
This work results from the project KI Data Tooling (19A20001O) funded by BMWI (Deutsches Bundesministerium für Wirtschaft und Energie/German Federal Ministry for Economic Affairs and Energy) and the project DeCoInt\(^2\) supported by the German Research Foundation (DFG) within the priority program SPP 1835: “Kooperativ interagierende Automobile”, grant number SI 674/11-2.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Heidecker, F., Hannan, A., Bieshaar, M., Sick, B. (2021). Towards Corner Case Detection by Modeling the Uncertainty of Instance Segmentation Networks. In: Del Bimbo, A., et al. Pattern Recognition. ICPR International Workshops and Challenges. ICPR 2021. Lecture Notes in Computer Science(), vol 12664. Springer, Cham. https://doi.org/10.1007/978-3-030-68799-1_26
Download citation
DOI: https://doi.org/10.1007/978-3-030-68799-1_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-68798-4
Online ISBN: 978-3-030-68799-1
eBook Packages: Computer ScienceComputer Science (R0)
