Skip to main content
SpringerLink
Log in
Book cover

International Joint Conference on Rough Sets

IJCRS 2022: Rough Sets pp 161–175Cite as

Uncertainty-Aware Deep Open-Set Object Detection

  • Conference paper
  • First Online:

  • 836 Accesses

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13633))

Abstract

Open-set object detection better simulates the real world compared with close-set object detection. Besides the classes of interest, it also pays attention to unknown objects in the environment. We extend the previous concept of open-set object detection, aiming to detect both known and unknown objects. Because unknown objects have different textural features from known classes and the background, we assume that detecting unknown instances will generate high uncertainty. Therefore, in this paper, we propose an uncertainty-aware open-set object detection framework based on faster R-CNN. We introduce evidential deep learning to the field of object detection to estimate the uncertainty of the predictions and perform more accurate classification in open-set conditions. The obtained uncertainty will be utilized to pseudo-label unknown instances in the training data. We also introduce a contrastive clustering module to separate the feature representations of each class during the training phase. We set an uncertainty-based unknown identifier at the inference phase to enhance the generalization of the detector. We conduct experiments on three different data splits, and our method outperforms the recent SOTA method. We also demonstrate each component in our method is effective and indispensable in our ablation studies.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Bendale, A., Boult, T.E.: Towards open set deep networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1563–1572 (2016)

    Google Scholar 

  2. Chen, L., Lou, Y., He, J., Bai, T., Deng, M.: Evidential neighborhood contrastive learning for universal domain adaptation (2022)

    Google Scholar 

  3. Dempster, A.P.: A generalization of Bayesian inference. J. Roy. Stat. Soc.: Ser. B (Methodol.) 30(2), 205–232 (1968)

    MathSciNet  MATH  Google Scholar 

  4. Dhamija, A., Gunther, M., Ventura, J., Boult, T.: The overlooked elephant of object detection: open set. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1021–1030 (2020)

    Google Scholar 

  5. Gal, Y., Ghahramani, Z.: Dropout as a Bayesian approximation: representing model uncertainty in deep learning. In: International Conference on Machine Learning, pp. 1050–1059. PMLR (2016)

    Google Scholar 

  6. Gawlikowski, J., et al.: A survey of uncertainty in deep neural networks. arXiv preprint arXiv:2107.03342 (2021)

  7. Geng, C., Huang, S.J., Chen, S.: Recent advances in open set recognition: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 43(10), 3614–3631 (2020)

    Article  Google Scholar 

  8. Girshick, R.: Fast R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1440–1448 (2015)

    Google Scholar 

  9. Han, Z., Zhang, C., Fu, H., Zhou, J.T.: Trusted multi-view classification. arXiv preprint arXiv:2102.02051 (2021)

  10. He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2961–2969 (2017)

    Google Scholar 

  11. Jaiswal, A., Babu, A.R., Zadeh, M.Z., Banerjee, D., Makedon, F.: A survey on contrastive self-supervised learning. Technologies 9(1), 2 (2020)

    Article  Google Scholar 

  12. Joseph, K., Khan, S., Khan, F.S., Balasubramanian, V.N.: Towards open world object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5830–5840 (2021)

    Google Scholar 

  13. Jøsang, A.: Subjective Logic: A Formalism for Reasoning Under Uncertainty. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-42337-1

    Book  MATH  Google Scholar 

  14. Lakshminarayanan, B., Pritzel, A., Blundell, C.: Simple and scalable predictive uncertainty estimation using deep ensembles. In: Advances in Neural Information Processing Systems, vol. 30 (2017)

    Google Scholar 

  15. Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017)

    Google Scholar 

  16. Liu, W., et al.: SSD: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2

    Chapter  Google Scholar 

  17. Liu, W., Yue, X., Chen, Y., Denoeux, T.: Trusted multi-view deep learning with opinion aggregation. In: The 36th AAAI Conference on Artificial Intelligence (AAAI-2022), vol. 36, pp. 7585–7593 (2022)

    Google Scholar 

  18. Malinin, A., Gales, M.: Predictive uncertainty estimation via prior networks. In: Advances in Neural Information Processing Systems, vol. 31 (2018)

    Google Scholar 

  19. Miller, D., Nicholson, L., Dayoub, F., Sünderhauf, N.: Dropout sampling for robust object detection in open-set conditions. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 3243–3249. IEEE (2018)

    Google Scholar 

  20. Miller, D., Sünderhauf, N., Milford, M., Dayoub, F.: Uncertainty for identifying open-set errors in visual object detection. IEEE Robot. Autom. Lett. 7(1), 215–222 (2022). https://doi.org/10.1109/LRA.2021.3123374

    Article  Google Scholar 

  21. Miller, D., Sünderhauf, N., Milford, M., Dayoub, F.: Uncertainty for identifying open-set errors in visual object detection. IEEE Robot. Autom. Lett. 7(1), 215–222 (2021)

    Article  Google Scholar 

  22. Molchanov, D., Lyzhov, A., Molchanova, Y., Ashukha, A., Vetrov, D.: Greedy policy search: a simple baseline for learnable test-time augmentation. arXiv preprint arXiv:2002.09103 (2020)

  23. Neal, L., Olson, M., Fern, X., Wong, W.-K., Li, F.: Open set learning with counterfactual images. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11210, pp. 620–635. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01231-1_38

    Chapter  Google Scholar 

  24. Oberdiek, P., Rottmann, M., Gottschalk, H.: Classification uncertainty of deep neural networks based on gradient information. In: Pancioni, L., Schwenker, F., Trentin, E. (eds.) ANNPR 2018. LNCS (LNAI), vol. 11081, pp. 113–125. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-99978-4_9

    Chapter  Google Scholar 

  25. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016)

    Google Scholar 

  26. Redmon, J., Farhadi, A.: YOLO9000: better, faster, stronger. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7263–7271 (2017)

    Google Scholar 

  27. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems, vol. 28 (2015)

    Google Scholar 

  28. Salimans, T., Kingma, D.P.: Weight normalization: a simple reparameterization to accelerate training of deep neural networks. In: Advances in Neural Information Processing Systems, vol. 29 (2016)

    Google Scholar 

  29. Scheirer, W.J., Jain, L.P., Boult, T.E.: Probability models for open set recognition. IEEE Trans. Pattern Anal. Mach. Intell. 36(11), 2317–2324 (2014)

    Article  Google Scholar 

  30. Scheirer, W.J., de Rezende Rocha, A., Sapkota, A., Boult, T.E.: Toward open set recognition. IEEE Trans. Pattern Anal. Mach. Intell. 35(7), 1757–1772 (2012)

    Article  Google Scholar 

  31. Sensoy, M., Kaplan, L., Kandemir, M.: Evidential deep learning to quantify classification uncertainty. In: Advances in Neural Information Processing Systems, vol. 31 (2018)

    Google Scholar 

  32. Stevens, W.: Efficient uncertainty estimation for open-set object detection. In: Epistemic Uncertainty Estimation for Object Detection in Open-Set Conditions, p. 91 (2021)

    Google Scholar 

  33. Valdenegro-Toro, M.: Deep sub-ensembles for fast uncertainty estimation in image classification. arXiv preprint arXiv:1910.08168 (2019)

  34. Welling, M., Teh, Y.W.: Bayesian learning via stochastic gradient Langevin dynamics. In: Proceedings of the 28th International Conference on Machine Learning (ICML-2011), pp. 681–688 (2011)

    Google Scholar 

  35. Wen, Y., Liu, W., Weller, A., Raj, B., Singh, R.: SphereFace2: binary classification is all you need for deep face recognition. arXiv preprint arXiv:2108.01513 (2021)

  36. Wen, Y., Tran, D., Ba, J.: BatchEnsemble: an alternative approach to efficient ensemble and lifelong learning. arXiv preprint arXiv:2002.06715 (2020)

  37. Yue, X., Chen, Y., Yuan, B., Lv, Y.: Three-way image classification with evidential deep convolutional neural networks. Cogn. Comput. 1–13 (2021). https://doi.org/10.1007/s12559-021-09869-y

  38. Zhou, X., Yue, X., Xu, Z., Denoeux, T., Chen, Y.: Deep neural networks with prior evidence for bladder cancer staging. In: 2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 1221–1226. IEEE (2021)

    Google Scholar 

  39. Zhu, X., Su, W., Lu, L., Li, B., Wang, X., Dai, J.: Deformable DETR: deformable transformers for end-to-end object detection. arXiv preprint arXiv:2010.04159 (2020)

Download references

Acknowledgments

This work was supported by National Natural Science Foundation of China (Serial Nos. 61976134, 61991410, 61991415), Natural Science Foundation of Shanghai (Serial No. 21ZR1423900) and Open Project Foundation of Intelligent Information Processing Key Laboratory of Shanxi Province, China (No. CICIP2021001).

Author information

Authors and Affiliations

  1. School of Computer Engineering and Science, Shanghai University, Shanghai, 200444, China

    Qi Hang, Zihao Li, Yudi Dong & Xiaodong Yue

  2. Artificial Intelligence Institute of Shanghai University, Shanghai, China

    Qi Hang & Xiaodong Yue

Authors
  1. Qi Hang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zihao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yudi Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaodong Yue
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaodong Yue .

Editor information

Editors and Affiliations

  1. University of Regina, Regina, SK, Canada

    JingTao Yao

  2. Iwate Prefectural University, Takizawa, Iwate, Japan

    Hamido Fujita

  3. Shanghai University, Shanghai, China

    Xiaodong Yue

  4. Tongji University, Shanghai, China

    Duoqian Miao

  5. University of Kansas, Lawrence, KS, USA

    Jerzy Grzymala-Busse

  6. Soochow University, Suzhou, Jiangsu, China

    Fanzhang Li

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Hang, Q., Li, Z., Dong, Y., Yue, X. (2022). Uncertainty-Aware Deep Open-Set Object Detection. In: Yao, J., Fujita, H., Yue, X., Miao, D., Grzymala-Busse, J., Li, F. (eds) Rough Sets. IJCRS 2022. Lecture Notes in Computer Science(), vol 13633. Springer, Cham. https://doi.org/10.1007/978-3-031-21244-4_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-21244-4_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-21243-7

  • Online ISBN: 978-3-031-21244-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation