Skip to main content
SpringerLink
Log in

Reliable Classification of Images by Calculating Their Credibility Using a Layer-Wise Activation Cluster Analysis of CNNs

  • Conference paper
  • First Online:
Deep Learning Theory and Applications (DeLTA 2022)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1858))

Included in the following conference series:

  • 193 Accesses

Abstract

An image classification model based on a Convolutional Neural Network architecture generally achieves a high classification performance over a wide range of image domains. However, the model is only able to obtain such a high performance on in-distribution samples. On out-of-distribution samples, in contrast, the performance of the model may be significantly decreased. To detect out-of-distribution samples, Papernot and McDaniel [38] introduced a method named DkNN, which is based on calculating a sample credibility score by a nearest neighbor classification in feature space of the hidden layers of the model. However, a nearest neighbor classification is memory-intensive and slow at inference. To address these issues, Lehmann and Ebner [26] suggested a method named LACA, which calculates the credibility score based on clustering instead of a nearest neighbor classification. Lehmann and Ebner [26] showed that for out-of-distribution samples with respect to models trained on MNIST, SVHN, or CIFAR-10, LACA is significantly faster at inference compared to DkNN, while obtaining a similar performance. In this work, we conducted additional experiments to test LACA on more complex datasets (Imagenette, Imagewoof). Our experiments show that LACA is significantly faster at inference compared to DkNN also for these more complex datasets. Furthermore, LACA computes meaningful credibility scores, while DkNN fails on these datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 54.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 69.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://github.com/fastai/imagenette#imagenette-1.

  2. 2.

    https://github.com/fastai/imagenette#imagewoof.

References

  1. Biggio, B., et al.: Evasion attacks against machine learning at test time. In: Blockeel, H., Kersting, K., Nijssen, S., Železný, F. (eds.) ECML PKDD 2013. LNCS (LNAI), vol. 8190, pp. 387–402. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40994-3_25

    Chapter  Google Scholar 

  2. Carrara, F., Falchi, F., Caldelli, R., Amato, G., Becarelli, R.: Adversarial image detection in deep neural networks. Multimedia Tools Appl. 78(3), 2815–2835 (2019)

    Article  Google Scholar 

  3. Chen, B., et al.: Detecting backdoor attacks on deep neural networks by activation clustering. In: Espinoza, H., Ó hÉigeartaigh, S., Huang, X., Hernández-Orallo, J., Castillo-Effen, M. (eds.) Workshop on SafeAI@AAAI. CEUR Workshop, vol. 2301. ceur-ws.org, Honolulu, HI, USA (2019)

    Google Scholar 

  4. Chen, T., Navratil, J., Iyengar, V., Shanmugam, K.: Confidence scoring using whitebox meta-models with linear classifier probes. In: Chaudhuri, K., Sugiyama, M. (eds.) AISTATS, vol. 89, pp. 1467–1475. PMLR, Naha, Japan (2019)

    Google Scholar 

  5. Clanuwat, T., Bober-Irizar, M., Kitamoto, A., Lamb, A., Yamamoto, K., Ha, D.: Deep learning for classical Japanese literature. ArXiv arXiv:1812.01718 (2018)

  6. Cohen, G., Sapiro, G., Giryes, R.: Detecting adversarial samples using influence functions and nearest neighbors. In: CVPR. pp. 14441–14450. IEEE, Seattle, WA, USA (2020)

    Google Scholar 

  7. Gal, Y.: Uncertainty in Deep Learning. Ph.D. thesis, Univ of Cambridge (2016)

    Google Scholar 

  8. Gal, Y., Ghahramani, Z.: Dropout as a bayesian approximation: representing model uncertainty in deep learning. In: Balcan, M., Weinberger, K. (eds.) ICML, vol. 48, pp. 1050–1059. PMLR, New York, NY, USA (2016)

    Google Scholar 

  9. Goodfellow, I., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. In: Bengio, Y., LeCun, Y. (eds.) ICLR. San Diego, CA, USA (2015)

    Google Scholar 

  10. Grosse, K., Manoharan, P., Papernot, N., Backes, M., McDaniel, P.: On the (statistical) detection of adversarial examples. ArXiv arXiv:1702.06280 (2017)

  11. He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: ICCV. pp. 1026–1034. IEEE, Santiago, Chile (2015)

    Google Scholar 

  12. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR, pp. 770–778. IEEE, Las Vegas, NV, USA (2016)

    Google Scholar 

  13. Hendrycks, D., Gimpel, K.: A baseline for detecting misclassified and out-of-distribution examples in neural networks. In: ICLR, Toulon, France (2017)

    Google Scholar 

  14. Hendrycks, D., Mazeika, M., Kadavath, S., Song, D.: Using self-supervised learning can improve model robustness and uncertainty. In: Wallach, H., Larochelle, H., Beygelzimer, A., d’Alché-Buc, F., Fox, E., Garnett, R. (eds.) NeurIPS, vol. 32, pp. 15637–15648. CAI, Vancouver, CA (2019)

    Google Scholar 

  15. Hendrycks, D., Zhao, K., Basart, S., Steinhardt, J., Song, D.: Natural adversarial examples. ArXiv arXiv:1907.07174 (2020)

  16. Huang, H., Li, Z., Wang, L., Chen, S., Dong, B., Zhou, X.: Feature space singularity for out-of-distribution detection. In: Espinoza, H., et al., (eds.) Workshop on SafeAI@AAAI. CEUR Workshop, vol. 2808. ceur-ws.org (2021)

    Google Scholar 

  17. Kim, H.: Torchattacks: A pytorch repository for adversarial attacks. ArXiv arXiv:2010.01950 (2020)

  18. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: Bengio, Y., LeCun, Y. (eds.) ICLR. San Diego, CA, USA (2015)

    Google Scholar 

  19. Krizhevsky, A.: Learning multiple layers of features from tiny images. University of Toronto, Technical Report (2009)

    Google Scholar 

  20. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) NIPS, vol. 25, pp. 1097–1105. CAI, Lake Tahoe, NV, USA (2012)

    Google Scholar 

  21. Kurakin, A., Goodfellow, I.J., Bengio, S.: Adversarial examples in the physical world. In: ICLR. Toulon, France (2017)

    Google Scholar 

  22. LeCun, Y., Cortes, C., Burges, C.: Mnist handwritten digit database. ATT Labs. https://yann.lecun.com/exdb/mnist 2 (2010)

  23. Lee, K., Lee, H., Lee, K., Shin, J.: Training confidence-calibrated classifiers for detecting out-of-distribution samples. In: ICLR. Vancouver, CA (2018)

    Google Scholar 

  24. Lee, K., Lee, K., Lee, H., Shin, J.: A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In: Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., Garnett, R. (eds.) NeurIPS, vol. 31, pp. 7167–7177. CAI, Montreal, CA (2018)

    Google Scholar 

  25. Lehmann, D., Ebner, M.: Layer-wise activation cluster analysis of CNNs to detect out-of-distribution samples. In: Farkaš, I., Masulli, P., Otte, S., Wermter, S. (eds.) ICANN 2021. LNCS, vol. 12894, pp. 214–226. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-86380-7_18

    Chapter  Google Scholar 

  26. Lehmann, D., Ebner, M.: Calculating the credibility of test samples at inference by a layer-wise activation cluster analysis of convolutional neural networks. In: Proceedings of the 3rd International Conference on Deep Learning Theory and Applications DeLTA 2022, pp. 34–43. INSTICC, SciTePress, Lisbon, Portugal (2022)

    Google Scholar 

  27. Lehmann, D., Ebner, M.: Subclass-based under sampling for class-imbalanced image classification. In: Proceedings of the 17th International Joint Conference on Computer Vision. Imaging and Computer Graphics Theory and Applications - Volume 5: VISAPP, pp. 493–500. SciTePress, INSTICC (2022)

    Google Scholar 

  28. Li, X., Li, F.: Adversarial examples detection in deep networks with convolutional filter statistics. In: ICCV, pp. 5775–5783. IEEE, Venice, Italy (2017)

    Google Scholar 

  29. Lin, Z., Roy, S.D., Li, Y.: Mood: multi-level out-of-distribution detection. In: CVPR, pp. 15308–15318. IEEE (2021)

    Google Scholar 

  30. Ma, X., et al.: Characterizing adversarial subspaces using local intrinsic dimensionality. In: ICLR, Vancouver, CA (2018)

    Google Scholar 

  31. MacQueen, J.B.: Some methods for classification and analysis of multivariate observations. In: Cam, L.M.L., Neyman, J. (eds.) Berkeley Symposium on Mathematical Statistics and Probability, vol. 1, pp. 281–297. University of California Press (1967)

    Google Scholar 

  32. Madry, A., Makelov, A., Schmidt, L., Tsipras, D., Vladu, A.: Towards deep learning models resistant to adversarial attacks. In: ICLR. Vancouver, CA (2018)

    Google Scholar 

  33. McInnes, L., Healy, J., Melville, J.: UMAP: Uniform manifold approximation and projection for dimension reduction. ArXiv arXiv:1802.03426 (2018)

  34. Meng, D., Chen, H.: Magnet: a two-pronged defense against adversarial examples. In: SIGSAC, pp. 135–147. ACM, Dallas, TX, USA (2017)

    Google Scholar 

  35. Metzen, J.H., Genewein, T., Fischer, V., Bischoff, B.: On detecting adversarial perturbations. In: ICLR, Toulon, France (2017)

    Google Scholar 

  36. Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., Ng, A.Y.: Reading digits in natural images with unsupervised feature learning. In: NIPS Workshop on Deep Learning and Unsupervised Feature Learning (2011)

    Google Scholar 

  37. Nguyen, A., Yosinski, J., Clune, J.: Multifaceted feature visualization: uncovering the different types of features learned by each neuron in deep neural networks. Visualization for Deep Learning workshop. In: International Conference in Machine Learning (2016). arXiv preprint arXiv:1602.03616

  38. Papernot, N., McDaniel, P.: Deep k-nearest neighbors: towards confident, interpretable and robust deep learning. ArXiv arXiv:1803.04765 (2018)

  39. Pearson, K.: LIII. On lines and planes of closest fit to systems of points in space. London, Edinburgh Dublin Philos. Mag. J. Sci. 2(11), 559–572 (1901)

    Google Scholar 

  40. Rousseeuw, P.J.: Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. J. Comput. Appl. Math. 20(1), 53–65 (1987)

    Article  MATH  Google Scholar 

  41. Russakovsky, O., et al.: Imagenet large scale visual recognition challenge. IJCV 115(3), 211–252 (2015)

    Article  MathSciNet  Google Scholar 

  42. Sastry, C.S., Oore, S.: Detecting out-of-distribution examples with gram matrices. In: ICML, vol. 119, pp. 8491–8501. PMLR (2020)

    Google Scholar 

  43. Smith, L.N.: Cyclical learning rates for training neural networks. In: WACV, pp. 464–472. IEEE (2017)

    Google Scholar 

  44. Szegedy, C., et al.: Intriguing properties of neural networks. In: Bengio, Y., LeCun, Y. (eds.) ICLR. Banff, CA (2014)

    Google Scholar 

  45. Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 818–833. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10590-1_53

    Chapter  Google Scholar 

  46. Zhang, H., Dauphin, Y.N., Ma, T.: Fixup initialization: residual learning without normalization. ArXiv arXiv:1901.09321 (2019)

Download references

Author information

Authors and Affiliations

  1. Institut für Mathematik und Informatik, Universität Greifswald, Walther-Rathenau-Straße 47, 17489, Greifswald, Germany

    Daniel Lehmann & Marc Ebner

Authors
  1. Daniel Lehmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marc Ebner
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Daniel Lehmann .

Editor information

Editors and Affiliations

  1. Instituto de Telecomunicações and University of Lisbon, Lisbon, Portugal

    Ana Fred

  2. University of Napoli Federico II, Napoli, Italy

    Carlo Sansone

  3. Ford Motor Company, Commerce Township, MI, USA

    Oleg Gusikhin

  4. Univ Paris-Est Creteil (UPEC), Créteil, France

    Kurosh Madani

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Lehmann, D., Ebner, M. (2023). Reliable Classification of Images by Calculating Their Credibility Using a Layer-Wise Activation Cluster Analysis of CNNs. In: Fred, A., Sansone, C., Gusikhin, O., Madani, K. (eds) Deep Learning Theory and Applications. DeLTA 2022. Communications in Computer and Information Science, vol 1858. Springer, Cham. https://doi.org/10.1007/978-3-031-37317-6_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-37317-6_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-37316-9

  • Online ISBN: 978-3-031-37317-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation