Abstract
Existing out-of-distribution (OOD) detection methods are mainly established on the use of output probability of softmax classifiers. However, the classifier-based approach only leverages the relationship between the input data and their associated labels, without making use of the information hidden in the abundant input data at all. Moreover, the inherent normalization characteristic in the softmax function is prone to make the output prediction probability large even for an OOD input. To address these issues, a generative-model-based approach is proposed. Specifically, to make full use of the input data and associated label, we propose to employ variational auto-encoder (VAE) to model the input data and their associated labels simultaneously. Moreover, to alleviate the issue of false large output probability, we transform the one-hot label into multi-label forms and propose to model the label with a multi-label branch in the VAE. Through experimental comparisons, it is verified that the model can effectively improve the OOD detection performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
DeVries, T., Taylor, G.W.: Learning confidence for out-of-distribution detection in neural networks. CoRR, abs/1802.04865 (2018)
Hendrycks, D., Gimpel, K.: A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint arXiv:1610.02136 (2016)
Salakhutdinov, R.R., Hinton, G.E.: Reducing the dimensionality of data with neural networks. Sci. Am. Assoc. Adv. Sci. 313, 504–507 (2006)
Kingma, D.P., Welling, M.: Auto-encoding variational Bayes. In: 2nd International Conference on Learning Representations, ICLR (2014)
Lee, K., Lee, H., Lee, K., Shin, J.: Training confidence-calibrated classifiers for detecting out-of-distribution samples. In: 6th International Conference on Learning Representations (ICLR) (2018)
Lee, K., Lee, K., Lee, H., Shin, J.: A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In: Advances in Neural Information Processing Systems, vol. 31: Annual Conference on Neural Information Processing Systems 2018 (NeurIPS), pp. 7167–7177 (2018)
Liang, S., Li, Y., Srikant, R.: Principled detection of out-of-distribution examples in neural networks. arXiv preprint arXiv:1706.02690 (2017)
Liang, S., Li, Y., Srikant, R.: Enhancing the reliability of out-of-distribution image detection in neural networks. In: 6th International Conference on Learning Representations (ICLR) (2018)
Munawar, A., Vinayavekhin, P., De Magistris, G.: Limiting the reconstruction capability of generative neural network using negative learning. In: 27th IEEE International Workshop on Machine Learning for Signal Processing (MLSP), pp. 1–6 (2017)
Nguyen, A.M., Yosinski, J., Clune, J.: Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp. 427–436 (2015)
Nguyen, K., O’Connor, B.: Posterior calibration and exploratory analysis for natural language processing models. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP, pp. 1587–1598 (2015)
Perera, P., Patel, V.M.: Deep transfer learning for multiple class novelty detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11544–11552 (2019)
Shalev, G., Adi, Y., Keshet, J.: Out-of-distribution detection using multiple semantic label representations. CoRR, abs/1808.06664 (2018)
Tian, K., Zhou, S., Fan, J., Guan, J.: Learning competitive and discriminative reconstructions for anomaly detection. In: The Thirty-Third AAAI Conference on Artificial Intelligence (AAAI), pp. 5167–5174 (2019)
Xia, Y., Cao, X., Wen, F., Hua, G., Sun, J.: Learning discriminative reconstructions for unsupervised outlier removal. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 1511–1519 (2015)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Yan, Z., Su, Q. (2022). OVAE: Out-of-Distribution Detection with Multi-label-enhanced Variational Autoencoders. In: Liao, X., et al. Big Data. BigData 2022. Communications in Computer and Information Science, vol 1496. Springer, Singapore. https://doi.org/10.1007/978-981-16-9709-8_16
Download citation
DOI: https://doi.org/10.1007/978-981-16-9709-8_16
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-16-9708-1
Online ISBN: 978-981-16-9709-8
eBook Packages: Computer ScienceComputer Science (R0)
