Cited By
View all- Zhang PHuang ZXu XBai G(2024)Effective and Robust Adversarial Training Against Data and Label CorruptionsIEEE Transactions on Multimedia10.1109/TMM.2024.339467726(9477-9488)Online publication date: 2-May-2024
Multi-head Siamese Prototype Learning against both Data and Label Corruption
Article No.: 61, Pages 1 - 7
Abstract
The training of the Deep Neural Network (DNN) has been seriously challenged by insidious noise in the dataset, including noise in raw data and errors in annotations. Existing methods usually limit their efforts to the defense of one particular kind of noise, which would be powerless when facing the coexistence of various noise. To deal with it, we propose a novel Multi-head Siamese Prototype Learning (MSPL) method to promote discriminative features and representative prototypes by modeling invariance in samples and sieving out incorrectness in labels. More specifically, a multi-head Siamese network structure is constructed, where prototype learning with the multi-consistency constraint is performed to improve the resilience of the model to noise. Under this regime, adversarial contrastive learning is performed to train the model with the dynamically generated vicious adversarial examples, further enhancing the invariant predictive ability against data noise. At the same time, to deal with label noise, an effective multi-granularity sample selection strategy is designed to filter out noisy labels by measuring the error distribution in both global and local (i.e., class-specific) perspectives. Semi-supervised learning is accordingly conducted to train the model with the resulting labelled data (i.e., data with clean labels) and unlabelled data (i.e., data with noisy labels). Extensive experiments on benchmarks demonstrate the effectiveness of the proposed method in the extremely noisy learning environment.
References
[1]
Devansh Arpit, Stanislaw Jastrzebski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S Kanwal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, 2017. A closer look at memorization in deep networks. In International conference on machine learning. PMLR, 233–242.
[2]
Battista Biggio and Fabio Roli. 2018. Wild patterns: Ten years after the rise of adversarial machine learning. Pattern Recognition 84 (2018), 317–331.
[3]
Zhi Chen, Jingjing Li, Yadan Luo, Zi Huang, and Yang Yang. 2020. Canzsl: cycle-consistent adversarial networks for zero-shot learning from natural language. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 874–883.
[4]
Zhi Chen, Sen Wang, Jingjing Li, and Zi Huang. 2020. Rethinking generative zero-shot learning: an ensemble learning perspective for recognising visual patches. In Proceedings of the ACM International Conference on Multimedia. 3413–3421.
[5]
Hao Cheng, Zhaowei Zhu, Xingyu Li, Yifei Gong, Xing Sun, and Yang Liu. 2020. Learning with instance-dependent label noise: a sample sieve approach. In Proceedings of the International Conference on Learning Representations.
[6]
Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc V Le. 2018. Autoaugment: learning augmentation policies from data. arXiv preprint arXiv:1805.09501 (2018).
[7]
Ji Feng, Qi-Zhi Cai, and Zhi-Hua Zhou. 2019. Learning to confuse: generating training time adversarial data with auto-encoder. In Proceedings of the International Conference on Neural Information Processing Systems. 11994–12004.
[8]
Liam Fowl, Micah Goldblum, Ping-yeh Chiang, Jonas Geiping, Wojciech Czaja, and Tom Goldstein. 2021. Adversarial Examples Make Strong Poisons. Advances in Neural Information Processing Systems 34 (2021), 30339–30351.
[9]
Aritra Ghosh, Himanshu Kumar, and PS Sastry. 2017. Robust loss functions under label noise for deep neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence. 1919–1925.
[10]
Jacob Goldberger and Ehud Ben-Reuven. 2017. Training deep neural-networks using a noise adaptation layer. In Proceedings of the International Conference on Learning Representations.
[11]
Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor W Tsang, and Masashi Sugiyama. 2018. Co-teaching: robust training of deep neural networks with extremely noisy labels. In Proceedings of the International Conference on Neural Information Processing Systems. 8536–8546.
[12]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 770–778.
[13]
Hanxun Huang, Xingjun Ma, Sarah Monazam Erfani, James Bailey, and Yisen Wang. 2020. Unlearnable Examples: Making Personal Data Unexploitable. In International Conference on Learning Representations.
[14]
Lu Jiang, Zhengyuan Zhou, Thomas Leung, Li-Jia Li, and Li Fei-Fei. 2018. Mentornet: learning data-driven curriculum for very deep neural networks on corrupted labels. In Proceedings of the International Conference on Machine Learning. 2304–2313.
[15]
Di Jin, Jiayi Shi, Rui Wang, Yawen Li, Yuxiao Huang, and Yu-Bin Yang. 2023. Trafformer: unify time and space in traffic prediction. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 8114–8122.
[16]
Nazmul Karim, Mamshad Nayeem Rizve, Nazanin Rahnavard, Ajmal Mian, and Mubarak Shah. 2022. UNICON: combating label noise through uniform selection and contrastive learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 9676–9686.
[17]
Alex Krizhevsky, Geoffrey Hinton, 2009. Learning multiple layers of features from tiny images. (2009).
[18]
Samuli Laine and Timo Aila. 2016. Temporal ensembling for semi-supervised learning. arXiv preprint arXiv:1610.02242 (2016).
[19]
Junnan Li, Richard Socher, and Steven CH Hoi. 2020. Dividemix: learning with noisy labels as semi-supervised learning. In Proceedings of the International Conference on Learning Representations.
[20]
Yawen Li, Liu Yang, Bohan Yang, Ning Wang, and Tian Wu. 2019. Application of interpretable machine learning models for the intelligent decision. Neurocomputing 333 (2019), 273–283.
[21]
Yawen Li, Ye Yuan, Yishu Wang, Xiang Lian, Yuliang Ma, and Guoren Wang. 2020. Distributed multimodal path queries. IEEE Transactions on Knowledge and Data Engineering 34, 7 (2020), 3196–3210.
[22]
Yang Liu and Hongyi Guo. 2020. Peer loss functions: learning from noisy labels without knowing noise rates. In Proceedings of the International Conference on Machine Learning. 6226–6236.
[23]
Xingjun Ma, Yisen Wang, Michael E Houle, Shuo Zhou, Sarah Erfani, Shutao Xia, Sudanthi Wijewickrema, and James Bailey. 2018. Dimensionality-driven learning with noisy labels. In Proceedings of the International Conference on Machine Learning. 3355–3364.
[24]
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2018. Towards Deep Learning Models Resistant to Adversarial Attacks. In International Conference on Learning Representations.
[25]
Duc Tam Nguyen, Chaithanya Kumar Mummadi, Thi Phuong Nhung Ngo, Thi Hoai Phuong Nguyen, Laura Beggel, and Thomas Brox. 2019. Self: learning to filter noisy labels with self-ensembling. In Proceedings of the International Conference on Learning Representations.
[26]
Giorgio Patrini, Alessandro Rozza, Aditya Krishna Menon, Richard Nock, and Lizhen Qu. 2017. Making deep neural networks robust to label noise: a loss correction approach. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1944–1952.
[27]
Yuntao Qu, Shasha Mo, and Jianwei Niu. 2021. DAT: training deep networks robust to label-noise by matching the feature distributions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 6821–6829.
[28]
Mengye Ren, Wenyuan Zeng, Bin Yang, and Raquel Urtasun. 2018. Learning to reweight examples for robust deep learning. In Proceedings of the International Conference on Machine Learning. 4334–4343.
[29]
Yanyao Shen and Sujay Sanghavi. 2019. Learning with bad training data via iterative trimmed loss minimization. In Proceedings of the International Conference on Machine Learning. 5739–5748.
[30]
Hwanjun Song, Minseok Kim, Dongmin Park, Yooju Shin, and Jae-Gil Lee. 2022. Learning from noisy labels with deep neural networks: A survey. IEEE Transactions on Neural Networks and Learning Systems (2022).
[31]
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research 15, 1 (2014), 1929–1958.
[32]
Sainbayar Sukhbaatar, Joan Bruna, Manohar Paluri, Lubomir Bourdev, and Rob Fergus. 2015. Training convolutional networks with noisy labels. In Proceedings of the International Conference on Learning Representations.
[33]
Daiki Tanaka, Daiki Ikami, Toshihiko Yamasaki, and Kiyoharu Aizawa. 2018. Joint optimization framework for learning with noisy labels. In Proceedings of the IEEE conference on computer vision and pattern recognition. 5552–5560.
[34]
Lue Tao, Lei Feng, Jinfeng Yi, Sheng-Jun Huang, and Songcan Chen. 2021. Better safe than sorry: preventing delusive adversaries with adversarial training. Advances in Neural Information Processing Systems 34 (2021), 16209–16225.
[35]
Antti Tarvainen and Harri Valpola. 2017. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In Proceedings of the 31st International Conference on Neural Information Processing Systems. 1195–1204.
[36]
Yisen Wang, Xingjun Ma, Zaiyi Chen, Yuan Luo, Jinfeng Yi, and James Bailey. 2019. Symmetric cross entropy for robust learning with noisy labels. In Proceedings of the IEEE International Conference on Computer Vision. 322–330.
[37]
Xiaobo Xia, Tongliang Liu, Bo Han, Nannan Wang, Mingming Gong, Haifeng Liu, Gang Niu, Dacheng Tao, and Masashi Sugiyama. 2020. Part-dependent label noise: towards instance-dependent label noise. In Proceedings of the International Conference in Neural Information Processing Systems. 7597–7610.
[38]
Guanhua Ye, Tong Chen, Yawen Li, Lizhen Cui, Quoc Viet Hung Nguyen, and Hongzhi Yin. 2023. Heterogeneous Collaborative Learning for Personalized Healthcare Analytics Via Messenger Distillation. IEEE Journal of Biomedical and Health Informatics (2023).
[39]
Guanhua Ye, Hongzhi Yin, Tong Chen, Hongxu Chen, Lizhen Cui, and Xiangliang Zhang. 2021. FENet: a frequency extraction network for obstructive sleep apnea detection. IEEE Journal of Biomedical and Health Informatics 25, 8 (2021), 2848–2856.
[40]
Guanhua Ye, Hongzhi Yin, Tong Chen, Miao Xu, Quoc Viet Hung Nguyen, and Jiangning Song. 2022. Personalized on-device e-health analytics with decentralized block coordinate descent. IEEE Journal of Biomedical and Health Informatics 26, 6 (2022), 2778–2786.
[41]
Da Yu, Huishuai Zhang, Wei Chen, Jian Yin, and Tie-Yan Liu. 2022. Availability attacks create shortcuts. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2367–2376.
[42]
Xingrui Yu, Bo Han, Jiangchao Yao, Gang Niu, Ivor Tsang, and Masashi Sugiyama. 2019. How does disagreement help generalization against label corruption?. In Proceedings of the International Conference on Machine Learning. 7164–7173.
[43]
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. 2021. Understanding deep learning (still) requires rethinking generalization. Commun. ACM 64, 3 (2021), 107–115.
[44]
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. 2017. mixup: Beyond empirical risk minimization. arXiv preprint arXiv:1710.09412 (2017).
[45]
Peng-Fei Zhang, Jiasheng Duan, Zi Huang, and Hongzhi Yin. 2021. Joint-teaching: learning to refine knowledge for resource-constrained unsupervised cross-modal retrieval. In Proceedings of the ACM International Conference on Multimedia. 1517–1525.
[46]
Peng-Fei Zhang, Yang Li, Zi Huang, and Xin-Shun Xu. 2021. Aggregation-based graph convolutional hashing for unsupervised cross-modal retrieval. IEEE Transactions on Multimedia 24 (2021), 466–479.
[47]
Peng-Fei Zhang, Yang Li, Zi Huang, and Hongzhi Yin. 2021. Privacy protection in deep multi-modal retrieval. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 634–643.
[48]
Peng-Fei Zhang, Yadan Luo, Zi Huang, Xin-Shun Xu, and Jingkuan Song. 2021. High-order nonlocal Hashing for unsupervised cross-modal retrieval. World Wide Web 24 (2021), 563–583.
[49]
Zhilu Zhang and Mert R Sabuncu. 2018. Generalized cross entropy loss for training deep neural networks with noisy labels. In Proceedings of the International Conference on Neural Information Processing Systems. 8792–8802.
Index Terms
- Multi-head Siamese Prototype Learning against both Data and Label Corruption
Recommendations
Robust Learning with Adversarial Perturbations and Label Noise: A Two-Pronged Defense Approach
MMAsia '22: Proceedings of the 4th ACM International Conference on Multimedia in AsiaDespite great success achieved, deep learning methods are vulnerable to noise in the training dataset, including adversarial perturbations and annotation noise. These harmful factors significantly influence the learning process of deep models, leading ...
Few-shot partial multi-label learning via prototype rectification
AbstractPartial multi-label learning (PML) models the scenario where each training sample is annotated with a candidate label set, among which only a subset corresponds to the ground-truth labels. Existing PML approaches generally promise that there are ...
How to handle noisy labels for robust learning from uncertainty
AbstractMost deep neural networks (DNNs) are trained with large amounts of noisy labels when they are applied. As DNNs have the high capacity to fit any noisy labels, it is known to be difficult to train DNNs robustly with noisy labels. These ...
Comments
Information & Contributors
Information
Published In
December 2023
745 pages
ISBN:9798400702051
DOI:10.1145/3595916
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 01 January 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
- ARC Centre of Excellence for Children and Families over the Life Course
- ARC Discovery Projects
Conference
MMAsia '23
Sponsor:
Acceptance Rates
Overall Acceptance Rate 59 of 204 submissions, 29%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 100Total Downloads
- Downloads (Last 12 months)61
- Downloads (Last 6 weeks)5
Reflects downloads up to 28 Feb 2025
Other Metrics
Citations
Cited By
View all- Zhang PHuang ZXu XBai G(2024)Effective and Robust Adversarial Training Against Data and Label CorruptionsIEEE Transactions on Multimedia10.1109/TMM.2024.339467726(9477-9488)Online publication date: 2-May-2024
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format