Optimizing GANs using Relativistic Discriminator with Margin Losses for Semi-supervised Learning
Pages 173 - 180
Abstract
We introduce a novel framework, termed RMGANs, that merges Relativistic Generative Adversarial Networks (RGANs) with Margin Losses. This framework capitalizes on the strengths of both RGANs’ discriminators, which assess the realism of real and fake data, and the advantages of Margin Losses, well-known for their ability to create distinct class separation. We apply this combination to the problem of semi-supervised learning (SSL). Our study delves into the architecture of RMGANs, provides a mathematical analysis to assess the impact of margin utilization on RMGAN losses, and offers guidance on choosing hyper-parameters. We also conduct experiments across the MNIST and CIFAR-10 datasets. The empirical results clearly demonstrate RMGANs’ effectiveness in achieving higher accuracy compared to the state-of-the-art work (MarginGAN) in the SSL fashion.
References
[1]
A. Agrawala. 1970. Learning with a probabilistic teacher. IEEE Transactions on Information Theory 16, 4 (1970), 373–379. https://doi.org/10.1109/TIT.1970.1054472
[2]
Avrim Blum and Tom Mitchell. 1998. Combining Labeled and Unlabeled Data with Co-Training. In Proceedings of the Eleventh Annual Conference on Computational Learning Theory (Madison, Wisconsin, USA) (COLT’ 98). Association for Computing Machinery, New York, NY, USA, 92–100. https://doi.org/10.1145/279943.279962
[3]
Yanwen Chong, Yun Ding, Qing Yan, and Shaoming Pan. 2020. Graph-based semi-supervised learning: A review. Neurocomputing 408 (2020), 216–230. https://doi.org/10.1016/j.neucom.2019.12.130
[4]
Jiankang Deng, Jia Guo, Jing Yang, Niannan Xue, Irene Kotsia, and Stefanos Zafeiriou. 2022. ArcFace: Additive Angular Margin Loss for Deep Face Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 10 (2022), 5962–5979. https://doi.org/10.1109/TPAMI.2021.3087709
[5]
Jinhao Dong and Tong Lin. 2019. MarginGAN: Adversarial Training in Semi-Supervised Learning. In Advances in Neural Information Processing Systems 32 (NIPS 2019). 10440–10449.
[6]
S. Fralick. 1967. Learning to recognize patterns without a teacher. IEEE Transactions on Information Theory 13, 1 (1967), 57–64. https://doi.org/10.1109/TIT.1967.1053952
[7]
Ian Goodfellow 2014. Generative Adversarial Nets. In Proc. NIPS 2014. Montreal, Canada, 2672–2680.
[8]
Alexia Jolicoeur-Martineau. 2019. The Relativistic Discriminator: A Key Element Missing from Standard GAN. In Proc. ICLR 2019. New Orleans, USA.
[9]
Samuli Laine and Timo Aila. 2017. Temporal Ensembling for Semi-Supervised Learning. arxiv:1610.02242 [cs.NE]
[10]
Dong-Hyun Lee. 2013. Pseudo-Label : The Simple and Efficient Semi-Supervised Learning Method for Deep Neural Networks. https://api.semanticscholar.org/CorpusID:18507866
[11]
Chongxuan Li, Kun Xu, Jun Zhu, and Bo Zhang. 2017. Triple Generative Adversarial Nets. In Advances in Neural Information Processing Systems, Vol. 31. Curran Associates, Inc.
[12]
Weiyang Liu, Yandong Wen, Zhiding Yu, Ming Li, Bhiksha Raj, and Le Song. 2018. SphereFace: Deep Hypersphere Embedding for Face Recognition. arXiv abs/1704.08063v4 (2018).
[13]
Cuong V. Nguyen, Tien-Dung Cao, Tram Truong-Huu, Khanh N. Pham, and Binh T. Nguyen. 2021. An Empirical Study on GANs with Margin Cosine Loss and Relativistic Discriminator. arxiv:2110.11293 [cs.CV]
[14]
Mohammad Peikari, Sherine Salama, Sharon Nofech-Mozes, and Anne L. Martel. 2018. A Cluster-then-label Semi-supervised Learning Approach for Pathology Image Classification. Scientific Reports 8, 1 (2018), 7193. https://doi.org/10.1038/s41598-018-24876-0
[15]
Alec Radford, Luke Metz, and Soumith Chintala. 2016. Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks. In 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds.). http://arxiv.org/abs/1511.06434
[16]
H. Scudder. 1965. Probability of error of some adaptive pattern-recognition machines. IEEE Transactions on Information Theory 11, 3 (1965), 363–371. https://doi.org/10.1109/TIT.1965.1053799
[17]
Jost Tobias Springenberg. 2016. Unsupervised And Semi-supervised Learning With Categorical Generative Adversarial Networks. In International Conference on Learning Representations.
[18]
Hao Wang 2018. CosFace: Large Margin Cosine Loss for Deep Face Recognition. In Proc. CVPR 2018. Utah, USA.
Index Terms
- Optimizing GANs using Relativistic Discriminator with Margin Losses for Semi-supervised Learning
Recommendations
Maximum margin semi-supervised learning with irrelevant data
Semi-supervised learning (SSL) is a typical learning paradigms training a model from both labeled and unlabeled data. The traditional SSL models usually assume unlabeled data are relevant to the labeled data, i.e., following the same distributions of ...
Multiview Semi-Supervised Learning with Consensus
Obtaining high-quality and up-to-date labeled data can be difficult in many real-world machine learning applications. Semi-supervised learning aims to improve the performance of a classifier trained with limited number of labeled data by utilizing the ...
Inductive Semi-supervised Multi-Label Learning with Co-Training
KDD '17: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data MiningIn multi-label learning, each training example is associated with multiple class labels and the task is to learn a mapping from the feature space to the power set of label space. It is generally demanding and time-consuming to obtain labels for training ...
Comments
Information & Contributors
Information
Published In
December 2023
1058 pages
ISBN:9798400708916
DOI:10.1145/3628797
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 07 December 2023
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
SOICT 2023
SOICT 2023: The 12th International Symposium on Information and Communication Technology
December 7 - 8, 2023
Ho Chi Minh, Vietnam
Acceptance Rates
Overall Acceptance Rate 147 of 318 submissions, 46%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 35Total Downloads
- Downloads (Last 12 months)15
- Downloads (Last 6 weeks)0
Reflects downloads up to 01 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format