Skip to main content
SpringerLink
Log in

Style-Neutralized Pattern Classification Based on Adversarially Trained Upgraded U-Net

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

Traditional machine learning approaches usually hold the assumption that data for model training and in real applications are created following the identical and independent distribution (i.i.d.). However, several relevant research topics have demonstrated that such condition may not always describe the real scenarios. One particular case is that the patterns are equipped with diverse and changeable style information. In this paper, a novel classification framework named Style Neutralization Generative Adversarial Classifier (SN-GAC), based on an upgraded U-Net architecture, and trained adversarially with the Generative Adversarial Network (GAN) framework, is introduced to accomplish the classification in such disparate and inconsistent data information case. The generative model in SN-GAC neutralizes style information from the original style-discriminative patterns (style-source) by building the mapping function from them to their style-free counterparts (corresponding standard examples, standard-target). A well-learned generator in the SN-GAC framework is capable of producing the targeted style-neutralized data (generated-target), satisfying the i.i.d. condition. Additionally, SN-GAC is trained adversarially, where an independent discriminator is used to surveil and supervise the training progress of the above-mentioned generator by distinguishing between the real and the generated. Simultaneously, an auxiliary classifier is also embedded in the discriminator to assign the correct class label of both the real and generated data. This process proves effective to aid the generator to produce high-quality human-readable style-neutralized patterns. It will then be further fine-tuned for the sake of promoting the final classification performance. Extensive experiments have adequately demonstrated the effectiveness of the proposed SN-GAC framework: it outperforms several relevant state-of-the-art baselines on two empirical data sets in the non-i.i.d. data classification task.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. The neural network–based SN-GAC can be readily extended to classification with large class numbers.

  2. Although the proposed SN-GAC model is evaluated only with dataset specifying groups of style patterns, it can also be applied in a more generalized way for any style-inconsistent classification situation.

  3. Such style inconsistency can be found when data are created by multiple sources, while each source is generating examples with a special kind of style information. The stylistic tendency caused differs from different sources.

  4. The source code of the SN-GAC model can be referred to via the online Github service: https://github.com/falconjhc/SN-GAC

  5. Paired input is not evaluated for conventional baselines in the “Experiments” section since style-neutralization cannot be achieved with traditional approaches.

  6. As suggested in [16], G is trained once after D-C is learned for five times to guarantee the best Wasserstein distance estimation at the current training progress.

  7. Although there exists a style shift between the cursive characters in testing and the isolated examples in training for a specific writer, as will be demonstrated in the“??” section, the writing style seems to be similar since they are written by the identical individual.

  8. The experiment setting as well as all the experimental results except the proposed SN-GAC model is referred to [2, 3].

  9. The “Heiti” font.

References

  1. Jiang H, Huang K, Zhang R. Field support vector regression. Proceedings of the International Conference on Neural Information Processing. Cham: Springer; 2017.

  2. Huang K, Jiang H, Zhang X. Field support vector machines. Proceedings of the 1st International Conference on Internet of Things and Machine Learning. ACM; 2017.

  3. Huang K, Jiang H, Zhang X. Field support vector machines. IEEE Transactions on Emerging Topics in Computational Intelligence 2017;1.6:454–63.

    Article  Google Scholar 

  4. Zhang X-Y, Huang K, Liu C-L. Pattern field classification with style normalized transformation. Twenty-Second International Joint Conference on Artificial Intelligence; 2011.

  5. Liu Z-Y, Qiao H, Yang X, Hoi SCH. Graph matching by simplified convex-concave relaxation procedure. Int J Comput Vis. 2014;109(3).

  6. Liu Z-Y, Qiao H. GNCCP-Graduated nonconvexityand concavity procedure. IEEE Trans Pattern Anal Mach Intell. 2013;36(6).

  7. Gourier N, Hall D, Crowley JL. Estimating face orientation from robust detection of salient facial features. ICPR International Workshop on Visual Observation of Deictic Gestures; 2004.

  8. Liu C-L, Yin F, Wang D-H, Wang Q-F. CASIA online and offline Chinese handwriting databases. 2011 International Conference on Document Analysis and Recognition. IEEE; 2011.

  9. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y. 2014. Generative adversarial nets. Advances in Neural Information Processing Systems.

  10. Isola P, Zhu J-Y, Zhou T, Efros AA. Image-to-image translation with conditional adversarial networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2017.

  11. Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation. Proceedings of the International Conference on Medical Image Computing and Computer-assisted Intervention. Cham: Springer; 2015.

  12. Lvmin Z, Ji Y, Lin X, Liu C. Style transfer for anime sketches with enhanced residual u-net and auxiliary classifier gan. 4th IAPR Asian Conference on Pattern Recognition. IEEE; 2017.

  13. Odena A, Olah C, Shlens J. Conditional image synthesis with auxiliary classifier gans. Proceedings of the 34th International Conference on Machine Learning; 2017.

  14. Mirza M, Osindero S. Conditional generative adversarial nets. arXiv:1411.1784.

  15. Salimans T, Goodfellow I, Zaremba W, Cheung V, Radford A, Chen X. Improved techniques for training gans. Advances in Neural Information Processing Systems. 2016.

  16. Gulrajani I, Ahmed F, Arjovsky M, Dumoulin V, Courville AC. Improved training of wasserstein gans. Advances in Neural Information Processing Systems. 2017.

  17. Yoshida Y, Miyato T. 2017. Spectral norm regularization for improving the generalizability of deep learning. arXiv:1705.10941.

  18. Miyato T, Kataoka T, Koyama M, Yoshida Y. Spectral normalization for generative adversarial networks. arXiv:1802.05957. 2018.

  19. Antoniou A, Storkey A, Edwards H. Data augmentation generative adversarial networks. arXiv:1711.04340. 2017.

  20. Shrivastava A, Pfister T, Tuzel O, Susskind J, Wang W, Webb R. Learning from simulated and unsupervised images through adversarial training. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2017.

  21. Wang T-C, Liu M-Y, Zhu J-Y, Tao A, Kautz J, Catanzaro B. High-resolution image synthesis and semantic manipulation with conditional gans. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2018.

  22. Taigman Y, Polyak A, Wolf L. Unsupervised cross-domain image generation. arXiv:1611.02200. 2016.

  23. Zhu J-Y, Park T, Isola P, Efros AA. Unpaired image-to-image translation using cycle-consistent adversarial networks. Proceedings of the IEEE International Conference on Computer Vision; 2017.

  24. Jiang H, Yang G, Huang K, Zhang R. W-Net: one-shot arbitrary-style chinese character generation with deep neural networks. Proceedings of the International Conference on Neural Information Processing; 2018.

  25. Evgeniou T, Pontil M. Regularized multi-task learning. Proceedings of the tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM; 2004.

  26. Sarkar P, Nagy G. Style consistent classification of isogenous patterns. IEEE Trans Pattern Anal Mach Intell 2005;27:1.

    Article  Google Scholar 

  27. Tenenbaum JB, Freeman WT. Separating style and content with bilinear models. Neural Comput 2000;12: 6.

    Article  Google Scholar 

  28. Zhong G, Huang K. 2018. Semi-supervised learning: background, applications and future directions. Nova Science Publishers Inc.

  29. Hsu C-W, Lin C-J. A comparison of methods for multiclass support vector machines. IEEE Trans Neural Netw 2002;2:13.

    Google Scholar 

  30. He K, et al. Identity mappings in deep residual networks. European conference on computer vision. Cham: Springer; 2016.

    Google Scholar 

  31. Huang K, Hussain A, Wang Q, Zhang R. Deep learning: fundamentals, theory, and applications. Springer; 2019. ISBN-13: 978-3540794516.

  32. Jiang Y, Lian Z, Tang Y, Xiao J. 2017. DCFOnt: an end-to-end deep chinese font generation system. SIGGRAPH Asia 2017 Technical Briefs. ACM.

  33. Johnson M, Schuster M, Le QV, Krikun M, Wu Y, Chen Z, Thorat N, et al. 2017. Google’s multilingual neural machine translation system: enabling zero-shot translation. Transactions of the Association for Computational Linguistics 5.

  34. Tian Y. 2017. Zi2Zi-Tensorflow. https://kaonashi-tyc.github.io/2017/04/06/zi2zi.html.

  35. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 2014;1:15.

    MathSciNet  MATH  Google Scholar 

  36. Cortes C, Vapnik V. Support-vector networks. Mach Learn 1995;20:3.

    MATH  Google Scholar 

  37. Cate H, Dalvi F, Hussain Z. Deepface: face generation using deep learning. Proceedings of the IEEE conference on computer vision and pattern recognition; 2014.

  38. Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems; 2012.

  39. Jing X-Y, Wong H-S, Zhang D. Face recognition based on 2D Fisherface approach. Pattern Recogn 2006; 4:39.

    MATH  Google Scholar 

  40. Kimura F, Takashina K, Tsuruoka S, Miyake Y. 1987. Modified quadratic discriminant functions and the application to Chinese character recognition. IEEE Transactions on Pattern Analysis & Machine Intelligence 1.

  41. Abadi M, Barham P, Chen J, Chen Z, Davis A, Dean J, Devin M, et al. Tensorflow: a system for large-scale machine learning. 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16); 2016.

  42. Wall ME, Rechtsteiner A, Rocha LM. Singular value decomposition and principal component analysis. A practical approach to microarray data analysis. Boston: Springer; 2003.

  43. Boser BE, Guyon IM, Vapnik VN. A training algorithm for optimal margin classifiers. Proceedings of the Fifth Annual Workshop on Computational Learning Theory. ACM; 1992.

  44. Internet Archive. GB 2312-1980: information technology—Chinese ideogram coded character set for information interchange (basic set). https://archive.org/details/GB2312-1980/page/n17.

Download references

Acknowledgments

Acknowledgment goes to Ms. Zijun CUI who offered assistance in designing several of the illustrations in this paper.

Funding

The work reported here was partially supported by the following: National Natural Science Foundation of China under grant no. 61876155; Natural Science Fund for Colleges and Universities in Jiangsu Province under grant no. 17KJD520010; Suzhou Science and Technology Program under grant no. SYG2-01712, SZS201613; Jiangsu University Natural Science Research Programme under grant no. 17KJB-520041; Key Program Special Fund in XJTLU (KSF-A-01).

Author information

Authors and Affiliations

  1. Department of EEE, Xi’an Jiaotong - Liverpool University, SIP, Suzhou, 215123, Jiangsu, People’s Republic of China

    Haochuan Jiang & Kaizhu Huang

  2. Department of MS, Xi’an Jiaotong - Liverpool University, SIP, Suzhou, 215123, Jiangsu, People’s Republic of China

    Rui Zhang

  3. Cyber and Cognitive Big Data Lab, Edinburgh Napier University, Scotland, UK

    Amir Hussain

Authors
  1. Haochuan Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kaizhu Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rui Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Amir Hussain
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kaizhu Huang.

Ethics declarations

Conflict of interests

The authors declare that they have no conflict of interest.

Ethical Approval

This article does not contain any studies with human participants performed by any of the authors.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jiang, H., Huang, K., Zhang, R. et al. Style-Neutralized Pattern Classification Based on Adversarially Trained Upgraded U-Net. Cogn Comput 13, 845–858 (2021). https://doi.org/10.1007/s12559-019-09660-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-019-09660-0

Keywords

Search

Navigation