Abstract
Chinese calligraphy is a valuable cultural heritage belonging to the world. It is liked by many people, and our mission is to endeavor to pursue calligraphy and make contributions to the business with technical means. The automatic recognition of the styles of calligraphy by image processing techniques has important meaning in arts collection and auction, etc. Traditional feature operators have some drawbacks that leave room for modern methods like convolutional neural network (CNN). However, most of the studies focus on the classification of five basic fonts that is somewhat different from styles. In this paper, four kinds of styles belonging to standard font are classified with a novel CNN structure where two squeeze-and-excitation modules that emphasize informative feature maps and suppress useless features are embedded after convolution layers, and a Haar transform layer that fuses the features is imposed before softmax layer. Experiment result shows the significance of the proposed structure over other networks in both font and style classification.
Similar content being viewed by others
References
Yao, C., Cheng, G.: Approximative Bayes optimality linear discriminant analysis for Chinese handwriting character recognition. Neurocomputing 207, 346–353 (2016)
Yang, W., Jin, L., Xie, Z., et al.: Improved deep convolutional neural network for online handwritten Chinese character recognition using domain-specific knowledge. In: 2015 13th international conference on document analysis and recognition (ICDAR). IEEE, pp. 551–555 (2015)
Gao, P., Wu, J., Yuan, L., et al.: Fast Chinese calligraphic character recognition with large-scale data. Multimed. Tools Appl. 74(17), 7221–7238 (2015)
Gao, P., Wu, J., Yuan, L., et al.: Fast Chinese calligraphic character recognition with large-scale data. Multimed. Tools Appl. 74(17), 7221–7238 (2015)
Tao, D., Lin, X., Jin, L., et al.: Principal component 2-D long short-term memory for font recognition on single Chinese characters. IEEE Trans. Cybern. 46(3), 756 (2016)
Zhang, X., Nagy, G.: Style comparisons in calligraphy. Doc. Recognit. Retr. XIX 8297(2), 263–271 (2012)
Zhang, Y., Liu, Y., He, J., et al.: Recognition of calligraphy style based on global feature defontor. In: IEEE International Conference on Multimedia and Expo. IEEE, pp. 1–6 (2013)
Gao, P., Wu, J., Lin, Y., et al.: Fast image-based chinese calligraphic character retrieval on large scale data. In: Acm/ieee-Cs Joint Conference on Digital Libraries. IEEE Press, pp. 211–218 (2014)
Shi, Z., Xu, B., Zheng, X., et al.: A Chinese character structure preserved denoising method for Chinese tablet calligraphy document images based on KSVD dictionary learning. Multimed. Tools Appl. 76(13), 1–16 (2016)
Imdad, A., Bres, S., Eglin, V., et al.: Writer identification using steered hermite features and SVM. In: International Conference on Document Analysis and Recognition. IEEE, pp. 839–843 (2017)
Vásquez, J.L., Travieso, C.M., Alonso, J.B.: Using calligraphies features for off line writer identification. In: International Carnahan Conference on Security Technology. IEEE, pp. 1–6 (2014)
Sun, R., Lian, Z., Tang, Y., et al.: Aesthetic visual quality evaluation of Chinese handwritings. In: International Conference on Artificial Intelligence. AAAI Press, pp. 2510–2516 (2015)
Hou, S., Xu, P.: An algorithm of calligraphy beautification based on improved velocity and width model. In: International Conference on Computer Science. IEEE, pp. 124–127 (2015)
Liu, L., Xia, W., Jin, L., et al.: A Kai style contour beautification method for Chinese handwriting characters. In: IEEE International Conference on Systems Man and Cybernetics. IEEE, pp. 3644–3649 (2010)
Xiao, J., Xiao, J., Xiao, J.: Automatic generation of large-scale handwriting fonts via style learning. In: SIGGRAPH ASIA 2016 Technical Briefs. ACM, p. 12 (2016)
Zhang, S.L., Xie, Z.K., Qi, Y.: Transformation of Chinese calligraphy style based on feature matrix editing. In; International Conference on Computer Science and Network Technology. IEEE, pp. 159–162 (2016)
Jie, C., Ma, Y.H.: Novel Chinese calligraphy style generation based on curve analogy. International Workshop on Intelligent Systems and Applications. IEEE, pp. 1–3 (2011)
Zhuang, Y., Lu, W., Wu, J.: Latent Style Model: Discovering Writing Styles for Calligraphy Works. Academic Press, Inc., Cambridge (2009)
Huang, S., Zhong, Z., Jin, L., et al.: DropRegion training of inception font network for high-performance Chinese font recognition[J]. Pattern Recogn. 77, 395–411 (2018)
Wang, X., Girshick, R., Gupta, A., et al.: Non-local Neural Networks, p. 10 (2017). arXiv preprint arXiv:1711.07971
Hu, J., Shen, L., Sun, G.: Squeeze-and-Excitation Networks, p. 7 (2017). arXiv preprint arXiv:1709.01507
Szegedy, C., Ioffe, S., Vanhoucke, V. et al: Inception-v4, inception-resnet and the impact of residual connections on learning. In: AAAI, vol. 4, no 12 (2017)
Huang, G., Liu, Z., Van Der Maaten, L., et al.: Densely connected convolutional networks[C]. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4700–4708 (2017)
Xie, S., Girshick, R., Dollár, P., et al.: Aggregated residual transformations for deep neural networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp. 5987–5995 (2017)
Haar, A.: Zur theorie der orthogonalen funktionensysteme. Math. Ann. 69(3), 331–371 (1910)
Daugman, J.G.: Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. JOSA A 2(7), 1160–1169 (1985)
Deepu, V., Madhvanath, S., Ramakrishnan, A.G.: Principal component analysis for online handwritten character recognition. In: International Conference on Pattern Recognition, vol. 2, pp. 327–330. IEEE (2004)
Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005. CVPR 2005, vol. 1, pp. 886–893. IEEE (2005)
Lécun, Y., Bottou, L., Bengio, Y., et al.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (2001)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. Commun. ACM 60(2), 2012 (2013)
Simonyan, K., Zisserman, A.,: Very deep convolutional networks for large-scale image recognition[J]. arXiv preprint arXiv:1409.1556 (2014)
Szegedy, C., Liu, W., Jia, Y., et al.: Going deeper with convolutions. In: Computer Vision and Pattern Recognition, pp. 1–9. IEEE (2015)
He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition[C]. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778 (2016)
Deng, J., Dong, W., Socher, R., et al.: ImageNet: a large-scale hierarchical image database. IEEE Conference on IEEE Computer Vision and Pattern Recognition, 2009. CVPR 2009, pp. 248–255 (2009)
Xu, K., Ba, J., Kiros, R.,: Show, attend and tell: neural image caption generation with visual attention[C]. In: International conference on machine learning, pp. 2048-2057 (2015)
Zhang, H., Goodfellow, I., Metaxas, D., et al.: Self-Attention Generative Adversarial Networks (2018). arXiv preprint arXiv:1805.08318
Oyallon, E., Mallat, S.: Deep roto-translation scattering for object classification. In: Computer Vision and Pattern Recognition. IEEE, pp. 2865–2873 (2015)
Oyallon, E., Belilovsky, E., Zagoruyko, S.: Scaling the scattering transform: deep hybrid networks[C]. In: Proceedings of the IEEE international conference on computer vision, pp. 5618–5627 (2017)
Nair, V., Hinton, G.E.: Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th International Conference on Machine Learning (ICML-10), pp. 807–814 (2010)
Ioffe, S., Szegedy, C.: Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift (2015). arXiv preprint arXiv:1502.03167
Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagating errors. Nature 323(6088), 533 (1986)
Abadi, M., Barham, P., Chen, J., et al.: Tensorflow: a system for large-scale machine learning. In: OSDI, vol. 16, pp. 265–283 (2016)
Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995)
Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. J. Mach. Learn. Res. 9, 249–256 (2010)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization[J]. arXiv preprint arXiv:1412.6980 (2014)
Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9(2605), 2579–2605 (2008)
Acknowledgement
This work is support by National Key Research and Development Plan (No. 2017YFB1402103), Xi’an Science and Technology Bureau Project (201805037YD15CG21(6)). The authors are grateful for the anonymous reviewers for their significant comments.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zhang, J., Guo, M. & Fan, J. A novel CNN structure for fine-grained classification of Chinese calligraphy styles. IJDAR 22, 177–188 (2019). https://doi.org/10.1007/s10032-019-00324-1
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10032-019-00324-1