Abstract
This paper studies the recognition of oracle character, the earliest known hieroglyphs in China. Essentially, oracle character recognition suffers from the problem of data limitation and imbalance. Recognizing the oracle characters of extremely limited samples, naturally, should be taken as the few-shot learning task. Different from the standard few-shot learning setting, our model has only access to large-scale unlabeled source Chinese characters and few labeled oracle characters. In such a setting, meta-based or metric-based few-shot methods are failed to be efficiently trained on source unlabeled data; and thus the only possible methodologies are self-supervised learning and data augmentation. Unfortunately, the conventional geometric augmentation always performs the same global transformations to all samples in pixel format, without considering the diversity of each part within a sample. Moreover, to the best of our knowledge, there is no effective self-supervised learning method for few-shot learning. To this end, this paper integrates the idea of self-supervised learning in data augmentation. And we propose a novel data augmentation approach, named Orc-Bert Augmentor pre-trained by self-supervised learning, for few-shot oracle character recognition. Specifically, Orc-Bert Augmentor leverages a self-supervised BERT model pre-trained on large unlabeled Chinese characters datasets to generate sample-wise augmented samples. Given a masked input in vector format, Orc-Bert Augmentor can recover it and then output a pixel format image as augmented data. Different mask proportion brings diverse reconstructed output. Concatenated with Gaussian noise, the model further performs point-wise displacement to improve diversity. Experimentally, we collect two large-scale datasets of oracle characters and other Chinese ancient characters for few-shot oracle character recognition and Orc-Bert Augmentor pre-training. Extensive experiments on few-shot learning demonstrate the effectiveness of our Orc-Bert Augmentor on improving the performance of various networks in the few-shot oracle character recognition.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Antoniou, A., Storkey, A.: Assume, augment and learn: unsupervised few-shot meta-learning via random labels and data augmentation. arXiv preprint arXiv:1902.09884 (2019)
Bhunia, A.K., Das, A., Bhunia, A.K., Kishore, P.S.R., Roy, P.P.: Handwriting recognition in low-resource scripts using adversarial learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019
Chen, Z., Fu, Y., Wang, Y.-X., Ma, L., Liu, W., Hebert, M.: Image deformation meta-networks for one-shot learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8680–8689 (2019)
Cubuk, E.D., Zoph, B., Mane, D., Vasudevan, V., Le, Q.V.: Autoaugment: learning augmentation strategies from data. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 113–123 (2019)
Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
DeVries, T., Taylor, G.W.: Dataset augmentation in feature space. arXiv preprint arXiv:1702.05538, 2017
Finn, C., Abbeel, P., Levine, S.: Model-agnostic meta-learning for fast adaptation of deep networks. ArXiv, abs/1703.03400 (2017)
Guo, J., Wang, C., Roman-Rangel, E., Chao, H., Rui, Y.: Building hierarchical representations for oracle character and sketch recognition. IEEE Trans. Image Process. 25(1), 104–118 (2015)
Gupta, A., Vedaldi, A., Zisserman, A.: Synthetic data for text localisation in natural images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2315–2324 (2016)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016)
Huang, G., Liu, Z., Weinberger, K.Q.: Densely connected convolutional networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2261–2269 (2017)
Huang, S., Wang, H., Liu, Y., Shi, X., Jin, L.: Obc306: a large-scale oracle bone character recognition dataset. In: 2019 International Conference on Document Analysis and Recognition (ICDAR), pp. 681–688 (2019)
Jaderberg, M., Simonyan, K., Vedaldi, A., Zisserman, A.: Reading text in the wild with convolutional neural networks. Int. J. Comput. Vis. 116(1), 1–20 (2016)
Meng, L., Lyu, B., Zhang, Z., Aravinda, C.V., Kamitoku, N., Yamazaki, K.: Oracle bone inscription detector based on SSD. In: Cristani, M., Prati, A., Lanz, O., Messelodi, S., Sebe, N. (eds.) ICIAP 2019. LNCS, vol. 11808, pp. 126–136. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30754-7_13
Keightley. D.N.: Graphs, words, and meanings: three reference works for shang oracle-bone studies, with an excursus on the religious role of the day or sun (1997)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)
Lemley, J., Bazrafkan, S., Corcoran, P.: Smart augmentation learning an optimal data augmentation strategy. IEEE Access 5, 5858–5869 (2017)
Li, Z., Zhou, F., Chen, F., Li, H.: Meta-SGD: learning to learn quickly for few shot learning. ArXiv, abs/1707.09835 (2017)
Lin, H., Fu, Y., Xue, X., Jiang, Y.-G.: Sketch-bert: learning sketch bidirectional encoder representation from transformers by self-supervised learning of sketch gestalt. In: The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020
Liu, B., Wang, X., Dixit, M., Kwitt, R., Vasconcelos, N.: Feature space transfer for data augmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9090–9098 (2018)
Liu, G., Gao, F.: Oracle-bone inscription recognition based on deep convolutional neural network. JCP 13(12), 1442–1450 (2018)
Luo, C., Zhu, Y., Jin, L., Wang, Y.: Learn to augment: joint data augmentation and network optimization for text recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13746–13755 (2020)
Mariani, G., Scheidegger, F., Istrate, R., Bekas, C., Malossi, C.: Bagan: data augmentation with balancing GAN. arXiv preprint arXiv:1803.09655 (2018)
Mayr, M., Stumpf, M., Nikolaou, A., Seuret, M., Maier, A., Christlein, V.: Spatio-temporal handwriting imitation. arXiv preprint arXiv:2003.10593 (2020)
Meng, L.: Recognition of oracle bone inscriptions by extracting line features on image processing. In: ICPRAM, pp. 606–611 (2017)
Ravi, S., Larochelle, H.: Optimization as a model for few-shot learning (2016)
Ren, M., et al.: Meta-learning for semi-supervised few-shot classification. arXiv preprint arXiv:1803.00676 (2018)
Shrivastava, A., Pfister, T., Tuzel, O., Susskind, J., Wang, W., Webb, R.: Learning from simulated and unsupervised images through adversarial training. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2107–2116 (2017)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)
Vinyals, O., Blundell, C., Lillicrap, T., Wierstra, D., et al.: Matching networks for one shot learning. In: Advances in Neural Information Processing Systems, pp. 3630–3638 (2016)
Wang, Y., Xu, C., Liu, C., Zhang, L., Fu, Y.: Instance credibility inference for few-shot learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12836–12845 (2020)
Xing, J., Liu, G., Xiong, J.: Oracle bone inscription detection: a survey of oracle bone inscription detection based on deep learning algorithm. In: Proceedings of the International Conference on Artificial Intelligence, Information Processing and Cloud Computing, pp. 1–8 (2019)
Yosinski, J., Clune, J., Bengio, Y., Lipson, H.: How transferable are features in deep neural networks? In: Advances in Neural Information Processing Systems, pp. 3320–3328 (2014)
Zagoruyko, S., Komodakis, N.: Wide residual networks. arXiv preprint arXiv:1605.07146, 2016
Zhang, X.-Y., Bengio, Y., Liu, C.-L.: Online and offline handwritten Chinese character recognition: a comprehensive study and new benchmark. Pattern Recogn. 61, 348–360 (2017)
Zhang, Y., Liang, S., Nie, S., Liu, W., Peng, S.: Robust offline handwritten character recognition through exploring writer-independent features under the guidance of printed data. Pattern Recogn. Lett. 106, 20–26 (2018)
Zhang, Y., Nie, S., Liu, W., Xu, X., Zhang, D., Shen, H.T.: Sequence-to-sequence domain adaptation network for robust text image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2740–2749 (2019)
Zhang, Y.-K., Zhang, H., Liu, Y.-G., Yang, Q., Liu, C.-Li.: Oracle character recognition by nearest neighbor classification with deep metric learning. In: 2019 International Conference on Document Analysis and Recognition (ICDAR), pp. 309–314. IEEE (2019)
Zhong, Z., Zhang, X.-Y., Yin, F., Liu, C.-L.: Handwritten Chinese character recognition with spatial transformer and deep residual networks. In: 2016 23rd International Conference on Pattern Recognition (ICPR), pp. 3440–3445. IEEE (2016)
Acknowledgement
This work was supported in part by NSFC Projects (U62076067), Science and Technology Commission of Shanghai Municipality Projects (19511120700, 19ZR1471800).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Han, W., Ren, X., Lin, H., Fu, Y., Xue, X. (2021). Self-supervised Learning of Orc-Bert Augmentor for Recognizing Few-Shot Oracle Characters. In: Ishikawa, H., Liu, CL., Pajdla, T., Shi, J. (eds) Computer Vision – ACCV 2020. ACCV 2020. Lecture Notes in Computer Science(), vol 12627. Springer, Cham. https://doi.org/10.1007/978-3-030-69544-6_39
Download citation
DOI: https://doi.org/10.1007/978-3-030-69544-6_39
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-69543-9
Online ISBN: 978-3-030-69544-6
eBook Packages: Computer ScienceComputer Science (R0)