Abstract
Incremental learning is a way relevant to human learning that utilizes samples in online sequence. In the paper, we propose an incremental learning method called Incremental Adaptive Learning Vector Quantization (IALVQ) which aims at classifying characters appearing in an online sequence with style consistency in local time periods. Such local consistency is present commonly in document images, in that the characters in a paragraph or text line are printed in the same font or written by the same person. Our IALVQ method updates the prototypes (parameters of classifier) incrementally to adapt to drifted concepts globally while utilize the style consistency locally. For style adaptation, a style transfer mapping (STM) matrix is calculated on a batch of samples of assumed same style. The STM matrix can be used both in training for prototypes updating and in testing for labels prediction. We consider supervised incremental learning and active incremental learning. In the latter way, class labels are attached only to samples that are assigned low confidence by the classifier. In our experiments on handwritten digits in the NIST Special Database 19, we evaluated the classification performance of IALVQ in two scenarios, interleaved test-then-train and style-specific classification. The results show that utilizing local style consistency can improve the accuracies of both two test scenarios, and for both supervised and active incremental learning modes.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Syed, N., Liu, H., Sung, K.: Incremental learning with support vector machines. In: International Joint Conference on Artificial Intelligence, pp. 352–356. Morgan Kaufmann Publishers, Sweden (1999)
Hoi, S.C.H., Wang, J., Zhao, P.: LIBOL: a library for online learning algorithms. J. Mach. Learn. Res. 15(1), 495–499 (2014)
Zhao, P., Hoi, S.C.: OTL: a framework of online transfer learning. In: Proceedings of the 27th International Conference on Machine Learning, pp. 1231–1238 (2010)
Zhang, X.Y., Liu, C.L.: Writer adaptation with style transfer mapping. IEEE Trans. Pattern Anal. Mach. Intell. 35(7), 1773–1787 (2013)
Rosenblatt, F.: The perceptron: a probabilistic model for information storage and organization in the brain. Psychol. Rev. 65(6), 386–408 (1958)
Kirstein, S., Wersing, H., Körner, E.: A biologically motivated visual memory architecture for online learning of objects. Neural Netw. 21(1), 65–77 (2008)
Xu, Y., Shen, F., Zhao, J.: An incremental learning vector quantization algorithm for pattern classification. Neural Comput. Appl. 21(6), 1205–1215 (2012)
Cesa-Bianchi, N., Conconi, A., Gentile, C.: A second-order perceptron algorithm. SIAM J. Comput. 34(3), 640–668 (2005)
Crammer, K., Dredze, M., Kulesza, A.: Multi-class confidence weighted algorithms. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 496–504 (2009)
Ushiku, Y., Hidaka, M., Harada, T.: Three guidelines of online learning for large-scale visual recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3574–3581 (2014)
Gama, J., ŽliobaitėI, B.A., Pechenizkiy, M., Bouchachia, A.: A survey on concept drift adaptation. ACM Comput. Surv. (CSUR) 46(4) (2014)
Sarkar, P., Nagy, G.: Style consistent classification of isogenous patterns. IEEE Trans. Pattern Anal. Mach. Intell. 27(1), 88–98 (2005)
Veeramachaneni, S., Nagy, G.: Adaptive classifiers for multisource OCR. Doc. Anal. Recognit. 6(3), 154–166 (2003)
Veeramachaneni, S., Nagy, G.: Style context with second-order statistics. IEEE Trans. Pattern Anal. Mach. Intell. 27(1), 14–22 (2005)
Huang, Z., Ding, K., Jin, L., et al.: Writer adaptive online handwriting recognition using incremental linear discriminant analysis. In: Proceedings of the Conference on Document Analysis and Recognition, pp. 91–95. IEEE (2009)
Ding, K., Jin, L.: Incremental MQDF learning for writer adaptive handwriting recognition. In: Proceedings of the Conference on Frontiers in Handwriting Recognition (ICFHR), pp. 559–564 (2010)
Kohonen, T., Hynninen, J., Kangas, J., Laaksonen, J., Torkkola, K.: LVQ PAK: the learning vector quantization program package. Technical report, Helsinki University of Technology (1995)
Kuncheva, L.I., Bezdek, J.C.: Nearest prototype classification: clustering, genetic algorithms, or random search? IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.) 28(1), 160–164 (1998)
Jin, X.B., Liu, C.L., Hou, X.: Regularized margin-based conditional log-likelihood loss for prototype learning. Pattern Recognit. 43(7), 2428–2438 (2010)
Bishop, C.M.: Pattern recognition and Machine Learning. Springer, New York (2006)
Grother, P.: Handprinted forms and character database, NIST special database 19. Technical report and CDROM (1995)
Liu, C.L., Sako, H., Fujisawa, H.: Performance evaluation of pattern classifiers for handwritten character recognition. Int. J. Doc. Anal. Recognit. 4(3), 191–204 (2002)
Duchi, J., Hazan, E., Singer, Y.: Adaptive subgradient methods for online learning and stochastic optimization. J. Mach. Learn. Res. 12, 2121–2159 (2011)
Acknowledgments
This work has been supported in part by the National Basic Research Program of China (973 Program) under Grant 2012CB316302 and the Strategic Priority Research Program of the CAS under Grant XDB02060009.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Shen, YY., Liu, CL. (2016). Incremental Learning Vector Quantization for Character Recognition with Local Style Consistency. In: Liu, CL., Hussain, A., Luo, B., Tan, K., Zeng, Y., Zhang, Z. (eds) Advances in Brain Inspired Cognitive Systems. BICS 2016. Lecture Notes in Computer Science(), vol 10023. Springer, Cham. https://doi.org/10.1007/978-3-319-49685-6_21
Download citation
DOI: https://doi.org/10.1007/978-3-319-49685-6_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-49684-9
Online ISBN: 978-3-319-49685-6
eBook Packages: Computer ScienceComputer Science (R0)