Abstract
Handwritten Chinese character recognition is difficult due to the unstructured and noisy nature of its training examples. There are often too few training examples for a statistical learner like SVM to overcome the noise and extract useful information reliably. Existing prior domain knowledge represents a valuable source of information for classifying handwritten characters. Explanation-based learning (EBL) provides a way to incorporating prior domain knowledge into the learner. The dynamic bias formed by the interaction of domain knowledge with training examples can yield solution knowledge of potential higher quality. Two EBL approaches, one that uses a special feature kernel function in SVM, the other uses a conventional kernel for the SVM but provides additional preference in choosing the classification hyperplane, are reported.
Similar content being viewed by others
References
Bartlett, P., Shawe-Taylor, J.: Generalization performance of support vector machines and other pattern classifiers. In: Schölkopf, B., Burges, C.J.C., Smola, A.J. (eds.) Advances in Kernel Methods: Support Vector Learning. MIT Press, Cambridge (1998)
Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines (2001). http://www.csie.ntu.edu.tw/~cjlin/libsvm
Cristianini, N., Shawe-Taylor, J., Elisseeff, A.: On kernel-target alignment. In: Dietterich, T.G., Becker, S., Ghahramani, Z. (eds.) Advances in Neural Information Processing Systems (NIPS), vol. 14, pp. 367–373. MIT Press, Cambridge (2002)
DeCoste D. and Schölkopf B. (2002). Training invariant support vector machines. Mach. Learn. 46: 161–190
Forsyth D.A. and Ponce J. (2002). Computer Vision—A Modern Approach. Prentice-Hall, Englewood Cliffs
Fung G.M., Mangasarian O.L. and Shavlik J.W. (2003). Knowledge-based nonlinear kernel classifiers. In: Schölkopf, B. and Warmuth, M.K. (eds) Learning Theory and Kernel Machines: 16th Annual Conference on Computational Learning Theory (COLT), Lecture Notes in Computer Science, vol. 2777, pp 102–113. Springer, Heidelberg
Fung G.M., Mangasarian O.L. and Shavlik J.W. (2003). Knowledge-based support vector machine classiers. In: Becker, S.T.S. and Obermayer, K. (eds) Advances in Neural Information Processing Systems (NIPS), vol. 15, pp 521–528. MIT Press, Cambridge
Kandola, J., Shawe-Taylor, J., Cristianini, N.: Optimizing kernel alignment over combinations of kernels. Technical Report NC-TR-02-121, NeuroCOLT (2002)
LeCun Y., Bottou L., Bengio Y. and Haffner P. (1998). Gradient-based learning applied to document recognition. In: Proceedings of the IEEE 86(11): 2278–2324
LeCun, Y., Jackel, L., Bottou, L., Brunot, A., Cortes, C., Denker, J., Drucker, H., Guyon, I., Muller, U., Sackinger, E., Simard, P., Vapnik, V.: Comparison of learning algorithms for handwritten digit recognition. In: Fogelman, F., Gallinari, P. (eds.) International Conference on Artificial Neural Networks, pp. 53–60 (1995)
Lim, S.H., Wang, L.L., DeJong, G.: Explanation-based feature construction. In: International Joint Conference on Artificial Intelligence (IJCAI), pp. 931–936 (2007)
Mangasarian O.L., Shavlik J.W. and Wild E.W. (2004). Knowledge-based kernel approximation. J. Mach. Learn. Res. 5: 1127–1141
Mitchell T. (1997). Machine Learning. McGraw-Hill, New York
Ong, C.S., Smola, A.: Machine learning with hyperkernels. In: Proceedings of the Twentieth International Conference on Machine Learning (ICML) (2003)
Saito T., Yamada H. and Yamamoto K. (1985). On the data base ETL 9 of handprinted characters in JIS Chinese characters and its analysis. IEICE Trans. J68-D(4): 757–764
Schölkopf, B., Burges, C., Vapnik, V.: Incorporating invariances in support vector learning machines. In: von der Malsburg, C., von Seelen, W., Vorbrüggen, J.C., Sendhoff, B. (eds.) Artificial Neural Networks—ICANN ’96, vol. 1112, pp. 47–52 (1996)
Schölkopf B., Simard P., Vapnik V. and Smola A.J. (1998). Prior knowledge in support vector kernels. In: Jordan, M.I., Kearns, M.J., and Solla, S.A. (eds) Advances in Neural Information Processing Systems (NIPS), vol. 10, pp 640–646. MIT Press, Cambridge
Shawe-Taylor J., Bartlett P.L., Williamson R.C. and Anthony M. (1998). Structural risk minimization over data-dependent hierarchies. IEEE Trans. Inform. Theor. 44(5): 1926–1940
Shawe-Taylor J. and Cristianini N. (2002). On the generalisation of soft margin algorithms. IEEE Trans. Inform. Theor. 48(10): 2721–2735
Shi D., Damper R.I. and Gunn S.R. (2003). Off-line handwritten chinese character recognition by radical decomposition. ACM Trans. Asian Lang. Process. 2(1): 27–48
Simard P.Y., Le Cun Y.A., Denker J.S. and Victorri B. (1998). Transformation invariance in pattern recognition—tangent distance and tangent propagation. In: Orr, G.B. and Müller, K.R. (eds) Neural Networks: Tricks of the Trade, Chap. 12., pp 239–274. Springer, Heidelberg
Simard, P.Y., Steinkraus, D., Platt, J.: Best practice for convolutional neural networks applied to visual document analysis. In: International Conference on Document Analysis and Recogntion (ICDAR), pp. 958–962. IEEE Comput. Soc., Los Alamitos (2003)
Sun, Q., DeJong, G.: Explanation-augmented SVM: an approach to incorporating domain knowledge into SVM learning. In: de Raedt, L., Wrobel, S. (eds) Proceedings of the 22nd International Machine Learning Conference. ACM Press, New York (2005)
Teow, L.N., Loe, K.F.: Handwritten digit recognition with a novel vision model that extracts linearly separable features. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, pp. 76–81 (2000)
Vapnik V.N. (1998). Statistical Learning Theory. Wiley, New York
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Sun, Q., Wang, LL., Lim, S.H. et al. Robustness through prior knowledge: using explanation-based learning to distinguish handwritten Chinese characters. IJDAR 10, 175–186 (2007). https://doi.org/10.1007/s10032-007-0053-1
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10032-007-0053-1