Skip to main content

Advertisement

Springer Nature Link
Log in

Confidence modeling for handwriting recognition: algorithms and applications

  • Original Paper
  • Published:
International Journal of Document Analysis and Recognition (IJDAR) Aims and scope Submit manuscript

Abstract

Confidence scoring can assist in determining how to use imperfect handwriting-recognition output. We explore a confidence-scoring framework for post-processing recognition for two purposes: deciding when to reject the recognizer's output, and detecting when to change recognition parameters e.g., to relax a word-set constraint. Varied confidence scores, including likelihood ratios and posterior probabilities, are applied to an Hidden-Markov-Model (HMM) based on-line recognizer. Receiver-operating characteristic curves reveal that we successfully reject 90% of word recognition errors while rejecting only 33% of correctly-recognized words. For isolated digit recognition, we achieve 90% correct rejection while limiting false rejection to 13%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Chigier, B.: Rejection and keyword spotting algorithms for a directory assistance city name recognition application. In: Proceedings of ICASSP 1992: IEEE International Conference on Acoustics, Speech and Signal Processing, vol. 2, pp. 93–96 San Francisco, California, U.S.A. (March, 1992)

  2. Evermann, G., Woodland, P.C.: Large vocabulary decoding and confidence estimation using word posterior probabilities. In: Proceedings of ICASSP 2000: IEEE International Conference on Acoustics, Speech, and Signal Processing, pp. 2366–2369 Istanbul, Turkey, (June 2000)

  3. Gorski, N.: Optimizing error-reject trade off in recognition systems. In: Proceedings of the 4th International Conference on Document Analysis and Recognition (ICDAR), vol. 2, pp. 1092–1096. Ulm, Germany (August 18–20, 1997)

  4. Guyon, I., Schomaker, L., Plamondon, R., Liberman, M., Janet, S.: UNIPEN project of on-line data exchange and recognizer benchmarks. In: Proceedings of the 12th International Conference on Pattern Recognition (ICPR '94), Jerusalem, pp. 29–33 (October 1994)

  5. Hazen, T.J., Burianek, T., Polifroni, J., Seneff, S.: Recognition confidence scoring for use in speech understanding systems. In: Proceedings ISCA Tutorial and Research Workshop: ASR2000, Paris, France, (September 2000)

  6. Kemp, T., Schaaf, T.: Estimating confidence using word lattices. In: Proceedings of the 5th European Conference on Speech Communication and Technology (Eurospeech '97), Rhodes, Greece, 22–25 September, pp. 827–830 (1997)

  7. Koo, M.-W., Lee, C.-H., Juang, B.-H.: Speech recognition and utterance verification based on a generalized confidence score. IEEE Trans. Speech Audio Proc. 9(8), pp. 821–832 (2001)

    Article  Google Scholar 

  8. Lifchitz, A., Maire, F.: A fast lexically constrained viterbi algorithm for on-line handwriting recognition. In: Proceedings of the 7th International Workshop on Frontiers in Handwriting Recognition (IWFHR-7), Amsterdam, The Netherlands, 11–13 September, pp. 313–322 (2000)

  9. Liu, C.-L., Nakagawa, M.: Precise candidate selection for large character set recognition by confidence evaluation. IEEE Trans. Pattern Anal. Mach. Intell. 22(6), pp. 636–641 (2000)

    Article  Google Scholar 

  10. Lleida, E., Rose, R.C.: Efficient decoding and training procedures for utterance verification in continuous speech recognition. In: Proceedings of ICASSP 1996: IEEE International Conference on Acoustics, Speech, and Signal Processing, Atlanta, Georgia, U.S.A., 7–10 May, v.ol. 1, pp. 507–510 (1996)

  11. Maison, B., Gopinath, R.: Robust confidence annotation and rejection for continuous speech recognition. In: Proceedings of ICASSP 2001: IEEE International Conference on Acoustics, Speech, and Signal Processing Salt Lake City, Utah, U.S.A., May (2001)

  12. Mangu, L., Brill, E., Stolcke, A.: Finding Consensus among Words: Lattice-Based Word Error Minimization. In: Proceedings of Eurospeech '99, Budapest, Hungary, 5–9 September, vol. 1, pp. 495–498 (1999)

  13. Moreno, P.J., Logan, B., Raj, B.: A boosting approach for confidence scoring. In: Proceedings of Eurospeech01: 7th European Conference on Speech Communication and Technology, Aalborg, Denmark, 3–7 September (2001)

  14. Nathan, K.S., Beigi, H.S.M. Subrahmonia, J., Clary, G.J., Maruyama, H.: Real-time on-line unconstrained handwriting recognition using statistical methods. In: Proceedings of ICASSP 1995: IEEE International Conference on Acoustics, Speech, and Signal Processing, Detroit, Michigan, U.S.A., 8–12 May, vol. 4, pp. 2619–2622 (1995)

  15. Perrone, M.P., Cooper, L.: When networks disagree: ensemble method for neural networks. In: Mammone, R.J. (ed.), Artificial Neural Networks for Speech and Vision, ch. 10. Chapman-Hall, London (1993)

  16. Perrone, M.P.: Improving regression estimation: averaging methods for variance reduction with extensions to general convex measure optimization. Ph.D. Thesis, Brown University Institute for Brain and Neural Systems (May, 1993)

  17. Pitrelli, J.F., Ratzlaff, E.H.: Quantifying the contribution of language modeling to writer-independent on-line handwriting recognition. In: Proceedings of the 7th International Workshop on Frontiers in Handwriting Recognition (IWFHR-7). Amsterdam, The Netherlands, 11–13 September, pp. 383–392 (2000)

  18. Pitrelli, J.F., Subrahmonia, J., Maison, B.: Toward island-of-reliability-driven very-large-vocabulary on-line handwriting recognition using character confidence scoring. In: Proceedings of ICASSP 2001: IEEE International Conference on Acoustics, Speech, and Signal Processing, Salt Lake City, Utah, U.S.A., 7–11 May, (2001)

  19. Plamondon, R., Srihari, S.N.: On-Line and Off-Line Handwriting Recognition: A Comprehensive Survey. IEEE Trans. Pattern Anal. Mach. Intell. 22(1), pp. 63–84 (2000)

    Article  Google Scholar 

  20. Sarkar, P., Baird, H.S., Henderson, J.: Triage of OCR output using “confidence” scores. In: Proceedings of the SPIE/IS&T 2002 Document Recognition & Retrieval Conference, San Jose, CA, U.S.A., January (2002)

  21. Schlüter, R., Wessel, F., Ney, H.: Speech recognition using context conditional word posterior probabilities. In: Proceedings of ICSLP 2000: International Conference on Spoken Language Processing, Beijing, P.R.C., 16–20 October (2000)

  22. Senior, A.W., Robinson, A.J.: An off-line cursive handwriting recognition system. IEEE Tran. Pattern Anal. Mach. Intell., 20(3), pp. 309–321 (1998)

    Article  Google Scholar 

  23. Stolcke, A., König, Y., Weintraub, M.: Explicit word error minimization in N-best list rescoring. In: Proceedings of Eurospeech '97, Rhodes, Greece, 22–25 September, vol. 1, pp. 163–166 (1997)

  24. Subrahmonia, J., Nathan, K., Perrone, M.: Writer dependent recognition of on-line unconstrained handwriting. In: Proceedings of ICASSP 1996: IEEE International Conference on Acoustics, Speech, and Signal Processing, Atlanta, Georgia, U.S.A., 7–10 May, vol. 6, pp. 3478–3481 (1996)

  25. Weintraub, M., Beaufays, F., Rivlin, Z., Konig, Y., Stolcke, A.: Neural-network based measures of confidence for word recognition. In: Proceedings of ICASSP 1997: IEEE International Conference on Acoustics, Speech, and Signal Processing, Munich, Germany, 21–24 April, vol. 2, pp. 887–890 (1997)

  26. Wessel, F., Macherey, K., Schlüter, R.: Using word probabilities as confidence measures. In: Proceedings of ICASSP 1998: IEEE International Conference on Acoustics, Speech, and Signal Processing, Seattle, Washington, U.S.A., vol. 1, pp. 225–228 (May, 1998)

  27. Wilpon, J.G., Rabiner, L.R., Lee, C.-H., Goldman, E.R.: Automatic recognition of keywords in unconstrained speech using hidden markov models. IEEE Trans. Acoustics, Speech, Signal Proc. 38(11), pp. 1870–1878 (1993)

    Article  Google Scholar 

Download references

Author information

Author notes

  1. Jayashree Subrahmonia

    Present address: IBM Silicon Valley Laboratory, 555 Bailey Avenue, San Jose, CA, 95141, USA

Authors and Affiliations

  1. IBM T. J. Watson Research Center, 1101 Kitchawan Road, Yorktown Heights, NY, 10598, USA

    John F. Pitrelli, Jayashree Subrahmonia & Michael P. Perrone

Authors
  1. John F. Pitrelli
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Jayashree Subrahmonia
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Michael P. Perrone
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to John F. Pitrelli.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Pitrelli, J.F., Subrahmonia, J. & Perrone, M.P. Confidence modeling for handwriting recognition: algorithms and applications. IJDAR 8, 35–46 (2006). https://doi.org/10.1007/s10032-005-0011-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10032-005-0011-8

Keywords