research-article

An analysis of binarization ground truthing

Author:

Elisa H. Barney SmithAuthors Info & Claims

DAS '10: Proceedings of the 9th IAPR International Workshop on Document Analysis Systems

Pages 27 - 34

https://doi.org/10.1145/1815330.1815334

Published: 09 June 2010 Publication History

Abstract

The accuracy of a binarization algorithm is often calculated relative to a ground truth image. Except for synthetically generated images, no ground truth image exists. Evaluating binarization on real images is preferred. The ground truthing between and among different operators is compared. Four direct metrics were used. The variability of the results of five different automatic binarization algorithms were compared to that of manual ground truth results. Significant variability in the ground truth results was found.

References

[1]

E. H. Barney Smith, L. Likforman-Sulem, and J. Darbon. Effect of pre-processing on binarization. In Proceedings SPIE Electronic Imaging Document Recognition and Retrieval, San Jose, California, USA, January 2010.

[2]

T. Ellis. Performance metrics and methods for tracking in surveillance. In 3rd IEEE International Workshop on Performance Evaluation of Tracking and Surveillance (PETS), pages 26--31, 2002.

[3]

B. Gatos, K. Ntirogiannis, and I. Pratikakis. ICDAR 2009 document image binarization contest (DIBCO 2009). In Proceedings of the Tenth International Conference on Document Analysis and Recognition, ICDAR-2009, pages 1375--1382, Barcelona, Spain, July 2009.

Digital Library

[4]

B. Gatos, I. Pratikakis, and S. J. Perantonis. Adaptive degraded document image binarization. Pattern Recognition, 39:317--327, 2006.

Digital Library

[5]

L. Goldmann, T. Adamek, P. Vajda, M. Karaman, R. Mörzinger, E. Galmar, T. Sikora, N. E. O£Connor, T. Ha-Minh, T. Ebrahimi, P. Schallauer, and B. Huet. Towards fully automatic image segmentation evaluation. In Advanced Concepts for Intelligent Vision Systems, volume 5259, pages 566--577, 2008.

Digital Library

[6]

W. Niblack. An Introduction to Digital Image Processing. Prentice- Hall, Englewood Cliffs, NJ, 1986.

Digital Library

[7]

K. Ntirogiannis, B. Gatos, and I. Pratikakis. An objective evaluation methodology for document image binarization techniques. In Proceedings of the 8th International Workshop on Document Analysis Systems (DAS'08), pages 217--224, Nara Japan, September 2008.

Digital Library

[8]

N. Otsu. A threshold selection method from gray level histograms. IEEE Trans. Syst. Man Cybern., 9:62--66, 1979.

[9]

E. Saund, J. Lind, and P. Sarkar. PixLabeler: User interface for pixel-level labeling of elements in document images. In Proceedings of the Tenth International Conference on Document Analysis and Recognition (ICDAR'09), pages 646--650, Barcelona, Spain, July 2009.

Digital Library

[10]

J. Sauvola and M. Pietikäinen. Adaptive document image binarization. Pattern Recognition, 33:225--236, 2000.

[11]

M. Sezgin and B. Sankur. Survey over image thresholding techniques and quantitative performance evaluation. Journal of Electronic Imaging, 13(1):146--168, 2004.

[12]

P. Stathis, E. Kavallieratou, and N. Papamarkos. An evaluation technique for binarization algorithms. Journal of Universal Computer Science, 14(18):3011--3030, 2008.

[13]

D. P. Young and J. M. Ferryman. Pets metrics: On-line performance evaluation service. In Proceedings - 2nd Joint IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance, VS-PETS, volume 2005, pages 317--324, Beijing, China, 2005.

Digital Library

[14]

Øivand Due Trier and A. K. Jain. Goal-directed evaluation of binarization methods. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(12):1191--1201, December 1995.

Digital Library

[15]

Øivind Due Trier and T. Taxt. Evaluation of binarization methods for document images. Transactions on Pattern Analysis and Machine Intelligence, 17(3), March 1995.

Digital Library

Cited By

Tensmeyer CMartinez T(2020)Historical Document Image Binarization: A ReviewSN Computer Science10.1007/s42979-020-00176-11:3Online publication date: 16-May-2020
https://doi.org/10.1007/s42979-020-00176-1
Santosh KSantosh K(2018)Graphics Recognition and Validation ProtocolDocument Image Analysis10.1007/978-981-13-2339-3_3(35-51)Online publication date: 19-Sep-2018
https://doi.org/10.1007/978-981-13-2339-3_3
Fedorchuk MLamiroy B(2017)Statistic metrics for evaluation of binary classifiers without ground-truth2017 IEEE First Ukraine Conference on Electrical and Computer Engineering (UKRCON)10.1109/UKRCON.2017.8100414(1066-1071)Online publication date: May-2017
https://doi.org/10.1109/UKRCON.2017.8100414
Show More Cited By

Index Terms

An analysis of binarization ground truthing
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Interest point and salient region detections
      2. Image and video acquisition

Recommendations

Effect of "Ground Truth" on Image Binarization
DAS '12: Proceedings of the 2012 10th IAPR International Workshop on Document Analysis Systems

Image binarization has a large effect on the rest of the document image analysis processes in character recognition. Algorithm development is still a major focus of research. Evaluation of image binarization has been done by comparison of the result of ...
An Efficient Ground Truthing Tool for Binarization of Historical Manuscripts
ICDAR '13: Proceedings of the 2013 12th International Conference on Document Analysis and Recognition

For the purpose of facilitating benchmark contributions for binarization methods, a new fast ground truthing approach, called the PhaseGT, is proposed. This approach is used for building the 1st groundtruthed Persian Heritage Image Binarization Dataset (...
A statistical tool based binarization method for document images
Abstract
Binarization of document images has great importance in several applications like historical document restoration, Optical Character Recognition (OCR). It is a challenging task due to small difference between foreground and background pixel ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

DAS '10: Proceedings of the 9th IAPR International Workshop on Document Analysis Systems

June 2010

490 pages

ISBN:9781605587738

DOI:10.1145/1815330

General Chairs:
David Doermann
University of Maryland, College Park
,
Venu Govindaraju
University at Buffalo, SUNY
,
Daniel Lopresti
Lehigh University
,
Prem Natarajan
Raytheon BBN Technologies

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 June 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

DAS '10

DAS '10: The Eighth IAPR International Workshop on Document Analysis Systems

June 9 - 11, 2010

Massachusetts, Boston, USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

30
Total Citations
View Citations
285
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)1

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Tensmeyer CMartinez T(2020)Historical Document Image Binarization: A ReviewSN Computer Science10.1007/s42979-020-00176-11:3Online publication date: 16-May-2020
https://doi.org/10.1007/s42979-020-00176-1
Santosh KSantosh K(2018)Graphics Recognition and Validation ProtocolDocument Image Analysis10.1007/978-981-13-2339-3_3(35-51)Online publication date: 19-Sep-2018
https://doi.org/10.1007/978-981-13-2339-3_3
Fedorchuk MLamiroy B(2017)Statistic metrics for evaluation of binary classifiers without ground-truth2017 IEEE First Ukraine Conference on Electrical and Computer Engineering (UKRCON)10.1109/UKRCON.2017.8100414(1066-1071)Online publication date: May-2017
https://doi.org/10.1109/UKRCON.2017.8100414
Fedorchuk MLamiroy B(2017)Binary Classifier Evaluation Without Ground Truth2017 Ninth International Conference on Advances in Pattern Recognition (ICAPR)10.1109/ICAPR.2017.8593175(1-6)Online publication date: Dec-2017
https://doi.org/10.1109/ICAPR.2017.8593175
Mishra AAlahari KJawahar C(2017)Unsupervised refinement of color and stroke features for text binarizationInternational Journal on Document Analysis and Recognition10.1007/s10032-017-0283-920:2(105-121)Online publication date: 1-Jun-2017
https://dl.acm.org/doi/10.1007/s10032-017-0283-9
Kesiman MPrum SSunarya IBurie JOgier J(2015)An analysis of ground truth binarized image variability of palm leaf manuscripts2015 International Conference on Image Processing Theory, Tools and Applications (IPTA)10.1109/IPTA.2015.7367134(229-233)Online publication date: Nov-2015
https://doi.org/10.1109/IPTA.2015.7367134
Kesiman MPrum SBurie JOgier J(2015)An initial study on the construction of ground truth binarized images of ancient palm leaf manuscriptsProceedings of the 2015 13th International Conference on Document Analysis and Recognition (ICDAR)10.1109/ICDAR.2015.7333843(656-660)Online publication date: 23-Aug-2015
https://dl.acm.org/doi/10.1109/ICDAR.2015.7333843
Nafchi HMoghaddam RCheriet M(2014)Phase-Based Binarization of Ancient Document Images: Model and ApplicationsIEEE Transactions on Image Processing10.1109/TIP.2014.232245123:7(2916-2930)Online publication date: Jul-2014
https://doi.org/10.1109/TIP.2014.2322451
Ramírez-Ortegón MRamírez-Ramírez LMessaoud IMärgner VCuevas ERojas R(2014)A model for the gray-intensity distribution of historical handwritten documents and its application for binarizationInternational Journal on Document Analysis and Recognition10.1007/s10032-013-0212-517:2(139-160)Online publication date: 1-Jun-2014
https://dl.acm.org/doi/10.1007/s10032-013-0212-5
Lamiroy B(2014)Interpretation, Evaluation and the Semantic Gap ... What if We Were on a Side-Track?Graphics Recognition. Current Trends and Challenges10.1007/978-3-662-44854-0_17(221-233)Online publication date: 19-Sep-2014
https://doi.org/10.1007/978-3-662-44854-0_17
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten