Skip to main content
SpringerLink
Log in

NCERT5K-IITRPR: A Benchmark Dataset for Non-textual Component Detection in School Books

  • Conference paper
  • First Online:
Document Analysis Systems (DAS 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13237))

Included in the following conference series:

  • 1666 Accesses

Abstract

The STEM subjects books heavily rely on Non-textual Components (NTCs) such as charts, geometric figures, and equations to demonstrate the underlying complex concepts. However, the accessibility of STEM subjects for Blind and Visually Impaired (BVIP) students is a primary concern, especially in developing countries such as India. BVIP uses assistive technologies (ATs) like optical character recognition (OCR) and screen readers for reading/writing purposes. While parsing, such ATs skip NTCs and mainly rely on alternative texts to describe these visualization components. Integration of effective and automated document layout parsing frameworks for extracting data from non-textual components of digital documents are required with existing ATs for making these NTCs accessible. Although, the primary concern is the absence of an adequately annotated textbook dataset on which layout recognition and other vision-based frameworks can be trained. To improve the accessibility and automated parsing of such books, we introduce a new NCERT5K-IITRPR dataset of National Council of Educational Research and Training (NCERT) school books. Twenty-three annotated books covering more than 5000 pages from the eighth to twelve standards have been considered. The NCERT label objects are structurally different from the existing document layout analysis (DLA) dataset objects and contain diverse label categories. We benchmark the NCERT5K-IITRPR dataset with multiple object detection methods. A systematic analysis of detectors shows the label complexity and fine-tuning necessity of the NCERT5K-IITRPR dataset. We hope that our dataset helps in improving the accessibility of NCERT Books for BVIP students.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://www.who.int/news-room/fact-sheets/detail/blindness-and-visual-impairment.

  2. 2.

    https://ncert.nic.in/pdf/accessibility/6.pdf.

  3. 3.

    https://ncert.nic.in/textbook.php.

  4. 4.

    https://pypi.org/project/pdf2image/.

  5. 5.

    https://github.com/tzutalin/labelImg.

  6. 6.

    https://github.com/ultralytics/yolov5.

References

  1. Vignesh, D., Gupta, N., Kalaivani, M., Goswami, A.K., Nongkynrih, B., Gupta, S.K.: Prevalence of visual impairment and its association with vision-related quality of life among elderly persons in a resettlement colony of Delhi. J. Fam. Med. Prim. Care 8(4), 1432 (2019)

    Article  Google Scholar 

  2. Gothwal, V.K., Kodavati, K., Subramanian, A.: Life in lockdown: impact of COVID-19 lockdown measures on the lives of visually impaired school-age children and their families in India. Ophthalmic Physiol. Optics. 42(2), 301–310 (2021)

    Article  Google Scholar 

  3. Tomar, G., Garg, V.: Making steam accessible for inclusive classroom. Glob. J. Enterp. Inf. Syst. 12(4), 94–101 (2020)

    Google Scholar 

  4. Dey, S., Vidhya, Y., Bhushan, S., Neerukonda, M.: Creating an accessible technology ecosystem for learning science and math: a case of visually impaired children in Indian schools (2019)

    Google Scholar 

  5. Jelas, Z.M., Ali, M.M.: Inclusive education in Malaysia: policy and practice. Int. J. Inclusive Educ. 18(10), 991–1003 (2014)

    Article  Google Scholar 

  6. Lamichhane, K.: Teaching students with visual impairments in an inclusive educational setting: a case from Nepal. Int. J. Incl. Educ. 21(1), 1–13 (2017)

    Article  MathSciNet  Google Scholar 

  7. Asamoah, E., Ofori-Dua, K., Cudjoe, E., Abdullah, A., Nyarko, J.A.: Inclusive education: perception of visually impaired students, students without disability, and teachers in Ghana. SAGE Open 8(4), 2158244018807791 (2018)

    Article  Google Scholar 

  8. Bansal, A., Garg, H., Jadhav, N., Kumar, S., Balakrishnan, M.: RAVI: reading assistant for visually impaired

    Google Scholar 

  9. Gao, L., Yi, X., Jiang, Z., Hao, L., Tang, Z.: ICDAR2017 competition on page object detection. In: 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), vol. 1, pp. 1417–1422. IEEE (2017)

    Google Scholar 

  10. Yi, X., Gao, L., Liao, Y., Zhang, X., Liu, R., Jiang, Z.: CNN based page object detection in document images. In: 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), vol. 1, pp. 230–235. IEEE (2017)

    Google Scholar 

  11. Siegel, N., Lourie, N., Power, R., Ammar, W.: Extracting scientific figures with distantly supervised neural networks. In: Proceedings of the 18th ACM/IEEE on Joint Conference on Digital Libraries, pp. 223–232 (2018)

    Google Scholar 

  12. Kavasidis, I., Pino, C., Palazzo, S., Rundo, F., Giordano, D., Messina, P., Spampinato, C.: A saliency-based convolutional neural network for table and chart detection in digitized documents. In: Ricci, E., Rota Bulò, S., Snoek, C., Lanz, O., Messelodi, S., Sebe, N. (eds.) ICIAP 2019. LNCS, vol. 11752, pp. 292–302. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30645-8_27

    Chapter  Google Scholar 

  13. Saha, R., Mondal, A., Jawahar, C.V.: Graphical object detection in document images. In: 2019 International Conference on Document Analysis and Recognition (ICDAR), pp. 51–58. IEEE (2019)

    Google Scholar 

  14. Singh, M., Goyal, P.: DeepDoT: deep framework for detection of tables in document images. In: Singh, S.K., Roy, P., Raman, B., Nagabhushan, P. (eds.) CVIP 2020. CCIS, vol. 1377, pp. 421–432. Springer, Singapore (2021). https://doi.org/10.1007/978-981-16-1092-9_35

    Chapter  Google Scholar 

  15. Singh, M., Goyal, P.: ChartSight: an automated scheme for assisting visually impaired in understanding scientific charts. (2021)

    Google Scholar 

  16. Yepes, A.J., Zhong, X., Burdick, D.: ICDAR 2021 competition on scientific literature parsing. arXiv preprint arXiv:2106.14616 (2021)

  17. Zhong, X., Tang, J., Yepes, A.J.: PubLayNet: largest dataset ever for document layout analysis. In: 2019 International Conference on Document Analysis and Recognition (ICDAR), pp. 1015–1022. IEEE (2019)

    Google Scholar 

  18. Mondal, A., Lipps, P., Jawahar, C.V.: IIIT-AR-13K: a new dataset for graphical object detection in documents. In: Bai, X., Karatzas, D., Lopresti, D. (eds.) DAS 2020. LNCS, vol. 12116, pp. 216–230. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-57058-3_16

    Chapter  Google Scholar 

  19. Li, M., et al.: DocBank: a benchmark dataset for document layout analysis. arXiv preprint arXiv:2006.01038 (2020)

  20. Li, M., Cui, L., Huang, S., Wei, F., Zhou, M., Li, Z.: TableBank: table benchmark for image-based table detection and recognition. In: Proceedings of the 12th Language Resources and Evaluation Conference, pp. 1918–1925. (2020)

    Google Scholar 

  21. Göbel, M., Hassan, T., Oro, E., Orsi, G.: ICDAR 2013 table competition. In 2013 12th International Conference on Document Analysis and Recognition, pp. 1449–1453. IEEE (2013)

    Google Scholar 

  22. Zheng, X., Burdick, D., Popa, L., Zhong, P., Wang, N.X.R.: Global table extractor (GTE): a framework for joint table identification and cell structure recognition using visual context. Winter Conference for Applications in Computer Vision (WACV) (2021)

    Google Scholar 

  23. Zhong, X., ShafieiBavani, E., Jimeno Yepes, A.: Image-based table recognition: data, model, and evaluation. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12366, pp. 564–580. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58589-1_34

    Chapter  Google Scholar 

  24. Shahab, A., Shafait, F., Kieninger, T., Dengel, A.: An open approach towards the benchmarking of table structure recognition systems. In: Proceedings of the 9th IAPR International Workshop on Document Analysis Systems, pp. 113–120 (2010)

    Google Scholar 

  25. Fang, J., Tao, X., Tang, Z., Qiu, R., Liu, Y.: Dataset, ground-truth and performance metrics for table detection evaluation. In: 2012 10th IAPR International Workshop on Document Analysis Systems, pp. 445–449. IEEE (2012)

    Google Scholar 

  26. Morris, D., Tang, P., Ewerth, R.: A neural approach for text extraction from scholarly figures. In: 2019 International Conference on Document Analysis and Recognition (ICDAR), pp. 1438–1443. IEEE (2019)

    Google Scholar 

  27. Luo, J., Li, Z., Wang, J., Lin, C.Y.: ChartOCR: data extraction from charts images via a deep hybrid framework. In: 2021 IEEE Winter Conference on Applications of Computer Vision (WACV). The Computer Vision Foundation (2021)

    Google Scholar 

  28. Wang, C.Y., Bochkovskiy, A., Liao, H.Y.M.: Scaled-YOLOv4: scaling cross stage partial network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 13029–13038 (2021)

    Google Scholar 

  29. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. Adv. Neural. Inf. Process. Syst. 28, 91–99 (2015)

    Google Scholar 

Download references

Acknowledgement

This research is supported by the DST under CSRI grant DST/CSRI/2018/234.

Author information

Authors and Affiliations

  1. Indian Institute of Technology Ropar, 140001, Rupnagar, Punjab, India

    Hadia Showkat Kawoosa, Mandhatya Singh, Manoj Manikrao Joshi & Puneet Goyal

Authors
  1. Hadia Showkat Kawoosa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mandhatya Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Manoj Manikrao Joshi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Puneet Goyal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Puneet Goyal .

Editor information

Editors and Affiliations

  1. Kyushu University, Fukuoka, Japan

    Seiichi Uchida

  2. Boise State University, BOISE, ID, USA

    Elisa Barney

  3. LIRIS UMR CNRS, Villeurbanne, France

    Véronique Eglin

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kawoosa, H.S., Singh, M., Joshi, M.M., Goyal, P. (2022). NCERT5K-IITRPR: A Benchmark Dataset for Non-textual Component Detection in School Books. In: Uchida, S., Barney, E., Eglin, V. (eds) Document Analysis Systems. DAS 2022. Lecture Notes in Computer Science, vol 13237. Springer, Cham. https://doi.org/10.1007/978-3-031-06555-2_31

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-06555-2_31

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-06554-5

  • Online ISBN: 978-3-031-06555-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation