Skip to main content

Advertisement

Springer Nature Link
Log in

A novel approach for scene text extraction from synthesized hazy natural images

  • Theoretical advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

The most important intricacy when processing natural scene text images is the existence of fog, smoke or haze. These intrusion elements decrease the contrast and disrupt the color fidelity of the image for various computer vision applications. In this paper, such a challenging issue is addressed. The intended work presents a novel method, that is, single image dehazing, based on transmission map. The contributions are performed in the following ways: (1) text extraction from hazy image is not straightforward due to lack of haze-free images and hazy images. To address this limitation, we introduce synthetic natural scene text image composed of pairs of synthetic hazy and corresponding haze-free images using mainstream datasets. Different from existing dehazing datasets, text in hazy images is considered compulsory content, which needs to be separated from background using the recovered image. For doing this, based on transmission map the scenic depth is calculated using haze density and color attenuation to generate depth map. In the next step, raw transmission map is computed, which is further refined using bilateral filtering to preserve edges and avoid possible noise; (2) text region proposals are estimated on the restored images using novel low-level connected component technique and character bounding is employed to complete the process. Finally, the experimentations are carried out on the images selected from standard datasets including MSRA-TD500, SVT and KAIST. The experimental outcomes demonstrate that the intended method performs better when compared with benchmark standard techniques and publically available dehazing datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Ansari GJ, Shah JH, Yasmin M, Sharif M, Fernandes SL (2018) A novel machine learning approach for scene text extraction. Future Gener Comput Syst 87:328–340

    Article  Google Scholar 

  2. Dai D, Yang W (2011) Satellite image classification via two-layer sparse coding with biased image representation. IEEE Geosci Remote Sens Lett 8:173–176

    Article  Google Scholar 

  3. Tian B, Li Y, Li B, Wen D (2014) Rear-view vehicle detection and tracking by combining multiple parts for complex urban surveillance. IEEE Trans Intell Transp Syst 15:597–606

    Article  Google Scholar 

  4. Zhang Z, Tao D (2012) Slow feature analysis for human action recognition. IEEE Trans Pattern Anal Mach Intell 34:436–450

    Article  Google Scholar 

  5. Huang S-C, Chen B-H, Cheng Y-J (2014) An efficient visibility enhancement algorithm for road scenes captured by intelligent transportation systems. IEEE Trans Intell Transp Syst 15:2321–2332

    Article  Google Scholar 

  6. Janai J, Güney F, Behl A, Geiger A (2017) Computer vision for autonomous vehicles: problems, datasets and state-of-the-art. arXiv preprint arXiv:1704.05519

  7. Li B, Ren W, Fu D, Tao D, Feng D, Zeng W, Wang Z (2019) Benchmarking single-image dehazing and beyond. IEEE Trans Image Process 28:492–505

    Article  MathSciNet  Google Scholar 

  8. Li Y, You S, Brown MS, Tan RT (2017) Haze visibility enhancement: a survey and quantitative benchmarking. Comput Vis Image Underst 165:1–16

    Article  Google Scholar 

  9. Huang S, Wu D, Yang Y, Zhu H (2018) Image dehazing based on robust sparse representation. IEEE Access 6:53907–53917

    Article  Google Scholar 

  10. Yao C, Bai X, Liu W, Ma Y, Tu Z (2012) Detecting texts of arbitrary orientations in natural images. In: 2012 IEEE conference on computer vision and pattern recognition, pp 1083–1090

  11. Lee S, Cho MS, Jung K, Kim JH (2010) Scene text extraction with edge constraint and text collinearity. In: 2010 20th international conference on pattern recognition, pp 3983–3986

  12. Wang K, Babenko B, Belongie S (2011) End-to-end scene text recognition. In: 2011 IEEE international conference on computer vision (ICCV), 2011, pp 1457–1464

  13. El Khoury J, Le Moan S, Thomas J-B, Mansouri A (2018) Color and sharpness assessment of single image dehazing. Multimed Tools Appl 77:15409–15430

    Article  Google Scholar 

  14. Salazar-Colores S, Ramos-Arreguín J-M, Echeverri CJO, Cabal-Yepez E, Pedraza-Ortega J-C, Rodriguez-Resendiz J (2018) Image dehazing using morphological opening, dilation and Gaussian filtering. Signal Image Video Process 12:1–7

    Article  Google Scholar 

  15. Gao Y, Chen H, Li H, Zhang W (2017) Single image dehazing using local linear fusion. IET Image Proc 12:637–643

    Article  Google Scholar 

  16. Park J, Kim K, Lee S, Won CS, Jung S-W (2016) Text-aware image dehazing using stroke width transform. In: 2016 IEEE international conference on image processing (ICIP), pp 2231–2235

  17. Berman D, Avidan S (2016) Non-local image dehazing. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1674–1682

  18. Ren W, Liu S, Zhang H, Pan J, Cao X, Yang M-H, Single image dehazing via multi-scale convolutional neural networks. In: European conference on computer vision, pp 154–169

  19. Li B, Peng X, Wang Z, Xu J, Feng D (2017) Aod-net: All-in-one dehazing network. In: Proceedings of the IEEE international conference on computer vision, pp 4770–4778

  20. Salazar-Colores S, Cruz-Aceves I, Ramos-Arreguin J-M (2018) Single image dehazing using a multilayer perceptron. J Electron Imaging 27:043022

    Article  Google Scholar 

  21. Tan RT (2008) Visibility in bad weather from a single image. In: IEEE conference on computer vision and pattern recognition, 2008. CVPR 2008, pp 1–8

  22. Tarel J-P, Hautiere N (2009) Fast visibility restoration from a single color or gray level image. In: 2009 IEEE 12th international conference on computer vision, pp 2201–2208

  23. Choi LK, You J, Bovik AC (2015) Referenceless prediction of perceptual fog density and perceptual image defogging. IEEE Trans Image Process 24:3888–3901

    Article  MathSciNet  Google Scholar 

  24. Galdran A, Vazquez-Corral J, Pardo D, Bertalmio M (2017) Fusion-based variational image dehazing. IEEE Signal Process Lett 24:151–155

    MATH  Google Scholar 

  25. Animesh C, Mohanty S, Dutta T, Gupta HP (2017) Fast text detection from single hazy image using smart device. In: 2017 IEEE international conference on multimedia and expo workshops (ICMEW), pp 423–428

  26. Fattal R (2008) Single image dehazing. ACM Trans Graph 27:72

    Article  Google Scholar 

  27. He K, Sun J, Tang X (2011) Single image haze removal using dark channel prior. IEEE Trans Pattern Anal Mach Intell 33:2341–2353

    Article  Google Scholar 

  28. Meng G, Wang Y, Duan J, Xiang S, Pan C (2013) Efficient image dehazing with boundary constraint and contextual regularization. In: Proceedings of the IEEE international conference on computer vision, pp 617–624

  29. Wang Z, Hou G, Pan Z, Wang G (2017) Single image dehazing and denoising combining dark channel prior and variational models. IET Comput Vis 12:393–402

    Article  Google Scholar 

  30. Liao B, Yin P, Xiao C (2018) Efficient image dehazing using boundary conditions and local contrast. Comput Gr 70:242–250

    Article  Google Scholar 

  31. Nair D, Sankaran P (2018) Color image dehazing using surround filter and dark channel prior. J Vis Commun Image Represent 50:9–15

    Article  Google Scholar 

  32. Tang K, Yang J, Wang J (2014) Investigating haze-relevant features in a learning framework for image dehazing. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2995–3000

  33. Zhu Q, Mai J, Shao L (2015) A fast single image haze removal algorithm using color attenuation prior. IEEE Trans Image Process 24:3522–3533

    Article  MathSciNet  Google Scholar 

  34. Cai B, Xu X, Jia K, Qing C, Tao D (2016) Dehazenet: an end-to-end system for single image haze removal. IEEE Trans Image Process 25:5187–5198

    Article  MathSciNet  Google Scholar 

  35. Li C, Guo J, Porikli F, Fu H, Pang Y (2018) A cascaded convolutional neural network for single image dehazing. IEEE Access 6:24877–24887

    Article  Google Scholar 

  36. Yin X-C, Pei W-Y, Zhang J, Hao H-W (2015) Multi-orientation scene text detection with adaptive clustering. IEEE Trans Pattern Anal Mach Intell 37:1930–1937

    Article  Google Scholar 

  37. Zhang Z, Zhang C, Shen W, Yao C, Liu W, Bai X Multi-oriented text detection with fully convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4159–4167

  38. Yao C, Bai X, Sang N, Zhou X, Zhou S, Cao Z (2016) Scene text detection via holistic, multi-channel prediction. arXiv preprint arXiv:1606.09002

  39. Wang Y, Shi C, Xiao B, Wang C, Qi C (2018) CRF based text detection for natural scene images using convolutional neural network and context information. Neurocomputing 295:46–58

    Article  Google Scholar 

  40. Lyu P, Yao C, Wu W, Yan S, Bai X (2018) Multi-oriented scene text detection via corner localization and region segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7553–7563

  41. Liao M, Shi B, Bai X (2018) Textboxes ++: a single-shot oriented scene text detector. IEEE Trans Image Process 27:3676–3690

    Article  MathSciNet  Google Scholar 

  42. Deng D, Liu H, Li X, Cai D (2018) Pixellink: detecting scene text via instance segmentation. In: Thirty-second AAAI conference on artificial intelligence

  43. Liu X, Liang D, Yan S, Chen D, Y. Qiao, Yan J (2018) Fots: fast oriented text spotting with a unified network. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5676–5685

  44. Jiang Y, Zhu X, Wang X, Yang S, Li W, Wang H, Fu P, Luo Z (2017) R2cnn: Rotational region cnn for orientation robust scene text detection. arXiv preprint arXiv:1706.09579

  45. Long S, Ruan J, Zhang W, He X, Wu W, Yao C (2018) Textsnake: a flexible representation for detecting text of arbitrary shapes. In: Proceedings of the European conference on computer vision (ECCV), pp 20–36

  46. Sakaridis C, Dai D, Van Gool L (2018) Semantic foggy scene understanding with synthetic data. International Journal of Computer Vision 126:1–20

    Article  Google Scholar 

  47. Shafina M, Aji S (2019) A single image haze removal method with improved airlight estimation using gradient thresholding. In: Krishna AN, Srikantaiah KC, Naveena C (eds) Integrated intelligent computing, communication and security, Springer, pp 651–659

  48. Park H, Park D, Han DK, Ko H (2014) Single image haze removal using novel estimation of atmospheric light and transmission. In: 2014 IEEE international conference on image processing (ICIP), pp 4502–4506

  49. Levin A, Lischinski D, Weiss Y (2007) A closed-form solution to natural image matting. IEEE Trans Pattern Anal Mach Intell 30:228–242

    Article  Google Scholar 

  50. He K, Sun J, Tang X (2012) Guided image filtering. IEEE Trans Pattern Anal Mach Intell 35:1397–1409

    Article  Google Scholar 

  51. Farbman Z, Fattal R, Lischinski D, Szeliski R (2008) Edge-preserving decompositions for multi-scale tone and detail manipulation. ACM Trans Gr 27:67

    Article  Google Scholar 

  52. Tomasi C, Manduchi R (1998) Bilateral filtering for gray and color images. In Iccv, p 2

  53. He L, Ren X, Gao Q, Zhao X, Yao B, Chao Y (2017) The connected-component labeling problem: a review of state-of-the-art algorithms. Pattern Recogn 70:25–43

    Article  Google Scholar 

  54. Sung M.-C, Jun B, Cho H, Kim D (2012) Scene text detection with robust character candidate extraction method. In: 2015 13th International conference on document analysis and recognition (ICDAR), pp 426–430

  55. Li J, Li G, Fan H (2018) Image dehazing using residual-based deep CNN. IEEE Access 6:26831–26842

    Article  Google Scholar 

  56. Zhang Y, Ding L, Sharma G (2017) Hazerd: an outdoor scene dataset and benchmark for single image dehazing. In: 2017 IEEE international conference on image processing (ICIP), pp 3205–3209

  57. Ancuti CO, Ancuti C, Timofte R, De Vleeschouwer C (2018) O-HAZE: a dehazing benchmark with real hazy and haze-free outdoor images. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 754–762

  58. Ancuti C, Ancuti CO, Timofte R, De Vleeschouwer C (2018) I-HAZE: a dehazing benchmark with real hazy and haze-free indoor images. In: International conference on advanced concepts for intelligent vision systems, pp 620-631

  59. Liu Z, Lin G, Yang S, Feng J, Lin W, Goh WL (2018) Learning markov clustering networks for scene text detection. arXiv preprint arXiv:1805.08365

Download references

Author information

Authors and Affiliations

  1. Department of Computer Sciences, COMSATS University Islamabad, Wah Campus, Islamabad, Pakistan

    Ghulam Jillani Ansari, Jamal Hussain Shah, Muhammad Sharif & Saeed ur Rehman

Authors
  1. Ghulam Jillani Ansari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jamal Hussain Shah
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Muhammad Sharif
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Saeed ur Rehman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jamal Hussain Shah.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ansari, G.J., Shah, J.H., Sharif, M. et al. A novel approach for scene text extraction from synthesized hazy natural images. Pattern Anal Applic 23, 1305–1322 (2020). https://doi.org/10.1007/s10044-019-00855-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-019-00855-7

Keywords