Skip to main content
SpringerLink
Log in

Image de-fencing using histograms of oriented gradients

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Image de-fencing is often used by digital photographers to remove regular or near-regular fence-like patterns from an image. The goal of image de-fencing is to remove a fence object from an image in such a seamless way that it appears as if the fence never existed in the image. This task is mainly challenging due to a wide range intra-class variation of fence, complexity of background, and common occlusions. We present a novel image de-fencing technique to automatically detect fences of regular and irregular patterns in an image. We use a data-driven approach that detects a fence using encoded images as feature descriptors. We use a variant of the histograms of oriented gradients (HOG) descriptor for feature representation. We modify the conventional HOG descriptor to represent each pixel rather than representing a full patch. We evaluated our algorithm on 41 different images obtained from various sources on the Internet based on a well-defined selection criteria. Our evaluation shows that the proposed algorithm is capable of detecting a fence object in a given image with more than 98% accuracy and 87% precision.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. These results are produced by using the source code available at: http://vision.cse.psu.edu/data/PAMI09Win7Matlab201264bit and http://www.di.unito.it/~farid/Research/defencing.html, respectively.

References

  1. Bertalmio, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: SIGGRAPH, pp. 417–424 (2000)

  2. Liu, Y., Belkina, T., Hays, J.H., Lublinerman, R.: Image de-fencing. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–8 (2008)

  3. Hays, J., Leordeanu, M., Efros, A.A., Liu, Y.: Discovering texture regularity as a higher-order correspondence problem. In: Proceedings of European Conference on Computer Vision (ECCV), pp. 522–535 (2006)

  4. Park, M., Brocklehurst, K., Collins, R.T., Liu, Y.: Image de-fencing revisited. In: Proceedings of Asian Conference on Computer Vision (ACCV), pp. 422–434 (2011)

  5. Farid, M.S., Mahmood, A., Grangetto, M.: Image de-fencing framework with hybrid inpainting algorithm. Signal Image Video Process. 10(7), 1193–1201 (2016)

    Article  Google Scholar 

  6. Qin Zou, Y., Cao, Q.L., Mao, Q., Wang, S.: Automatic inpainting by removing fence-like structures in rgbd images. Mach. Vis. Appl. 25(7), 1841–1858 (2014)

    Article  Google Scholar 

  7. Kumar, V., Mukherjee, J., Mandal, S.K.D.: Image defencing via signal demixing. In: Proceedings of 10th Indian Conference on Computer Vision, Graphics and Image Processing (ICVGIP), pp. 1–8 (2016)

  8. Khalid, M., Yousaf, M.M.: Parallel image de-fencing: technique, analysis and performance evaluation. In: Advanced Computer and Communication Engineering Technology, pp. 979–988. Springer (2016)

  9. Zhang, Q., Yuan, Y., Lu, X.: Image de-fencing with hyperspectral camera. In Proceedings of the International Conference on Computer, Information and Telecommunication Systems (CITS), pp. 1–5 (2016)

  10. Jonna, S., Satapathy, S., Sahay, R.R.: Stereo image de-fencing using smartphones. In: Proceedings of the International Conference Acoustics, Speech and Signal Processing (ICASSP), pp. 1792–1796 (2017)

  11. Xue, T., Rubinstein, M., Liu, C., Freeman, W.T.: A computational approach for obstruction-free photography. ACM Trans. Graph. 34(4), 1–11 (2015)

    Article  Google Scholar 

  12. Liu, W., Mu, Y., Yan, S.: Video de-fencing. IEEE Trans. Circuits Syst. Video Technol. 24(7), 1111–1121 (2014)

    Article  Google Scholar 

  13. Khasare, V.S., Sahay, R.R., Kankanhalli, M.S.: Seeing through the fence: image de-fencing using a video sequence. In: IEEE Proceedings of International Conference on Image Processing (ICIP), pp. 1351–1355 (2013)

  14. Negi, C.S., Mandal, K., Sahay, R.R., Kankanhalli, M.S.: Super-resolution de-fencing: simultaneous fence removal and high-resolution image recovery using videos. In: Proceedings of IEEE International Conference on Multimedia and Expo Workshops (ICMEW), pp. 1–6 (2014)

  15. Yi, R., Wang, J., Tan, P.: Automatic fence segmentation in videos of dynamic scenes. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp. 705–713 (2016)

  16. Jonna, S., Nakka, K.K., Sahay, R.R.: My camera can see through fences: a deep learning approach for image de-fencing. In: Asian Conference on Pattern Recognition (ACPR), pp. 261–265 (2015)

  17. Jonna, S., Nakka, K.K., Sahay, R.R.: Deep learning based fence segmentation and removal from an image using a video sequence. In: Computer Vision—ECCV Workshops, pp. 836–851. Springer, 2016

  18. Jonna, S., Voleti, V.S., Sahay, R.R., Kankanhalli, M.S.: A multimodal approach for image de-fencing and depth inpainting. In: International Conference on Advances in Pattern Recognition (ICAPR), pp. 1–6 (2015)

  19. Jonna, S., Nakka, K.K., Khasare, V.S., Sahay, R.R., Kankanhalli, M.S.: Detection and removal of fence occlusions in an image using a video of the static/dynamic scene. J. Opt. Soc. Am. 33(10), 1917–1930 (2016)

    Article  Google Scholar 

  20. Yamashita, A., Matsui, A., Kaneko, T.: Fence removal from multi-focus images. In: Proceedings of the International Conference on Pattern Recognition (ICPR), pp. 4532–4535 (2010)

  21. Li, Y., Wang, Y., Piao, Y.: Extraction of thin occlusions from digital images. In: Proceedings of SPIE 10255, Selected Papers of the Chinese Society for Optical Engineering Conferences (2017)

  22. Criminisi, A., Perez, P., Toyama, K.: Region filling and object removal by exemplar-based image inpainting. IEEE Trans. Image Process. 13(9), 1200–1212 (2004)

    Article  Google Scholar 

  23. Mirkamali, S.S., Nagabhushan, P.: Object removal by depth-wise image inpainting. Signal Image Video Process. 9(8), 1785–1794 (2015)

    Article  Google Scholar 

  24. Lee, J., Lee, D.K., Park, R.H.: Robust exemplar-based inpainting algorithm using region segmentation. IEEE Trans. Consum. Electron. 58(2), 553–561 (2012)

    Article  Google Scholar 

  25. Yamashita, A., Tsurumi, F., Kaneko, T., Asama, H.: Automatic removal of foreground occluder from multi-focus images. In: IEEE International Conference on Robotics and Automation, pp. 5410–5416 (2012)

  26. Kumar, V., Mukherjee, J., Mandal, S.K.D.: Combinatorial exemplar-based image inpainting. In: International Workshop on Combinatorial Image Analysis, pp. 284–298 (2015)

  27. Yamauchi, H., Haber, J., Seidel, H.P.: Image restoration using multiresolution texture synthesis and image inpainting. In: Proceedings of Computer Graphics International, pp. 120–125 (2003)

  28. Wei, Y., Liu, S.: Domain-based structure-aware image inpainting. Signal Image Video Process. 10(5), 911–919 (2016)

    Article  MathSciNet  Google Scholar 

  29. Farid, M.S., Khan, H., Mahmood, A.: Image inpainting based on pyramids. In: Proceedings of IEEE International Conference on Signal Processing (ICSP), pp. 711–715 (2010)

  30. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp. 886–893 (2005)

  31. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. (IJCV) 60(2), 91–110 (2004)

    Article  Google Scholar 

  32. Belongie, S., Malik, J., Puzicha, J.: Matching shapes. In: The 8th ICCV, pp. 454–461 (2001)

  33. Freeman, W.T., Roth, M.: Orientation histograms for hand gesture recognition. Technical report, International Workshop on Automatic Face and Gesture Recognition. IEEE Computer Society (1995)

  34. Serra, J.: Image Analysis and Mathematical Morphology. Academic Press Inc., Orlando (1983)

    Google Scholar 

  35. Dalal, N.: Finding people in images and videos. Ph.D. thesis, Institut National Polytechnique de Grenoble/INRIA (2006)

Download references

Author information

Authors and Affiliations

  1. Punjab University College of Information Technology, University of the Punjab, Lahore, Pakistan

    Madiha Khalid, Muhammad Murtaza Yousaf, Kashif Murtaza & Syed Mansoor Sarwar

Authors
  1. Madiha Khalid
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Muhammad Murtaza Yousaf
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kashif Murtaza
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Syed Mansoor Sarwar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Madiha Khalid.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (jpg 54 KB)

Supplementary material 2 (jpg 120 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Khalid, M., Yousaf, M.M., Murtaza, K. et al. Image de-fencing using histograms of oriented gradients. SIViP 12, 1173–1180 (2018). https://doi.org/10.1007/s11760-018-1266-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-018-1266-0

Keywords

Search

Navigation