Skip to main content
SpringerLink
Log in

Canny edge detection and Hough transform for high resolution video streams using Hadoop and Spark

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

Nowadays, video cameras are increasingly used for surveillance, monitoring, and activity recording. These cameras generate high resolution image and video data at large scale. Processing such large scale video streams to extract useful information with time constraints is challenging. Traditional methods do not offer scalability to process large scale data. In this paper, we propose and evaluate cloud services for high resolution video streams in order to perform line detection using Canny edge detection followed by Hough transform. These algorithms are often used as preprocessing steps for various high level tasks including object, anomaly, and activity recognition. We implement and evaluate both Canny edge detector and Hough transform algorithms in Hadoop and Spark. Our experimental evaluation using Spark shows an excellent scalability and performance compared to Hadoop and standalone implementations for both Canny edge detection and Hough transform. We obtained a speedup of 10.8\(\times\) and 9.3\(\times\) for Canny edge detection and Hough transform respectively using Spark. These results demonstrate the effectiveness of parallel implementation of computer vision algorithms to achieve good scalability for real-world applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Anjum, A., Abdullah, T., Tariq, M., Baltaci, Y., Antonopoulos, N.: Video stream analysis in clouds: an object detection and classification framework for high performance video analytics. IEEE Trans. Cloud Comput. (2016)

  2. Arsh, S., Bhatt, A., Kumar, P.: Distributed image processing using hadoop and HIPI. In: 2016 International Conference on Advances in Computing, Communications and Informatics, ICACCI 2016, Jaipur, India, September 21–24, 2016, pp. 2673–2676 (2016)

  3. Arthanari, J., Baskaran, R.: Enhancement of video streaming analysis using cluster-computing framework. Clust. Comput. 3 (2018)

  4. Arunkumar, P., Shantharajah, S., Geetha, M.: Improved canny detection algorithm for processing and segmenting text from the images. Clust. Comput., pp. 1–7 (2018)

  5. Chen, L., Chen, H., Pan, Y., Chen, Y.: A fast efficient parallel Hough transform algorithm on LARPBS. J. Supercomput. 29(2), 185–195 (2004)

    Article  Google Scholar 

  6. Chen, J., Li, K., Tang, Z., Bilal, K., Yu, S., Weng, C., Li, K.: A parallel random forest algorithm for big data in a Spark cloud computing environment. IEEE Trans. Parallel Distrib. Syst. 28(4), 919–933 (2017)

    Article  Google Scholar 

  7. Deanm, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)

    Article  Google Scholar 

  8. Gentsos, C., Sotiropoulou, C.-L., Nikolaidis, S., Vassiliadis, N.: Real-time canny edge detection parallel implementation for fpgas. In: 2010 17th IEEE International Conference on Electronics, Circuits, and Systems (ICECS), pp. 499–502. IEEE (2010)

  9. Halyo, V., LeGresley, P., Lujan, P., Karpusenko, V., Vladimirov, A.: First evaluation of the CPU, GPGPU and MIC architectures for real time particle tracking based on Hough transform at the LHC. J. Instrum. 9(04), P04005 (2014)

    Article  Google Scholar 

  10. Huang, W., Meng, L., Zhang, D., Zhang, W.: In-memory parallel processing of massive remotely sensed data using an Apache Spark on Hadoop YARN model. IEEE J. Sel. Topics Appl. Earth Obs. Remote Sens. 10(1), 3–19 (2017)

    Article  Google Scholar 

  11. Ismail El-Helw, R. H.: Scaling mapreduce vertically and horizontally. In: SC14: International Conference for High Performance Computing, Networking, Storage and Analysis (2014)

  12. Jatrniko, W., Arsa, D.M.S., Wisesa, H., Jati, G., Ma’sum, M.A.: A review of big data analytics in the biomedical field. In: International Workshop on Big Data and Information Security (IWBIS), pp. 31–41. IEEE (2016)

  13. Lv, J., Wu, B., Yang, S., Jia, B., Qiu, P.: Efficient large scale near-duplicate video detection base on Spark. In: 2016 IEEE International Conference on Big Data (Big Data), pp. 957–962. IEEE (2016)

  14. Mukhopadhyay, P., Chaudhuri, B.B.: A survey of hough transform. Pattern Recognit. 48(3), 993–1010 (2015)

    Article  Google Scholar 

  15. Rathore, M.M., Son, H., Ahmad, A., Paul, A., Jeon, G.: Real-time big data stream processing using GPU with Spark over Hadoop ecosystem. Int. J. Parallel Program. pp. 1–17 (2017)

  16. Sweeney, C., Liu, L., Arietta, S., Lawrence, J.: HIPI: A Hadoop Image Processing Interface for Image-Based Mapreduce Tasks. University of Virginia, Chris (2011)

    Google Scholar 

  17. van den Braak, G.-J., Nugteren, C., Mesman, B., Corporaal, H.: Fast hough transform on gpus: Exploration of algorithm trade-offs. In: International Conference on Advanced Concepts for Intelligent Vision Systems, pp. 611–622. Springer (2011)

  18. Waghule, D.R., Ochawar, R.S.: Overview on edge detection methods. In: 2014 International Conference on Electronic Systems, Signal Processing and Computing Technologies (ICESC), pp. 151–155. IEEE (2014)

  19. Xu, Q., Varadarajan, S., Chakrabarti, C., Karam, L.J.: A distributed canny edge detector: algorithm and FPGA implementation. IEEE Trans. Image Process. 23(7), 2944–2960 (2014)

    Article  MathSciNet  Google Scholar 

  20. Yam-Uicab, R., Lopez-Martinez, J., Trejo-Sanchez, J., Hidalgo-Silva, H., Gonzalez-Segura, S.: A fast hough transform algorithm for straight lines detection in an image using gpu parallel computing with CUDA-C. J. Supercomput. 73(11), 4823–4842 (2017)

    Article  Google Scholar 

  21. Yaseen, M.U., Anjum, A., Rana, O., Hill, R.: Cloud-based scalable object detection and classification in video streams. Future Gener. Comput. Syst. 80, 286–298 (2018)

    Article  Google Scholar 

  22. Zaharia, M., Chowdhury, M., Franklin, M.J., Shenker, S., Stoica, I.: Spark: cluster computing with working sets. HotCloud 10, 10–10 (2010)

    Google Scholar 

  23. Zaharia, M., Xin, R.S., Wendell, P., Das, T., Armbrust, M., Dave, A., Meng, X., Rosen, J., Venkataraman, S., Franklin, M.J., et al.: Apache spark: a unified engine for big data processing. Commun. ACM 59(11), 56–65 (2016)

    Article  Google Scholar 

Download references

Acknowledgements

This work was made possible by NPRP Grant # 7-481-1-088 from the Qatar National Research Fund (a member of Qatar Foundation). The statements made herein are solely the responsibility of the authors.

Author information

Authors and Affiliations

  1. Punjab University College of Information Technology, Lahore, Pakistan

    Bilal Iqbal, Waheed Iqbal & Nazar Khan

  2. Information Technology University, Lahore, Pakistan

    Arif Mahmood

  3. Department of Computer Science, Qatar University, Doha, Qatar

    Abdelkarim Erradi

Authors
  1. Bilal Iqbal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Waheed Iqbal
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nazar Khan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Arif Mahmood
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Abdelkarim Erradi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Waheed Iqbal.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Iqbal, B., Iqbal, W., Khan, N. et al. Canny edge detection and Hough transform for high resolution video streams using Hadoop and Spark. Cluster Comput 23, 397–408 (2020). https://doi.org/10.1007/s10586-019-02929-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-019-02929-x

Keywords

Search

Navigation