Skip to main content
SpringerLink
Log in

Granulated deep learning and Z-numbers in motion detection and object recognition

  • IAPR-MedPRAI
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

The article deals with the problems of motion detection, object recognition, and scene description using deep learning in the framework of granular computing and Z-numbers. Since deep learning is computationally intensive, whereas granular computing, on the other hand, leads to computation gain, a judicious integration of their merits is made so as to make the learning mechanism computationally efficient. Further, it is shown how the concept of z-numbers can be used to quantify the abstraction of semantic information in interpreting a scene, where subjectivity is of major concern, through recognition of its constituting objects. The system, thus developed, involves recognition of both static objects in the background and moving objects in foreground separately. Rough set theoretic granular computing is adopted where rough lower and upper approximations are used in defining object and background models. During deep learning, instead of scanning the entire image pixel by pixel in the convolution layer, we scan only the representative pixel of each granule. This results in a significant gain in computation time. Arbitrary-shaped and sized granules, as expected, perform better than regular-shaped rectangular granules or fixed-sized granules. The method of tracking is able to deal efficiently with various challenging cases, e.g., tracking partially overlapped objects and suddenly appeared objects. Overall, the granulated system shows a balanced trade-off between speed and accuracy as compared to pixel level learning in tracking and recognition. The concept of using Z-numbers, in providing a granulated linguistic description of a scene, is unique. This gives a more natural interpretation of object recognition in terms of certainty toward scene understanding.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Yilmaz A, Javed O, Shah M (2006) Object tracking: a survey. Acm Comput Surv (CSUR) 38(4):13

    Article  Google Scholar 

  2. Zadeh LA (1997) Toward a theory of fuzzy information granulation and its centrality in human reasoning and fuzzy logic. Fuzzy Sets Syst 90(2):111–127

    Article  MathSciNet  Google Scholar 

  3. Pal SK, Uma Shankar B, Mitra P (2005) Granular computing, rough entropy and object extraction. Pattern Recognit Lett 26(16):2509–2517

    Article  Google Scholar 

  4. Pawlak Z (2012) Rough sets: theoretical aspects of reasoning about data, vol 9. Springer, Berlin

    MATH  Google Scholar 

  5. Debarati Chakraborty B, Shankar U, Pal SK (2013) Granulation, rough entropy and spatiotemporal moving object detection. Appl Soft Comput 13(9):4001–4009

    Article  Google Scholar 

  6. Chakraborty DB, Pal SK (2016) Neighborhood granules and rough rule-base in tracking. Nat Comput 15(3):359–370

    Article  MathSciNet  Google Scholar 

  7. Pal SK, Chakraborty DB (2017) Granular flow graph, adaptive rule generation and tracking. IEEE Trans Cybern 47(12):4096–4107

    Article  Google Scholar 

  8. Pal SK (2018) Data science and technology: challenges, opportunities and national relevance. In: 14th annual convocation speech: convocation address, National Institute of Technology, Calicut, India, Sept 29

  9. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436

    Article  Google Scholar 

  10. Erhan D, Szegedy C, Toshev A, Anguelov D (2014) Scalable object detection using deep neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2147–2154

  11. Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: towards real-time object detection with region proposal networks. In: Advances in neural information processing systems, pp 91–99

  12. Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: Unified, real-time object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 779–788

  13. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC (2016) Ssd: single shot multibox detector. In: European conference on computer vision, Springer, pp 21–37

  14. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105

  15. Kavukcuoglu K, Sermanet P, Boureau YL, Gregor K, Mathieu M, Cun YL (2010) Learning convolutional feature hierarchies for visual recognition. In: Advances in neural information processing systems, pp 1090–1098

  16. Erhan Gundogdu A, Alatan A (2018) Good features to correlate for visual tracking. IEEE Trans Image Process 27(5):2526–2540

    Article  MathSciNet  Google Scholar 

  17. He Z, Fan Y, Zhuang J, Dong Y, Bai H (2017) Correlation filters with weighted convolution responses. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1992–2000

  18. Oron S, Bar-Hillel A, Levi D, Avidan S (2015) Locally orderless tracking. Int J Comput Vis 111(2):213–228

    Article  MathSciNet  Google Scholar 

  19. Zhong W, Huchuan L, Yang MH (2014) Robust object tracking via sparse collaborative appearance model. IEEE Trans Image Process 23(5):2356–2368

    Article  MathSciNet  Google Scholar 

  20. Karpathy A, Toderici G, Shetty S, Leung T, Sukthankar R, Fei-Fei L (2014) Large-scale video classification with convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1725–1732

  21. Ji S, Wei X, Yang M, Kai Y (2013) 3d convolutional neural networks for human action recognition. IEEE Trans Pattern Anal Mach Intell 35(1):221–231

    Article  Google Scholar 

  22. Gan W, Lee MS, Wu CH, Kuo CCJ (2018) Online object tracking via motion-guided convolutional neural network (mgnet). J Vis Commun Image Represent 53:180–191

    Article  Google Scholar 

  23. Held D, Thrun S, Savarese S (2016) Learning to track at 100 fps with deep regression networks. In: European conference on computer vision. Springer, pp 749–765

  24. Zadeh Lotfi A (2011) A note on z-numbers. Inf Sci 181(14):2923–2932

    Article  Google Scholar 

  25. Pal SK, Mitra S (1992) Multilayer perceptron, fuzzy sets, classifiaction. IEEE Trans Neural Netw 3(5):683–697

    Article  Google Scholar 

  26. Pan SJ, Yang Q et al (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22(10):1345–1359

    Article  Google Scholar 

  27. Lin TY, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollár P, Zitnick CL (2014) Microsoft coco: common objects in context. In: European conference on computer vision, Springer, pp 740–755

  28. Zadeh LA (1996) \({\text{ Fuzzy }} {\text{ logic }}= {\text{ computing }} {\text{ with }} {\text{ words}}\). IEEE Trans Fuzzy Syst 4(2):103–111

    Article  Google Scholar 

  29. Banerjee R, Pal S (2013) The z-number enigma: a study through an experiment. In: Soft computing: state of the art theory and novel applications, Springer, pp 71–88

  30. Possegger H, Sternig S, Mauthner T, Roth PM, Bischof H (2013) Robust real-time tracking of multiple objects by volumetric mass densities. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)

  31. Visual tracker benchmark data. http://cvlab.hanyang.ac.kr/tracker_benchmark/datasets.html

  32. Kristan M, Matas J, Leonardis A, Vojir T, Pflugfelder Roman, Fernandez Gustavo, Nebehay Georg, Porikli Fatih, Čehovin Luka (2016) A novel performance evaluation methodology for single-target trackers. IEEE Trans Pattern Anal Mach Intell 38(11):2137–2155

    Article  Google Scholar 

  33. Jurrasic intro dataset. https://www.youtube.com/watch?v=lc0UehYemQA

  34. Ferryman J, Shahrokni A (2009) Pets2009: dataset and challenge. In: 2009 twelfth IEEE international workshop on performance evaluation of tracking and surveillance (PETS-Winter), pp 1–6. IEEE

  35. Head pose estimation in the wild using convolutional neural networks and adaptive gradient methods. https://github.com/mpatacchiola/deepgaze/blob/master/examples/ex_motion_detectors_comparison_video/cars.avi/

  36. Pal SK, Ghosh A, Uma Shankar B (2000) Segmentation of remotely sensed images with fuzzy thresholding, and quantitative evaluation. Int J Remote Sens 21(11):2269–2300

    Article  Google Scholar 

  37. Taormina R, Chau KW, Sivakumar B (2015) Neural network river forecasting through baseflow separation and binary-coded swarm optimization. J Hydrol 529:1788–1797

    Article  Google Scholar 

  38. Wu CL, Chau KW (2011) Rainfall-runoff modeling using artificial neural network coupled with singular spectrum analysis. J Hydrol 399(3–4):394–409

    Article  Google Scholar 

Download references

Acknowledgements

Valuable discussion with Ms. Romi Banerjee is gratefully acknowledged. S.K. Pal acknowledges the INSA Distinguished Professorship. D. Bhunia Chakraborty acknowledges CSIR for providing her Research Associateship.

Author information

Authors and Affiliations

  1. Center for Soft Computing Research, Indian Statistical Institute, Kolkata, 700 108, India

    Sankar K. Pal, Debasmita Bhoumik & Debarati Bhunia Chakraborty

Authors
  1. Sankar K. Pal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Debasmita Bhoumik
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Debarati Bhunia Chakraborty
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sankar K. Pal.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pal, S.K., Bhoumik, D. & Bhunia Chakraborty, D. Granulated deep learning and Z-numbers in motion detection and object recognition. Neural Comput & Applic 32, 16533–16548 (2020). https://doi.org/10.1007/s00521-019-04200-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-019-04200-1

Keywords

Search

Navigation