Skip to main content
SpringerLink
Log in

Shearlet Features for Pedestrian Detection

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

A long-time, hand-crafted features governed by a directional image analysis have been the base for the best performing pedestrian detection algorithms. In the past few years, approaches using convolutional neural networks have taken over the leadership concerning detection quality. We investigate in which way shearlets can be used for an improved hand-crafted feature computation in order to reduce the gap to CNNs. Shearlets are a relatively new mathematical framework for multiscale signal analysis, which can be seen as an extension of the wavelet framework. Shearlets are designed to capture directional information and can therefore be used for detecting the orientation of edges in images. We use this characteristic to compute image features with high informative content for pedestrian detection. Furthermore, we provide experimental results using these features and show that they outperform the results obtained by the currently best performing hand-crafted features for pedestrian detection.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. Benchmark results are available and updated frequently at http://www.vision.caltech.edu/Image_Datasets/CaltechPedestrians.

  2. \(\left| \mathscr {SH}_{\psi }\left( f\right) \left( a,s,t\right) \right| \) is said to decay rapidly if for any \(N\in \mathbb {N}\), there is a \(c_{N}>0\) such that \(\left| \mathscr {SH}_{\psi }\left( f\right) \left( a,s,t\right) \right| \le c_{N}a^{N}\), as \(a\rightarrow 0\).

References

  1. Appel, R., Fuchs, T., Dollár, P., Perona, P.: Quickly boosting decision trees—pruning underachieving features early. In: ICML (3), JMLR Proceedings, vol. 28, pp. 594–602. JMLR.org (2013)

  2. Benenson, R., Mathias, M., Tuytelaars, T., Gool, L.V.: Seeking the strongest rigid detector. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3666–3673 (2013). https://doi.org/10.1109/CVPR.2013.470

  3. Benenson, R., Omran, M., Hosang, J., Schiele, B.: Ten years of pedestrian detection, what have we learned? In: ECCV, CVRSUAD Workshop (2014)

  4. Brazil, G., Yin, X., Liu, X.: Illuminating pedestrians via simultaneous detection & segmentation. In: Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy (2017)

  5. Cai, Z., Fan, Q., Feris, R.S., Vasconcelos, N.: A unified multi-scale deep convolutional neural network for fast object detection. In: Computer Vision—ECCV 2016—14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV, pp. 354–370 (2016). https://doi.org/10.1007/978-3-319-46493-0_22

  6. Cai, Z., Saberian, M.J., Vasconcelos, N.: Learning complexity-aware cascades for deep pedestrian detection. In: 2015 IEEE International Conference on Computer Vision, ICCV 2015, Santiago, Chile, December 7–13, 2015, pp. 3361–3369 (2015). https://doi.org/10.1109/ICCV.2015.384

  7. Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. PAMI-8(6), 679–698 (1986). https://doi.org/10.1109/TPAMI.1986.4767851

  8. Chui, C.K.: An Introduction to Wavelets. Wavelet analysis and its applications. Academic Press, Cambridge (1992)

    Google Scholar 

  9. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: CVPR, pp. 886–893 (2005)

  10. Dollár, P.: Piotr’s Computer Vision Matlab Toolbox (PMT). http://vision.ucsd.edu/~pdollar/toolbox/doc/index.html. Accessed 7 July 2017

  11. Dollár, P., Appel, R., Belongie, S., Perona, P.: Fast feature pyramids for object detection. IEEE Trans. Pattern Anal. Mach. Intell. 36(8), 1532–1545 (2014). https://doi.org/10.1109/TPAMI.2014.2300479

    Article  Google Scholar 

  12. Dollár, P., Belongie, S., Perona, P.: The fastest pedestrian detector in the west. In: Proceedings of BMVC, pp. 68.1–11 (2010). https://doi.org/10.5244/C.24.68

  13. Dollár, P., Tu, Z., Perona, P., Belongie, S.: Integral channel features. In: Proceedings of BMVC, pp. 91.1–91.11 (2009). https://doi.org/10.5244/C.23.91

  14. Dollár, P., Wojek, C., Schiele, B., Perona, P.: Pedestrian detection: A benchmark. In: CVPR (2009)

  15. Dollár, P., Wojek, C., Schiele, B., Perona, P.: Pedestrian detection: an evaluation of the state of the art. IEEE Trans. Pattern Anal. Mach. Intell. 34(4), 743–761 (2012). https://doi.org/10.1109/TPAMI.2011.155

    Article  Google Scholar 

  16. Du, X., El-Khamy, M., Lee, J., Davis, L.S.: Fused DNN: a deep neural network fusion approach to fast and robust pedestrian detection. CoRR. arXiv:1610.03466 (2016)

  17. Duval-Poo, M.A., Odone, F., Vito, E.D.: Edges and corners with shearlets. IEEE Trans. Image Process. 24(11), 3768–3780 (2015). https://doi.org/10.1109/TIP.2015.2451175

    Article  MathSciNet  MATH  Google Scholar 

  18. Easley, G.R., Labate, D.: Shearlets: Multiscale Analysis for Multivariate Data. In: Kutyniok, G., Labate, D. (eds.) Image processing using shearlets, pp. 283–325. Birkhäuser, Boston (2012). https://doi.org/10.1007/978-0-8176-8316-0_8

    Google Scholar 

  19. Grohs, P., Keiper, S., Kutyniok, G., Schfer, M.: \(\alpha \)-molecules. Appl. Comput. Harmon. Anal. 41(1), 297–336 (2016). https://doi.org/10.1016/j.acha.2015.10.009. (Sparse representations with applications in imaging science, data analysis and beyond)

    Article  MathSciNet  MATH  Google Scholar 

  20. Guo, K., Kutyniok, G., Labate, D.: Sparse multidimensional representations using anisotropic dilation and shear operators. In: Wavelets and splines: Athens 2005, Mod. Methods Math., pp. 189–201. Nashboro Press, Brentwood, TN (2006)

  21. Guo, K., Labate, D., Lim, W.Q.: Edge analysis and identification using the continuous shearlet transform. Appl. Comput. Harmon. Anal. 27(1), 24–46 (2009). https://doi.org/10.1016/j.acha.2008.10.004

    Article  MathSciNet  MATH  Google Scholar 

  22. Guo, K., Labate, D., Lim, W.Q., Weiss, G., Wilson, E.: Wavelets with composite dilations. Electron. Res. Announc. Am. Math. Soc. 10(9), 78–87 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  23. Häuser, S.: Shearlet coorbit spaces, shearlet transforms and applications in imaging. Ph.D. thesis, Technische Universität Kaiserslautern (2014)

  24. Häuser, S., Steidl, G.: Fast finite shearlet transform: a tutorial. arXiv:1202.1773 (2014)

  25. Kittipoom, P., Kutyniok, G., Lim, W.Q.: Construction of compactly supported shearlet frames. Constr. Approx. 35(1), 21–72 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  26. Kutyniok, G., Labate, D.: Construction of regular and irregular shearlet frames. J. Wavelet Theory Appl. 1(1), 1–10 (2007)

    Google Scholar 

  27. Kutyniok, G., Labate, D.: Introduction to shearlets. In: Kutyniok, G., Labate, D. (eds.) Shearlets: Multiscale Analysis for Multivariate Data, pp. 1–38. Birkhäuser, Boston (2012). https://doi.org/10.1007/978-0-8176-8316-0_1

    Chapter  Google Scholar 

  28. Kutyniok, G., Lim, W.Q., Reisenhofer, R.: Shearlab 3d: Faithful digital shearlet transforms based on compactly supported shearlets. ACM Trans. Math. Softw. 42(1), 5:1–5:42 (2016). https://doi.org/10.1145/2740960

    Article  MathSciNet  MATH  Google Scholar 

  29. Kutyniok, G., Shahram, M., Zhuang, X.: Shearlab: a rational design of a digital parabolic scaling algorithm. SIAM J. Imaging Sci. 5(4), 1291–1332 (2012). https://doi.org/10.1137/110854497

    Article  MathSciNet  MATH  Google Scholar 

  30. Labate, D., Lim, W.Q., Kutyniok, G., Weiss, G.: Sparse multidimensional representation using shearlets. In: Papadakis, M., Laine, A.F., Unser, M.A. (eds.) Wavelets XI, SPIE proceedings, vol. 5914, pp. 254–262 (2005). https://doi.org/10.1117/12.613494

  31. Lee, J.M.: Smooth Maps. In: Introduction to Smooth Manifolds, 1st edn., pp. 30–59. Springer, New York (2003). https://doi.org/10.1007/978-0-387-21752-9

  32. Li, J., Liang, X., Shen, S., Xu, T., Yan, S.: Scale-aware fast r-cnn for pedestrian detection. CoRR. arXiv:1510.08160 (2015)

  33. Li, S., Shen, Y.: Shearlet frames with short support. CoRR. arXiv:1101.4725 (2011)

  34. Lim, W.Q.: The discrete shearlet transform: a new directional transform and compactly supported shearlet frames. IEEE Trans. Image Process. 19(5), 1166–1180 (2010). https://doi.org/10.1109/TIP.2010.2041410

    Article  MathSciNet  MATH  Google Scholar 

  35. Mallat, S.: A Wavelet Tour of Signal Processing. Academic Press, San Diego (1999)

    MATH  Google Scholar 

  36. Mallat, S., Zhong, S.: Characterization of signals from multiscale edges. IEEE Trans. Pattern Anal. Mach. Intell. 14(7), 710–732 (1992). https://doi.org/10.1109/34.142909

    Article  Google Scholar 

  37. Meyer, Y.: Oscillating Patterns in Image Processing and Nonlinear Evolution Equations. American Mathematical Society, Providence (2001)

    Book  MATH  Google Scholar 

  38. Nam, W., Dollár, P., Han, J.H.: Local decorrelation for improved pedestrian detection. In: Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8–13 2014, Montreal, Quebec, Canada, pp. 424–432 (2014)

  39. Ohn-Bar, E., Trivedi, M.M.: To boost or not to boost? on the limits of boosted trees for object detection. In: IEEE International Conference on Pattern Recognition (2016)

  40. Ouyang, W., Zhou, H., Li, H., Li, Q., Yan, J., Wang, X.: Jointly learning deep features, deformable parts, occlusion and classification for pedestrian detection. IEEE Trans. Pattern Anal. Mach. Intell. PP(99), 1–1 (2017). https://doi.org/10.1109/TPAMI.2017.2738645

  41. Paisitkriangkrai, S., Shen, C., van den Hengel, A.: Strengthening the effectiveness of pedestrian detection with spatially pooled features. In: European Conference on Computer Vision (ECCV’14), Zurich (2014)

  42. Papageorgiou, C., Poggio, T.: A trainable system for object detection. Int. J. Comput. Vision 38(1), 15–33 (2000). https://doi.org/10.1023/A:1008162616689

    Article  MATH  Google Scholar 

  43. Schwartz, W.R., da Silva, R.D., Davis, L.S., Pedrini, H.: A novel feature descriptor based on the shearlet transform. In: ICIP, pp. 1033–1036. IEEE (2011)

  44. Tian, Y., Luo, P., Wang, X., Tang, X.: Deep learning strong parts for pedestrian detection. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 1904–1912 (2015). https://doi.org/10.1109/ICCV.2015.221

  45. Viola, P.A., Jones, M.J.: Rapid object detection using a boosted cascade of simple features. In: 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2001), 8–14 December 2001, Kauai, HI, USA, pp. 511–518 (2001). https://doi.org/10.1109/CVPR.2001.990517

  46. Yi, S., Labate, D., Easley, G.R., Krim, H.: A shearlet approach to edge analysis and detection. IEEE Trans. Image Process. 18(5), 929–941 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  47. Zhang, L., Lin, L., Liang, X., He, K.: Is faster R-CNN doing well for pedestrian detection? In: Computer Vision—ECCV 2016—14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part II, pp. 443–457 (2016). https://doi.org/10.1007/978-3-319-46475-6_28

  48. Zhang, S., Bauckhage, C., Cremers, A.B.: Informed haar-like features improve pedestrian detection. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 947–954 (2014). https://doi.org/10.1109/CVPR.2014.126

  49. Zhang, S., Benenson, R., Omran, M., Hosang, J., Schiele, B.: How far are we from solving pedestrian detection? In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1259–1267 (2016). https://doi.org/10.1109/CVPR.2016.141

  50. Zhang, S., Benenson, R., Schiele, B.: Filtered channel features for pedestrian detection. In: CVPR (2015)

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics/Computer Science, Philipps-University of Marburg, 35032, Marburg, Germany

    Lienhard Pfeifer

  2. ITK Engineering GmbH, Computer Vision Team, 35457, Lollar, Germany

    Lienhard Pfeifer

Authors
  1. Lienhard Pfeifer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lienhard Pfeifer.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pfeifer, L. Shearlet Features for Pedestrian Detection. J Math Imaging Vis 61, 292–309 (2019). https://doi.org/10.1007/s10851-018-0834-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-018-0834-9

Keywords

Search

Navigation