Skip to main content
SpringerLink
Log in

Exploiting street-level panoramic images for large-scale automated surveying of traffic signs

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Accurate and up-to-date inventories of traffic signs contribute to efficient road maintenance and a high road safety. This paper describes a system for the automated surveying of road signs from street-level images. This is an extremely challenging task, as the involved capturings are non-densely sampled, captured under a wide range of weather conditions and signs may be distorted. The described system is designed in a generic and learning-based fashion, which enables the recognition of different sign appearance classes with the same algorithms, based on class-specific training data. The system starts with detection of the signs visible within each image, using a detection cascade. Next, the 3D position of the signs that are detected consequently within consecutive capturings is calculated. Afterwards, each positioned road sign is classified to retrieve its sign type, thereby exploiting all detections used during positioning of the respective sign. The presented system is intended for large-scale application and currently supports 11 sign appearance classes, containing 176 different sign types. Performance evaluations conducted on a large, real-world dataset (68,010 images) show that our approach accurately positions 95.5 % of the 3,385 present signs, where 96.3 % of them are also correctly classified. Furthermore, our system localized 98.5 % of the signs in at least a single image. Our system design allows for appending a limited manual correction stage to attain a very high performance, so that sign inventories can be created cost effectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Arthur, D., Vassilvitskii, S.: k-means++: the advantages of careful seeding. In: Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms. SODA ’07, pp. 1027–1035. Society for Industrial and Applied Mathematics, Philadelphia (2007)

  2. Bonaci, I., Kusalic, I., Kovacek, I., Kalafatic, Z., Segvic, S.: Addressing false alarms and localization inaccuracy in traffic sign detection and recognition. In: Proceeding of the 16th Computer Vision Winter Workshop, pp. 1–8 (2011)

  3. Burghouts, G.J., Geusebroek, J.M.: Performance evaluation of local colour invariants. Comput. Vis. Image Underst. 113, 48–62 (2009)

    Article  Google Scholar 

  4. Comanicu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24, 603–619 (2002)

    Article  Google Scholar 

  5. Creusen, I., Hazelhoff, L., de With, P.: Color transformation for improved traffic sign detection. In: Proceeding of the 2012 19th IEEE International Conference on Image Processing (ICIP), pp. 461–464 (2012). doi:10.1109/ICIP.2012.6466896

  6. Creusen, I., Wijnhoven, R.G.J., Herbschleb, E., de With, P.: Color exploitation in hog-based traffic sign detection. In: Proceeding of the 2010 17th IEEE International Conference on Image Processing (ICIP), pp. 2669–2672 (2010). doi:10.1109/ICIP.2010.5651637

  7. Csurka, G., Dance, C.R., Fan, L., Willamowski, J., Bray, C.: Visual categorization with bags of keypoints. In: Proceeding of the European Conference on Computer Vision (ECCV) (2004)

  8. Dalal, N., Triggs, B.: Histogram of oriented gradients for human detection. In: Proceeding of the IEEE Computer Vision and Pattern Recognition (CVPR), vol. 1, pp. 886–893 (2005)

  9. Hazelhoff, L., Creusen, I., de With, P.: Robust classification of traffic signs using multi-view cues. In: Proceeding of the 2012 19th IEEE International Conference on Image Processing (ICIP), pp. 457–460 (2012). doi:10.1109/ICIP.2012.6466895

  10. Hazelhoff, L., Creusen, I.M., De With, P.H.N.: Robust detection, classification and positioning of traffic signs from street-level panoramic images for inventory purposes. In: Proceeding of the Workshop on Applications of Computer Vision (WACV), pp. 313–320 (2012)

  11. Hazelhoff, L., Creusen, I.M., van de Wouw, D.W.J.M., de With, P.H.N.: Large-scale classification of traffic signs under real-world conditions. In: Proceeding of the SPIE 8304B–34 (2012)

  12. Houben, S., Stallkamp, J., Salmen, J., Schlipsing, M., Igel, C.: Detection of traffic signs in real-world images: The German Traffic Sign Detection Benchmark. In: Proceeding of the International Joint Conference on Neural Networks, p. 1288 (2013)

  13. Jurie, F., Triggs, B.: Creating efficient codebooks for visual recognition. In: ICCV, pp. 604–610 (2005)

  14. Lafuente-Arroyo, S., Maldonado-Bascon, S., Gil-Jimenez, P., Acevedo-Rodriguez, J., Lopez-Sastre, R.: A tracking system for automated inventory of road signs. In: Proceeding of the 2007 IEEE Intelligent Vehicles Symposium, pp. 166–171 (2007). doi:10.1109/IVS.2007.4290109

  15. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. (IJCV) 60(2) (2004)

  16. Maldonado-Bascon, S., Lafuente-Arroyo, S., Gil-Jimenez, P., Gomez-Moreno, H., Lopez-Ferreras, F.: Road-sign detection and recognition based on support vector machines. IEEE Trans. Intell. Transp. Syst. 8(2), 264–278 (2007). doi:10.1109/TITS.2007.895311

  17. Mathias, M., Timofte, R., Benenson, R., Gool, L.J.V.: Traffic sign recognition—how far are we from the solution? In: IJCNN, pp. 1–8. IEEE, New York (2013)

  18. Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. IEEE Trans. Pattern Anal. Mach. Intell. 27(10), 1615–1630 (2005). doi:10.1109/TPAMI.2005.188

    Article  Google Scholar 

  19. Mogelmose, A., Trivedi, M., Moeslund, T.: Vision-based traffic sign detection and analysis for intelligent driver assistance systems: perspectives and survey. IEEE Trans. Intell. Transp. Syst. 13(4), 1484–1497 (2012). doi:10.1109/TITS.2012.2209421

    Article  Google Scholar 

  20. Overett, G., Petersson, L.: Large scale sign detection using hog feature variants. In: Proceeding of the 2011 IEEE Intelligent Vehicles Symposium (IV), pp. 326–331 (2011). doi:10.1109/IVS.2011.5940549

  21. Stallkamp, J., Schlipsing, M., Salmen, J., Igel, C.: Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition. Neural Netw. 0 (2012). doi:10.1016/j.neunet.2012.02.016

  22. Thain, D., Tannenbaum, T., Livny, M.: Distributed computing in practice: the condor experience. Concurr. Pract. Exp. 17(2–4), 323–356 (2005)

    Article  Google Scholar 

  23. Timofte, R., Gool, L.V.: Sparse representation based projections. In: Proceedings of the British Machine Vision Conference, pp. 61.1–61.12. BMVA Press, USA (2011). http://dx.doi.org/10.5244/C.25.61

  24. Timofte, R., Zimmermann, K., Van Gool, L.: Multi-view traffic sign detection, recognition, and 3d localisation. In: Proceeding of the 2009 Workshop on Applications of Computer Vision (WACV), pp. 1–8 (2009). doi:10.1109/WACV.2009.5403121

  25. Timofte, R., Zimmermann, K., Van Gool, L.: Multi-view traffic sign detection, recognition, and 3d localisation. Mach. Vis. Appl. (2011). doi:10.1007/s00138-011-0391-3

Download references

Author information

Authors and Affiliations

  1. CycloMedia Technology B.V., Van Voordenpark 1b, 5301 KP , Zaltbommel, The Netherlands

    Lykele Hazelhoff & Ivo M. Creusen

  2. Eindhoven University of Technology, Den Dolech 2, 5600 MB , Eindhoven, The Netherlands

    Peter H. N. de With

Authors
  1. Lykele Hazelhoff
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ivo M. Creusen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Peter H. N. de With
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lykele Hazelhoff.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hazelhoff, L., Creusen, I.M. & de With, P.H.N. Exploiting street-level panoramic images for large-scale automated surveying of traffic signs. Machine Vision and Applications 25, 1893–1911 (2014). https://doi.org/10.1007/s00138-014-0628-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-014-0628-z

Keywords

Search

Navigation