Skip to main content
SpringerLink
Log in

Fine localization and distortion resistant detection of multi-class barcode in complex environments

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Barcode, including one-dimensional (1D) barcode and two-dimensional (2D) barcode, can be seen almost anywhere in our lives. In many barcode-based mobile systems, different barcodes will appear simultaneously with different angles, shapes, and image quality. Barcode localization is a significant prerequisite for barcode decoding in these applications. In this paper, we propose a region-based end-to-end network to finely localize and classify 1D barcode and Quick Response (QR) code in complex environments. Two special layers are designed in our network. One is a quadrilateral regression layer to localize arbitrary quadrilateral bounding boxes, and another is a Multi-scale Spatial Pyramid Pooling (MSPP) layer to improve the detection accuracy of small-scale barcodes. Extensive experiments on existing public datasets and our own dataset have verified the effectiveness of proposed layers. We also demonstrate that our method can resist some distortions by simulating barcode images of different image qualities. What’s more, a human decoding experiment is also performed to prove the effectiveness of our method as a preprocessor for QR code decoding.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Al-Khalifa HS (2008) Utilizing qr code and mobile phones for blinds and visually impaired people. In: International conference on computers helping people with special needs (ICCHP), pp 1065–1069

  2. Anezaki T, Eimon K, Tansuriyavong S, Yagi Y (2011) Development of a human-tracking robot using qr code recognition. In: Frontiers of computer vision (FCV), pp 1–6

  3. Ardón P, Dragone M, Erden MS (2018) Reaching and grasping of objects by humanoid robots through visual servoing. In: Haptic: science, technology, and applications, pp 353–365

  4. Belussi L, Hirata N (2011) Fast qr code detection in arbitrarily acquired images. In: SIBGRAPI conference on graphics, patterns and images (SIBGRAPI), pp 281–288

  5. Busta M, Neumann L, Matas J (2017) Deep textspotter: an end-to-end trainable scene text localization and recognition framework. In: The IEEE international conference on computer vision (ICCV)

  6. Cai Z, Fan Q, Feris RS, Vasconcelos N (2016) A unified multi-scale deep convolutional neural network for fast object detection. In: European conference on computer vision (ECCV), pp 354–370

  7. Chang Y, Tsai S, Chang Y, Wang T (2007) A novel wayfinding system based on geo-coded qr codes for individuals with cognitive impairments. In: International ACM SIGACCESS conference on computers and accessibility, pp 231–232

  8. Dai J, Li Y, He K, Sun J (2016) R-fcn: object detection via region-based fully convolutional networks. In: Advances in neural information processing systems (NIPS), pp 379–387

  9. Dollár P, Appel R, Belongie S, Perona P (2014) Fast feature pyramids for object detection. IEEE Trans Pattern Anal Mach Intell (PAMI) 36(8):1532–1545

    Article  Google Scholar 

  10. Dubská M, Herout A, Havel J (2016) Real-time precise detection of regular grids and matrix codes. J Real-Time Image Process (JRTIP), 193–200

  11. Ebrahim Y, Abdelsalam W, Ahmed M, Chau SC (2005) Proposing a hybrid tag-camera-based identification and navigation aid for the visually impaired. In: Second IEEE consumer communications and networking conference (CCNC), pp 172–177

  12. Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: 2014 IEEE Conference on computer vision and pattern recognition, pp 580–587. https://doi.org/10.1109/CVPR.2014.81

  13. Grósz T, Bodnár P, Tóth L, Nyúl LG (2014) Qr code localization using deep neural networks. In: IEEE International workshop on machine learning for signal processing (MLSP), pp 1–6

  14. Hansen DK, Nasrollahi K, Rasmusen CB, Moeslund TB (2017) Real-time barcode detection and classification using deep learning. In: International joint conference on computational intelligence (IJCCI), pp 321–327

  15. He K, Zhang X, Ren S, Sun J (2015) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Mach Intell (PAMI) 37(9):1904–1916

    Article  Google Scholar 

  16. Hu GX, Yang Z, Hu L, Huang L, Han JM (2018) Small object detection with multiscale features. International Journal of Digital Multimedia Broadcasting (JDMB

  17. Katona M, Nyúl LG (2012) A novel method for accurate and efficient barcode detection with morphological operations. In: International conference on signal image technology and internet based systems (SITIS), pp 307–314

  18. Lin TY, Dollar P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. In: The IEEE conference on computer vision and pattern recognition (CVPR)

  19. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C, Berg AC (2016) Ssd: single shot multibox detector. In: European conference on computer vision (ECCV), pp 21–37

  20. Ma J, Shao W, Ye H, Wang L, Wang H, Zheng Y, Xue X (2018) Arbitrary-oriented scene text detection via rotation proposals. IEEE Trans Multimed (TMM) 20(11):3111–3122

    Article  Google Scholar 

  21. Nguyen TV, Le T, Bui Q, Tran M, Duong A (2012) Smart shopping assistant: a multimedia and social media augmented system with mobile devices to enhance customers’ experience and interaction. In: Pacific Asia conference on information systems (PACIS) 2012 proceedings, p 95

  22. Pandya KH, Galiyawala HJ (2014) A survey on qr codes: in context of research and application. Int J Emerg Technol Adv Eng (JETAE) 4(3):258–262

    Google Scholar 

  23. Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified real-time object detection. In: The IEEE Conference on computer vision and pattern recognition (CVPR)

  24. Ren S, He R, Girshick K, Sun J (2015) Faster r-cnn: towards real-time object detection with region proposal networks. In: Neural information processing systems (NIPS), pp 91–99

  25. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:abs/1409.1556

  26. Sörös G, Flörkemeier C (2013) Blur-resistant joint 1d and 2d barcode localization for smartphones. In: International conference on mobile and ubiquitous multimedia (MUM), pp 11:1–11:8

  27. Szentandrási I, Herout A, Dubská M (2012) Fast detection and recognition of qr codes in high-resolution images. In: Conference on computer graphics (CCG), pp 129–136

  28. Uijlings JRR, van de Sande KEA, Gevers T, Smeulders AWM (2013) Selective search for object recognition. Int J Comput Vis 104(2):154–171. https://doi.org/10.1007/s11263-013-0620-5

  29. Wachenfeld S, Terlunen S, Jiang X (2008) Robust recognition of 1-d barcodes using camera phones. In: International conference on pattern recognition (ICPR), pp 1–4

  30. Walsh A (2010) Qr codes – using mobile phones to deliver library instruction and help at the point of need. J Inform Liter 4:1

    Article  Google Scholar 

  31. Xu W, McCloskey S (2011) 2d barcode localization and motion deblurring using a flutter shutter camera. In: IEEE Workshop on applications of computer vision (WACV), pp 159–165

  32. Zamberletti A, Gallo I, Albertini S (2013) Robust angle invariant 1d barcode detection. In: Asian conference on pattern recognition (ACPR), pp 160–164

Download references

Author information

Authors and Affiliations

  1. Department of Electronic Engineering, Shanghai Jiao Tong University, Shanghai, China

    Jiahe Zhang, Xiongkuo Min, Jun Jia, Zehao Zhu, Jia Wang & Guangtao Zhai

Authors
  1. Jiahe Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiongkuo Min
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jun Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zehao Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jia Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Guangtao Zhai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guangtao Zhai.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, J., Min, X., Jia, J. et al. Fine localization and distortion resistant detection of multi-class barcode in complex environments. Multimed Tools Appl 80, 16153–16172 (2021). https://doi.org/10.1007/s11042-019-08578-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-08578-x

Keywords

Search

Navigation