skip to main content
10.1145/3409334.3452063acmconferencesArticle/Chapter ViewAbstractPublication Pagesacm-seConference Proceedingsconference-collections
research-article

A computer vision pipeline for automatic large-scale inventory tracking

Published: 10 May 2021 Publication History

Abstract

Monitoring and tracking inventory is one of the most important aspects of administrating any large-scale enterprise operation that involves physical goods. One of the most evident examples of such operations is automotive manufacturing, especially for servicing a global customer base. We present a software solution of Intelligent Process Automation (IPA) that utilizes state-of-the-art computer vision (CV) and other algorithmic techniques to locate, detect, and manage inventory storage logistics using label information from simple warehouse images. When used in conjunction with a recently developed robotic imaging system, our pipeline can be shown to replace the need for costly, error-prone human input to the inventory tracking system. This paper outlines the technical and practical application of IPA fueled by deep learning. The specific motivation for this project was to address a critical need of Mercedes-Benz U.S. International (MBUSI), but the techniques could be applied more generally to other inventory management contexts. We also discuss how our pipeline produces an inexpensive, efficient, and generalizable solution that provides the capability to retrieve data from an unpredictable environment, in contrast to previous approaches.

References

[1]
M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Isard, Y. Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Levenberg, D. Mané, R. Monga, S. Moore, D. Murray, C. Olah, M. Schuster, J. Shlens, B. Steiner, I. Sutskever, K. Talwar, P. Tucker, V. Vanhoucke, V. Vasudevan, F. Viégas, O. Vinyals, P. Warden, M. Wattenberg, M. Wicke, Y. Yu, and X. Zheng. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. https://www.tensorflow.org/.
[2]
A. Bochkovskiy, C. Wang, and H. M. Liao. 2020. YOLOv4: Optimal Speed and Accuracy of Object Detection. arXiv:2004.10934.
[3]
BoofCV 2019. Comparison of QR Code Performance. https://boofcv.org/index.php?title=Performance:QrCode.
[4]
G. Bradski. 2000. The OpenCV Library. Dr. Dobb's Journal of Software Tools (November 2000), 122--125.
[5]
J. Canny. 1986. A Computational Approach to Edge Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence PAMI-8, 6 (1986), 679--698.
[6]
L. Chandrasekar and G. Durga. 2014. Implementation of Hough Transform for Image Processing Applications. In 2014 International Conference on Communication and Signal Processing. 843--847.
[7]
CoralUSB 2019. Coral USB Accelerator. https://coral.ai/products/accelerator/.
[8]
C. Dong, C. C. Loy, and X. Tang. 2018. Accelerating the Super-Resolution Convolutional Neural Network. In Computer Vision - ECCV 2016 - 14th European Conference. 391--407.
[9]
Gaussian Blur 2020. Convolution and Smoothening Images. https://docs.opencv.org/master/d4/d13/tutorial_py_filtering.html.
[10]
K. Katircioglu and Y. Li. 2015. Machine Vision Technology for Shelf Inventory Management. https://patents.google.com/patent/US20150262116A1/en#citedBy. US Patent no: US20150262116A1.
[11]
W. Lai, J. Huang, N. Ahuja, and M. Yang. 2019. Fast and Accurate Image Super-Resolution with Deep Laplacian Pyramid Networks. In IEEE Transactions on Pattern Analysis and Machine Intelligence. 41(11): 2599--2613.
[12]
T. Lin, M. Maire, S. Belongie, L. Bourdev, R. Girshick, J. Hays, P. Perona, D. Ramanan, C. L. Zitnick, and P. DollÃąr. 2014. Microsoft COCO: Common Objects in Context. In Computer Vision - ECCV 2014 - 13th European Conference. 740--755.
[13]
W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C. Fu, and A. C. Berg. 2016. SSD: Single Shot MultiBox Detector. In Computer Vision - ECCV 2016 - 14th European Conference. 21âĂŞ37.
[14]
P. Maragos and R. Schafer. 1986. Applications of Morphological Filtering to Image Analysis and Processing. In ICASSP '86. IEEE International Conference on Acoustics, Speech, and Signal Processing, Vol. 11. 2067--2070.
[15]
NVIDIAJetson 2018. NVIDIA Autonomous Machines. https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/.
[16]
Otsu's Binarization 2020. Image Thresholding. https://docs.opencv.org/master/d7/d4d/tutorial_py_thresholding.html.
[17]
A. D. Patel and A. R. Chowdhury. 2020. Vision-based Object Classification using Deep Learning for Inventory Tracking in Automated Warehouse Environment. In 2020 20th International Conference on Control, Automation and Systems (ICCAS). 145--150.
[18]
RaspberryPi 2019. Raspberry Pi 4 Model B. https://www.raspberrypi.org/products/raspberry-pi-4-model-b/.
[19]
J. Redmon, S. Divvala, R. B. Girshick, and A. Farhadi. 2016. You Only Look Once: Unified, Real-Time Object Detection. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), 779--788.
[20]
J. Redmon and A. Farhadi. 2018. YOLOv3: An Incremental Improvement. CoRR abs/1804.02767 (2018). arXiv:1804.02767 http://arxiv.org/abs/1804.02767.
[21]
M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L. Chen. 2019. MobileNetV2: Inverted Residuals and Linear Bottlenecks. arXiv:1801.04381.
[22]
W. Shi, J. Caballero, F. HuszÃąr, J. Totz, A. P. Aitken, R. Bishop, D. Rueckert, and Z. Wang. 2016. Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network. In IEEE Conference on Computer Vision and Pattern Recognition. 1874--1883.
[23]
R. Smith. 2007. An Overview of the Tesseract OCR Engine. In Proc. Ninth Int. Conference on Document Analysis and Recognition (ICDAR). 629--633.
[24]
M. Stern and B. Bekritsky. 2011. Real-time Automatic RFID Inventory ControlSsystem. https://patents.google.com/patent/US8077041B2/en. US Patent no: US8077041B2.
[25]
Tzutalin. 2015. LabelImg. Free Software: MIT License. https://github.com/tzutalin/labelImg.
[26]
F. J. Valente and A. C. Neto. 2017. Intelligent Steel Inventory Tracking with IoT / RFID. In 2017 IEEE International Conference on RFID Technology Application (RFID-TA). 158--163.
[27]
H. Yang and S. Yang. 2009. Connectionless Indoor Inventory Tracking in Zigbee RFID Sensor Network. In 2009 35th Annual Conference of IEEE Industrial Electronics. 2618--2623.
[28]
Zbar Software 2011. Zbar Barcode Reader. http://zbar.sourceforge.net/.
[29]
X. Zhou, C. Yao, H. Wen, Y. Wang, S. Zhou, W. He, and J. Liang. 2017. EAST: An Efficient and Accurate Scene Text Detector. In IEEE Conference on Computer Vision and Pattern Recognition. 2641--2651.
[30]
ZXing Library 2008. ZXing Image Processing Library. https://opensource.google/projects/zxing.

Cited By

View all
  • (2023)A Comprehensive Framework for Industrial Sticker Information Recognition Using Advanced OCR and Object Detection TechniquesApplied Sciences10.3390/app1312732013:12(7320)Online publication date: 20-Jun-2023
  • (2023)Training industrial engineers in Logistics 4.0Computers and Industrial Engineering10.1016/j.cie.2023.109550184:COnline publication date: 1-Oct-2023
  • (2022)Application of Deep Learning Techniques and Bayesian Optimization with Tree Parzen Estimator in the Classification of Supply Chain Pricing Datasets of Health MedicationsApplied Sciences10.3390/app12191016612:19(10166)Online publication date: 10-Oct-2022

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ACMSE '21: Proceedings of the 2021 ACM Southeast Conference
April 2021
263 pages
ISBN:9781450380683
DOI:10.1145/3409334
  • Conference Chair:
  • Kazi Rahman,
  • Program Chair:
  • Eric Gamess
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 May 2021

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. computer vision
  2. deep learning
  3. intelligent process automation
  4. large-scale tracking
  5. object detection
  6. pattern recognition
  7. robotic process automation

Qualifiers

  • Research-article

Conference

ACM SE '21
Sponsor:
ACM SE '21: 2021 ACM Southeast Conference
April 15 - 17, 2021
Virtual Event, USA

Acceptance Rates

Overall Acceptance Rate 502 of 1,023 submissions, 49%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)62
  • Downloads (Last 6 weeks)3
Reflects downloads up to 18 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2023)A Comprehensive Framework for Industrial Sticker Information Recognition Using Advanced OCR and Object Detection TechniquesApplied Sciences10.3390/app1312732013:12(7320)Online publication date: 20-Jun-2023
  • (2023)Training industrial engineers in Logistics 4.0Computers and Industrial Engineering10.1016/j.cie.2023.109550184:COnline publication date: 1-Oct-2023
  • (2022)Application of Deep Learning Techniques and Bayesian Optimization with Tree Parzen Estimator in the Classification of Supply Chain Pricing Datasets of Health MedicationsApplied Sciences10.3390/app12191016612:19(10166)Online publication date: 10-Oct-2022

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media