research-article

Take Goods from Shelves: A Dataset for Class-Incremental Object Detection

Authors:

Yu-Gang JiangAuthors Info & Claims

ICMR '19: Proceedings of the 2019 on International Conference on Multimedia Retrieval

Pages 271 - 278

https://doi.org/10.1145/3323873.3325033

Published: 05 June 2019 Publication History

Abstract

Object detection for automatic visual checkout in self-service vending machines is attracting significant attention in the retail industry. However, several critical challenges have not received enough attention. First, large-scale, high-quality retail image datasets are urgently demanded to train and evaluate the detection models. Second, the trained models should be able to cope with the frequently added new goods at low cost, while most cutting-edge models cannot. In this paper, we propose a new hierarchical large-scale object detection dataset, called Take Goods from Shelves (TGFS), containing 38K images of 24 fine-grained and 3 coarse classes. A preliminary method for solving the goods-adding problem, called Faster R-CNN Class-incremental Object Detector (FCIOD), is also described and evaluated. In addition, several popular methods are benchmarked on the TGFS dataset.

References

[1]

Jifeng Dai, Yi Li, Kaiming He, and Jian Sun. 2016. R-FCN: Object Detection via Region-based Fully Convolutional Networks. arXiv (2016).

[2]

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. ImageNet: A Large-Scale Hierarchical Image Database. In CVPR.

[3]

Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. 2010. The Pascal Visual Object Classes (VOC) Challenge. IJCV, Vol. 88, 2 (2010), 303--338.

Digital Library

[4]

Patrick Follmann, Bertram Drost, and Tobias Böttger. 2018. Acquire, Augment, Segment & Enjoy: Weakly Supervised Instance Segmentation of Supermarket Products. ArXiv (2018).

[5]

Ross Girshick. 2015. Fast R-CNN. In ICCV.

Digital Library

[6]

Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. 2014. Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. In CVPR.

Digital Library

[7]

Yu Hao, Yanwei Fu, Yu-Gang Jiang, and Qi Tian. 2019. An End-to-End Architecture for Class-incremental Object Detection With Knowledge Distillation. In ICME.

[8]

Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. 2017. Mask R-CNN. In ICCV.

[9]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Deep Residual Learning for Image Recognition. ArXiv (2015).

[10]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In CVPR.

[11]

Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. Distilling the Knowledge in a Neural Network. ArXiv (2015).

[12]

Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. 2017. Densely Connected Convolutional Networks. In CVPR.

[13]

Philipp Jund, Nichola Abdo, Andreas Eitel, and Wolfram Burgard. 2016. The Freiburg Groceries Dataset. ArXiv (2016).

[14]

Heechul Jung, Jeongwoo Ju, Minju Jung, and Junmo Kim. 2016. Less-forgetting Learning in Deep Neural Networks. ArXiv (2016).

[15]

Alex Krizhevsky. 2014. One weird trick for parallelizing convolutional neural networks. ArXiv (2014).

[16]

Zhizhong Li and Derek Hoiem. 2018. Learning without forgetting. IEEE TPAMI, Vol. 40, 12 (2018), 2935--2947.

Digital Library

[17]

Zeming Li, Chao Peng, Gang Yu, Xiangyu Zhang, Yangdong Deng, and Jian Sun. 2017. Light-Head R-CNN: In Defense of Two-Stage Object Detector. ArXiv (2017).

[18]

Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014. Microsoft COCO: Common Objects in Context. ArXiv (2014).

[19]

Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C Berg. 2016. SSD: Single Shot MultiBox Detector. In ECCV.

[20]

Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. 2017. iCaRL: Incremental Classifier and Representation Learning. In CVPR.

[21]

Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. 2016. You Only Look Once: Unified, Real-Time Object Detection. In CVPR.

[22]

Joseph Redmon and Ali Farhadi. 2017. YOLO9000: Better, Faster, Stronger. In CVPR.

[23]

Joseph Redmon and Ali Farhadi. 2018. YOLOv3: An Incremental Improvement. ArXiv (2018).

[24]

Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. In NIPS.

Digital Library

[25]

Jose Rivera-Rubio, Saad Idrees, Ioannis Alexiou, Lucas Hadjilucas, and Anil A Bharath. 2014. Small Hand-held Object Recognition Test (SHORT). In WACV. 524--531.

[26]

Zhiqiang Shen, Zhuang Liu, Jianguo Li, Yu-Gang Jiang, Yurong Chen, and Xiangyang Xue. 2017. DSOD: Learning Deeply Supervised Object Detectors From Scratch. In ICCV.

[27]

Konstantin Shmelkov, Cordelia Schmid, and Karteek Alahari. 2017. Incremental Learning of Object Detectors Without Catastrophic Forgetting. In ICCV.

[28]

Karen Simonyan and Andrew Zisserman. 2014. Very Deep Convolutional Networks for Large-Scale Image Recognition. ArXiv (2014).

[29]

Luchao Tian, Mingchen Li, Yu Hao, Jun Liu, Guyue Zhang, and Yan Qiu Chen. 2018. Robust 3-D Human Detection in Complex Environments With a Depth Camera. IEEE TMM, Vol. 20, 9 (2018), 2249--2261.

Digital Library

[30]

Jasper RR Uijlings, Koen EA Van De Sande, Theo Gevers, and Arnold WM Smeulders. 2013. Selective Search for Object Recognition. IJCV, Vol. 104, 2 (2013), 154--171.

Digital Library

[31]

Gül Varol and Ridvan Salih Kuzu. 2015. Toward retail product recognition on grocery shelves. In ICGIP.

[32]

C Lawrence Zitnick and Piotr Dollár. 2014. Edge Boxes: Locating Object Proposals from Edges. In ECCV.

Cited By

Ding LSong XHe YWang CDong SWei XGong Y(2024)Domain Incremental Object Detection Based on Feature Space Topology Preserving StrategyIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2023.328526334:1(424-437)Online publication date: Jan-2024
https://doi.org/10.1109/TCSVT.2023.3285263
Jiang LCheng HYe XOuyang Y(2024)Multi-Teacher Distillation for Incremental Object DetectionICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP48485.2024.10447964(5520-5524)Online publication date: 14-Apr-2024
https://doi.org/10.1109/ICASSP48485.2024.10447964
Belouadah EDapogny ABailly K(2024)MultIOD: Rehearsal-free Multihead Incremental Object Detector2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW63382.2024.00414(4107-4117)Online publication date: 17-Jun-2024
https://doi.org/10.1109/CVPRW63382.2024.00414
Show More Cited By

Index Terms

Take Goods from Shelves: A Dataset for Class-Incremental Object Detection
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Object detection
      2. Computer vision tasks

Recommendations

Branding Conspicuous Goods: An Analysis of the Effects of Social Influence and Competition

Branding decisions are critical for the success of new products. Prior research on branding and brand extension has primarily focused on how branding influences consumers' perceptions of product quality. However, consumers of conspicuous goods care not ...
Channel Coordination of Storable Goods
This paper investigates how a manufacturer should respond to the retailer’s and consumers’ stockpiling ability by contracting with the retailer.
Manufacturers of consumer-packaged goods invest heavily in trade promotions (i.e., temporary wholesale price discounts), but retailer stockpiling often yields trade promotions unprofitable. In this paper, we investigate how a manufacturer should respond ...
Durable Products with Multiple Used Goods Markets: Product Upgrade and Retail Pricing Implications

Used goods markets are currently important transaction channels for durable products. For some durable products, such markets first appeared when retailers started buying back used products from “old” customers and selling them to new ones for a profit (...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMR '19: Proceedings of the 2019 on International Conference on Multimedia Retrieval

June 2019

427 pages

ISBN:9781450367653

DOI:10.1145/3323873

General Chairs:
Abdulmotaleb El Saddik
University of Ottawa, Canada
,
Alberto Del Bimbo
University of Florence, Italy
,
Zhongfei Zhang
Binghamton University, State University of New York, USA
,
Program Chairs:
Alexander Hauptmann
Carnegie Mellon University, USA
,
K. Selcuk Candan
Arizona State University, USA
,
Marco Bertini
University of Florence, Italy
,
Lexing Xie
Australia National University, Australia
,
Xiao-Yong Wei
Sichuan University, China

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 June 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Natural Science Foundation of China

Conference

ICMR '19

Sponsor:

SIGMM

ICMR '19: International Conference on Multimedia Retrieval

June 10 - 13, 2019

Ottawa ON, Canada

Acceptance Rates

Overall Acceptance Rate 254 of 830 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

21
Total Citations
View Citations
575
Total Downloads

Downloads (Last 12 months)37
Downloads (Last 6 weeks)1

Reflects downloads up to 27 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Ding LSong XHe YWang CDong SWei XGong Y(2024)Domain Incremental Object Detection Based on Feature Space Topology Preserving StrategyIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2023.328526334:1(424-437)Online publication date: Jan-2024
https://doi.org/10.1109/TCSVT.2023.3285263
Jiang LCheng HYe XOuyang Y(2024)Multi-Teacher Distillation for Incremental Object DetectionICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP48485.2024.10447964(5520-5524)Online publication date: 14-Apr-2024
https://doi.org/10.1109/ICASSP48485.2024.10447964
Belouadah EDapogny ABailly K(2024)MultIOD: Rehearsal-free Multihead Incremental Object Detector2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW63382.2024.00414(4107-4117)Online publication date: 17-Jun-2024
https://doi.org/10.1109/CVPRW63382.2024.00414
Sheshappanavar SAnvekar TKundargi SWang YKambhamettu C(2024)A Benchmark Grocery Dataset of Realworld Point Clouds From Single View2024 International Conference on 3D Vision (3DV)10.1109/3DV62453.2024.00011(516-527)Online publication date: 18-Mar-2024
https://doi.org/10.1109/3DV62453.2024.00011
Zhao YYang DCao SCai BMaryamah MSolihin M(2024)Object detection in smart indoor shopping using an enhanced YOLOv8n algorithmIET Image Processing10.1049/ipr2.1328418:14(4745-4759)Online publication date: 21-Nov-2024
https://doi.org/10.1049/ipr2.13284
Pietrini RPaolanti MMancini AFrontoni EZingaretti P(2024)Shelf Management: A deep learning-based system for shelf visual monitoringExpert Systems with Applications10.1016/j.eswa.2024.124635255(124635)Online publication date: Dec-2024
https://doi.org/10.1016/j.eswa.2024.124635
Rane N(2023)YOLO and Faster R-CNN object detection for smart Industry 4.0 and Industry 5.0: applications, challenges, and opportunitiesSSRN Electronic Journal10.2139/ssrn.4624206Online publication date: 2023
https://doi.org/10.2139/ssrn.4624206
Yang DZhou YHong XZhang AWei XZeng LQiao ZWang WEl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)Pseudo Object Replay and Mining for Incremental Object DetectionProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3611952(153-162)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3581783.3611952
Li ZXu PChang XYang LZhang YYao LChen X(2023)When Object Detection Meets Knowledge Distillation: A SurveyIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.325754645:8(10555-10579)Online publication date: Aug-2023
https://doi.org/10.1109/TPAMI.2023.3257546
Liu GZhao WZhao J(2023)Decoupled Mutual Distillation for Incremental Object Detection2023 IEEE International Conference on Multimedia and Expo (ICME)10.1109/ICME55011.2023.00142(798-803)Online publication date: Jul-2023
https://doi.org/10.1109/ICME55011.2023.00142
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten