Abstract
Weakly supervised object localization (WSOL) is a task of localizing an object in an image only using image-level labels. To tackle the WSOL problem, most previous studies have followed the conventional class activation mapping (CAM) pipeline: (i) training CNNs for a classification objective, (ii) generating a class activation map via global average pooling (GAP) on feature maps, and (iii) extracting bounding boxes by thresholding based on the maximum value of the class activation map. In this work, we reveal the current CAM approach suffers from three fundamental issues: (i) the bias of GAP that assigns a higher weight to a channel with a small activation area, (ii) negatively weighted activations inside the object regions and (iii) instability from the use of the maximum value of a class activation map as a thresholding reference. They collectively cause the problem that the localization to be highly limited to small regions of an object. We propose three simple but robust techniques that alleviate the problems, including thresholded average pooling, negative weight clamping, and percentile as a standard for thresholding. Our solutions are universally applicable to any WSOL methods using CAM and improve their performance drastically. As a result, we achieve the new state-of-the-art performance on three benchmark datasets of CUB-200–2011, ImageNet-1K, and OpenImages30K.
W. Bae and J. Noh—Equal contribution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bazzani, L., Bergamo, A., Anguelov, D., Torresani, L.: Self-Taught Object Localization With Deep Networks. In: WACV (2016)
Benenson, R., Popov, S., Ferrari, V.: Large-scale Interactive Object Segmentation with Human Annotators. In: CVPR (2019)
Chattopadhay, A., Sarkar, A., Howlader, P., Balasubramanian, V.N.: Grad-Cam++: Improved Visual Explanations for Deep Convolutional Networks. In: WACV (2018)
Choe, J., Oh, S.J., Lee, S., Chun, S., Akata, Z., Shim, H.: Evaluating Weakly Supervised Object Localization Methods Right. In: CVPR (2020)
Choe, J., Park, J.H., Shim, H.: Improved Techniques for Weakly-Supervised Object Localization. Arxiv:1802.07888 (2018)
Choe, J., Shim, H.: Attention-Based Dropout Layer for Weakly Supervised Object Localization. In: CVPR (2019)
Dai, J., Li, Y., He, K., Sun, J.: R-FCN: Object Detection via Region-Based Fully Convolutional Networks. In: NeurIPS (2016)
Durand, T., Mordan, T., Thome, N., Cord, M.: WILDCAT: Weakly Supervised Learning of Deep ConvNets for Image Classification, Pointwise Localization and Segmentation. In: CVPR (2017)
Durand, T., Thome, N., Cord, M.: WELDON: Weakly Supervised Learning of Deep Convolutional Neural Networks. In: CVPR (2016)
Everingham, M., Eslami, S.M.A., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The Pascal Visual Object Classes Challenge: A Retrospective. IJCV (2015)
He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask R-CNN. In: ICCV (2017)
He, K., Zhang, X., Ren, S., Sun, J.: Deep Residual Learning for Image Recognition. In: CVPR (2016)
Hou, Q., Jiang, P., Wei, Y., Cheng, M.M.: Self-Erasing Network for Integral Object Attention. In: NeurIPS (2018)
Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., Adam, H.: MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. Arxiv:1704.04861 (2017)
Hu, J., Shen, L., Sun, G.: Squeeze-and-Excitation Networks. In: CVPR (2018)
Ioffe, S., Szegedy, C.: Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. In: ICML (2015)
Kim, D., Cho, D., Yoo, D., Kweon, I.: Two-Phase Learning for Weakly Supervised Object Localization. In: ICCV (2017)
Li, K., Wu, Z., Peng, K.C., Ernst, J., Fu, Y.: Tell Me Where to Look: Guided Attention Inference Network. In: CVPR (2018)
Lin, M., Chen, Q., Yan, S.: Network in Network. In: ICLR (2014)
Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal Loss for Dense Object Detection. In: ICCV (2017)
Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft COCO: Common Objects in Context. In: ECCV (2014)
Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., Berg, A.C.: SSD: Single Shot Multibox Detector. In: ECCV (2016)
Oquab, M., Bottou, L., Laptev, I., Sivic, J.: Is Object Localization for Free? - Weakly-Supervised Learning With Convolutional Neural Networks. In: CVPR (2015)
Pinheiro, P.O., Collobert, R.: From Image-Level to Pixel-Level Labeling With Convolutional Networks. In: CVPR (2015)
Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You Only Look Once: Unified, Real-Time Object Detection. In: CVPR (2016)
Redmon, J., Farhadi, A.: YOLO9000: Better, Faster. CVPR, Stronger. In (2017)
Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. In: NeurIPS (2015)
Russakovsky, O., Deng, J., SU, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C., Fei-Fei, L.: Imagenet Large Scale Visual Recognition Challenge. IJCV (2015)
Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-Cam: Visual Explanations From Deep Networks via Gradient-Based Localization. In: ICCV (2017)
Simonyan, K., Zisserman, A.: Very Deep Convolutional Networks for Large-Scale Image Recognition. Arxiv:1409.1556 (2014)
Singh, K.K., Lee, Y.J.: Hide-And-Seek: Forcing a Network to Be Meticulous for Weakly-Supervised Object and Action Localization. In: ICCV (2017)
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going Deeper with Convolutions. In: CVPR (2015)
Wah, C., Branson, S., Welinder, P., Perona, P., Belongie, S.: The Caltech-UCSD Birds-200-2011 Dataset. Tech. Rep. Cns-Tr-2011-001, California Institute of Technology (2011)
Wei, Y., Feng, J., Liang, X., Cheng, M.M., Zhao, Y., Yan, S.: Object Region Mining With Adversarial Erasing: A Simple Classification to Semantic Segmentation Approach. In: CVPR (2017)
Wei, Y., Shen, Z., Cheng, B., Shi, H., Xiong, J., Feng, J., Huang, T.: TS2C: Tight Box Mining with Surrounding Segmentation Context for Weakly Supervised Object Detection. In: ECCV (2018)
Xue, H., Wan, F., Jiao, J., Ji, X., Qixiang, Y.: DANet: Divergent Activation for Weakly supervised Object Localization. In: ICCV (2019)
Yun, S., Han, D., Oh, S.J., Chun, S., Choe, J., Yoo, Y.: CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features. In: ICCV (2019)
Zhang, X., Wei, Y., Feng, J., Yang, Y., Huang, T.S.: Adversarial Complementary Learning for Weakly Supervised Object Localization. In: CVPR (2018)
Zhang, X., Wei, Y., Kang, G., Yang, Y., Huang, T.: Self-Produced Guidance for Weakly-Supervised Object Localization. In: ECCV (2018)
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning Deep Features for Discriminative Localization. In: CVPR (2016)
Acknowledgements
We appreciate Hyunwoo Kim and Jinhwan Seo for their valuable comments. This work was supported by AIR Lab (AI Research Lab) in Hyundai Motor Company through HMC-SNU AI Consortium Fund, and the ICT R&D program of MSIT/IITP (No. 2019-0-01309, Development of AI technology for guidance of a mobile robot to its goal with uncertain maps in indoor/outdoor environments and No.2019-0-01082, SW StarLab).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Bae, W., Noh, J., Kim, G. (2020). Rethinking Class Activation Mapping for Weakly Supervised Object Localization. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, JM. (eds) Computer Vision – ECCV 2020. ECCV 2020. Lecture Notes in Computer Science(), vol 12360. Springer, Cham. https://doi.org/10.1007/978-3-030-58555-6_37
Download citation
DOI: https://doi.org/10.1007/978-3-030-58555-6_37
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-58554-9
Online ISBN: 978-3-030-58555-6
eBook Packages: Computer ScienceComputer Science (R0)