BSDIB: Balanced Self-Distillation Information Bottleneck for Long-Tailed Visual Recognition
Authors: 
Xin Cai, Shan TanAuthors Info & Claims
Xin Cai, Shan TanAuthors Info & ClaimsPages 37 - 40
Abstract
Real-world data are usually long-tailed, which results in poor performance of visual recognition systems. Current prevailing methods decouple the training procedure into representation learning and classification. However, the two stages in such methods are independent, resulting in sub-optimal solutions. In this paper, we propose a one-stage method, namely balanced self-distillation information bottleneck (BSDIB), to take care of both representation learning and classification simultaneously. Specifically, a re-weighting strategy is used to obtain an unbiased classifier, and the class-dependent self-distillation information bottleneck is utilized to promote data representations. Extensive experiments on three commonly-used long-tailed datasets indicate that BSDIB achieves SOTA performance and the visualization results show that our approach focuses more on the target object.
References
[1]
Olga Russakovsky, Jia Deng, Hao Su (2015). Imagenet large scale visual recognition challenge. International journal of computer vision, vol. 115, pp. 211-252
[2]
Mateusz Buda, Atsuto Maki and Maciej A Mazurowski (2018). A systematic study of the class imbalance problem in convolutional neural networks. Neural networks, vol. 106, pp. 249-259
[3]
Kaidi Cao, Colin Wei, Adrien Gaidon (2019). Learning imbalanced datasets with label-distribution-aware margin loss. in NeurIPS, vol. 32
[4]
Jiawei Ren, Cunjun Yu, Shunan Sheng (2020). Balanced meta-softmax for long-tailed visual recognition. in NeurIPS, vol. 33, pp. 4175-4186
[5]
Bingyi Kang, Saining Xie, Marcus Rohrbach (2020). Decoupling representation and classifier for long-tailed recognition. in ICLR
[6]
Boyan Zhou, Quan Cui, XiuShen Wei (2020). Bbn: Bilateral-branch network with cumulative learning for long-tailed visual recognition. in CVPR, pp. 9719-9728
[7]
Zhisheng Zhong, Jiequan Cui, Shu Liu (2021). Improving calibration for long-tailed recognition. in CVPR, pp. 16489-16498
[8]
Naftali Tishby, Fernando C Pereira and William Bialek (2000). The information bottleneck method. arXiv preprint physics/0004057
[9]
Alexander A Alemi, Ian Fischer, Joshua V Dillon (2017). Deep Variational Information Bottleneck. in ICLR
[10]
Xudong Tian, Zhizhong Zhang, Shaohui Lin (2021). Farewell to mutual information: Variational distillation for cross-modal person re-identification. in CVPR, pp. 1522-1531
[11]
TsungYi Lin, Priya Goyal, Ross Girshick (2017). Focal loss for dense object detection. in ICCV, pp. 2980-2988
[12]
Bingyi Kang, Yu Li, Sa Xie (2021). Exploring balanced feature spaces for representation learning. in ICLR
[13]
Tianhong Li, Peng Cao, Yuan Yuan (2022). Targeted supervised contrastive learning for long-tailed recognition. in CVPR, pp. 6918-6928
[14]
Songyang Zhang, Zeming Li, Shipeng Yan (2021). Distribution alignment: A unified framework for long-tail visual recognition. in CVPR, pp. 2361-2370
[15]
Youngkyu Hong, Seungju Han, Kwanghee Choi (2021). Disentangling label distribution for long-tailed visual recognition. in CVPR, pp. 6626-6636
[16]
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das (2017). Grad-cam: Visual explanations from deep networks via gradient-based localization. in ICCV, pp. 618-626
Recommendations
Long-Tailed Recognition Based on Self-attention Mechanism
Advanced Intelligent Computing Technology and ApplicationsAbstractThe long-tailed distribution data poses significant challenges for visual classification tasks. The existing solutions can be categorized into three main categories, i.e., class re-balancing, information augmentation, and module improvement. ...
Cross-View Representation Learning for Multi-View Logo Classification with Information Bottleneck
MM '21: Proceedings of the 29th ACM International Conference on MultimediaMulti-view logo classification is a challenging task due to the cross-view misalignment of logo image varies under different viewpoints, large intra-classes and small inter-classes variation of logo appearance. Cross-view data can represent objects from ...
Multi-view Semantic Consistency based Information Bottleneck for Clustering
AbstractMulti-view clustering leverages diverse information sources for unsupervised clustering. While existing methods primarily focus on learning a fused representation matrix, they often overlook the impact of private information and noise. To ...
Highlights- A novel clustering approach extracts essential information from multi-view features, enhancing interpretability.
- It uses semantic consistency and information bottleneck theory for discriminative representation, focusing on crucial ...
Comments
Information & Contributors
Information
Published In
April 2023
296 pages
ISBN:9798400707544
DOI:10.1145/3616901
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 05 March 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
Conference
FAIML 2023
FAIML 2023: 2023 International Conference on Frontiers of Artificial Intelligence and Machine Learning
April 14 - 16, 2023
Beijing, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 19Total Downloads
- Downloads (Last 12 months)19
- Downloads (Last 6 weeks)2
Reflects downloads up to 01 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format