Skip Abstract Section
Abstract
Existing methods based on transfer learning leverage auxiliary information to help tail generalization and improve the performance of the tail classes. However, they cannot fully exploit the relationships between auxiliary information and tail classes and bring irrelevant knowledge to the tail classes. To solve this problem, we propose a hierarchical CNN with knowledge complementation, which regards hierarchical relationships as auxiliary information and transfers relevant knowledge to tail classes. First, we integrate semantics and clustering relationships as hierarchical knowledge into the CNN to guide feature learning. Then, we design a complementary strategy to jointly exploit the two types of knowledge, where semantic knowledge acts as a prior dependence and clustering knowledge reduces the negative information caused by excessive semantic dependence (i.e., semantic gaps). In this way, the CNN facilitates the utilization of the two complementary hierarchical relationships and transfers useful knowledge to tail data to improve long-tailed classification accuracy. Experimental results on public benchmarks show that the proposed model outperforms existing methods. In particular, our model improves accuracy by 3.46% compared with the second-best method on the long-tailed tieredImageNet dataset.
- [1] . 2015. To combat multi-class imbalanced problems by means of over-sampling techniques. Transactions on Knowledge and Data Engineering 28, 1 (2015), 238–251.Google Scholar
Digital Library
- [2] . 2019. Learning imbalanced datasets with label-distribution-aware margin loss. In International Conference on Neural Information Processing Systems. 1567–1578.Google Scholar
- [3] . 2018. Deep self-evolution clustering. Transactions on Pattern Analysis and Machine Intelligence 42, 4 (2018), 809–823.Google ScholarDigital Library
- [4] . 2002. SMOTE: Synthetic minority over-sampling technique. Artificial Intelligence Research 16, 1 (2002), 321–357.Google ScholarDigital Library
- [5] . 2021. Hierarchy-aware label semantics matching network for hierarchical text classification. In Annual Meeting of the Association for Computational Linguistics. 4370–4379.Google ScholarCross Ref
- [6] . 2019. Class-balanced loss based on effective number of samples. In Conference on Computer Vision and Pattern Recognition. 9268–9277.Google ScholarCross Ref
- [7] . 2009. ImageNet: A large-scale hierarchical image database. In Conference on Computer Vision and Pattern Recognition. 248–255.Google ScholarCross Ref
- [8] . 2012. Hedging your bets: Optimizing accuracy-specificity trade-offs in large scale visual recognition. In Conference on Computer Vision and Pattern Recognition. 3450–3457.Google Scholar
- [9] . 2022. Imbalanced sample selection with deep reinforcement learning for fault diagnosis. Transactions on Industrial Informatics 18, 4 (2022), 2518–2527.Google ScholarCross Ref
- [10] . 2021. Embedding semantic hierarchy in discrete optimal transport for risk minimization. In International Conference on Acoustics, Speech and Signal Processing. 2835–2839.Google Scholar
- [11] . 2021. Long-tailed multi-label visual recognition by collaborative training on uniform and re-balanced samplings. In Conference on Computer Vision and Pattern Recognition. 15089–15098.Google ScholarCross Ref
- [12] . 2016. Deep residual learning for image recognition. In Conference on Computer Vision and Pattern Recognition. 770–778.Google ScholarCross Ref
- [13] . 2016. Learning deep representation for imbalanced classification. In Conference on Computer Vision and Pattern Recognition. 5375–5384.Google ScholarCross Ref
- [14] . 2022. A dynamic time warping approach for handling class imbalanced medical datasets with missing values: A case study of protein localization site prediction. Expert Systems with Applications 192 (2022), 116437.Google ScholarDigital Library
- [15] . 2020. Semantic hierarchy-based convolutional neural networks for image classification. In International Joint Conference on Neural Networks. 1–8.Google Scholar
- [16] . 2000. The class imbalance problem: Significance and strategies. In International Conference on Artificial Intelligence. 111–117.Google Scholar
- [17] . 2020. M2m: Imbalanced classification via major-to-minor translation. In Conference on Computer Vision and Pattern Recognition. 13896–13905.Google ScholarCross Ref
- [18] . 2015. Evaluation measures for hierarchical classification: A unified view and novel approaches. Data Mining and Knowledge Discovery 29, 3 (2015), 820–865.Google ScholarDigital Library
- [19] . 2009. Learning Multiple Layers of Features from Tiny Images. Master’s thesis, University of Toronto. 1–58.Google Scholar
- [20] . 2017. Focal loss for dense object detection. In International Conference on Computer Vision. 2980–2988.Google ScholarCross Ref
- [21] . 2017. Clustering-based undersampling in class-imbalanced data. Information Sciences 409 (2017), 17–26.Google ScholarCross Ref
- [22] . 2021. Cluster-wise hierarchical generative model for deep amortized clustering. In Conference on Computer Vision and Pattern Recognition. 15109–15118.Google ScholarCross Ref
- [23] . 2019. Large-scale long-tailed recognition in an open world. In Conference on Computer Vision and Pattern Recognition. 2537–2546.Google ScholarCross Ref
- [24] . 2022. Semantic-gap-oriented feature selection and classifier construction in multilabel learning. Transactions on Cybernetics 52, 1 (2022), 101–115.Google ScholarCross Ref
- [25] . 2022. FW-SMOTE: A feature-weighted oversampling approach for imbalanced classification. Pattern Recognition 124 (2022), 108511.Google ScholarDigital Library
- [26] . 2021. Objective-based hierarchical clustering of deep embedding vectors. In Conference on Artificial Intelligence. 9055–9063.Google ScholarCross Ref
- [27] . 2022. Visual vs internal attention mechanisms in deep neural networks for image classification and object detection. Pattern Recognition 123 (2022), 108411.Google ScholarDigital Library
- [28] . 2001. WordNet: An electronic lexical database. Applied Psycholinguistics 22, 1 (2001), 131–134.Google ScholarCross Ref
- [29] . 2018. Meta-learning for semi-supervised few-shot classification. In International Conference on Learning Representations.Google Scholar
- [30] . 1999. Image retrieval: Current techniques, promising directions, and open issues. Journal of Visual Communication and Image Representation 10, 1 (1999), 39–62.Google ScholarDigital Library
- [31] . 2015. ImageNet large scale visual recognition challenge. International Journal of Computer Vision 115, 3 (2015), 211–252.Google ScholarDigital Library
- [32] . 2022. Discriminative feature generation for classification of imbalanced data. Pattern Recognition 122 (2022), 108302.Google ScholarDigital Library
- [33] . 2012. Inverse random under sampling for class imbalance problem and its application to multi-label classification. Pattern Recognition 45, 10 (2012), 3738–3750.Google ScholarDigital Library
- [34] . 2021. Equalization loss v2: A new gradient balance approach for long-tailed object detection. In Conference on Computer Vision and Pattern Recognition. 1685–1694.Google ScholarCross Ref
- [35] . 2020. Equalization loss for long-tailed object recognition. In Conference on Computer Vision and Pattern Recognition. 11662–11671.Google ScholarCross Ref
- [36] . 2018. The iNaturalist species classification and detection dataset. In Conference on Computer Vision and Pattern Recognition. 8769–8778.Google ScholarCross Ref
- [37] . 2017. Granular computing: From granularity optimization to multi-granularity joint problem solving. Granular Computing 2, 3 (2017), 105–120.Google ScholarCross Ref
- [38] . 2021. Seesaw loss for long-tailed instance segmentation. In Conference on Computer Vision and Pattern Recognition. 9695–9704.Google ScholarCross Ref
- [39] . 2023. Coarse-to-fine: Progressive knowledge transfer based multi-task convolutional neural network for intelligent large-scale fault diagnosis. Transactions on Neural Networks and Learning Systems 34, 2 (2023), 761–774.Google Scholar
- [40] . 2017. Learning to model the tail. In Conference on Neural Information Processing Systems. 7032–7042.Google Scholar
- [41] . 2016. SUN database: Exploring a large collection of scene categories. International Journal of Computer Vision 119, 1 (2016), 3–22.Google ScholarDigital Library
- [42] . 2010. SUN database: Large-scale scene recognition from abbey to zoo. In Conference on Computer Vision and Pattern Recognition. 3485–3492.Google ScholarCross Ref
- [43] . 2022. Deep image clustering by fusing contrastive learning and neighbor relation mining. Knowledge-Based Systems 238 (2022), 107967.Google ScholarDigital Library
- [44] . 2021. Imbalanced classification based on minority clustering synthetic minority oversampling technique with wind turbine fault detection application. Transactions on Industrial Informatics 17, 9 (2021), 5867–5875.Google ScholarCross Ref
- [45] . 2022. Solving the long-tailed problem via intra-and inter-category balance. In International Conference on Acoustics, Speech and Signal Processing. 2355–2359.Google Scholar
- [46] . 2021. A recursive regularization based feature selection framework for hierarchical classification. Transactions on Knowledge and Data Engineering 33, 7 (2021), 2833–2846.Google ScholarCross Ref
- [47] . 2022. Predicting local protein 3D structures using clustering deep recurrent neural network. Transactions on Computational Biology and Bioinformatics 19, 1 (2022), 593–604.Google ScholarDigital Library
- [48] . 2020. BBN: Bilateral-branch network with cumulative learning for long-tailed visual recognition. In Conference on Computer Vision and Pattern Recognition. 9719–9728.Google ScholarCross Ref
- [49] . 2013. Jointly learning visually correlated dictionaries for large-scale visual recognition applications. Transactions on Pattern Analysis and Machine Intelligence 36, 4 (2013), 715–730.Google ScholarDigital Library
- [50] . 2022. Deep collaborative multi-task network: A human decision process inspired model for hierarchical image classification. Pattern Recognition 124 (2022), 108449.Google ScholarDigital Library
- [51] . 2022. Label independent memory for semi-supervised few-shot video classification. Transactions on Pattern Analysis and Machine Intelligence 44, 1 (2022), 273–285.Google ScholarDigital Library
Index Terms
- Hierarchical Convolutional Neural Network with Knowledge Complementation for Long-Tailed Classification
Recommendations
Multi-task convolutional neural network with coarse-to-fine knowledge transfer for long-tailed classification
AbstractLong-tailed classifications make it very challenging to deal with class-imbalanced problems using deep convolutional neural networks (CNNs). Existing solutions based on re-balancing methods perform well and use single-task CNNs to train each fine-...
Hierarchical long-tailed classification based on multi-granularity knowledge transfer driven by multi-scale feature fusion
AbstractLong-tailed learning is attracting increasing attention due to the unbalanced distributions of real-world data. The aim is to train well-performing depth models. Traditional knowledge transfer methods for long-tailed learning are classified into ...
Highlights- We propose a multi-scale feature fusion network about channel and spatial features.
- We investigate a multi-granularity relationship of class space.
- We explore a vertical transfer of coarse- to fine-grained knowledge.
Tacit Knowledge Transfer within Enterprises during Industry Conversion
ICIII '08: Proceedings of the 2008 International Conference on Information Management, Innovation Management and Industrial Engineering - Volume 03Enterprises are confronted with the challenge of reengineering technological capability during industry conversion. Tacit knowledge accumulated in the former industry is vital for reengineering new technological capability. The research aims at the ...
Comments