Efficient Multi-model Vision Transformer Based on Feature Fusion for Classification of DFUC2021 Challenge

Abstract

Diabetic Foot Ulcers (DFU) is a serious problem and one out of three people with diabetes could be affected by this disease. The rapid rise in the occurrences of DFUs over the last few decades is a major challenge for healthcare systems. DFU with ischemia and infection could be a more serious problem and can cause death. Early detection of DFU and regular monitoring by patients can be useful to overcome the disease. The improvement of patient care and minimization of drawbacks in healthcare systems are also very important. We proposed different pre-trained transformers and fine-tuned them on the DFUC-21 dataset for multi-class classification problems. Before using the Transformer, different pre-trained CNN-based models were fine-tuned on this challenging dataset. When the Transformer was applied to the DFUC-21 dataset, we got favorable results as compared to pre-trained-CNN based models. After several experiments, we have chosen the five best transformers based on pre-trained models and only used two pre-trained transformers in parallel to extract and fuse features from the last layers of the Multi-Model. The proposed model produced Macro-Average F1 0.557 on the validation dataset and 0.569 on a testing dataset and achieved 4^th position on the leader board. The proposed model could be useful for the automatic classification of diabetic foot ulcers. The code is publicly available at: https://github.com/RespectKnowledge/DFUC2021.