Loading [a11y]/accessibility-menu.js
A Visual and Textual Information Fusion-Based Zero-Shot Framework for Hazardous Material Placard Detection and Recognition | IEEE Journals & Magazine | IEEE Xplore

A Visual and Textual Information Fusion-Based Zero-Shot Framework for Hazardous Material Placard Detection and Recognition


Impact Statement:Although existing deep learning-based methods detect and recognize hazardous material placards effectively on the categories with a huge amount of training data, they fal...Show More

Abstract:

Automatically detecting and recognizing hazardous material placards using computer vision-based methods ensures safe operations and proper management of dangerous freight...Show More
Impact Statement:
Although existing deep learning-based methods detect and recognize hazardous material placards effectively on the categories with a huge amount of training data, they fall behind in the classes that are rarely seen. Even worse, they cannot recognize the types of placards that are not included in the training process. Our proposed framework enhances the performance of imbalanced classes and opens up the recognition of new classes that have never appeared in the training dataset. Compared with the previous state-of-the-art method, our framework improved the overall F1 score from 79.97% to 92.77%. As a very practical solution, our framework will promote the automation of hazardous material freight monitoring, consequently, improving the safety of transportation, as well as the efficiency of the global supply chain.

Abstract:

Automatically detecting and recognizing hazardous material placards using computer vision-based methods ensures safe operations and proper management of dangerous freight transportation. Deep learning-based object detection methods provide viable and practical solutions to varied applications. However, contemporary deep learning-based methods suffer from imbalanced and unseen classes, which are very common in real-life data. Thus, this study, drawing attention to this hitherto neglected challenge in real-world applications, proposes a deep learning-based zero-shot framework to detect and recognize the hazardous material placards of both imbalanced and open classes. A logarithmic weighted cross-entropy is proposed to balance the closed classes during training. In addition, a logarithmic weighted confidence fusion strategy is designed to fuse the separately extracted visual and textual information. The experiments on real-world transportation data demonstrated the proposed framework's ef...
Published in: IEEE Transactions on Artificial Intelligence ( Volume: 5, Issue: 4, April 2024)
Page(s): 1755 - 1768
Date of Publication: 24 July 2023
Electronic ISSN: 2691-4581

Funding Agency:


References

References is not available for this document.