Loading [MathJax]/extensions/MathMenu.js
Multi-Level Objective Alignment Transformer for Fine-Grained Oral Panoramic X-Ray Report Generation | IEEE Journals & Magazine | IEEE Xplore

Multi-Level Objective Alignment Transformer for Fine-Grained Oral Panoramic X-Ray Report Generation


Abstract:

Automatically generated oral panoramic X-ray report is highly beneficial for improving the efficiency of dental diagnosis. However, recent solutions adopt holistic method...Show More

Abstract:

Automatically generated oral panoramic X-ray report is highly beneficial for improving the efficiency of dental diagnosis. However, recent solutions adopt holistic methods, resulting in a cursory description of the oral condition. This may lead to reports lacking details, such as specific sites or lesion contours. Therefore, we propose a Multi-Level objective Alignment Transformer(MLAT) network, which integrates all tooth and disease objects into a positional alignment graph to extract fine-grained object-level features. Specifically, we introduce a novel Object-Level Collaborative Encoder (OLCE) module, which uses a positional alignment graph to construct object relationships. OLCE enhances object-level feature extraction by eliminating interference information between pathologically unrelated objects. In addition, we build a high-quality panoramic X-ray image-report dataset consisting of 562 sets of images and reports labeled by 13 experienced dental specialists. Experiments on the collected dataset show that the proposed MLAT significantly outperforms the state-of-the-art baselines by more than 5% in 4 different metrics, including BLEUs, Meteor, Rouge, and BERTScore.
Published in: IEEE Transactions on Multimedia ( Volume: 26)
Page(s): 7462 - 7474
Date of Publication: 22 February 2024

ISSN Information:

Funding Agency:


References

References is not available for this document.