Cross-Modal Collaborative Evolution Reinforced by Semantic Coupling for Image Registration and Fusion | IEEE Journals & Magazine | IEEE Xplore

Cross-Modal Collaborative Evolution Reinforced by Semantic Coupling for Image Registration and Fusion


Abstract:

Joint image registration and fusion aim to align and integrate source images to generate an image with salient targets and rich texture details. Current methods pursue sp...Show More

Abstract:

Joint image registration and fusion aim to align and integrate source images to generate an image with salient targets and rich texture details. Current methods pursue spatially optimal deformation fields. However, these methods often overlook local semantic alignment, leading to exacerbated heterogeneity in cascaded fusion and vision tasks. To address these issues, we propose a collaborative evolution network reinforced by semantic coupling for image registration and fusion, named CE-SCNet. First, to correct spatial misalignments, we design a multiscale deformation estimator (MSDE). This module is to estimate spatial deformation fields by modeling global relationships across multiple scales. Second, to further enhance semantic alignment and mitigate heterogeneity, we design the semantic interaction module (SIM). This module is to integrate contextual information within the semantic domain for feature coupling. Third, to reconstruct images with high visual perception, we design the feature discrimination module (FDM) and the detail awareness module (DAM). Both modules are to capture texture information from multiple perspectives. Finally, to optimize the joint paradigm, we construct a multilabel semantic loss. Extensive experimental validations have shown that CE-SCNet significantly outperforms state-of-the-art methods in alleviating semantic misalignments. The semantic segmentation experiments demonstrate that CE-SCNet can adapt to the semantic demands of high-level vision tasks.
Article Sequence Number: 5000112
Date of Publication: 13 November 2024

ISSN Information:

Funding Agency:


Contact IEEE to Subscribe

References

References is not available for this document.