BioSAM: Generating SAM Prompts From Superpixel Graph for Biological Instance Segmentation | IEEE Journals & Magazine | IEEE Xplore

BioSAM: Generating SAM Prompts From Superpixel Graph for Biological Instance Segmentation


Superpixel-guided Prompt Generation is composed of two stages: (a) Superpixel Graph Construction generates sufficient superpixels as graph nodes and extracts node and edg...

Abstract:

Proposal-free instance segmentation methods have significantly advanced the field of biological image analysis. Recently, the Segment Anything Model (SAM) has shown an ex...Show More

Abstract:

Proposal-free instance segmentation methods have significantly advanced the field of biological image analysis. Recently, the Segment Anything Model (SAM) has shown an extraordinary ability to handle challenging instance boundaries. However, directly applying SAM to biological images that contain instances with complex morphologies and dense distributions fails to yield satisfactory results. In this work, we propose BioSAM, a new biological instance segmentation framework generating SAM prompts from a superpixel graph. Specifically, to avoid over-merging, we first generate sufficient superpixels as graph nodes and construct an initialized graph. We then generate initial prompts from each superpixel and aggregate them through a graph neural network (GNN) by predicting the relationship of superpixels to avoid over-segmentation. We employ the SAM encoder embeddings and the SAM-assisted superpixel similarity as new features for the graph to enhance its discrimination capability. With the graph-based prompt aggregation, we utilize the aggregated prompts in SAM to refine the segmentation and generate more accurate instance boundaries. Comprehensive experiments on four representative biological datasets demonstrate that our proposed method outperforms state-of-the-art methods.
Superpixel-guided Prompt Generation is composed of two stages: (a) Superpixel Graph Construction generates sufficient superpixels as graph nodes and extracts node and edg...
Published in: IEEE Journal of Biomedical and Health Informatics ( Volume: 29, Issue: 1, January 2025)
Page(s): 273 - 284
Date of Publication: 04 October 2024

ISSN Information:

PubMed ID: 39365724

Funding Agency:


References

References is not available for this document.