Context-FPN and Memory Contrastive Learning for Partially Supervised Instance Segmentation

Abstract

Partially supervised instance segmentation aims to segment objects on both limited seen categories and novel unseen categories (without annotated masks), thereby eliminating expensive demands of mask annotation for new categories. Existing work mainly utilize the pipeline model of detection first and then segmentation, and explores how to provide more discriminative regions of interest for the class-agnostic mask head, but these methods do not perform well when faced with complex scenes. In this work, we propose a novel method, named CCMask, that combines Context Feature Pyramid Network (Context-FPN) and Memory Contrastive Learning Head (MCL Head) to achieve effective class-agnostic mask segmentation. Specifically, we introduce a Context-FPN to obtain context-rich feature map via context extraction module, which will benefit the subsequent task heads. In the MCL Head, we employ foreground/background query memory queue to store queries from recent training batches, this helps the MCL Head learns the general concepts of foreground and background. These strategies collectively contribute to improve the discrimination between foreground and background. Exhaustive experiments on COCO dataset demonstrate that our method achieves state-of-the-art results.