Abstract
For the instance segmentation task, instance representation directly determines the quality of generated masks, so achieving efficient and accurate instance representation is crucial. Grid-based or box-based instance representation contains redundant information from background or other instances, activation-based instance representation includes a small part of the instance. The instance representation based on current methods is not accurate enough. In order to represent more information of an instance under the condition of excluding irrelevant information, this paper proposes multi-area representation (MAR), which is in the form of a scale-adaptive multi-area activation map generated by a multi-branch structure. MAR can adapt to the structure and pose of an instance, thereby representing the shape and size of the instance. Experiments show that, compared with SparseInst, MARInst can improve the performance of instance segmentation and keep the inference speed and training memory almost unchanged. In particular, MARInst achieved 30.3% AP on the MS COCO 2017 val, and 1.6% AP higher than SparseInst when using the same ResNet-50 backbone, proving the effectiveness of the proposed method.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
baseline means that SparseInst does not employ G-IAM and data augmentation.
References
Bolya, D., Zhou, C., Xiao, F., Lee, Y.J.: YOLACT: real-time instance segmentation. In: CVPR (2019)
Cheng, T., et al.: Sparse instance activation for real-time instance segmentation. In: CVPR (2022)
De Brabandere, B., Neven, D., Van Gool, L.: Semantic instance segmentation with a discriminative loss function. arXiv preprint arXiv:1708.02551 (2017)
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: CVPR (2009)
Gao, N., et al.: SSAP: single-shot instance segmentation with affinity pyramid. In: ICCV (2019)
Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: AISTATS (2010)
He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask r-cnn. In: ICCV (2017)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR (2016)
Lee, Y., Park, J.: CenterMask: real-time anchor-free instance segmentation. In: CVPR (2020)
Lin, T.Y., Dollár, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: CVPR (2017)
Lin, T.-Y., et al.: Microsoft COCO: Common Objects in Context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48
Liu, S., Jia, J., Fidler, S., Urtasun, R.: SGN: sequential grouping networks for instance segmentation. In: ICCV (2017)
Liu, S., Qi, L., Qin, H., Shi, J., Jia, J.: Path aggregation network for instance segmentation. In: CVPR (2018)
Liu, Y., et al.: Affinity Derivation and Graph Merge for Instance Segmentation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11207, pp. 708–724. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01219-9_42
Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. In: ICLR (2019)
Peng, S., Jiang, W., Pi, H., Li, X., Bao, H., Zhou, X.: Deep snake for real-time instance segmentation. In: CVPR (2020)
Qi, L., et al.: Pointins: point-based instance segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 44(10), 6377–6392 (2021)
Tian, Z., Shen, C., Chen, H.: Conditional Convolutions for Instance Segmentation. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12346, pp. 282–298. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58452-8_17
Vaswani, A., et al.: Attention is all you need. In: NeurIPS (2017)
Wang, X., Kong, T., Shen, C., Jiang, Y., Li, L.: SOLO: Segmenting Objects by Locations. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12363, pp. 649–665. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58523-5_38
Wang, X., Zhang, R., Kong, T., Li, L., Shen, C.: Solov2: dynamic and fast instance segmentation. In: NIPS (2020)
Wu, Y., Kirillov, A., Massa, F., Lo, W.Y., Girshick, R.: Detectron2. https://github.com/facebookresearch/detectron2 (2019)
Xie, E., et al.: PolarMask: single shot instance segmentation with polar representation. In: CVPR (2020)
Xie, E., Wang, W., Ding, M., Zhang, R., Luo, P.: Polarmask++: enhanced polar representation for single-shot instance segmentation and beyond. IEEE Trans. Pattern Anal. Mach. Intell. 44(9), 5385–5400 (2021)
Yang, H., Zheng, L., Barzegar, S.G., Zhang, Y., Xu, B.: BorderPointsMask: one-stage instance segmentation with boundary points representation. Neurocomputing 467, 348–359 (2022)
Zhang, R., Tian, Z., Shen, C., You, M., Yan, Y.: Mask encoding for single shot instance segmentation. In: CVPR (2020)
Zhang, T., Wei, S., Ji, S.: E2ec: an end-to-end contour-based method for high-quality high-speed instance segmentation. In: CVPR (2022)
Acknowledgment
This work is supported by The National Key R &D Program of China (No. 2021ZD0111902), NSFC (U21B2038, 61876012), Foundation for China university Industry-university Research Innovation (No. 2021JQR023).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, H., Wang, L., Li, S., Xu, K., Yin, B. (2023). Scale-Adaptive Multi-area Representation for Instance Segmentation. In: Lu, H., et al. Image and Graphics . ICIG 2023. Lecture Notes in Computer Science, vol 14358. Springer, Cham. https://doi.org/10.1007/978-3-031-46314-3_5
Download citation
DOI: https://doi.org/10.1007/978-3-031-46314-3_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-46313-6
Online ISBN: 978-3-031-46314-3
eBook Packages: Computer ScienceComputer Science (R0)