Skip to main content
SpringerLink
Log in

Lightweight Image Matting via Efficient Non-local Guidance

  • Conference paper
  • First Online:
Computer Vision – ACCV 2022 (ACCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13842))

Included in the following conference series:

  • 269 Accesses

Abstract

Natural image matting aims to estimate the opacity of foreground objects. Most existing approaches involve prohibitive parameters, daunting computational complexity, and redundant dependency. In this paper, we propose a lightweight matting method termed LiteMatting, which learns the local smoothness of color space and affinities between neighboring pixels to estimate the alpha mattes. Specifically, a modified mobile block is adopted to construct an encoder-decoder framework, which reduces parameters while retaining sufficient spatial and channel information. In addition, a Long-Short Range Pyramid Pooling Module (LSRPPM) is introduced to extend the reception field by capturing long-range dependency between regions distributed discretely. Finally, an Efficient Non-Local Block (ENB) is presented for guiding high-level semantics propagation from low-level detail features to refine the alpha mattes. Extensive experiments demonstrate that our method achieves a favorable trade-off between accuracy and efficiency. Compared with most state-of-the-art approaches, our method attains an immense descent in parameters and FLOPs with 30\(\%\) and 13\(\%\), respectively, while achieving an improvement of over 15\(\%\) in SAD metrics. Code and model are available at https://github.com/kzx2018/LiteMatting.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Sengupta, S., Jayaram, V., Curless, B., Seitz, S.M., Kemelmacher-Shlizerman, I.: Background matting: the world is your green screen. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2291–2300 (2020)

    Google Scholar 

  2. Lin, S., Ryabtsev, A., Sengupta, S., Curless, B.L., Seitz, S.M., Kemelmacher-Shlizerman, I.: Real-time high-resolution background matting. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8762–8771 (2021)

    Google Scholar 

  3. Lin, S., Yang, L., Saleemi, I., Sengupta, S.: Robust high-resolution video matting with temporal guidance. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 238–247 (2022)

    Google Scholar 

  4. Chen, Y., Guan, J., Cham, W.K.: Robust multi-focus image fusion using edge model and multi-matting. IEEE Trans. Image Process. 27, 1526–1541 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  5. Ke, Z., et al.: Is a green screen really necessary for real-time portrait matting? arXiv preprint arXiv:2011.11961 (2020)

  6. Xu, N., Price, B., Cohen, S., Huang, T.: Deep image matting. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2970–2979 (2017)

    Google Scholar 

  7. Hou, Q., Liu, F.: Context-aware image matting for simultaneous foreground and alpha estimation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4130–4139 (2019)

    Google Scholar 

  8. Li, Y., Lu, H.: Natural image matting via guided contextual attention. Proc. AAAI Conf. Artif. Intell. 34, 11450–11457 (2020)

    Google Scholar 

  9. Lu, H., Dai, Y., Shen, C., Xu, S.: Indices matter: learning to index for deep image matting. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3266–3275 (2019)

    Google Scholar 

  10. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: MobileNetV2: inverted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4510–4520 (2018)

    Google Scholar 

  11. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  12. Wang, R., Xie, J., Han, J., Qi, D.: Improving deep image matting via local smoothness assumption. arXiv preprint arXiv:2112.13809 (2021)

  13. Liu, Y., Yu, J., Han, Y.: Understanding the effective receptive field in semantic image segmentation. Multimedia Tools Appl. 77(17), 22159–22171 (2018). https://doi.org/10.1007/s11042-018-5704-3

    Article  Google Scholar 

  14. Wang, X., Girshick, R., Gupta, A., He, K.: Non-local neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7794–7803 (2018)

    Google Scholar 

  15. Ruzon, M.A., Tomasi, C.: Alpha estimation in natural images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2000 (Cat. No. PR00662). Volume 1, pp. 18–25. IEEE (2000)

    Google Scholar 

  16. Gastal, E.S., Oliveira, M.M.: Shared sampling for real-time alpha matting. In: Computer Graphics Forum, vol. 29, pp. 575–584. Wiley Online Library (2010)

    Google Scholar 

  17. Guan, Y., Chen, W., Liang, X., Ding, Z., Peng, Q.: Easy matting-a stroke based approach for continuous image matting. In: Computer Graphics Forum, vol. 25, pp. 567–576. Wiley Online Library (2006)

    Google Scholar 

  18. Feng, X., Liang, X., Zhang, Z.: A cluster sampling method for image matting via sparse coding. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 204–219. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_13

    Chapter  Google Scholar 

  19. Chuang, Y.Y., Curless, B., Salesin, D.H., Szeliski, R.: A Bayesian approach to digital matting. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2001, vol. 2, pp. 264–271. IEEE (2001)

    Google Scholar 

  20. Wang, J., Cohen, M.F.: Optimized color sampling for robust matting. In: 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE (2007)

    Google Scholar 

  21. He, K., Rhemann, C., Rother, C., Tang, X., Sun, J.: A global sampling method for alpha matting. In: CVPR 2011, pp. 2049–2056. IEEE (2011)

    Google Scholar 

  22. Sun, J., Jia, J., Tang, C.K., Shum, H.Y.: Poisson matting. In: ACM SIGGRAPH 2004 Papers, pp. 315–321 (2004)

    Google Scholar 

  23. Bai, X., Sapiro, G.: A geodesic framework for fast interactive image and video segmentation and matting. In: 2007 IEEE 11th International Conference on Computer Vision, pp. 1–8. IEEE (2007)

    Google Scholar 

  24. Levin, A., Rav-Acha, A., Lischinski, D.: Spectral matting. IEEE Trans. Pattern Anal. Mach. Intell. 30, 1699–1712 (2008)

    Article  Google Scholar 

  25. Lee, P., Wu, Y.: Nonlocal matting. In: CVPR 2011, pp. 2193–2200. IEEE (2011)

    Google Scholar 

  26. Levin, A., Lischinski, D., Weiss, Y.: A closed-form solution to natural image matting. IEEE Trans. Pattern Anal. Mach. Intell. 30, 228–242 (2007)

    Article  Google Scholar 

  27. Chen, Q., Li, D., Tang, C.K.: KNN matting. IEEE Trans. Pattern Anal. Mach. Intell. 35, 2175–2188 (2013)

    Article  Google Scholar 

  28. Aksoy, Y., Ozan Aydin, T., Pollefeys, M.: Designing effective inter-pixel information flow for natural image matting. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 29–37 (2017)

    Google Scholar 

  29. Lutz, S., Amplianitis, K., Smolic, A.: AlphaGAN: generative adversarial networks for natural image matting. arXiv preprint arXiv:1807.10088 (2018)

  30. Cai, S., et al.: Disentangled image matting. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8819–8828 (2019)

    Google Scholar 

  31. Yu, H., Xu, N., Huang, Z., Zhou, Y., Shi, H.: High-resolution deep image matting. Proc. AAAI Conf. Artif. Intell. 35, 3217–3224 (2021)

    Google Scholar 

  32. Sun, Y., Tang, C.K., Tai, Y.W.: Semantic image matting. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11120–11129 (2021)

    Google Scholar 

  33. Liu, Y., et al.: Tripartite information mining and integration for image matting. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7555–7564 (2021)

    Google Scholar 

  34. Jiang, W., Yu, D., Xie, Z., Li, Y., Yuan, Z., Lu, H.: Trimap-guided feature mining and fusion network for natural image matting. arXiv preprint arXiv:2112.00510 (2021)

  35. Cheng, H., Xu, S., Jiang, X., Wang, R.: Deep image matting with flexible guidance input. arXiv preprint arXiv:2110.10898 (2021)

  36. Goel, A., Kumar, M., Sudheendra, P., Team, V., et al.: IamAlpha: instant and adaptive mobile network for alpha matting (2021)

    Google Scholar 

  37. Liu, Y., Xie, J., Qiao, Y., Tang, Y., Yang, X.: Prior-induced information alignment for image matting. IEEE Trans. Multimedia 24, 2727–2738 (2021)

    Article  Google Scholar 

  38. Dai, Y., Price, B., Zhang, H., Shen, C.: Boosting robustness of image matting with context assembling and strong data augmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11707–11716 (2022)

    Google Scholar 

  39. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    Chapter  Google Scholar 

  40. Le, H., Mai, L., Price, B., Cohen, S., Jin, H., Liu, F.: Interactive boundary prediction for object selection. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) Computer Vision – ECCV 2018. LNCS, vol. 11218, pp. 20–36. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01264-9_2

    Chapter  Google Scholar 

  41. Wu, Y., He, K.: Group normalization. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11217, pp. 3–19. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01261-8_1

    Chapter  Google Scholar 

  42. Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with Atrous separable convolution for semantic image segmentation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11211, pp. 833–851. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01234-2_49

    Chapter  Google Scholar 

  43. Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J.: Pyramid scene parsing network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2881–2890 (2017)

    Google Scholar 

  44. Hou, Q., Zhang, L., Cheng, M.M., Feng, J.: Strip pooling: rethinking spatial pooling for scene parsing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4003–4012 (2020)

    Google Scholar 

  45. Zhu, Z., Xu, M., Bai, S., Huang, T., Bai, X.: Asymmetric non-local neural networks for semantic segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 593–602 (2019)

    Google Scholar 

  46. Zhong, Y., Li, B., Tang, L., Tang, H., Ding, S.: Highly efficient natural image matting. arXiv preprint arXiv:2110.12748 (2021)

  47. He, K., Sun, J., Tang, X.: Fast matting using large kernel matting Laplacian matrices. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 2165–2172. IEEE (2010)

    Google Scholar 

  48. Liu, Q., Xie, H., Zhang, S., Zhong, B., Ji, R.: Long-range feature propagating for natural image matting. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 526–534 (2021)

    Google Scholar 

  49. Dai, Y., Lu, H., Shen, C.: Learning affinity-aware upsampling for deep image matting. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6841–6850 (2021)

    Google Scholar 

  50. Tang, J., Aksoy, Y., Oztireli, C., Gross, M., Aydin, T.O.: Learning-based sampling for natural image matting. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3055–3063 (2019)

    Google Scholar 

  51. Ma, N., Zhang, X., Zheng, H.-T., Sun, J.: ShuffleNet V2: practical guidelines for efficient CNN architecture design. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) Computer Vision – ECCV 2018. LNCS, vol. 11218, pp. 122–138. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01264-9_8

    Chapter  Google Scholar 

  52. Tan, M., Le, Q.: EfficientNet: rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114. PMLR (2019)

    Google Scholar 

  53. Liu, Z., Mao, H., Wu, C.Y., Feichtenhofer, C., Darrell, T., Xie, S.: A ConvNet for the 2020s. arXiv preprint arXiv:2201.03545 (2022)

  54. Forte, M., Pitié, F.: \( f \), \( b \), alpha matting. arXiv preprint arXiv:2003.07711 (2020)

Download references

Acknowledgement

This work was supported by the National Natural Science Foundation of China (Nos. 61872112 and 6207071409) and the Taishan Scholars Program of Shandong Province (No. tsqn201812106).

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Harbin Institute of Technology, Weihai, 264209, China

    Zhaoxiang Kang, Zonglin Li, Qinglin Liu, Yuhe Zhu, Hongfei Zhou & Shengping Zhang

Authors
  1. Zhaoxiang Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zonglin Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qinglin Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuhe Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hongfei Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Shengping Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zonglin Li .

Editor information

Editors and Affiliations

  1. University of Wollongong, Wollongong, NSW, Australia

    Lei Wang

  2. University of Bonn, Bonn, Germany

    Juergen Gall

  3. University of Adelaide, Adelaide, SA, Australia

    Tat-Jun Chin

  4. National Institute of Informatics, Tokyo, Japan

    Imari Sato

  5. Johns Hopkins University, Baltimore, MD, USA

    Rama Chellappa

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kang, Z., Li, Z., Liu, Q., Zhu, Y., Zhou, H., Zhang, S. (2023). Lightweight Image Matting via Efficient Non-local Guidance. In: Wang, L., Gall, J., Chin, TJ., Sato, I., Chellappa, R. (eds) Computer Vision – ACCV 2022. ACCV 2022. Lecture Notes in Computer Science, vol 13842. Springer, Cham. https://doi.org/10.1007/978-3-031-26284-5_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-26284-5_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-26283-8

  • Online ISBN: 978-3-031-26284-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation