Skip to main content
SpringerLink
Log in

Adaptive FSP: Adaptive Architecture Search with Filter Shape Pruning

  • Conference paper
  • First Online:
Computer Vision – ACCV 2022 (ACCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13841))

Included in the following conference series:

Abstract

Deep Convolutional Neural Networks (CNNs) have high memory footprint and computing power requirements, making their deployment in embedded devices difficult. Network pruning has received attention in reducing those requirements of CNNs. Among the pruning methods, Stripe-Wise Pruning (SWP) achieved a further network compression than conventional filter pruning methods and can obtain optimal kernel shapes of filters. However, the model pruned by SWP has filter redundancy because some filters have the same kernel shape. In this paper, we propose the Filter Shape Pruning (FSP) method, which prunes the networks using the kernel shape while maintaining the receptive fields. To obtain an architecture that satisfies the target FLOPs with the FSP method, we propose the Adaptive Architecture Search (AAS) framework. The AAS framework adaptively searches for the architecture that satisfies the target FLOPs with the layer-wise threshold. The layer-wise threshold is calculated at each iteration using the metric that reflects the filter’s influence on accuracy and FLOPs together. Comprehensive experimental results demonstrate that the FSP can achieve a higher compression ratio with an acceptable reduction in accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The baseline obtained by SWP influences the performance of our method. The better the SWP baseline, the better the performance compared with our results. We report the SWP re-implementation results used as the baseline.

References

  1. Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 834–848 (2017)

    Article  Google Scholar 

  2. Chen, W., Wilson, J., Tyree, S., Weinberger, K., Chen, Y.: Compressing neural networks with the hashing trick. In: International Conference on Machine Learning, pp. 2285–2294. PMLR (2015)

    Google Scholar 

  3. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)

    Google Scholar 

  4. Dong, X., Huang, J., Yang, Y., Yan, S.: More is less: a more complicated network with less inference complexity. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5840–5848 (2017)

    Google Scholar 

  5. Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. In: Advances in Neural Information Processing Systems, vol. 28 (2015)

    Google Scholar 

  6. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  7. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  8. He, Y., Kang, G., Dong, X., Fu, Y., Yang, Y.: Soft filter pruning for accelerating deep convolutional neural networks. In: IJCAI International Joint Conference on Artificial Intelligence (2018)

    Google Scholar 

  9. He, Y., Zhang, X., Sun, J.: Channel pruning for accelerating very deep neural networks. In: Proceedings of the IEEE International Conference on Computer vision, pp. 1389–1397 (2017)

    Google Scholar 

  10. Howard, A.G., et al.: MobileNets: efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017)

  11. Huang, Z., Wang, N.: Data-driven sparse structure selection for deep neural networks. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 304–320 (2018)

    Google Scholar 

  12. Huo, Z., Wang, C., Chen, W., Li, Y., Wang, J., Wu, J.: Balanced stripe-wise pruning in the filter. In: ICASSP 2022–2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4408–4412. IEEE (2022)

    Google Scholar 

  13. Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images (2009)

    Google Scholar 

  14. Li, D., Chen, S., Liu, X., Sun, Y., Zhang, L.: Towards optimal filter pruning with balanced performance and pruning speed. In: Proceedings of the Asian Conference on Computer Vision (2020)

    Google Scholar 

  15. Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016)

  16. Li, Y., Gu, S., Mayer, C., Gool, L.V., Timofte, R.: Group sparsity: the hinge between filter pruning and decomposition for network compression. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8018–8027 (2020)

    Google Scholar 

  17. Lin, M., Ji, R., Wang, Y., Zhang, Y., Zhang, B., Tian, Y., Shao, L.: HRank: filter pruning using high-rank feature map. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1529–1538 (2020)

    Google Scholar 

  18. Lin, M., Ji, R., Zhang, Y., Zhang, B., Wu, Y., Tian, Y.: Channel pruning via automatic structure search. In: Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence, pp. 673–679 (2021)

    Google Scholar 

  19. Lin, S., Ji, R., Yan, C., Zhang, B., Cao, L., Ye, Q., Huang, F., Doermann, D.: Towards optimal structured CNN pruning via generative adversarial learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2790–2799 (2019)

    Google Scholar 

  20. Liu, G., Zhang, K., Lv, M.: SOKs: automatic searching of the optimal kernel shapes for stripe-wise network pruning. IEEE Trans. Neural Netw. Learn. Syst. (2022)

    Google Scholar 

  21. Liu, W., et al.: SSD: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2

    Chapter  Google Scholar 

  22. Liu, Y., Guan, B., Xu, Q., Li, W., Quan, S.: SMOF: squeezing more out of filters yields hardware-friendly CNN pruning. arXiv preprint arXiv:2110.10842 (2021)

  23. Liu, Z., Li, J., Shen, Z., Huang, G., Yan, S., Zhang, C.: Learning efficient convolutional networks through network slimming. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2736–2744 (2017)

    Google Scholar 

  24. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)

    Google Scholar 

  25. Loshchilov, I., Hutter, F.: SGDR: stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983 (2016)

  26. Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017)

  27. Meng, F., et al.: Pruning filter in filter. Adv. Neural. Inf. Process. Syst. 33, 17629–17640 (2020)

    Google Scholar 

  28. Ning, X., Zhao, T., Li, W., Lei, P., Wang, Yu., Yang, H.: DSA: more efficient budgeted pruning via differentiable sparsity allocation. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12348, pp. 592–607. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58580-8_35

    Chapter  Google Scholar 

  29. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016)

    Google Scholar 

  30. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems, vol. 28 (2015)

    Google Scholar 

  31. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  32. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  33. Singh, P., Verma, V.K., Rai, P., Namboodiri, V.P.: Play and Prune: adaptive filter pruning for deep model compression. arXiv preprint arXiv:1905.04446 (2019)

  34. Sui, Y., Yin, M., Xie, Y., Phan, H., Aliari Zonouz, S., Yuan, B.: Adv. Neural. Inf. Process. Syst. 34, 24604–24616 (2021)

    Google Scholar 

  35. Sui, Y., Yin, M., Xie, Y., Phan, H., Aliari Zonouz, S., Yuan, B.: CHIP: channel independence-based pruning for compact neural networks. In: Advances in Neural Information Processing Systems, vol. 34 (2021)

    Google Scholar 

  36. Wang, H., Zhang, Q., Wang, Y., Yu, L., Hu, H.: Structured pruning for efficient convnets via incremental regularization. In: 2019 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE (2019)

    Google Scholar 

  37. Wang, W., Fu, C., Guo, J., Cai, D., He, X.: COP: customized deep model compression via regularized correlation-based filter-level pruning. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence, pp. 3785–3791 (2019)

    Google Scholar 

  38. Xu, Y., et al.: TRP: trained rank pruning for efficient deep neural networks. In: Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence, pp. 977–983 (2021)

    Google Scholar 

  39. Zhang, X., Zhou, X., Lin, M., Sun, J.: ShuffleNet: an extremely efficient convolutional neural network for mobile devices. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6848–6856 (2018)

    Google Scholar 

  40. Zhao, K., Jain, A., Zhao, M.: Adaptive activation-based structured pruning. arXiv preprint arXiv:2201.10520 (2022)

Download references

Acknowledgements

This work was supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No.2022-0-01172, DRAM PIM Design Base Technology Development) and by the National Research Foundation of Korea(NRF) grant funded by the Korea government(MSIT) (No.2019R1A5A1027055).

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, Pohang University of Science and Technology, Pohang, South Korea

    Aeri Kim, Seungju Lee, Eunji Kwon & Seokhyeong Kang

Authors
  1. Aeri Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Seungju Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Eunji Kwon
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Seokhyeong Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Seokhyeong Kang .

Editor information

Editors and Affiliations

  1. University of Wollongong, Wollongong, NSW, Australia

    Lei Wang

  2. University of Bonn, Bonn, Germany

    Juergen Gall

  3. University of Adelaide, Adelaide, SA, Australia

    Tat-Jun Chin

  4. National Institute of Informatics, Tokyo, Japan

    Imari Sato

  5. Johns Hopkins University, Baltimore, MD, USA

    Rama Chellappa

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 225 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kim, A., Lee, S., Kwon, E., Kang, S. (2023). Adaptive FSP: Adaptive Architecture Search with Filter Shape Pruning. In: Wang, L., Gall, J., Chin, TJ., Sato, I., Chellappa, R. (eds) Computer Vision – ACCV 2022. ACCV 2022. Lecture Notes in Computer Science, vol 13841. Springer, Cham. https://doi.org/10.1007/978-3-031-26319-4_32

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-26319-4_32

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-26318-7

  • Online ISBN: 978-3-031-26319-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation