Abstract
Convolutional neural network (CNN) is a basic neural network widely used in vision tasks. Many CNNs alleviate the redundancy in feature maps to reduce model complexity. Inspired by digital signal processing theories, this paper reviews discrete fourier transform (DFT), finding its similarities with standard convolution. In particular, DFT has a fast algorithm called FFT, which sparks our thinking: can we learn from the idea of FFT to realize a more efficient convolution filter? Based on the butterfly operation of FFT, we propose a novel butterfly convolution (BFConv). In addition, we illustrate that group weight sharing convolution is a basic unit of BFConv. Compared with the traditional group convolution structure, BFConv constructs group residual-like connections and increases the range of receptive fields for each sub-feature layer. Without changing the network architecture, we integrate BFConv into ResNet-50, ShuffleNet and VGG-16. Experimental results on CIFAR-10 and ImageNet demonstrate the above BFConv-equipped networks reduce parameters and computation, achieving similar or higher accuracy. Remarkably, when ResNet-50 embedded BFConv reaches nearly half of the compression ratio of the model, it performs favorably against its state-of-the-art competitors.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: CVPR (2018)
Xu, Z.-Q.J., Zhang, Y., Xiao, Y.: Training behavior of deep neural network in frequency domain. In: Gedeon, T., Wong, K.W., Lee, M. (eds.) ICONIP 2019. LNCS, vol. 11953, pp. 264â274. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-36708-4_22
Howard, A.G., et al.: Mobilenets: efficient convolutional neural networks for mobile vision applications. arXiv:1704.04861 (2017)
Szegedy, C., et al.: Going deeper with convolutions. In: CVPR (2015)
Gueguen, L., Sergeev, A., Kadlec, B., Liu, R., Yosinski, J.: Faster neural networks straight from JPEG. In: NeurIPS (2018)
Xu, K., Qin, M., Sun, F., Wang, Y., Chen, Y., Ren, F.: Learning in the frequency domain. In: CVPR (2020)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR (2016)
Zhang, Q., Yang, Y.: SA-Net: shuffle attention for deep convolutional neural networks. In: ICASSP (2021)
Wang, Q., Wu, B., Zhu, P., Li, P., Zuo, W., Hu, Q.: ECA-Net: efficient channel attention for deep convolutional neural networks. In: CVPR (2020)
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.-C.: MobileNetV2: inverted residuals and linear bottlenecks. In: CVPR (2018)
Zhang, X., Zhou, X., Lin, M., Sun, J.: ShuffleNet: an extremely efficient convolutional neural network for mobile devices. In: CVPR (2018)
Howard, A., Sandler, M., Chu, G., Chen, L.-C., Chen, B., Tan, M.: Searching for MobileNetV3. In: CVPR (2019)
Chollet, F.: Xception: deep learning with depthwise separable convolutions. In: CVPR (2017)
Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: ICML (2015)
Xie, S., Girshick, R., Dollar, P., Tu, Z., He, K.: Aggregated residual transformations for deep neural networks. In: CVPR (2017)
Singh, P., Verma, V.K., Rai, P., Namboodiri, V.P.: Hetconv: Heterogeneous kernel-based convolutions for deep CNNs. In: CVPR (2019)
Hou, Q., Zhou, D., Feng, J.: Coordinate attention for efficient mobile network design. In: CVPR (2021)
Chen, Y., et al.: Drop an Octave: reducing spatial redundancy in convolutional neural networks with octave convolution. In: ICCV (2019)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: NeurIPS (2012)
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: CVPR (2016)
An, J., Liu, F., Zhao, J., Shen, F.: IC networks: remodeling the basic unit for convolutional neural networks. arXiv:2102.03495 (2021)
Vanhoucke, V.: Learning visual representations at scale. In: ICLR (Invited Talk) (2014)
Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A.: Inception-v4, inception-ResNet and the impact of residual connections on learning. In: AAAI (2017)
Han, K., Wang, Y., Tian, Q., Guo, J., Xu, C., Xu, C.: Ghostnet: more features from cheap operations. In: CVPR (2020)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: ICLR (2015)
Luo, J.-H., Wu, J., Lin, W.: ThiNet: a filter level pruning method for deep neural network compression. In: ICCV (2017)
Zhang, Q., et al.: Split to be slim: an overlooked redundancy in vanilla convolution. In: IJCAI (2020)
Krizhevsky, A., Hinton, G.: Learning multiple layers of features from tiny images. Technical report, Citeseer (2009)
Yu, R., et al.: NISP: pruning networks using neuron importance score propagation. In: CVPR (2018)
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: CVPR (2009)
Huang, Z., Wang, N.: Data-driven sparse structure selection for deep neural networks. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11220, pp. 317â334. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01270-0_19
Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-CAM: visual explanations from deep networks via gradient-based localization. In: ICCV (2017)
Wang, Y., Xu, C., Xu, C., Xu, C., Tao, D.: Learning versatile filters for efficient convolutional neural networks. In: NeurIPS (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
Âİ 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Yang, D., Yu, X., Sun, Y., Zhuang, F., He, Q., Ye, S. (2021). BFConv: Improving Convolutional Neural Networks with Butterfly Convolution. In: Mantoro, T., Lee, M., Ayu, M.A., Wong, K.W., Hidayanto, A.N. (eds) Neural Information Processing. ICONIP 2021. Lecture Notes in Computer Science(), vol 13111. Springer, Cham. https://doi.org/10.1007/978-3-030-92273-3_4
Download citation
DOI: https://doi.org/10.1007/978-3-030-92273-3_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-92272-6
Online ISBN: 978-3-030-92273-3
eBook Packages: Computer ScienceComputer Science (R0)