Guangping Li
ABSTRACT
Adder neural network (AdderNet) is a new kind of deep learning model in which the original massive multiplications in convolutions are replaced by additions. The overall energy consumption using adder network is reduced significantly. However, there is a classification accuracy drop when using AdderNet in image classification task. In this paper, we present an adder wavelet transform (AWT) layer instead of the existing down-sampling operations. Based on the AWT layer, we propose a novel adder neural network (AddWaveNet) to improve classification accuracy. Experimental results on CIFAR-10 and CIFAR-100 datasets show that our proposed AddWaveNets achieves significant improvements in classification accuracy and powerful ability of feature learning compared to state-of-the-art quantization networks. Specifically, based on ResNet-32 backbones, AddWaveNet achieves 0.24% accuracy improvement on the CIFAR-10 benchmark and 0.52% accuracy improvement on the CIFAR-100 benchmark.
- Hanting Chen, Yunhe Wang, Chunjing Xu, Boxin Shi, Chao Xu, Qi Tian, and Chang Xu. 2020. AdderNet: Do we really need multiplications in deep learning?. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 1468–1477.Google Scholar
Cross Ref
- Xinghao Chen, Chang Xu, Minjing Dong, Chunjing Xu, and Yunhe Wang. 2021. An empirical study of adder neural networks for object detection. Advances in Neural Information Processing Systems 34 (2021), 6894–6905.Google Scholar
- William Dally. 2015. High-performance hardware for machine learning. Nips Tutorial 2 (2015), 3.Google Scholar
- Minh N Do and Martin Vetterli. 2005. The contourlet transform: an efficient directional multiresolution image representation. IEEE Transactions on image processing 14, 12 (2005), 2091–2106.Google ScholarDigital Library
- Shi Guo, Zifei Yan, Kai Zhang, Wangmeng Zuo, and Lei Zhang. 2019. Toward convolutional blind denoising of real photographs. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 1712–1722.Google ScholarCross Ref
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770–778.Google ScholarCross Ref
- Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. 2017. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 4700–4708.Google ScholarCross Ref
- Itay Hubara, Matthieu Courbariaux, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio. 2016. Binarized neural networks. Advances in neural information processing systems 29 (2016).Google Scholar
- Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems 25 (2012).Google Scholar
- Qiufu Li, Linlin Shen, Sheng Guo, and Zhihui Lai. 2021. WaveCNet: Wavelet Integrated CNNs to Suppress Aliasing Effect for Noise-Robust Image Classification. IEEE Transactions on Image Processing 30 (2021), 7074–7089.Google ScholarDigital Library
- Wenshuo Li, Hanting Chen, Mingqiang Huang, Xinghao Chen, Chunjing Xu, and Yunhe Wang. 2021. Winograd Algorithm for AdderNet. In International Conference on Machine Learning. PMLR, 6307–6315.Google Scholar
- Xiaofan Lin, Cong Zhao, and Wei Pan. 2017. Towards accurate binary convolutional neural network. Advances in neural information processing systems 30 (2017).Google Scholar
- Pengju Liu, Hongzhi Zhang, Wei Lian, and Wangmeng Zuo. 2019. Multi-level wavelet convolutional neural networks. IEEE Access 7 (2019), 74973–74985.Google ScholarCross Ref
- Harry Nyquist. 1928. Certain topics in telegraph transmission theory. Transactions of the American Institute of Electrical Engineers 47, 2 (1928), 617–644.Google ScholarCross Ref
- Mónica Penedo, William A Pearlman, Pablo G Tahoces, Miguel Souto, and Juan J Vidal. 2003. Region-based wavelet coding methods for digital mammography. IEEE transactions on medical imaging 22, 10 (2003), 1288–1296.Google ScholarCross Ref
- Mohammad Rastegari, Vicente Ordonez, Joseph Redmon, and Ali Farhadi. 2016. Xnor-net: Imagenet classification using binary convolutional neural networks. In European conference on computer vision. Springer, 525–542.Google ScholarCross Ref
- Dongwei Ren, Wangmeng Zuo, Qinghua Hu, Pengfei Zhu, and Deyu Meng. 2019. Progressive image deraining networks: A better and simpler baseline. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3937–3946.Google ScholarCross Ref
- Dehua Song, Yunhe Wang, Hanting Chen, Chang Xu, Chunjing Xu, and DaCheng Tao. 2021. Addersr: Towards energy efficient image super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 15648–15657.Google ScholarCross Ref
- Dehua Song, Chang Xu, Xu Jia, Yiyi Chen, Chunjing Xu, and Yunhe Wang. 2020. Efficient residual dense block search for image super-resolution. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 12007–12014.Google ScholarCross Ref
- Mingxing Tan and Quoc Le. 2019. Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference on machine learning. PMLR, 6105–6114.Google Scholar
- Mingxing Tan, Ruoming Pang, and Quoc V Le. 2020. Efficientdet: Scalable and efficient object detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 10781–10790.Google ScholarCross Ref
- Andrew Tao, Karan Sapra, and Bryan Catanzaro. 2020. Hierarchical multi-scale attention for semantic segmentation. arXiv preprint arXiv:2005.10821 (2020).Google Scholar
- Yixing Xu, Chang Xu, Xinghao Chen, Wei Zhang, Chunjing Xu, and Yunhe Wang. 2020. Kernel based progressive distillation for adder neural networks. Advances in Neural Information Processing Systems 33 (2020), 12322–12333.Google Scholar
- Shanshan Zhao, Mingming Gong, Huan Fu, and Dacheng Tao. 2021. Adaptive context-aware multi-modal network for depth completion. IEEE Transactions on Image Processing 30 (2021), 5264–5276.Google ScholarCross Ref
- Xiangyu Zhao, Peng Huang, and Xiangbo Shu. 2022. Wavelet-Attention CNN for image classification. Multimedia Systems (2022), 1–10.Google Scholar
- Shuchang Zhou, Yuxin Wu, Zekun Ni, Xinyu Zhou, He Wen, and Yuheng Zou. 2016. Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients. arXiv preprint arXiv:1606.06160 (2016).Google Scholar
Index Terms
- Adder Wavelet for Better Image Classification under Adder Neural Network
Recommendations
A dyadic multi-resolution deep convolutional neural wavelet network for image classification
For almost the past four decades, image classification has gained a lot of attention in the field of pattern recognition due to its application in various fields. Given its importance, several approaches have been proposed up to now. In this paper, we ...
Wavelet-Attention CNN for image classification
AbstractThe feature learning methods based on convolutional neural network (CNN) have successfully produced tremendous achievements in image classification tasks. However, the inherent noise and some other factors may weaken the effectiveness of the ...
Deep CNN for Classification of Image Contents
IPMV '21: Proceedings of the 2021 3rd International Conference on Image Processing and Machine VisionIn recent years the classification of images has made great progress and has been used in many fields. However, it may not be possible to classify images perfectly through the CNN because of overfitting and gradient vanishing. Most existing CNNs have ...
Comments