Abstract
Inception module is proposed in GoogLeNet, which improves performance by increasing the width of the network. Multiple branches are computed in parallel, which makes the inception module naturally take the advantage of GPU high-performance computing. In this paper, we propose a parallel implementation of the inception module to accelerate the training and test of the inception networks. However, convolution neural networks are prone to overfitting due to the huge amount of parameters. We propose random inception module to avoid overfitting and accelerate inception module. In order to demonstrate the effectiveness of the proposed methods, we compare the performance of the random inception module with original inception module on CIFAR-10 dataset. Experimental results show our parallel inception module obtains over 2.8\(\times \) speedup compared with Caffe. And our proposed RIM indeed behaves in a manner of regularization and speeds up convergence.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Cuda toolkit documentation v10.0.130. https://docs.nvidia.com/cuda
Abadi, M., et al.: Tensorflow: a system for large-scale machine learning, pp. 265–283 (2016)
Bergstra, J., et al.: Theano: A CPU and GPU math compiler in Python. In: Proceedings of the 9th Python in Science Conference, pp. 3–10 (2010)
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11211, pp. 833–851. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01234-2_49
Ghiasi, G., Lin, T., Le, Q.V.: Dropblock: a regularization method for convolutional networks. In: Neural Information Processing Systems, pp. 10750–10760 (2018)
Gulshan, V., et al.: Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 316(22), 2402–2410 (2016)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700–4708 (2017)
Jia, Y., et al.: Caffe: convolutional architecture for fast feature embedding. CoRR abs/1408.5093 (2014). http://arxiv.org/abs/1408.5093
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)
Rumelhart, D.E., Hinton, G.E., Williams, R.J., et al.: Learning representations by back-propagating errors. Cogn. Model. 5(3), 1 (1988)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations (2015)
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)
Szegedy, C., et al.: Going deeper with convolutions. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2015
Acknowledgements
This work is partially supported by the National Natural Science Foundation (61872200), the National Key Research and Development Program of China (2016YFC0400709), the Science and Technology Commission of Tianjin Binhai New Area (BHXQKJXM-PT-ZJSHJ-2017005), the Natural Science Foundation of Tianjin (18YFYZCG00060) and Nankai University (91922299).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Gao, Y., Xie, K., Guo, S., Wang, K., Kang, H., Li, T. (2019). Random Inception Module and Its Parallel Implementation. In: Yew, PC., Stenström, P., Wu, J., Gong, X., Li, T. (eds) Advanced Parallel Processing Technologies. APPT 2019. Lecture Notes in Computer Science(), vol 11719. Springer, Cham. https://doi.org/10.1007/978-3-030-29611-7_8
Download citation
DOI: https://doi.org/10.1007/978-3-030-29611-7_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-29610-0
Online ISBN: 978-3-030-29611-7
eBook Packages: Computer ScienceComputer Science (R0)
