Abstract
In this paper, parameter optimization of Convolutional Neural Network architectures is proposed by applying the Particle Swarm Optimization algorithm, where the parameters that are optimized are: the number of layers, number of convolution filters, the filter size, and the batch size. The optimized architecture is applied in two sign language databases, the American Sign Language Alphabet and American Sign Language MNIST. This research aims to analyze the performance of the proposed architecture, focusing on obtaining a better recognition rate of the signs to achieve an increase in the recognition and computational performance of the Convolutional Neural Network architecture and subsequently propose tools for assisted communication for the deaf community. The achieved results indicate that the proposed method is effective since a recognition rate above 99% was obtained in both study cases, achieving a reduction in processing times and computational cost.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Adeleh, E., D. Vajiheh, B. Ali, & S. Vahid. 2017. Improved particle swarm optimization through orthogonal experimental design. In 2nd conference on swarm intelligence and evolutionary computation (CSIEC2017), 153–158.
Aggarwal, C.C. 2018. Neural networks and deep learning. New York: Springer.
Bin, L.Y., G.Y. Huann, and L.K. Yun. 2019. Study of convolutional neural network in recognizing static American sign language. In 2019 IEEE International Conference on Signal and Image Processing Applications (ICSIPA), 41–45.
Castillo, O., and P. Melin. 1998. A new fuzzy-fractal-genetic method for automated mathematical modelling and simulation of robotic dynamic systems. In 1998 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE 1998) Proceedings, vol. 2, 1182–1187.
Castillo, O., and P. Melin. 2003. Intelligent adaptive model-based control of robotic dynamic systems with a hybrid fuzzy-neural approach. Applied Soft Computing 3 (4): 363–378.
Cheng, J., P.-S. Wang, G. Li, Q.-H. Hu, and H.-Q. Lu. 2018. Recent advances in efficient computation of deep convolutional neural networks. Frontiers of Information Technology & Electronic Engineering 19: 64–77.
Engelbrecht, P. 2007a. Computational intelligence: An introduction, 2nd ed. Wiley: University of Pretoria, South Africa.
Engelbrecht, P. 2007b. Computacional intelligence. South Africa: WILEY.
Fernandes, F.E., and G.G. Yen. 2019. Particle swarm optimization of deep neural networks architectures for image classification. Swarm and Evolutionary Computation 49: 62–74.
Fielding, B., and L. Zhang. 2018. Evolving image classification architectures with enhanced particle swarm optimisation. IEEE Access 6: 68560–68575.
Fukushima, K. 1980. A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biological Cybernetics 36: 193–202.
Giaquinto, and G. Fornarelli. 2009. PSO-based cloning template design for CNN associative memories. IEEE Transactions on Neural Networks 20 (11): 1837–1841.
Gaxiola, F., P. Melin, F. Valdez, J.R. Castro, and O. Castillo. 2016. Optimization of type-2 fuzzy weights in backpropagation learning for neural networks using GAs and PSO. Applied Soft Computing 38: 860–871.
Gonzalez, B., P. Melin, and F. Valdez. 2019. Particle swarm algorithm for the optimization of modular neural networks in pattern recognition. Hybrid Intelligent Systems in Control, Pattern Recognition and Medicine 827: 59–69.
Goodfellow, I., Y. Bengio, and A. Courville. 2016. Deep learning. Cambridge: MIT Press.
Hemanth, J.D., O. Deperlioglu, and U. Kose. 2020. An enhanced diabetic retinopathy detection and classification approach using deep convolutional neural network. Neural Computing and Applications 32: 707–721.
Huang, J., W. Zhou, H. Li, and W. Li. 2019. Attention-based 3D-CNNs for large-vocabulary sign language recognition. IEEE Transactions on Circuits and Systems for Video Technology 29 (9): 2822–2832.
Hubel, D.H., and T.N. Wiesel. 1959. Receptive fields of single neurons in the cat’s striate cortex. The Journal of Physiology 148: 574–591.
Kaggle. 2017. Sign language MNIST. https://www.kaggle.com/datamunge/sign-language-mnist. Accessed 8 Feb 2020.
Kaggle. 2018. American Sign Language dataset. https://www.kaggle.com/grassknoted/asl-alphabet. Accessed 10 Feb 2020.
Kim, P. 2017. MATLAB deep learning. Seoul: Apress.
Miramontes, I., P. Melin, and G. Prado-Arechiga. 2020. Particle swarm optimization of modular neural networks for obtaining the trend of blood pressure. In Intuitionistic and type-2 fuzzy logic enhancements in neural and optimization algorithms: Theory and applications, vol. 862, 225–236.
Peter, S.E., and I.J. Reglend. 2017. Sequential wavelet-ANN with embedded ANN-PSO hybrid electricity price forecasting model for Indian energy exchange. Neural Computing and Applications 28: 2277–2292.
Poma, Y., P. Melin, C. I. González, and G.E. Martinez. 2020a. Optimal recognition model based on convolutional neural networks and fuzzy gravitational search algorithm method. In Hybrid intelligent systems in control, pattern recognition and medicine, vol. 827, 71–81.
Poma, Y., P. Melin, C.I. González, and G.E. Martinez. 2020b. Filter size optimization on a convolutional neural network using FGSA. In Intuitionistic and type-2 fuzzy logic enhancements in neural and optimization algorithms, 391–403.
Sánchez, D., and P. Melin. 2014. Optimization of modular granular neural networks using hierarchical genetic algorithms for human recognition using the ear biometric measure. Engineering Applications of Artificial Intelligence 27: 41–56.
Sanchez, M.A., O. Castillo, J.R. Castro, and P. Melin. 2014. Fuzzy granular gravitational clustering algorithm for multivariate data. Information Sciences 279: 498–511.
Sánchez, D., P. Melin, J. Carpio, and H. Puga. 2017a. Comparison of optimization techniques for modular neural networks applied to human recognition. In Nature-Inspired Design of Hybrid Intelligent Systems, 225–241. Cham: Springer.
Sánchez, D., P. Melin, and O. Castillo. 2017b. Optimization of modular granular neural networks using a firefly algorithm for human recognition. Engineering Applications of Artificial Intelligence 64: 172–186.
Sánchez, D., P. Melin, and O. Castillo. 2020. Comparison of particle swarm optimization variants with fuzzy dynamic parameter adaptation for modular granular neural networks for human recognition. Journal of Intelligent & Fuzzy Systems 38 (3): 3229–3252.
Schmidhuber, J. 2015. Deep learning in neural networks: An overview. Neural Networks 61: 85–117.
Varela-Santos, S., and P. Melin. 2020. Classification of x-ray images for pneumonia detection using texture features and neural networks. In Intuitionistic and type-2 fuzzy logic enhancements in neural and optimization algorithms: Theory and applications, vol. 862, 237–253.
Xianwei, J., and L. Mingzhou. 2019. An eight-layer convolutional neural network with stochastic pooling, batch normalization and dropout for fingerspelling recognition of Chinese sign language, 19. Springer Link.
Xianwei, J., M. Lu, and S.-H. Wang. 2019. An eight-layer convolutional neural network with stochastic pooling, batch normalization and dropout for fingerspelling recognition of Chinese sign language, 1–19. Springer Multimedia Tools and Applications.
Xiaojing, Y., J. Qingju, and L. Xinke. 2019. Center particle swarm optimization algorithm. In 2019 IEEE 3rd Information Technology, Networking, Electronic and Automation Control Conference (ITNEC), 2084–2087.
Yannakakis, G.N., and J. Togelius. 2015. A panorama of artificial and computational intelligence in games. IEEE Transactions on Computational Intelligence and AI in Games 7 (4): 317–335.
Zou, Z., B. Shuai, and G. Wang. 2016. Learning contextual dependence with convolutional hierarchical recurrent neural networks. IEEE Transactions on Image Processing 25 (7): 2983–2996.
Acknowledgments
We thank the Tijuana Institute of Technology, and the financial support provided by our sponsor CONACYT with the scholarship number: 954950.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Fregoso, J., Gonzalez, C.I., Martinez, G.E. (2021). Parameter Optimization of a Convolutional Neural Network Using Particle Swarm Optimization. In: Castillo, O., Melin, P. (eds) Fuzzy Logic Hybrid Extensions of Neural and Optimization Algorithms: Theory and Applications. Studies in Computational Intelligence, vol 940. Springer, Cham. https://doi.org/10.1007/978-3-030-68776-2_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-68776-2_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-68775-5
Online ISBN: 978-3-030-68776-2
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)