Abstract
Training a deep neural network usually requires a high computational cost. Nowadays, the most common way to carry out this task is through the use of GPUs due to their efficiency implementing complicated algorithms for this kind of tasks. However, training several neural networks, each with different hyperparameters, is still a very heavy task. Typically, clusters include one or more GPUs that could be used for deep learning. This paper proposes and analyzes a distributed parallel procedure to train multiple Convolutional Neural Networks (CNNs) for EEG classification, in a heterogeneous CPU-GPU cluster and in a Desktop PC. The procedure is implemented in C++ and with the MPI library to dynamically distribute the hyperparameters among the nodes, which are responsible for training the corresponding CNN by using Python, Keras, and TensorFlow. The proposed algorithm has been analyzed considering running times and energy measures, showing that when more nodes are used, the procedure scales linearly and the lowest running time is obtained. However, the desktop PC provides the best energy results.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Asensio-Cubero, J., Gan, J., Palaniappan, R.: Multiresolution analysis over simple graphs for brain computer interfaces. J. Neural Eng. 10(4), 21–26 (2013)
Bellman, R.: Adaptive Control Processes: A Guided Tour. Princeton University Press, Princeton (1961)
Birbaumer, N., Cohen, L.: Brain-computer interfaces: communication and restoration of movement in paralysis. J. Physiol. 579(3), 621–636 (2007)
Brumberg, J.S., Burnison, J.D., Pitt, K.M.: Using motor imagery to control brain-computer interfaces for communication. In: Schmorrow, D.D.D., Fidopiastis, C.M.M. (eds.) AC 2016. LNCS (LNAI), vol. 9743, pp. 14–25. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-39955-3_2
Collet, P.: Why GPGPUs for evolutionary computation? In: Tsutsui, S., Collet, P. (eds.) Massively Parallel Evolutionary Computation on GPGPUs. NCS, pp. 3–14. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-37959-8_1
Ortega, J., Asensio-Cubero, J., Gan, J., Ortiz, A.: Classification of motor imagery tasks for BCI with multiresolution analysis and multiobjective feature selection. BioMed. Eng. OnLine 15(1), 73 (2016)
Ortega, J., Ortiz, A., Martín-Smith, P., Gan, J.Q., González-Peñalver, J.: Deep belief networks and multiobjective feature selection for BCI with multiresolution analysis. In: Rojas, I., Joya, G., Catala, A. (eds.) IWANN 2017. LNCS, vol. 10305, pp. 28–39. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-59153-7_3
Owens, J., et al.: A survey of general-purpose computation on graphics hardware. Comput. Graph. Forum 26(1), 80–113 (2007)
Pfurtscheller, G.: EEG event-related desynchronization (ERD) and event-related synchronization (ERS). Electroencephalogr. Clin. Neurophysiol. 103(1), 26 (1997)
Raju, K., Niranjan, N.: A survey on techniques for cooperative cpu-gpu computing. Sustain. Comput.: Inf. Syst. 19, 72–85 (2018)
Tabar, Y., Halici, U.: A novel deep learning approach for classification of EEG motor imagery signals. J. Neural Eng. 14(1), 016003 (2016)
The Open MPI Project: OpenMPI documentation. https://www.open-mpi.org/doc/. Accessed 19 Nov 2018
Wawrzynek, J., Asanovic, K., Kingsbury, B., Johnson, D., Beck, J., Morgan, N.: Spert-II: a vector microprocessor system. Computer 29(3), 79–86 (1996)
Wei, R., Zhang, X., Dang, X., Li, G.: Classification for motion game based on EEG sensing. In: ITM Web Conferences, vol. 11, p. 05002 (2017)
Zou, Y., Jin, X., Li, Y., Guo, Z., Wang, E., Xiao, B.: Mariana: tencent deep learning platform and its applications. VLDB 7(13), 1772–1777 (2014)
Acknowledgements
This research was funded by grants TIN2015-67020-P (Spanish “Ministerio de Economía y Competitividad”), PGC2018-098813-B-C31 (Spanish “Ministerio de Ciencia, Innovación y Universidades”), and ERDF funds. We would also like to thank the BCI Laboratory of the University of Essex for allowing us to use their databases. The Titan Xp used for this research was donated by the NVIDIA Corporation.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Escobar, J.J., Ortega, J., Damas, M., Kızıltepe, R.S., Gan, J.Q. (2019). Energy-Time Analysis of Convolutional Neural Networks Distributed on Heterogeneous Clusters for EEG Classification. In: Rojas, I., Joya, G., Catala, A. (eds) Advances in Computational Intelligence. IWANN 2019. Lecture Notes in Computer Science(), vol 11507. Springer, Cham. https://doi.org/10.1007/978-3-030-20518-8_74
Download citation
DOI: https://doi.org/10.1007/978-3-030-20518-8_74
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-20517-1
Online ISBN: 978-3-030-20518-8
eBook Packages: Computer ScienceComputer Science (R0)