Abstract:
Photonic Deep Learning (DL) accelerators are among the most promising approaches for providing fast and energy efficient neural network implementations for several applic...Show MoreMetadata
Abstract:
Photonic Deep Learning (DL) accelerators are among the most promising approaches for providing fast and energy efficient neural network implementations for several applications. However, photonic accelerators require using different activation functions compared to those typically used in DL. This renders the training process especially difficult to tune, often requiring several trials just for selecting the appropriate initialization hyper-parameters for the network. This process becomes even more difficult for recurrent networks, where exploding gradient phenomena can further destabilize the training process. In this paper, we propose an adaptive data-driven initialization approach for recurrent photonic neural networks. The proposed method is activation-agnostic, while it takes into account the actual distribution of the data used to train the network, overcoming a number of significant limitations of existing approaches. The proposed method is simple and easy to implement, yet it leads to significant improvements in the performance of DL models, as it was experimentally demonstrated using two large-scale challenging time-series datasets.
Date of Conference: 12-14 October 2020
Date Added to IEEE Xplore: 28 September 2020
Print ISBN:978-1-7281-3320-1
Print ISSN: 2158-1525