Abstract
Artificial Neural Networks (ANNs) are thinly based on biological neural pathways. In an ANN, each node computes its activation by applying a non-linearity to a weighted sum of its inputs. While this formulation has been wildly successful for a variety of tasks, it is still a far cry from its biological counterpart, largely due to ANNs lack of phase information during computation. In this paper, we adapt ANNs to operate on complex values which naturally allows the inclusion of phase information during the forward pass. We demonstrate that our complex-valued architecture generally performs better compared to real-valued and other complex-valued networks in similar conditions. Additionally, we couple our model with a biologically inspired form of dimensionality reduction and present our findings on the MNIST and MusicNet data sets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Allard, W.K., Chen, G., Maggioni, M.: Multi-scale geometric methods for data sets II: geometric multi-resolution analysis. Appl. Comput. Harmon. Anal. 32(3), 435–462 (2012)
Cole, E., Cheng, J., Pauly, J., Vasanawala, S.: Analysis of deep complex-valued convolutional neural networks for mri reconstruction and phase-focused applications. Magn. Reson. Med. 86, 1093–1099 (2021). https://onlinelibrary.wiley.com/doi/10.1002/mrm.28733
Dramsch, J., Lüthje, M., Christensen, A.N.: Complex-valued neural networks for machine learning on non-stationary physical data. Comput. Geosci. 146, 104643 (2021). ISSN 0098–3004. https://doi.org/10.1016/j.cageo.2020.104643
Gao, J., Deng, B., Qin, Y., Wang, H., Li, X.: Enhanced radar imaging using a complex-valued convolutional neural network. IEEE Geosci. Remote Sens. Lett. 16(1), 35–39 (2019). https://doi.org/10.1109/LGRS.2018.2866567
Hornickel, J., Kraus, N.: Unstable representation of sound: a biological marker of dyslexia. J. Neurosci. 33(8), 3500–3504 (2013). ISSN 0270–6474. https://doi.org/10.1523/JNEUROSCI.4205-12.2013, https://www.jneurosci.org/content/33/8/3500
LeCun, Y.: The MNIST database of handwritten digits (1998). http://yann.lecun.com/exdb/mnist/
Matsumura, M., Chen, D., Sawaguchi, T., Kubota, K., Fetz, E.E.: Synaptic interactions between primate precentral cortex neurons revealed by spike-triggered averaging of intracellular membrane potentials in vivo. J. Neurosci. 16(23), 7757–7767 (1996). ISSN 0270–6474. https://doi.org/10.1523/JNEUROSCI.16-23-07757.1996, https://www.jneurosci.org/content/16/23/7757
Mönning, N., Manandhar, S.: Evaluation of complex-valued neural networks on real-valued classification tasks. CoRR, abs/1811.12351 (2018). http://arxiv.org/abs/1811.12351
Mönning, N., Manandhar, S.: Evaluation of complex-valued neural networks on real-valued classification tasks (2018)
Popa, C.-A.: Complex-valued convolutional neural networks for real-valued image classification, pp. 816–822 (2017). https://doi.org/10.1109/IJCNN.2017.7965936
Prechelt, L.: Early stopping-but when? pp. 55–69 (1998)
Purwins, H., Li, B., Virtanen, T., Schlüter, J., Chang, S.-Y., Sainath, T.N.: Deep learning for audio signal processing. CoRR, abs/1905.00078 (2019). http://arxiv.org/abs/1905.00078
Reichert, D.P., Serre, T.: Neuronal synchrony in complex-valued deep networks (2014)
Rosenblatt, F.: Perceptron simulation experiments. Proc. IRE 48(3), 301–309 (1960). https://doi.org/10.1109/JRPROC.1960.287598
Singer, W.: Neural synchrony: a versatile code for the definition of relations? Neuron 24, 49–65 (1999)
Stanley, G.B.: Reading and writing the neural code. Nat. Neurosci. 16, 259-263 (2013). https://www.nature.com/articles/nn.3330
Stevens, C., Zador, A.: Novel integrate-and-fire-like model of repetitive firing in cortical neurons (1998)
Tan, M., Le, Q.V.: Efficientnet: rethinking model scaling for convolutional neural networks. CoRR, abs/1905.11946 (2019). http://arxiv.org/abs/1905.11946
Thickstun, J., Harchaoui, Z., Kakade, S.: Learning features of music from scratch (2017)
Trabelsi, C., et al.: Deep complex networks, Negar Rostamzadeh (2018)
Uhlhaas, P.J., et al.: Neural synchrony in the cortical networks: history, concept and current status. Front. Integr. Neurosci. 3, 17 (2009)
Waddington, D., Dickey, C., Hershcovitch, M., Seshadri, S.: An architecture for memory centric active storage (MCAS). arXiv preprint arXiv:2103.00007 (2021)
Waddington, D.G., Hershcovitch, M., Dickey, C.: PYMM: heterogeneous memory programming for python data science. In: PLOS 2021: Proceedings of the 11th Workshop on Programming Languages and Operating Systems, Virtual Event, Germany, 25 October 2021, pp. 31–37. ACM (2021). https://doi.org/10.1145/3477113.3487266
Wood, A., et al.: Non-volatile memory accelerated geometric multi-scale resolution analysis. In: 2021 IEEE High Performance Extreme Computing Conference (HPEC), pp. 1–7. IEEE (2021)
Wyse, L.: Audio spectrogram representations for processing with convolutional neural networks. In: Workshop on Deep Learning for Music (2017)
Xu, Y., Kong, Q., Huang, Q., Wang, W., Plumbley, M.D.: Attention and localization based on a deep convolutional recurrent model for weakly supervised audio tagging. CoRR, abs/1703.06052 (2017). http://arxiv.org/abs/1703.06052
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 IFIP International Federation for Information Processing
About this paper
Cite this paper
Yu, R., Wood, A., Cohen, S., Hershcovitch, M., Waddington, D., Chin, P. (2022). Biologically Plausible Complex-Valued Neural Networks and Model Optimization. In: Maglogiannis, I., Iliadis, L., Macintyre, J., Cortez, P. (eds) Artificial Intelligence Applications and Innovations. AIAI 2022. IFIP Advances in Information and Communication Technology, vol 646. Springer, Cham. https://doi.org/10.1007/978-3-031-08333-4_30
Download citation
DOI: https://doi.org/10.1007/978-3-031-08333-4_30
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-08332-7
Online ISBN: 978-3-031-08333-4
eBook Packages: Computer ScienceComputer Science (R0)
