Skip to main content
SpringerLink
Log in
Book cover

International Symposium on Neural Networks

ISNN 2018: Advances in Neural Networks – ISNN 2018 pp 64–71Cite as

Complex-Valued Stacked Denoising Autoencoders

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10878))

Abstract

Stacking layers of denoising autoencoders, which are trained to reconstruct corrupted versions of their inputs, results in a type of deep neural network architecture called stacked denoising autoencoders. This paper introduces a model of complex-valued stacked denoising autoencoders, which can be used to build complex-valued deep neural networks. Experiments done using the MNIST and FashionMNIST datasets show superior performance of the complex-valued stacked denoising autoencoders with respect to their real-valued counterparts, both in terms of reconstruction error, and in terms of classification error.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Arjovsky, M., Shah, A., Bengio, Y.: Unitary evolution recurrent neural networks. In: ICML 2016 (2016). https://arxiv.org/abs/1511.06464

  2. Baldi, P., Lu, Z.: Complex-valued autoencoders. Neural Netw. 33, 136–147 (2012)

    Article  Google Scholar 

  3. Bengio, Y.: Learning deep architectures for AI. Found. Trends Mach. Learn. 2(1), 1–127 (2009)

    Article  Google Scholar 

  4. Bengio, Y., LeCun, Y., Hinton, G.: Deep learning. Nature 521, 436–444 (2015)

    Article  Google Scholar 

  5. Bengio, Y., Lamblin, P., Popovici, D., Larochelle, H.: Greedy layer-wise training of deep networks. In: NIPS 2006, pp. 153–160 (2006)

    Google Scholar 

  6. Bruna, J., Mallat, S.: Invariant scattering convolution networks. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1872–1886 (2013)

    Article  Google Scholar 

  7. Erhan, D., Bengio, Y., Courville, A., Manzagol, P.A., Vincent, P., Bengio, S.: Why does unsupervised pre-training help deep learning? J. Mach. Learn. Res. 11, 625–660 (2010)

    MathSciNet  MATH  Google Scholar 

  8. Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016)

    MATH  Google Scholar 

  9. Guberman, N.: On complex valued convolutional neural networks. Master’s thesis, School of Computer Science and Engineering, The Hebrew University of Jerusalem (2016). https://arxiv.org/abs/1602.09046

  10. Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)

    Article  MathSciNet  Google Scholar 

  11. Hinton, G.E., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527–1554 (2006)

    Article  MathSciNet  Google Scholar 

  12. Jing, L., Shen, Y., Dubcek, T., Peurifoy, J., Skirlo, S., LeCun, Y., Tegmark, M., Soljacic, M.: Tunable efficient unitary neural networks (EUNN) and their application to RNNs. In: ICML 2017 (2017). https://arxiv.org/abs/1612.05231

  13. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: ICLR 2015 (2015)

    Google Scholar 

  14. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  15. Popa, C.A.: Complex-valued convolutional neural networks for real-valued image classification. In: 2017 International Joint Conference on Neural Networks (IJCNN). IEEE, May 2017

    Google Scholar 

  16. Sarroff, A.M., Shepardson, V., Casey, M.A.: Learning representations using complex-valued nets (2015). https://arxiv.org/abs/1511.06351

  17. Schmidhuber, J.: Deep learning in neural networks: an overview. Neural Netw. 61, 85–117 (2015)

    Article  Google Scholar 

  18. Tygert, M., Bruna, J., Chintala, S., LeCun, Y., Piantino, S., Szlam, A.: A mathematical motivation for complex-valued convolutional networks. Neural Comput. 28(5), 815–825 (2016)

    Article  Google Scholar 

  19. Vincent, P., Larochelle, H., Bengio, Y., Manzagol, P.A.: Extracting and composing robust features with denoising autoencoders. In: ICML 2008 (2008)

    Google Scholar 

  20. Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., Manzagol, P.A.: Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res. 11, 3371–3408 (2010)

    MathSciNet  MATH  Google Scholar 

  21. Wisdom, S., Powers, T., Hershey, J.R., Roux, J.L., Atlas, L.: Full-capacity unitary recurrent neural networks. In: NIPS 2016 (2016). https://arxiv.org/abs/1611.00035

  22. Xiao, H., Rasul, K., Vollgraf, R.: Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine and Learning Algorithms (2017). https://arxiv.org/abs/1708.07747

Download references

Acknowledgement

This work was supported by research grant no. PCD-TC-2017-41 of the Polytechnic University Timişoara, Romania.

Author information

Authors and Affiliations

  1. Department of Computer and Software Engineering, Polytechnic University Timişoara, Blvd. V. Pârvan, No. 2, 300223, Timişoara, Romania

    Călin-Adrian Popa

Authors
  1. Călin-Adrian Popa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Călin-Adrian Popa .

Editor information

Editors and Affiliations

  1. Texas A&M University at Qatar, Doha, Qatar

    Tingwen Huang

  2. Sichuan University, Chengdu, China

    Jiancheng Lv

  3. Southeast University, Nanjing, China

    Changyin Sun

  4. United Institute of Informatics Problems, Minsk, Belarus

    Alexander V. Tuzikov

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Popa, CA. (2018). Complex-Valued Stacked Denoising Autoencoders. In: Huang, T., Lv, J., Sun, C., Tuzikov, A. (eds) Advances in Neural Networks – ISNN 2018. ISNN 2018. Lecture Notes in Computer Science(), vol 10878. Springer, Cham. https://doi.org/10.1007/978-3-319-92537-0_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-92537-0_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-92536-3

  • Online ISBN: 978-3-319-92537-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation