Skip to main content
SpringerLink
Log in

Novel Multi-layer Non-negative Tensor Factorization with Sparsity Constraints

  • Conference paper
Adaptive and Natural Computing Algorithms (ICANNGA 2007)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 4432))

Included in the following conference series:

Abstract

In this paper we present a new method of 3D non-negative tensor factorization (NTF) that is robust in the presence of noise and has many potential applications, including multi-way blind source separation (BSS), multi-sensory or multi-dimensional data analysis, and sparse image coding. We consider alpha- and beta-divergences as error (cost) functions and derive three different algorithms: (1) multiplicative updating; (2) fixed point alternating least squares (FPALS); (3) alternating interior-point gradient (AIPG) algorithm. We also incorporate these algorithms into multilayer networks. Experimental results confirm the very useful behavior of our multilayer 3D NTF algorithms with multi-start initializations.

Invited paper.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Workshop on tensor decompositions and applications. CIRM, Marseille, France (2005)

    Google Scholar 

  2. Hazan, T., Polak, S., Shashua, A.: Sparse image coding using a 3D non-negative tensor factorization. In: International Conference of Computer Vision (ICCV), pp. 50–57 (2005), http://www.cs.huji.ac.il/~tamirr/papers/NTF-iccv05.pdf

  3. Heiler, M., Schnörr, C.: Controlling sparseness in non-negative tensor factorization. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 56–67. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  4. Smilde, A., Bro, R., Geladi, P.: Multi-way Analysis: Applications in the Chemical Sciences. John Wiley and Sons, New York (2004)

    Google Scholar 

  5. Berry, M., Browne, M., Langville, A., Pauca, P., Plemmons, R.: Algorithms and applications for approximate nonnegative matrix factorization. Computational Statistics and Data Analysis, submitted (2006)

    Google Scholar 

  6. Cichocki, A., Zdunek, R., Amari, S.-i.: Csiszár’s divergences for non-negative matrix factorization: Family of new algorithms. In: Rosca, J.P., Erdogmus, D., Príncipe, J.C., Haykin, S. (eds.) ICA 2006. LNCS, vol. 3889, pp. 32–39. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  7. Cichocki, A., Amari, S.-i., Zdunek, R., Kompass, R., Hori, G., He, Z.: Extended SMART algorithms for non-negative matrix factorization. In: Rutkowski, L., Tadeusiewicz, R., Zadeh, L.A., Żurada, J.M. (eds.) ICAISC 2006. LNCS (LNAI), vol. 4029, pp. 548–562. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  8. Cichocki, A., Zdunek, R.: NMFLAB for Signal and Image Processing. Technical report, Laboratory for Advanced Brain Signal Processing, BSI, RIKEN, Saitama, Japan (2006)

    Google Scholar 

  9. Cichocki, A., Zdunek, R.: NTFLAB for Signal Processing. Technical report, Laboratory for Advanced Brain Signal Processing, BSI, RIKEN, Saitama, Japan (2006)

    Google Scholar 

  10. Dhillon, I., Sra, S.: Generalized nonnegative matrix approximations with Bregman divergences. In: Neural Information Proc. Systems, Vancouver, Canada, December 2005, pp. 283–290 (2005)

    Google Scholar 

  11. Hoyer, P.: Non-negative matrix factorization with sparseness constraints. Journal of Machine Learning Research 5, 1457–1469 (2004)

    MathSciNet  Google Scholar 

  12. Kim, M., Choi, S.: Monaural music source separation: Nonnegativity, sparseness, and shift-invariance. In: Rosca, J., Erdogmus, D., Príncipe, J.C., Haykin, S. (eds.) ICA 2006. LNCS, vol. 3889, pp. 617–624. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  13. Lee, D.D., Seung, H.S.: Learning the parts of objects by nonnegative matrix factorization. Nature 401, 788–791 (1999)

    Article  Google Scholar 

  14. Vasilescu, M.A.O., Terzopoulos, D.: Multilinear analysis of image ensembles: TensorFaces. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2350, pp. 447–460. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  15. Morup, M., Hansen, L.K., Herrmann, C.S., Parnas, J., Arnfred, S.M.: Parallel factor analysis as an exploratory tool for wavelet transformed event-related EEG. NeuroImage 29(3), 938–947 (2006)

    Article  Google Scholar 

  16. Miwakeichi, F., Martínez-Montes, E., Valdés-Sosa, P.A., Nishiyama, N., Mizuhara, H., Yamaguchi, Y.: Decomposing EEG data into space-time-frequency components using Parallel Factor Analysis. NeuroImage 22, 1035–1045 (2004)

    Article  Google Scholar 

  17. Amari, S.: Differential-Geometrical Methods in Statistics. Springer, Heidelberg (1985)

    MATH  Google Scholar 

  18. Merritt, M., Zhang, Y.: An interior-point gradient method for large-scale totally nonnegative least squares problems. J. Optimization Theory and Applications 126(1), 191–202 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  19. Cichocki, A., Zdunek, R.: Multilayer nonnegative matrix factorization. Electronics Letters 42(16), 947–948 (2006)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. RIKEN Brain Science Institute, Wako-shi, Japan

    Andrzej Cichocki, Rafal Zdunek & Shun-ichi Amari

  2. POSTECH, Korea

    Seungjin Choi

  3. Dept. of Mathematics and Computer Science, Wake Forest University, USA

    Robert Plemmons

Authors
  1. Andrzej Cichocki
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rafal Zdunek
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Seungjin Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Robert Plemmons
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shun-ichi Amari
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Bartlomiej Beliczynski Andrzej Dzielinski Marcin Iwanowski Bernardete Ribeiro

Rights and permissions

Reprints and permissions

Copyright information

© 2007 Springer Berlin Heidelberg

About this paper

Cite this paper

Cichocki, A., Zdunek, R., Choi, S., Plemmons, R., Amari, Si. (2007). Novel Multi-layer Non-negative Tensor Factorization with Sparsity Constraints. In: Beliczynski, B., Dzielinski, A., Iwanowski, M., Ribeiro, B. (eds) Adaptive and Natural Computing Algorithms. ICANNGA 2007. Lecture Notes in Computer Science, vol 4432. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-71629-7_31

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-71629-7_31

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-71590-0

  • Online ISBN: 978-3-540-71629-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation