Skip to main content
SpringerLink
Log in

Optimization Using Fourier Expansion over a Geodesic for Non-negative ICA

  • Conference paper
  • First Online:
Independent Component Analysis and Blind Signal Separation (ICA 2004)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 3195))

Abstract

We propose a new algorithm for the non-negative ICA problem, based on the rotational nature of optimization over a set of square orthogonal (orthonormal) matrices W, i.e. where W TW=WW T=I n. Using a truncated Fourier expansion of J(t), we obtain a Newton-like update step along the steepest-descent geodesic, which automatically approximates to a usual (Taylor expansion) Newton update step near to a minimum. Experiments confirm that this algorithm is effective, and it compares favourably with existing non-negative ICA algorithms. We suggest that this approach could modified for other algorithms, such as the normal ICA task.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 74.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Lee, D.D., Seung, H.S.: Algorithms for non-negative matrix factorization. In: Leen, T.K., Dietterich, T.G., Tresp, V. (eds.) Proceedings of NIPS*2000 Advances in Neural Information Processing Systems 13, pp. 556–562. MIT Press, Cambridge (2001)

    Google Scholar 

  2. Paatero, P., Tapper, U.: Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values. Environmetrics 5, 111–126 (1994)

    Article  Google Scholar 

  3. Harpur, G.F.: Low Entropy Coding with Unsupervised Neural Networks. PhD thesis, Department of Engineering, University of Cambridge (1997)

    Google Scholar 

  4. Cichocki, A., Georgiev, P.: Blind source separation algorithms with matrix constraints. IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences E86–A, 522–531 (2003)

    Google Scholar 

  5. Plumbley, M.D.: Conditions for nonnegative independent component analysis. IEEE Signal Processing Letters 9, 177–180 (2002)

    Article  Google Scholar 

  6. Plumbley, M.D.: Algorithms for nonnegative independent component analysis. IEEE Transactions on Neural Networks 14, 534–543 (2003)

    Article  Google Scholar 

  7. Plumbley, M.D., Oja, E.: A nonnegative PCA algorithm for independent component analysis. IEEE Transactions on Neural Networks 15, 66–76 (2004)

    Article  Google Scholar 

  8. Brockett, R.W.: Dynamical systems that sort lists, diagonalize matrices, and solve linear programming problems. Linear Algebra and its Applications 146, 79–91 (1991)

    Article  MathSciNet  Google Scholar 

  9. Edelman, A., Arias, T.A., Smith, S.T.: The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Anal. Appl. 20, 303–353 (1998)

    Article  MathSciNet  Google Scholar 

  10. Nishimori, Y.: Learning algorithm for ICA by geodesic flows on orthogonal group. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN 1999), Washington, DC, vol. 2, pp. 933–938 (1999)

    Google Scholar 

  11. Douglas, S.C.: Self-stabilized gradient algorithms for blind source separation with orthogonality constraints. IEEE Transactions on Neural Networks 11, 1490–1497 (2000)

    Article  Google Scholar 

  12. Fiori, S.: A theory for learning by weight flow on Stiefel-Grassman manifold. Neural Computation 13, 1625–1647 (2001)

    Article  Google Scholar 

  13. Hyvärinen, A., Karhunen, J., Oja, E.: Independent Component Analysis. John Wiley & Sons, Chichester (2001)

    Book  Google Scholar 

  14. Gallier, J., Xu, D.: Computing exponentials of skew-symmetric matrices and logarithms of orthogonal matrices. International Journal of Robotics and Automation 18, 10–20 (2003)

    Google Scholar 

  15. Comon, P.: Independent component analysis - a new concept? Signal Processing 36, 287–314 (1994)

    Article  Google Scholar 

  16. Fiori, S., Rossi, R.: Stiefel-manifold learning by improved rigid-body theory appliedto ICA. International Journal of Neural Systems 13, 273–290 (2003)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electronic Engineering, Queen Mary, University of London, Mile End Road, London, E1 4NS, UK

    Mark D. Plumbley

Authors
  1. Mark D. Plumbley
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Dept. of Architecture and Computer Technology, University of Granada, Spain

    Carlos G. Puntonet  & Alberto Prieto  & 

Rights and permissions

Reprints and permissions

Copyright information

© 2004 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Plumbley, M.D. (2004). Optimization Using Fourier Expansion over a Geodesic for Non-negative ICA. In: Puntonet, C.G., Prieto, A. (eds) Independent Component Analysis and Blind Signal Separation. ICA 2004. Lecture Notes in Computer Science, vol 3195. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30110-3_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-30110-3_7

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-23056-4

  • Online ISBN: 978-3-540-30110-3

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation