Skip to main content
SpringerLink
Log in

Complex Infomax: Convergence and Approximation of Infomax with Complex Nonlinearities

  • Published:
Journal of VLSI signal processing systems for signal, image and video technology Aims and scope Submit manuscript

Abstract

Independent component analysis (ICA) for separating complex-valued sources is needed for convolutive source-separation in the frequency domain, or for performing source separation on complex-valued data, such as functional magnetic resonance imaging or radar data. Previous complex Infomax approaches that use nonlinear functions in the updates have proposed using bounded (and hence non-analytic) nonlinearities. In this paper, we propose using an analytic (and hence unbounded) complex nonlinearity for Infomax for processing complex-valued sources. We show by simulation examples that using an analytic nonlinearity for processing complex data has a number of advantages. First, when compared to split-complex approaches (i.e., approaches that split the real and imaginary data into separate channels), the shape of the performance surface is improved resulting in better convergence characteristics. We also show that using an analytic complex-valued function for the nonlinearity is more effective in generating the higher order statistics required to establish independence when compared to complex nonlinear functions, i.e., functions that are ℂ → ℂ

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. V. D. Calhoun, T. Adalí, G. D. Pearlson, P. C. van Zijl, and J. J. Pekar, “Independent Component Analysis of FMRI Data in the Complex Domain,” Magn Reson. Med., vol. 48, 2002, pp. 180–192.

    Article  Google Scholar 

  2. V. D. Calhoun, T. Adalí, G. D. Pearlson, and J. J. Pekar, “On Complex Infomax Applied to Complex FMRI Data,” in Proc. ICASSP, Orlando, FL, 2002.

  3. G. Desodt and D. Muller, “Complex ICA Applied to the Separation of Radar Signals,” in Proc. EUSIPCO, Barcelona, Spain, vol. I, 1990, pp. 665–668.

    Google Scholar 

  4. Y. Chen and C. Hou, “High Resolution Adaptive Bearing Estimation Using a Complex-Weighted Neural Network,” in Proc. ICASSP, San Francisco, CA, vol. II, 1992, pp. 317–320.

    Google Scholar 

  5. A. Back and A. C. Tsoi, “Blind Deconvolution of Signals Using a Complex Recurrent Network,” in Proc. NNSP, Edinburgh, Scotland, 1994.

  6. P. Comon, “Independent Component Analysis - A New Concept?,” Signal Proc., vol. 36, 1994, pp. 287–314.

    Article  MATH  Google Scholar 

  7. B. Laheld and J. F. Cardoso, “Adaptive Source Separation With Uniform Performance,” in Proc. EUSIPCO, Edinburgh, UK, vol. II, 1994, pp. 183–186.

    Google Scholar 

  8. S. Fiori, A. Uncini, and F. Piazza, “Neural Blind Separation of Complex Sources By Extended Apex Algorithm (EAPEX),” in Proc. International Symposium on Circuits and Systems (ISCAS), Falmouth, MA, vol. V, 1999, pp. 627–630.

    Google Scholar 

  9. E. Bingham and A. Hyvarinen, “A Fast Fixed-Point Algorithm for Independent Component Analysis of Complex-Valued Signals,” Int. J. Neural Syst., vol. 10, 2000, pp. 1–8.

    Article  Google Scholar 

  10. H. Sawada, R. Mukai, S. Araki, and S. Makino, “A Polar-Coordinate Based Activation Function For Frequency Domain Blind Source Separation,” in Proc. Int. Conf. on ICA and BSS, San Diego, CA, 2001, pp. 663–668.

  11. P. Smaragdis, “Blind Separation of Convolved Mixtures in the Frequency Domain,” Neurocomputing, vol. 22, 1998, pp. 21–34.

    Article  MATH  Google Scholar 

  12. J. Annemueller, T. J. Sejnowski, and S. Makeig, “Complex Spectral-Domain Independent Component Analysis of Electroencephalographic Data,” in Proc. Int. Conf. on ICA and BSS, 2003.

  13. J. F. Cardoso and A. Souloumiac, “Blind Beamforming for Non Gaussian Signals,” IEE-Proceeding-F, vol. 140, 1993, pp. 362–370.

    Google Scholar 

  14. R. V. Churchill and J. W. Brown, Complex Variables and Applications, New York, NY: McGraw-Hill, Inc., 1984.

    MATH  Google Scholar 

  15. A. J. Bell and T. J. Sejnowski, “An Information Maximisation Approach to Blind Separation and Blind Deconvolution,” Neural Comput., vol. 7, 1995, pp. 1129–1159.

    Article  Google Scholar 

  16. T. W. Lee, M. Girolami, A. J. Bell, and T. J. Sejnowski, “A Unifying Information-Theoretic Framework for Independent Component Analysis,” Int. J. Comp. and Math. with Appl., vol. 31, 2000, pp. 1–21.

    Article  MathSciNet  Google Scholar 

  17. M. J. McKeown, S. Makeig, G. G. Brown, T. P. Jung, S. S. Kindermann, A. J. Bell, and T. J. Sejnowski, “Analysis of FMRI Data by Blind Separation Into Independent Spatial Components,” Hum. Brain Map., vol. 6, 1998, pp. 160–188.

    Article  Google Scholar 

  18. S. Makeig, T. P. Jung, A. J. Bell, D. Ghahremani, and T. J. Sejnowski, “Blind Separation of Auditory Event-Related Brain Responses into Independent Components,” Proc. Natl. Acad. Sci., vol. 94, 1997, pp. 10979–10984.

    Article  Google Scholar 

  19. B. W. Koehler, T. W. Lee, and R. Orglmeister, “Improving the Performance of Infomax Using Statistical Signal Processing Techniques,” in Proc. ICANN, Lausanne, Switzerland, 1997, pp. 535–540.

  20. A. Hyvarinen, “Survey on Independent Component Analysis,” Neural Computing Surveys, vol. 1, 1999, pp. 94–128.

    Google Scholar 

  21. J. Annemueller, T. J. Sejnowski, and S. Makeig, “Complex Independent Component Analysis of Frequency-Domain Electroencephalographic Data,” Neural Netw., vol. 16, 2003, pp. 1311–1323.

    Article  Google Scholar 

  22. T. Kim and T. Adalí, “Fully Complex Backpropagation for Constant Envelope Signal Processing,” in Proc. NNSP, Sydney, Australia, 2000, pp. 231–240.

  23. T. Kim and T. Adalí, “Fully Complex Multi-Layer Perceptron Network For Nonlinear Signal Processing,” Journal of VLSI Signal Processing Systems for Signal, image, and Video Technology, vol. 32, 2002, pp. 29–43.

    Article  MATH  Google Scholar 

  24. T. Kim and T. Adalí, “Universal Approximation of Fully Complex Feed-Forward Neural Networks,” in Proc. ICASSP, Orlando, FL, 2002.

  25. S. Amari, A. Cichocki, and H. H. Yang, “A New Learning Algorithm for Blind Signal Separation,” Advances in Neural Information Processing Systems, vol. 8, 1996, pp. 757–763.

    Google Scholar 

  26. J. F. Cardoso, “Infomax and Maximum Likelihood for Source Separation,” IEEE Letters on Signal Processing, vol. 4, 1997, pp. 112–114.

    Article  Google Scholar 

  27. N. Benvenuto and F. Piazza, “On the Complex Backpropagation Algorithm,” IEEE Trans. Signal Proc., vol. 40, 1992, pp. 967–969.

    Article  Google Scholar 

  28. H. Leung and S. Haykin, “The Complex Backpropagation Algorithm,” IEEE Trans. Signal Proc., vol. 39, 1991, pp. 2101–2104.

    Article  Google Scholar 

  29. G. M. Georgiou and C. Koutsougeras, “Complex Domain Backpropagation,” IEEE Trans. Circuits and Systems, vol. 39, 1992, pp. 330–334.

    Article  MATH  Google Scholar 

  30. T. Kim and T. Adalí, “Approximation by Fully Complex Multilayer Perceptrons,” Neural Comput., vol. 15, 2003, pp. 1641–1666.

    Article  MATH  Google Scholar 

  31. R. Der, “Blind Signal Separation,” Telecom.and Signal Proc.Lab Technical Report, 2001.

  32. T. W. Lee, M. Girolami, and T. J. Sejnowski, “Independent Component Analysis Using an Extended Infomax Algorithm for Mixed Subgaussian and Supergaussian Sources,” Neural Comput., vol. 11, 1999, pp. 417–441.

    Article  Google Scholar 

  33. B. A. Pearlmutter and L. C. Parra, “A Context-Sensitive Generalization of ICA,” in Proc. Int. Conf. on Neural Inf. Proc., Hong Kong, 1996.

  34. D. T. Pham and P. Garat, “Blind Separation of Mixture of Independent Sources Through a Quasi-Maximum Likelihood Approach,” IEEE Trans. Signal Proc., vol. 45, 1997, pp. 1712–1725.

    Article  MATH  Google Scholar 

  35. P. McCullagh, “Quasi-Likelihood Functions,” The Annals of Statistics, vol. 11, 1983, pp. 59–67.

    Article  MathSciNet  MATH  Google Scholar 

  36. E. Kreyszig, Advanced Engineering Mathematics, New York, NY: John Wiley & Sons, Inc., 1993.

    MATH  Google Scholar 

  37. V. D. Calhoun, T. Adalí, G. D. Pearlson, and J. J. Pekar, “Spatial and Temporal Independent Component Analysis of Functional MRI Data Containing a Pair of Task-Related Waveforms,” Hum. Brain Map., vol. 13, 2001, pp. 43–53.

    Article  Google Scholar 

  38. V.D.Calhoun, T.Adalí, and G.D.Pearlson, “Independent Components Analysis Applied to FMRI Data: A Generative Model for Validating Results,” to appear Journal of VLSI Signal Proc. Systems, 2004.

  39. T. W. Lee, Independent Component Analysis, Norwell, MA: Kluwer Academic Publishers, 1998.

    MATH  Google Scholar 

  40. T. Adalí, T. Kim, and V. D. Calhoun, “Independent Component Analysis By Complex Nonlinearities,” in Proc. ICASSP, Montreal, Canada, 2004.

  41. V. D. Calhoun and T. Adalí, “Complex ICA for FMRI Analysis: Performance of Several Approaches,” in Proc. ICASSP, Hong Kong, China, 2003.

  42. A. Hyvarinen and E. Oja, “A Fast Fixed-Point Algorithm for Independent Component Analysis,” Neural Comput., vol. 9, 1997, pp. 1483–1492.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Olin Neuropsychiatry Research Center, Institute of Living, 200 Retrat Ave., Hartford, CT, 06106

    Vince Calhoun Ph.D.

  2. Dept. of Psychiatry, Yale University, New Haven, CT, 06520

    Vince Calhoun Ph.D.

  3. Dept. of CSEE, University of Maryland Baltimore County, Baltimore, MD, 21250

    Tülay Adali

Authors
  1. Vince Calhoun Ph.D.
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tülay Adali
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vince Calhoun Ph.D..

Additional information

This work was supported in part by the National Science Foundation Career Award, NSF NCR-9703161 (to TA) and the National Institutes of Health 1 R01 EB 000840-01 (to VC).

Vince Calhoun received a bachelor’s degree in Electrical Engineering from the University of Kansas, Lawrence, Kansas, in 1991, master’s degrees in Biomedical Engineering and Information Systems from Johns Hopkins University, Baltimore, in 1993 and 1996, respectively, and the Ph.D. degree in electrical engineering from the University of Maryland Baltimore County, Baltimore, in 2002. He worked as a Senior Research Engineer in Psychiatric Neuro-Imaging at Johns Hopkins from 1993 until 2002.

He is currently the Director of the Medical Image Analysis Laboratory and an associate adjunct professor at Yale University. He is associate editor of the IEEE signal processing letters and on the editorial board for the Journal of Human Brain Mapping.

Dr. Calhoun is a member of the IEEE, the American Scientific Affiliation, the Organization for Human Brain Mapping, and the International Society for Magnetic Resonance in Medicine.

He has organized workshops for human brain mapping (HBM), the society of biological psychiatry (SOBP), and the international conference of independent component analysis and blind source separation (ICA). He is currently serving on the IEEE Machine Learning for Signal Processing (MLSP) Technical Committee and was the general chair for MLSP 2005 in Mystic, CT.

He works primarily with magnetic resonance imaging (functional imaging, diffusion tensor imaging, and structural imaging) and electroencephalography (EEG) data and is the author of more than 70 refereed articles in journals and conference proceedings in the areas of image processing, data fusion, adaptive signal processing, neural networks, statistical signal processing, and pattern recognition.

Tülay Adalı received the B.S. degree from Middle East Technical University, Ankara, Turkey, in 1987 and the M.S. and Ph.D. degrees from North Carolina State University, Raleigh, in 1988 and 1992 respectively, all in electrical engineering. In 1992, she joined the Department of Electrical Engineering at the University of Maryland Baltimore County, Baltimore, where she currently is a professor.

She has worked in the organization of a number of international conference and workshops including the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) and the IEEE International Workshop on Machine Learning for Signal Processing (MLSP).

She was the general co-chair for the NNSP workshops 2001-2003.

She is the past chair and a current member of the IEEE Machine Learning for Signal Processing Technical Committee and is serving on the IEEE Signal Processing Society conference board. She is an associate editor for the IEEE Transactions on Signal Processing and the Journal of VLSI Signal Processing Systems. She has also guest-edited a number of special issues for the IEEE Transactions on Neural Networks and the VLSI Signal Processing Systems on biomedical, multimedia, and data mining applications of neural networks.

She has authored or co-authored more than 175 refereed publications in the areas of statistical signal processing, neural computation, adaptive signal processing, biomedical data analysis, bioinformatics, and communications.

Dr. Adalı is the recipient of a 1997 National Science Foundation CAREER Award.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Calhoun, V., Adali, T. Complex Infomax: Convergence and Approximation of Infomax with Complex Nonlinearities. J VLSI Sign Process Syst Sign Image Video Technol 44, 173–190 (2006). https://doi.org/10.1007/s11265-006-7514-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11265-006-7514-5

Keywords

Search

Navigation