Skip to main content
Springer Nature Link
Log in

Fast and incoherent dictionary learning algorithms with application to fMRI

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

In this paper, the problem of dictionary learning and its analogy to source separation is addressed. First, we extend the well-known method of K-SVD to incoherent K-SVD, to enforce the algorithm to achieve an incoherent dictionary. Second, a fast dictionary learning algorithm based on steepest descent method is proposed. The main advantage of this method is high speed since both coefficients and dictionary elements are updated simultaneously rather than column-by-column. Finally, we apply the proposed methods to both synthetic and real functional magnetic resonance imaging data for the detection of activated regions in the brain. The results of our experiments confirm the effectiveness of the proposed ideas. In addition, we compare the quality of results and empirically prove the superiority of the proposed dictionary learning methods over the conventional algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. Note that some of the sparse recovery techniques, such as FOCUSS, uses a relaxed version of (1) by replacing \(\ell _0\)-norm with \(\ell _1\)-norm defined as \(\left\Vert\mathbf{x}\right\Vert_1=\sum _i{|x_i|}\). This can convexify the cost function.

  2. See [27] for a detailed discussion on this method.

  3. http://www.cs.technion.ac.il/~ronrubin/software.html.

  4. http://code.soundsoftware.ac.uk/embedded/incoherentdl/SMALL\$_\$incoherentDL.html.

  5. SMALLbox: http://small-project.eu/software-data/smallbox/.

  6. Since this method works only for non-negative data, we were not able to use it for synthetic fMRI data in previous section.

References

  1. Sezer O.G., Harmanci O., Guleryuz O.G.: Sparse orthonormal transforms for image compression. In: Proceedings of the 15th IEEE International Conference on Image Processing, ICIP 2008, pp. 149–152 (2008)

  2. Aharon, M., Elad, M., Bruckstein, A.: K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Signal Process. 54(11), 4311–4322 (2006)

    Article  Google Scholar 

  3. Elad, M., Starck, J., Querre, P., Donoho, D.: Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA). Appl. Computat. Harmon. Anal. 19(3), 340–358 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  4. Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  5. Candes, E.J., Romberg, J., Tao, T.: Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52(2), 489–509 (February 2006)

    Google Scholar 

  6. Engan, K., Aase, S.O., Husoy, J.H.: Method of optimal directions for frame design. In: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing. ICASSP’99, Washington, DC, USA, pp. 2443–2446 (1999)

  7. Kreutz-Delgado, K., Murray, J.F., Rao, B.D., Engan, K., Lee, T.W., Sejnowski, T.J.: Dictionary learning algorithms for sparse representation. Neural Computat. 15(2), 349–396 (2003)

    Article  MATH  Google Scholar 

  8. Olshausen, B.A., Field, D.J.: Sparse coding with an overcomplete basis set: a strategy employed by v1? Vis. Res. 37(23), 3311–3325 (1997)

    Google Scholar 

  9. Mairal, J., Bach, F., Ponce, J., Sapiro G.: Online dictionary learning for sparse coding. In: Proceedings of the 26th Annual International Conference on Machine Learning, ICML ’09, ACM, New York, NY, USA, pp. 689–696 (2009)

  10. Ramirez, I., Lecumberry, F., Sapiro, G.: Universal priors for sparse modeling. In: Proceedings of the 3rd IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing CAMSAP’09, pp. 197–200 (2009)

  11. Ramirez, I., Lecumberry, F., Sapiro G.: Sparse modeling with universal priors and learned incoherent dictionaries. NIPS (2009, submitted)

  12. Yaghoobi, M., Daudet, L., Davies, ME.: Structured and incoherent parametric dictionary design. In; Proceedings of the IEEE International Conference on Acoustics, Speech, and, Signal Processing, ICASSP’10, pp. 5486–5489 (2010)

  13. Ramirez, I., Sprechmann, P., Sapiro, G.: Classification and clustering via dictionary learning with structured incoherence and shared features. in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3501–3508 (2010)

  14. Mailhe, B., Barchiesi, D., Plumbley, M.D.: INK-SVD: learning incoherent dictionaries for sparse representations. In: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP’12 (2012 accepted)

  15. Daniele, B., Plumbley, M.D.: Learning incoherent dictionaries for sparse approximation using iterative projections and rotations. IEEE Trans. Audio Speech Lang. (2012, submitted)

  16. Hyvärinen, A., Karhunen, J., Oja E.: Independent Component Analysis, Wiley-Interscience, London (2001)

  17. Abolghasemi, V., Ferdowsi, S., Sanei, S.: Blind separation of image sources via adaptive dictionary learning. IEEE Trans. Image Process. 21(6), 2921–2930 (2012)

    Article  MathSciNet  Google Scholar 

  18. Filipovi, M., Kopriva, I.: A comparison of dictionary based approaches to inpainting and denoising with an emphasis to independent component analysis learned dictionaries. Inverse Probl. Imaging 5(4), 815–841 (2011)

    Article  MathSciNet  Google Scholar 

  19. Zibulevsky, M., Pearlmutter, B.A.: Blind source separation by sparse decomposition. Neural Comput. 13, 863–882 (1999)

    Article  Google Scholar 

  20. Daubechies, I., Roussos, E., Takerkart, S., Benharrosh, M., Golden, C., D’Ardenne, K., Richter, W., Cohen, JD., Haxby, J.: Independent component analysis for brain fMRI does not select for independence. Proc. Natl. Acad. Sci. 106(26), 10415–10422 (2009)

    Google Scholar 

  21. Ferdowsi, S., Abolghasemi, V., Sanei, S.: A constrained NMF algorithm for BOLD detection in fMRI. In: Proceedings of the IEEE International Workshop on Machine Learning for Signal Processing (MLSP), pp. 77–82 (2010)

  22. Lee, K., Ye, J.C.: A Data-Driven fMRI Analysis Using K-SVD Sparse Dictionary Learning. International Society of Magnetic Resonance in medicine ISMRM, Stockholm, Sweden (2010)

    Google Scholar 

  23. Kangjoo, L., Sungho, T., Jong, C.Y.: A data-driven sparse GLM for fMRI analysis using sparse dictionary learning with MDL criterion. IEEE Trans. Med. Imaging 30(5), 1076–1089 (2011)

    Article  Google Scholar 

  24. Tropp, J.A., Gilbert, A.C.: Signal recovery from random measurements via orthogonal matching pursuit. IEEE Trans. Inf. Theory 53(12), 4655–4666 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  25. Chen, S.S., Donoho, D., Saunders, M.A.: Atomic decomposition by basis pursuit. SIAM J. Sci. Comput. 20(1), 33–61 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  26. Gorodnitsky, I., Rao, B.: Sparse signal reconstruction from limited data using focuss: a re-weighted minimum norm algorithm. IEEE Trans. Signal Process. 45(3), 600–616 (March 1997)

    Google Scholar 

  27. Abolghasemi, V., Ferdowsi, S., Sanei, S.: On optimization of the measurement matrix for compressive sensing. In: Proceedings of the 18th European Signal Processing Conference EUSIPCO’10, Aalborg, Denmark, pp. 427–431 (2010)

  28. Cichocki, A., Zdunek, R., Phan, A.H., Amari, S.: Applications to Exploratory Multi-way Data Analysis and Blind Source Separation. Nonnegative Matrix and Tensor Factorizations. Wiley, London (2009)

  29. Bobin, J., Moudden, Y., Starck, JL.: Enhanced source separation by morphological component analysis. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP’06, vol. 5 (2006)

  30. Wright, S.J., Nowak, R.D., Figueiredo, M.A.T.: Sparse reconstruction by separable approximation. IEEE Trans. Signal Process. 57(7), 2479–2493 (2009)

    Article  MathSciNet  Google Scholar 

  31. Combettes, P.L., Wajs, V.R.: Signal recovery by proximal forward-backward splitting. Multiscale Model. Simul. 4(4), 1168–1200 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  32. Tseng, P.: Convergence of block coordinate descent method for non-differentiable minimization. J. Optim. Theory Appl. 109, 475–494 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  33. Peyré, G., Fadili, J., Starck, J.L.: Learning the morphological diversity. SIAM J. Imaging Sci. 3(3), 646–669 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  34. Bobin, J., Starck, J.L., Fadili, J., Moudden, Y.: Sparsity and morphological diversity in blind source separation. IEEE Trans. Image Process. 16(11), 2662–2674 (2007)

    Google Scholar 

  35. Mathews, V., Xie, Z.: A stochastic gradient adaptive filter with gradient adaptive step size. IEEE Trans. Signal Process. 41(6), 2075–2087 (1993)

    Article  Google Scholar 

  36. Douglas, S.: Generalized gradient adaptive step sizes for stochastic gradient adaptive filters. In: Proceedings of the International Conference on Acoustics, Speech, and, Signal Processing, ICASSP’95, vol. 2, pp. 1396–1399 (1995)

  37. Li, Y., Cichocki, A., Amari, S.: Analysis of sparse representation and blind source separation. Neural Comput. 16, 1193–1234 (2004)

    Article  MATH  Google Scholar 

  38. Mishali, M., Eldar, Y.C.: Sparse source separation from orthogonal mixtures. In: Proceedings of the IEEE International Conference on Acoustics, Speech and, Signal Processing. ICASSP’09, pp. 3145–3148 (2009)

  39. Dobigeon, N., Tourneret, J.Y.: Bayesian orthogonal component analysis for sparse representation. IEEE Trans. Signal Process. 58(5), 2675–2685 (2010)

    Article  MathSciNet  Google Scholar 

  40. Papadias, C., Kuzminskiy, A.: Blind source separation with randomized Gram–Schmidt orthogonalization for short burst systems. In: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP’04, vol. 5, pp. 809–812 (2004)

  41. MLSP-Lab. http://mlsp.umbc.edu/simulated_fmri_data.html

  42. The FastICA package for MATLAB. http://research.ics.tkk.fi/ica/fastica/

  43. Statitstical parameter mapping (SPM) website. http://www.fil.ion.ucl.ac.uk/spm/

  44. Qian, Y., Jia, S., Zhou, J., Robles-Kelly, A.: Hyperspectral unmixing via \(L_{1/2}\) sparsity-constrained nonnegative matrix factorization. IEEE Trans. Geosci. Remote Sens. 49, 4282–4297 (2011)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Electrical Engineering, Shahrood University, Shahrood, Iran

    Vahid Abolghasemi

  2. Faculty of Engineering and Physical Sciences, University of Surrey, Guildford, GU2 7XH, UK

    Saideh Ferdowsi & Saeid Sanei

Authors
  1. Vahid Abolghasemi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Saideh Ferdowsi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Saeid Sanei
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vahid Abolghasemi.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Abolghasemi, V., Ferdowsi, S. & Sanei, S. Fast and incoherent dictionary learning algorithms with application to fMRI. SIViP 9, 147–158 (2015). https://doi.org/10.1007/s11760-013-0429-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-013-0429-2

Keywords

Search

Navigation