Abstract
Recently, sparse coding has been widely used in many applications ranging from image recovery to pattern recognition. The low mutual coherence of a dictionary is an important property that ensures the optimality of the sparse code generated from this dictionary. Indeed, most existing dictionary learning methods for sparse coding either implicitly or explicitly tried to learn an incoherent dictionary, which requires solving a very challenging non-convex optimization problem. In this paper, we proposed a hybrid alternating proximal algorithm for incoherent dictionary learning, and established its global convergence property. Such a convergent incoherent dictionary learning method is not only of theoretical interest, but also might benefit many sparse coding based applications.
Chapter PDF
Similar content being viewed by others
References
Aharon, M., Elad, M., Bruckstein, A.: K-SVD: An algorithm for designing of overcomplete dictionaries for sparse representation. IEEE Trans. Signal Process. (2006)
Rakotomamonjy, A.: Direct optimization of the dictionary learning. IEEE Trans. Signal Process. (2013)
Attouch, H., Bolte, J., Redont, P., Soubeyran, A.: Proximal alternating minimization and projection methods for nonconvex problems: An approach based on the kurdyka-lojasiewicz inequality. Math. Oper. Res. 35(2), 438–457 (2010)
Attouch, H., Bolte, J., Svaiter, B.F.: Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized gauss–seidel methods. Math. Program. Ser. A. 137(1-2), 91–129 (2013)
Bao, C., Cai, J., Ji, H.: Fast sparsity-based orthogonal dictionary learning for image restoration. In: ICCV (2013)
Bao, C., Ji, H., Quan, Y., Shen, Z.: ℓ0 norm based dictioanry learning by proximal method with global convergence. In: CVPR (2014)
Barchiesi, D., Plumbley, M.D.: Learning incoherent dictionaries for sparse approximation using iterative projections and rotations. IEEE Trans. Signal Process. (2013)
Bobin, J., Starck, J.L., Fadili, J.M., Moudden, Y., Donoho, D.L.: Morphological component analysis: An adaptive thresholding strategy. IEEE Trans. Image Process. 16(11) (2007)
Bolte, J., Sabach, S., Teboulle, M.: Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math. Program. Ser. A., 1–36 (2013)
Chen, S., Donoho, D., Saunders, M.: Atomic decomposition by basis pursuit. SIAM J. Sci. Comput. (1999)
Dai, W., Xu, T., Wang, W.: Dictionary learning and update based on simultaneous code-word optimzation (simco). In: ICASSP. IEEE (2012)
Fei-Fei, L., Fergus, R., Perona, P.: Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. In: CVPR Workshop of Generative Model Based Vision (WGMBV). IEEE (2004)
Georghiades, A.S., Belhumeur, P.N., Kriegman, D.J.: From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE Trans. Pattern Anal. Mach. Intell. (2001)
Jenatton, R., Mairal, J., Bach, F.R., Obozinski, G.R.: Proximal methods for sparse hierarchical dictionary learning. In: ICML (2010)
Jiang, Z., Lin, Z., Davis, L.: Learning a dicscriminative dictionary for sparse coding via label consistent K-SVD. In: CVPR (2011)
Mailhé, B., Barchiesi, D., Plumbley, M.D.: INK-SVD: Learning incoherent dictionaries for sparse representations. In: ICASSP (2012)
Mairal, J., Bach, F., Ponce, J., Sapiro, G.: Online learning for matrix factorization and sparse coding. JMLR (2010)
Mairal, J., Bach, F., Ponce, J., Sapiro, G., Zisserman, A.: Supervised dictionary learning. In: NIPS (2009)
Martínez, A., Benavente, R.: The ar face database. Tech. rep., Computer Vision Center (1998)
Parikh, N., Boyd, S.: Proximal algorithms. Found. Trends optim. 1(3), 123–231 (2013)
Ramirez, I., Sprechmann, P., Sapiro, G.: Classification and clustering via dictionary learning with structured incoherence and shared features. In: CVPR. IEEE (2010)
Rockafellar, R.T., Wets, R.J.B.: Variational analysis: grundlehren der mathematischen wissenschaften, vol. 317. Springer (1998)
Schnass, K., Vandergheynst, P.: Dictionary preconditioning for greedy algorithms. IEEE Trans. Signal Process. 56(5), 1994–2002 (2008)
Tosic, I., Frossard, P.: Dictionary learning. IEEE Signal Process. Mag. (2011)
Tropp, A.: Greed is good: algorithmic results for sparse approximation. IEEE Trans. Inf. Theory (2004)
Wright, J., Ma, Y., Mairal, J., Sapiro, G., Huang, T.S., Yan, S.: Sparse representation for computer vision and pattern recognition. Proc. IEEE 98(6), 1031–1044 (2010)
Xu, Y., Yin, W.: A fast patch-dictionary method for the whole image recovery. UCLA CAM report (2013)
Zhang, Q., Li, B.: Discriminative K-SVD for dictionary learning in face recognition. In: CVPR (2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
1 Electronic Supplementary Material
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Bao, C., Quan, Y., Ji, H. (2014). A Convergent Incoherent Dictionary Learning Algorithm for Sparse Coding. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds) Computer Vision – ECCV 2014. ECCV 2014. Lecture Notes in Computer Science, vol 8694. Springer, Cham. https://doi.org/10.1007/978-3-319-10599-4_20
Download citation
DOI: https://doi.org/10.1007/978-3-319-10599-4_20
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-10598-7
Online ISBN: 978-3-319-10599-4
eBook Packages: Computer ScienceComputer Science (R0)