Abstract
A novel criterion to the well-known dictionary learning technique, K-SVD, is proposed. The approach exploits the L1-norm as the cost function for the dictionary update stage of K-SVD in order to provide robustness against impulsive noise and outlier input samples. The optimization algorithm successfully retrieves the first principal component of the input samples via greedy search methods and a parameter-free implementation. The final product is Robust K-SVD, a fast, reliable and intuitive algorithm. The results thoroughly detail how, under a wide range of noisy scenarios, the proposed technique outperforms K-SVD in terms of dictionary estimation and processing time. Recovery of Discrete Cosine Transform (DCT) bases and estimation of intrinsic dictionaries from noisy grayscale patches highlight the enhanced performance of Robust K-SVD and illustrate the circumvention of a misplaced assumption in sparse modeling problems: the availability of untampered, noiseless, and outlier-free input samples for training.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Aharon, M., Elad, M., Bruckstein, A.: K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Signal Process. 54(11), 4311–4322 (2006). https://doi.org/10.1109/TSP.2006.881199
Baccini, A., Besse, P., Falguerolles, A.: A L1-norm PCA and a heuristic approach. Ordinal Symb. Data Anal. 1(1), 359–368 (1996)
Bryt, O., Elad, M.: Compression of facial images using the K-SVD algorithm. J. Vis. Commun. Image Represent. 19(4), 270–282 (2008). https://doi.org/10.1016/j.jvcir.2008.03.001
Chen, S., Donoho, D., Saunders, M.: Atomic decomposition by basis pursuit. SIAM Rev. 43(1), 129–159 (2001). https://doi.org/10.1137/S003614450037906X
Ding, C., Zhou, D., He, X., Zha, H.: R1-PCA: rotational invariant L1-norm principal component analysis for robust subspace factorization. In: Proceedings of the 23rd International Conference on Machine Learning, pp. 281–288. ACM (2006). https://doi.org/10.1145/1143844.1143880
Elad, M., Aharon, M.: Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans. Image Process. 15(12), 3736–3745 (2006). https://doi.org/10.1109/TIP.2006.881969
Engan, K., Aase, S.O., Husoy, J.H.: Method of optimal directions for frame design. In: Proceedings of the 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 5, pp. 2443–2446. IEEE (1999). https://doi.org/10.1109/ICASSP.1999.760624
Georghiades, A.S., Belhumeur, P.N., Kriegman, D.J.: From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE Trans. Pattern Anal. Mach. Intell. 23(6), 643–660 (2001). https://doi.org/10.1109/34.927464
Ke, Q., Kanade, T.: Robust L1 norm factorization in the presence of outliers and missing data by alternative convex programming. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 1, pp. 739–746. IEEE (2005). https://doi.org/10.1109/CVPR.2005.309
Kwak, N.: Principal component analysis based on L1-norm maximization. IEEE Trans. Pattern Anal. Mach. Intell. 30(9), 1672–1680 (2008). https://doi.org/10.1109/TPAMI.2008.114
Lewicki, M.S., Olshausen, B.A.: Probabilistic framework for the adaptation and comparison of image codes. JOSA A 16(7), 1587–1601 (1999). https://doi.org/10.1364/JOSAA.16.001587
Loza, C.A., Principe, J.C.: A robust maximum correntropy criterion for dictionary learning. In: 2016 IEEE 26th International Workshop on Machine Learning for Signal Processing (MLSP), pp. 1–6. IEEE (2016). https://doi.org/10.1109/MLSP.2016.7738898
Mallat, S.G., Zhang, Z.: Matching pursuits with time-frequency dictionaries. IEEE Trans. Signal Process. 41(12), 3397–3415 (1993). https://doi.org/10.1109/78.258082
Mukherjee, S., Basu, R., Seelamantula, C.S.: L1-K-SVD: a robust dictionary learning algorithm with simultaneous update. Signal Process. 123, 42–52 (2016). https://doi.org/10.1016/j.sigpro.2015.12.008
Tropp, J.A., Gilbert, A.C.: Signal recovery from random measurements via orthogonal matching pursuit. IEEE Trans. Inf. Theory 53(12), 4655–4666 (2007). https://doi.org/10.1109/TIT.2007.909108
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Loza, C.A. (2018). Robust K-SVD: A Novel Approach for Dictionary Learning. In: Hernández Heredia, Y., Milián Núñez, V., Ruiz Shulcloper, J. (eds) Progress in Artificial Intelligence and Pattern Recognition. IWAIPR 2018. Lecture Notes in Computer Science(), vol 11047. Springer, Cham. https://doi.org/10.1007/978-3-030-01132-1_21
Download citation
DOI: https://doi.org/10.1007/978-3-030-01132-1_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-01131-4
Online ISBN: 978-3-030-01132-1
eBook Packages: Computer ScienceComputer Science (R0)