Abstract
Unsupervised feature selection (UFS) as an effective method to reduce time complexity and storage burden has been widely applied to various machine learning tasks. The selected features should model data distribution, preserve data reconstruction and maintain manifold structure. However, most UFS methods don’t consider these three factors simultaneously. Motivated by this, we propose a novel joint dictionary learning method, which handles these three key factors simultaneously. In joint dictionary learning, an intrinsic space shared by feature space and pseudo label space is introduced, which can model cluster structure and reveal data reconstruction. To ensure the sparseness of intrinsic space, the \({ \ell _{1}}\)-norm regularization is imposed on the representation coefficients matrix. The joint learning of robust sparse regression model and spectral clustering can select features that maintain data distribution and manifold structure. An efficient algorithm is designed to solve the proposed optimization problem. Experimental results on various types of benchmark datasets validate the effectiveness of our method.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alelyani, S., Tang, J., Liu, H.: Feature selection for clustering: a review. In: Aggarwal, C.C., Reddy, C.K. (eds.) Data Clustering: Algorithms and Applications, pp. 29–60. CRC Press, Boca Raton (2013)
Boyd, S.P., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011). https://doi.org/10.1561/2200000016
Cai, D., Zhang, C., He, X.: Unsupervised feature selection for multi-cluster data. In: Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 333–342 (2010). https://doi.org/10.1145/1835804.1835848
He, X., Cai, D., Niyogi, P.: Laplacian score for feature selection. In: Advances in Neural Information Processing Systems, vol. 18, pp. 507–514 (2005)
Kong, X., Yu, P.S.: Semi-supervised feature selection for graph classification. In: Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 793–802 (2010)
Li, J., Tang, J., Liu, H.: Reconstruction-based unsupervised feature selection: an embedded approach. In: Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, pp. 2159–2165 (2017)
Li, J., Wu, L., Dani, H., Liu, H.: Unsupervised personalized feature selection. In: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence (2018)
Li, Z., Yang, Y., Liu, J., Zhou, X., Lu, H.: Unsupervised feature selection using nonnegative spectral analysis. In: Proceedings of the Twenty-Sixth AAAI Conference on Artificial Intelligence (2012)
von Luxburg, U.: A tutorial on spectral clustering. Stat. Comput. 17(4), 395–416 (2007). https://doi.org/10.1007/s11222-007-9033-z
Nie, F., Huang, H., Cai, X., Ding, C.H.: Efficient and robust feature selection via joint \({\ell }_{2, 1}\)-norms minimization. In: Advances in Neural Information Processing Systems, pp. 1813–1821 (2010). https://doi.org/10.1007/978-3-319-10690-8_12
Qian, M., Zhai, C.: Robust unsupervised feature selection. In: IJCAI 2013, Proceedings of the 23rd International Joint Conference on Artificial Intelligence, pp. 1621–1627 (2013)
Wang, S., Tang, J., Liu, H.: Embedded unsupervised feature selection. In: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, pp. 470–476 (2015)
Zhao, Z., Liu, H.: Spectral feature selection for supervised and unsupervised learning. In: Machine Learning, Proceedings of the Twenty-Fourth International Conference, pp. 1151–1157 (2007). https://doi.org/10.1145/1273496.1273641
Zhu, F., Shao, L., Yu, M.: Cross-modality submodular dictionary learning for information retrieval. In: Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, pp. 1479–1488 (2014). https://doi.org/10.1145/2661829.2661926
Zhu, P., Hu, Q., Zhang, C., Zuo, W.: Coupled dictionary learning for unsupervised feature selection. In: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, pp. 2422–2428 (2016)
Zhu, P., Xu, Q., Hu, Q., Zhang, C.: Co-regularized unsupervised feature selection. Neurocomputing 275, 2855–2863 (2018). https://doi.org/10.1016/j.neucom.2017.11.061
Zhu, X., Zhu, Y., Zhang, S., Hu, R., He, W.: Adaptive hypergraph learning for unsupervised feature selection. In: Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, pp. 3581–3587 (2017)
Acknowledgment
This work was partially supported by the National Natural Science Foundation of China (No. 61473259) and the Hunan Provincial Science & Technology Project Foundation (2018TP1018, 2018RS3065).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Fan, Y., Dai, J., Zhang, Q., Liu, S. (2019). Joint Dictionary Learning for Unsupervised Feature Selection. In: Tetko, I., Kůrková, V., Karpov, P., Theis, F. (eds) Artificial Neural Networks and Machine Learning – ICANN 2019: Deep Learning. ICANN 2019. Lecture Notes in Computer Science(), vol 11728. Springer, Cham. https://doi.org/10.1007/978-3-030-30484-3_4
Download citation
DOI: https://doi.org/10.1007/978-3-030-30484-3_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-30483-6
Online ISBN: 978-3-030-30484-3
eBook Packages: Computer ScienceComputer Science (R0)