Abstract
In recent years, data volume is getting larger along with the fast development of Internet technologies. Some datasets contain a huge number of labels, dimensions and data points. As a result, some of them cannot be loaded by typical classifiers, and some of them require very long and unacceptable time for execution. Extreme multi-label classification is designed for these challenges. Extreme multi-label classification differs from traditional multi-label classification in a number of ways including the need for lower execution time, training at an extreme scale with millions of data points, features and labels, etc. In order to enhance the practicality, in this paper, we focus on designing an extreme multi-label classification approach which can be performed on a single person-al computer. We devise a two-phase framework for dealing with the above issues. In the reweighting phase, the prediction precision is improved by paying more attention on hard-to-classify instances and increasing the diversity of the model. In the pretesting phase, trees with lower quality will be removed from the prediction model for reducing the model size and increasing the prediction precision. Experiments on real world datasets will verify that the pro-posed method is able to generate better prediction results and the model size is successfully shrunk down.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Jain, H., Prabhu, Y., Varma, M.: Extreme multi-label loss functions for recommendation, tagging, ranking & other missing label applications. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 935–944. ACM (2016)
Prabhu, Y., Varma, M.: FastXML: a fast, accurate and stable tree-classifier for extreme multi-label learning. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 263–272. ACM (2014)
Bhatia, K., Jain, H., Kar, P., Varma, M., Jain, P.: Sparse local embeddings for extreme multi-label classification. In: Advances in Neural Information Processing Systems, pp. 730–738 (2015)
Babbar, R., Shoelkopf, B.: DiSMEC: distributed sparse machines for extreme multi-label classification. In: Proceedings of the Tenth ACM International Conference on Web Search and Data Mining, pp. 721–729. ACM (2017)
Yen, I.E., Huang, X., Zhong, K., Ravikumar, P., Dhillon, I.S.: PD-sparse: a primal and dual sparse approach to extreme multiclass and multilabel classification. In: Proceedings of The 33rd International Conference on Machine Learning, pp. 3069–3077. IEEE (2016)
Xu, C., Tao, D., Xu, C.: Robust extreme multi-label learning. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1275–1284. ACM (2016)
Yu, H.F., Jain, P., Kar, P., Dhillon, I.S.: Large-scale multi-label learning with missing labels. In: Proceedings of The 31st International Conference on Machine Learning, pp. 593–601. IEEE (2014)
Adnan, Md.N., Islam, Md.Z.: Forest CERN: a new decision forest building technique. In: Bailey, J., Khan, L., Washio, T., Dobbie, G., Huang, J.Z., Wang, R. (eds.) PAKDD 2016. LNCS, vol. 9651, pp. 304–315. Springer, Cham (2016). doi:10.1007/978-3-319-31753-3_25
Adnan, Md N., Islam, Md Z.: On improving random forest for hard-to-classify records. In: Li, Jinyan, Li, Xue, Wang, Shuliang, Li, Jianxin, Sheng, Quan Z. (eds.) ADMA 2016. LNCS, vol. 10086, pp. 558–566. Springer, Cham (2016). doi:10.1007/978-3-319-49586-6_39
Snoek, C.G., Worring, M., Van Gemert, J.C., Geusebroek, J.M., Smeulders, A.W.: The challenge problem for automated detection of 101 semantic concepts in multimedia. In: Proceedings of the 14th Annual ACM International Conference on Multimedia, pp. 421– 430. ACM (2006)
Katakis, I., Tsoumakas, G., Vlahavas, I: Multilabel text classification for automated tag suggestion. In: ECML/PKDD Discovery Challenge (2008)
Tsoumakas, G., Katakis, I., Vlahavas, I.: Effective and efficient multilabel classification in domains with large number of labels. In: ECML/PKDD 2008 Workshop on Mining Multidimensional Data, pp. 30–44 (2008)
Mencia, E.L., Fürnkranz, J.: Efficient pairwise multilabel classification for large-scale problems in the legal domain. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 50–65 (2008)
McAuley, J., Leskovec, J.: Hidden factors and hidden topics: understanding rating dimensions with review text. In: Proceedings of the 7th ACM conference on Recommender systems, pp. 165–172. ACM (2013)
Zubiaga, A.: Enhancing navigation on wikipedia with social tags. Preprint (2012)
Partalas, I., Kosmopoulos, A., Baskiotis, N., Artieres, T., Paliouras, G., Gaussier, E., Androutsopoulos, I., Amini, M.-R., Galinari, P.: LSHTC: A benchmark for large-scale text classification. Preprint (2015)
The Extreme Classification Repository. http://research.microsoft.com/en-us/um/people/manik/downloads/XC/XMLRepository.html
John, G.H., Langley, P.: Estimating continuous distributions in Bayesian classifiers. In: Proceedings of the Eleventh Conference on Uncertainty in Artificial Intelligence, pp. 338–345 (1995)
Safavian, S.R., Landgrebe, D.: A survey of decision tree classifier methodology. IEEE Trans. Syst. Man Cybern. 21(3), 660–674 (1991)
Weston, J., Makadia, A., Yee, H.: Label partitioning for sublinear ranking. In: Proceeding of ICML, pp. 181–189 (2013)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)
Suykens, J.A., Vandewalle, J.: Least squares support vector machine classifiers. Neural Process. Lett. 9, 293–300 (1999)
Al Bataineh, M., Al-Qudah, Z.: A novel gene identification algorithm with Bayesian classification. In: Biomedical Signal Processing and Control, pp. 6–15 (2017)
Goodman, K.E., Lessler, J., Cosgrove, S.E., Harris, A.D., Lautenbach, E., Han, J.H., Tamma, P.D.: A clinical decision tree to predict whether a bacteremic patient is infected with an extended-spectrum β-Lactamase–producing organism. In: Clinical Infectious Diseases, pp. 896–903 (2016)
Esteva, A., Kuprel, B., Novoa, R.A., Ko, J., Swetter, S.M., Blau, H.M., Thrun, S.: Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 115–118 (2017)
Geng, Y., Chen, J., Fu, R., Bao, G., Pahlavan, K.: Enlighten wearable physiological monitoring systems: on-body RF characteristics based human motion classification using a support vector machine. IEEE Trans. Mobile Comput. 15, 656–671 (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Lin, ZZ., Dai, BR. (2017). Reweighting Forest for Extreme Multi-label Classification. In: Bellatreche, L., Chakravarthy, S. (eds) Big Data Analytics and Knowledge Discovery. DaWaK 2017. Lecture Notes in Computer Science(), vol 10440. Springer, Cham. https://doi.org/10.1007/978-3-319-64283-3_21
Download citation
DOI: https://doi.org/10.1007/978-3-319-64283-3_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-64282-6
Online ISBN: 978-3-319-64283-3
eBook Packages: Computer ScienceComputer Science (R0)