Abstract
In supervised learning scenarios, some applications require solve a classification problem wherein labels are not given as a single ground truth. Instead, the criteria of a set of experts is used to provide labels aimed at compensating for the erroneous influence with respect to a single labeler as well as the error bias (excellent or lousy) due to the level of perception and experience of each expert. This paper aims to briefly outline mathematical developments on support vector machines (SVM), and overview SVM-based approaches for multiple expert learning (MEL). Such MEL approaches are posed by modifying the formulation of a least-squares SVM, which enables to obtain a set of reliable, objective labels while penalizing the evaluation quality of each expert. Particularly, this work studies both two-class (binary) MEL classifier (BMLC) and its extension to multiclass through one-against all (OaA-MLC) including penalization of each expert’s influence. Formal mathematical developments are stated, as well as remarkable discussion on key aspects about the least-squares SVM formulation and penalty factors are provided.
Supported by SDAS Research Group (www.sdas-group.com).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alzubi, J., Nayyar, A., Kumar, A.: Machine learning from theory to algorithms: an overview. In: Journal of Physics: Conference Series, vol. 1142, p. 012012. IOP Publishing (2018)
Chang, V., et al.: Generation of a HER2 breast cancer gold-standard using supervised learning from multiple experts. In: Stoyanov, D. (ed.) LABELS/CVII/STENT -2018. LNCS, vol. 11043, pp. 45–54. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01364-6_6
Danenas, P., Garsva, G., Simutis, R.: Development of discriminant analysis and majority-voting based credit risk assessment classifier. In: Proceedings of the 2011 International Conference on Artificial Intelligence, ICAI 2011, vol. 1, pp. 204–209 (2011)
Dekel, O., Shamir, O.: Good learners for evil teachers. In: Proceedings of the 26th Annual International Conference on Machine Learning, ICML 2009, pp. 233–240. Association for Computing Machinery, New York (2009). https://doi.org/10.1145/1553374.1553404
Gil-Gonzalez, J., Alvarez-Meza, A., Orozco-Gutierrez, A.: Learning from multiple annotators using kernel alignment. Pattern Recogn. Lett. 116, 150–156 (2018). https://doi.org/10.1016/j.patrec.2018.10.005
Hsu, C.W., Lin, C.J.: A comparison of methods for multiclass support vector machines. Neural Netw. IEEE Trans. 13(2), 415–425 (2002)
Liu, Y., Zheng, Y.F.: One-against-all multi-class SVM classification using reliability measures. In: IEEE International Joint Conference on Neural Networks, vol. 2, pp. 849–854. IEEE (2005)
Mahapatra, D.: Combining multiple expert annotations using semi-supervised learning and graph cuts for medical image segmentation. Comput. Vis. Image Underst. 151, 114–123 (2016). https://doi.org/10.1016/j.cviu.2016.01.006
Murillo, S., Peluffo, D.H., Castellanos, G.: Support vector machine-based approach for multi-labelers problems. In: European Symposium on Artificial Neural Networks, Computational Inteligence and Machine Learning (2013)
Murillo-Rendón, S., Peluffo-Ordóñez, D., Arias-Londoño, J.D., Castellanos-Domínguez, C.G.: Multi-labeler analysis for bi-class problems based on soft-margin support vector machines. In: Ferrández Vicente, J.M., Álvarez Sánchez, J.R., de la Paz López, F., Toledo Moreo, F.J. (eds.) IWINAC 2013. LNCS, vol. 7930, pp. 274–282. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-38637-4_28
Nir, G., et al.: Automatic grading of prostate cancer in digitized histopathology images learning from multiple experts. Med. Image Anal. 50, 167–180 (2018)
Peluffo-Ordóñez, D.H., Rendón, S.M., Arias-Londoño, J.D., Castellanos-Domínguez, G.: A multi-class extension for multi-labeler support vector machines. In: European Symposium on Artificial Neural Networks, Computational Inteligence and Machine Learning (2014)
Raykar, V., et al.: Learning from crowds. J. Mach. Learn. Res. 11, 1297–1322 (2010). http://jmlr.org/papers/v11/raykar10a.html
Raykar, V.C., et al.: Supervised learning from multiple experts : whom to trust when everyone lies a bit. In: ACM International Conference Proceeding Series. vol. 382, pp. 1–8. ACM Press, New York (2009). https://doi.org/10.1145/1553374.1553488, http://portal.acm.org/citation.cfm?doid=1553374.1553488
Rodrigues, F., Pereira, F., Ribeiro, B.: Learning from multiple annotators: distinguishing good from random labelers. Pattern Recogn. Lett. 34(12), 1428–1436 (2013). https://doi.org/10.1016/j.patrec.2013.05.012
Rodrigues, F., Pereira, F., Ribeiro, B.: Gaussian process classification and active learning with multiple annotators. In: Xing, E.P., Jebara, T. (eds.) Proceedings of the 31st International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 32, pp. 433–441. PMLR, Bejing, China (22–24 Jun 2014). http://proceedings.mlr.press/v32/rodrigues14.html
Schölkopf, B., Smola, A.J.: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, Cambridge (2001)
Shalev-Shwartz, S., Ben-David, S.: Understanding Machine Learning: From Theory to Algorithms. Cambridge University Press, Cambridge (2014)
Umaquinga-Criollo, A.C., Tamayo-Quintero, J.D., Moreno-García, M.N., Riascos, J.A., Peluffo-Ordóñez, D.H.: Multi-expert methods evaluation on financial and economic data: introducing bag of experts. In: de la Cal, E.A., Villar Flecha, J.R., Quintián, H., Corchado, E. (eds.) HAIS 2020. LNCS (LNAI), vol. 12344, pp. 437–449. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-61705-9_36
Yan, Y., Rosales, R., Fung, G., Subramanian, R., Dy, J.: Learning from multiple annotators with varying expertise. Mach. Learn. 95(3), 291–327 (2014). https://doi.org/10.1007/s10994-013-5412-1
Zhang, J., Wu, X., Sheng, V.S.: Imbalanced multiple noisy labeling. IEEE Trans. Knowl. Data Eng. 27(2), 489–503 (2015)
Acknowledgments
This work is supported by the project PN223LH010-005 “Desarrollo de nuevos modelos y métodos matemáticos para la toma de decisiones”. As well, authors acknowledge the valuable support given by the SDAS Research Group (www.sdas-group.com).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Umaquinga-Criollo, A.C., Tamayo-Quintero, J.D., Moreno-García, M.N., Aalaila, Y., Peluffo-Ordóñez, D.H. (2021). Developments on Support Vector Machines for Multiple-Expert Learning. In: Yin, H., et al. Intelligent Data Engineering and Automated Learning – IDEAL 2021. IDEAL 2021. Lecture Notes in Computer Science(), vol 13113. Springer, Cham. https://doi.org/10.1007/978-3-030-91608-4_57
Download citation
DOI: https://doi.org/10.1007/978-3-030-91608-4_57
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-91607-7
Online ISBN: 978-3-030-91608-4
eBook Packages: Computer ScienceComputer Science (R0)