Abstract
The use of machine learning into economics scenarios results appealing since it allows for automatically testing economic models and predict consumer/client behavior to support decision-making processes. The finance market typically uses a set of expert labelers or Bureau credit scores given by governmental or private agencies such as Experian, Equifax, and Creditinfo, among others. This work focuses on introducing a so-named Bag of Expert (BoE): a novel approach for creating multi-expert Learning (MEL) frameworks aimed to emulate real experts labeling (human-given labels) using neural networks. The MEL systems “learn” to perform decision-making tasks by considering a uniform number of labels per sample or individuals along with respective descriptive variables. The BoE is created similarly to Generative Adversarial Network (GANs), but rather than using noise or perturbation by a generator, we trained a feed-forward neural network to randomize sampling data, and either add or decrease hidden neurons. Additionally, this paper aims to investigate the performance on economics-related datasets of several state-of-the-art MEL methods, such as GPC, GPC-PLAT, KAAR, MA-LFC, MA-DGRL, and MA-MAE. To do so, we develop an experimental framework composed of four tests: the first one using novice experts; the second with proficient experts; the third is a mix of novices, intermediate and proficient experts, and the last one uses crowd-sourcing. Our BoE method presents promising results and can be suitable as an alternative to properly assess the reliability of both MEL methods and conventional labeler generators (i.e., virtual expert labelers).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Attigeri, G., Manohara Pai, M., Pai, R.: Framework to predict NPA/willful defaults in corporate loans: a big data approach. Int. J. Electr. Comput. Eng. 9(5), 3786–3797 (2019)
Bebis, G., Georgiopoulos, M.: Feed-forward neural networks. IEEE Potentials 13(4), 27–31 (1994)
Blitzer, J., Crammer, K., Kulesza, A., Pereira, F., Wortman, J.: Learning bounds for domain adaptation (2009)
Chang, V.: Generation of a HER2 breast cancer gold-standard using supervised learning from multiple experts. In: Stoyanov, D., et al. (eds.) LABELS/CVII/STENT -2018. LNCS, vol. 11043, pp. 45–54. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01364-6_6
Crammer, K., Kearns, M., Wortman, J.: Learning from multiple sources. J. Mach. Learn. Res. 9, 1757–1774 (2008)
Danenas, P., Garsva, G., Simutis, R.: Development of discriminant analysis and majority-voting based credit risk assessment classifier. vol. 1, pp. 204–209 (2011)
Dekel, O., Shamir, O.: Good Learners for Evil Teachers. Association for Computing Machinery, New York (2009)
Donmez, P., Carbonell, J.G., Schneider, J.: Efficiently Learning the Accuracy of Labeling Sources for Selective Sampling. Association for Computing Machinery, New York (2009)
Dua, D., Graff, C.: UCI machine learning repository
Gil-Gonzalez, J., Alvarez-Meza, A., Orozco-Gutierrez, A.: Learning from multiple annotators using kernel alignment. Pattern Recogn. Lett. 116, 150–156 (2018)
Groot, P., Birlutiu, A., Heskes, T.: Learning from multiple annotators with Gaussian processes. In: Honkela, T., Duch, W., Girolami, M., Kaski, S. (eds.) ICANN 2011. LNCS, vol. 6792, pp. 159–164. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-21738-8_21
Kim, M.J., Min, S.H., Han, I.: An evolutionary approach to the combination of multiple classifiers to predict a stock price index. Expert Syst. Appl. 31(2), 241–247 (2006)
Klepac, G.: Customer profiling in complex analytical environments using swarm intelligence algorithms. Int. J. Swarm Intell. Res. (IJSIR) 7(3), 43–70 (2016)
Lee, T., Cho, J., Kwon, D., Sohn, S.: Global stock market investment strategies based on financial network indicators using machine learning techniques. Expert Syst. Appl. 117, 228–242 (2019)
Long, C., Hua, G., Kapoor, A.: A joint gaussian process model for active visual recognition with expertise estimation in crowdsourcing. Int. J. Comput. Vis. 116(2), 136–160 (2016)
Mahapatra, D.: Combining multiple expert annotations using semi-supervised learning and graph cuts for medical image segmentation. Comput. Vis. Image Underst. 151, 114–123 (2016)
Murillo Rendón, S.: Metodología para el aprendizaje de máquina a partir de múltiples expertos en procesos de clasificación de bioseñales. Ph.D. thesis
Nir, G., et al.: Automatic grading of prostate cancer in digitized histopathology images: learning from multiple experts. Med. Image Anal. 50, 167–180 (2018)
Patwardhan, S., Yadav, D., Parlikar, S.: A review of role of data mining techniques in portfolio management. J. Adv. Res. Dyn. Control Syst. 11(2 Special Issue), 674–681 (2019)
Peluffo-Ordóñez, D., Murillo-Rendón, S., Arias-Londoño, J., Castellanos-Domínguez, G.: A multi-class extension for multi-labeler support vector machines, pp. 701–706 (2014)
Raykar, V., et al.: Supervised learning from multiple experts : whom to trust when everyone lies a bit, vol. 382 (2009)
Raykar, V., et al.: Learning from crowds. J. Mach. Learn. Res. 11, 1297–1322 (2010)
Rodrigues, F., Pereira, F., Ribeiro, B.: Learning from multiple annotators: distinguishing good from random labelers. Pattern Recogn. Lett. 34(12), 1428–1436 (2013)
Rodrigues, F., Pereira, F., Ribeiro, B.: Gaussian process classification and active learning with multiple annotators. In: Xing, E.P., Jebara, T. (eds.) Proceedings of the 31st International Conference on Machine Learning. Proceedings of Machine Learning Research PMLR, Bejing, China, 22–24 June 2014, vol. 32, pp. 433–441 (2014)
Valizadegan, H., Nguyen, Q., Hauskrecht, M.: Learning classification models from multiple experts. J. Biomed. Inform. 46(6), 1125–1135 (2013)
Wang, W., Zhou, Z.: Learnability of multi-instance multi-label learning. Chin. Sci. Bull. 57(19), 2488–2491 (2012)
Wiebe, J., Mihalcea, R.: Word sense and subjectivity, vol. 1, pp. 1065–1072 (2006)
Yan, Y., Rosales, R., Fung, G., Subramanian, R., Dy, J.: Learning from multiple annotators with varying expertise. Mach. Learn. 95(3), 291–327 (2013). https://doi.org/10.1007/s10994-013-5412-1
Yun, H., Lee, M., Kang, Y., Seok, J.: Portfolio management via two-stage deep learning with a joint cost. Expert Syst. Appl. 143, 113041 (2020)
Zhang, J., Wu, X., Sheng, V.S.: Imbalanced multiple noisy labeling. IEEE Trans. Knowl. Data Eng. 27(2), 489–503 (2015)
Zhang, Q., Yang, L.T., Chen, Z., Li, P., Bu, F.: An adaptive dropout deep computation model for industrial IoT big data learning with crowdsourcing to cloud computing. IEEE Trans. Ind. Inform. 15(4), 2330–2337 (2019)
Acknowledgment
This work is supported by the Smart Data Analysis Systems Group - SDAS Research Group (http://sdas-group.com)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Umaquinga-Criollo, A.C., Tamayo-Quintero, J.D., Moreno-García, M.N., Riascos, J.A., Peluffo-Ordóñez, D.H. (2020). Multi-expert Methods Evaluation on Financial and Economic Data: Introducing Bag of Experts. In: de la Cal, E.A., Villar Flecha, J.R., Quintián, H., Corchado, E. (eds) Hybrid Artificial Intelligent Systems. HAIS 2020. Lecture Notes in Computer Science(), vol 12344. Springer, Cham. https://doi.org/10.1007/978-3-030-61705-9_36
Download citation
DOI: https://doi.org/10.1007/978-3-030-61705-9_36
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-61704-2
Online ISBN: 978-3-030-61705-9
eBook Packages: Computer ScienceComputer Science (R0)