Abstract
Categorical attributes are present in datasets used in machine learning (ML) tasks. Since most ML algorithms only accept numeric inputs, categorical instances must be converted to numbers. There are different encoding techniques to accomplish this task. During this conversion, it is important to preserve the underlying pattern in the dataset. Otherwise, there may be a loss of information that can negatively affect the performance of supervised learning algorithms. In this paper, we present an encoding technique based on finding those numbers or codes that preserve the relationship between the categorical attribute and the other variables of the dataset. We solved six supervised classification problems using the proposed technique with five different ML algorithms. Additionally, we compare the performance of the proposed technique with other ten encoding techniques. We found that the proposed technique outperforms the most commonly used encoding techniques for certain trained ML algorithms. On average, CESAMMO remained within the top 5 techniques in terms of performance of the 12 encoders tested.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Kuhn, M., Johnson, K.: Feature Engineering and Selection: A Practical Approach for Predictive Models. CRC Press, Boca Raton (2019)
Hancock, J.T., Khoshgoftaar, T.M.: Survey on categorical data for neural networks. J. Big Data 7(1), 1–41 (2020)
Zheng, A., Casari, A.: Feature Engineering for Machine Learning: Principles and Techniques for Data Scientists. O’Reilly Media Inc., Boston (2018)
Micci-Barreca, D.: A preprocessing scheme for high-cardinality categorical attributes in classification and prediction problems. ACM SIGKDD Explor. Newsl. 3(1), 27–32 (2001)
Prokhorenkova, L., et al.: CatBoost: unbiased boosting with categorical features. Adv. Neural Inf. Process. Syst. 31 (2018)
Pargent, F., et al.: Regularized target encoding outperforms traditional methods in supervised machine learning with high cardinality features. Comput. Stat., 1–22 (2022)
De La Bourdonnaye, F., Daniel, F.: Evaluating categorical encoding methods on a real credit card fraud detection database. arXiv preprint arXiv:2112.12024 (2021)
Seca, D., Mendes-Moreira, J.: Benchmark of encoders of nominal features for regression. In: Rocha, Á., Adeli, H., Dzemyda, G., Moreira, F., Ramalho Correia, A.M. (eds.) WorldCIST 2021. AISC, vol. 1365, pp. 146–155. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-72657-7_14
Kuri-Morales, A.F.: Categorical encoding with neural networks and genetic algorithms. In: WSEAS Proceedings of the 6th International Conference on Applied Informatics and. Computing Theory, pp. 167–175 (2015)
Kuri-Morales, A.: Pattern discovery in mixed data bases. In: Martínez-Trinidad, J.F., Carrasco-Ochoa, J.A., Olvera-López, J.A., Sarkar, S. (eds.) MCPR 2018. LNCS, vol. 10880, pp. 178–188. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-92198-3_18
Kuri-Morales, A., Cartas-Ayala, A.: Polynomial multivariate approximation with genetic algorithms. In: Sokolova, M., van Beek, P. (eds.) AI 2014. LNCS (LNAI), vol. 8436, pp. 307–312. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-06483-3_30
Cheney, E.W.: Introduction to Approximation Theory. McGraw-Hill Book Company, New York (1966)
Rana, R., Singhal, R., et al.: Chi-square test and its application in hypothesis testing. J. Pract. Cardiovasc. Sci. 1(1), 69 (2015)
Cowles, M.: Statistics in Psychology: An Historical Perspective. Psychology Press, London (2005)
Valdez-Valenzuela, E., Kuri-Morales, A., Gomez-Adorno, H.: Measuring the effect of categorical encoders in machine learning tasks using synthetic data. In: Batyrshin, I., Gelbukh, A., Sidorov, G. (eds.) MICAI 2021. LNCS (LNAI), vol. 13067, pp. 92–107. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-89817-5_7
McGinnis, W.D., et al.: Category encoders: a scikit-learn-contrib package of transformers for encoding categorical data. J. Open Source Softw. 3(21), 501 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Valdez-Valenzuela, E., Kuri-Morales, A., Gomez-Adorno, H. (2022). CESAMMO: Categorical Encoding by Statistical Applied Multivariable Modeling. In: Pichardo Lagunas, O., Martínez-Miranda, J., Martínez Seis, B. (eds) Advances in Computational Intelligence. MICAI 2022. Lecture Notes in Computer Science(), vol 13612. Springer, Cham. https://doi.org/10.1007/978-3-031-19493-1_14
Download citation
DOI: https://doi.org/10.1007/978-3-031-19493-1_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-19492-4
Online ISBN: 978-3-031-19493-1
eBook Packages: Computer ScienceComputer Science (R0)