Abstract
This work implements the BFGS (Broyden-Fletcher-Goldfarb-Shanno) optimization method for training the type-1 and singleton fuzzy logic system applied to solve binary classification problems. The BFGS is a quasi-Newton method that approximates the second-order information using the gradient of the cost function. Additionally, the Golden Section method is used to obtain the step size for each line search in a descent direction. The effectiveness of the proposed method is demonstrated by using well-established classification metrics evaluated in popular datasets from the literature. Comparisons between the proposed approach and well-known gradient-based methods available are also provided, showing that the BFGS achieves improved performance in terms of accuracy, mean squared error, and the number of epoch demanded during the training phase.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Datasets provided by Knowledge Extraction based on Evolutionary Learning (KEEL) Repository
Datasets provided by UCI (University of California, Irvine) Machine Learning Repository (Lichman 2013)
References
Alcalá, R., Cordón, O., Casillas, J., Herrera, F., & Zwir, S. (2000). Learning and tuning fuzzy rule-based systems for linguistic modeling. In C. T. Leondes (Ed.), Knowledge-Based Systems (pp. 889–941). San Diego: Academic Press.
Alcalá-Fdez, J., Fernández, A., Luengo, J., Derrac, J., García, S., Sánchez, L., & Herrera, F. (2011). Keel data-mining software tool: Data set repository, integration of algorithms and experimental analysis framework. Journal of Multiple-Valued Logic & Soft Computing,17.
Berger, V. W., & Zhou, Y. (2014). Kolmogorov–Smirnov test: Overview. Wiley statsref: Statistics reference online.
Chu, C.-W., Liang, G.-S., & Liao, C.-T. (2008). Controlling inventory by combining ABC analysis and fuzzy classification. Computers & Industrial Engineering, 55(4), 841–851.
de Aguiar, E. P., Nogueira, F. M., Amaral, R. P., Fabri, D. F., Rossignoli, S. C., Ferreira, J. G., et al. (2016). EANN 2014: A fuzzy logic system trained by conjugate gradient methods for fault classification in a switch machine. Neural Computing and Applications, 27(5), 1175–1189.
Dhimish, M., Holmes, V., Mehrdadi, B., Dales, M., & Mather, P. (2017). Photovoltaic fault detection algorithm based on theoretical curves modelling and fuzzy classification system. Energy, 140, 276–290.
Finotti Amaral, R. P., Menezes, I. F., & Ribeiro, M. V. (2020). An extension of the type-1 and singleton fuzzy logic system trained by scaled conjugate gradient methods for multiclass classification problems. Neurocomputing, 411, 149–163.
Finotti Amaral, R. P., Ribeiro, M. V., & de Aguiar, E. P. (2019). Type-1 and singleton fuzzy logic system trained by a fast scaled conjugate gradient methods for dealing with binary classification problems. Neurocomputing, 355, 57–70.
Ganji, M. F., & Abadeh, M. S. (2011). A fuzzy classification system based on ant colony optimization for diabetes disease diagnosis. Expert Systems with Applications, 38(12), 14650–14659.
Hollander, M., Wolfe, D. A., & Chicken, E. (2013). Nonparametric statistical methods, vol. 751. John Wiley & Sons.
Karaboga, D., & Kaya, E. (2019). Adaptive network based fuzzy inference system (ANFIS) training approaches: A comprehensive survey. Artificial Intelligence Review, 52(4), 2263–2293.
Kohavi, R. (1995). A study of cross-validation and bootstrap for accuracy estimation and model selection. In Ijcai, vol. 14. Stanford, CA (pp. 1137–1145).
Lichman, M. (2013). UCI machine learning repository. http://archive.ics.uci.edu/ml.
Mendel, J. (2001). Uncertain rule-based fuzzy logic system: Introduction and new directions. Prentice–Hall PTR.
Mishra, S. K., & Ram, B. (2019). Introduction to Unconstrained Optimization with R. Springer Nature.
Moore, D. S., & Kirkland, S. (2007). The basic practice of statistics, vol. 2. WH Freeman New York.
Nilashi, M., Ibrahim, O., Ahmadi, H., & Shahmoradi, L. (2017). A knowledge-based system for breast cancer classification using fuzzy logic method. Telematics and Informatics, 34(4), 133–144.
Nocedal, J., & Wright, S. (2006). Numerical optimization. Springer Science & Business Media.
Press, W. H., Teukolsky, S. A., Vetterling, W. T., & Flannery, B. P. (1992). Numerical recipes in Fortran 77: The art of scientific computing, vol. 2. Cambridge university press.
Scales, L. (1985). Introduction to non-linear optimization. Macmillan International Higher Education.
Schaefer, G., Závišek, M., & Nakashima, T. (2009). Thermography based breast cancer analysis using statistical features and fuzzy classification. Pattern Recognition, 42(6), 1133–1137.
Sokolova, M., Japkowicz, N., & Szpakowicz, S. (2006). Beyond accuracy, f-score and roc: A family of discriminant measures for performance evaluation. In A. Sattar & B.-H. Kang (Eds.), AI 2006: Advances in Artificial Intelligence (pp. 1015–1021). Berlin, Heidelberg: Springer.
Stehman, S. (1996). Estimating the kappa coefficient and its variance under stratified random sampling. Photogrammetric Engineering and Remote Sensing, 62(4), 401–407.
Theodoridis, S., & Koutroumbas, K. (2009). Pattern recognition, 4th ed. Academic Press.
Wang, L., & Mendel,J. M. (1992). Back-propagation fuzzy system as nonlinear dynamic system identifiers. In 1992 Proc. IEEE International Conference on Fuzzy Systems (pp. 1409–1418).
Acknowledgements
This work was supported by the National Council for Scientific and Technological Development (CNPq - Conselho Nacional de Desenvolvimento Científico e Tecnolgico - Brazil). It was also financed by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001. We also thank the Carlos Chagas Filho Research Support Foundation (FAPERJ, E-26/200.260/2020). The authors acknowledge the support provided by the Tecgraf Institute of Technical-Scientific Software Development of PUC-Rio (Tecgraf/PUC-Rio), Brazil. Any opinions, findings, conclusions, or recommendations expressed here are those of the authors and do not necessarily reflect the views of the sponsors.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of interest
We wish to confirm that there are no known conflicts of interest associated with this publication.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Calderano, P.H.S., Mateus Gheorghe, d.C.R., Teixeira, R.S. et al. Type-1 and singleton fuzzy logic system binary classifier trained by BFGS optimization method. Fuzzy Optim Decis Making 22, 149–168 (2023). https://doi.org/10.1007/s10700-022-09387-y
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10700-022-09387-y