Abstract
A new family of techniques, called conformal predictors, have very recently been developed to hedge the estimates of machine learning methods, by providing two parameters, credibility and confidence, which can assess the level of trust that can be attributed to their outputs. In this paper, the main steps required to extend this approach to fuzzy logic classifiers are reported. The more delicate aspect is the definition of an appropriate nonconformity score, which has to be based on the fuzzy membership function to preserve the specificities of Fuzzy Logic. Various examples of increasing complexity are introduced, to describe the main properties of fuzzy logic based conformal predictors and to compare their performance with alternative approaches. The obtained results are quite promising, since conformal predictors based on fuzzy classifiers outperform solutions based on the nearest neighbour in terms of ambiguity, robustness and interpretability.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Vapnik, V.N.: The Nature of Statistical Learning Theory, 2nd edn. Springer, New York (2000)
Theodoridis, S., Koutroumbas, K.: Pattern Recognition. Elsevier, Academic Press, Canada (2009)
Bishop, C.M.: Pattern Recognition and Machine Learning. Springer Science+Bussiness Media Singapore, Singapore (2006)
Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Identification, 2nd edn.Wiley-Interscience, New York (2000)
Vovk, V., Gammerman, A., Shafer, G.: Algorithmic Learning in a Random World. Springer, New York (2005)
Vega, J., Murari, A., Pereira, A., Gonz´alez, S., Pastor, I.: Rev. Sci. Instrum. 81, 10E118 (2010)
Ross, T.J.: Fuzzy Logic with Engineering Applications, 2nd edn. Wiley, Singapore (2004)
Riid, A., Ruistern, E.: Transparent fuzzy systems in modelling and control. In: Casillas, J., Cordon, O., Herrera, F., Magdalena, L. (eds.): Interpretability Issues in Fuzzy Modeling, pp. 452–476. Springer (2003)
Nauck, D., Kruse, R.: How the learning of rule weights affects the interpretability of fuzzy systems. In: Proceedings IEEE International Conference on Fuzzy Systems 1998. pp. 1235–1240. Anchorage (1998)
Papadopoulos, H., Vovk, V., Gammerman, A.: Regression conformal prediction with nearest neighbours. J. Artif. Intell. Res. 40, 815–840 (2011). doi:10.1613/jair.3198
Nouretdinov, I., Melluish, T., Vovk, V.: Ridge regression confidence machine. In: Proceedings of the 18th International Conference on Machine Learning (ICML’01), pp. 385–392. Morgan Kaufmann, San Francisco (2001)
Proedrou, K., Nouretdinov, I., Vovk, V., Gammerman, A.: Transductive confidence machines for pattern recognition. In: Proceedings of the 13th European Conference on Machine Learning (ECML’02), ser. LNCS, vol. 2430, pp. 381–390. Springer (2002)
Papadopoulos, H., Haralambous, H.: Reliable prediction intervals with regression neural networks. Neural Netw. (2011). doi:10.1016/j.neunet.2011.05.008
Bhattacharyya, S.: Confidence in predictions from random tree ensembles. In: Proceedings of the 11th IEEE International Conference on Data Mining (ICDM2011), pp. 71–80. Springer (2011)
Gammerman, A., Vovk, V., Burford, B., Nouretdinov, I., Luo, Z., Chervonenkis, A., Waterfield, M., Cramer, R., Tempst, P., Villanueva, J., Kabir, M., Camuzeaux, S., Timms, J., Menon, U., Jacobs, I.: Serum proteomic abnormality predating screen detection of ovarian cancer. Comput. J. 52(3), 326–333 (2009)
Papadopoulos, H., Papatheocharous, E., Andreou, A.S.: Reliable confidence intervals for software effort estimation. In: Proceedings of the 2nd Workshop on Artificial Intelligence Techniques in Software Engineering (AISEW 2009), ser. CEUR Workshop Proceedings, vol. 475. CEUR-WS. org, [Online]. Available: http://ceur-ws.org/Vol-475/AISEW2009/22-pp-211-220-208.pdf
Bellotti, T., Luo, Z., Gammerman, A., Delft, F.W.V., Saha, V.: Qualified predictions for microarray and proteomics pattern diagnostics with confidence machines. Int. J. Neural Syst. 15(4), 247–258 (2005)
Zhang, J., Li, G., Hu, M., Li, J., Luo, Z.: Recognition of hypoxia EEG with a preset confidence level based on EEG analysis. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN 2008), part of the IEEE World Congress on Computational Intelligence (WCCI 2008), pp.3005–3008. IEEE (2008)
Lambrou, A., Papadopoulos, H., Kyriacou, E., Pattichis, C.S., Pattichis, M.S., Gammerman, A., Nicolaides, A.: Evaluation of the risk of stroke with confidence predictions based on ultrasound carotid image analysis. Int. J. Artif. Intell. Tools 21(04), 1240016 (2012). [Online]. Available doi:10.1142/S0218213012400167
Shahmuradov, A., Solovyev, V.V., Gammerman, A.J.: Plant promoter prediction with confidence estimation. Nucleic Acids Res. 33(3), 1069–1076 (2005)
Papadopoulos, H., Vovk, V., Gammerman, A.: Conformal prediction with neural networks. In: Proceedings of the 19th IEEE International Conference on Tools with Artificial Intelligence (ICTAI’07), vol. 2, pp. 388–395. IEEE Computer Society (2007)
Papadopoulos, H., Gammerman, A., Vovk, V.: Reliable diagnosis of acute abdominal pain with conformal prediction. Eng. Intell. Syst. 17(2–3), 127–137. CRL Publishing (2009)
Lambrou, A., Papadopoulos, H., Gammerman, A.: Reliable confidence measures for medical diagnosis with evolutionary algorithms. IEEE Trans. Inf. Technol. Biomed. 15(1), 93–99 (2011). IEEE
Lambrou, A., Papadopoulos, H., Gammerman, A. (2009)
Abonyi, J., Feil, B.: Cluster Analysis for Data Mining and System Identification. Birkauser Verlag AG, Basel (2007)
Author information
Authors and Affiliations
Consortia
Corresponding author
Additional information
See the Appendix of F. Romanelli et al., Proceedings of the 24th IAEA Fusion Energy Conference 2012, San Diego, US
Rights and permissions
About this article
Cite this article
Murari, A., Vega, J., Mazon, D. et al. Preliminary numerical investigations of conformal predictors based on fuzzy logic classifiers. Ann Math Artif Intell 74, 155–180 (2015). https://doi.org/10.1007/s10472-014-9399-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10472-014-9399-5