Skip to main content
Springer Nature Link
Log in

RIONIDA: A Novel Algorithm for Imbalanced Data Combining Instance-Based Learning and Rule Induction

  • Conference paper
  • First Online:
Rough Sets (IJCRS 2024)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14839))

Included in the following conference series:

  • 201 Accesses

Abstract

The article presents the RIONIDA learning algorithm based on combination of two widely-used empirical approaches: rule induction and instance-based learning for imbalanced data classification. The algorithm is a substantial extension of the well-known RIONA algorithm developed for balanced data.

RIONIDA is relatively fast and significantly outperforms the state-of-the-art algorithms analysed in the paper.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    \(\rho _a\) is used to define for a given \(v \in V_a\) a neighbourhood of similar values for v. RIONIDA as RIONA learns (as default setting) SVDM metrics [10] for symbolic attributes; and for numerical attributes uses Euclidean metric on \(\mathbb {R}\).

  2. 2.

    For \(s=1\), this definition is equivalent to conditions used in RIONA. For \(s=0\) we have the rule covering only the test example and the training examples identical with the test example for all numerical attributes and distanced by 0 for all symbolic attributes. For \(s<0\), the premise of this rule is always false (formally speaking, not satisfied by any example) what relates to elimination of consistency checking and in consequence to working as the kNN algorithm. The parameter s such that \(0<s<1\) defines the scaling of the satisfiability area of the rule.

  3. 3.

    We have not used AUC measure because of (i) the criticism about it (see e.g. [22, 32]); (ii) BRACID, one of the important learning algorithms that we wanted to compare with, does not return probabilities for the two decision classes (only the deterministic decision is returned).

  4. 4.

    Only mammography data set is not publicly available and was supported by Nitesh Chawla [6].

  5. 5.

    In fact, we describe in detail only the simplified version of experiments (see also Subsect. 7.3).

  6. 6.

    The only exception was for F-measure and kNN: RIONIDA achieved better average rank (2.85) than kNN (4.45) but the difference between RIONIDA and kNN is not statistically significant for scores computed in such a way (adjusted p-value in Finner statistical test for kNN was 0.09). However, one should also bear in mind that real meta-learning algorithm using kNN would obtain worse results than we took into comparison (as it was mentioned, we took maximal values of possible scores).

References

  1. Rseslib 3: Rough set and machine learning open source in Java. http://rseslib.mimuw.edu.pl

  2. Aha, D.W. (ed.): Lazy Learning, 1st edn. Springer, Dordrecht (1997). https://doi.org/10.1007/978-94-017-2053-3

  3. Aha, D.W., Kibler, D., Albert, M.K.: Instance-based learning algorithms. Mach. Learn. 6(1), 37–66 (1991). https://doi.org/10.1023/A:1022689900470

    Article  Google Scholar 

  4. Almasri, A., Celebi, E., Alkhawaldeh, R.S.: EMT: ensemble meta-based tree model for predicting student performance. Sci. Program. 2019, 1–13 (2019). https://doi.org/10.1155/2019/3610248. Article No. 3610248

  5. Batista, G.E.A.P.A., Prati, R.C., Monard, M.C.: A study of the behavior of several methods for balancing machine learning training data. ACM SIGKDD Explor. Newsl. 6(1), 20–29 (2004). https://doi.org/10.1145/1007730.1007735

  6. Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: SMOTE: synthetic minority over-sampling technique. J. Artif. Intell. Res. 16, 321–357 (2002). https://doi.org/10.1613/jair.953

    Article  Google Scholar 

  7. Cormen, T.H., Leiserson, C.E., Rivest, R.L., Stein, C.: Introduction to Algorithms, 2nd edn. The MIT Press and McGraw-Hill Book Company, Cambridge (2001)

    Google Scholar 

  8. Demšar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)

    MathSciNet  Google Scholar 

  9. Dey, N., Borah, S., Babo, R., Ashour, A.S. (eds.): Social Network Analytics: Computational Research Methods and Techniques, 1st edn. Academic Press, London (2019). https://doi.org/10.1016/C2017-0-02844-6

    Book  Google Scholar 

  10. Domingos, P.: Unifying instance-based and rule-based induction. Mach. Learn. 24(2), 141–168 (1996). https://doi.org/10.1007/BF00058656

    Article  MathSciNet  Google Scholar 

  11. Fernández, A., García, S., Galar, M., Prati, R.C., Krawczyk, B., Herrera, F.: Learning from Imbalanced Data Sets, 1st edn. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-98074-4

    Book  Google Scholar 

  12. Frank, E., Witten, I.H.: Generating accurate rule sets without global optimization. In: Proceedings of the 15th International Conference on Machine Learning (ICML 1998), pp. 144–151. Morgan Kaufmann, San Francisco (1998)

    Google Scholar 

  13. Friedman, M.: The use of ranks to avoid the assumption of normality implicit in the analysis of variance. J. Am. Stat. Assoc. 32(200), 675–701 (1937). https://doi.org/10.1080/01621459.1937.10503522

    Article  Google Scholar 

  14. Fürnkranz, J., Widmer, G.: Incremental reduced error pruning. In: Proceedings of the 11th International Conference on Machine Learning (ICML 1994), pp. 70–77. Morgan Kaufmann, San Francisco (1994). https://doi.org/10.1016/B978-1-55860-335-6.50017-9

  15. Góra, G., Skowron, A., Wojna, A.: Explainability in RIONA algorithm combining rule induction and instance-based learning. In: Ganzha, M., Maciaszek, L.A., Paprzycki, M., Ślȩzak, D. (eds.) Proceedings of the 18th Conference on Computer Science and Intelligence Systems, FedCSIS 2023, Warsaw, Poland, 17–20 September 2023. Annals of Computer Science and Information Systems, vol. 31, pp. 485–496. IEEE (2023). https://annals-csis.org/proceedings/2023/

  16. Góra, G.: Combining instance-based learning and rule-based methods for imbalanced data. Ph.D. thesis, University of Warsaw, Warsaw (2022). https://www.mimuw.edu.pl/sites/default/files/gora_grzegorz_rozprawa_doktorska.pdf

  17. Góra, G., Skowron, A.: On kNN class weights for optimising G-mean and F1-score. In: Campagner, A., Urs Lenz, O., Xia, S., Ślęzak, D., Wąs, J., Yao, J. (eds.) IJCRS 2023. LNCS, vol. 14481, pp. 414–430. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-50959-9_29

    Chapter  Google Scholar 

  18. Góra, G., Wojna, A.: RIONA: a classifier combining rule induction and K-NN method with automated selection of optimal neighbourhood. In: Elomaa, T., Mannila, H., Toivonen, H. (eds.) ECML 2002. LNCS, vol. 2430, pp. 111–123. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-36755-1_10

    Chapter  Google Scholar 

  19. Grama, L., Rusu, C.: Choosing an accurate number of mel frequency cepstral coefficients for audio classification purpose. In: Proceedings of the 10th International Symposium on Image and Signal Processing and Analysis (ISPA 2017), pp. 225–230 (2017). https://doi.org/10.1109/ISPA.2017.8073600

  20. Grama, L., Rusu, C.: Adding audio capabilities to TIAGo service robot. In: 2018 International Symposium on Electronics and Telecommunications (ISETC), pp. 1–4 (2018). https://doi.org/10.1109/ISETC.2018.8583897

  21. Grzymala-Busse, J.W., Goodwin, L.K., Grzymala-Busse, W.J., Zheng, X.: An approach to imbalanced data sets based on changing rule strength. In: Pal, S.K., Polkowski, L., Skowron, A. (eds.) Rough-Neural Computing: Techniques for Computing with Words. Cognitive Technologies, pp. 543–553. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  22. Hand, D.J.: Measuring classifier performance: a coherent alternative to the area under the ROC curve. Mach. Learn. 77(1), 103–123 (2009). https://doi.org/10.1007/s10994-009-5119-5

    Article  Google Scholar 

  23. He, H., Ma, Y.: Imbalanced Learning: Foundations, Algorithms, and Applications, 1st edn. Wiley-IEEE Press, Piscataway (2013)

    Book  Google Scholar 

  24. Kaur, H., Pannu, H.S., Malhi, A.K.: A systematic review on imbalanced data challenges in machine learning: applications and solutions. ACM Comput. Surv. 52(4), 1–36 (2019). https://doi.org/10.1145/3343440

    Article  Google Scholar 

  25. Li, J., Dong, G., Ramamohanarao, K., Wong, L.: DeEPs: a new instance-based lazy discovery and classification system. Mach. Learn. 54(2), 99–124 (2004). https://doi.org/10.1023/B:MACH.0000011804.08528.7d

    Article  Google Scholar 

  26. Lichman, M.: UCI machine learning repository (2013). http://archive.ics.uci.edu/ml

  27. Napierała, K.: Improving rule classifiers for imbalanced data. Ph.D. thesis, Poznań University of Technology, Poznań (2012)

    Google Scholar 

  28. Napierała, K., Stefanowski, J.: Types of minority class examples and their influence on learning classifiers from imbalanced data. J. Intell. Inf. Syst. 46(3), 563–597 (2016). https://doi.org/10.1007/s10844-015-0368-1

    Article  Google Scholar 

  29. de Oliveira Almeida, R., Valente, G.T.: Predicting metabolic pathways of plant enzymes without using sequence similarity: Models from machine learning. The Plant Genome 13(3), e20043 (2020). https://doi.org/10.1002/tpg2.20043

    Article  Google Scholar 

  30. Pawlak, Z., Skowron, A.: Rough sets: some extensions. Inf. Sci. 177(28–40), 1 (2007). https://doi.org/10.1016/j.ins.2006.06.006

    Article  MathSciNet  Google Scholar 

  31. Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo (1993)

    Google Scholar 

  32. Raeder, T., Forman, G., Chawla, N.V.: Learning from imbalanced data: evaluation matters. In: Holmes, D.E., Jain, L.C. (eds.) Data Mining: Foundations and Intelligent Paradigms. Intelligent Systems Reference Library, vol. 23, pp. 315–331. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-23166-7_12

    Chapter  Google Scholar 

  33. Russell, S.J., Norvig, P.: Artificial Intelligence: A Modern Approach, 4th edn. Pearson Education, Hoboken (2021)

    Google Scholar 

  34. Rusu, C., Grama, L.: Recent developments in acoustical signal classification for monitoring. In: 2017 5th International Symposium on Electrical and Electronics Engineering (ISEEE), pp. 1–10 (2017). https://doi.org/10.1109/ISEEE.2017.8170705

  35. Salzberg, S.: A nearest hyperrectangle learning method. Mach. Learn. 6, 251–276 (1991)

    Article  Google Scholar 

  36. Stefanowski, J.: Rough set based rule induction techniques for classification problems. In: Proceedings of 6th European Congress on Intelligent Techniques & Soft Computing (EUFIT 1998),D vol. 1, pp. 109–113. Verlag Mainz, Aachen (1998)

    Google Scholar 

  37. Vapnik, V.N.: Statistical Learning Theory, 1st edn. Wiley-Interscience, New York (1998)

    Google Scholar 

  38. Wettschereck, D., Dietterich, T.G.: An experimental comparison of the nearest-neighbor and nearest-hyperrectangle algorithms. Mach. Learn. 19, 5–27 (1995)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Warsaw, Stefana Banacha 2, 02-097, Warsaw, Poland

    Grzegorz Góra

  2. Systems Research Institute PAS, Newelska 6, 01-447, Warsaw, Poland

    Andrzej Skowron

Authors
  1. Grzegorz Góra
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrzej Skowron
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Grzegorz Góra .

Editor information

Editors and Affiliations

  1. Saint Mary's University, Halifax, NS, Canada

    Mengjun Hu

  2. Ghent University, Ghent, Belgium

    Chris Cornelis

  3. California State University, San Bernardino, CA, USA

    Yan Zhang

  4. Saint Mary's University, Halifax, NS, Canada

    Pawan Lingras

  5. University of Warsaw, Warsaw, Poland

    Dominik Ślęzak

  6. University of Regina, Regina, SK, Canada

    JingTao Yao

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Góra, G., Skowron, A. (2024). RIONIDA: A Novel Algorithm for Imbalanced Data Combining Instance-Based Learning and Rule Induction. In: Hu, M., Cornelis, C., Zhang, Y., Lingras, P., Ślęzak, D., Yao, J. (eds) Rough Sets. IJCRS 2024. Lecture Notes in Computer Science(), vol 14839. Springer, Cham. https://doi.org/10.1007/978-3-031-65665-1_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-65665-1_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-65664-4

  • Online ISBN: 978-3-031-65665-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics