Skip to main content
SpringerLink
Log in

Learning cluster-based classification systems with ant colony optimization algorithms

  • Published:
Swarm Intelligence Aims and scope Submit manuscript

Abstract

Classification is a data mining task the goal of which is to learn a model, from a training dataset, that can predict the class of a new data instance, while clustering aims to discover natural instance-groupings within a given dataset. Learning cluster-based classification systems involves partitioning a training set into data subsets (clusters) and building a local classification model for each data cluster. The class of a new instance is predicted by first assigning the instance to its nearest cluster and then using that cluster’s local classification model to predict the instance’s class. In this paper, we present an ant colony optimization (ACO) approach to building cluster-based classification systems. Our ACO approach optimizes the number of clusters, the positioning of the clusters, and the choice of classification algorithm to use as the local classifier for each cluster. We also present an ensemble approach that allows the system to decide on the class of a given instance by considering the predictions of all local classifiers, employing a weighted voting mechanism based on the fuzzy degree of membership in each cluster. Our experimental evaluation employs five widely used classification algorithms: naïve Bayes, nearest neighbour, Ripper, C4.5, and support vector machines, and results are reported on a suite of 54 popular UCI benchmark datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  • Abdallah, Z. S., Gaber, M., Srinivasan, B., & Krishnaswamy, S. (2012). CBARS: Cluster based classification for activity recognition systems. In A. Hassanien, A. Salem, R. Ramadan & T. Kim (Eds.), Advanced machine learning technologies and applications (Proceedings AMLTA-2012), Communications in Computer and Information Science (CCIS) (Vol. 322, pp. 82–91). Berlin: Springer.

  • Abdelbar, A. M., & Salama, K. M. (2016). Clustering with the ACO\({_{\mathbb{R}}}\) algorithm. In: M. Dorigo, M. Birattari, X. Li, M. López-Ibáñez, K. Ohkura, C. Pinciroli & T. Stützle (Eds.), Swarm intelligence (Proceedings ANTS-2016), Lecture Notes in Computer Science (Vol. 9882, pp. 291–292). Berlin: Springer.

  • Arthur, D., & Vassilvitskii, S. (2007). k-means\(++\): The advantages of careful seeding. In Proceedings ACM-SIAM Symposium on Discrete Algorithms (SODA-2007) (pp. 1027–1035). Philadelphia, PA: Society for Industrial and Applied Mathematics.

  • Ashok, L., & Messinger, D. (2012). A spectral image clustering algorithm based on ant colony optimization. In Proceedings SPIE 8390, Algorithms and Technologies for Multispectral, Hyperspectral, and Ultraspectral Imagery XVIII (p. 83901P). SPIE Press.

  • Asuncion, A., & Newman, D. (2007). UCI Machine Learning Repository. http://www.ics.uci.edu/~mlearn/MLRepository.html.

  • Bezdek, J., Ehrlich, R., & Full, W. (1984). FCM: The fuzzy \(c\)-means clustering algorithm. Computers & Geosciences, 10(2–3), 191–203.

    Article  Google Scholar 

  • Bishop, C. M. (2007). Pattern recognition and machine learning. Berlin: Springer.

    MATH  Google Scholar 

  • Breiman, L. (1996). Bagging predictors. Machine Learning, 24(2), 123–140.

    MATH  Google Scholar 

  • Dempster, A., Laird, N., & Rubin, D. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Series B (Methodological), 39(1), 1–8.

    MathSciNet  MATH  Google Scholar 

  • Dorigo, M., & Stützle, T. (2004). Ant colony optimization. Cambridge: MIT Press.

    MATH  Google Scholar 

  • Franca, F., Coelho, G., & Zuben, F. (2008). bicACO: An ant colony inspired biclustering algorithm. In M. Dorigo, M. Birattari, C. Blum, M. Clerc, T. Stützle & A. Winfield (Eds.), Ant colony optimization and swarm intelligence (Proceedings ANTS-2008), Lecture Notes in Computer Science (Vol. 5217, pp. 401–402). Berlin: Springer.

  • Freund, Y., & Schapire, R. (1997). Decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1), 119–139.

    Article  MathSciNet  MATH  Google Scholar 

  • Gan, G., Ma, C., & Wu, J. (2007). Data clustering: Theory, algorithms, and applications. Philadelphia, PA: SIAM Press.

    Book  MATH  Google Scholar 

  • Han, J., & Kamber, M. (2000). Data mining: Concepts and techniques. San Francisco, CA: Morgan Kaufmann.

    MATH  Google Scholar 

  • Jafar, M., & Sivakumar, R. (2010). Ant-based clustering algorithms: A brief survey. International Journal of Computer Theory and Engineering, 2, 787–796.

    Article  Google Scholar 

  • Kao, Y., & Cheng, K. (2016). An ACO-based clustering algorithm. In: M. Dorigo, L. M. Gambardella, M. Birattari, A. Martinoli, R. Poli & T. Stützle (Eds.), Swarm intelligence (Proceedings ANTS-2006), Lecture Notes in Computer Science (Vol. 4150, pp 210–221). Berlin: Springer.

  • Kaufman, L., & Rousseeuw, P. (1987). Clustering by means of medoids. In Y. Dodge (Ed.), Statistical data analysis based on the L1 norm and related methods (pp. 405–416). Amsterdam: North-Holland.

    Google Scholar 

  • Liao, T., Socha, K., Montes de Oca, M., Stützle, T., & Dorigo, M. (2014). Ant colony optimization for mixed-variable optimization problems. IEEE Transactions on Evolutionary Computation, 18(4), 503–518.

    Article  Google Scholar 

  • Liu, X. Y., & Fu, H. (2010). An effective clustering algorithm with ant colony. Journal of Computers, 5, 598–605.

    Google Scholar 

  • Liu, Y., Li, Z., Xiong, H., Gao, X., & Wu, J. (2010). Understanding of internal clustering validation measures. In Proceedings IEEE conference on data mining (pp. 911–916). IEEE Press.

  • MacQueen, J. (1967). Some methods of classification and analysis of multivariate observations. In Proceedings fifth Berkeley symposium on mathematical statistics and probability (Vol. 1, pp. 281–297). University of California Press.

  • Martens, D., De Backer, M., Haesen, R., Vanthienen, J., Snoeck, M., & Baesens, B. (2007). Classification with ant colony optimization. IEEE Transactions on Evolutionary Computation, 11(5), 651–665.

    Article  Google Scholar 

  • Mehrotra, K., Ozgencil, N., & McCracken, N. (2007). Squeezing the last drop: Cluster-based classification algorithm. Statistics & Probability Letters, 77, 1288–1299.

    Article  MathSciNet  MATH  Google Scholar 

  • Menendez, H., Otero, F., & Camacho, D.(2014a). MACOC: A medoid-based ACO clustering algorithm. In: M. Dorigo, M. Birattari, S. Garnier, H. Hamann, M. Montes de Oca, C. Solnon & T. Stützle (Eds.,) Swarm Intelligence (Proceedings ANTS-2014), Lecture Notes in Computer Science (Vol. 8667, pp. 122–133). Berlin: Springer.

  • Menendez, H., Otero, F., & Camacho, D. (2014b). SACOC: A spectral-based ACO clustering algorithm. In Proceedings eighth international symposium on intelligent distributed computing, Studies in Computational Intelligence (Vol. 570, pp. 185–194). Berlin: Springer

  • Otero, F. E., Freitas, A. A., & Johnson, C. G. (2009). Handling continuous attributes in ant colony classification algorithms. In IEEE Symposium on Computational Intelligence in Data Mining (CIDM 2009) (pp 225–231). IEEE Press.

  • Otero, F. E., Freitas, A. A., & Johnson, C. G. (2012). Inducing decision trees with an ant colony optimization algorithm. Applied Soft Computing, 12(11), 3615–3626.

    Article  Google Scholar 

  • Otero, F. E., Freitas, A. A., & Johnson, C. (2013). A new sequential covering strategy for inducing classification rules with ant colony algorithms. IEEE Transactions on Evolutionary Computation, 17(1), 64–74.

    Article  Google Scholar 

  • Ozgencil, N. E. (2007). Cluster based classification for semantic role labeling. Ph.D. thesis, Syracuse University, Syracuse, NY, USA.

  • Parpinelli, R. S., Lopes, H. S., & Freitas, A. A. (2002). Data mining with an ant colony optimization algorithm. IEEE Transactions on Evolutionary Computation, 6(4), 321–332.

    Article  MATH  Google Scholar 

  • Rokach, L. (2010). Ensemble-based classifiers. Artificial Intelligence Review, 33(1–2), 1–39.

    Article  Google Scholar 

  • Salama, K. M., & Abdelbar, A. M. (2015). Learning neural network structures with ant colony algorithms. Swarm Intelligence, 9(4), 229–265.

    Article  Google Scholar 

  • Salama, K. M., & Abdelbar, A. M. (2016). Using ant colony optimization to build cluster-based classification systems. In M. Dorigo, M. Birattari, X. Li, M. López-Ibáñez, K. Ohkura, C. Pinciroli & T. Stützle (Eds.), Swarm Intelligence (Proceedings ANTS-2016), Lecture Notes in Computer Science (Vol. 9882, pp. 210–221). Berlin: Springer.

  • Salama, K. M., & Freitas, A. A. (2013a). Clustering-based Bayesian multi-net classifier construction with ant colony optimization. In IEEE Congress on Evolutionary Computation (IEEE CEC) (pp. 3079–3086). IEEE Press.

  • Salama, K. M., & Freitas, A. A. (2013b). Learning Bayesian network classifiers using ant colony optimization. Swarm Intelligence, 7(2–3), 229–254.

    Article  Google Scholar 

  • Salama, K. M., & Freitas, A. A. (2014a). ABC-Miner\(+\): Constructing Markov blanket classifiers with ant colony algorithms. Memetic Computing, 6(3), 183–206.

    Article  Google Scholar 

  • Salama, K. M., & Freitas, A. A. (2014b). Classification with cluster-based Bayesian multi-nets using ant colony optimization. Swarm and Evolutionary Computation, 18, 54–70.

    Article  Google Scholar 

  • Salama, K. M., & Freitas, A. A. (2015). Ant colony algorithms for constructing Bayesian multi-net classifiers. Intelligent Data Analysis, 19(2), 233–257.

    Google Scholar 

  • Salama, K. M., & Otero, F. E. (2014). Learning multi-tree classification models with ant colony optimization. In Proceedings sixth international conference on evolutionary computation theory and applications (ECTA-2014), Lecture Notes in Computer Science (pp. 38–48). Berlin: Springer.

  • Salama, K. M., Abdelbar, A. M., & Freitas, A. A. (2011). Multiple pheromone types and other extensions to the Ant-Miner classification rule discovery algorithm. Swarm Intelligence, 5(3–4), 149–182.

    Article  Google Scholar 

  • Salama, K. M., Abdelbar, A. M., & Anwar, I. M. (2016). Data reduction for classification with ant colony optimization. Intelligent Data Analysis, 20(5), 1021–1059.

    Article  Google Scholar 

  • Salama, K. M., Abdelbar, A. M., Helal, A. Z., & Freitas, A. A. (2017). Instance-based classification with ant colony optimization. Intelligent Data Analysis (to appear).

  • Schapire, R. (1990). The strength of weak learnability. Machine Learning, 5(2), 197–227.

    Google Scholar 

  • Shelokar, P. S., Jayaraman, V. K., & Kulkarni, B. D. (2004). An ant colony approach for clustering. Analytica Chimica Acta, 509(2), 187–195.

    Article  Google Scholar 

  • Socha, K., & Blum, C. (2007). An ant colony optimization algorithm for continuous optimization: Application to feed-forward neural network training. Neural Computing & Applications, 16, 235–247.

    Article  Google Scholar 

  • Socha, K., & Dorigo, M. (2008). Ant colony optimization for continuous domains. European Journal of Operational Research, 185, 1155–1173.

    Article  MathSciNet  MATH  Google Scholar 

  • Tan, P. N., Steinbach, M., & Kumar, V. (2005). Introduction to data mining. Reading: Addison Wesley.

    Google Scholar 

  • Witten, I. H., & Frank, E. (2010). Data mining: Practical machine learning tools and techniques. San Francisco, CA: Morgan Kaufmann.

    MATH  Google Scholar 

  • Woniak, M., Graña, M., & Corchado, E. (2014). A survey of multiple classifier systems as hybrid systems. Information Fusion, 16, 3–17.

    Article  Google Scholar 

  • Xu, R., & Wunsch, D. (2009). Clustering. Piscataway, NJ: IEEE Press.

    Google Scholar 

  • Yen, S. J., & Lee, Y. S. (2009). Cluster-based under-sampling approaches for imbalanced data distributions. Expert Systems with Applications, 36, 5718–5727.

    Article  Google Scholar 

  • Zhang, X., & Xiao, W. (2012). Clustering based two-stage text classification requiring minimal training data. Computer Science and Information Systems, 9(4), 1627–1643.

    Article  Google Scholar 

Download references

Acknowledgements

The partial support of a grant from the Brandon University Research Council (BURC) is gratefully acknowledged. The authors would like to thank the anonymous reviewers and the guest editor for their insightful comments which substantially improved the paper.

Author information

Authors and Affiliations

  1. School of Computing, University of Kent, Canterbury, UK

    Khalid M. Salama

  2. Department of Mathematics and Computer Science, Brandon University, Brandon, MB, Canada

    Ashraf M. Abdelbar

Authors
  1. Khalid M. Salama
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ashraf M. Abdelbar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ashraf M. Abdelbar.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Salama, K.M., Abdelbar, A.M. Learning cluster-based classification systems with ant colony optimization algorithms. Swarm Intell 11, 211–242 (2017). https://doi.org/10.1007/s11721-017-0138-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11721-017-0138-5

Keywords

Search

Navigation