Skip to main content
SpringerLink
Log in

Covariance Matrix Adaptation Evolution Strategy for Convolutional Neural Network in Text Classification

  • Conference paper
  • First Online:
Progress in Artificial Intelligence and Pattern Recognition (IWAIPR 2021)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 13055))

Abstract

Text classification has become relevant in recent years because of its usefulness in supporting different text mining solutions. Neural networks for this purpose have benefited from the creation of word embedding for learning semantics among words in a corpus. However, artificial neural network training by conventional methods present several theoretical and computational limitations. In this work, we develop a hybrid training method that combines gradient-based methods and Covariance Matrix Adaptation Evolution Strategy to train Convolutional Neural Network for the text classification task. For this, the training process is divided into two stages taking advantage of the speed of the gradient-based methods for learning the parameters of the convolutional filters and the application of the Covariance Matrix Adaptation Evolution Strategy for learning the weights of the fully connected layer. Our proposal was evaluated using a Spanish dataset for text classification, taken off the EcuRed Cuban Encyclopedia, divided into five classes. The proposed method increases the accuracy significantly of the convolutional network applied to the text classification.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Available at http://dcc.uchile.cl/~jperez/word-embeddings/glove-sbwc.i25.vec.gz.

  2. 2.

    https://www.ecured.cu/EcuRed:Enciclopedia_cubana.

  3. 3.

    https://www.sketchengine.eu/.

  4. 4.

    Dataset available at: https://github.com/ogtoledano/ecured_five_tags.

  5. 5.

    Source code at: https://github.com/ogtoledano/Text_Cat_Based_EDA.

References

  1. Hansen, N., Ostermeier, A.: Completely derandomized self-adaptation in evolution strategies. Evol. Comput. 9(2), 159–195 (2001)

    Article  Google Scholar 

  2. Hartmann, J., Huppertz, J., Schamp, C., Heitmann, M.: Comparing automated text classification methods. Int. J. Res. Mark. 36(1), 20–38 (2019)

    Article  Google Scholar 

  3. Indolia, S., Kumar, A., Mishra, S.P., Asopa, P.: Conceptual understanding of convolutional neural network- a deep learning approach. Proc. Comput. Sci. 132, 679–688 (2018)

    Article  Google Scholar 

  4. Kiefer, J., Wolfowitz, J.: Stochastic estimation of the maximum of a regression function. Ann. Math. Stat. 23(3), 462–466 (1952)

    Article  MathSciNet  Google Scholar 

  5. Kingma, D.P., Ba, J.L.: Adam: a method for stochastic optimization. In 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings, pp. 1–15 (2015)

    Google Scholar 

  6. Kowsari, K., Meimandi, K.J., Heidarysafa, M., Mendu, S., Barnes, L., Brown, D.: Text classification algorithms: a survey (2019)

    Google Scholar 

  7. Larrañaga, P., Lozano, J.A.: Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation, vol. 2. Springer, Heidelberg (2002). https://doi.org/10.1007/978-1-4615-1539-5

    Book  MATH  Google Scholar 

  8. Le, L.T., Nguyen, H., Dou, J., Zhou, J.: A comparative study of PSO-ANN, GA-ANN, ICA-ANN, and ABC-ANN in estimating the heating load of buildings’ energy efficiency for smart city planning. Appl. Sci. (Switzerland) 9(13), 2630 (2019)

    Google Scholar 

  9. Liang, Y., et al.: Fusion of heterogeneous attention mechanisms in multi-view convolutional neural network for text classification. Inf. Sci. 548, 295–312 (2021)

    Article  Google Scholar 

  10. Luan, Y., Lin, S.: Research on text classification based on CNN and LSTM. In: Proceedings of 2019 IEEE International Conference on Artificial Intelligence and Computer Applications, ICAICA 2019, pp. 352–355 (2019)

    Google Scholar 

  11. Madera, J., Dorronsoro, B.: Estimation of distribution algorithms. In: Alba, E., Martí, R. (eds.) Metaheuristic Procedures for Training Neural Networks, 1 edn., pp. 87–108. Springer, Boston (2006). https://doi.org/10.1007/0-387-33416-5_5. ISBN: 978-0-387-33415-8

  12. Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems, pp. 1–9 (2013)

    Google Scholar 

  13. Minaee, S., Kalchbrenner, N., Cambria, E., Nikzad, N., Chenaghlu, M., Gao, J.: Deep learning based text classification: a comprehensive review (2020)

    Google Scholar 

  14. Nair, V., Mohapatra, S.K., Malhotra, R.: A machine learning algorithm for product classification based on unstructured text description. Int. J. Eng. Res. Technol. 7(06), 404–407 (2018)

    Google Scholar 

  15. Ojha, V.K., Abraham, A., Snásel, V.: Metaheuristic design of feedforward neural networks : a review of two decades of research. Eng. Appl. Artif. Intell. 60, 97–116 (2017)

    Article  Google Scholar 

  16. Pennington, J., Socher, R., Manning, C.D.: GloVe: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543. Association for Computational Linguistics (2014)

    Google Scholar 

  17. Raunak, V., Metze, F.: Effective dimensionality reduction for word embeddings. In: Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019), , Florence, pp. 235–243. Association for Computational Linguistics (2019)

    Google Scholar 

  18. Reddy, T., Williams, R., Breazeal, C.: Text classification for AI education. In: Proceedings of the 52nd ACM Technical Symposium on Computer Science Education, SIGCSE 2021, pp. 1381. Association for Computing Machinery, New York (2021)

    Google Scholar 

  19. Rojas-Delgado, J., Trujillo-Rasúa, R., Bello, R.: A continuation approach for training Artificial Neural Networks with meta-heuristics. Pattern Recogn. Lett. 125, 373–380 (2019)

    Article  Google Scholar 

  20. Shen, H.: Towards a mathematical understanding of the difficulty in learning with feedforward neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 811–820 (2018)

    Google Scholar 

  21. Suyan, W., Entong, S., Binyang, L., Jiangrui, W.: TextCNN-based text classification for E-government. In: International Conference on Information Science and Control Engineering, ICISCE 2019, pp. 929–934 (2019)

    Google Scholar 

  22. Wang, R., Li, Z., Cao, J., Chen, T., Wang, L.: Convolutional recurrent neural networks for text classification. In: Proceedings of the International Joint Conference on Neural Networks, vol. 2019-July, no. 2018, pp. 1–6 (2019)

    Google Scholar 

  23. Wu, H., Liu, Y., Wang, J.: Review of text classification methods on deep learning. Comput. Mater. Continua 63(3), 1309–1321 (2020)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Universidad de las Ciencias Informáticas (UCI), La Habana, Cuba

    Orlando Grabiel Toledano-López & Héctor González

  2. Universidad de Camagüey, Camagüey, Cuba

    Julio Madera

  3. Universidad Tecnológica de La Habana “José Antonio Echeverría”, La Habana, Cuba

    Alfredo Simón Cuevas

Authors
  1. Orlando Grabiel Toledano-López
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Julio Madera
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Héctor González
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alfredo Simón Cuevas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Orlando Grabiel Toledano-López .

Editor information

Editors and Affiliations

  1. Universidad de las Ciencias Informáticas, La Habana, Cuba

    Yanio Hernández Heredia

  2. Universidad de las Ciencias Informáticas, La Habana, Cuba

    Vladimir Milián Núñez

  3. Universidad de las Ciencias Informáticas, La Habana, Cuba

    José Ruiz Shulcloper

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Toledano-López, O.G., Madera, J., González, H., Cuevas, A.S. (2021). Covariance Matrix Adaptation Evolution Strategy for Convolutional Neural Network in Text Classification. In: Hernández Heredia, Y., Milián Núñez, V., Ruiz Shulcloper, J. (eds) Progress in Artificial Intelligence and Pattern Recognition. IWAIPR 2021. Lecture Notes in Computer Science(), vol 13055. Springer, Cham. https://doi.org/10.1007/978-3-030-89691-1_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-89691-1_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-89690-4

  • Online ISBN: 978-3-030-89691-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation