Skip to main content
SpringerLink
Log in
Book cover

International Conference on Computational Science and Its Applications

ICCSA 2012: Computational Science and Its Applications – ICCSA 2012 pp 618–630Cite as

Evaluation of Normalization Techniques in Text Classification for Portuguese

  • Conference paper

  • 2384 Accesses

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 7335))

Abstract

Text classification is an important task of Artificial Intelligence. Normally, this task uses large textual datasets whose representation is feasible because of normalization and selection techniques. In the literature, we can find three normalization techniques: stemming, lemmatization, and nominalization. Nevertheless, it is difficult to choose the most suitable technique for the text classification task. In this paper, we investigate this question experimentally by applying five different classifiers to four textual datasets in the Portuguese language. Additionally, the classification results are evaluated using unigrams, bigrams, and the combination of unigrams and bigrams. The results indicate that, in general, the number of terms obtained by each of the cases and the comprehensibility required in the results of the classification can be used as criteria to define the most suitable technique for the text classification task.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Alvares, R.V., Garcia, A.C.B., Ferraz, I.: STEMBR: A Stemming Algorithm for the Brazilian Portuguese Language. In: Bento, C., Cardoso, A., Dias, G. (eds.) EPIA 2005. LNCS (LNAI), vol. 3808, pp. 693–701. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  2. Arampatzis, A., van der Weide, T., Koster, C., van Bommel, P.: Linguistically-motivated Information Retrieval, pp. 201–222. Marcel Dekker, NY (2000)

    Google Scholar 

  3. Aranha, C.N.: Uma Abordagem de Pré-Processamento Automático para Mineração de Textos em Português: sob o Enfoque da Inteligência Computacional. PhD thesis, Departamento de Engenharia Elétrica - PUC - Rio de Janeiro (2007)

    Google Scholar 

  4. Bekkerman, R., Allan, J.: Using bigrams in text categorization. Technical Report IR-408, Center of Intelligent Information Retrieval, UMass Amherst (2004)

    Google Scholar 

  5. Brill, E.: Transformation-based error-driven learning of natural language: A case study in part of speech tagging. Computational Linguistics, 543–565 (1995)

    Google Scholar 

  6. Conrado, M.S.: O efeito do uso de diferentes formas de geração de termos na compreensibilidade e representatividade dos termos em coleções textuais na Língua Portuguesa. Master’s thesis, Instituto de Ciências Matemáticas e de Computação - USP, São Carlos, SP (2009)

    Google Scholar 

  7. Conrado, M.S., Marcacini, R.M., Moura, M.F., Rezende, S.O.: O efeito do uso de diferentes formas de geração de termos na compreensibilidade e representatividade dos termos em coleções textuais na Língua Portuguesa. In: Proceedings of II Web and Text Intelligence - 7th Brazilian Symposium in Information and Human Language Technology, São Carlos, SP (2009)

    Google Scholar 

  8. das Nunes, M.G.V.: The design of a lexicon for brazilian portuguese: Lessons learned and perspectives. In: Proceedings of the II Workshop on Computational Processing of Written and Spoken Portuguese, Curitiba, pp. 61–70 (1996)

    Google Scholar 

  9. Demšar, J.: Statistical comparison of classifiers over multiple data sets. Journal of Machine Learning Research 7(1), 1–30 (2006)

    MATH  Google Scholar 

  10. Ebecken, N.F.F., Lopes, M.C.S., de Aragão, M.C.: Mineração de Textos. In: Rezende, S.O. (ed.) Sistemas Inteligentes: Fundamentos e Aplicações, 1st edn., Manole, ch. 13, pp. 337–364 (2003)

    Google Scholar 

  11. Gonzalez, M. A. I.: Termos e Relacionamentos em Evidência na Recuperação de Informação. PhD thesis, Instituto de Informática - UFRGS, Porto Alegre (2005)

    Google Scholar 

  12. Gonzalez, M.A.I., de Lima, V.L.S., de Lima, J.V.: Tools for Nominalization: An Alternative for Lexical Normalization. In: Vieira, R., Quaresma, P., Nunes, M.d.G.V., Mamede, N.J., Oliveira, C., Dias, M.C. (eds.) PROPOR 2006. LNCS (LNAI), vol. 3960, pp. 100–109. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  13. Braga, Í.A., Monard, M.C., Matsubara, E.T.: Combining unigrams and bigrams in semi-supervised text classification. In: 14th Portuguese Conference on Artificial Intelligence - New Trends in Artificial Intelligence, Aveiro, Portugal, pp. 489–500 (2009)

    Google Scholar 

  14. Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The WEKA data mining software: An update. In: Explorations of Special Interest Group on Knowledge Discovery and Data Mining, vol. 11, pp. 10–18 (2009)

    Google Scholar 

  15. Keerthi, S.S., Shevade, S.K., Bhattacharyya, C., Murthy, K.R.K.: Improvements to platt’s smo algorithm for svm classifier design. Neural Comput. 13(3), 637–649 (2001)

    Article  MATH  Google Scholar 

  16. Manning, C.D., Raghavan, P., Schütze, H.: Introduction to Information Retrieval. Cambridge University Press (2008)

    Google Scholar 

  17. Manning, C.D., Raghavan, P., Schütze, H.: Language models for information retrieval. In: An Introduction to Information Retrieval, ch. 12. Cambridge University Press (2008)

    Google Scholar 

  18. Maziero, E.G., del Rosario Castro Jorge, M.L., Pardo, T.A.S.: Identifying multidocument relations. In: Proceedings of 7th International Workshop on Natural Language Processing and Cognitive Science, Funchal/Madeira, Portugal, vol. 1, pp. 60–69 (2010)

    Google Scholar 

  19. Mccallum, A., Nigam, K.: A comparison of event models for naive bayes text classification. In: AAAI Magazine - Workshop on ’Learning for Text Categorization, pp. 1–8 (1998)

    Google Scholar 

  20. Miner, G., Elder, J., Hill, T., Nisbet, R., Delen, D.: Practical Text Mining and Statistical Analysis for Non-structured Text Data Applications. Elsevier Science (2012)

    Google Scholar 

  21. Mitra, V., Wang, C.-J., Banerjee, S.: Text classification: A least square support vector machine approach. Appl. Soft Comput. 7(3), 908–914 (2007)

    Article  Google Scholar 

  22. Nuipian, V., Meesad, P., Boonrawd, P.: Improve abstract data with feature selection for classification techniques. Advanced Materials Research 403-408, 3699–3703 (2011)

    Article  Google Scholar 

  23. Orengo, V.M., Huyck, C.: A stemming algorithm for portuguese language. In: Proceedings of Eigth Symposium on String Processing and Information Retrieval, Chile, pp. 186–193 (2001)

    Google Scholar 

  24. Quinlan, J.R.: C4.5: programs for machine learning. Morgan Kaufmann Publishers Inc., San Francisco (1993)

    Google Scholar 

  25. Ratnaparkhi, A.: A maximum entropy model for part-of-speech tagging. In: Proceedings of the Empirical Methods in Natural Language Processing Conference, pp. 491–497. University of Pennsylvania (1996)

    Google Scholar 

  26. Read, J., Webster, J., Fang, A.C.: In: Proceedings of the 24th Pacific Asia Conference on Language, Information and Computation, Sendai, Japan

    Google Scholar 

  27. Řehůřek, R., Sojka, P.: Automated Classification and Categorization of Mathematical Knowledge. In: Autexier, S., Campbell, J., Rubio, J., Sorge, V., Suzuki, M., Wiedijk, F. (eds.) AISC 2008, Calculemus 2008, and MKM 2008. LNCS (LNAI), vol. 5144, pp. 543–557. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  28. Schmid, H.: Probabilistic part-of-speech tagging using decision trees. In: Proceedings of International Conference on New Methods in Language Processing, pp. 44–49 (1994)

    Google Scholar 

  29. Sebastiani, F.: Machine learning in automated text categorization. ACM Computing Surveys 34(1), 1–47 (2002)

    Article  Google Scholar 

  30. Silic, A., Chauchat, J.-H., Basic, B.-D., Morin, A.: N-grams and morphological normalization in text classification: A comparison on a croatian-english parallel corpus. In: Neves, J., Santos, M.-F., Machado, J. (eds.) 13th Portuguese Conference on Artificial Intelligence, Guimaraes, Portugal

    Google Scholar 

  31. Snedecor, G.W., Cochran, W.G.: Statistical Methods, 6th edn. Iowa State University Press, Ames (1967)

    Google Scholar 

  32. Soares, M.V., Prati, R.C., Monard, M.C.: PreTexT II: Descrição da reestruturação da ferramenta de pré-processamento de textos. Technical Report 333, Instituto de Ciências Matemáticas e de Computação - USP, São Carlos, SP (2008)

    Google Scholar 

  33. Su, J., Zhang, H., Ling, C.X., Matwin, S.: Discriminative parameter learning for bayesian networks. In: Proceedings of the 25th International Conference on Machine Learning, pp. 1016–1023. ACM, New York (2008)

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Sao Paulo University (USP), P.O. Box 668, 13561-970, Sao Carlos, SP, Brazil

    Merley da Silva Conrado & Solange Oliveira Rezende

  2. Pontifical Catholic University of Peru (PUCP), P.O. Box 1761, Lima, 32, Peru

    Víctor Antonio Laguna Gutiérrez

Authors
  1. Merley da Silva Conrado
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Víctor Antonio Laguna Gutiérrez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Solange Oliveira Rezende
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Laboratory of Urban and Territorial Systems, University of Basilicata, 10, Viale dell’Ateneo Lucano, 85100, Potenza, Italy

    Beniamino Murgante

  2. Department of Mathematics and Computer Science, University of Perugia, Via Vanvitelli 1, 06123, Perugia, Italy

    Osvaldo Gervasi

  3. Department of Cyber Security Science, Federal University of Technology, Gidan Kwano Campus, Minna, Nigeria

    Sanjay Misra

  4. Faculty of Engineering, Department of Electronics Engineering and Telecommunications, State University of Rio de Janeiro, Rua Sao Francisco Xavier, 524, 50. andar, sala 5145-F, Maracana, 20.550-013, Rio de Janeiro, RJ, Brazil

    Nadia Nedjah

  5. Department of Production and Systems, University of Minho, Campus de Gualtar, 4710-057, Braga, Portugal

    Ana Maria A. C. Rocha

  6. School of Business Systems, Monash University, 3800, Clayton, VIC, Australia

    David Taniar

  7. Department of Intelligent Informatics, Kyushu Sangyo University, 2-3-1 Matsukadai, Higashi-ku, 813-8503, Fukuoka, Japan

    Bernady O. Apduhan

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

da Silva Conrado, M., Laguna Gutiérrez, V.A., Rezende, S.O. (2012). Evaluation of Normalization Techniques in Text Classification for Portuguese. In: Murgante, B., et al. Computational Science and Its Applications – ICCSA 2012. ICCSA 2012. Lecture Notes in Computer Science, vol 7335. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-31137-6_47

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-31137-6_47

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-31136-9

  • Online ISBN: 978-3-642-31137-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation