Skip to main content
SpringerLink
Log in
Book cover

International Work-Conference on Artificial Neural Networks

IWANN 2019: Advances in Computational Intelligence pp 621–631Cite as

Document Model with Attention Bidirectional Recurrent Network for Gender Identification

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 11506))

Abstract

Author profiling is an important statistical and semantic processing task Author profiling is an important statistical and semantic processing task in the field of natural language processing (NLP). It refers to the extraction of information from author’s texts such as gender, age and other kinds of personality traits. Author profiling can be applied in various fields like marketing, security and forensics. In this work, we explore how bi-directional deep learning architectures can be used to learn the abstract and higher-level features of the document, which could be employed to identify the author’s gender. To deal with this, we extend Bidirectional Long Short-Term Memory Networks Language Models with an attention mechanism. The originality of our approach lays in its ability to capture the most important semantic information in a sentence. The experimental results on Facebook and twitter corpus show that our method outperformed the majority of the existing methods.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    https://techcrunch.com/2017/06/27/facebook-2-billion-users/.

  2. 2.

    Wikipedia, “WikimediaDownloads.” https://dumps.wikimedia.org/arwiki/20170401/, 2017. [Online; accessed 10-April-2017].

References

  1. Poulston, A., Stevenson, M., Bontcheva, K.: Topic models and n–gram language models for author profiling. In: Proceedings of CLEF 2015 Evaluation Labs (2015)

    Google Scholar 

  2. Alvarez-Carmona, M.A., et al.: INAOE’s participation at PAN’15: Author profiling task. Working Notes Papers of the CLEF (2015)

    Google Scholar 

  3. Argamon, S., Koppel, M., Fine, J., Shimoni, A.R.: Gender, genre, and writing style in formal written texts. Text-The Hague Then Amsterdam Then Berlin 23(3), 321–346 (2003)

    Google Scholar 

  4. Aslam, T., Krsul, I., Spafford, E.H.: Use of a taxonomy of security faults (1996)

    Google Scholar 

  5. Bamman, D., Eisenstein, J., Schnoebelen, T.: Gender identity and lexical variation in social media. J. Sociolinguistics 18(2), 135–160 (2014)

    Article  Google Scholar 

  6. Bassem, B., Zrigui, M.: Enhancing deep learning gender identification with gated recurrent units architecture in social text. Computación y Sistemas 22(3), 757–766 (2018)

    Google Scholar 

  7. González-Gallardo, C.E., et al.: Tweets classification using corpus dependent tags, character and POS N-grams. In: Proceedings of CLEF 2015 Evaluation Labs (2015)

    Google Scholar 

  8. Chaski, C.E.: Who wrote it? Steps toward a science of authorship identification. Natl. Inst. Justice J. 233, 15–22 (1997)

    Google Scholar 

  9. Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014)

  10. Clauset, A., Moore, C., Newman, M.E.: Hierarchical structure and the prediction of missing links in networks. Nature 453(7191), 98 (2008)

    Article  Google Scholar 

  11. Collobert, R., et al.: Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12, 2493–2537 (2011)

    MATH  Google Scholar 

  12. Ding, H., Samadzadeh, M.H.: Extraction of Java program fingerprints for software authorship identification. J. Syst. Softw. 72(1), 49–57 (2004)

    Article  Google Scholar 

  13. Estival, D., et al.: Author Profiling for English and Arabic Emails (2008)

    Google Scholar 

  14. Gehring, W.J., et al.: A neural system for error detection and compensation. Psychol. Sci. 4(6), 385–390 (1993)

    Article  Google Scholar 

  15. Gokturk, S.B., et al.: System and method for providing objectified image renderings using recognition information from images. U.S. Patent No. 9,430,719, 30 August 2016

    Google Scholar 

  16. Hochreiter, S., Schmidhuber, J.: LSTM can solve hard long time LAG problems. In: Advances in Neural Information Processing Systems, pp. 473–479 (1997)

    Google Scholar 

  17. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  18. Inches, G., Crestani, F.: Overview of the International Sexual Predator Identification Competition at PAN-2012. CLEF (Online working notes/labs/workshop), vol. 30 (2012)

    Google Scholar 

  19. Joachims, T.: Text categorization with support vector machines: learning with many relevant features. In: Nédellec, C., Rouveirol, C. (eds.) ECML 1998. LNCS, vol. 1398, pp. 137–142. Springer, Heidelberg (1998). https://doi.org/10.1007/BFb0026683

    Chapter  Google Scholar 

  20. Kalchbrenner, N., Blunsom, P.: Recurrent continuous translation models. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (2013)

    Google Scholar 

  21. Kalchbrenner, N., Grefenstette, E., Blunsom, P.: A convolutional neural network for modelling sentences. arXiv preprint arXiv:1404.2188 (2014)

  22. Kodiyan, D., et al.: Author profiling with bidirectional RNNs using attention with GRUs: notebook for PAN at CLEF 2017. In: CLEF 2017 Evaluation Labs and Workshop–Working Notes Papers, Dublin, Ireland, 11–14 September 2017 (2017)

    Google Scholar 

  23. LeCun, L.B., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  24. Werlen, L.M.: Statistical learning methods for profiling analysis. In: Proceedings of CLEF 2015 Evaluation Labs (2015)

    Google Scholar 

  25. Maraoui, M., Terbeh, N., Zrigui, M.: Arabic discourse analysis based on acoustic, prosodic and phonetic modeling: elocution evaluation, speech classification and pathological speech correction. Int. J. Speech Technol. 21(4), 1071–1090 (2018)

    Article  Google Scholar 

  26. Martinc, M., Škrjanec, I., Zupan, K., Pollak, S.: Pan 2017: Author Profiling, gender and Language Variety Prediction. CLEF (Working Notes) 2017. CEUR Workshop Proceedings 1866, CEUR-WS.org (2017)

    Google Scholar 

  27. Miura, Y. et al.: Author Profiling with Word+Character Neural Attention Network. CLEF (Working Notes) 2017. CEUR Workshop Proceedings 1866, CEUR-WS.org (2017)

    Google Scholar 

  28. Peersman, C., Daelemans, W., Van Vaerenbergh, L.: Predicting age and gender in online social networks. In: Proceedings of the 3rd International Workshop on Search and Mining User-Generated Contents, pp. 37–44. ACM (2011)

    Google Scholar 

  29. Pham, D.D., Tran, G.B., Pham, S.B.: Author profiling for Vietnamese blogs. In: International Conference on Asian Language Processing, IALP 2009, pp. 190–194. IEEE (2009)

    Google Scholar 

  30. Rangel, F., et al.: Overview of the 5th author profiling task at PAN 2017: gender and language variety identification in Twitter. Working Notes Papers of the CLEF (2017)

    Google Scholar 

  31. Rangel, F., Rosso, P., Montes-y-Gómez, M., et al.: Overview of the 6th author profiling task at PAN 2018: multimodal gender identification in Twitter. Working Notes Papers of the CLEF (2018)

    Google Scholar 

  32. Säily, T.: Variation in morphological productivity in the BNC: Sociolinguistic and methodological considerations. Corpus Linguist. Linguist. Theory 7(1), 119–141 (2011)

    Article  Google Scholar 

  33. Sallis, P.J., et al.: Identified: software authorship analysis with case-based reasoning (1998)

    Google Scholar 

  34. Sap, M., et al.: Developing age and gender predictive lexica over social media. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1146–1151 (2014)

    Google Scholar 

  35. Socher, R., et al.: Recursive deep models for semantic compositionality over a sentiment treebank. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (2013)

    Google Scholar 

  36. Wang, P., et al.: Semantic clustering and convolutional neural network for short text categorization. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), vol. 2, pp. 352–357 (2015)

    Google Scholar 

  37. Wang, Y., Huang, M., Zhao, L.: Attention-based LSTM for aspect-level sentiment classification. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 606–615 (2016)

    Google Scholar 

  38. Williams, J.D., Zweig, G.: End-to-end LSTM-based dialog control optimized with supervised and reinforcement learning. arXiv preprint arXiv:1606.01269 (2016)

  39. Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., Hovy, E.: Hierarchical attention networks for document classification. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1480–1489 (2016)

    Google Scholar 

  40. Yih, X.H., Meek, C.: Semantic parsing for single-relation question answering. In: Proceedings of ACL 2014 (2014)

    Google Scholar 

  41. Yin, W., et al.: Comparative study of CNN and RNN for natural language processing. arXiv preprint arXiv:1702.01923 (2017)

  42. Zhou, C., Sun, C., Liu, Z., Lau, F.: A C-LSTM neural network for text classification. arXiv preprint arXiv:1511.08630 (2015)

  43. Zhou, P., et al.: Attention-based bidirectional long short-term memory networks for relation classification. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), vol. 2, pp. 207–212 (2016)

    Google Scholar 

  44. Zhou, T., Shen, T., Long, G., Jiang, J., Pan, S., Zhang, C.: DiSAN: directional self-attention network for RNN/CNN-free language understanding. In: Thirty-Second AAAI Conference on Artificial Intelligence, April 2018

    Google Scholar 

  45. Zrigui, M., Ayadi, R., Mars, M., Maraoui, M.: Arabic text classification framework based on latent Dirichlet allocation. J. Comput. Inform. Technol. 20(2), 125–140 (2012)

    Google Scholar 

  46. Zrigui, M., Charhad, M., Zouaghi, A.: A framework of indexation and document video retrieval based on the conceptual graphs. J. Comput. Inform. Technol. 18(3), 245–256 (2010)

    Article  Google Scholar 

  47. Zouaghi, A., Zrigui, M., Antoniadis, G.: Compréhension automatique de la parole arabe spontanée. Traitement Automatique des Langues 49(1), 141–166 (2008)

    Google Scholar 

  48. Zouaghi, A., Merhbene, L., Zrigui, M.: A hybrid approach for Arabic word sense disambiguation. Int. J. Comput. Process. Lang. 24(02), 133–151 (2012)

    Article  Google Scholar 

  49. Zouaghi, A., Zrigui, M., Antoniadis, G., Merhbene, L.: Contribution to semantic analysis of Arabic language. In: Adv. Artif. Intell. (2012)

    Google Scholar 

  50. Zouaghi, A., Merhbene, L., Zrigui, M.: Combination of information retrieval methods with LESK algorithm for Arabic word sense disambiguation. Artif. Intell. Rev. 38(4), 257–269 (2012)

    Article  Google Scholar 

  51. Zouaghi, A., Zrigui, M., Ahmed, M.B., Riadi, L.: Évaluation des performances d’un modèle de langage stochastique pour la compréhension de la parole arabe spontanée. In: Proceedings of the TALN (2007)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Research Laboratory in Algebra, Numbers Theory and Intelligent Systems, University of Monastir, Monastir, Tunisia

    Bassem Bsir & Mounir Zrigui

Authors
  1. Bassem Bsir
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mounir Zrigui
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Bassem Bsir or Mounir Zrigui .

Editor information

Editors and Affiliations

  1. University of Granada, Granada, Spain

    Ignacio Rojas

  2. University of Malaga, Malaga, Spain

    Gonzalo Joya

  3. Polytechnic University of Catalonia, Barcelona, Spain

    Andreu Catala

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bsir, B., Zrigui, M. (2019). Document Model with Attention Bidirectional Recurrent Network for Gender Identification. In: Rojas, I., Joya, G., Catala, A. (eds) Advances in Computational Intelligence. IWANN 2019. Lecture Notes in Computer Science(), vol 11506. Springer, Cham. https://doi.org/10.1007/978-3-030-20521-8_51

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-20521-8_51

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-20520-1

  • Online ISBN: 978-3-030-20521-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation