Abstract
One of the main research challenges in Semantic Role Labeling (SRL) is the development of systems for languages other than English. For Brazilian Portuguese, a corpus with appropriate manually-annotated data, PropBank.Br, has recently become available. Proposals for implementing SRL systems using this corpus have already been made, but no standard way of comparing their results is available. We present a benchmark for comparing SRL systems for Brazilian Portuguese, based on the CoNLL Shared Tasks on SRL for English. Training and test data sets, evaluation metrics and a baseline system are provided as part of this benchmark. These resources have been used to implement a supervised SRL system which outperforms the baseline (17 points better in F 1 measure). Most importantly, the benchmark proved to be useful for evaluating the performance of SRL systems for Brazilian Portuguese.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Alva-Manchego, F., Rosa, J.L.: Towards Semi-supervised Brazilian Portuguese Semantic Role Labeling: Building a Benchmark. In: Caseli, H., Villavicencio, A., Teixeira, A., Perdigão, F. (eds.) PROPOR 2012. LNCS (LNAI), vol. 7243, pp. 210–217. Springer, Heidelberg (2012)
Bick, E.: The Parsing System PALAVRAS – Automatic Grammatical Analysis of Portuguese in a Constraint Grammar Framework. Aarhus University Press (2000)
Buchholz, S., Marsi, E.: CoNLL-X Shared Task on Multilingual Dependency Parsing. In: 10th Conference on Computational Natural Language Learning (CoNNL 2006), pp. 149–164. ACL, New York City (2006)
Carreras, X., Màrquez, L.: Introduction to the CoNLL-2004 Shared Task – Semantic Role Labeling. In: 8th Conference on Computational Natural Language Learning (CoNNL 2004) – Shared Task, pp. 89–97. ACL, Boston (2004)
Carreras, X., Màrquez, L.: Introduction to the CoNLL-2005 Shared Task – Semantic Role Labeling. In: 9th Conference on Computational Natural Language Learning (CoNNL 2005) – Shared Task, pp. 152–164. ACL, Ann Arbor (2005)
Duran, M.S., Aluísio, S.M.: Propbank-Br – A Brazilian Treebank Annotated with Semantic Role Labels. In: 8th International Conference on Language Resources and Evaluation (LREC 2012), Istanbul, Turkey, pp. 1862–1867 (2012)
Fonseca, E.R., Rosa, J.L.: An Architecture for Semantic Role Labeling on Portuguese. In: Caseli, H., Villavicencio, A., Teixeira, A., Perdigão, F. (eds.) PROPOR 2012. LNCS (LNAI), vol. 7243, pp. 204–209. Springer, Heidelberg (2012)
Hajic, J., Ciaramita, M., Johansson, R., Kawahara, D., Martí, M.A., Màrquez, L., Mayers, A., Nivre, J., Padó, S., Stepánek, J., Stranák, P., Surdeanu, M., Xue, N., Zhang, Y.: The CoNLL-2009 Shared Task: Syntactic and Semantic Dependencies in Multiple Languages. In: 13th Conference on Computational Natural Language Learning (CoNNL 2009) – Shared Task, pp. 1–18. ACL, Boulder (2009)
Màrquez, L., Carreras, X., Litkowski, K.C., Stevenson, S.: Semantic Role Labeling – An Introduction to the Special Issue. Computational Linguistics 34(2), 145–159 (2008)
Palmer, M., Gildea, D., Kingsbury, P.: The Proposition Bank – An Annotated Corpus of Semantic Roles. Computational Linguistics 31(1), 71–106 (2005)
Palmer, M., Gildea, D., Xue, N.: Semantic Role Labeling, 1st edn. Morgan & Claypool Publishers (2010)
Pradhan, S.S., Ward, W., Martin, J.H.: Towards Robust Semantic Role Labeling. Computational Linguistics 34(2), 289–310 (2008)
Santos, D., Bick, E., Afonso, S.: Floresta sintá(c)tica: apresentação e história do projecto. Encontro Um passeio pela Floresta Sintá(c)tica (Setembro 2007)
Sequeira, J., Gonçalves, T., Quaresma, P.: Semantic Role Labeling for Portuguese – A Preliminary Approach –. In: Caseli, H., Villavicencio, A., Teixeira, A., Perdigão, F. (eds.) PROPOR 2012. LNCS (LNAI), vol. 7243, pp. 193–203. Springer, Heidelberg (2012)
Surdeanu, M., Johansson, R., Meyers, A., Màrquez, L., Nivre, J.: The CoNLL 2008 Shared Task on Joint Parsing of Syntactic and Semantic Dependencies. In: 12th Conference on Computational Natural Language Learning (CoNNL 2008), pp. 159–177. ACL, Manchester (2008)
Xue, N., Palmer, M.: Calibrating Features for Semantic Role Labeling. In: 2004 Conference on Empirical Methods in Natural Language Processing (EMNLP 2004), pp. 88–94. ACL, Barcelona (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Alva-Manchego, F.E., Rosa, J.L.G. (2012). Semantic Role Labeling for Brazilian Portuguese: A Benchmark. In: Pavón, J., Duque-Méndez, N.D., Fuentes-Fernández, R. (eds) Advances in Artificial Intelligence – IBERAMIA 2012. IBERAMIA 2012. Lecture Notes in Computer Science(), vol 7637. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-34654-5_49
Download citation
DOI: https://doi.org/10.1007/978-3-642-34654-5_49
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-34653-8
Online ISBN: 978-3-642-34654-5
eBook Packages: Computer ScienceComputer Science (R0)