Skip to main content
SpringerLink
Log in

Question Answering of Bar Exams by Paraphrasing and Legal Text Analysis

  • Conference paper
  • First Online:
New Frontiers in Artificial Intelligence (JSAI-isAI 2016)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 10247))

Included in the following conference series:

Abstract

Our legal question answering system combines legal information retrieval and textual entailment, and exploits paraphrasing and sentence-level analysis of queries and legal statutes. We have evaluated our system using the training data from the competition on legal information extraction/entailment (COLIEE)-2016. The competition focuses on the legal information processing required to answer yes/no questions from Japanese legal bar exams, and it consists of three phases: legal ad-hoc information retrieval (Phase 1), textual entailment (Phase 2), and a combination of information retrieval and textual entailment (Phase 3). Phase 1 requires the identification of Japan civil law articles relevant to a legal bar exam query. For this phase, we have used an information retrieval approach using TF-IDF and a Ranking SVM. Phase 2 requires decision on yes/no answer for previously unseen queries, which we approach by comparing the approximate meanings of queries with relevant articles. Our meaning extraction process uses a selection of features based on a kind of paraphrase, coupled with a condition/conclusion/exception analysis of articles and queries. We also identify synonym relations using word embedding, and detect negation patterns from the articles. Our heuristic selection of attributes is used to build an SVM model, which provides the basis for ranking a decision on the yes/no questions. Experimental evaluation show that our method outperforms previous methods. Our result ranked highest in the Phase 3 in the COLIEE-2016 competition.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Notes

  1. 1.

    https://webdocs.cs.ualberta.ca/~miyoung2/COLIEE2016/.

  2. 2.

    https://code.google.com/p/word2vec.

  3. 3.

    Lucene can be downloaded from http://lucene.apache.org/core/.

  4. 4.

    http://excite.translation.jp/world/.

  5. 5.

    https://translate.google.com/.

  6. 6.

    The SVM function in Weka is provided by libsvm https://www.csie.ntu.edu.tw/~cjlin/libsvm/, and the linear kernal is from liblinear https://www.csie.ntu.edu.tw/~cjlin/liblinear/.

References

  1. Jones, K.S.: A statistical interpretation of term specicity and its application in retrieval. In: Willett, P. (ed.) Document Retrieval Systems, pp. 132–142. Taylor Graham Publishing, London (1988)

    Google Scholar 

  2. Joachims, T.: Optimizing search engines using clickthrough data. In: Proceedings of 8th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2002, pp. 133–142. ACM, New York (2002)

    Google Scholar 

  3. Maxwell, K.T., Oberlander, J., Croft, W.B.: Feature-based selection of dependency paths in ad hoc information retrieval. In: Proceedings of 51st Annual Meeting of the Association for Computational Linguistics, (vol. 1: Long Papers), pp. 507–516. Association for Computational Linguistics, Sofia, August 2013

    Google Scholar 

  4. Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The WEKA data mining software: an update. SIGKDD Explor. 11(1), 10–18 (2009)

    Article  Google Scholar 

  5. Bdour, W.N., Gharaibeh, N.K.: Development of yes/no Arabic question answering system. Int. J. Artif. Intell. Appl. 4(1), 51–63 (2013)

    Google Scholar 

  6. Nielsen, R.D., Ward, W., Martin, J.H.: Toward dependency path based entailment. In: Proceedings of 2nd PASCAL Challenges Workshop on RTE (2006)

    Google Scholar 

  7. Zanzotto, F.M., Moschitti, A., Pennacchiotti, M., Pazienza, M.T.: Learning textual entailment from examples. In: Proceedings of 2nd PASCAL Challenges Workshop on RTE (2006)

    Google Scholar 

  8. Harmeling, S.: An extensible probabilistic transformation-based approach to the third recognizing textual entailment challenge. In: Proceedings of ACL PASCAL Workshop on Textual Entailment and Paraphrasing (2007)

    Google Scholar 

  9. Marsi, E., Krahmer, E., Bosma, W.: Dependency-based paraphrasing for recognizing textual entailment. In: Proceedings of ACL PASCAL Workshop on Textual Entailment and Paraphrasing (2007)

    Google Scholar 

  10. Kim, M.-Y., Xu, Y., Goebel, R.: Alberta-KXG: legal question answering using ranking SVM and syntactic/semantic similarity. In: 8th International Workshop on Juris-Informatics (JURISIN), 2014

    Google Scholar 

  11. Kim, M.-Y., Xu, Y., Goebel, R.: A convolutional neural network in legal question answering. In: JURISIN Workshop (2015)

    Google Scholar 

  12. Sultan, M.A., Bethard, S., Sumner, T.: Back to basics for monolingual alignment: exploiting word similarity and contextual evidence. Trans. Assoc. Comput. Linguist. 2, 219–230 (2014)

    Article  Google Scholar 

  13. Berant, J., Percy, L.: Semantic parsing via paraphrasing. In: Proceedings of Conference of the Association for Computational Linguistics (ACL), pp. 1415–1425 (2014)

    Google Scholar 

  14. Zhang, W., Ming, Z., Zhang, Y., Liu, T., Chua, T.S.: Exploring key concept paraphrasing based on pivot language translation for question retrieval. In: AAAI, pp. 410–416 (2015)

    Google Scholar 

  15. Manning, C.D., Surdeanu, M., Bauer, J., Finkel, J., Bethard, S.J., McClosky, D.: The Stanford CoreNLP natural language processing toolkit. In: Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pp. 55–60 (2014)

    Google Scholar 

  16. Kim, M.-Y., Goebel, R., Kano, Y., Satoh, K.: COLIEE-2016: evaluation of the competition on legal information extraction and entailment. In: Tenth International Workshop on Juris-Informatics (JURISIN) (2016)

    Google Scholar 

  17. Carvalho, D.S., Tran, V.D., Tran, K.V., Lai, V.D., Nguyen, M.-L.: Lexical to discourse-level corpus modeling for legal question answering. In: Tenth International Workshop on Juris-Informatics (JURISIN) (2016). (Submission ID: JNLN1)

    Google Scholar 

  18. Taniguchi, R., Kano, Y.: Legal yes/no question answering system using case-role analysis. In: Tenth International Workshop on Juris-Informatics (JURISIN) (2016). (Submission ID: KIS)

    Google Scholar 

  19. Kim, K., Heo, S., Jung, S., Hong, K., Rhim, Y.-Y.: An ensemble based legal information retrieval and entailment system. In: Tenth International Workshop on Juris-Informatics (JURISIN) (2016). (Submission ID: iLis7)

    Google Scholar 

  20. Do, P.-K., Nguyen, H.-T., Tran, C.-X., Nguyen, M.-T., Minh, N.L.: Legal question answering using ranking SVM and deep convolutional neural network. In: Tenth International Workshop on Juris-Informatics (JURISIN) (2016). (Submission ID: JNLN3)

    Google Scholar 

  21. Onodera, D., Yoshioka, M.: Civil code article information retrieval system based on legal terminology and civil code article structure. In: Tenth International Workshop on Juris-Informatics (JURISIN) (2016). (Submission ID: HUKB)

    Google Scholar 

  22. Nguyen, T.-S., Phan, V.-A., Nguyen, T.-H., Trieu, H.-L., Chau, N.-P., Pham, T.-T., Nguyen, L.-M.: Legal information extraction/entailment using SVM-ranking and tree-based convolutional neural network. In: Tenth International Workshop on Juris-Informatics (JURISIN) (2016). (Submission ID: JNLN2)

    Google Scholar 

  23. John, A.K., Di Caro, L., Boella, G., Bartolini, C.: Team-normas’ participation at the COLIEE 2016 bar legal exam competition. In: Tenth International Workshop on Juris-Informatics (JURISIN) (2016). (Submission ID: N01)

    Google Scholar 

Download references

Acknowledgements

This research was supported by the Alberta Machine Intelligence Institute (www.amii.ca). We are indebted to Ken Satoh of the National Institute for Informatics, who had the vision to create the COLIEE competition.

Author information

Authors and Affiliations

  1. Alberta Machine Intelligence Institute, University of Alberta, Edmonton, AB, Canada

    Ying Xu & Randy Goebel

  2. iLab Tongji, School of Software Engineering, Tongji University, Shanghai, China

    Yao Lu

  3. Department of Science, University of Alberta, Augustana Campus, Camrose, AB, Canada

    Mi-Young Kim

Authors
  1. Mi-Young Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ying Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yao Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Randy Goebel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mi-Young Kim .

Editor information

Editors and Affiliations

  1. Graduate School of Business Sciences, University of Tsukuba, Tokyo, Japan

    Setsuya Kurahashi

  2. Fujitsu Laboratories Ltd., Kanagawa, Japan

    Yuiko Ohta

  3. Chiba University, Chiba, Japan

    Sachiyo Arai

  4. National Institute of Informatics, Tokyo, Japan

    Ken Satoh

  5. Ochanomizu University, Tokyo, Japan

    Daisuke Bekki

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Kim, MY., Xu, Y., Lu, Y., Goebel, R. (2017). Question Answering of Bar Exams by Paraphrasing and Legal Text Analysis. In: Kurahashi, S., Ohta, Y., Arai, S., Satoh, K., Bekki, D. (eds) New Frontiers in Artificial Intelligence. JSAI-isAI 2016. Lecture Notes in Computer Science(), vol 10247. Springer, Cham. https://doi.org/10.1007/978-3-319-61572-1_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-61572-1_20

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-61571-4

  • Online ISBN: 978-3-319-61572-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation