Skip to main content
SpringerLink
Log in

RuCAM: Comparative Argumentative Machine for the Russian Language

  • Conference paper
  • First Online:
Analysis of Images, Social Networks and Texts (AIST 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14486))

  • 58 Accesses

Abstract

Comparative question answering is one of the question answering subtasks which requires not only to choose between two (or more) objects, but also to explain the choice and support it with arguments. ChatGPT-like models are able nowadays to generate a coherent answer in a natural language, however, they are not fully reliable as they are not publicly accessible and tend to hallucinate. Another solution is a Comparative Argument Machine (CAM), which however, has been developed for English only. In this paper, we describe the development of RuCAM—comparative argumentative machine for Russian, as well as the challenges of the system adaptation for another language. It is the first open-domain system to argumentatively compare objects in Russian with respect to information extracted from the OSCAR corpus. We also introduce several datasets for the RuCAM subtasks: comparative question classification, object and aspect identification, comparative sentences classification. We provide models for each subtask and compare them with the existing baselines.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 74.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://compare.com.

  2. 2.

    https://quora.com, https://stackexchange.com.

  3. 3.

    https://rucam.ltdemos.informatik.uni-hamburg.de.

  4. 4.

    https://github.com/stefanrer/MCQA_RUS.

  5. 5.

    https://www.elastic.co.

  6. 6.

    https://www.elastic.co/guide/en/elasticsearch/guide/current/scoring-theory.html.

References

  1. Beloucif, M., Yimam, S.M., Stahlhacke, S., Biemann, C.: Elvis vs. M. Jackson: who has more albums? Classification and identification of elements in comparative questions. In: Calzolari, N., et al. (eds.) Proceedings of the Thirteenth Language Resources and Evaluation Conference, LREC 2022, Marseille, France, 20–25 June 2022, pp. 3771–3779. European Language Resources Association (2022). https://aclanthology.org/2022.lrec-1.402

  2. Bondarenko, A., Ajjour, Y., Dittmar, V., Homann, N., Braslavski, P., Hagen, M.: Towards understanding and answering comparative questions. In: Candan, K.S., Liu, H., Akoglu, L., Dong, X.L., Tang, J. (eds.) WSDM 2022: The Fifteenth ACM International Conference on Web Search and Data Mining, Virtual Event/Tempe, AZ, USA, 21–25 February 2022, pp. 66–74. ACM (2022). https://doi.org/10.1145/3488560.3498534

  3. Bondarenko, A., et al.: Comparative web search questions. In: Caverlee, J., Hu, X.B., Lalmas, M., Wang, W. (eds.) WSDM 2020: The Thirteenth ACM International Conference on Web Search and Data Mining, Houston, TX, USA, 3–7 February 2020, pp. 52–60. ACM (2020). https://doi.org/10.1145/3336191.3371848

  4. Bondarenko, A., et al.: Overview of Touché 2020: argument retrieval. In: Arampatzis, A., et al. (eds.) CLEF 2020. LNCS, vol. 12260, pp. 384–395. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58219-7_26

    Chapter  Google Scholar 

  5. Bondarenko, A., et al.: Overview of touché 2022: argument retrieval. In: Faggioli, G., Ferro, N., Hanbury, A., Potthast, M. (eds.) Proceedings of the Working Notes of CLEF 2022 - Conference and Labs of the Evaluation Forum, Bologna, Italy, 5th–8th September 2022. CEUR Workshop Proceedings, vol. 3180, pp. 2867–2903. CEUR-WS.org (2022). https://ceur-ws.org/Vol-3180/paper-247.pdf

  6. Bondarenko, A., et al.: Overview of Touché 2021: argument retrieval. In: Candan, K.S., et al. (eds.) CLEF 2021. LNCS, vol. 12880, pp. 450–467. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-85251-1_28

    Chapter  Google Scholar 

  7. Cao, M., Dong, Y., Cheung, J.: Hallucinated but factual! inspecting the factuality of hallucinations in abstractive summarization. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, Dublin, Ireland (Volume 1: Long Papers), pp. 3340–3354. Association for Computational Linguistics (2022). https://doi.org/10.18653/v1/2022.acl-long.236

  8. Chekalina, V., Bondarenko, A., Biemann, C., Beloucif, M., Logacheva, V., Panchenko, A.: Which is better for deep learning: Python or matlab? Answering comparative questions in natural language. In: Gkatzia, D., Seddah, D. (eds.) Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, EACL 2021, Online, 19–23 April 2021, pp. 302–311. Association for Computational Linguistics (2021). https://doi.org/10.18653/v1/2021.eacl-demos.36

  9. Chistova, E.: End-to-end argument mining over varying rhetorical structures. In: Findings of the Association for Computational Linguistics: ACL 2023, Toronto, Canada, pp. 3376–3391. Association for Computational Linguistics (2023). https://doi.org/10.18653/v1/2023.findings-acl.209. https://aclanthology.org/2023.findings-acl.209

  10. Dale, D., Voita, E., Barrault, L., Costa-jussà, M.R.: Detecting and mitigating hallucinations in machine translation: model internal workings alone do well, sentence similarity Even better. In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics, Toronto, Canada (Volume 1: Long Papers), pp. 36–50. Association for Computational Linguistics (2023). https://aclanthology.org/2023.acl-long.3

  11. Fishcheva, I., Goloviznina, V., Kotelnikov, E.V.: Traditional machine learning and deep learning models for argumentation mining in Russian texts. CoRR abs/2106.14438 (2021). https://arxiv.org/abs/2106.14438

  12. Fishcheva, I., Kotelnikov, E.: Cross-lingual argumentation mining for Russian texts. In: van der Aalst, W.M.P., et al. (eds.) AIST 2019. LNCS, vol. 11832, pp. 134–144. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-37334-4_12

    Chapter  Google Scholar 

  13. Goloviznina, V., Fishchev, I., Peskisheva, T., Kotelnikov, E.: Aspect-based argument generation in Russian. In: Computational Linguistics and Intellectual Technologies: Papers from the Annual Conference “Dialogue” (2023)

    Google Scholar 

  14. Kotelnikov, E.V., Loukachevitch, N.V., Nikishina, I., Panchenko, A.: RuArg-2022: argument mining evaluation. In: Computational Linguistics and Intellectual Technologies: Papers from the Annual Conference “Dialogue” (2022)

    Google Scholar 

  15. Ng, N., Yee, K., Baevski, A., Ott, M., Auli, M., Edunov, S.: Facebook FAIR’s WMT19 news translation task submission. In: Proceedings of the Fourth Conference on Machine Translation, Florence, Italy (Volume 2: Shared Task Papers, Day 1), pp. 314–319. Association for Computational Linguistics (2019). https://doi.org/10.18653/v1/W19-5333. https://aclanthology.org/W19-5333

  16. Nikolaev, K., Malafeev, A.: Russian-language question classification: a new typology and first results. In: van der Aalst, W.M.P., et al. (eds.) AIST 2017. LNCS, vol. 10716, pp. 72–81. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-73013-4_7

    Chapter  Google Scholar 

  17. Nikolaev, K., Malafeev, A.: Russian Q &A method study: from Naive Bayes to convolutional neural networks. In: van der Aalst, W.M.P., et al. (eds.) AIST 2018. LNCS, vol. 11179, pp. 121–126. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-11027-7_12

    Chapter  Google Scholar 

  18. Ortiz Suárez, P.J., Sagot, B., Romary, L.: Asynchronous pipelines for processing huge corpora on medium to low resource infrastructures. In: Proceedings of the Workshop on Challenges in the Management of Large Corpora (CMLC-7) 2019. Cardiff, 22nd July 2019, pp. 9–16. Leibniz-Institut für Deutsche Sprache, Mannheim (2019). https://doi.org/10.14618/ids-pub-9021. http://nbn-resolving.de/urn:nbn:de:bsz:mh39-90215

  19. Panchenko, A., Bondarenko, A., Franzek, M., Hagen, M., Biemann, C.: Categorizing comparative sentences. In: Stein, B., Wachsmuth, H. (eds.) Proceedings of the 6th Workshop on Argument Mining, ArgMining@ACL 2019, Florence, Italy, 1 August 2019, pp. 136–145. Association for Computational Linguistics (2019). https://doi.org/10.18653/v1/w19-4516

  20. Pavlichenko, N., Stelmakh, I., Ustalov, D.: CrowdSpeech and VoxDIY: benchmark dataset for crowdsourced audio transcription. In: Vanschoren, J., Yeung, S. (eds.) NeurIPS Datasets and Benchmarks 2021, December 2021, virtual (2021)

    Google Scholar 

  21. Schildwächter, M., Bondarenko, A., Zenker, J., Hagen, M., Biemann, C., Panchenko, A.: Answering comparative questions: better than ten-blue-links? In: Azzopardi, L., Halvey, M., Ruthven, I., Joho, H., Murdock, V., Qvarfordt, P. (eds.) Proceedings of the 2019 Conference on Human Information Interaction and Retrieval, CHIIR 2019, Glasgow, Scotland, UK, 10–14 March 2019, pp. 361–365. ACM (2019). https://doi.org/10.1145/3295750.3298916

Download references

Acknowledgements

This work was supported by the DFG through the project “ACQuA: Answering Comparative Questions with Arguments” (grants BI 1544/7- 1 and HA 5851/2- 1) as part of the priority program “RATIO: Robust Argumentation Machines” (SPP 1999).

Author information

Authors and Affiliations

  1. HSE University, Moscow, Russia

    Maria Maslova, Stefan Rebrikov & Anton Artsishevski

  2. Universität Hamburg, Hamburg, Germany

    Sebastian Zaczek, Chris Biemann & Irina Nikishina

Authors
  1. Maria Maslova
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stefan Rebrikov
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Anton Artsishevski
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sebastian Zaczek
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Chris Biemann
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Irina Nikishina
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Irina Nikishina .

Editor information

Editors and Affiliations

  1. National Research University Higher School of Economics, Moscow, Russia

    Dmitry I. Ignatov

  2. Krasovskii Institute of Mathematics and Mechanics of Russian Academy of Sciences, Yekaterinburg, Russia

    Michael Khachay

  3. University of Oslo, Oslo, Norway

    Andrey Kutuzov

  4. American University of Armenia, Yerevan, Armenia

    Habet Madoyan

  5. Artificial Intelligence Research Institute, Moscow, Russia

    Ilya Makarov

  6. University of Hamburg, Hamburg, Germany

    Irina Nikishina

  7. Skolkovo Institute of Science and Technology, Moscow, Russia

    Alexander Panchenko

  8. Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates

    Maxim Panov

  9. University of Florida, Gainesville, FL, USA

    Panos M. Pardalos

  10. National Research University Higher School of Economics, Nizhny Novgorod, Russia

    Andrey V. Savchenko

  11. Apptek, Aachen, Germany

    Evgenii Tsymbalov

  12. Kazan Federal University, Kazan, Russia

    Elena Tutubalina

  13. MTS AI, Moscow, Russia

    Sergey Zagoruyko

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Maslova, M., Rebrikov, S., Artsishevski, A., Zaczek, S., Biemann, C., Nikishina, I. (2024). RuCAM: Comparative Argumentative Machine for the Russian Language. In: Ignatov, D.I., et al. Analysis of Images, Social Networks and Texts. AIST 2023. Lecture Notes in Computer Science, vol 14486. Springer, Cham. https://doi.org/10.1007/978-3-031-54534-4_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-54534-4_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-54533-7

  • Online ISBN: 978-3-031-54534-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation