Skip to main content
SpringerLink
Log in

Evaluation of Question-Answering System About Conversational Agent’s Personality

  • Chapter
  • First Online:
Dialogues with Social Robots

Part of the book series: Lecture Notes in Electrical Engineering ((LNEE,volume 427))

Abstract

We develop a question-answering system for questions that ask about a conversational agent’s personality based on large-scale question-answer pairs created by hand. In casual dialogues, the speaker sometimes asks his conversation partner questions about favorites or experiences. Since this behavior also appears in conversational dialogues with a dialogue system, systems must be developed to respond to such questions. However, the effectiveness of personality-question-answering for conversational agents has not been investigated. Our user-machine chat experiments show that our question-answering system, which estimates appropriate answers with 60.7 % accuracy for the personality questions in our conversation corpus, significantly improves user’s subjective evaluations.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Shibata, M., Nishiguchi, T., Tomiura, Y.: Dialog system for open-ended conversation using web documents. Informatica 33, 277–284 (2009), http://www.informatica.si/PDF/Informatica_2009_3.pdf#page=37

  2. Ritter, A., Cherry, C., Dolan, W.: Data-driven response generation in social media. In: Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pp. 583–593 (2011). http://ai.cs.washington.edu/www/media/papers/paper2.pdf

  3. Wong, W., Cavedon, L., Thangarajah, J., Padgham, L.: Strategies for mixed-initiative conversation management using question-answer pairs. In: Proceedings of the 24th International Conference on Computational Linguistics. pp. 2821–2834 (2012). https://www.aclweb.org/anthology-new/C/C12/C12-1172.pdf

  4. Meguro, T., Higashinaka, R., Minami, Y., Dohsaka, K.: Controlling listening-oriented dialogue using partially observable markov decision processes. In: Proceedings of the 23rd International Conference on Computational Linguistics, pp. 761–769 (2010)

    Google Scholar 

  5. Higashinaka, R., Imamura, K., Meguro, T., Miyazaki, C., Kobayashi, N., Sugiyama, H., Hirano, T., Makino, T., Matsuo, Y.: Towards an open-domain conversational system fully based on natural language processing. In: Proceedings of the 25th International Conference on Computational Linguistics, pp. 928–939 (2014)

    Google Scholar 

  6. Bickmore, T., Cassell, J.: Relational agents: a model and implementation of building user trust. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 396–403 (2001). http://dl.acm.org/citation.cfm?id=365304

  7. Tidwell, L.C., Walther, J.B.: Computer-mediated communication effects on disclosure, impressions, and interpersonal evaluations. Hum. Commun. Res. 28(3), 317–348 (2002)

    Article  Google Scholar 

  8. Nisimura, R., Nishihara, Y., Tsurumi, R., Lee, A., Saruwatari, H., Shikano, K.: Takemaru-kun.: Speech-oriented information system for real world research platform. In: Proceedings of the First International Workshop on Language Understanding and Agents for Real World Interaction, pp. 70–78 (2003). http://library.naist.jp:12180/dspace/handle/10061/8226

  9. Weizenbaum, J.: ELIZA—a computer program for the study of natural language communication between man and machine. Commun. ACM 9(1), 36–45 (Jan 1966). http://portal.acm.org/citation.cfm?doid=365153.365168

  10. Caspi, A., Roberts, B.W., Shiner, R.L.: Personality development: stability and change. Ann. Rev. Psychol. 56, 453–484 (Jan 2005). http://www.ncbi.nlm.nih.gov/pubmed/15709943

  11. John, O.P., Srivastava, S.: The Big Five trait taxonomy: history, measurement, and theoretical perspectives. No. 510 (1999). http://scholar.google.com/scholar?hl=en&btnG=Search&q=intitle:The+Big-Five+Trait+Taxonomy:+History,+Measurement,+and+Theoretical+Perspectives#0

  12. Mairesse, F., Walker, M.: PERSONAGE: Personality generation for dialogue. In: Proceedings of the Annual Meeting of the Association For Computational Linguistics, pp. 496–503 (2007). http://acl.ldc.upenn.edu/P/P07/P07-1063.pdf

  13. Batacharia, B., Levy, D., Catizone, R., Krotov, A., Wilks, Y.: CONVERSE: a conversational companion. Machine conversations, pp. 205–215 (1999). http://link.springer.com/chapter/10.1007/978-1-4757-5687-6_17

  14. Traum, D., Georgila, K., Artstein, R., Leuski, A.: Evaluating spoken dialogue processing for time-offset interaction. In: Proceedings of the 16th Annual SIGdial Meeting on Discourse and Dialogue, pp. 199–208 (2015)

    Google Scholar 

  15. Leuski, A., Traum, D.: Creating virtual human dialogue using information retrieval techniques. AI Mag. 32(2), 42–56 (2011). http://www.aaai.org/ojs/index.php/aimagazine/article/view/2347

  16. Sugiyama, H., Meguro, T., Higashinaka, R.: Large-scale collection and analysis of personal question-answer pairs for conversational agents. In: Proceedings of Intelligent Virtual Agents, pp. 420–433 (2014)

    Google Scholar 

  17. Sugiyama, H., Meguro, T., Higashinaka, R., Minami, Y.: Open-domain utterance generation using phrase pairs based on dependency relations. In: Proceedings of Spoken Language Technology Workshop, pp. 60–65 (2014)

    Google Scholar 

  18. Turian, J., Ratinov, L., Bengio, Y.: Word representations: A simple and general method for semi-supervised learning. In: Proceedings of the 48th Annual Meeting of the Association for Computational Linquistics. pp. 384–394. July (2010)

    Google Scholar 

  19. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Proceedings of the 27th Annual Conference on Neural Information Processing Systems, pp. 1–9 (2013)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. NTT Communication Science Laboratories, 2-4, Hikari-dai, Seika-cho, Souraku-gun, Kyoto, Japan

    Hiroaki Sugiyama, Toyomi Meguro & Ryuichiro Higashinaka

  2. NTT Media Intelligence Laboratories, 1-1, Hikari-no-oka, Yokosuka-shi, Kanagawa, Japan

    Ryuichiro Higashinaka

Authors
  1. Hiroaki Sugiyama
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Toyomi Meguro
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ryuichiro Higashinaka
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hiroaki Sugiyama .

Editor information

Editors and Affiliations

  1. Institute of Behavioural Sciences, University of Helsinki Institute of Behavioural Sciences, Helsinki, Finland

    Kristiina Jokinen

  2. University of Helsinki , Helsinki, Finland

    Graham Wilcock

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer Science+Business Media Singapore

About this chapter

Cite this chapter

Sugiyama, H., Meguro, T., Higashinaka, R. (2017). Evaluation of Question-Answering System About Conversational Agent’s Personality. In: Jokinen, K., Wilcock, G. (eds) Dialogues with Social Robots. Lecture Notes in Electrical Engineering, vol 427. Springer, Singapore. https://doi.org/10.1007/978-981-10-2585-3_14

Download citation

  • DOI: https://doi.org/10.1007/978-981-10-2585-3_14

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-10-2584-6

  • Online ISBN: 978-981-10-2585-3

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation