Skip to main content
SpringerLink
Log in
Book cover

Dialogues with Social Robots pp 257–269Cite as

HELPR: A Framework to Break the Barrier Across Domains in Spoken Dialog Systems

  • Chapter
  • First Online:

Part of the book series: Lecture Notes in Electrical Engineering ((LNEE,volume 427))

Abstract

People usually interact with intelligent agents (IAs) when they have certain goals to be accomplished. Sometimes these goals are complex and may require interacting with multiple applications, which may focus on different domains. Current IAs may be of limited use in such cases and the user needs to directly manage the task at hand. An ideal personal agent would be able to learn, over time, these tasks spanning different resources. In this article, we address the problem of cross-domain task assistance in the context of spoken dialog systems, and describe our approach about discovering such tasks and how IAs learn to talk to users about the task being carried out. Specifically we investigate how to learn user activity patterns in a smartphone environment that span multiple apps and how to incorporate users’ descriptions about their high-level intents into human-agent interaction.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    Dataset: http://appdialogue.com.

  2. 2.

    \(F_1=2\times Precision\times Recall/(Precision+Recall)\).

  3. 3.

    Toolkit: https://radimrehurek.com/gensim/models/word2vec.html.

  4. 4.

    Model: https://drive.google.com/file/d/0B7XkCwpI5KDYNlNUTTlSS21pQmM/edit?usp=sharing.

  5. 5.

    Toolkit: https://www.airpair.com/nlp/keyword-extraction-tutorial.

  6. 6.

    MAP@K computed as: \(\sum _{k=1}^K {precision(k)*relevance(k)} / K\).

References

  1. Harrison, C., Xiao, R., Schwarz, J., Hudson, S.E.: Touchtools: leveraging familiarity and skill with physical tools to augment touch interaction. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2913–2916 (2014)

    Google Scholar 

  2. Sun, M., Chen, Y.N., Rudnicky, A.I.: Understanding user’s cross-domain intentions in spoken dialog systems. In: Proceedings of NIPS workshop on Machine Learning for SLU and Interaction (2015)

    Google Scholar 

  3. Chen, Y.N., Sun, M., Rudnicky, A.I.: Leveraging behavioral patterns of mobile applications for personalized spoken language understanding. In: Proceedings of the 18th ACM International Conference on Multimodal Interaction (ICMI), pp. 83–86 (2015)

    Google Scholar 

  4. Young, S.: Using POMDPs for dialog management. In: Proceedings of the IEEE Spoken Language Technology Workshop (SLT), pp. 8–13 (2006)

    Google Scholar 

  5. Sun, M., Rudnicky, A.I., Winter, U.: User adaptation in automotive environments. In: Proceedings of the 4th International Conference on Advances in Human-Factors and Ergonomics (AHFE), pp. 156–165 (2012)

    Google Scholar 

  6. Pappu, A., Sun, M., Sridharan, S., Rudnicky, A.I.: Situated multiparty interactions between humans and agents. In: Proceedings of the 15th International Conference on Human-Computer Interaction: Interaction Modalities and Techniques, pp. 107–116 (2013)

    Google Scholar 

  7. Lunati, J.M., Rudnicky, A.I.: Spoken language interfaces: the OM system. In: Proceedings of the Conference on Human Factors in Computing Systems, pp. 453–454 (1991)

    Google Scholar 

  8. Rudnicky, A.I., Lunati, J.M., Franz, A.M.: Spoken language recognition in an office management domain. In: Proceedings of the International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pp. 829–832. IEEE (1991)

    Google Scholar 

  9. Lin, B.S., Wang, H.M., Lee, L.S.: A distributed architecture for cooperative spoken dialogue agents with coherent dialogue state and history. In: Proceedings of the Automatic Speech Recognition and Understanding Workshop (ASRU), vol. 99, p. 4 (1999)

    Google Scholar 

  10. Nakano, M., Sato, S., Komatani, K., Matsuyama, K., Funakoshi, K., Okuno, H.G.: A two-stage domain selection framework for extensible multi-domain spoken dialogue systems. In: Proceedings of the 12th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL), pp. 18–29. ACL (2011)

    Google Scholar 

  11. Chen, Y.N., Rudnicky, A.I.: Dynamically supporting unexplored domains in conversational interactions by enriching semantics with neural word embeddings. In: Proceedings of the IEEE Spoken Language Technology Workshop (SLT), pp. 590–595. IEEE (2014)

    Google Scholar 

  12. Li, Q., Tür, G., Hakkani-Tür, D., Li, X., Paek, T., Gunawardana, A., Quirk, C.: Distributed open-domain conversational understanding framework with domain independent extractors. In: Proceedings of the IEEE Spoken Language Technology Workshop (SLT), pp. 566–571. IEEE (2014)

    Google Scholar 

  13. Ryu, S., Lee, D., Lee, I., Han, S., Lee, G.G., Kim, M., Kim, K.: A hierarchical domain model-based multi-domain selection framework for multi-domain dialog systems. In: Proceedings of the 24th International Conference on Computational Linguistics (ACL) (2012)

    Google Scholar 

  14. Ryu, S., Song, J., Koo, S., Kwon, S., Lee, G.G.: Detecting multiple domains from users utterance in spoken dialog system. In: Proceedings of the 6th International Workshop on Spoken Dialogue Systems (IWSDS), pp. 101–111 (2015)

    Google Scholar 

  15. Sun, M., Chen, Y.N., Hua, Z., Tamres-Rudnicky, Y., Dash, A., Rudnicky, A.I.: AppDialogue: Multi-app dialogues for intelligent assistants. In: Proceedings of the 10th International Conference on Language Resources and Evaluation (LREC) (2016)

    Google Scholar 

  16. Lucchese, C., Orlando, S., Perego, R., Silvestri, F., Tolomei., G.: Identifying task-based sessions in search engine query logs. In: Proceedings of the 4th ACM International Conference on Web Search and Data Mining, pp. 277–286. ACM (2011)

    Google Scholar 

  17. Stenetorp, P., Pyysalo, S., Topić, G., Ohta, T., Ananiadou, S., Tsujii, J.: BRAT: a web-based tool for NLP-assisted text annotation. In: Proceedings of the Demonstrations at the 13th Conference of the European Chapter of the Association for Computational Linguistics (EACL), pp. 102–107. Association for Computational Linguistics (2012)

    Google Scholar 

  18. Bohus, D., Rudnicky, A.I.: Sorry, I didn’t catch that!-an investigation of non-understanding errors and recovery strategies. In: Proceedings of the 6th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL) (2005)

    Google Scholar 

  19. Shen, X., Tan, B., Zhai., C.: Implicit user modeling for personalized search. In: Proceedings of the 14th ACM International Conference on Information and Knowledge Management, pp. 824–831 (2005)

    Google Scholar 

  20. Sun, M., Chen, Y.N., Rudnicky., A.I.: Learning OOV through semantic relatedness in spoken dialog systems. In: Proceedings of the 16th Annual Conference of the International Speech Communication Association (Interspeech), pp. 1453–1457 (2015)

    Google Scholar 

  21. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. In: Proceedings of the Workshop at International Conference on Learning Representations (ICLR) (2013)

    Google Scholar 

  22. Tibshirani, R., Walther, G., Hastie., T.: Estimating the number of clusters in a data set via the gap statistic. J. R. Stat. Soc.: Series B (Stat. Method.) 63, 411–423 (2001)

    Google Scholar 

  23. Duda, R., Hart, P., Stork, D.: Pattern Classification. Wiley (2012)

    Google Scholar 

  24. Fiscus, J.G.: A post-processing system to yield reduced word error rates: recognizer output voting error reduction (ROVER). In: Proceedings of the Automatic Speech Recognition and Understanding Workshop (ASRU), pp. 347–352 (1997)

    Google Scholar 

  25. Ganesan, K., Zhai, C., Han, J.: Opinosis: a graph-based approach to abstractive summarization of highly redundant opinions. In: Proceedings of the 23rd International Conference on Computational Linguistics (COLING), pp. 340–348. ACL (2010)

    Google Scholar 

  26. Liu, F., Flanigan, J., Thomson, S., Sadeh, N., Smith, N.A.: Toward abstractive summarization using semantic representations. In: Proceedings of the 14th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologie (NAACL-HLT), pp. 1077–1086 (2015)

    Google Scholar 

  27. Witten, I.H., Paynter, G.W., Frank, E., Gutwin, C., Nevill-Manning, C.G.: KEA: practical automatic keyphrase extraction. In: Proceedings of the 4th ACM Conference on Digital Libraries, pp. 254–255 (1999)

    Google Scholar 

  28. Medelyan, O.: Human-competitive automatic topic indexing. Ph.D. thesis, University of Waikato (2009)

    Google Scholar 

  29. Berry, M.W., Kogan., J.: Text Mining: Applications and Theory. Wiley (2010)

    Google Scholar 

  30. Mihalcea, R., Tarau, P.: TextRank: bringing order into texts. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 404–411. ACL (2004)

    Google Scholar 

  31. Hulth, A.: Improved automatic keyword extraction given more linguistic knowledge. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 216–223. ACL (2003)

    Google Scholar 

Download references

Acknowledgements

This work was supported in part by Yahoo! InMind, and by the General Motors Advanced Technical Center.

Author information

Authors and Affiliations

  1. School of Computer Science, Carnegie Mellon University, Pittsburgh, USA

    Ming Sun, Yun-Nung Chen & Alexander I. Rudnicky

Authors
  1. Ming Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yun-Nung Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alexander I. Rudnicky
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ming Sun .

Editor information

Editors and Affiliations

  1. Institute of Behavioural Sciences, University of Helsinki Institute of Behavioural Sciences, Helsinki, Finland

    Kristiina Jokinen

  2. University of Helsinki , Helsinki, Finland

    Graham Wilcock

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer Science+Business Media Singapore

About this chapter

Cite this chapter

Sun, M., Chen, YN., Rudnicky, A.I. (2017). HELPR: A Framework to Break the Barrier Across Domains in Spoken Dialog Systems. In: Jokinen, K., Wilcock, G. (eds) Dialogues with Social Robots. Lecture Notes in Electrical Engineering, vol 427. Springer, Singapore. https://doi.org/10.1007/978-981-10-2585-3_20

Download citation

  • DOI: https://doi.org/10.1007/978-981-10-2585-3_20

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-10-2584-6

  • Online ISBN: 978-981-10-2585-3

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation