Skip to main content
Springer Nature Link
Log in

End-to-End Memory Networks: A Survey

  • Conference paper
  • First Online:
Intelligent Computing (SAI 2020)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 1229))

Included in the following conference series:

  • 1172 Accesses

Abstract

Constructing a dialog system which can speak naturally with a human is considered as a major challenge of artificial intelligence. End-to-end dialog system is taken to be a primary research topic in the area of conversational systems. Since an end-to-end dialog system is structured based on learning a dialog policy from transactional dialogs in a defined extent, therefore, useful datasets are required for evaluating the learning procedures. In this paper, different deep learning techniques are applied to the Dialog bAbI datasets. On this dataset, the performance of the proposed techniques is analyzed. The performance results demonstrate that all the proposed techniques attain decent precisions on the Dialog bAbI datasets. The best performance is obtained utilizing end-to-end memory network with a unified weight tying scheme (UN2N).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Araujo, T.: Living up to the chatbot hype: The influence of anthropomorphic design cues and communicative agency framing on conversational agent and company perceptions. Comput. Hum. Behav. 85, 183–189 (2018)

    Article  Google Scholar 

  2. Hill, J., Ford, W.R., Farreras, I.G.: Real conversations with artificial intelligence: a comparison between human-human online conversations and human-chatbot conversations. Comput. Hum. Behav. 49, 245–250 (2015)

    Article  Google Scholar 

  3. Quarteroni, S.: A chatbot-based interactive question answering system. In: 11th Workshop on the Semantics and Pragmatics of Dialogue, pp. 83–90 (2007)

    Google Scholar 

  4. Young, S., Gasic, M., Thomson, B., Williams, J.D.: POMDP-based statistical spoken dialog systems: a review. Proc. IEEE 101, 1160–1179 (2013)

    Article  Google Scholar 

  5. Shawar, B.A., Atwell, E.: Chatbots: are they really useful? LDV Forum 22, 29–49 (2007)

    Google Scholar 

  6. Dote, Y., Hoft, R.G.: Intelligent Control Power Electronics Systems. Oxford Univ. Press, Oxford (1998)

    Google Scholar 

  7. Mohanty, S.: Estimation of vapour liquid equilibria for the system carbon dioxide-difluoromethane using artificial neural networks. Int. J. Refrig. 29, 243249 (2006)

    Article  Google Scholar 

  8. Razvarz, S., Jafari, R., Yu, W., Khalili, A.: PSO and NN Modeling for photocatalytic removal of pollution in wastewater. In: 14th International Conference on Electrical Engineering, Computing Science and Automatic Control (CCE) Electrical Engineering, pp. 1–6 (2017)

    Google Scholar 

  9. Jafari, R., Yu, W.: Artificial neural network approach for solving strongly degenerate parabolic and burgers-fisher equations. In: 12th International Conference on Electrical Engineering, Computing Science and Automatic Control (2015). https://doi.org/10.1109/ICEEE.2015.7357914

  10. Jafari, R., Razvarz, S., Gegov, A.: A new computational method for solving fully fuzzy nonlinear systems. In: Computational Collective Intelligence. ICCCI 2018. Lecture Notes in Computer Science, vol. 11055, pp. 503–512. Springer, Cham (2018)

    Google Scholar 

  11. Razvarz, S., Jafari, R.: ICA and ANN modeling for photocatalytic removal of pollution in wastewater. Math. Comput. Appl. 22, 38–48 (2017)

    Google Scholar 

  12. Razvarz, S., Jafari, R., Gegov, A., Yu, W., Paul, S.: Neural network approach to solving fully fuzzy nonlinear systems. In: Fuzzy modeling and control Methods Application and Research, pp. 45–68. Nova science publisher Inc., New York (2018). ISBN: 978-1-53613-415-5

    Google Scholar 

  13. Razvarz, S., Jafari, R.: Intelligent techniques for photocatalytic removal of pollution in wastewater. J. Elect. Eng. 5, 321–328 (2017). https://doi.org/10.17265/2328-2223/2017.06.004

    Article  Google Scholar 

  14. Graupe, D.: Chapter 112. In: Chen, W., Mlynski, D.A. (eds.): Principles of Artificial Neural Networks. Advanced Series in Circuits and Systems, 1st edn. vol. 3, p. 4e189. World Scientific (1997)

    Google Scholar 

  15. Jafari, R., Yu, W., Li, X.: Solving fuzzy differential equation with Bernstein neural networks. In: IEEE International Conference on Systems, Man, and Cybernetics, Budapest, Hungary, pp. 1245–1250 (2016)

    Google Scholar 

  16. Jafari, R. Yu, W.: Uncertain nonlinear system control with fuzzy differential equations and Z-numbers. In: 18th IEEE International Conference on Industrial Technology, Canada, pp. 890–895 (2017). https://doi.org/10.1109/ICIT.2017.7915477

  17. Jafarian, A., Measoomy, N.S., Jafari, R.: Solving fuzzy equations using neural nets with a new learning algorithm. J. Adv. Comput. Res. 3, 33–45 (2012)

    Google Scholar 

  18. Werbos, P.J.: Neuro-control and elastic fuzzy logic: capabilities, concepts, and applications. IEEE Trans. Ind. Electron. 40, 170180 (1993)

    Article  Google Scholar 

  19. Jafari, R., Yu, W., Razvarz, S., Gegov, A.: Numerical methods for solving fuzzy equations: a Survey. Fuzzy Sets Syst. (2019). ISSN 0165-0114, https://doi.org/10.1016/j.fss.2019.11.003

  20. Kim, J.H., Kim, K.S., Sim, M.S., Han, K.H., Ko, B.S.: An application of fuzzy logic to control the refrigerant distribution for the multi type air conditioner. In: Proceedings of IEEE International Fuzzy Systems Conference, vol. 3, pp. 1350–1354 (1999)

    Google Scholar 

  21. Wakami, N., Araki, S., Nomura, H.: Recent applications of fuzzy logic to home appliances. In: Proceedings of IEEE International Conference on Industrial Electronics, Control, and Instrumentation, Maui, HI, pp. 155–160 (1993)

    Google Scholar 

  22. Jafari, R., Razvarz, S.: Solution of fuzzy differential equations using fuzzy Sumudu transforms. In: IEEE International Conference on Innovations in Intelligent Systems and Applications, pp. 84–89 (2017)

    Google Scholar 

  23. Jafari, R., Razvarz, S., Gegov, A., Paul, S.: Fuzzy modeling for uncertain nonlinear systems using fuzzy equations and Z-numbers. In: Advances in Computational Intelligence Systems: Contributions Presented at the 18th UK Workshop on Computational Intelligence, Advances in Intelligent Systems and Computing, 5–7 September 2018, Nottingham, UK, vol. 840, pp. 66–107. Springer, Cham (2018)

    Google Scholar 

  24. Jafari, R., Razvarz, S.: Solution of fuzzy differential equations using fuzzy Sumudu transforms. Math. Comput. Appl. 23, 1–15 (2018)

    MathSciNet  MATH  Google Scholar 

  25. Jafari, R., Razvarz, S., Gegov, A.: Solving differential equations with z-numbers by utilizing fuzzy Sumudu transform. In: Intelligent Systems and Applications. IntelliSys 2018. Advances in Intelligent Systems and Computing, vol. 869, pp. 1125-1138. Springer, Cham (2019)

    Google Scholar 

  26. Yu, W., Jafari, R.: Modeling and Control of Uncertain Nonlinear Systems with Fuzzy Equations and Z-Number. IEEE Press Series on Systems Science and Engineering. Wiley-IEEE Press, John Wiley & Sons, Inc., Hoboken (2019). ISBN-13: 978-1119491552

    Google Scholar 

  27. Negoita, C.V., Ralescu, D.A.: Applications of Fuzzy Sets to Systems Analysis. Wiley, New York (1975)

    Book  Google Scholar 

  28. Zadeh, L.A.: Probability measures of fuzzy events. J. Math. Anal. Appl. 23, 421–427 (1968)

    Article  MathSciNet  Google Scholar 

  29. Zadeh, L.A.: Calculus of fuzzy restrictions. In: Fuzzy Sets and Their Applications to Cognitive and Decision Processes, pp. 1-39. Academic Press, New York (1975)

    Google Scholar 

  30. Zadeh, L.A.: Fuzzy logic and the calculi of fuzzy rules and fuzzy graphs. Multiple Valued Logic 1, 1–38 (1996)

    MATH  Google Scholar 

  31. Razvarz, S., Jafari, R.: Experimental study of Al2O3 nanofluids on the thermal efficiency of curved heat pipe at different tilt angle. J. Nanomater. 2018, 1–7 (2018)

    Article  Google Scholar 

  32. Razvarz, S., Vargas-Jarillo, C., Jafari, R.: Pipeline monitoring architecture based on observability and controllability analysis. In: IEEE International Conference on Mechatronics (ICM), Ilmenau, Germany, vol. 1, pp. 420–423 (2019). https://doi.org/10.1109/ICMECH.2019.872287

  33. Razvarz, S., Vargas-jarillo, C., Jafari, R., Gegov, A.: Flow control of fluid in pipelines using PID controller. IEEE Access 7, 25673–25680 (2019). https://doi.org/10.1109/ACCESS.2019.2897992

    Article  Google Scholar 

  34. Razvarz, S., Jafari, R.: Experimental study of Al2O3 nanofluids on the thermal efficiency of curved heat pipe at different tilt angle. In: 2nd International Congress on Technology Engineering and Science, ICONTES, Malaysia (2016)

    Google Scholar 

  35. Jafari, R., Razvarz, S., Gegov, A.: Neural network approach to solving fuzzy nonlinear equations using Z-numbers. IEEE Trans. Fuzzy Syst. (2019). https://doi.org/10.1109/TFUZZ.2019.2940919

    Article  Google Scholar 

  36. Jafari, R., Yu, W., Li, X., Razvarz, S.: Numerical solution of fuzzy differential equations with Z-numbers using Bernstein neural networks. Int. J. Comput. Intell. Syst. 10, 1226–1237 (2017)

    Article  Google Scholar 

  37. Jafari, R., Yu, W., Li, X.: Numerical solution of fuzzy equations with Z-numbers using neural networks. In: Intelligent Automation and Soft Computing, pp. 1–7 (2017)

    Google Scholar 

  38. Sutskever, I., Martens, J., Hinton, G.E.: Generating text with recurrent neural networks. In: Proceedings of ICML-2011 (2011)

    Google Scholar 

  39. Vinyals, O., Toshev, A., Bengio, S., Erhan, D.: Show and tell: a neural image caption generator. In: Proceedings of CVPR-2015 (2015)

    Google Scholar 

  40. Liu, F., Perez, J.: Gated end-to-end memory networks. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers. Association for Computational Linguistics, Valencia, Spain, pp. 1–10 (2017)

    Google Scholar 

  41. Bordes, A., Weston, J.: Learning end-to-end goal-oriented dialog, arXiv preprint. arXiv:1605.07683 (2016)

  42. Williams, J.D., Zweig, G.: End-to-end LSTM-based dialog control optimized with supervised and reinforcement learning, arXiv preprint arXiv:1606.01269 (2016)

  43. Kumar, A., Irsoy, O., Su, J., Bradbury, J., English, R., Pierce, B., Ondruska, P., Gulrajani, I., Socher, R.: Ask me anything: dynamic memory networks for natural language processing. In: Proceedings of ICML-2016 (2016)

    Google Scholar 

  44. Weston, J., Chopra, S., Bordes, A.: Memory networks. In: International Conference on Learning Representations (ICLR) (2015)

    Google Scholar 

  45. Dodge, J., Gane, A., Zhang, X., Bordes, A., Chopra, S., Miller, A.H., Szlam, A., Weston, J.: Evaluating prerequisite qualities for learning end-to-end dialog systems. In: Proceedings of ICLR-2016 (2016)

    Google Scholar 

  46. Henderson, M., Thomson, B., Young, S.: Word-based dialog state tracking with recurrent neural networks. In: Proceedings of SIGDIAL-2014 (2014)

    Google Scholar 

  47. Sukhbaatar, S., Szlam, A., Weston, J., Fergus, R.: End-to-end memory networks. In: Proceedings of Advances in Neural Information Processing Systems (NIPS 2015), Montreal, Canada, pp. 2440–2448 (2015)

    Google Scholar 

  48. Liu, F., Cohn, T., Baldwin, T.: Improving end-to-end memory networks with unified weight tying. In: Proceedings of the 15th Annual Workshop of The Australasian Language Technology Association (ALTW 2017), Brisbane, Australia, pp. 16–24 (2017)

    Google Scholar 

  49. Henderson, M., Thomson, B., Williams, J.D.: The second dialog state tracking challenge. In: Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL 2014), Philadelphia, USA, pp. 263–272 (2014)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Design, University of Leeds, Leeds, LS2 9JT, UK

    Raheleh Jafari

  2. Departamento de Control Automático, CINVESTAV-IPN (National Polytechnic Institute), Mexico City, Mexico

    Sina Razvarz

  3. School of Computing, University of Portsmouth, Buckingham Building, Portsmouth, PO1 3HE, UK

    Alexander Gegov

Authors
  1. Raheleh Jafari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sina Razvarz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alexander Gegov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Raheleh Jafari .

Editor information

Editors and Affiliations

  1. Faculty of Science and Engineering, Saga University, Saga, Japan

    Kohei Arai

  2. The Science and Information (SAI) Organization, Bradford, West Yorkshire, UK

    Supriya Kapoor

  3. The Science and Information (SAI) Organization, Bradford, West Yorkshire, UK

    Rahul Bhatia

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Jafari, R., Razvarz, S., Gegov, A. (2020). End-to-End Memory Networks: A Survey. In: Arai, K., Kapoor, S., Bhatia, R. (eds) Intelligent Computing. SAI 2020. Advances in Intelligent Systems and Computing, vol 1229. Springer, Cham. https://doi.org/10.1007/978-3-030-52246-9_20

Download citation

Publish with us

Policies and ethics