skip to main content
10.1145/3266037.3266110acmconferencesArticle/Chapter ViewAbstractPublication PagesuistConference Proceedingsconference-collections
poster

Engagement Learning: Expanding Visual Knowledge by Engaging Online Participants

Published: 11 October 2018 Publication History

Abstract

Most artificial intelligence (AI) systems to date have focused entirely on performance, and rarely if at all on their social interactions with people and how to balance the AIs' goals against their human collaborators'. Learning quickly from interactions with people poses both social challenges and is unresolved technically. In this paper, we introduce engagement learning: a training approach that learns to trade off what the AI needs---the knowledge value of a label to the AI---against what people are interested to engage with---the engagement value of the label. We realize our goal with ELIA (Engagement Learning Interaction Agent), a conversational AI agent who's goal is to learn new facts about the visual world by asking engaging questions of people about the photos they upload to social media. Our current deployment of ELIA on Instagram receives a response rate of 26%.

Supplementary Material

suppl.mov (upp1106.mp4)
Supplemental video

References

[1]
Robert B Cialdini and Robert B Cialdini. 2007. Influence: The psychology of persuasion. Collins New York.
[2]
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 248--255.
[3]
Kieran Healy and Alan Schussman. 2003. The ecology of open-source software development. Technical Report. Technical report, University of Arizona, USA.
[4]
Benjamin Mako Hill. 2013. Almost Wikipedia: Eight early Encyclopedia projects and the mechanisms of collective action. Massachusetts Institute of Technology (2013), 1--38.
[5]
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation 9, 8 (1997), 1735--1780.
[6]
Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, and others. 2017. Visual genome: Connecting language and vision using crowdsourced dense image annotations. International Journal of Computer Vision 123, 1 (2017), 32--73.
[7]
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems. 1097--1105.
[8]
David D Lewis and William A Gale. 1994. A sequential algorithm for training text classifiers. In Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval. Springer-Verlag New York, Inc., 3--12.
[9]
Christopher Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven Bethard, and David McClosky. 2014. The Stanford CoreNLP natural language processing toolkit. In Proceedings of 52nd annual meeting of the association for computational linguistics: system demonstrations. 55--60.
[10]
Ishan Misra, Ross Girshick, Rob Fergus, Martial Hebert, Abhinav Gupta, and Laurens van der Maaten. 2017. Learning by Asking Questions. arXiv preprint arXiv:1712.01238 (2017).
[11]
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. 2013. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 (2013).
[12]
Gina Neff and Peter Nagy. 2016. Talking to Bots: Symbiotic Agency and the Case of Tay. (2016).
[13]
Jan Peters and Stefan Schaal. 2008. Reinforcement learning of motor skills with policy gradients. Neural networks 21, 4 (2008), 682--697.
[14]
Justin Reich, Richard Murnane, and John Willett. 2012. The state of wiki usage in US K--12 schools: Leveraging Web 2.0 data warehouses to assess quality and equity in online learning environments. Educational Researcher 41, 1 (2012), 7--15.
[15]
Tobias Scheffer, Christian Decomain, and Stefan Wrobel. 2001. Active hidden markov models for information extraction. In International Symposium on Intelligent Data Analysis. Springer, 309--318.
[16]
Richard S Sutton and Andrew G Barto. 1998. Reinforcement learning: An introduction. Vol. 1. MIT press Cambridge.
[17]
Yongdong Wang. 2016. Your next new best friend might be a robot. (2016).
[18]
Xingdi Yuan, Tong Wang, Caglar Gulcehre, Alessandro Sordoni, Philip Bachman, Sandeep Subramanian, Saizheng Zhang, and Adam Trischler. 2017. Machine Comprehension by Text-to-Text Neural Question Generation. arXiv preprint arXiv:1705.02012 (2017).

Cited By

View all
  • (2023)Positive Artificial Intelligence in Education (P-AIED): A RoadmapInternational Journal of Artificial Intelligence in Education10.1007/s40593-023-00357-y34:3(732-792)Online publication date: 3-Aug-2023
  • (2021)Social Media as a Design and Research Site in HCI: Mapping Out Opportunities and Envisioning Future UsesExtended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411763.3441311(1-5)Online publication date: 8-May-2021

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
UIST '18 Adjunct: Adjunct Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology
October 2018
251 pages
ISBN:9781450359498
DOI:10.1145/3266037
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 October 2018

Check for updates

Author Tags

  1. computer vision
  2. engagement learning
  3. natural language generation
  4. reinforcement learning
  5. scene understanding

Qualifiers

  • Poster

Conference

UIST '18

Acceptance Rates

UIST '18 Adjunct Paper Acceptance Rate 80 of 375 submissions, 21%;
Overall Acceptance Rate 355 of 1,733 submissions, 20%

Upcoming Conference

UIST '25
The 38th Annual ACM Symposium on User Interface Software and Technology
September 28 - October 1, 2025
Busan , Republic of Korea

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)12
  • Downloads (Last 6 weeks)1
Reflects downloads up to 14 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2023)Positive Artificial Intelligence in Education (P-AIED): A RoadmapInternational Journal of Artificial Intelligence in Education10.1007/s40593-023-00357-y34:3(732-792)Online publication date: 3-Aug-2023
  • (2021)Social Media as a Design and Research Site in HCI: Mapping Out Opportunities and Envisioning Future UsesExtended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411763.3441311(1-5)Online publication date: 8-May-2021

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media