research-article

Simulating IBM Watson in the Classroom

Authors:

Wlodek W. Zadrozny,

Sean Gallagher,

Adarsh AvadhaniAuthors Info & Claims

SIGCSE '15: Proceedings of the 46th ACM Technical Symposium on Computer Science Education

Pages 72 - 77

https://doi.org/10.1145/2676723.2677287

Published: 24 February 2015 Publication History

Abstract

IBM Watson exemplifies multiple innovations in natural language processing and question answering. In addition, Watson uses most of the known techniques in these two domains as well as many methods from related domains. Hence, there is pedagogical value in a rigorous understanding of its function. The paper provides the description of a text analytics course focused on building a simulator of IBM Watson, conducted in Spring 2014 at UNC Charlotte. We believe this is the first time a simulation containing all the major Watson components was created in a university classroom. The system achieved a respectable (close to) 20% accuracy on Jeopardy! questions, and there remain many known and new avenues of improving performance that can be explored in the future. The code and documentation are available on GitHub. The paper is a joint effort of the teacher and some of the students who were leading teams implementing component technologies, and therefore deeply involved in making the class successful.

References

[1]

Deep blue (chess computer). http://en.wikipedia.org/wiki/Deep_Blue (chess computer).

[2]

Excitement Open Platform. http://hltfbk.github.io/Excitement-Open-Platform/.

[3]

"IBM J. RES. and DEV" 56 (3.4). 2012.

[4]

C. Biemann et al. Jobimtext project. Retrieved September 03 2014 from http://sourceforge.net/projects/jobimtext/, 2014.

[5]

G. Booch. Watson under the hood: Booch Describes Jeopardy Champ's System's Architecture. Retrieved September 03 2014 from http://searchsoa.techtarget.com/news/2240038723/Watson-under-the-hood-Booch-describes-Jeopardy-champs-systems-architecture.

[6]

E. Brown, E. Epstein, J. W. Murdock, and T.-H. Fin. Tools and methods for building Watson. IBM Research Report RC 25356.

[7]

E. Brown, D. Ferrucci, A. Lally, and W. W. Zadrozny. System and method for providing answers to questions, September 2012. US Patent 8,275,803.

[8]

M. Campbell, A. J. Hoane Jr, and F. Hsu. Deep Blue. Artificial intelligence, 134(1):57--83, 2002.

Digital Library

[9]

J. Chu-Carroll, J. Fan, N. Schlaefer, and W. Zadrozny. Textual resource acquisition and engineering. IBM Journal of Research and Development, 56(3.4):4:1--4:11, May 2012.

Digital Library

[10]

E. A. Epstein, M. I. Schor, B. S. Iyer, A. Lally, E. Brown, and J. Cwiklik. Making Watson fast. IBM Journal of Research and Development, 56(3.4):15:1--15:12, May 2012.

Digital Library

[11]

J. Fan, D. Ferrucci, D. C. Gondek, and W. W. Zadrozny. System and method for providing questions and answers with deferred type evaluation, December 2012. US Patent 8,332,394.

[12]

D. Ferrucci et al. Building Watson: An overview of the DeepQA project. AI Magazine, 31(3), 2010.

[13]

D. Ferrucci and A. Lally. UIMA: an architectural approach to unstructured information processing in the corporate research environment. Natural Language Engineering, 10(3-4):327--348, 2004.

Digital Library

[14]

D. A. Ferrucci et al. Introduction to "This is Watson". IBM J. RES. and DEV, 56(3), 2012.

Digital Library

[15]

D. Fisher. Indri retrieval model. Retrieved September 03 2014 from http://sourceforge.net/p/lemur/wiki/Indri Retrieval Model/, 2012.

[16]

R. Gaizauskas, H. Cunningham, Y. Wilks, P. Rodgers, and K. Humphreys. GATE: an environment to support research and development in natural language engineering. In Tools with Artificial Intelligence, 1996., Proceedings Eighth IEEE International Conference on, pages 58--66. IEEE, 1996.

Digital Library

[17]

S. Gallagher, W. Zadrozny, W. Shalaby, and A. Avadhani. Watsonsim: Overview of a question answering engine. arxiv.org, 2014.

[18]

A. Gliozzo, O. Biran, S. Patwardhan, and K. McKeown. Semantic technologies in IBM Watson. ACL 2013, http://www.aclweb.org/anthology/W13--3413, page 85, 2013.

[19]

M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I. H. Witten. The WEKA data mining software: an update. ACM SIGKDD explorations newsletter, 11(1):10--18, 2009.

Digital Library

[20]

J. Hendler. Watson at RPI. Retrieved September 03 2014 from http://www.slideshare.net/jahendler/watson-summer-review82013final.

[21]

J. Hendler and S. Ells. Retrieved September 03 2014 from http://www.slideshare.net/jahendler/why-watson-won-a-cognitive-perspective?nextslideshow=1.

[22]

IBM. IBM Research: Cognitive Computing. Retrieved September 03 2014 from http://www.research.ibm.com/cognitive-computing/.

[23]

A. Lally et al. Question analysis: How Watson reads a clue. IBM J. RES. and DEV, 56(3/4), 2012.

Digital Library

[24]

T. Mikolav et al. Word2Vec project. Retrieved September 03 2014 from http://code.google.com/p/word2vec/, 2014.

[25]

R. Muir et al. Lucene Similarity class reference. Retrieved September 03 2014 from http://lucene.apache.org/core/4_9_0/core/org/apache/lucene/search/similarities/TFIDFSimilarity.html, 2014.

[26]

J. W. Murdock et al. Textual evidence gathering and analysis. IBM J. RES. and DEV, 56(3.4), 2012.

Digital Library

[27]

D. A. Patterson. Computer science education in the 21st century. Commun. ACM, 49(3):27--30, Mar. 2006.

Digital Library

[28]

N. Schlaefer, J. Chu-Carroll, E. Nyberg, J. Fan, W. Zadrozny, and D. Ferrucci. Statistical source expansion for question answering. In Proceedings of the 20th ACM International Conference on Information and Knowledge Management, CIKM '11, pages 345--354, New York, NY, USA, 2011. ACM.

Digital Library

[29]

A. Trotman, C. L. Clarke, I. Ounis, S. Culpepper, M.-A. Cartright, and S. Geva. Open Source Information Retrieval: A Report on the SIGIR 2012 Workshop. SIGIR Forum, 46(2):95--101, 2012.

Digital Library

[30]

C. Wang et al. Relation extraction and scoring in DeepQA. IBM J. RES. and DEV, 56(3/4), 2012.

Digital Library

[31]

M. Wollowski. Teaching with Watson. Fifth AAAI Symposium on Educational Advances in Artificial Intelligence, http://www.aaai.org/ocs/index.php/EAAI/EAAI14/paper/view/8584/8677, 2014.

Cited By

Radanliev P(2024)Artificial intelligence: reflecting on the past and looking towards the next paradigm shiftJournal of Experimental & Theoretical Artificial Intelligence10.1080/0952813X.2024.2323042(1-18)Online publication date: 28-Feb-2024
https://doi.org/10.1080/0952813X.2024.2323042
Fountis ALemonakis C(2024)Artificial Intelligence (ΑΙ) in Education—Current TrendsProceedings of Second International Conference on Intelligent System10.1007/978-981-99-8976-8_18(191-201)Online publication date: 12-Apr-2024
https://doi.org/10.1007/978-981-99-8976-8_18
Khazanchi RKhazanchi P(2021)Artificial Intelligence in EducationHandbook of Research on Critical Issues in Special Education for School Rehabilitation Practices10.4018/978-1-7998-7630-4.ch014(256-277)Online publication date: 2021
https://doi.org/10.4018/978-1-7998-7630-4.ch014
Show More Cited By

Index Terms

Simulating IBM Watson in the Classroom
1. Social and professional topics
  1. Professional topics
    1. Computing education
      1. Computing education programs
        Computer science education
      2. Model curricula

Recommendations

ConversIAmo: Improving Italian Question Answering Exploiting IBM Watson Services
Text, Speech, and Dialogue
Abstract
Chatbots, conversational interfaces and NLP have achieved considerable improvements and are spreading more and more in everyday applications. Solutions on the market allow their implementation easily in different languages, but the proposals for ...
Ibm websphere portal for multiplatforms v5.1 handbook
IBM Pseries Certification Study Guide Update

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGCSE '15: Proceedings of the 46th ACM Technical Symposium on Computer Science Education

February 2015

766 pages

ISBN:9781450329668

DOI:10.1145/2676723

General Chairs:
Adrienne Decker
Rochester Institute of Technology
,
Kurt Eiselt
University of British Columbia
,
Program Chairs:
Carl Alphonce
University at Buffalo
,
Jodi Tims
Baldwin Wallace College

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCSE: ACM Special Interest Group on Computer Science Education

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 February 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGCSE '15

Sponsor:

SIGCSE

SIGCSE '15: The 46th ACM Technical Symposium on Computer Science Education

March 4 - 7, 2015

Missouri, Kansas City, USA

Acceptance Rates

SIGCSE '15 Paper Acceptance Rate 105 of 289 submissions, 36%;

Overall Acceptance Rate 1,787 of 5,146 submissions, 35%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
588
Total Downloads

Downloads (Last 12 months)25
Downloads (Last 6 weeks)2

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Radanliev P(2024)Artificial intelligence: reflecting on the past and looking towards the next paradigm shiftJournal of Experimental & Theoretical Artificial Intelligence10.1080/0952813X.2024.2323042(1-18)Online publication date: 28-Feb-2024
https://doi.org/10.1080/0952813X.2024.2323042
Fountis ALemonakis C(2024)Artificial Intelligence (ΑΙ) in Education—Current TrendsProceedings of Second International Conference on Intelligent System10.1007/978-981-99-8976-8_18(191-201)Online publication date: 12-Apr-2024
https://doi.org/10.1007/978-981-99-8976-8_18
Khazanchi RKhazanchi P(2021)Artificial Intelligence in EducationHandbook of Research on Critical Issues in Special Education for School Rehabilitation Practices10.4018/978-1-7998-7630-4.ch014(256-277)Online publication date: 2021
https://doi.org/10.4018/978-1-7998-7630-4.ch014
Dalal PAggarwal GTejasvee S(2021)Preeminent Development Boards to Design Sustainable Integrated Model of a Smart Healthcare System under IoTIOP Conference Series: Materials Science and Engineering10.1088/1757-899X/1022/1/0120041022(012004)Online publication date: 19-Jan-2021
https://doi.org/10.1088/1757-899X/1022/1/012004
Müller SBergande BBrune PMottok J(2018)Robot TutoringProceedings of the 3rd European Conference of Software Engineering Education10.1145/3209087.3209093(45-49)Online publication date: 14-Jun-2018
https://dl.acm.org/doi/10.1145/3209087.3209093
Gallagher SZadrozny W(2016)Leveraging Large Corpora Using Internet Search for Question Answering2016 IEEE/WIC/ACM International Conference on Web Intelligence (WI)10.1109/WI.2016.0090(532-535)Online publication date: Oct-2016
https://doi.org/10.1109/WI.2016.0090
Murtaza SLak PBener APischdotchian A(2016)How to Effectively Train IBM WatsonProceedings of the 2016 49th Hawaii International Conference on System Sciences (HICSS)10.1109/HICSS.2016.210(1663-1670)Online publication date: 5-Jan-2016
https://dl.acm.org/doi/10.1109/HICSS.2016.210

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten