Skip to main content
SpringerLink
Log in
Book cover

Revolutionizing Education with Digital Ink pp 129–140Cite as

An Intelligent Sketch-Based Educational Interface for Learning Complex Written East Asian Phonetic Symbols

  • Chapter
  • First Online:

Part of the book series: Human–Computer Interaction Series ((HCIS))

Abstract

Literacy in written phonetic symbols for East Asian languages is important but challenging for novice language students with only English language fluency, but pedagogical approaches such as rote writing practice and written technique have successfully assisted students towards written mastery. Researchers and developers are also adapting these approaches into intelligent educational apps for students to exploit interactive computing technologies for learning written East Asian language phonetic symbols, whether they are full-time students in K12 or higher education enrolled in conventional classrooms or non-traditional students studying the subject as a passionate interest. However, related pen and touch educational computing apps for sketching practice of East Asian language phonetic symbols provide limited assessment and flexibility of students’ input and sketching style, while related recognition systems either focus more on expert users’ writing styles or cannot provide assessment for more complex phonetic symbols. In this article, we describe our preliminary work on an intelligent sketch-based educational interface developed specifically for assessing students’ sketched input of complex East Asian language phonetic symbols. The interface system relies on template matching from expert users’ sketched training data and various heuristics for assessing the visual structure and technical correctness of students’ more complex written phonetic symbols. From our evaluations of separate sketching data from both novice and expert writers, we were able to achieve reasonably robust performance for both visual structure and technical correctness of our workbook interface for complex written East Asian language phonetic symbols.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Anthony L, Wobbrock J (2012) \({\$}\)N-protractor: a fast and accurate multistroke recognizer. In: Proceedings of graphics interface 2012, GI ’12, Canadian Information Processing Society, Mississauga, pp 117–120

    Google Scholar 

  2. Cornerstone Studio: Hiragana - read and write [application] (2015). Accessed 30 June 2015

    Google Scholar 

  3. Dixon D, Prasad M, Hammond T (2010) iCanDraw: using sketch recognition and corrective feedback to assist a user in drawing human faces. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI ’10. ACM, New York, pp 897–906

    Google Scholar 

  4. Hammond T, Davis R (2003) LADDER: a language to describe drawing, display, and editing in sketch recognition. In: Proceedings of the 18th international joint conference on artificial intelligence, IJCAI ’03. AAAI, Palo Alto, pp 461–467

    Google Scholar 

  5. LaViola J Jr (2007) Advances in mathematical sketching: moving toward the paradigm’s full potential. IEEE Comput Graph Appl 27(1):38–48

    Article  Google Scholar 

  6. Legendarya/Imaginactiva: Hiragana - learn Japanese [application] (2015). Accessed 14 Jan 2016

    Google Scholar 

  7. Paulson B, Hammond T (2008) Paleosketch: accurate primitive sketch recognition and beautification. In: Proceedings of the 13th international conference on intelligent user interfaces, IUI ’08. ACM, New York, pp 1–10

    Google Scholar 

  8. Shahzad N, Paulson B, Hammond T (2009) Urdu qaeda: recognition system for isolated urdu characters. In: Proceedings of the workshop on sketch recognition at the 14th international conference of intelligent user interfaces, IUI-SR ’09

    Google Scholar 

  9. Sigma Sky, LLC: Write bopomofo [application] (2015). Accessed 14 Jan 2016

    Google Scholar 

  10. Taele P, Hammond T (2009) Hashigo: a next-generation sketch interactive system for Japanese kanji. In: Proceedings of the twenty-first innovative applications artificial intelligence conference, IAAI ’09. AAAI, Palo Alto, pp 153–158

    Google Scholar 

  11. Taele P, Hammond T (2010) Lamps: a sketch recognition-based teaching tool for mandarin phonetic symbols i. J Vis Lang Comput 21(2):109–120

    Article  Google Scholar 

  12. Taele P, Hammond T (2015) Enhancing instruction of written east asian languages with sketch recognition-based “intelligent language workbook” interfaces. In: Hammond T, Valentine S, Adler A, Payton M (eds) The impact of pen and touch technology on education. Springer, Berlin, pp 119–126

    Google Scholar 

  13. Taele P, Barreto L, Hammond T (2015) Maestoso: an intelligent educational sketching tool for learning music theory. In: Proceedings of the twenty-seventh innovative applications of artificial intelligence conference, IAAI ’15. AAAI, Palo Alto, pp 3999–4005

    Google Scholar 

  14. Valentine S, Field M, Hammond T, Smith A (2011) A shape comparison technique for use in sketch-based tutoring systems. In: Proceedings of the 2011 intelligent user interfaces workshop on sketch recognition, IUI SR ’11

    Google Scholar 

  15. Valentine S, Vides F, Lucchese G, Turner D, Kim Hh, Li W, Linsey J, Hammond T (2012) Mechanix: A sketch-based tutoring system for statics courses. In: Proceedings of the twenty-fourth innovative applications of artificial intelligence conference, IAAI ’12. AAAI, Palo Alto, pp 2253–2260

    Google Scholar 

  16. Valentine S, Lara-Garduno R, Linsey J, Hammond T (2015) Mechanix: a sketch based tutoring system that automatically corrects hand-sketched statics homework, The impact of pen and touch technology on education

    Google Scholar 

  17. Vatavu RD, Anthony L, Wobbrock JO (2012) Gestures as point clouds: a \({\$}\)p recognizer for user interface prototypes. In: Proceedings of the 14th ACM international conference on multimodal interaction. ACM, New York, pp 273–280

    Google Scholar 

  18. Wobbrock J, Wilson A, Li Y (2007) Gestures without libraries, toolkits or training: a \({\$}\)1 recognizer for user interface prototypes. In: Proceedings of the 20th annual ACM symposium on user interface software and technology, UIST ’07. ACM, New York, pp 159–168

    Google Scholar 

  19. Wolin A, Eoff B, Hammond T (2008) Shortstraw: a simple and effective corner finder for polylines. In: Proceedings of the fifth eurographics conference on sketch-based interfaces and modeling, SBIM ’08. Eurographics Association, Geneva, pp 33–40

    Google Scholar 

  20. Xiong Y, LaViola J Jr (2010) Technical section: a shortstraw-based algorithm for corner finding in sketch-based interfaces. Comput Graph 34(5):513–527

    Article  Google Scholar 

Download references

Acknowledgments

I would first like to give my great appreciation to the members of the Sketch Recognition Lab for sharing their many combined years of experience in shaping the direction of this research project. There have also been many influential East Asian language instructors in my life that allowed me to incorporate my language knowledge into intelligent sketch-based educational applications, especially to my Japanese language instructors Mr. George Adams and Ms. Kazue Kurokawa from Texas A&M University’s Department of International Studies, and my Chinese language instructor Dr. Wen-Hua Teng from the University of Texas’ Department of Asian Studies. Lastly, this project is funded in part by Microsoft (Microsoft Surface Hub Grant: Sketch and Gesture Recognition for Collaborative and Design Interfaces on the Surface Hub).

Author information

Authors and Affiliations

  1. Sketch Recognition Lab, Computer Science and Engineering Department, Texas A&M University, College Station, TX, USA

    Paul Taele & Tracy Hammond

Authors
  1. Paul Taele
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tracy Hammond
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Paul Taele .

Editor information

Editors and Affiliations

  1. Texas A&M University, College Station, Texas, USA

    Tracy Hammond

  2. Dept of Computer Science & Engineering, Texas A&M University, College Station, Texas, USA

    Stephanie Valentine

  3. Raytheon BBN Technologies, Cambridge, Massachusetts, USA

    Aaron Adler

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Taele, P., Hammond, T. (2016). An Intelligent Sketch-Based Educational Interface for Learning Complex Written East Asian Phonetic Symbols. In: Hammond, T., Valentine, S., Adler, A. (eds) Revolutionizing Education with Digital Ink. Human–Computer Interaction Series. Springer, Cham. https://doi.org/10.1007/978-3-319-31193-7_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-31193-7_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-31191-3

  • Online ISBN: 978-3-319-31193-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation