Tracy Hammond
Randall Davis
ABSTRACT
Sketch recognition systems are currently being developed for many domains, but can be time consuming to build if they are to handle the intricacies of each domain. In order to aid sketch-based user interface developers, we have developed tools to simplify the development of a new sketch recognition interface. We created LADDER, a language to describe how sketched diagrams in a domain are drawn, displayed, and edited. We then automatically transform LADDER structural descriptions into domain specific shape recognizers, editing recognizers, and shape exhibitors for use in conjunction with a domain independent sketch recognition system, creating a sketch recognition system for that domain. We have tested our framework by writing several domain descriptions and automatically generating a domain specific sketch recognition system from each description.
- Alvarado C. A natural sketching environment: bringing the computer into early stages of mechanical design. Master's thesis, MIT, 2000.Google Scholar
- Landay JA, Myers BA. Interactive sketching for the early stages of user interface design. In: Proceedings of CHI '95: Human Factors in Computing Systems, 1995. p. 43--50. Google ScholarDigital Library
- Stahovich T. Sketchit: a sketch interpretation tool for conceptual mechanism design. Technical Report, MIT AI Laboratory, 1996. Google ScholarDigital Library
- Hammond T, Davis R. Tahuti: a geometrical sketch recognition system for uml class diagrams, AAAI Spring Symposium on Sketch Understanding 2002; 59--68.Google Scholar
- Damm CH, Hansen KM, Thomsen M. Tool support for cooperative object-oriented design: gesture based modeling on an electronic whiteboard. In: CHI 2000, CHI; 2000. p. 518--25. Google ScholarDigital Library
- Ideogramic, Ideogramic UML#8482;, Ideogramic ApS, Denmark, http://www.ideogramic.com/products/, 2001.Google Scholar
- Lank E, Thorley JS, Chen SJ-S. An interactive system for recognizing hand drawn UML diagrams. In: Proceedings for CASCON 2000; 2000. p. 7. Google ScholarDigital Library
- Lin J, Newman MW, Hong JI, Landay J. DENIM: finding a tighter fit between tools and practice for web site design. In: CHI Letters: Human Factors in Computing Systems, CHI; 2000. p. 510--7. Google ScholarDigital Library
- Caetano A, Goulart N, Fonseca M, Jorge J. JavaSketchIt: issues in sketching the look of user interfaces, Sketch Understanding. Papers from the 2002 AAAI Spring Symposium.Google Scholar
- Lecolinet E. Designing guis by sketch drawing and visual programming. In: Proceedings of the International Conference on Advanced Visual Interfaces (AVI 1998), AVI, New York: ACM Press, 1998. p. 274--6. URLciteseer. nj.nec.com/lecolinet98designing.html Google ScholarDigital Library
- Do EY-L. Vr sketchpad---create instant 3d worlds by sketching on a transparent window. In: de Vries B, van Leeuwen JP, Achten HH, editors. CAAD Futures 2001. 2001. p. 161--72.Google Scholar
- Mahoney JV, Fromherz MPJ. Handling ambiguity in constraint-based recognition of stick figure sketches. SPIE Document Recognition and Retrieval IX Conference, San Jose, CA.Google Scholar
- Pittman J, Smith I, Cohen P, Oviatt S, Yang T. Quickset: a multimodal interface for military simulations. Proceedings of the Sixth Conference on Computer-Generated Forces and Behavioral Representation, 1996. p. 217--24.Google Scholar
- Hse H, Shilman M, Newton AR, Landay J. Sketch-based user interfaces for collaborative object-oriented modeling. Berkley CS260 Class Project (December 1999).Google Scholar
- Zue, Glass, Conversational interfaces: advances and challenges. Proceedings of IEEE, 2000. p. 1166--80.Google Scholar
- VoiceXML Forum, http://www.voicexml.org/specs/VoiceXML-100.pdf, Voice eXtensible Markup Language, 1st ed. (07 March 2000).Google Scholar
- Hammond T, Davis R. LADDER: a language to describe drawing, display, and editing in sketch recognition. Proceedings of the 2003 International Joint Conference on Artificial Intelligence (IJCAI). Google ScholarDigital Library
- Hammond T, Davis R. Automatically transforming symbolic shape descriptions for use in sketch recognition. Proceedings of the 19th National Conference on Artificial Intelligence (AAAI-04).Google Scholar
- Rubine D. Specifying gestures by example. Computer Graphics 1991;25(4):329--37. Google ScholarDigital Library
- Long AC. Quill: a gesture design tool for pen-based user interfaces, Eecs department, computer science division, UC Berkeley, Berkeley, CA, December 2001.Google Scholar
- Sezgin TM. Feature point detection and curve approximation for early processing in sketch recognition. Master's thesis, Massachusetts Institute of Technology, June 2001.Google Scholar
- Alvarado C, Davis R. Sketchread: a multi-domain sketch recognition engine. In: Proceedings of UIST '04, 2004. p. 23--32. Google ScholarDigital Library
- Friedman-Hill E, Jess, the Java expert system shell. http://herzberg.ca.sandia.gov/jess, 2001.Google Scholar
- Stiny G, Gips J. Shape grammars and the generative specification of painting and sculpture. In: Freiman CV, editor. Information processing, vol. 71. Amsterdam: North-Holland, 1972. p. 1460--5.Google Scholar
- Gips J. Computer implementation of shape grammars. NSF/MIT Workshop on Shape Computation.Google Scholar
- Futrelle RP, Nikolakis N. Efficient analysis of complex diagrams using constraint-based parsing. In: ICDAR-95 (International Conference on Document Analysis and Recognition), Montreal, Canada; 1995. p. 782--90. Google ScholarDigital Library
- Bimber O, Encarnacao LM, Stork A. A multi-layered architecture for sketch-based interaction within virtual environments. Computer and Graphics (Special Issue on Calligraphic Interfaces: Towards a New Generation of Interactive Systems) 2000; 24(6): 851--67.Google Scholar
- Mahoney JV, Fromherz MPJ. Three main concerns in sketch recognition and an approach to addressing them. In: Sketch Understanding, Papers from the 2002 AAAI Spring Symposium. Stanford, CA: AAAI Press, 2002. p. 105--12.Google Scholar
- Caetano A, Goulart N, Fonseca M, Jorge J. Sketching user interfaces with visual patterns. Proceedings of the First Ibero-American Symposium in Computer Graphics (SIACG02), 2002. p. 271--9.Google Scholar
- Shilman M, Pasula H, Russell S, Newton R. Statistical visual language models for ink parsing. In: Sketch Understanding, Papers from the 2002 AAAI Spring Symposium. Stanford, CA: AAAI Press, 2002. p. 126--32.Google Scholar
- Lee S-W. Recognizing circuit symbols with attributed graph matching. In: Baird H, Bunke H, Yamamoto K, editors. Structured document image analysis, 1992. p. 340--58.Google ScholarCross Ref
- Keating JP, Mason RL. Some practical aspects of covariance estimation. Pattern Recognition Letters 1985; 3(5):295--350.Google Scholar
- Calhoun C, Stahovich TF, Kurtoglu T, Kara LB. Recognizing multi-stroke symbols. Sketch Understanding. Papers from the 2002 AAAI Spring Symposium, 2002. p. 15--23.Google Scholar
- Gross MD, Do EY-L. Demonstrating the electronic cocktail napkin: a paper-like interface for early design. 'Common Ground' appeared in ACM Conference on Human Factors in Computing (CHI), 1996. p. 5--6. Google ScholarDigital Library
- Jacob RJK, Deligiannidis L, Morrison S. A software model and specification language for non-WIMP = user interfaces. ACM Transactions on Computer-Human Interaction 1999;6(1):1--46 URLciteseer.nj.nec.com/jacob99software.html. Google ScholarDigital Library
- Costagliola G, Tortora G, Orefice S, Lucia D. Automatic generation of visual programming environments. IEEE Computer 1995; 56--65. Google ScholarDigital Library
- Veselova O. Perceptually based learning of shape descriptions. Master's thesis, Massachusetts Institute of Technology, Cambridge, MA, 2003.Google Scholar
- Hammond T, Davis R. Shady: a shape description debugger for use in sketch recognition. AAAI Fall Symposium on Making Pen-Based Interaction Intelligent and Natural.Google Scholar
- LADDER, a sketching language for user interface developers
Recommendations
LADDER, a sketching language for user interface developers
Sketch recognition systems are currently being developed for many domains, but can be time consuming to build if they are to handle the intricacies of each domain. In order to aid sketch-based user interface developers, we have developed tools to ...
Sketching reality: Realistic interpretation of architectural designs
In this article, we introduce sketching reality, the process of converting a freehand sketch into a realistic-looking model. We apply this concept to architectural designs. As the sketch is being drawn, our system periodically interprets its 2.5D-...
Sketching User Experiences: The Hands-on Course
CHI EA '15: Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing SystemsWhen designing novel user interfaces, paper-pencil sketches can support the design thinking process and are valuable for communicating design ideas to others. This hands-on course will demonstrate how to integrate sketching into researchers' and ...
Comments