skip to main content
10.1145/1935701.1935712acmconferencesArticle/Chapter ViewAbstractPublication PagesteiConference Proceedingsconference-collections
research-article

Midas: a declarative multi-touch interaction framework

Published: 22 January 2010 Publication History

Abstract

Over the past few years, multi-touch user interfaces emerged from research prototypes into mass market products. This evolution has been mainly driven by innovative devices such as Apple's iPhone or Microsoft's Surface tabletop computer. Unfortunately, there seems to be a lack of software engineering abstractions in existing multi-touch development frameworks. Many multi-touch applications are based on hard-coded procedural low level event processing. This leads to proprietary solutions with a lack of gesture extensibility and cross-application reusability. We present Midas, a declarative model for the definition and detection of multi-touch gestures where gestures are expressed via logical rules over a set of input facts. We highlight how our rule-based language approach leads to improvements in gesture extensibility and reusability. Last but not least, we introduce JMidas, an instantiation of Midas for the Java programming language and describe how JMidas has been applied to implement a number of innovative multi-touch gestures.

References

[1]
F. P. Brooks, Jr. No Silver Bullet: Essence and Accidents of Software Engineering. IEEE Computer, 20(4):10--19, April 1987.
[2]
F. Echtler and G. Klinker. A Multitouch Software Architecture. In Proc. of NordiCHI 2008, 5th Nordic Conference on Human-Computer Interaction, pages 463--466, Lund, Sweden, 2008.
[3]
C. L. Forgy. RRete: A Fast Algorithm for the Many Pattern/Many Object Pattern Match Problem. Artificial Intelligence, 19(1):17--37, 1982.
[4]
E. Friedman-Hill. Jess in Action: Java Rule-Based Systems. Manning Publications, July 2003.
[5]
L. Hoste. Experiments with the SunSPOT Accelerometer. Project report, Vrije Universiteit Brussel, 2009.
[6]
M. Kaltenbrunner, T. Bovermann, R. Bencina, and E. Costanza. TUIO: A Protocol for Table-Top Tangible User Interfaces. In Proc. of GW 2005, 6th Intl. Workshop on Gesture in Human-Computer Interaction and Simulation, Ile de Berder, France, May 2005.
[7]
D. Kammer, M. Keck, G. Freitag, and M. Wacker. Taxonomy and Overview of Multi-touch Frameworks: Architecture, Scope and Features. In Proc. of Workshop on Engineering Patterns for Multi-Touch Interfaces, Berlin, Germany, June 2010.
[8]
I. Maier, T. Rompf, and M. Odersky. Deprecating the Observer Pattern. Technical Report EPFL-REPORT-148043, Ecole Polytechnique Fédeérale de Lausanne, Lausanne, Switzerland, 2010.
[9]
L. H. Nakatani and J. A. Rohrlich. Soft Machines: A Philosophy of User-Computer Interface Design. In Proc. of CHI '83, ACM Conference on Human Factors in Computing Systems, pages 19--23, Boston, USA, December 1983.
[10]
A. D. Nardi. Grafiti: Gesture Recognition mAnagement Framework for Interactive Tabletop Interfaces. Master's thesis, University of Pisa, 2008.
[11]
P. Ramanahally, S. Gilbert, T. Niedzielski, D. Velázquez, and C. Anagnost. Sparsh UI: A Multi-Touch Framework for Collaboration and Modular Gesture Recognition. In Proc. of WINVR 2009, Conference on Innovative Virtual Reality, pages 1--6, Chalon-sur-SaoÆne, France, February 2009.
[12]
D. Rubine. Specifying Gestures by Example. In Proc. of ACM SIGGRAPH '91, 18th Intl. Conference on Computer Graphics and Interactive Techniques, pages 329--337, Las Vegas, USA, August 1991.
[13]
B. Signer, U. Kurmann, and M. C. Norrie. iGesture: A General Gesture Recognition Framework. In Proc. of ICDAR 2007, 9th Intl. Conference on Document Analysis and Recognition, pages 954--958, Curitiba, Brazil, September 2007.
[14]
J. Stewart, B. B. Bederson, and A. Druin. Single Display Groupware: A Model for Co-present Collaboration. In Proc. of CHI '99, ACM Conference on Human Factors in Computing Systems, pages 286--293, Pittsburgh, USA, May 1999.

Cited By

View all
  • (2022)PassWalk: Spatial Authentication Leveraging Lateral Shift and Gaze on Mobile HeadsetsProceedings of the 30th ACM International Conference on Multimedia10.1145/3503161.3548252(952-960)Online publication date: 10-Oct-2022
  • (2021)Building Adaptive Touch Interfaces—Case Study 6Intelligent Computing for Interactive System Design10.1145/3447404.3447426(379-406)Online publication date: 23-Feb-2021
  • (2020)DG3: Exploiting Gesture Declarative Models for Sample Generation and Online RecognitionProceedings of the ACM on Human-Computer Interaction10.1145/33978704:EICS(1-21)Online publication date: 18-Jun-2020
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
TEI '11: Proceedings of the fifth international conference on Tangible, embedded, and embodied interaction
January 2011
470 pages
ISBN:9781450304788
DOI:10.1145/1935701
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 January 2010

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. declarative programming
  2. gesture framework
  3. multi-touch interaction
  4. rule language

Qualifiers

  • Research-article

Conference

TEI'11
Sponsor:

Acceptance Rates

Overall Acceptance Rate 393 of 1,367 submissions, 29%

Upcoming Conference

TEI '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)11
  • Downloads (Last 6 weeks)0
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2022)PassWalk: Spatial Authentication Leveraging Lateral Shift and Gaze on Mobile HeadsetsProceedings of the 30th ACM International Conference on Multimedia10.1145/3503161.3548252(952-960)Online publication date: 10-Oct-2022
  • (2021)Building Adaptive Touch Interfaces—Case Study 6Intelligent Computing for Interactive System Design10.1145/3447404.3447426(379-406)Online publication date: 23-Feb-2021
  • (2020)DG3: Exploiting Gesture Declarative Models for Sample Generation and Online RecognitionProceedings of the ACM on Human-Computer Interaction10.1145/33978704:EICS(1-21)Online publication date: 18-Jun-2020
  • (2019)Implementing Multi-Touch Gestures with Touch Groups and Cross EventsProceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300585(1-12)Online publication date: 2-May-2019
  • (2018)G-GeneProceedings of the ACM on Human-Computer Interaction10.1145/32290952:EICS(1-17)Online publication date: 19-Jun-2018
  • (2018)The TUIO 2.0 ProtocolProceedings of the ACM on Human-Computer Interaction10.1145/32290902:EICS(1-35)Online publication date: 19-Jun-2018
  • (2018)Learning to Recognize Touch Gestures: Recurrent vs. Convolutional Features and Dynamic Sampling2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018)10.1109/FG.2018.00026(114-121)Online publication date: May-2018
  • (2017)CircGRProceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces10.1145/3132272.3134139(12-21)Online publication date: 17-Oct-2017
  • (2017)ProbUIProceedings of the 2017 CHI Conference on Human Factors in Computing Systems10.1145/3025453.3025502(4640-4653)Online publication date: 2-May-2017
  • (2017)Gesture Modelling and Recognition by Integrating Declarative Models and Pattern Recognition AlgorithmsImage Analysis and Processing - ICIAP 201710.1007/978-3-319-68560-1_8(84-95)Online publication date: 13-Oct-2017
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media