Loading web-font TeX/Main/Regular
- Framework: A Software Framework for Multimodal Human-Computer Interaction Systems | IEEE Journals & Magazine | IEEE Xplore

{\rm HCI}^{\wedge}2 Framework: A Software Framework for Multimodal Human-Computer Interaction Systems


Abstract:

This paper presents a novel software framework for the development and research in the area of multimodal human-computer interface (MHCI) systems. The proposed software f...Show More

Abstract:

This paper presents a novel software framework for the development and research in the area of multimodal human-computer interface (MHCI) systems. The proposed software framework, which is called the HCI2 Framework, is built upon publish/subscribe (P/S) architecture. It implements a shared-memory-based data transport protocol for message delivery and a TCP-based system management protocol. The latter ensures that the integrity of system structure is maintained at runtime. With the inclusion of bridging modules, the HCI2 Framework is interoperable with other software frameworks including Psyclone and ActiveMQ. In addition to the core communication middleware, we also present the integrated development environment (IDE) of the HCI2 Framework. It provides a complete graphical environment to support every step in a typical MHCI system development process, including module development, debugging, packaging, and management, as well as the whole system management and testing. The quantitative evaluation indicates that our framework outperforms other similar tools in terms of average message latency and maximum data throughput under a typical single PC scenario. To demonstrate HCI2 Framework's capabilities in integrating heterogeneous modules, we present several example modules working with a variety of hardware and software. We also present an example of a full system developed using the proposed HCI2 Framework, which is called the CamGame system and represents a computer game based on hand-held marker(s) and low-cost camera(s).
Published in: IEEE Transactions on Cybernetics ( Volume: 43, Issue: 6, December 2013)
Page(s): 1593 - 1606
Date of Publication: 16 October 2013

ISSN Information:

PubMed ID: 24235258

Contact IEEE to Subscribe

References

References is not available for this document.