ABSTRACT
An important criterion for many intelligent user interfaces is that the interface be multimodal, that is, allow the user to interact with the system using a variety of different input channels. In addition to user interface interactions per se, the system may need to process input from multiple channels to make decisions in response to interface interactions or for other related purposes.The multimodal event parsing system described in our paper has been implemented in a working system called CERA, the Complex Event Recognition Architecture. CERA, developed under contract with NASA, has been used to identify complex events across multiple sensor channels in an advanced life support system demonstration project.We will demonstrate:
The CERA event recognition language,
The CERA event recognition engine at work,
A custom development environment for writing and debugging CERA event recognizers,
Visualization tools for complex event display,
Integrating CERA with various toolkits and projects.
- ANSI Common Lisp standard (X3.226-1994).Google Scholar
- Eclispe Integrated Development Environment. Available at http://www.eclipse.org.Google Scholar
- Firby, R. James. The RAP Language Manual, Version 2. Available at http://www.inetmi.com/ci/tech/rap-zzGoogle Scholar
- I/NET, Inc. Conversational Interface for dialogue management. http://www.inetmi.com/ci/Google Scholar
- Demonstration of the complex event recognition architecture for multimodal event parsing
Recommendations
Multimodal Event Detection on Chinese Glyphs
Advanced Intelligent Computing Technology and ApplicationsAbstractAt present, most of the event detection researches focus on the single text modality, and multimodal event extraction is in its infancy, which mainly used the feature information of images to enhance text information. However, obtaining the ...
Multimodal Attentive Fusion Network for audio-visual event recognition
AbstractEvent classification is inherently sequential and multimodal. Therefore, deep neural models need to dynamically focus on the most relevant time window and/or modality of a video. In this study, we propose the Multimodal Attentive ...
Highlights- State-of-the-art audio and visual interactions in neural networks are relatively simple.
Structural and event based multimodal video data modeling
ADVIS'06: Proceedings of the 4th international conference on Advances in Information SystemsIn this paper, a structural and event based multimodal video data model (SEBM) is proposed. SEBM supports three different modalities that are visual, auditory and textual modalities for video database systems and it can dissolve these three modalities ...
Comments