Dan Bohus
Sean Andrist
ABSTRACT
We demonstrate an open, extensible platform for developing and studying multimodal, integrative-AI systems. The platform provides a time-aware, stream-based programming model for parallel coordinated computation, a set of tools for data visualization, processing, and learning, and an ecosystem of pluggable AI components. The demonstration will showcase three applications built on this platform and highlight how the platform can significantly accelerate development and research in multimodal interactive systems.
- 2017. Platform for Situated Intelligence. http://aka.ms/psi. (2017).Google Scholar
- Itseez. 2015. Open Source Computer Vision Library. https://github.com/itseez/ opencv. (2015).Google Scholar
- Morgan Quigley, Brian Gerkey, Ken Conley, Josh Faust, Tully Foote, Jeremy Leibs, Eric Berger, Rob Wheeler, and Andrew Ng. 2009. ROS: an open-source Robot Operating System. In Proc. of the IEEE Intl. Conf. on Robotics and Automation (ICRA) Workshop on Open Source Robotics. Kobe, Japan.Google Scholar
- Gabriel Skantze and Samer Al Moubayed. 2012. IrisTK: a statechart-based toolkit for multi-party face-to-face interaction. In Proceedings of the 14th ACM international conference on Multimodal interaction (ICMI’12). ACM, 69–76. Google ScholarDigital Library
- Johannes Wagner, Florian Lingenfelser, Tobias Baur, Ionut Damian, Felix Kistler, and Elisabeth André. 2013. The social signal interpretation (SSI) framework: Multimodal signal processing and recognition in real-time. In Proceedings of the 21st ACM international conference on Multimedia (MM’13). ACM, 831–834. Abstract 1 Introduction 2 Platform for Situated Intelligence 2.1 Runtime 2.2 Tools 2.3 Components 3 Demonstration Acknowledgments References Google ScholarDigital Library
Index Terms
- Rapid development of multimodal interactive systems: a demonstration of platform for situated intelligence
Recommendations
Demonstrating a framework for rapid development of physically situated interactive systems
HRI '19: Proceedings of the 14th ACM/IEEE International Conference on Human-Robot InteractionWe demonstrate an open, extensible framework for enabling faster development and study of physically situated interactive systems. The framework provides a programming model for parallel coordinated computation centered on temporal streams of data, a ...
Facilitating multiparty dialog with gaze, gesture, and speech
ICMI-MLMI '10: International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal InteractionWe study how synchronized gaze, gesture and speech rendered by an embodied conversational agent can influence the flow of conversations in multiparty settings. We begin by reviewing a computational framework for turn-taking that provides the foundation ...
Natural Communication about Uncertainties in Situated Interaction
ICMI '14: Proceedings of the 16th International Conference on Multimodal InteractionPhysically situated, multimodal interactive systems must often grapple with uncertainties about properties of the world, people, and their intentions and actions. We present methods for estimating and communicating about different uncertainties in ...
Comments