Abstract
The dialogue components we developed provide the infrastructure of the disseminated industrial prototype RadSpeech—a semantic speech dialogue system for radiologists. The major contribution of this paper is the description of a new speech-based interaction scenario of RadSpeech where two radiologists use two independent but related mobile speech devices (iPad and iPhone) and collaborate via a connected large screen installation using related speech commands. With traditional user interfaces, users may browse or explore patient data, but little to no help is given when it comes to structuring the collaborative user input and annotate radiology images in real-time with ontology-based medical annotations. A distinctive feature is that the interaction design includes the screens of the mobile devices for touch screen interaction for more complex tasks rather than the simpler ones such as a mere remote control of the image display on the large screen.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
This work is part of THESEUS-RadSpeech (see www.dfki.de/RadSpeech/) to implement dialogue applications for medical use case scenarios. It has been supported by the German Federal Ministry of Economics and Technology (01MQ07016).
References
Fensel, D., Hendler, J.A., Lieberman, H., Wahlster, W. (eds.): Spinning the Semantic Web: Bringing the World Wide Web to Its Full Potential. MIT Press, Cambridge (2003)
Hall, F.M.: The radiology report of the future. Radiology 251(2), 313–316 (2009)
Mejino, J.L., Rubin, D.L., Brinkley, J.F.: FMA-RadLex: An application ontology of radiological anatomy derived from the foundational model of anatomy reference ontology. In: Proceedings of AMIA Symposium, pp. 465–469, 2008. http://stanford.edu/~rubin/pubs/097.pdf
Möller, M., Ernst, P., Dengel, A., Sonntag, D.: Representing the international classification of diseases version 10 in OWL. In: Proceedings of the International Conference on Knowledge Engineering and Ontology Development (KEOD), Valencia, Spain, 2010
Oviatt, S.: Ten myths of multimodal interaction. Commun. ACM 42(11), 74–81 (1999). citeseer.nj.nec.com/oviatt99ten.html
Pfleger, N.: FADE: an integrated approach to multimodal fusion and discourse processing. In: Proceedings of the Dotoral Spotlight at ICMI 2005, Trento, Italy, 2005
Pieraccini, R., Huerta, J.: Where do we go from here? research and commercial spoken dialog systems. In: Proceedings of the 6th SIGDdial Worlshop on Discourse and Dialogue, pp. 1–10, 2005
Reithinger, N., Fedeler, D., Kumar, A., Lauer, C., Pecourt, E., Romary, L.: MIAMM: A multimodal dialogue system using haptics. In: van Kuppevelt, J., Dybkjaer, L., Bernsen, N.O. (eds.) Advances in Natural Multimodal Dialogue Systems. Springer, Berlin (2005)
Reithinger, N., Sonntag, D.: An integration framework for a mobile multimodal dialogue system accessing the semantic web. In: Proceedings of Interspeech, Lisbon, Portugal, pp. 841–844 (2005)
Sonntag, D.: Ontologies and Adaptivity in Dialogue for Question Answering. AKA and IOS Press, Heidelberg (2010)
Sonntag, D., Deru, M., Bergweiler, S.: Design and implementation of combined mobile and touchscreen-based multimodal web 3.0 interfaces. In: Proceedings of the International Conference on Artificial Intelligence (ICAI), pp. 974–979 (2009)
Sonntag, D., Möller, M.: Unifying semantic annotation and querying in biomedical image repositories. In: Proceedings of International Conference on Knowledge Management and Information Sharing (KMIS) (2009)
Sonntag, D., Reithinger, N., Herzog, G., Becker, T.: a discourse and dialogue infrastructure for industrial dissemination. Proceedings of IWSDS—Spoken Dialogue Systems for Ambient Environment, Chapter. Lecture Notes in Artificial Intelligence, pp. 132–143. Springer, Berlin (2010)
Sonntag, D., Sonnenberg, G., Nesselrath, R., Herzog, G.: Supporting a rapid dialogue engineering process. In: Proceedings of the First International Workshop On Spoken Dialogue Systems Technology (IWSDS) (2009)
Wahlster, W.: SmartKom: symmetric multimodality in an adaptive and reusable dialogue shell. In: Krahl, R., Günther, D. (eds.) Proceedings of the Human Computer Interaction Status Conference 2003, pp. 47–62. DLR, Berlin, Germany (2003)
Weiss, D.L., Langlotz, C.: Structured reporting: Patient care enhancement or productivity nightmare? Radiology 249(3), 739–747 (2008)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer Science+Business Media New York
About this paper
Cite this paper
Sonntag, D., Schulz, C. (2014). A Multimodal Multi-device Discourse and Dialogue Infrastructure for Collaborative Decision-Making in Medicine. In: Mariani, J., Rosset, S., Garnier-Rizet, M., Devillers, L. (eds) Natural Interaction with Robots, Knowbots and Smartphones. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-8280-2_4
Download citation
DOI: https://doi.org/10.1007/978-1-4614-8280-2_4
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4614-8279-6
Online ISBN: 978-1-4614-8280-2
eBook Packages: EngineeringEngineering (R0)