Abstract
In this paper, we present a multimodal discourse ontology that serves as a knowledge representation and annotation framework for the discourse understanding component of an artificial personal office assistant. The ontology models components of natural language, multimodal communication, multi-party dialogue structure, meeting structure, and the physical and temporal aspects of human communication. We compare our models to those from the research literature and from similar applications. We also highlight some annotations which have been made in conformance with the ontology as well as some algorithms which have been trained on these data and suggest elements of the ontology that may be of immediate interest for further annotation by human or automated means.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Romano Jr., N.C., Nunamaker Jr., J.F.: Meeting analysis: Findings from research and practice. In: Proceedings of the 34th Hawaii International Conference on System Sciences (2001)
Lisowska, A., Popescu-Belis, A., Armstrong, S.: User query analysis for the specificationand evaluation of a dialogue processing and retrieval system. In: Proceedings of the 4th International Conference on Language Resources and Evaluation (2004)
Reidsma, D., Rienks, R., Jovanović, N.: Meeting modelling in the context of multimodal research. In: Bengio, S., Bourlard, H. (eds.) MLMI 2004. LNCS, vol. 3361, pp. 22–35. Springer, Heidelberg (2005)
Bachler, M.S., Shum, S.J.B., Roure, D.C.D., Michaelides, D.T., Page, K.R.: Ontologicalmediation of meeting structure: Argumentation, annotation, and navigation. In: Proceedings of the 1st International Workshop on Hypermedia and the Semantic Web (2003)
Marchand-Maillet, S.: Meeting record modelling for enhanced browsing. Technical Report 03.01, Computer Vision and Multimedia Laboratory, Computing Centre, University of Geneva, Switzerland (2003)
Banerjee, S., Rose, C., Rudnicky, A.: The necessity of a meeting recording and playback system, and the benefit of topic–level annotations to meeting browsing. In: Costabile, M.F., Paternó, F. (eds.) INTERACT 2005. LNCS, vol. 3585, pp. 643–656. Springer, Heidelberg (2005)
Barker, K., Porter, B., Clark, P.: A library of generic concepts for composing knowledge bases. In: Proceedings of the 1st International Conference on Knowledge Capture (2001)
Clark, P., Porter, B.: KM - The Knowledge Machine 2.0: Users manual (2004), http://www.cs.utexas.edu/users/mfkb/RKF/km.html
Popescu-Belis, A.: Dialogue acts: One or more dimensions? ISSCO Working Paper 62. University of Geneva (2005)
Clark, H.H., Krych, M.A.: Speaking while monitoring addressees for understanding. Journal of Memory and Language 50, 62–81 (2004)
Quek, F., McNeill, D., Bryll, R., Duncan, S., Ma, X.F., Kirbas, C., McCullough, K.E., Ansari, R.: Multimodal human discourse: Gesture and speech. ACM Transactions on Computer-Human Interaction 9(3), 171–193 (2002)
Farrar, S., Langendoen, T.: A linguistic ontology for the semantic web. Glot International 7(3), 97–100 (2003)
Ide, N., Romary, L., de la Clergerie, E.: International standard for a linguistic annotation framework. In: Proceedings of the HLT-NAACL Workshop on the Software Engineering and Architecture of Language Technology (2003)
Clark, A., Popescu-Belis, A.: Multi-level dialogue act tags. In: Proceedings of the 5th SIGdial Workshop on Discourse and Dialogue (2004)
Shriberg, E., Dhillon, R., Bhagat, S., Ang, J., Carvey, H.: The ICSI Meeting Recorder Dialog Act Corpus. In: Proceedings of the 5th SIGdial Workshop on Discourse and Dialogue (2004)
Lemon, O., Gruenstein, A.: Multithreaded context for robust conversational interfaces: Context-sensitive speech recognition and interpretation of corrective fragments. ACM Transactions on Computer-Human Interaction 11(3) (2004)
Traum, D., Bos, J., Cooper, R., Larsson, S., Lewin, I., Matheson, C., Poesio, M.: A model of dialogue moves and information state revision. Task Oriented Instructional Dialogue (TRINDI): Deliverable 2.1. University of Gothenburg (1999)
Pallotta, V., Niekrasz, J., Purver, M.: Collaborative and argumentative models of natural discussions. In: Proceedings of the 5th Workshop on Computational Models of Natural Argument (2005)
Dielmann, A., Renals, S.: Dynamic bayesian networks for meeting structuring. In: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (2004)
Reiter, S., Rigoll, G.: Segmentation and classification of meeting events using multiple classifier fusion and dynamic programming. In: Proceedings of the International Conference on Pattern Recognition (2004)
McCowan, I., Gatica-Perez, D., Bengio, S., Lathoud, G., Barnard, M., Zhang, S.: Automatic analysis of multimodal group actions in meetings. IEEE Transactions on Pattern Analysis and Machine Intelligence 27(3), 305–317 (2005)
McCowan, I., Bengio, S., Gatica-Perez, D., Lathoud, G., Monay, F., Moore, D., Wellner, P., Bourlard, H.: Modeling human interaction in meetings. In: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (2003)
Banerjee, S., Rudnicky, A.: Using simple speech-based features to detect the state of a meeting and the roles of the meeting participants. In: Proceedings of the 8th International Conference on Spoken Language Processing (2004)
Galley, M., McKeown, K., Fosler-Lussier, E., Jing, H.: Discourse segmentation of multi-party conversation. In: Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics (2003)
Gruenstein, A., Niekrasz, J., Purver, M.: Meeting structure annotation: Data and tools. In: Proceedings of the 6th SIGdial Workshop on Discourse and Dialogue, Lisbon, Portugal (2005)
Niekrasz, J., Purver, M., Dowding, J., Peters, S.: Ontology-based discourse understanding for a persistent meeting assistant. In: Proceedings of the AAAI Spring Symposium Workshop on Persistent Assistants: Living and Working with AI (2005)
Blei, D., Moreno, P.: Topic segmentation with an aspect hidden Markov model. In: Proceedings of the 24th Annual International Conference on Research and Development in Information Retrieval, pp. 343–348 (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Niekrasz, J., Purver, M. (2006). A Multimodal Discourse Ontology for Meeting Understanding. In: Renals, S., Bengio, S. (eds) Machine Learning for Multimodal Interaction. MLMI 2005. Lecture Notes in Computer Science, vol 3869. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11677482_14
Download citation
DOI: https://doi.org/10.1007/11677482_14
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-32549-9
Online ISBN: 978-3-540-32550-5
eBook Packages: Computer ScienceComputer Science (R0)