Abstract
The problem of modality detection in so called communicative interactivity is addressed. Multiple audio and video recordings of human communication are analyzed within this framework, based on fusion of the extracted features. At the decision level, support vector machines (SVMs) are utilized to segregate between the communication modalities. The proposed approach is verified through simulations on real world recordings.
Similar content being viewed by others
References
M. Chen, “Visualizing the Pulse of a Classroom,” in Proceedings of the Eleventh ACM International Conference on Multimedia, ACM Press, 2003, pp. 555–561.
J.J. Gibson, “The Theory of Affordances,” in Perceiving, Acting and Knowing, R. Shaw and J. Bransford (Eds.), Erlbaum, Hillsdale, NJ, 1977.
T.M. Rutkowski, M. Yokoo, D. Mandic, K. Yagi, Y. Kameda, K. Kakusho and M. Minoh, “Identification and Tracking of Active Speaker’s Position in Noisy Environments,” in Proceedings of International Workshop on Acoustic Echo and Noise Control (IWAENC2003), Kyoto, Japan, 2003, pp. 283–286.
T.M. Rutkowski, K. Kakusho, V.V. Kryssanov and M. Minoh, “Evaluation of the communication atmosphere,” Lect. Notes Comput. Sci., vol. 3213, 2004, pp. 364–370.
T.M. Rutkowski, Y. Yamakata, K. Kakusho and M. Minoh, “Smart sensor mesh—intelligent sensor clusters configuration based on communicative affordances principle,” Lecture Notes in Artificial Intelligence, vol. 3490, 2005, pp. 147–157.
T.M. Rutkowski and D. Mandic, “Communicative interactivity—a multimodal communicative situation classification approach,” Lect. Notes Comput. Sci., vol. 3697, 2005, pp. 741–746.
S. Furui, “Digital Speech Processing, Synthesis, and Recognition—Second Edition, Revised and Expanded. 2nd edn. Signal Processing and Communications Series,” Marcell Dekker, Inc., New York, Basel, 2001.
D.D. Lee and H.S. Seung, “Learning the Parts of Objects by Non-Negative Matrix Factorization,” Nature, vol. 401, 1999, pp. 788–791.
A. Hyvarinen, J. Karhunen, E. Oja, “Independent Component Analysis,” Wiley, 2001.
T.M., Rutkowski, S. Seki, Y. Yamakata, K. Kakusho and M. Minoh, “Toward the Human Communication Efficiency Monitoring from Captured Audio and Video Media in Real Environments,” Lect. Notes Comput. Sci., vol. 2774, 2003, pp. 1093–1100.
C. Shannon and W. Weaver, “The Mathematical Theory of Communication,” University of Illinois Press, Urbana, 1949.
V. Kryssanov and K. Kakusho, “From Semiotics of Hypermedia to Physics of Semiosis: A view from System Theory,” Semiotica, vol. 154, no. 1/4, 2005, pp. 11–38.
C.W. Hsu and C.J. Lin, “A Comparison of Methods for Multi-Class Support Vector Machines,” IEEE Trans. Neural Netw., vol. 13, 2002, pp. 415–425.
V. Cherkassky and F. Mulier, “Learning from Data. Adaptive and Learning Systems for Signal Processing, Communication, and Control,” Wiley, USA (1998).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Rutkowski, T.M., Mandic, D. & Barros, A.K. A Multimodal Approach to Communicative Interactivity Classification. J VLSI Sign Process Syst Sign Im 49, 317–328 (2007). https://doi.org/10.1007/s11265-007-0081-6
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11265-007-0081-6