Skip to main content
Log in

Abstract

The problem of modality detection in so called communicative interactivity is addressed. Multiple audio and video recordings of human communication are analyzed within this framework, based on fusion of the extracted features. At the decision level, support vector machines (SVMs) are utilized to segregate between the communication modalities. The proposed approach is verified through simulations on real world recordings.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. M. Chen, “Visualizing the Pulse of a Classroom,” in Proceedings of the Eleventh ACM International Conference on Multimedia, ACM Press, 2003, pp. 555–561.

  2. J.J. Gibson, “The Theory of Affordances,” in Perceiving, Acting and Knowing, R. Shaw and J. Bransford (Eds.), Erlbaum, Hillsdale, NJ, 1977.

    Google Scholar 

  3. T.M. Rutkowski, M. Yokoo, D. Mandic, K. Yagi, Y. Kameda, K. Kakusho and M. Minoh, “Identification and Tracking of Active Speaker’s Position in Noisy Environments,” in Proceedings of International Workshop on Acoustic Echo and Noise Control (IWAENC2003), Kyoto, Japan, 2003, pp. 283–286.

  4. T.M. Rutkowski, K. Kakusho, V.V. Kryssanov and M. Minoh, “Evaluation of the communication atmosphere,” Lect. Notes Comput. Sci., vol. 3213, 2004, pp. 364–370.

    Article  Google Scholar 

  5. T.M. Rutkowski, Y. Yamakata, K. Kakusho and M. Minoh, “Smart sensor mesh—intelligent sensor clusters configuration based on communicative affordances principle,” Lecture Notes in Artificial Intelligence, vol. 3490, 2005, pp. 147–157.

    Google Scholar 

  6. T.M. Rutkowski and D. Mandic, “Communicative interactivity—a multimodal communicative situation classification approach,” Lect. Notes Comput. Sci., vol. 3697, 2005, pp. 741–746.

    Google Scholar 

  7. S. Furui, “Digital Speech Processing, Synthesis, and Recognition—Second Edition, Revised and Expanded. 2nd edn. Signal Processing and Communications Series,” Marcell Dekker, Inc., New York, Basel, 2001.

    Google Scholar 

  8. D.D. Lee and H.S. Seung, “Learning the Parts of Objects by Non-Negative Matrix Factorization,” Nature, vol. 401, 1999, pp. 788–791.

    Article  Google Scholar 

  9. A. Hyvarinen, J. Karhunen, E. Oja, “Independent Component Analysis,” Wiley, 2001.

  10. T.M., Rutkowski, S. Seki, Y. Yamakata, K. Kakusho and M. Minoh, “Toward the Human Communication Efficiency Monitoring from Captured Audio and Video Media in Real Environments,” Lect. Notes Comput. Sci., vol. 2774, 2003, pp. 1093–1100.

    Article  Google Scholar 

  11. C. Shannon and W. Weaver, “The Mathematical Theory of Communication,” University of Illinois Press, Urbana, 1949.

    MATH  Google Scholar 

  12. V. Kryssanov and K. Kakusho, “From Semiotics of Hypermedia to Physics of Semiosis: A view from System Theory,” Semiotica, vol. 154, no. 1/4, 2005, pp. 11–38.

    Google Scholar 

  13. C.W. Hsu and C.J. Lin, “A Comparison of Methods for Multi-Class Support Vector Machines,” IEEE Trans. Neural Netw., vol. 13, 2002, pp. 415–425.

    Article  Google Scholar 

  14. V. Cherkassky and F. Mulier, “Learning from Data. Adaptive and Learning Systems for Signal Processing, Communication, and Control,” Wiley, USA (1998).

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Tomasz M. Rutkowski.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Rutkowski, T.M., Mandic, D. & Barros, A.K. A Multimodal Approach to Communicative Interactivity Classification. J VLSI Sign Process Syst Sign Im 49, 317–328 (2007). https://doi.org/10.1007/s11265-007-0081-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11265-007-0081-6

Keywords

Navigation