short-paper

The RAP system: automatic feedback of oral presentation skills using multimodal analysis and low-cost sensors

Authors:

Federico Domínguez,

Gabriel Falcones,

Jaime CastellsAuthors Info & Claims

LAK '18: Proceedings of the 8th International Conference on Learning Analytics and Knowledge

Pages 360 - 364

https://doi.org/10.1145/3170358.3170406

Published: 07 March 2018 Publication History

Abstract

Developing communication skills in higher education students could be a challenge to professors due to the time needed to provide formative feedback. This work presents RAP, a scalable system to provide automatic feedback to entry-level students to develop basic oral presentation skills. The system improves the state-of-the-art by analyzing posture, gaze, volume, filled pauses and the slides of the presenters through data captured by very low-cost sensors. The system also provides an off-line feedback report with multimodal recordings of their performance. An initial evaluation of the system indicates that the system's feedback highly agrees with human feedback and that students considered that feedback useful to develop their oral presentation skills.

References

[1]

Michael Argyle and Mark Cook. 1976. Gaze and mutual gaze. (1976).

[2]

K. Audhkhasi, K. Kandhway, O. D. Deshmukh, and A. Verma. 2009. Formant-based technique for automatic filled-pause detection in spontaneous spoken english. In 2009 IEEE International Conference on Acoustics, Speech and Signal Processing. 4857--4860.

Digital Library

[3]

Lucas Azaïs, Adrien Payan, Tianjiao Sun, Guillaume Vidal, Tina Zhang, Eduardo Coutinho, Florian Eyben, and Björn Schuller. 2015. Does my Speech Rock? Automatic Assessment of Public Speaking Skills. In Sixteenth Annual Conference of the International Speech Communication Association.

[4]

Ligia Batrinca, Giota Stratou, Ari Shapiro, Louis-Philippe Morency, and Stefan Scherer. 2013. Cicero-towards a multimodal virtual audience platform for public speaking training. In International Workshop on Intelligent Virtual Agents. Springer, 116--128.

[5]

Zhe Cao, Tomas Simon, Shih-En Wei, and Yaser Sheikh. 2017. Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields. In CVPR.

[6]

Lei Chen, Chee Wee Leong, Gary Feng, Chong Min Lee, and Swapna Somasundaran. 2015. Utilizing multimodal cues to automatically evaluate public speaking performance. In Affective Computing and Intelligent Interaction (ACII), 2015 International Conference on. IEEE, 394--400.

Digital Library

[7]

Vanessa Echeverria, Bruno Guaman, and Katherine Chiluiza. 2015. Mirroring Teachers' Assessment of Novice Students' Presentations through an Intelligent Tutor System. In Computer Aided System Engineering (APCASE), 2015 Asia-Pacific Conference on. IEEE, 264--269.

Digital Library

[8]

Tian Gan, Yongkang Wong, Bappaditya Mandal, Vijay Chandrasekhar, and Mohan S Kankanhalli. 2015. Multi-sensor self-quantification of presentations. In Proceedings of the 23rd ACM international conference on Multimedia. ACM, 601--610.

Digital Library

[9]

Fasih Haider, Loredana Cerrato, Nick Campbell, and Saturnino Luz. 2016. Presentation quality assessment using acoustic information and hand movements. In Acoustics, Speech and Signal Processing (ICASSP), 2016 IEEE International Conference on. IEEE, 2812--2816.

Digital Library

[10]

Steve Johnson, Sarah Veitch, and Silvia Dewiyanti. 2015. A Framework to Embed Communication Skills across the Curriculum: A Design-Based Research Approach. Journal of University Teaching and Learning Practice 12, 4 (2015), 6.

[11]

Stephen E Lucas. 1999. Teaching public speaking. Teaching communication: Theory, research and methods (1999), 75--84.

[12]

Gonzalo Luzardo, Bruno Guamán, Katherine Chiluiza, Jaime Castells, and Xavier Ochoa. 2014. Estimation of Presentations Skills Based on Slides and Audio Features. In Proceedings of the 2014 ACM workshop on Multimodal Learning Analytics. ACM, Istanbul, Turkey, 37--44.

Digital Library

[13]

Xavier Ochoa. 2017. Handbook of Learning Analytics. Society for Learning Analytics Research (SoLAR), Chapter Multimodal Learning Analytics, 129--141. 978-0-9952408-0-3.

[14]

Jan Schneider, Dirk Börner, P Rosmalen, and Marcus Specht. 2017. Presentation Trainer: what experts and computers can tell about your nonverbal communication. Journal of Computer Assisted Learning 33, 2 (2017), 164--177.

Digital Library

[15]

Aaron W Siegman. 1987. Nonverbal behavior and communication. Psychology Press. 37--64 pages.

[16]

Sarah Stawiski, Amy Germuth, Preston Yarborough, Vernal Alford, and Leotis Parrish. 2017. Infusing Twenty-First-Century Skills into Engineering Education. Journal of Business and Psychology 32, 3 (2017), 335--346.

[17]

Rainer Stiefelhagen and Jie Zhu. 2002. Head Orientation and Gaze Direction in Meetings. In CHI '02 Extended Abstracts on Human Factors in Computing Systems (CHI EA '02). ACM, New York, NY, USA, 858--859.

Digital Library

[18]

Bernie Trilling and Charles Fadel. 2009. 21st century skills: Learning for life in our times. John Wiley & Sons.

[19]

Torsten Wörtwein, Mathieu Chollet, Boris Schauerte, Louis-Philippe Morency, Rainer Stiefelhagen, and Stefan Scherer. 2015. Multimodal public speaking performance assessment. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction. ACM, 43--50.

Digital Library

Cited By

Sung HNathan M(2025)Unraveling temporally entangled multimodal interactions: investigating verbal and nonverbal contributions to collaborative construction of embodied math knowledgeInternational Journal of Educational Technology in Higher Education10.1186/s41239-025-00504-622:1Online publication date: 14-Feb-2025
https://doi.org/10.1186/s41239-025-00504-6
Fang YHuang SOgan A(2025)A Cross-Cultural Confusion Model for Detecting and Evaluating Students’ Confusion In a Large ClassroomProceedings of the 15th International Learning Analytics and Knowledge Conference10.1145/3706468.3706528(473-483)Online publication date: 3-Mar-2025
https://dl.acm.org/doi/10.1145/3706468.3706528
Sung HNathan M(2024)Your body tells how you engage in collaboration: Machine‐detected body movements as indicators of engagement in collaborative math knowledge buildingBritish Journal of Educational Technology10.1111/bjet.1347355:5(1950-1973)Online publication date: 10-May-2024
https://doi.org/10.1111/bjet.13473
Show More Cited By

Index Terms

The RAP system: automatic feedback of oral presentation skills using multimodal analysis and low-cost sensors
1. Applied computing
  1. Education
    1. Computer-assisted instruction

Recommendations

A multi-joint model of quiet, upright stance accounts for the "uncontrolled manifold" structure of joint variance

The upright body in quiet stance is usually modeled as a single-link inverted pendulum. This agrees with most of the relevant sensory organs being at the far end of the pendulum, i.e., the eyes and the vestibular system in the head. Movement of the body ...
Development of posture adjustment supporting device of the head and neck for swallowing rehabilitation
i-CREATe '13: Proceedings of the 7th International Convention on Rehabilitation Engineering and Assistive Technology

Background & Aim: Stroke patients often develop dysphagia and sometimes have aspiration pneumonia. To prevent aspiration, dyshagic stroke patients usually undergo a program of comprehensive swallowing rehabilitation, which involves the provision of ...
A Randomized Controlled Trial on Posture Improvement and Trunk Muscle Strength by Round Shoulder Posture Improvement Exercise

The aim of this article was to examine investigates the effects of posture-improving exercises on round shoulder posture RSP and back muscle strength. 35 college students participated in this study and were measured for both shoulder height and muscle ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

LAK '18: Proceedings of the 8th International Conference on Learning Analytics and Knowledge

March 2018

489 pages

ISBN:9781450364003

DOI:10.1145/3170358

Conference Chairs:
Abelardo Pardo
The University of Sydney, AUS
,
Kathryn Bartimote-Aufflick
The University of Sydney, AUS
,
Grace Lynch
Society for Learning Analytics Research, AUS
,
Program Chairs:
Simon Buckingham Shum
University of Technology Sydney, AUS
,
Rebecca Ferguson
The Open University, UK
,
Agathe Merceron
Beuth University of Applied Sciences, GER
,
Xavier Ochoa
Escuela Superior Politécnica del Litoral, EC

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 March 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

LAK '18

LAK '18: International Conference on Learning Analytics and Knowledge

March 7 - 9, 2018

New South Wales, Sydney, Australia

Acceptance Rates

LAK '18 Paper Acceptance Rate 35 of 115 submissions, 30%;

Overall Acceptance Rate 236 of 782 submissions, 30%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

58
Total Citations
View Citations
936
Total Downloads

Downloads (Last 12 months)88
Downloads (Last 6 weeks)11

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Sung HNathan M(2025)Unraveling temporally entangled multimodal interactions: investigating verbal and nonverbal contributions to collaborative construction of embodied math knowledgeInternational Journal of Educational Technology in Higher Education10.1186/s41239-025-00504-622:1Online publication date: 14-Feb-2025
https://doi.org/10.1186/s41239-025-00504-6
Fang YHuang SOgan A(2025)A Cross-Cultural Confusion Model for Detecting and Evaluating Students’ Confusion In a Large ClassroomProceedings of the 15th International Learning Analytics and Knowledge Conference10.1145/3706468.3706528(473-483)Online publication date: 3-Mar-2025
https://dl.acm.org/doi/10.1145/3706468.3706528
Sung HNathan M(2024)Your body tells how you engage in collaboration: Machine‐detected body movements as indicators of engagement in collaborative math knowledge buildingBritish Journal of Educational Technology10.1111/bjet.1347355:5(1950-1973)Online publication date: 10-May-2024
https://doi.org/10.1111/bjet.13473
Huang ZHe QMaher KDeng XLai YMa CQin SLiu YWang H(2024)SpeechMirror: A Multimodal Visual Analytics System for Personalized Reflection of Online Public Speaking EffectivenessIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332693230:1(606-616)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TVCG.2023.3326932
Sung HRau MVan Veen B(2024)Development of an Intelligent Tutoring System That Assesses Internal Visualization Skills in Engineering Using Multimodal TriangulationIEEE Transactions on Learning Technologies10.1109/TLT.2024.339639317(1625-1638)Online publication date: 2-May-2024
https://dl.acm.org/doi/10.1109/TLT.2024.3396393
Shirol AGadad JM V(2024)Unleashing Potential: Transforming Oral Presentations Through Multimodal Learning Analytics2024 IEEE International Conference on Teaching, Assessment and Learning for Engineering (TALE)10.1109/TALE62452.2024.10834387(1-7)Online publication date: 9-Dec-2024
https://doi.org/10.1109/TALE62452.2024.10834387
Lee EYang EHuh JOh U(2024)EcoScript: A Real-Time Presentation Supporting Tool using a Speech Recognition Model2024 IEEE International Conference on Information Reuse and Integration for Data Science (IRI)10.1109/IRI62200.2024.00031(96-101)Online publication date: 7-Aug-2024
https://doi.org/10.1109/IRI62200.2024.00031
Domínguez FEras LCollaguazo A(2024)Insights on the Distribution of Nonverbal and Verbal Oral Presentation Skills in an Educational InstitutionSN Computer Science10.1007/s42979-024-02785-65:5Online publication date: 25-Apr-2024
https://dl.acm.org/doi/10.1007/s42979-024-02785-6
Krogstie JSharma K(2024)Enhancing Our Understanding of Business Process Model Comprehension Using Biometric DataEnterprise, Business-Process and Information Systems Modeling10.1007/978-3-031-61007-3_13(159-174)Online publication date: 31-May-2024
https://doi.org/10.1007/978-3-031-61007-3_13
Xie YKonomi S(2024)Developing a Human-Centered AI Environment to Enhance Financial Literacy of College Students: A Systematic ReviewCross-Cultural Design10.1007/978-3-031-60913-8_25(360-374)Online publication date: 29-Jun-2024
https://dl.acm.org/doi/10.1007/978-3-031-60913-8_25
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten