skip to main content
10.1145/3290605.3300681acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

Face and Ecological Validity in Simulations: Lessons from Search-and-Rescue HRI

Published: 02 May 2019 Publication History

Abstract

In fields where in situ performance cannot be measured, ecological validity is difficult to estimate. Drawing on theory from social psychology and virtual reality, we argue that face validity can be a useful proxy for ecological validity. We provide illustrative examples of this relationship from work in search-and-rescue HRI, and conclude with some practical guidelines for the construction of immersive simulations in general.

Supplementary Material

MP4 File (paper451.mp4)

References

[1]
H. L. Akin, N. Ito, A. Jacoff, A. Kleiner, J. Pellenz, and A. Visser, "Robocup rescue robot and simulation leagues," AI magazine, vol. 34, no. 1, p. 78, 2012.
[2]
R. R. Murphy, Disaster robotics. MIT press, 2014.
[3]
T. D. Parsons, "Virtual reality for enhanced ecological validity and experimental control in the clinical, affective and social neurosciences," Frontiers in human neuroscience, vol. 9, p. 660, 2015.
[4]
U. Neisser, "on social knowing"," Personality and Social Psychology Bulletin, vol. 6, no. 4, pp. 601--605, 1980.
[5]
L. Palen, S. Vieweg, J. Sutton, S. B. Liu, and A. Hughes, "Crisis informatics: Studying crisis in a networked world," in Third International Conference on e-Social Science, pp. 7--9, Citeseer, 2007.
[6]
S. Vieweg, A. L. Hughes, K. Starbird, and L. Palen, "Microblogging during two natural hazards events: what twitter may contribute to situational awareness," in Proceedings of the SIGCHI conference on human factors in computing systems, pp. 1079--1088, ACM, 2010.
[7]
K. Starbird and L. Palen, Pass it on?: Retweeting in mass emergency. International Community on Information Systems for Crisis Response and Management, 2010.
[8]
G. Bennett, G. Lindgaard, B. Tsuji, K. H. Connelly, and K. A. Siek, "Reality testing: Hci challenges in non-traditional environments," in CHI'06 Extended Abstracts on Human Factors in Computing Systems, pp. 1679--1682, ACM, 2006.
[9]
C. L. Bethel and R. R. Murphy, "Affective expression in appearance constrained robots," in Proceedings of the 1st ACM SIGCHI/SIGART conference on Human-robot interaction, pp. 327--328, ACM, 2006.
[10]
Z. Henkel, C. L. Bethel, R. R. Murphy, and V. Srinivasan, "Evaluation of proxemic scaling functions for social robotics," IEEE Transactions on Human-Machine Systems, vol. 44, no. 3, pp. 374--385, 2014.
[11]
V. Srinivasan, C. L. Bethel, and R. R. Murphy, "Evaluation of head gaze loosely synchronized with real-time synthetic speech for social robots," IEEE Transactions on Human-Machine Systems, vol. 44, no. 6, pp. 767--778, 2014.
[12]
R. Stallings, "Methodological issues," in Handbook of disaster research (E. L. Q. H. Rodriguez and R. R. Dynes, eds.), pp. 55--82, New York: Springer, 2006.
[13]
F. G. Miller and H. Brody, "What makes placebo-controlled trials unethical?," The American Journal of Bioethics, vol. 2, no. 2, pp. 3--9, 2002.
[14]
U. D. of Health and H. Services, "The belmont report," 1979.
[15]
N. Chaytor and M. Schmitter-Edgecombe, "The ecological validity of neuropsychological tests: A review of the literature on everyday cognitive skills," Neuropsychology review, vol. 13, no. 4, pp. 181--197, 2003.
[16]
J. H. Korn, Illusions of reality: A history of deception in social psychology. SUNY Press, 1997.
[17]
C. Deniaud, V. Honnet, B. Jeanne, and D. Mestre, "The concept of presence" as a measure of ecological validity in driving simulators," Journal of Interaction Science, vol. 3, no. 1, p. 1, 2015.
[18]
R. T. Hays, J. W. Jacobs, C. Prince, and E. Salas, "Flight simulator training effectiveness: A meta-analysis," Military psychology, vol. 4, no. 2, pp. 63--74, 1992.
[19]
K. Lewin, "Field theory in social science," 1951.
[20]
S. Milgram, "Behavioral study of obedience.," The Journal of abnormal and social psychology, vol. 67, no. 4, p. 371, 1963.
[21]
C. L. Sheridan and R. G. King, "Obedience to authority with an authentic victim.," in Proceedings of the annual convention of the American Psychological Association, American Psychological Association, 1972.
[22]
S. A. Haslam, S. D. Reicher, and M. E. Birney, "Nothing by mere authority: Evidence that in an experimental analogue of the milgram paradigm participants are motivated not by orders but by appeals to science," Journal of Social Issues, vol. 70, no. 3, pp. 473--488, 2014.
[23]
E. A. Caspar, J. F. Christensen, A. Cleeremans, and P. Haggard, "Coercion changes the sense of agency in the human brain," Current biology, vol. 26, no. 5, pp. 585--592, 2016.
[24]
D. Doli'ski, T. Grzyb, M. Folwarczny, P. Grzybaa, K. Krzyszycha, K. Martynowska, and J. Trojanowski, "Would you deliver an electric shock in 2015? obedience in the experimental paradigm developed by stanley milgram in the 50 years following the original studies," Social Psychological and Personality Science, vol. 8, no. 8, pp. 927--933, 2017.
[25]
S. Milgram, "Obedience to authority," 1974.
[26]
I. Rock and S. Palmer, "The legacy of gestalt psychology," Scientific American, vol. 263, no. 6, pp. 84--91, 1990.
[27]
E. Aronson and G. Lindzey, The handbook of social psychology. AddisonWesley Publishing Company, 1968.
[28]
E. E. Jones and H. Sigall, "The bogus pipeline: A new paradigm for measuring affect and attitude.," Psychological Bulletin, vol. 76, no. 5, p. 349, 1971.
[29]
J. J. Cummings and J. N. Bailenson, "How immersive is enough? a meta-analysis of the effect of immersive technology on user presence," Media Psychology, vol. 19, no. 2, pp. 272--309, 2016.
[30]
M. Slater, M. Usoh, and A. Steed, "Depth of presence in virtual environments," Presence: Teleoperators & Virtual Environments, vol. 3, no. 2, pp. 130--144, 1994.
[31]
B. G. Witmer and M. J. Singer, "Measuring presence in virtual environments: A presence questionnaire," Presence, vol. 7, no. 3, pp. 225--240, 1998.
[32]
T. Schubert, F. Friedmann, and H. Regenbrecht, "The experience of presence: Factor analytic insights," Presence: Teleoperators & Virtual Environments, vol. 10, no. 3, pp. 266--281, 2001.
[33]
J. Lessiter, J. Freeman, E. Keogh, and J. Davidoff, "A cross-media presence questionnaire: The itc-sense of presence inventory," Presence: Teleoperators & Virtual Environments, vol. 10, no. 3, pp. 282--297, 2001.
[34]
M. Lombard, T. B. Ditton, D. Crane, B. Davis, G. Gil-Egui, K. Horvath, J. Rossman, and S. Park, "Measuring presence: A literature-based approach to the development of a standardized paper-and-pencil instrument," in Third international workshop on presence, delft, the netherlands, vol. 240, pp. 2--4, 2000.
[35]
C. Rosakranse and S. Y. Oh, "Measuring presence: The use trends of five canonical presence questionnaires from 1998--2012," in Challenging presence: Proceedings of the 15th international conference on presence, pp. 25--30, 2014.
[36]
B. E. Insko, "Measuring presence: Subjective, behavioral and physiological methods.," 2003.
[37]
N. Yee and J. Bailenson, "The proteus effect: The effect of transformed self-representation on behavior," Human communication research, vol. 33, no. 3, pp. 271--290, 2007.
[38]
S. J. G. Ahn, J. N. Bailenson, and D. Park, "Short-and long-term effects of embodied experiences in immersive virtual environments on environmental locus of control and behavior," Computers in Human Behavior, vol. 39, pp. 235--245, 2014.
[39]
B. K. Wiederhold, D. P. Jang, M. Kaneda, I. Cabral, Y. Lurie, T. May, I. Kim, M. D. Wiederhold, and S. Kim, "An investigation into physiological responses in virtual environments: an objective measurement of presence," Towards cyberpsychology: Mind, cognitions and society in the internet age, 2001.
[40]
Z. Henkel, N. Rashidi, A. Rice, and R. Murphy, "Survivor buddy: A social medium robot," in Proceedings of the 6th international conference on Human-robot interaction, pp. 387--388, ACM, 2011.
[41]
M. Slater and A. Steed, "A virtual presence counter," Presence: Teleoperators & Virtual Environments, vol. 9, no. 5, pp. 413--434, 2000.
[42]
M. Slater, A. Brogni, and A. Steed, "Physiological responses to breaks in presence: A pilot study," in Presence 2003: The 6th Annual International Workshop on Presence, vol. 157, Citeseer, 2003.
[43]
B. Rey, V. Parkhutik, J. Tembl, and M. Alcañiz, "Breaks in presence in virtual environments: An analysis of blood flow velocity responses," Presence: Teleoperators and Virtual Environments, vol. 20, no. 3, pp. 273-- 286, 2011.
[44]
T. Marsh, K. Yang, C. Shahabi, W. L. Wong, L. Nocera, E. Carriazo, A. Varma, H. Yoon, and C. Kyriakakis, "Automating the detection of breaks in continuous user experience with computer games," in CHI'05 Extended Abstracts on Human Factors in Computing Systems, pp. 1629-- 1632, ACM, 2005.
[45]
V. Groom, V. Srinivasan, C. L. Bethel, R. Murphy, L. Dole, and C. Nass, "Responses to robot social roles and social role framing," in Collaboration Technologies and Systems (CTS), 2011 International Conference on, pp. 194--203, IEEE, 2011.
[46]
L. D. Dole, D. M. Sirkin, R. M. Currano, R. R. Murphy, and C. I. Nass, "Where to look and who to be designing attention and identity for search-and-rescue robots," in Human-Robot Interaction (HRI), 2013 8th ACM/IEEE International Conference on, pp. 119--120, IEEE, 2013.
[47]
L. D. Dole, D. M. Sirkin, R. R. Murphy, and C. I. Nass, "Robots need humans in the loop to improve the hopefulness of disaster survivors," in Robot and Human Interactive Communication (RO-MAN), 2015 24th IEEE International Symposium on, pp. 707--714, IEEE, 2015.

Cited By

View all
  • (2024)Portobello: Extending Driving Simulation from the Lab to the RoadProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642341(1-13)Online publication date: 11-May-2024
  • (2024)Vid2Real HRI: Align video-based HRI study designs with real-world settings2024 33rd IEEE International Conference on Robot and Human Interactive Communication (ROMAN)10.1109/RO-MAN60168.2024.10731413(542-548)Online publication date: 26-Aug-2024
  • (2024)Assessment and treatment of visuospatial neglect using active learning with Gaussian processes regressionArtificial Intelligence in Medicine10.1016/j.artmed.2024.102770149:COnline publication date: 1-Mar-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems
May 2019
9077 pages
ISBN:9781450359702
DOI:10.1145/3290605
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 May 2019

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. dependents
  2. ecological validity
  3. emergence
  4. experimental realism
  5. face validity
  6. human-robot interaction
  7. immersion
  8. mundane realism
  9. presence
  10. search and rescue
  11. transposition
  12. virtual reality

Qualifiers

  • Research-article

Conference

CHI '19
Sponsor:

Acceptance Rates

CHI '19 Paper Acceptance Rate 703 of 2,958 submissions, 24%;
Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025
ACM CHI Conference on Human Factors in Computing Systems
April 26 - May 1, 2025
Yokohama , Japan

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)40
  • Downloads (Last 6 weeks)5
Reflects downloads up to 10 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Portobello: Extending Driving Simulation from the Lab to the RoadProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642341(1-13)Online publication date: 11-May-2024
  • (2024)Vid2Real HRI: Align video-based HRI study designs with real-world settings2024 33rd IEEE International Conference on Robot and Human Interactive Communication (ROMAN)10.1109/RO-MAN60168.2024.10731413(542-548)Online publication date: 26-Aug-2024
  • (2024)Assessment and treatment of visuospatial neglect using active learning with Gaussian processes regressionArtificial Intelligence in Medicine10.1016/j.artmed.2024.102770149:COnline publication date: 1-Mar-2024
  • (2024)Application of time series analysis to improve the validity of Immersive virtual environments for collecting occupant thermal state and adaptive behavioral intention dataAdvanced Engineering Informatics10.1016/j.aei.2024.10244961:COnline publication date: 1-Aug-2024
  • (2024)Co-existing with Drones: A Virtual Exploration of Proxemic Behaviours and Users’ Insights on Social DronesInternational Journal of Social Robotics10.1007/s12369-024-01111-716:3(547-567)Online publication date: 6-Mar-2024
  • (2023)Simulating Wearable Urban Augmented Reality Experiences in VR: Lessons Learnt from Designing Two Future Urban InterfacesMultimodal Technologies and Interaction10.3390/mti70200217:2(21)Online publication date: 16-Feb-2023
  • (2023)Behavior as a Function of Video Quality in an Ecologically Valid ExperimentProceedings of the 2023 ACM International Conference on Interactive Media Experiences10.1145/3573381.3597235(415-418)Online publication date: 12-Jun-2023
  • (2023)Modeling Adaptive Expression of Robot Learning Engagement and Exploring Its Effects on Human TeachersACM Transactions on Computer-Human Interaction10.1145/357181330:5(1-48)Online publication date: 23-Sep-2023
  • (2023)Grab It, While You Can: A VR Gesture Evaluation of a Co-Designed Traditional Narrative by Indigenous PeopleProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580894(1-13)Online publication date: 19-Apr-2023
  • (2023)Towards enhancing ecological validity in user studies: a systematic review of guidelines and implications for QoE researchQuality and User Experience10.1007/s41233-023-00059-28:1Online publication date: 5-Jul-2023
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media