skip to main content
10.1145/3036331.3036351acmotherconferencesArticle/Chapter ViewAbstractPublication PagesiccmsConference Proceedingsconference-collections
research-article

A New Credibility Assessment Framework for Training Simulators

Published: 20 January 2017 Publication History

Abstract

To solve the issues in credibility assessment for training simulators such as the afterthought assessment, both human-in-the-loop and hardware-in-the-loop simulation, a new credibility assessment framework for training simulators is presented comprehensively. An overview of representative credibility assessment methodologies for training simulators is summarized and the problems which including the complex tailoring process of existing credibility assessment framework and the one-sided assessment measurement are analyzed. Then, a new credibility assessment framework which comprises project processes, technical processes and information artifacts is developed. Different evaluation perspectives including system function evaluation, system performance evaluation, sense fidelity evaluation and training effectiveness evaluation are considered in technical processes. Finally, an application case is given to demonstrate how to use the framework proposed to assess the credibility of training simulators.

References

[1]
Lemmers A., Roza M., Voogd J., and F. van Heck. 2003. V&V study of an F-16 familiarization training simulator. Fall SIW Proceeding, 13F-SIW-016.
[2]
Department of Defense. 1996. Verification Validation and Accreditation (VV&A) Recommended Practice Guide. Technical Report. USA: US Department of Defense.
[3]
IEEE. 1997. Verification, Validation, and Accreditation for Distributed Interactive Simulation, IEEE Std 1278.4-1997(R2002). Technical Report.
[4]
IEEE, An Overlay to the High Level Architecture Federation Development and Execution Process, IEEE Std 1516. Technical Report.
[5]
SISO. 2013. GM-VV Vol. 3: Reference Manual, SISO-GUIDE-001.3-2013. Technical Report.
[6]
Roza M., Lemmers A., F. van Heck. 2013. Verification and Validation of Distributed Air Operation Mission Training Environment. In Proceedings of the AIAA Modeling and Simulation Technology Conference, Boston, USA.
[7]
Voogd, J., M. Roza, A. van Lier. 2011. The GM-VV Tailored for a Naval Ship-Handling Training Simulation. 2011 International Simulation Multi-Conference, (Den Haag, The Netherlands, June 2011).
[8]
Brade, D., S. Pohl, S. Youngblood. 2005. Preliminary Convention Report: First Combined Con-vention on International VV&A Standardization En-deavors (CConVV&A). Munich, Germany October 20-23.
[9]
Brade, D. et al. 2005. WEAG THALES JP11.20 2005. Final state of the REVVA methodology. Spring SIW-05, San Diego.
[10]
ITOP. 2004. General Procedure for Modeling and Simulation Verification and Validation Information Exchange. Technical Report. International Test Operations Procedure 1-1-002, WGE 7.2.
[11]
Lynn M., Michelle F. 2015. A validation study of driving errors using a driving simulator. Transportation Research Part F, (Jan. 2015), 14--21.
[12]
Geraint L. H., Thomas P. C., Pritam S., et al. 2015. Validation of a pediatric single-port laparoscopic surgery simulator. Journal of Pediatric Surgery (Mar. 2015) 1762--1766.
[13]
Ganesh S., Baichun L., Amie M., et al. 2015. "Face validation of the Virtual Electrosurgery Skill Trainer". Surg Endosc (Mar. 2015), 730--738.
[14]
VanderVliet G. M., Wilkinson C. H., Roscoe M. F. 2001. Verification, Validation, and Accreditation of a flight simulator: the Jship experience. In Proceedings of the AIAA Modeling and Simulation Technology Conference, (Montreal, Canada, August 2001).
[15]
Roza, M. J. Voogd, D. Sebalj. 2014. The Generic Methodology for Verification and Validation (GM-VV) to support acceptance of models, simulations and data. National Aerospace Laboratory NLR, (Mar. 2014).
[16]
Voogd, J., M. Roza, A. Lemmers. 2013. Implementing GM-VV enterprise, project and technical levels. Simulation Interoperability Workshop, (Orlando, FL, September 16-20, 2013).

Cited By

View all
  • (2021)Simulation Credibility Assessment Methodology With FPGA-based Hardware-in-the-Loop PlatformIEEE Transactions on Industrial Electronics10.1109/TIE.2020.298212268:4(3282-3291)Online publication date: Apr-2021
  • (2018)Credibility evaluation of hardware-in-the-loop simulation systems2018 Chinese Control And Decision Conference (CCDC)10.1109/CCDC.2018.8407782(3794-3799)Online publication date: Jun-2018
  • (2018)A Credibility Assessment Method for Training Simulations from the View of Training EffectivenessMethods and Applications for Modeling and Simulation of Complex Systems10.1007/978-981-13-2853-4_25(316-329)Online publication date: 18-Oct-2018

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
ICCMS '17: Proceedings of the 8th International Conference on Computer Modeling and Simulation
January 2017
207 pages
ISBN:9781450348164
DOI:10.1145/3036331
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

  • Central Queensland University
  • University of Canberra: University of Canberra

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 January 2017

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. credibility assessment
  2. performance evaluation
  3. sense fidelity evaluation
  4. training effectiveness evaluation
  5. training simulator

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

Conference

ICCMS '17

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)10
  • Downloads (Last 6 weeks)2
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2021)Simulation Credibility Assessment Methodology With FPGA-based Hardware-in-the-Loop PlatformIEEE Transactions on Industrial Electronics10.1109/TIE.2020.298212268:4(3282-3291)Online publication date: Apr-2021
  • (2018)Credibility evaluation of hardware-in-the-loop simulation systems2018 Chinese Control And Decision Conference (CCDC)10.1109/CCDC.2018.8407782(3794-3799)Online publication date: Jun-2018
  • (2018)A Credibility Assessment Method for Training Simulations from the View of Training EffectivenessMethods and Applications for Modeling and Simulation of Complex Systems10.1007/978-981-13-2853-4_25(316-329)Online publication date: 18-Oct-2018

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media