Abstract
When a model is built based on expert knowledge, a small data set will, in many cases, form the base for the model. It must be possible to validate the trustworthiness and model improvement potential of the provided information from humans or machines. In this study, we have investigated how to evaluate the information from humans to improve the model itself. We used evaluation research and collected the research data with the help of focus group interviews and questionnaires. The result of the study suggests a way to determine the trustworthiness of answers from humans and how to understand if these answers indicate a change to the underlying expert model. The introduction of divergence, and candidate areas, made it possible to evaluate the trustworthiness and changes to the expert model. These were deemed valuable by practitioners.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Added as a pre-calculated value in the second version of the questionnaire.
References
Burkov, A.: The Hundred-Page Machine Learning Book. Andriy Burkov (2019)
Davidson-Pilon, C.: Bayesian Methods for Hackers. 1st edn. Addison Wesley Data & Analytics Series (2016)
Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press (2016)
Jarrahi, M.H., Memariani, A., Guha, S.: The Principles of Data-Centric AI (DCAI). arXiv preprint arXiv:2211.14611 (2022)
Lunn, D., Jackson, C., Best, N., Thomas, A., Spiegelhalter, D.: The BUGS Book. CRC Press (2013)
Pearl, J., Glymour, M., Jewell, N.P.: Causal Inference in Statistics. First edn. John Wiley and Sons Inc. (2016)
Powers, D.M.W.: The problem with kappa. In: Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics, pp. 345–355 (2012)
Robson, C.: Real World Research: A Resource for Users of Social Research Methods in Applied Settings, 3rd edn. Wiley, Chichester (2011)
RStudio: RStudio. https://rstudio.com/. Accessed 12 Sept 2019
Scutari, M., Silander, T., Ness, R.: bnlearn. https://cran.r-project.org/web/packages/bnlearn/index.html. Accessed 10 Oct 2022
Scutari, M., Denis, J.B.: Bayesian Networks. CRC Press (2015)
Silvander, J.: Towards intent-driven systems based on context frames. Doctoral dissertation, Blekinge Institute of Technology (2021)
Singh, S., Ali, N., Lundberg, L.: Smart and adaptive architecture for a dedicated Internet of Things network comprised of diverse entities: a proposal and evaluation. Sensors 22(8), 3017 (2022)
Strickland, E.: Andrew Ng: Unbiggen AI. https://spectrum.ieee.org/andrew-ng-data-centric-ai. Accessed 05 Dec 2022
Youden, W.J.: Index for rating diagnostic tests. Cancer 3(1), 32–35 (1950)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Silvander, J., Singh, S.P. (2023). Validating Trust in Human Decisions to Improve Expert Models Based on Small Data Sets. In: Shishkov, B. (eds) Business Modeling and Software Design. BMSD 2023. Lecture Notes in Business Information Processing, vol 483. Springer, Cham. https://doi.org/10.1007/978-3-031-36757-1_17
Download citation
DOI: https://doi.org/10.1007/978-3-031-36757-1_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-36756-4
Online ISBN: 978-3-031-36757-1
eBook Packages: Computer ScienceComputer Science (R0)