FlexCCT: A Methodological Framework and Software for Ratings Analysis and Wisdom of the Crowd Applications | IEEE Journals & Magazine | IEEE Xplore

FlexCCT: A Methodological Framework and Software for Ratings Analysis and Wisdom of the Crowd Applications


Abstract:

Flexible cultural consensus theory (FlexCCT) provides an integrated framework and tool set for analyzing and aggregating ratings. It utilizes a likelihood-based statistic...Show More

Abstract:

Flexible cultural consensus theory (FlexCCT) provides an integrated framework and tool set for analyzing and aggregating ratings. It utilizes a likelihood-based statistical model to create aggregate ratings weighted for rater competencies and rater biases. It has features for the analysis of multiple rating cultures and for consensus adjusted reliability. Multiple optimization algorithms are implemented for FlexCCT, along with a range of model identifiability options to restrict certain subsets of the model parameters. Bootstrapping- and jackknifing-based methods are implemented to give confidence intervals for parameters. A holdout validation method that generates a measure of “prediction log-likelihood” allows for the testing of solution reliability and stability. Empirical work demonstrates the utility of FlexCCT on unsupervised wisdom of the crowd problems, and an example shows how FlexCCT can be used to analyze educational grading data.
Published in: IEEE Transactions on Computational Social Systems ( Volume: 5, Issue: 2, June 2018)
Page(s): 358 - 370
Date of Publication: 05 March 2018

ISSN Information:

Funding Agency:


References

References is not available for this document.