Abstract
Crowdsourcing platforms tend to provide redundant noise labels when annotating data for machine learning, which promotes truth inference to be proposed for crowdsourced labels denoising. Existing truth inference methods have effectively improved the model’s inference quality by utilizing Graph Neural Networks (GNNs) to mine potential crowdsourcing relationships. However, the GNN-based methods fail to reflect crowdsourcing behaviors comprehensively when mining crowdsourcing relationships in a task-worker graph, which limits their performance. To cope with this challenge, we propose a Label Enhanced Graph Attention Network for truth inference, which realizes GNN-based truth inference on complete crowdsourcing behaviors to explore latent crowdsourcing relationships between tasks and workers more effectively. Specifically, we construct an attentive label semantic propagation module based on a graph attention network, which propagates crowdsourced label semantics into the process of information aggregation to represent complete crowdsourcing behaviors. In addition, we design a label-aware graph encoder, which captures the semantics of crowdsourced labels more efficiently by reducing the difference between the predicted and the actual distribution of crowdsourced labels. Extensive experiments on eight real-world datasets demonstrate that LEGAT outperforms eight state-of-the-art methods.
This work was supported by the Program for Innovative Research Team in University of the Ministry of Education under grant IRT_17R32, and the National Natural Science Foundation of China under grant 62120106008.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Balayn, A., Soilis, P., Lofi, C., Yang, J., Bozzon, A.: What do you mean? Interpreting image classification with crowdsourced concept extraction and analysis. In: Proceedings of WWW, pp. 1937–1948. ACM (2021)
Shi, Z. R., Lizarondo, L., Fang, F.: A recommender system for crowdsourcing food rescue platforms. In: Proceedings of WWW, pp. 857–865. ACM / IW3C2 (2021)
Qi, G.J., Aggarwal, C.C., Han, J., Huang, T.: Mining collective intelligence in diverse groups. In: Proceedings of WWW, pp. 1041–1052. ACM (2013)
Hanlu, W., Ma, T., Lingfei, W., Fangli, X., Ji, S.: Exploiting heterogeneous graph neural networks with latent worker/task correlation information for label aggregation in crowdsourcing. ACM Trans. Knowl. Discov. Data 16(2), 1–18 (2021)
Wu, G., Zhuo, X., Bao, X., Hu, X., Hong, R., Wu, X.: Crowdsourcing truth inference via reliability-driven multi-view graph embedding. ACM Trans. Knowl. Discov. Data, 17(5), 65:1–65:26 (2023)
Velickovic, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.. Graph attention networks. In: Proceedings of ICLR. OpenReview.net (2018)
Li, Q., Li, Y., Gao, J., Zhao, B., Fan, W., Han, J.: Resolving conflicts in heterogeneous data by truth discovery and source reliability estimation. In: Proceedings of SIGMOD, pp. 1187–1198. ACM (2014)
Gao, Q.L.Y.L.J., Lu, S., Zhao, B., Demirbas, M., Fan, W., Han, J.: A confidence-aware approach for truth discovery on long-tail data. Proc. VLDB Endow. 8(4), 425–436 (2014)
Zhou, D., Basu, S., Mao, Y., Platt, J.: Learning from the wisdom of crowds by minimax entropy. In: Proceedings of NeurIPS, pp. 2204–2212. Curran Associates Inc., (2012)
Zhu, S., Yingnian, W., Mumford, D.: Minimax entropy principle and its application to texture modeling. Neural Comput. 9(8), 1627–1660 (1997)
Qi Li, Yaliang Li, Jing Gao, Bo Zhao, Wei Fan, and Jiawei Han. ZenCrowd: leveraging probabilistic reasoning and crowdsourcing techniques for large-scale entity linking. In: Proceedings of WWW, pp. 1187–1198. ACM (2014)
Whitehill, J., Wu, T.F., Bergsma, J., Movellan, J., Ruvolo, P.: Whose vote should count more: optimal integration of labels from labelers of unknown expertise. In: Proceedings of NeurIPS, pp. 2035–2043. Curran Associates Inc., (2009)
Karger, D., Oh, S., Shah, D.: Iterative learning for reliable crowdsourcing systems. In: Proceedings of NeurIPS, pp. 1953–1961. Curran Associates Inc., (2011)
Dawid, A.P., Skene, A.M.: Maximum likelihood estimation of observer error-rates using the EM algorithm. J. R. Stat. Soc. C-appl. 28(1), 20–28 (1979)
Li, Y., IP Rubinstein, B., Cohn, T.: Truth inference at scale: a Bayesian model for adjudicating highly redundant crowd annotations. In: Proceedings of WWW, pp. 1028–1038. ACM (2019)
Zhang, J., Sheng, V.S., Jian, W., Xindong, W.: Multi-class ground truth inference in crowdsourcing with clustering. IEEE Trans. Knowl. Data Eng. 28(4), 1080–1085 (2016)
Wu, G., Zhou, L., Xia, J., Li, L., Bao, X., Wu, X.: Crowdsourcing truth inference based on label confidence clustering. ACM Trans. Knowl. Discov. Data 17(4), 46:1–46:20 (2023)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhao, N., Zhuo, X., Wu, G., Zhang, Z. (2023). Label Enhanced Graph Attention Network for Truth Inference. In: Iliadis, L., Papaleonidas, A., Angelov, P., Jayne, C. (eds) Artificial Neural Networks and Machine Learning – ICANN 2023. ICANN 2023. Lecture Notes in Computer Science, vol 14257. Springer, Cham. https://doi.org/10.1007/978-3-031-44216-2_38
Download citation
DOI: https://doi.org/10.1007/978-3-031-44216-2_38
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44215-5
Online ISBN: 978-3-031-44216-2
eBook Packages: Computer ScienceComputer Science (R0)