Skip to main content

Label Enhanced Graph Attention Network for Truth Inference

  • Conference paper
  • First Online:
Artificial Neural Networks and Machine Learning – ICANN 2023 (ICANN 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14257))

Included in the following conference series:

  • 1227 Accesses

Abstract

Crowdsourcing platforms tend to provide redundant noise labels when annotating data for machine learning, which promotes truth inference to be proposed for crowdsourced labels denoising. Existing truth inference methods have effectively improved the model’s inference quality by utilizing Graph Neural Networks (GNNs) to mine potential crowdsourcing relationships. However, the GNN-based methods fail to reflect crowdsourcing behaviors comprehensively when mining crowdsourcing relationships in a task-worker graph, which limits their performance. To cope with this challenge, we propose a Label Enhanced Graph Attention Network for truth inference, which realizes GNN-based truth inference on complete crowdsourcing behaviors to explore latent crowdsourcing relationships between tasks and workers more effectively. Specifically, we construct an attentive label semantic propagation module based on a graph attention network, which propagates crowdsourced label semantics into the process of information aggregation to represent complete crowdsourcing behaviors. In addition, we design a label-aware graph encoder, which captures the semantics of crowdsourced labels more efficiently by reducing the difference between the predicted and the actual distribution of crowdsourced labels. Extensive experiments on eight real-world datasets demonstrate that LEGAT outperforms eight state-of-the-art methods.

This work was supported by the Program for Innovative Research Team in University of the Ministry of Education under grant IRT_17R32, and the National Natural Science Foundation of China under grant 62120106008.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://www.mturk.com/.

  2. 2.

    https://github.com/lazyloafer/TiReMGE/tree/main/data.

  3. 3.

    https://github.com/yuanli/truth-inference-at-scale/tree/master/data.

  4. 4.

    https://github.com/orchidproject/active-crowd-toolkit/tree/master/Data.

  5. 5.

    https://sites.google.com/site/treccrowd/2011.

References

  1. Balayn, A., Soilis, P., Lofi, C., Yang, J., Bozzon, A.: What do you mean? Interpreting image classification with crowdsourced concept extraction and analysis. In: Proceedings of WWW, pp. 1937–1948. ACM (2021)

    Google Scholar 

  2. Shi, Z. R., Lizarondo, L., Fang, F.: A recommender system for crowdsourcing food rescue platforms. In: Proceedings of WWW, pp. 857–865. ACM / IW3C2 (2021)

    Google Scholar 

  3. Qi, G.J., Aggarwal, C.C., Han, J., Huang, T.: Mining collective intelligence in diverse groups. In: Proceedings of WWW, pp. 1041–1052. ACM (2013)

    Google Scholar 

  4. Hanlu, W., Ma, T., Lingfei, W., Fangli, X., Ji, S.: Exploiting heterogeneous graph neural networks with latent worker/task correlation information for label aggregation in crowdsourcing. ACM Trans. Knowl. Discov. Data 16(2), 1–18 (2021)

    Google Scholar 

  5. Wu, G., Zhuo, X., Bao, X., Hu, X., Hong, R., Wu, X.: Crowdsourcing truth inference via reliability-driven multi-view graph embedding. ACM Trans. Knowl. Discov. Data, 17(5), 65:1–65:26 (2023)

    Google Scholar 

  6. Velickovic, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.. Graph attention networks. In: Proceedings of ICLR. OpenReview.net (2018)

    Google Scholar 

  7. Li, Q., Li, Y., Gao, J., Zhao, B., Fan, W., Han, J.: Resolving conflicts in heterogeneous data by truth discovery and source reliability estimation. In: Proceedings of SIGMOD, pp. 1187–1198. ACM (2014)

    Google Scholar 

  8. Gao, Q.L.Y.L.J., Lu, S., Zhao, B., Demirbas, M., Fan, W., Han, J.: A confidence-aware approach for truth discovery on long-tail data. Proc. VLDB Endow. 8(4), 425–436 (2014)

    Article  Google Scholar 

  9. Zhou, D., Basu, S., Mao, Y., Platt, J.: Learning from the wisdom of crowds by minimax entropy. In: Proceedings of NeurIPS, pp. 2204–2212. Curran Associates Inc., (2012)

    Google Scholar 

  10. Zhu, S., Yingnian, W., Mumford, D.: Minimax entropy principle and its application to texture modeling. Neural Comput. 9(8), 1627–1660 (1997)

    Article  Google Scholar 

  11. Qi Li, Yaliang Li, Jing Gao, Bo Zhao, Wei Fan, and Jiawei Han. ZenCrowd: leveraging probabilistic reasoning and crowdsourcing techniques for large-scale entity linking. In: Proceedings of WWW, pp. 1187–1198. ACM (2014)

    Google Scholar 

  12. Whitehill, J., Wu, T.F., Bergsma, J., Movellan, J., Ruvolo, P.: Whose vote should count more: optimal integration of labels from labelers of unknown expertise. In: Proceedings of NeurIPS, pp. 2035–2043. Curran Associates Inc., (2009)

    Google Scholar 

  13. Karger, D., Oh, S., Shah, D.: Iterative learning for reliable crowdsourcing systems. In: Proceedings of NeurIPS, pp. 1953–1961. Curran Associates Inc., (2011)

    Google Scholar 

  14. Dawid, A.P., Skene, A.M.: Maximum likelihood estimation of observer error-rates using the EM algorithm. J. R. Stat. Soc. C-appl. 28(1), 20–28 (1979)

    Google Scholar 

  15. Li, Y., IP Rubinstein, B., Cohn, T.: Truth inference at scale: a Bayesian model for adjudicating highly redundant crowd annotations. In: Proceedings of WWW, pp. 1028–1038. ACM (2019)

    Google Scholar 

  16. Zhang, J., Sheng, V.S., Jian, W., Xindong, W.: Multi-class ground truth inference in crowdsourcing with clustering. IEEE Trans. Knowl. Data Eng. 28(4), 1080–1085 (2016)

    Article  Google Scholar 

  17. Wu, G., Zhou, L., Xia, J., Li, L., Bao, X., Wu, X.: Crowdsourcing truth inference based on label confidence clustering. ACM Trans. Knowl. Discov. Data 17(4), 46:1–46:20 (2023)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Gongqing Wu .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhao, N., Zhuo, X., Wu, G., Zhang, Z. (2023). Label Enhanced Graph Attention Network for Truth Inference. In: Iliadis, L., Papaleonidas, A., Angelov, P., Jayne, C. (eds) Artificial Neural Networks and Machine Learning – ICANN 2023. ICANN 2023. Lecture Notes in Computer Science, vol 14257. Springer, Cham. https://doi.org/10.1007/978-3-031-44216-2_38

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-44216-2_38

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-44215-5

  • Online ISBN: 978-3-031-44216-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics