Abstract
Crowdsourcing is a popular approach for crowd workers collaborating to have tasks done. However, some workers communicate with each other and share answers during the crowdsourcing process. This is referred to as “collusion”. Copying from others and submitting repeated answers are detrimental to the quality of the tasks. Existing studies on collusion detection focus on ground truth problems (e.g., labeling tasks) and require a fixed threshold to be set in advance. In this paper, we aim to detect collusion behavior of workers in an adaptive way, and propose an Adaptive Clustering Based Collusion Detection approach (ACCD) for a broad range of task types and data types solved via crowdsourcing (e.g., continuous rating with or without distributions). Extensive experiments on both real-world and synthetic datasets show the superiority of ACCD over state-of-the-art approaches.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Adams, S.A.: Maintaining the collision of accounts: crowdsourcing sites in health care as brokers in the co-production of pharmaceutical knowledge. Inf. Commun. Soc. 17(6), 657–669 (2014)
Campello, R.J., Moulavi, D., Sander, J.: Density-based clustering based on hierarchical density estimates. In: Pacific-Asia Conference on Knowledge Discovery and Data Mining, pp. 160–172. Springer (2013). https://doi.org/10.1007/978-3-642-37456-2_14
Celis, L.E., Reddy, S.P., Singh, I.P., Vaya, S.: Assignment techniques for crowdsourcing sensitive tasks. In: Proceedings of the 19th ACM Conference on Computer-Supported Co-operative Work & Social Computing, pp. 836–847 (2016)
Chang, J.C., Amershi, S., Kamar, E.: Revolt: Collaborative crowdsourcing for labeling ma- chine learning datasets. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pp. 2334–2346 (2017)
Chen, P.P., Sun, H.L., Fang, Y.L., Huai, J.P.: Collusion-proof result inference in crowdsourc- ing. J. Comput. Sci. Technol. 33(2), 351–365 (2018)
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition, pp. 248–255. IEEE (2009)
Ester, M., Kriegel, H.P., Sander, J., Xu, X., et al.: A density-based algorithm for discovering clusters in large spatial databases with noise. In: KDD, vol. 96, pp. 226–231 (1996)
Fang, Y., Sun, H., Li, G., Zhang, R., Huai, J.: Effective result inference for context-sensitive tasks in crowdsourcing. In: Navathe, S., Wu, W., Shekhar, S., Du, X., Wang, X., Xiong, H. (eds.) Database Systems for Advanced Applications. DASFAA 2016. Lecture Notes in Computer Science, vol. 9642, pp. 33–48. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-32025-0_3
Gadiraju, U., Kawase, R., Dietze, S., Demartini, G.: Understanding malicious behavior in crowdsourcing platforms: the case of online surveys. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pp. 1631–1640 (2015)
Howe, J., et al.: The rise of crowdsourcing. Wired Mag. 14(6), 1–4 (2006)
KhudaBukhsh, A.R., Carbonell, J.G., Jansen, P.J.: Detecting non-adversarial collusion in crowdsourcing. In: Second AAAI Conference on Human Computation and Crowdsourcing (2014)
Kriegel, H.P., Kröger, P., Sander, J., Zimek, A.: Density-based clustering. Wiley Interdiscip. Rev. Data Min. Knowl. Discov. 1(3), 231–240 (2011)
Lev, O., Polukarov, M., Bachrach, Y., Rosenschein, J.S.: Mergers and collusion in all-pay auctions and crowdsourcing contests. In: Proceedings of the 2013 International Conference on Autonomous Agents and Multi-agent Systems, pp. 675–682 (2013)
Liu, X., Lu, M., Ooi, B.C., Shen, Y., Wu, S., Zhang, M.: Cdas: A crowdsourcing data analytics system. arXiv preprint arXiv:1207.0143 (2012)
Marcus, A., Karger, D., Madden, S., Miller, R., Oh, S.: Counting with the crowd. Proceed. VLDB Endow. 6(2), 109–120 (2012)
Niazi Torshiz, M., Amintoosi, H.: Collusion-resistant worker selection in social crowdsensing systems. Comput. Knowl. Eng. 1(1), 9–20 (2018)
Nouri, Z., Wachsmuth, H., Engels, G.: Mining crowdsourcing problems from discussion forums of workers. In: Proceedings of the 28th International Conference on Computational Linguistics, pp. 6264–6276 (2020)
Sheng, V.S., Provost, F., Ipeirotis, P.G.: Get another label? improving data quality and data mining using multiple, noisy labelers. In: Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 614–622 (2008)
Song, C., Liu, K., Zhang, X.: Collusion detection and ground truth inference in crowdsourcing for labeling tasks. J. Mach. Learn. Res. 22(190), 1–45 (2021)
Sun, H., Dong, B., Zhang, B., Wang, W.H., Kantarcioglu, M.: Sensitive task assignments in crowdsourcing markets with colluding workers. In: 2018 IEEE 34th International Conference on Data Engineering (ICDE), pp. 377–388. IEEE (2018)
Von Ahn, L., Maurer, B., McMillen, C., Abraham, D., Blum, M.: recaptcha: human-based character recognition via web security measures. Science 321(5895), 1465–1468 (2008)
Wang, G., et al.: Serf and turf: crowdturfing for fun and profit. In: Proceedings of the 21st international conference on World Wide Web, pp. 679–688 (2012)
Xiang, Q., Nevat, I., Zhang, P., Zhang, J.: Collusion-resistant spatial phenomena crowdsourcing via mixture of gaussian processes regression. In: TRUST@ AAMAS, pp. 19–30 (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Xu, R., Li, G., Jin, W., Chen, A., Sheng, V.S. (2023). Adaptive Clustering-Based Collusion Detection in Crowdsourcing. In: Huang, DS., Premaratne, P., Jin, B., Qu, B., Jo, KH., Hussain, A. (eds) Advanced Intelligent Computing Technology and Applications. ICIC 2023. Lecture Notes in Computer Science(), vol 14089. Springer, Singapore. https://doi.org/10.1007/978-981-99-4752-2_22
Download citation
DOI: https://doi.org/10.1007/978-981-99-4752-2_22
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-4751-5
Online ISBN: 978-981-99-4752-2
eBook Packages: Computer ScienceComputer Science (R0)