Abstract
Crowdsourcing is evolving as a distributed problem-solving and business production model in recent years. In crowdsourcing paradigm, tasks are distributed to networked people to complete such that a company’s production cost can be greatly reduced. In crowdsourcing systems, task recommendation can help workers to find their right tasks faster as well as help requesters to receive good quality output quicker. However, previously proposed classification based task recommendation approach, which is the only one in the literature, does not consider the dynamic scenarios of new workers and new tasks in the crowdsourcing system. In this paper, we propose a Task Recommendation (TaskRec) framework based on a unified probabilistic matrix factorization, aiming to recommend tasks to workers in dynamic scenarios. Unlike traditional recommendation systems, workers do not provide their ratings on tasks in crowdsourcing systems, thus we infer user ratings from their interacting behaviors. This conversion helps task recommendation in crowdsourcing systems. Complexity analysis shows that our framework is efficient and is scalable to large datasets. Finally, we conduct experiments on real-world datasets for performance evaluation. Experimental results show that TaskRec outperforms the state-of-the-art approach.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Amazon Mechanical Turk website: http://www.mturk.com.
CrowdFlower website: http://crowdflower.com.
Samasource website: http://samasource.org.
NAACL 2010 workshop:http://sites.google.com/site/amtworkshop2010/data-1.
References
Yuen MC, King I, Leung KS (2011) A survey of crowdsourcing systems. In: SocialCom ’11: Proceedings of the Third IEEE International Conference on Social Computing, IEEE Computer Society, p. 766–773
Yuen MC, Chen LJ, King I (2009) A survey of human computation systems. In: CSE ’09: Proceedings of IEEE International Conference on Computational Science and Engineering, IEEE Computer Society, p. 723–728
Zhou TC, Ma H, King I, Lyu MR (2009) Tagrec: leveraging tagging wisdom for recommendation. In: Proceedings of the 2009 International Conference on Computational Science and Engineering, vol 04, Washington, DC, USA. IEEE Computer Society, p. 194–199
Ma H, King I, Lyu MR (2007) Effective missing data prediction for collaborative filtering. In: Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’07, New York, NY, USA, ACM, p. 39–46
Cheng C, Yang H, King I, Lyu MR (2012) Fused matrix factorization with geographical and social influence in location-based social networks. In: AAAI ’12: Proceedings of Twenty-Sixth Conference on Artificial Intelligence, Toronto, Canada
Schein AI, Popescul A, Ungar LH, Pennock DM (2002) Methods and metrics for cold-start recommendations. In: Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’02, New York, NY, USA, ACM, p. 253–260
Ambati V, Vogel S, Carbonell J (2011) Towards task recommendation in micro-task markets. In: AAAI ’11: Proceedings of the 25th AAAI Workshop in Human Computation. AAAI Publications
Stewart O, Lubensky D, Huerta JM (2010) Crowdsourcing participation inequality: a scout model for the enterprise domain. In: Proceedings of the ACM SIGKDD Workshop on Human Computation, HCOMP ’10, New York, NY, USA, ACM, p. 30–33
Ross J, Irani L, Six Silberman M, Zaldivar A, Tomlinson B (2010) Who are the crowdworkers? Shifting demographics in mechanical turk. In: Proceedings of the 28th of the International Conference Extended Abstracts on Human Factors in Computing Systems, CHI EA ’10, New York, NY, USA, ACM, p. 2863–2872
Chilton LB, Horton JJ, Miller RC, Azenkot S (2010) Task search in a human computation market. In: Proceedings of the ACM SIGKDD Workshop on Human Computation, HCOMP ’10, New York, NY, USA, ACM, p. 1–9
Koren Y, Bell R, Volinsky C (2009) Matrix factorization techniques for recommender systems. Computer 42(8):30–37
Yang S, Ye M (2013) Global minima analysis of Lee and Seung’s NMF algorithms. Neural Process Lett 38(1):29–51
Yang S, Yi Z (2010) Convergence analysis of non-negative matrix factorization for BSS algorithm. Neural Process Lett 31(1):45–64
Ruslan Salakhutdinov, Andriy Mnih (2008) Probabilistic matrix factorization. In: Advances in neural information processing systems, vol 20
Yuen MC, King I, Leung KS (2011) Task matching in crowdsourcing. In: CPSCom ’11: Proceedings of The 4th IEEE International Conference on Cyber, Physical and Social Computing, IEEE Computer Society, p. 409–412
Yuen MC, King I, Leung KS (2012) Task recommendation in crowdsourcing systems. In: KDD ’12: Proceedings of ACM KDD 2012 Workshop on Data Mining and Knowledge Discovery with Crowdsourcing (CrowdKDD), ACM
Zhang H, Horvitz E, Miller RC, Parkes DC (2011) Crowdsourcing general computation. In: CHI ’11: Proceedings of ACM CHI 2011 Workshop on Crowdsourcing and Human Computation, ACM
Salakhutdinov R, Mnih A (2007) Probabilistic matrix factorization. In: NIPS ’07: Proceedings of the Twenty-First Annual Conference on Neural Information Processing Systems, Curran Associates Inc.
Delbert D, Brendan JF, Delbert D, Brendan JF (2004) Probabilistic sparse matrix factorization. Technical report, University of Toronto
Yuen MC, King I, Leung KS (2012) Taskrec: probabilistic matrix factorization in task recommendation in crowdsourcing systems. In: ICONIP (2), p. 516–525
Acknowledgments
This work was partially supported by grants from the Research Grants Council of the Hong Kong Special Administrative Region, China (CUHK 413212) and Direct Grant (CUHK 2050498).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Yuen, MC., King, I. & Leung, KS. TaskRec: A Task Recommendation Framework in Crowdsourcing Systems. Neural Process Lett 41, 223–238 (2015). https://doi.org/10.1007/s11063-014-9343-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11063-014-9343-z