Abstract
In recent years, spatial crowdsourcing has emerged as an important new framework, in which each spatial task requires a set of right crowd-workers in the near vicinity to the target locations. Previous studies have focused on spatial task assignment in the static crowdsourcing environment. These algorithms may achieve local optimality by neglecting the uncertain features inherent in real-world crowdsourcing environments, where workers may join or leave during run time. Moreover, spatial task assignment is more complicated when large-scale crowd-workers exist in crowdsourcing environments. The large-scale nature of task assignments poses a significant challenge to uncertain spatial crowdsourcing. In this paper, we propose a novel algorithm combining spatial optimization and multi-agent temporal difference learning (SMATDL). The combination of grid-based optimization and multi-agent learning can achieve higher adaptability and maintain greater efficiency than traditional learning algorithms in the face of large-scale crowdsourcing problems. The SMATDL algorithm decomposes the uncertain crowdsourcing problem into numerous sub-problems by means of a grid-based optimization approach. In order to adapt to the change in the large-scale environment, each agent utilizes temporal difference learning to handle its own spatial region optimization in online crowdsourcing. As a result, multiple agents in SMATDL collaboratively learn to optimize their efforts in accomplishing the global assignment problems efficiently. Through extensive experiments, we illustrate the effectiveness and efficiency of our proposed algorithms on the experimental data sets.












Similar content being viewed by others
References
Awal, G.K., & Bharadwaj, K.K. (2014). Team formation in social networks based on collective intelligence – an evolutionary approach. Applied Intelligence, 41(2), 627–648.
Azevedo, C.R., & Von Zuben, F.J. (2015). Learning to anticipate flexible choices in multiple criteria decision-making under uncertainty. IEEE Transactions on Cybernetics, 46(3), 778–791.
Bloembergen, D., Tuyls, K., Hennes, D., et al. (2015). Evolutionary dynamics of multi-agent learning: a survey. Journal of Artificial Intelligence Research, 53(1), 659–697.
Dai, P., Lin, C.H., Mausam, et al. (2013). POMDP-based control of workflows for crowdsourcing. Artificial Intelligence, 202(9), 52–85.
Demartini, G. (2015). Hybrid human–machine information systems: challenges and opportunities. Computer Networks, 5–13.
Deng, D., Shahabi, C., Demiryurek, U., et al. (2016). Task selection in spatial crowdsourcing from worker’s perspective. Geoinformatica, 20(3), 529–568.
Doan, A., Ramakrishnan, R., Halevy, A.Y. (2011). Crowdsourcing systems on the World-Wide Web Communications of the ACM.
Guo, B., Liu, Y., Wang, L., et al. (2018). Task allocation in spatial crowdsourcing: current state and future directions. IEEE Internet of Things Journal, 5(3), 1749–1764.
Hassan, U.U., & Curry, E. (2016). Efficient task assignment for spatial crowdsourcing: a combinatorial fractional optimization approach with semi-bandit learning. Expert Systems with Applications, 58(C), 36–56.
Hung, N.Q.V., Thang, D.C., Tam, N.T., et al. (2017). Answer validation for generic crowdsourcing tasks with minimal efforts. Vldb Journal, 26(6), 1–26.
Jin, C., Allen-Zhu, Z., Bubeck, S., Jordan, M.I. (2018). Is q-learning provably efficient?. In Advances in neural information processing systems (pp. 4864–4874).
Kazemi, L., Shahabi, C., Chen, L. (2013). Geotrucrowd:trustworthy query answering with spatial crowdsourcing // ACM Sigspatial International Conference on Advances in Geographic Information Systems, ACM, 314–323.
Li, S., Xu, L.D., Zhao, S., et al. (2015). The internet of things: a survey. Information Systems Frontiers, 17(2), 243–259.
Li, S., Xu, L.D., Zhao, S., et al. (2018). 5G internet of things: a survey. Journal of Industrial Information Integration, 1–9.
Li, Y., Cao, B., Xu, L.D., et al. (2014). An efficient recommendation method for improving business process modeling. IEEE Transactions on Industrial Informatics, 10(1), 502–513.
Liu, A., Wang, W., Shang, S., et al. (2018). Efficient task assignment in spatial crowdsourcing with worker and task privacy protection. GeoInformatica, 22(2), 335–362.
Liu, X., Lu, M., Ooi, B.C., et al. (2012). CDAS: a crowdsourcing data analytics system. Proceedings of the VLDB Endowment, 5(10), 1040–1051.
Liu, Y., Alexandrova, T., Nakajima, T., et al. (2013). Using stranger as sensors: temporal and geo-sensitive question answering via social media. In Proceedings of the 22nd international conference on World Wide Web, WWW (pp. 803–814).
Mnih, V., Badia, A.P., Mirza, M., et al. (2016). Asynchronous methods for deep reinforcement learning. In International conference on machine learning (pp. 1928–1937).
Mnih, V., Kavukcuoglu, K., Silver, D., et al. (2015). Human-level control through deep reinforcement learning. Nature, 518(7540), 529–533.
Parameswaran, A., Sarma, A.D., Garcia-Molina, H., et al. (2011). Human-assisted graph search: it’s okay to ask questions. Proceedings of the Vldb Endowment, 4(5), 267–278.
Peng, X., Gu, J., Tan, T.H., et al. (2018). Crowdservice: optimizing mobile crowdsourcing and service composition. ACM Transactions on Internet Technology, 18(2), A1–A23.
Schulman, J., Levine, S., Abbeel, P., Jordan, M., Moritz, P. (2015). Trust region policy optimization. In International conference on machine learning (pp. 1889–1897).
Shen, W., Hao, Q., Yoon, H.J., et al. (2006). Applications of agent-based systems in intelligent manufacturing: an updated review. Advanced Engineering Informatics, 20(4), 415–431.
Sprague, N., & Ballard, D. (2003). Multiple-goal reinforcement learning with modular Sarsa(0) // International Joint Conference on Artificial Intelligence, Morgan Kaufmann Publishers Inc., 1445–1447.
Sun, Y., Tan, W., Li, L.X., et al. (2016). A new method to identify collaborative partners in social service provider networks. Information Systems Frontiers, 18(3), 565–578.
Sun, Y., Wang, J., Tan, W., et al. (2018). Dynamic worker-and-task assignment on uncertain spatial crowdsourcing. In International conference on computer supported cooperative work in design, CSCWD (pp. 755–760).
Sutton, R.S., & Barto, A.G. (2018). Reinforcement learning: an introduction. Cambridge: MIT Press.
Tan, W., Shen, W., Xu, L.D., et al. (2008). A business process intelligence system for enterprise process performance management. IEEE Transactions on Systems, Man, and Cybernetics, 38(6), 745–756.
Tran, L., To, H., Fan, L., et al. (2018). A real-time framework for task assignment in hyperlocal spatial crowdsourcing. ACM Transactions on Intelligent Systems and Technology, 9(3), 37:1-37:26.
Watkins, C.J.C.H. (1989). Learning from delayed rewards. PhD thesis, Cambridge University, Cambridge, England.
Xiong, P., Zhang, L., Zhu, T. (2017). Reward-based spatial crowdsourcing with differential privacy preservation. Enterprise Information Systems, 11(10), 1500–1517.
Xu, L.D., He, W., Li, S., et al. (2014). Internet of things in industries: a survey. IEEE Transactions on Industrial Informatics, 10(4), 2233–2243.
Yang, P., Zhang, N., Zhang, S., et al. (2017). Identifying the most valuable workers in fog-assisted spatial crowdsourcing. IEEE Internet of Things Journal, 4(4), 1193–1203.
Zhang, X., Yang, Z., Liu, Y., et al. (2019). On reliable task assignment for spatial crowdsourcing. IEEE Transactions on Emerging Topics in Computing, 7(1), 174–186.
Zhao, Y., & Zhu, Q. (2014). Evaluation on crowdsourcing research: current status and future direction. Information Systems Frontiers, 16(3), 417–434.
Acknowledgments
This work was supported in part by Anhui Provincial Natural Science Foundation under Grant No.1908085MF191, the Key Program of University Natural Science Foundation of Anhui Province under Grant No. KJ2017A414 and the National Natural Science Foundation of China under Grant No. 61272036.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Sun, Y., Tan, W. Combining Spatial Optimization and Multi-Agent Temporal Difference Learning for Task Assignment in Uncertain Crowdsourcing. Inf Syst Front 22, 1447–1465 (2020). https://doi.org/10.1007/s10796-019-09938-6
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10796-019-09938-6