skip to main content
10.1145/2740908.2744109acmotherconferencesArticle/Chapter ViewAbstractPublication PagesthewebconfConference Proceedingsconference-collections
other

The Dynamics of Micro-Task Crowdsourcing: The Case of Amazon MTurk

Published: 18 May 2015 Publication History

Abstract

Micro-task crowdsourcing is rapidly gaining popularity among research communities and businesses as a means to leverage Human Computation in their daily operations. Unlike any other service, a crowdsourcing platform is in fact a marketplace subject to human factors that affect its performance, both in terms of speed and quality. Indeed, such factors shape the \emph{dynamics} of the crowdsourcing market. For example, a known behavior of such markets is that increasing the reward of a set of tasks would lead to faster results. However, it is still unclear how different dimensions interact with each other: reward, task type, market competition, requester reputation, etc.
In this paper, we adopt a data-driven approach to (A) perform a long-term analysis of a popular micro-task crowdsourcing platform and understand the evolution of its main actors (workers, requesters, tasks, and platform). (B) We leverage the main findings of our five year log analysis to propose features used in a predictive model aiming at determining the expected performance of any batch at a specific point in time. We show that the number of tasks left in a batch and how recent the batch is are two key features of the prediction. (C) Finally, we conduct an analysis of the demand (new tasks posted by the requesters) and supply (number of tasks completed by the workforce) and show how they affect task prices on the marketplace.

Cited By

View all
  • (2024)Exploring Determinants of Second-Hand Apparel Purchase Intention and Word of Mouth: A Stimulus–Organism–Response ApproachSustainability10.3390/su1611444516:11(4445)Online publication date: 24-May-2024
  • (2024)Platform-mediated work in Poland: Worker characteristics and prevalence in societyInternational Journal of Management and Economics10.2478/ijme-2024-0007Online publication date: 22-Feb-2024
  • (2024)Explaining crowdworker behaviour through computational rationalityBehaviour & Information Technology10.1080/0144929X.2024.2329616(1-22)Online publication date: 24-Apr-2024
  • Show More Cited By

Index Terms

  1. The Dynamics of Micro-Task Crowdsourcing: The Case of Amazon MTurk

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    WWW '15 Companion: Proceedings of the 24th International Conference on World Wide Web
    May 2015
    1602 pages
    ISBN:9781450334730
    DOI:10.1145/2740908
    Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

    Sponsors

    • IW3C2: International World Wide Web Conference Committee

    In-Cooperation

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 18 May 2015

    Check for updates

    Author Tags

    1. crowdsourcing
    2. forecasting
    3. tracking
    4. trend identification

    Qualifiers

    • Other

    Conference

    WWW '15
    Sponsor:
    • IW3C2

    Acceptance Rates

    Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)6
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 18 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Exploring Determinants of Second-Hand Apparel Purchase Intention and Word of Mouth: A Stimulus–Organism–Response ApproachSustainability10.3390/su1611444516:11(4445)Online publication date: 24-May-2024
    • (2024)Platform-mediated work in Poland: Worker characteristics and prevalence in societyInternational Journal of Management and Economics10.2478/ijme-2024-0007Online publication date: 22-Feb-2024
    • (2024)Explaining crowdworker behaviour through computational rationalityBehaviour & Information Technology10.1080/0144929X.2024.2329616(1-22)Online publication date: 24-Apr-2024
    • (2023)Efficacy of an Unguided, Digital Single-Session Intervention for Internalizing Symptoms in Web-Based Workers: Randomized Controlled TrialJournal of Medical Internet Research10.2196/4541125(e45411)Online publication date: 7-Jul-2023
    • (2023)On Dynamically Pricing Crowdsourcing TasksACM Transactions on Knowledge Discovery from Data10.1145/354401817:2(1-27)Online publication date: 20-Feb-2023
    • (2023)Identify Javascript Trends in Crowdsourcing Small TasksEvaluation of Novel Approaches to Software Engineering10.1007/978-3-031-36597-3_9(179-204)Online publication date: 8-Jul-2023
    • (2022)Does Sustainable Consumption Behaviour Influence Luxury Services Purchase Intention?Sustainability10.3390/su1413790614:13(7906)Online publication date: 29-Jun-2022
    • (2022)Evaluating the crowd quality for subjective questions based on a Spark computing environmentFuture Generation Computer Systems10.1016/j.future.2020.01.010106:C(426-437)Online publication date: 22-Apr-2022
    • (2022)Hourly Wages in Crowdworking: A Meta-AnalysisBusiness & Information Systems Engineering10.1007/s12599-022-00769-564:5(553-573)Online publication date: 30-Aug-2022
    • (2021)A Brief Taxonomy of Hybrid IntelligenceForecasting10.3390/forecast30300393:3(633-643)Online publication date: 1-Sep-2021
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media