skip to main content
10.1145/2660398.2660403acmconferencesArticle/Chapter ViewAbstractPublication PagesgroupConference Proceedingsconference-collections
research-article

Turk-Life in India

Published: 09 November 2014 Publication History

Abstract

Previous studies on Amazon Mechanical Turk (AMT), the most well-known marketplace for microtasks, show that the largest population of workers on AMT is U.S. based, while the second largest is based in India. In this paper, we present insights from an ethnographic study conducted in India to introduce some of these workers or "Turkers" -- who they are, how they work and what turking means to them. We examine the work they do to maintain their reputations and their work-life balance. In doing this, we illustrate how AMT's design practically impacts on turk-work. Understanding the "lived work" of crowdwork is a valuable first step for technology design.

References

[1]
Bederson, B. B. and Quinn, A. J. 2011. Web workers unite! addressing challenges of online laborers. In Proceedings of CHI EA '11. ACM (2011), 97--106.
[2]
Bourne, K. A., and Forman, P. J. (2013). Living in a Culture of Overwork: An Ethnographic Study of Flexibility. Journal of Management Inquiry, 25 March 2013.
[3]
Crabtree, A. 2003. Designing Collaborative Systems: A Practical Guide to Ethnography, Springer.
[4]
Crabtree, A., Nichols, D. M., O'Brien, J., Rouncefield, M., & Twidale, M. B. 2000. Ethnomethodologically informed ethnography and information system design. Journal of the American Society for Information Science, 51(7), 666--682.
[5]
Felsteiner, A. 2011. Working the crowd: employment and labor law in the crowdsourcing industry. Berkeley Journal of Employment & Labor Law 32(1), 143--204.
[6]
Felstiner, A. 2013. The weakness of crowds. Limn, Crowds and Clouds (2), Retrieved from http://limn.it/the-weakness-ofcrowds/.
[7]
Garfinkel, H., 1967. Studies in ethnomethodology. Englewood Cliffs, N.J.: Prentice-Hall.
[8]
Gawade, M., Vaish, R., Waihumbu, M, N., Davis, J., 2012. Exploring Employment Opportunities through Microtasks via Cybercafes. In Proceedings of IEEE GHTC'12 77--82.
[9]
Howe, J., 2006. The rise of crowdsourcing. Wired Magazine Issue 14.06. Available at http://www.wired.com/wired/archive/14.06/crowds.html
[10]
Hughes: Hughes, J. A., Randall, D., & Shapiro, D. (1992, December). Faltering from ethnography to design. In Proceedings of the 1992 ACM conference on Computer-supported cooperative work, 115--122.
[11]
Hughes, J., King, V., Rodden, T., Andersen, H. (1994) Moving Out from the Control Room: Ethnography in System Design. In Proceedings of ACM CSCW'94 Conference on Computer-Supported Cooperative Work, 429--439.
[12]
Ipeirotis, P. 2010. Analyzing the Amazon Mechanical Turk marketplace. CeDER Working Papers No. CeDER-10-04. Available at http://hdl.handle.net/2451/29801
[13]
Ipeirotis, P. 2010. Demographics of Mechanical Turk. CeDER Working Papers No. CeDER-10-01. http://hdl.handle.net/2451/29585
[14]
Irani, L. and Silberman, M. S. 2013. Turkopticon: Interrupting Worker Invisibility in Amazon Mechanical Turk. In Proceedings of CHI 2013, ACM Press.
[15]
Jensen, R. E., & Bjørn, P. (2012). Divergence and Convergence in Global Software Development: Cultural Complexities as Social Worlds. From research to practice in the design of cooperative systems. pp. 123--136
[16]
Kaufmann, N., Schulze, T., and Veit, D. 2011. More than fun and money. Worker Motivation in Crowdsourcing -- A Study on Mechanical Turk. In Proceedings of AMCIS 2011.
[17]
Khanna, S., Ratan, A., Davis, J., Thies, W. 2010. Evaluating and Improving the Usability of Mechanical Turk for Low-Income Workers in India. In Proceedings of ACM DEV 2010.
[18]
Kittur,A., Chi, E. H., and Suh, B. 2008. Crowdsourcing user studies with Mechanical Turk. In Proceedings of the CHI '08. ACM, 453--456.
[19]
Kittur, A., Nickerson, J. V., Bernstein, M., Gerber, E., Shaw, A., Zimmerman, J., Lease, M. and Horton, J. 2013. The future of crowd work. In Proceedings of CSCW '13, ACM press, 1301--1318.
[20]
Kochhar, S., Mazzochi, S., Paritosh, P. 2010. The Anatomy of a Large-Scale Human Computation Engine. HCOMP'10. ACM press. 10--17.
[21]
Le, J., Edmonds, A., Hester, V., Biewald, L. 2010. Ensuring quality in crowdsourced search relevance evaluation: The effects of training question distribution. SIGIR'10. 17--20.
[22]
Little, G., Chilton, L. B., Goldman, M. and Miller, R. C. 2009. Turkit: tools for iterative tasks on mechanical turk. In Proceedings of KDD-HCOMP '09, 2009.
[23]
Martin, D., Hanrahan, B. J., O'Neill, J., Gupta, N., 2014. Being a Turker. In Proceedings of ACM CSCW '14, 224--235.
[24]
O'Neill, J., Roy, S., Grasso, A., Martin, D., 2013. Form Digitization in BPO: From outsourcing to crowdsourcing? In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'13).
[25]
O'Neill J., Martin D. 2013. Relationship-based Business Process Crowdsourcing? In Proceedings of IFIP Conference on Human-Computer Interaction'13. 429--446.
[26]
Pontin, J., 2007. Artificial Intelligence, With Help From the Humans. The New York Times. Available at http://www.nytimes.com/2007/03/25/business/yourmoney/25Stream.html
[27]
Randall, D., Harper, R. & Rouncefield, M. (2007). Fieldwork for Design: Theory and Practice. Springer Verlag, New York.
[28]
Ratan, A. L., Satpathy, S., Zia, L., Toyama, K., Blagsvedt, S., Pawar, U. S., Subramaniam, T., 2009. Kelsa+: digital literacy for low-income office workers. Proceedings of the 3rd International Conference on Information and Communication Technologies and Development, 2009.
[29]
Ross, J., Irani, L., Silberman, M. S., Zaldivar, A., and Tomlinson, B. 2010. Who are the crowdworkers': shifting demographics in mechanical turk. In Proceedings of CHI EA '10, ACM (2010), 2863--2872.
[30]
Shaw, A., 2010. For love or for money? A list experiment on the motivations behind crowdsourcing work. Available at https://crowdflower.com/blog/2010/08/for-love-or-for-money-a-list-experiment-on-the-motivations-behind-crowdsourcing-work/
[31]
Silberman, M. S. 2010. What's fair? Rational action and its residuals in an electronic market. Unpublished manuscript. http://www.scribd.com/doc/86592724/Whats-Fair
[32]
Silberman, M. S., Irani, L., and Ross, J. 2010. Ethics and Tactics of Professional Crowdwork. ACM XRDS, 17, 2.
[33]
Sommerville, I., Rodden, T., Sawyer, P. & Bentley, R. 1992. Sociologists can be surprisingly useful in interactive systems design. In proceedings of HCI '92, CUP. People and computers, 341--341.
[34]
Star, S.L. and Strauss, A.L. 1998. Layers of silence, arenas of voice: The ecology of visible and invisible work?. CSCW Journal, 8: 9--30.
[35]
Tanney, J. (2013) Rules, Reason and Self-knowledge. Harvard University Press
[36]
Thies, W., Ratan, A., Davis, J.: 2011. Paid crowdsourcing as a vehicle for global development. In: CHI '11 Workshop on Crowdsourcing and Human Computation

Cited By

View all
  • (2024)»Künstliche Künstliche Intelligenz«»Artificial Artificial Intelligence«PROKLA. Zeitschrift für kritische Sozialwissenschaft10.32387/prokla.v54i217.215854:217(627-649)Online publication date: 11-Dec-2024
  • (2024)A Culturally-Aware AI Tool for Crowdworkers: Leveraging Chronemics to Support Diverse Work StylesProceedings of the ACM on Human-Computer Interaction10.1145/36868998:CSCW2(1-34)Online publication date: 8-Nov-2024
  • (2024)‘Your Duties Are To Sweep A Floor Remotely’: Low Information Quality in Job Advertisements is a Barrier to Low-Income Job-Seekers’ Successful Use of Digital PlatformsProceedings of the 3rd Annual Meeting of the Symposium on Human-Computer Interaction for Work10.1145/3663384.3663403(1-20)Online publication date: 25-Jun-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
GROUP '14: Proceedings of the 2014 ACM International Conference on Supporting Group Work
November 2014
340 pages
ISBN:9781450330435
DOI:10.1145/2660398
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 November 2014

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. amazon mechanical turk (amt)
  2. crowdsourcing
  3. crowdworkers
  4. ethnography
  5. relationship-based crowdsourcing
  6. requesters
  7. turkers

Qualifiers

  • Research-article

Conference

GROUP '14
Sponsor:
GROUP '14: 2014 ACM Conference on Supporting Groupwork
November 9 - 12, 2014
Florida, Sanibel Island, USA

Acceptance Rates

GROUP '14 Paper Acceptance Rate 27 of 90 submissions, 30%;
Overall Acceptance Rate 125 of 405 submissions, 31%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)32
  • Downloads (Last 6 weeks)4
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)»Künstliche Künstliche Intelligenz«»Artificial Artificial Intelligence«PROKLA. Zeitschrift für kritische Sozialwissenschaft10.32387/prokla.v54i217.215854:217(627-649)Online publication date: 11-Dec-2024
  • (2024)A Culturally-Aware AI Tool for Crowdworkers: Leveraging Chronemics to Support Diverse Work StylesProceedings of the ACM on Human-Computer Interaction10.1145/36868998:CSCW2(1-34)Online publication date: 8-Nov-2024
  • (2024)‘Your Duties Are To Sweep A Floor Remotely’: Low Information Quality in Job Advertisements is a Barrier to Low-Income Job-Seekers’ Successful Use of Digital PlatformsProceedings of the 3rd Annual Meeting of the Symposium on Human-Computer Interaction for Work10.1145/3663384.3663403(1-20)Online publication date: 25-Jun-2024
  • (2024)Explaining crowdworker behaviour through computational rationalityBehaviour & Information Technology10.1080/0144929X.2024.232961644:3(552-573)Online publication date: 24-Apr-2024
  • (2024)New Digital Economic GeographyDigital Geographies—Urbanisation, Economy, and Modelling10.1007/978-981-97-9278-8_6(599-833)Online publication date: 18-Dec-2024
  • (2023)“Sometimes It’s Like Putting the Track in Front of the Rushing Train”: Having to Be ‘On Call’ for Work Limits the Temporal Flexibility of CrowdworkersACM Transactions on Computer-Human Interaction10.1145/363514531:2(1-45)Online publication date: 4-Dec-2023
  • (2023)What You Show is What You Get! Gestures for Microtask CrowdsourcingCompanion Proceedings of the 28th International Conference on Intelligent User Interfaces10.1145/3581754.3584175(255-258)Online publication date: 27-Mar-2023
  • (2023)Print Your Money: Cash-Like Experiences with Digital MoneyExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3585772(1-7)Online publication date: 19-Apr-2023
  • (2023)Do you see what I see? A social capital perspective on microtask gig worker opportunity recognition within electronic networks of practiceInternational Journal of Information Management10.1016/j.ijinfomgt.2022.10261569(102615)Online publication date: Apr-2023
  • (2023)Undue Influence or Exploitation — A Qualitative Inquiry into an Ethical Dilemma Around Payment in Crowd Work-Based Research in the U.S.Computer Supported Cooperative Work (CSCW)10.1007/s10606-023-09472-933:3(329-369)Online publication date: 23-May-2023
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media