skip to main content
10.1145/2464464.2464508acmconferencesArticle/Chapter ViewAbstractPublication PageswebsciConference Proceedingsconference-collections
research-article

Collabmap: crowdsourcing maps for emergency planning

Published: 02 May 2013 Publication History

Abstract

In this paper, we present a software tool to help emergency planners at Hampshire County Council in the UK to create maps for high-fidelity crowd simulations that require evacuation routes from buildings to roads. The main feature of the system is a crowdsourcing mechanism that breaks down the problem of creating evacuation routes into microtasks that a contributor to the platform can execute in less than a minute. As part of the mechanism we developed a concensus-based trust mechanism that filters out incorrect contributions and ensures that the individual tasks are complete and correct. To drive people to contribute to the platform, we experimented with different incentive mechanisms and applied these over different time scales, the aim being to evaluate what incentives work with different types of crowds, including anonymous contributors from Amazon Mechanical Turk. The results of the 'in the wild' deployment of the system show that the system is effective at engaging contributors to perform tasks correctly and that users respond to incentives in different ways. More specifically, we show that purely social motives are not good enough to attract a large number of contributors and that contributors are averse to the uncertainty in winning rewards. When taken altogether, our results suggest that a combination of incentives may be the best approach to harnessing the maximum number of resources to get socially valuable tasks (such for planning applications) performed on a large scale.

References

[1]
M. S. Bernstein, G. Little, R. C. Miller, B. Hartmann, M. S. Ackerman, D. R. Karger, D. Crowell, and K. Panovich. Soylent: a word processor with a crowd inside. In Proceedings of UIST 2010, pages 313--322, 2010.
[2]
M. G. Findley, M. C. Gleave, R. N. Morello, and D. L. Nielson. Extrinsic, intrinsic, and social incentives for crowdsourcing development information in uganda: A field experiment. Working Paper, 2012.
[3]
K. Heimerl, B. Gawalt, K. Chen, T. Parikh, and B. Hartmann. Communitysourcing: engaging local crowds to perform expert work via physical kiosks. In Proceedings of the 2012 ACM annual conference on Human Factors in Computing Systems, CHI '12, pages 1539--1548, New York, NY, USA, 2012. ACM.
[4]
C. Heipke. Crowdsourcing geospatial data. ISPRS Journal of Photogrammetry and Remote Sensing, 65(6):550--557, 2010.
[5]
J. Heyman and D. Ariely. Effort for payment a tale of two markets. Psychological Science, 15(11):787--793, 2004.
[6]
W. S. Lasecki, R. Wesley, A. Kulkarni, and J. P. Bigham. Speaking with the crowd. In Proceedings of the Symposium on User Interface Software and Technology (UIST 2012), 2012.
[7]
W. Mason and D. J. Watts. Financial incentives and the "performance of crowds". In Proceedings of the ACM SIGKDD Workshop on Human Computation, HCOMP '09, pages 77--85, New York, NY, USA, 2009. ACM.
[8]
J. Nielsen. Participation inequality: Encouraging more users to contribute, 2006. http://www.useit.com/alertbox/participation_inequality.html.
[9]
A. J. Quinn and B. B. Bederson. Human computation: a survey and taxonomy of a growing field. In Proceedings of the 2011 annual conference on Human factors in computing systems, CHI '11, pages 1403--1412, New York, NY, USA, 2011. ACM.
[10]
J. C. Tang, M. Cebrian, N. A. Giacobe, H.-W. Kim, T. Kim, and D. B. Wickert. Reflecting on the darpa red balloon challenge. Commun. ACM, 54(4):78--85, Apr. 2011.
[11]
S. Van Wart, K. J. Tsai, and T. Parikh. Local ground: a paper-based toolkit for documenting local geo-spatial knowledge. In Proceedings of the First ACM Symposium on Computing for Development, ACM DEV '10, pages 11:1--11:10, New York, NY, USA, 2010. ACM.
[12]
L. Von Ahn, R. Liu, and M. Blum. Peekaboom: a game for locating objects in images. In Proceedings of the SIGCHI conference on Human Factors in computing systems, pages 55--64. ACM, 2006.
[13]
H. Zhang, E. Horvitz, and R. C. Miller. Crowdsourcing general computation. In CHI Workshop on Human Computation, 2011.
[14]
H. Zhang, E. Law, R. Miller, K. Gajos, D. Parkes, and E. Horvitz. Human computation tasks with global constraints. In Proceedings of the 2012 ACM annual conference on Human Factors in Computing Systems, CHI '12, pages 217--226, New York, NY, USA, 2012. ACM.
[15]
M. Zook, M. Graham, T. Shelton, and S. Gorman. Volunteered geographic information and crowdsourcing disaster relief: A case study of the haitian earthquake. World Medical and Health Policy, 2(2), 2010.

Cited By

View all
  • (2022)Efficient and adaptive incentive selection for crowdsourcing contestsApplied Intelligence10.1007/s10489-022-03593-253:8(9204-9234)Online publication date: 6-Aug-2022
  • (2021)Incremental Inference of Provenance TypesProvenance and Annotation of Data and Processes10.1007/978-3-030-80960-7_9(145-162)Online publication date: 9-Jul-2021
  • (2019)TheoryThe Theory and Practice of Social Machines10.1007/978-3-030-10889-2_2(43-102)Online publication date: 15-Feb-2019
  • Show More Cited By

Index Terms

  1. Collabmap: crowdsourcing maps for emergency planning

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    WebSci '13: Proceedings of the 5th Annual ACM Web Science Conference
    May 2013
    481 pages
    ISBN:9781450318891
    DOI:10.1145/2464464
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 02 May 2013

    Permissions

    Request permissions for this article.

    Check for updates

    Qualifiers

    • Research-article

    Funding Sources

    Conference

    WebSci '13
    Sponsor:
    WebSci '13: Web Science 2013
    May 2 - 4, 2013
    Paris, France

    Acceptance Rates

    Overall Acceptance Rate 245 of 933 submissions, 26%

    Upcoming Conference

    Websci '25
    17th ACM Web Science Conference
    May 20 - 24, 2025
    New Brunswick , NJ , USA

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)5
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 28 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2022)Efficient and adaptive incentive selection for crowdsourcing contestsApplied Intelligence10.1007/s10489-022-03593-253:8(9204-9234)Online publication date: 6-Aug-2022
    • (2021)Incremental Inference of Provenance TypesProvenance and Annotation of Data and Processes10.1007/978-3-030-80960-7_9(145-162)Online publication date: 9-Jul-2021
    • (2019)TheoryThe Theory and Practice of Social Machines10.1007/978-3-030-10889-2_2(43-102)Online publication date: 15-Feb-2019
    • (2018)Crowdsourcing Rural Network Maintenance and Repair via Network MessagingProceedings of the 2018 CHI Conference on Human Factors in Computing Systems10.1145/3173574.3173641(1-12)Online publication date: 21-Apr-2018
    • (2018)Crowdsourcing Methods for Data Collection in Geophysics: State of the Art, Issues, and Future DirectionsReviews of Geophysics10.1029/2018RG00061656:4(698-740)Online publication date: 5-Dec-2018
    • (2018)Provenance Network AnalyticsData Mining and Knowledge Discovery10.1007/s10618-017-0549-332:3(708-735)Online publication date: 26-Dec-2018
    • (2015)Embracing CrowdsourcingState and Local Government Review10.1177/0160323X1557518447:1(57-67)Online publication date: 17-Mar-2015
    • (2015)Incentive Mechanisms for Social ComputingProceedings of the 2015 IEEE International Conference on Self-Adaptive and Self-Organizing Systems Workshops10.1109/SASOW.2015.32(162-167)Online publication date: 21-Sep-2015
    • (2015)Supporting Multilevel Incentive Mechanisms in Crowdsourcing Systems: An Artifact-Centric ViewCrowdsourcing10.1007/978-3-662-47011-4_6(91-111)Online publication date: 29-May-2015
    • (2014)HCI as a means to prosociality in the economyProceedings of the SIGCHI Conference on Human Factors in Computing Systems10.1145/2556288.2557367(2955-2964)Online publication date: 26-Apr-2014
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media