skip to main content
10.1145/3025453.3025464acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

The Effect of Peripheral Micro-tasks on Crowd Ideation

Published: 02 May 2017 Publication History

Abstract

Research has explored different ways of improving crowd ideation, such as presenting examples or employing facilitators. While such support is usually generated through peripheral tasks delegated to crowd workers who are not part of the ideation, it is possible that the ideators themselves could benefit from the extra thought involved in doing them. Therefore, we iterate over an ideation system in which ideators can perform one of three peripheral tasks (rating originality and usefulness, similarity, or idea combination) on demand. In controlled experiments with workers on Mechanical Turk, we compare the effects of these secondary tasks to simple idea exposure or no support at all, examining usage of the inspirations, fluency, breadth, and depth of ideas generated. We find tasks to be as good or better than exposure, although this depends on the period of ideation and the fluency level. We also discuss implications of inspiration size, homogeneity, and frequency.

References

[1]
Teresa M. Amabile. 1983. The social psychology of creativity: A componential conceptualization. Journal of personality and social psychology 45, 2: 357.
[2]
Michael S. Bernstein, Greg Little, Robert C. Miller, Björn Hartmann, Mark S. Ackerman, David R. Karger, David Crowell, and Katrina Panovich. 2010. Soylent: a word processor with a crowd inside. In Proceedings of the 23nd annual ACM symposium on User interface software and technology, 313--322. Retrieved January 11, 2016 from http://dl.acm.org/citation.cfm?id=1866078
[3]
Joel Chan, Steven Dang, and Steven P. Dow. 2016. Comparing Different Sensemaking Approaches for LargeScale Ideation. Retrieved March 11, 2016 from http://joelchan.me/files/2016-chi-sensemaking-ideation.pdf
[4]
Joel Chan, Steven Dang, and Steven P. Dow. 2016. Improving Crowd Innovation with Expert Facilitation.
[5]
Lydia B. Chilton, Greg Little, Darren Edge, Daniel S. Weld, and James A. Landay. 2013. Cascade: Crowdsourcing taxonomy creation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 1999-- 2008. Retrieved January 11, 2016 from http://dl.acm.org/citation.cfm?id=2466265
[6]
Arthur Cropley. 2006. In praise of convergent thinking. Creativity research journal 18, 3: 391--404.
[7]
Alan R. Dennis and Joseph S. Valacich. 1993. Computer brainstorms: More heads are better than one. Journal of applied psychology 78, 4: 531.
[8]
Alan R. Dennis and Mike L. Williams. 2003. Electronic Brainstorming: Theory, Research, and Future Directions. In Group creativity: Innovation through collaboration. Oxford University Press.
[9]
Michael Diehl and Wolfgang Stroebe. 1987. Productivity loss in brainstorming groups: Toward the solution of a riddle. Journal of Personality and Social Psychology 53, 3: 497--509.
[10]
Karen Leggett Dugosh, Paul B. Paulus, Evelyn J. Roland, and Huei-Chuan Yang. 2000. Cognitive stimulation in brainstorming. Journal of Personality and Social Psychology 79, 5: 722--735.
[11]
Beth A. Hennessey and Teresa M. Amabile. 2010. Creativity. Annual Review of Psychology 61, 1: 569--598.
[12]
Alex Ivanov and Dianne Cyr. 2006. The Concept Plot: a concept mapping visualization tool for asynchronous webbased brainstorming sessions. Information Visualization 5, 3: 185--191.
[13]
D. G. Jansson and S. M. Smith. 1991. Design Fixation. Design Studies 12, 1: 3--11.
[14]
Aniket Kittur, Ed H. Chi, and Bongwon Suh. 2008. Crowdsourcing user studies with Mechanical Turk. In Proceedings of the SIGCHI conference on human factors in computing systems, 453--456. Retrieved August 17, 2015 from http://dl.acm.org/citation.cfm?id=1357127
[15]
Aniket Kittur, Boris Smus, Susheel Khamkar, and Robert E. Kraut. 2011. Crowdforge: Crowdsourcing complex work. In Proceedings of the 24th annual ACM symposium on User interface software and technology, 43-- 52. Retrieved August 4, 2015 from http://dl.acm.org/citation.cfm?id=2047202
[16]
Nicholas W. Kohn, Paul B. Paulus, and YunHee Choi. 2011. Building on the ideas of others: An examination of the idea combination process. Journal of Experimental Social Psychology 47, 3: 554--561.
[17]
Aaron Kozbelt, Ronald A. Beghetto, and Mark A. Runco. 2010. Theories of creativity. The Cambridge handbook of creativity: 20--47.
[18]
Filip Krynicki. 2014. Methods and models for the quantitative analysis of crowd brainstorming. Retrieved April 5, 2016 from https://uwspace.uwaterloo.ca/handle/10012/8347
[19]
Richard L. Marsh, Joshua D. Landau, and Jason L. Hicks. 1996. How examples may (and may not) constrain creativity. Memory & cognition 24, 5: 669--680.
[20]
Brent A. Nelson, Jamal O. Wilson, David Rosen, and Jeannette Yen. 2009. Refined metrics for measuring ideation effectiveness. Design Studies 30, 6: 737--743.
[21]
Charlan J. Nemeth, Bernard Personnaz, Marie Personnaz, and Jack A. Goncalo. 2004. The liberating role of conflict in group creativity: A study in two countries. European Journal of Social Psychology 34, 4: 365--374.
[22]
Bernard A. Nijstad, Michael Diehl, and Wolfgang Stroebe. 2003. Cognitive Stimulation and Interference in Idea-Generating Groups. In Group Creativity: Innovation Through Collaboration. Oxford University Press.
[23]
Bernard A. Nijstad and Wolfgang Stroebe. 2006. How the group affects the mind: A cognitive model of idea generation in groups. Personality and social psychology review 10, 3: 186--213.
[24]
Bernard A. Nijstad, Wolfgang Stroebe, and Hein FM Lodewijkx. 2002. Cognitive stimulation and interference in groups: Exposure effects in an idea generation task. Journal of experimental social psychology 38, 6: 535--544.
[25]
Alex F. Osborn. 1963. Applied imagination; principles and procedures of creative problem-solving. Scribner, New York.
[26]
Jonathan A. Plucker and Matthew C. Makel. 2010. Assessment of Creativity. In The Cambridge Handbook of Creativity, James C. Kaufman and Robert J. Sternberg (eds.). Cambridge University Press, Cambridge, 48--73. Retrieved November 29, 2016 from http://ebooks.cambridge.org/ref/id/CBO9780511763205A0
[27]
Pao Siangliulue, Kenneth C. Arnold, Krzysztof Z. Gajos, and Steven P. Dow. 2015. Toward Collaborative Ideation at Scale: Leveraging Ideas from Others to Generate More Creative and Diverse Ideas. 937--945.
[28]
Pao Siangliulue, Joel Chan, Steven P. Dow, and Krzysztof Z. Gajos. 2016. IdeaHound: Improving Largescale Collaborative Ideation with Crowd-Powered Real-time Semantic Modeling. 609--624.
[29]
Pao Siangliulue, Joel Chan, Krzysztof Z. Gajos, and Steven P. Dow. 2015. Providing Timely Examples Improves the Quantity and Quality of Generated Ideas. 83--92.
[30]
Steven M. Smith. 2003. The constraining effects of initial ideas. In Group creativity: Innovation through collaboration, Paul B. Paulus and Bernard A. Nijstad (eds.). Oxford University Press, New York, NY, US, 15--31.
[31]
Lixiu Yu, Aniket Kittur, and Robert E. Kraut. 2014. Distributed analogical idea generation: inventing with crowds. 1245--1254.
[32]
Lixiu Yu, Aniket Kittur, and Robert E. Kraut. 2014. Searching for analogical ideas with crowds. 1225--1234.
[33]
Lixiu Yu and Jeffrey V. Nickerson. 2011. Cooks or cobblers?: crowd creativity through combination. In Proceedings of the SIGCHI conference on human factors in computing systems, 1393--1402. Retrieved October 12, 2015 from http://dl.acm.org/citation.cfm?id=1979147

Cited By

View all
  • (2022)Scaling Creative Inspiration with Fine-Grained Functional Aspects of IdeasProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3517434(1-15)Online publication date: 29-Apr-2022
  • (2022)Microtasking Activities in Crowdsourced Software Development: A Systematic Literature ReviewIEEE Access10.1109/ACCESS.2022.314840010(24721-24737)Online publication date: 2022
  • (2022)Online Communication for Team Creativity in Tech Companies: Barriers and Tool DesignCross-Cultural Design. Applications in Business, Communication, Health, Well-being, and Inclusiveness10.1007/978-3-031-06050-2_2(13-28)Online publication date: 26-Jun-2022
  • Show More Cited By

Index Terms

  1. The Effect of Peripheral Micro-tasks on Crowd Ideation

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems
    May 2017
    7138 pages
    ISBN:9781450346559
    DOI:10.1145/3025453
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 02 May 2017

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. creativity
    2. crowdsourcing
    3. ideation
    4. microtasks

    Qualifiers

    • Research-article

    Funding Sources

    Conference

    CHI '17
    Sponsor:

    Acceptance Rates

    CHI '17 Paper Acceptance Rate 600 of 2,400 submissions, 25%;
    Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

    Upcoming Conference

    CHI 2025
    ACM CHI Conference on Human Factors in Computing Systems
    April 26 - May 1, 2025
    Yokohama , Japan

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)41
    • Downloads (Last 6 weeks)2
    Reflects downloads up to 13 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2022)Scaling Creative Inspiration with Fine-Grained Functional Aspects of IdeasProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3517434(1-15)Online publication date: 29-Apr-2022
    • (2022)Microtasking Activities in Crowdsourced Software Development: A Systematic Literature ReviewIEEE Access10.1109/ACCESS.2022.314840010(24721-24737)Online publication date: 2022
    • (2022)Online Communication for Team Creativity in Tech Companies: Barriers and Tool DesignCross-Cultural Design. Applications in Business, Communication, Health, Well-being, and Inclusiveness10.1007/978-3-031-06050-2_2(13-28)Online publication date: 26-Jun-2022
    • (2021)Hugging with a Shower Curtain: Older Adults' Social Support Realities During the COVID-19 PandemicProceedings of the ACM on Human-Computer Interaction10.1145/34796075:CSCW2(1-31)Online publication date: 18-Oct-2021
    • (2021)IdeateRelate: An Examples Gallery That Helps Creators Explore Ideas in Relation to Their OwnProceedings of the ACM on Human-Computer Interaction10.1145/34794965:CSCW2(1-18)Online publication date: 18-Oct-2021
    • (2021)Directed Diversity: Leveraging Language Embedding Distances for Collective Creativity in Crowd IdeationProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445782(1-35)Online publication date: 6-May-2021
    • (2021)FashionQ: An AI-Driven Creativity Support Tool for Facilitating Ideation in Fashion DesignProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445093(1-18)Online publication date: 6-May-2021
    • (2020)Evaluating Creativity Support Tools in HCI ResearchProceedings of the 2020 ACM Designing Interactive Systems Conference10.1145/3357236.3395474(457-476)Online publication date: 3-Jul-2020
    • (2020)Spinneret: Aiding Creative Ideation through Non-Obvious Concept AssociationsProceedings of the 2020 CHI Conference on Human Factors in Computing Systems10.1145/3313831.3376746(1-13)Online publication date: 21-Apr-2020
    • (2019)Discovering the Sweet Spot of Human-Computer ConfigurationsProceedings of the ACM on Human-Computer Interaction10.1145/33592973:CSCW(1-30)Online publication date: 7-Nov-2019
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media