Work in Progress

Disagreement, Agreement, and Elaboration in Crowdsourced Deliberation: Ideation Through Elaborated Perspectives

Authors:

Tanja Aitamurto,

Jorge SaldivarAuthors Info & Claims

CHI EA '23: Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems

Article No.: 88, Pages 1 - 10

https://doi.org/10.1145/3544549.3585708

Published: 19 April 2023 Publication History

CHI EA '23: Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems

Disagreement, Agreement, and Elaboration in Crowdsourced Deliberation: Ideation Through Elaborated Perspectives

Pages 1 - 10

Abstract
Supplementary Material
References

Abstract

In this study, we examined disagreement, agreement, and elaboration (rationale sharing) and their association with idea generation in a crowdsourced deliberation that took place within a crowdsourced policymaking process led by a national government. We analyzed the comments posted to the crowdsourced deliberation process and found that the elaboration of perspectives was a key element in idea generation. Disagreement contributed to ideation most when it was elaborated—i.e., when the participants justified their stances—and when it was accompanied by elaborated agreement. The findings suggest that in the design of the technologies and processes facilitating crowdsourced policymaking and other applications for civic engagement, there should be a particular focus on encouraging elaboration because elaboration can contribute to productive ideation as well as constructive argumentation. Elaboration could be fostered by deploying features from deliberation and argumentation technologies, which are designed to encourage participants to elaborate on their stances.

Supplementary Material

MP4 File (3544549.3585708-video-preview.mp4)

Video Preview

Download
2.92 MB

MP4 File (3544549.3585708-talk-video.mp4)

Pre-recorded Video Presentation

Download
39.64 MB

References

[1]

Tanja Aitamurto and Kaiping Chen. 2017. The value of crowdsourcing in public policymaking: Epistemic, democratic and economic value. The Theory and Practice of Legislation 5, 1 (Feb. 2017), 55–72. https://doi.org/10.1080/20508840.2017.1282665

[2]

Tanja Aitamurto and Hélène Landemore. 2015. Five design principles for crowdsourced policymaking: Assessing the case of crowdsourced off-road traffic law in Finland. Journal of Social Media for Organizations 2, 1 (Feb. 2015), 1–19. http://www2.mitre.org/public/jsmo/pdfs/02-01-5-design-principles.pdf

[3]

Tanja Aitamurto and Hélène Landemore. 2016. Crowdsourced deliberation: The case of the law on off-road traffic in Finland. Policy & Internet 8, 2 (June 2016), 174–196. https://doi.org/10.1002/poi3.115

[4]

Tanja Aitamurto and Jorge Saldivar. 2017. Examining the quality of crowdsourced deliberation: Respect, reciprocity and lack of common-good orientation. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems (Denver, CO, USA) (CHI EA ’17). ACM, New York, NY, USA, 2314–2321. https://doi.org/10.1145/3027063.3053248

Digital Library

[5]

Tanja Aitamurto and Jorge Saldivar. 2017. Motivating participation in crowdsourced policymaking: The interplay of epistemic and interactive aspects. Proceedings of the ACM on Human-Computer Interaction 1, CSCW, Article 18 (Dec. 2017), 22 pages. https://doi.org/10.1145/3134653

Digital Library

[6]

Sherry R. Arnstein. 1969. A ladder of citizen participation. Journal of the American Institute of Planners 35, 4 (1969), 216–224. https://doi.org/10.1080/01944366908977225

[7]

Laura W. Black and Anna Wiederhold. 2014. Discursive strategies of civil disagreement in public dialogue groups. Journal of Applied Communication Research 42, 3 (July 2014), 285–306. https://doi.org/10.1080/00909882.2014.911938

[8]

Kirsten Boehner and Carl DiSalvo. 2016. Data, design and civics: An exploratory study of civic tech. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, CA, USA) (CHI ’16). ACM, New York, NY, USA, 2970–2981. https://doi.org/10.1145/2858036.2858326

Digital Library

[9]

Peter Bruce, Andrew Bruce, and Peter Gedeck. 2020. Practical Statistics for Data Scientists: 50+ Essential Concepts Using R and Python (2nd ed.). O’Reilly Media, Sebastopol, CA, USA.

[10]

Moira Burke, Cameron Marlow, and Thomas Lento. 2009. Feed me: Motivating newcomer contribution in social network sites. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Boston, MA, USA) (CHI ’09). ACM, New York, NY, USA, 945–954. https://doi.org/10.1145/1518701.1518847

Digital Library

[11]

Yvonne Chan and Roy P. Walmsley. 1997. Learning and understanding the Kruskal-Wallis one-way analysis-of-variance-by-ranks test for differences among three or more independent groups. Physical Therapy 77, 12 (Dec. 1997), 1755–1761. https://doi.org/10.1093/ptj/77.12.1755

[12]

Kaiping Chen and Tanja Aitamurto. 2019. Barriers for crowd’s impact in crowdsourced policymaking: Civic data overload and filter hierarchy. International Public Management Journal 22, 1 (Jan. 2019), 99–126. https://doi.org/10.1080/10967494.2018.1488780

[13]

Quanze Chen, Jonathan Bragg, Lydia B. Chilton, and Dan S. Weld. 2019. Cicero: Multi-turn, contextual argumentation for accurate crowdsourcing. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland, UK) (CHI ’19). ACM, New York, NY, USA, Article 531, 14 pages.

Digital Library

[14]

Joshua Cohen. 1989. Deliberation and democratic legitimacy. In The Good Polity: Normative Analysis of the State, Alan P. Hamlin and Philip Pettit (Eds.). Wiley-Blackwell, New York, NY, USA, 17–34.

[15]

Eric Corbett and Christopher Le Dantec. 2021. Designing civic technology with trust. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). ACM, New York, NY, USA, Article 173, 17 pages. https://doi.org/10.1145/3411764.3445341

Digital Library

[16]

Kevin M. Esterling, Archon Fung, and Taeku Lee. 2015. How much disagreement is good for democratic deliberation?Political Communication 32, 4 (Oct. 2015), 529–551. https://doi.org/10.1080/10584609.2014.969466

[17]

James S. Fishkin. 2009. When the People Speak: Deliberative Democracy and Public Consultation. Oxford University Press, Oxford, UK.

[18]

Bruno S. Frey. 1994. Direct democracy: Politico-economic lessons from Swiss experience. The American Economic Review 84, 2 (1994), 338–342. https://www.jstor.org/stable/2117855

[19]

Joshua C. Gellers. 2016. Crowdsourcing global governance: Sustainable development goals, civil society, and the pursuit of democratic legitimacy. International Environmental Agreements: Politics, Law and Economics 16, 3 (June 2016), 415–432. https://doi.org/10.1007/s10784-016-9322-0

[20]

Homero Gil de Zúñiga, Sebastián Valenzuela, and Brian E. Weeks. 2016. Motivations for political discussion: Antecedents and consequences on civic engagement. Human Communication Research 42, 4 (Oct. 2016), 533–552. https://doi.org/10.1111/hcre.12086

[21]

Amy Gutmann and Dennis F. Thompson. 1996. Democracy and Disagreement. Belknap Press, Cambridge, MA, USA.

[22]

Jürgen Habermas. 1984. The Theory of Communicative Action. Vol. 1: Reason and the Rationalization of Society. Beacon Press, Boston, MA, USA.

[23]

Barbora Haltofova. 2018. Using crowdsourcing to support civic engagement in strategic urban development planning: A case study of Ostrava, Czech Republic. Journal of Competitiveness 10, 2 (June 2018), 85–103. https://doi.org/10.7441/joc.2018.02.06

[24]

Ronald R. Hocking. 2013. Methods and Applications of Linear Models: Regression and the Analysis of Variance (3rd ed.). Wiley, Hoboken, NJ, USA, Chapter 5: Collinearity in multiple linear regression, 142–181.

[25]

Simo Hosio, Jorge Goncalves, Vassilis Kostakos, and Jukka Riekki. 2015. Crowdsourcing public opinion using urban pervasive technologies: Lessons from real-life experiments in Oulu. Policy & Internet 7, 2 (June 2015), 203–222. https://doi.org/10.1002/poi3.90

[26]

Luca Iandoli, Ivana Quinto, Paolo Spada, Mark Klein, and Raffaele Calabretta. 2018. Supporting argumentation in online political debate: Evidence from an experiment of collective deliberation. New Media & Society 20, 4 (April 2018), 1320–1341. https://doi.org/10.1177/1461444817691509

[27]

Shanto Iyengar, Robert C. Luskin, and James S. Fishkin. 2003. Facilitating informed public opinion: Evidence from face-to-face and online deliberative polls. (Aug. 2003). Paper presented at the Annual Meeting of the American Political Science Association, Philadelphia, PA, USA.

[28]

Ian G. Johnson, Dalya Al-Shahrabi, and John Vines. 2020. From creating spaces for civic discourse to creating resources for action. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI) (CHI ’20). ACM, New York, NY, USA, 1–14. https://doi.org/10.1145/3313831.3376464

Digital Library

[29]

Ian G. Johnson, Alistair MacDonald, Jo Briggs, Jennifer Manuel, Karen Salt, Emma Flynn, and John Vines. 2017. Community Conversational: Supporting and capturing deliberative talk in local consultation processes. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, CO, USA) (CHI ’17). ACM, New York, NY, USA, 2320–2333. https://doi.org/10.1145/3025453.3025559

Digital Library

[30]

Hyunwoo Kim, Eun-Young Ko, Donghoon Han, Sung-Chul Lee, Simon T. Perrault, Jihee Kim, and Juho Kim. 2019. Crowdsourcing perspectives on public policy from stakeholders. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland, UK) (CHI EA ’19). ACM, New York, NY, USA, Article LBW1220, 6 pages. https://doi.org/10.1145/3290607.3312769

Digital Library

[31]

Jini Kim, Chorong Kim, and Ki-Young Nam. 2022. ThinkWrite: Design interventions for empowering user deliberation in online petition. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI EA ’22). ACM, New York, NY, USA, Article 428, 8 pages. https://doi.org/10.1145/3491101.3519644

Digital Library

[32]

Mark Klein. 2011. The MIT Deliberatorium: Enabling large scale deliberation about complex systemic problems. In Proceedings of the 2011 International Conference on Collaboration Technologies and Systems (CTS) (Philadelphia, PA, USA). IEEE, New York, NY, USA, 161. https://doi.org/10.1109/CTS.2011.5928678

[33]

Travis Kriplean, Jonathan Morgan, Deen Freelon, Alan Borning, and Lance Bennett. 2012. Supporting reflective public thought with ConsiderIt. In Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work (Seattle, WA, USA) (CSCW ’12). ACM, New York, NY, USA, 265–274. https://doi.org/10.1145/2145204.2145249

Digital Library

[34]

Travis Kriplean, Michael Toomim, Jonathan Morgan, Alan Borning, and Amy J. Ko. 2012. Is this what you meant? Promoting listening on the web with Reflect. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Austin, TX, USA) (CHI ’12). ACM, New York, NY, USA, 1559–1568. https://doi.org/10.1145/2207676.2208621

Digital Library

[35]

Hélène Landemore. 2017. Inclusive constitution making and religious rights: Lessons from the Icelandic experiment. The Journal of Politics 79, 3 (July 2017), 762–779. https://doi.org/10.1086/690300

[36]

Hélène Landemore. 2020. Open Democracy: Reinventing Popular Rule for the Twenty-First Century. Princeton University Press, Princeton, NJ, USA.

[37]

Hélène Landemore and Scott E. Page. 2015. Deliberation and disagreement: Problem solving, prediction, and positive dissensus. Politics, Philosophy & Economics 14, 3 (Aug. 2015), 229–254. https://doi.org/10.1177/1470594X14544284

[38]

Sung-chul Lee, Jihee Kim, and Juho Kim. 2017. Micro-NGO: Crowd-driven social activism via a chat-based online platform. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (Portland, OR, USA) (CSCW ’17). ACM, New York, NY, USA.

[39]

Narges Mahyar, Michael R. James, Michelle M. Ng, Reginald A. Wu, and Steven P. Dow. 2018. CommunityCrit: Inviting the public to improve and evaluate urban design ideas through micro-activities. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montréal, QC, Canada) (CHI ’18). ACM, New York, NY, USA, Article 195, 14 pages. https://doi.org/10.1145/3173574.3173769

Digital Library

[40]

Rousiley C. M. Maia, Gabriella Hauber, Thais Choucair, and Neylson J. B. Crepalde. 2021. What kind of disagreement favors reason-giving? Analyzing online political discussions across the broader public sphere. Political Studies 69, 1 (Feb. 2021), 108–128. https://doi.org/10.1177/0032321719894708

[41]

Henry B. Mann and Donald R. Whitney. 1947. On a test of whether one of two random variables is stochastically larger than the other. The Annals of Mathematical Statistics 18, 1 (March 1947), 50–60. https://doi.org/10.1214/aoms/1177730491

[42]

Matti Nelimarkka, Brandie Nonnecke, Sanjay Krishnan, Tanja Aitamurto, Daniel Catterson, Camille Crittenden, Chris Garland, Conrad Gregory, Ching-Chang (Allen) Huang, Gavin Newsom, Jay Patel, John Scott, and Ken Goldberg. 2014. Comparing three online civic engagement platforms using the "spectrum of public participation" framework. UC Berkeley: Center for Information Technology Research in the Interest of Society (CITRIS). Retrieved from https://escholarship.org/uc/item/0bz755bj.

[43]

Won No, Laurie Mook, and Daniel Schugurensky. 2017. Ideation in an online participatory platform: Towards a conceptual framework. Information Polity: The International Journal of Government & Democracy in the Information Age 22, 2 (Oct. 2017), 101–116. https://doi.org/10.3233/IP-170417

Digital Library

[44]

Beth Simone Noveck. 2015. Smart Citizens, Smarter State: The Technologies of Expertise and the Future of Governing. Harvard University Press, Cambridge, MA, USA.

[45]

Victoria Palacin, Síle Ginnane, Maria Angela Ferrario, Ari Happonen, Annika Wolff, Sara Piutunen, and Niina Kupiainen. 2019. SENSEI: Harnessing community wisdom for local environmental monitoring in Finland. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland, UK) (CHI EA ’19). ACM, New York, NY, USA, Article CS01, 8 pages. https://doi.org/10.1145/3290607.3299047

Digital Library

[46]

John Parkinson and Jane J. Mansbridge (Eds.). 2012. Deliberative systems: Deliberative democracy at the large scale. Cambridge University Press, Cambridge, UK.

[47]

Carole Pateman. 1970. Participation and Democratic Theory. Cambridge University Press, Cambridge, UK. https://doi.org/10.1017/CBO9780511720444

[48]

Paul B. Paulus, Lauren E. Coursey, and Jared B. Kenworthy. 2019. Divergent and convergent collaborative creativity. In The Palgrave Handbook of Social Creativity Research, Izabela Lebuda and Vlad Petre Glăveanu (Eds.). Springer International Publishing, Cham, Switzerland, 245–262. https://doi.org/10.1007/978-3-319-95498-1_16

[49]

Vincent Price, Joseph N. Cappella, and Lilach Nir. 2002. Does disagreement contribute to more deliberative opinion?Political Communication 19, 1 (Jan. 2002), 95–112. https://doi.org/10.1080/105846002317246506

[50]

John Prpić, Araz Taeihagh, and James Melton. 2015. The fundamentals of policy crowdsourcing. Policy & Internet 7, 3 (Sept. 2015), 340–361. https://doi.org/10.1002/poi3.102

[51]

Ivana Quinto, Luca Landoli, and Anna De Liddo. 2021. Designing online collaboration for the individual and social good: A collective argumentation approach. In Proceedings of the 27th Annual Americas Conference on Information Systems(AMCIS 2021). ACM, New York, NY, USA, USA, Article 1407, 11 pages.

[52]

Jorge Saldivar, Cristhian Parra, Marcelo Alcaraz, Rebeca Arteta, and Luca Cernuzzi. 2019. Civic technology for social innovation: A systematic literature review. Computer Supported Cooperative Work (CSCW) 28, 1–2 (April 2019), 169–207. https://doi.org/10.1007/s10606-018-9311-7

Digital Library

[53]

Václav Štětka and Jaromír Mazák. 2014. Whither slacktivism? Political engagement and social media use in the 2013 Czech Parliamentary elections. Cyberpsychology: Journal of Psychosocial Research on Cyberspace 8, 3, Article 7 (Dec. 2014), 21 pages. https://doi.org/10.5817/CP2014-3-7

[54]

Jennifer Stromer-Galley. 2007. Measuring deliberation’s content: A coding scheme. Journal of Public Deliberation 3, 1, Article 12 (2007), 35 pages. https://doi.org/10.16997/jdd.50

[55]

Jennifer Stromer-Galley, Lauren Bryant, and Bruce Bimber. 2015. Context and medium matter: Expressing disagreements online and face-to-face in political deliberations. Journal of Deliberative Democracy 11, 1, Article 1 (May 2015), 22 pages. https://doi.org/10.16997/jdd.218

[56]

Jennifer Stromer-Galley and Peter Muhlberger. 2009. Agreement and disagreement in group deliberation: Effects on deliberation satisfaction, future engagement, and decision legitimacy. Political Communication 26, 2 (May 2009), 173–192. https://doi.org/10.1080/10584600902850775

[57]

Nick Taylor, Loraine Clarke, Martin Skelly, and Sara Nevay. 2018. Strategies for engaging communities in creating physical civic technologies. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montréal, QC, Canada) (CHI ’18). ACM, New York, NY, USA, Article 507, 12 pages. https://doi.org/10.1145/3173574.3174081

Digital Library

[58]

Robert Vogel, Evelina Moulder, and Mike Huggins. 2014. The extent of public participation. Public Management 96, 2 (March 2014), 6–10.

[59]

Dawn Walker. 2018. "Data justice" by design: Building engagement through civic technologies. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (Montréal, QC, Canada) (CHI EA ’18). ACM, New York, NY, USA, Article DC20, 4 pages. https://doi.org/10.1145/3170427.3173024

Digital Library

[60]

Lu Xiao. 2011. The practice of rationale sharing in virtual group learning activities: An empirical study. In Proceedings of the 2011 IEEE 11th International Conference on Advanced Learning Technologies (Athens, GA, USA) (ICALT 2011). IEEE, New York, NY, USA, 521–523. https://doi.org/10.1109/ICALT.2011.160

Digital Library

[61]

Lu Xiao. 2013. The effects of a shared free form rationale space in collaborative learning activities. Journal of Systems and Software 86, 7 (July 2013), 1727–1737. https://doi.org/10.1016/j.jss.2012.07.042

Digital Library

[62]

Lu Xiao. 2014. Effects of rationale awareness in online ideation crowdsourcing tasks. Journal of the Association for Information Science and Technology 65, 8 (Aug. 2014), 1707–1720. https://doi.org/10.1002/asi.23079

Digital Library

Cited By

Frappier TBressa NHuron S(2024)Jumping to Conclusions: A Visual Comparative Analysis of Online Debate Platform LayoutsProceedings of the 13th Nordic Conference on Human-Computer Interaction10.1145/3679318.3685377(1-15)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3679318.3685377

Recommendations

Examining the Quality of Crowdsourced Deliberation: Respect, Reciprocity and Lack of Common-Good Orientation
CHI EA '17: Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems

We examine deliberative quality of crowdsourced deliberation in this paper. Analyzing data from two crowdsourced policy-making processes, we found a good quality deliberation with respect, reciprocity, and storytelling according to the standards in the ...
Beyond Transactional Democracy: A Study of Civic Tech in Canada
CSCW

Technologies are increasingly enrolled in projects to involve civilians in the work of policy-making, often under the label of 'civic technology'. But conventional forms of participation through transactions such as voting provide limited opportunities ...
The democratic potential of civic applications

Recently, digital democratic applications have increased in presence and scope. This study clarifies how civic applications – bottom-up technologies that use open data to solve governance and policy challenges – can contribute to democratic ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI EA '23: Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems

April 2023

3914 pages

ISBN:9781450394222

DOI:10.1145/3544549

Editors:
Albrecht Schmidt
LMU Munich, Germany
,
Kaisa Väänänen
Tampere University, Finland
,
Tesh Goyal
Google Research, USA
,
Per Ola Kristensson
University of Cambridge, UK
,
Anicia Peters
University of Namibia, Namibia

Copyright © 2023 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 April 2023

Check for updates

Author Tags

Qualifiers

Work in progress
Research
Refereed limited

Funding Sources

Strategic Research Council at the Academy of Finland

Conference

CHI '23

Sponsor:

SIGCHI

CHI '23: CHI Conference on Human Factors in Computing Systems

April 23 - 28, 2023

Hamburg, Germany

Acceptance Rates

Overall Acceptance Rate 6,164 of 23,696 submissions, 26%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
217
Total Downloads

Downloads (Last 12 months)116
Downloads (Last 6 weeks)36

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Frappier TBressa NHuron S(2024)Jumping to Conclusions: A Visual Comparative Analysis of Online Debate Platform LayoutsProceedings of the 13th Nordic Conference on Human-Computer Interaction10.1145/3679318.3685377(1-15)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3679318.3685377

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View full text|Download PDF

View Table of Conten