Skip to main content
SpringerLink
Log in

Enabling Large-Scale Deliberation Using Attention-Mediation Metrics

  • Published:
Computer Supported Cooperative Work (CSCW) Aims and scope Submit manuscript

Abstract

Humanity now finds itself faced with a range of highly complex and controversial challenges—such as climate change, the spread of disease, international security, scientific collaborations, product development, and so on—that call upon us to bring together large numbers of experts and stakeholders to deliberate collectively on a global scale. Collocated meetings can however be impractically expensive, severely limit the concurrency and thus breadth of interaction, and are prone to serious dysfunctions such as polarization and hidden profiles. Social media such as email, blogs, wikis, chat rooms, and web forums provide unprecedented opportunities for interacting on a massive scale, but have yet to realize their potential for helping people deliberate effectively, typically generating poorly-organized, unsystematic and highly redundant contributions of widely varying quality. Large-scale argumentation systems represent a promising approach for addressing these challenges, by virtue of providing a simple systematic structure that radically reduces redundancy and encourages clarity. They do, however, raise an important challenge. How can we ensure that the attention of the deliberation participants is drawn, especially in large complex argument maps, to where it can best serve the goals of the deliberation? How can users, for example, find the issues they can best contribute to, assess whether some intervention is needed, or identify the results that are mature and ready to “harvest”? Can we enable, for large-scale distributed discussions, the ready understanding that participants typically have about the progress and needs of small-scale, collocated discussions?. This paper will address these important questions, discussing (1) the strengths and limitations of current deliberation technologies, (2) how argumentation technology can help address these limitations, and (3) how we can use attention-mediation metrics to enhance the effectiveness of large-scale argumentation-based deliberations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9

Similar content being viewed by others

Notes

  1. Based on personal communication with Catherine Spence, Information Technology Enterprise Architect, Computing Director/Manager at Intel.

  2. The one exception we are aware of (the Open Meeting Project’s mediation of the 1994 National Policy Review (Hurwitz 1996)) was effectively a comment collection system rather than a deliberation system, since the participants predominantly offered reactions to a large set of pre-existing policy documents, rather than interacting with each other to create new policy options.

References

  • Adomavicius, G., & Tuzhilin, A. (2005). Toward the Next Generation of Recommender Systems: A Survey of the State-of-the-Art and Possible Extensions. IEEE Transactions on Knowledge and Data Engineering, 17(6), 734–749.

    Article  Google Scholar 

  • Benbasat, I., & Lim, J. (2000). Information technology support for debiasing group judgments: an empirical evaluation. Organizational Behavior and Human Decision Processes, 83(1), 167–183.

    Article  Google Scholar 

  • Benkler, Y. (2006). The Wealth of Networks: How Social Production Transforms Markets and Freedom. Yale University Press.

  • Boland, R. J., Maheshwari, A. K., Te’eni, D., Schwartz, D., & Tenkasi, R. V. (1992). Sharing Perspectives in Distributed Decision Making. Computer-Supported Cooperative Work.

  • Bolstad, W. M. (2010). Understanding Computational Bayesian Statistics. John Wiley.

  • Bonaccorsi, A., & Rossi, C. (2004). Altruistic individuals, selfish firms? The structure of motivation in Open Source software.

  • Cappella, J. N., Price, V., & Nir, L. (2002). Argument Repertoire as a Reliable and Valid Measure of Opinion Quality: Electronic Dialogue During Campaign 2000. Political Communication, 19(1), 73–93.

    Article  Google Scholar 

  • Carr, C. S. (2003). Using computer supported argument visualization to teach legal argumentation. In P. A. Kirschner, S. J. B. Shum, & C. S. Carr (Eds.), Visualizing argumentation: software tools for collaborative and educational sense-making (pp. 75–96). Springer-Verlag.

  • Chklovski, T., Ratnakar, V., & Gil, Y. (2005). User interfaces with semi-formal representations: a study of designing argumentation structures. Proceedings of the 10th international conference on Intelligent user interfaces, 130–136.

  • Clark, H. H., & Brennan, S. E. (1991). Grounding in communication. In L. B. Resnick, J. M. Levine, & S. D. Teasley (Eds.), Perspectives on socially shared cognition (pp. 127–149). Washington, DC, US: American Psychological Association.

    Chapter  Google Scholar 

  • Conklin, J. (2005). Dialogue Mapping: Building Shared Understanding of Wicked Problems. John Wiley and Sons, Ltd.

  • Convertino, G., Billman, D., Shrager, J., Pirolli, P., & Massar, J. P. (2008). The CACHE Study: Group Effects in Computer-Supported Collaborative Analysis. Journal of Computer Supported Cooperative Work, 17(4), 353–393.

    Article  Google Scholar 

  • Cook, M. B., & Smallman, H. S. (2007). Visual Evidence Landscapes: Reducing Bias in Collaborative Intelligence Analysis. Human Factors and Ergonomics Society Annual Meeting Proceedings.

  • Dennis, A. R., & Valacich, J. S. (1993). Computer brainstorms: More heads are better than one. Journal of Applied Psychology, 78(4), 531–537.

    Article  Google Scholar 

  • Eemeren, F. H. v., & Grootendorst, R. (2003). A Systematic Theory of Argumentation: The Pragma-dialectical Approach. Cambridge University Press.

  • Erickson, T., Halverson, C., Kellogg, W. A., Laff, M., & Wolf, T. (2002). Social Translucence: Designing Social Infrastructures that Make Collective Activity Visible. Communications of the ACM, 45(4), 40–44.

    Article  Google Scholar 

  • Farnham, S., Chesley, H. R., McGhee, D. E., Kawal, R., & Landau, J. (2000). Structured online interactions: improving the decision-making of small discussion groups. Computer Supported Cooperative Work.

  • Finholt, T. A. (2002). Collaboratories. Annual Review of Information Science and Technology, 36(1), 73–107.

    Article  Google Scholar 

  • Gladwell, M. (2002). The Tipping Point: How Little Things Can Make a Big Difference. Back Bay Books.

  • Gopal, A., & Prasad, P. (2000). Understanding GDSS in Symbolic Context: Shifting the Focus from Technology to Interaction. MIS Quarterly, 24(3), 509–546.

    Article  Google Scholar 

  • Group, R. D. F. D. A. W. (2008). SPARQL Query Language for RDF. http://www.w3.org/TR/rdf-sparql-query/.

  • Hars, A., & Ou, S. (2002). Working for Free? Motivations for Participating in Open-Source Projects. International Journal of Electronic Commerce, 6(3), 25–39.

    Google Scholar 

  • Heng, M. S. H., & de Moor, A. (2003). From Habermas' s communicative theory to practice on the internet. Information Systems Journal, 13(4), 331–352.

    Article  Google Scholar 

  • Householder, A. S. (1958). Unitary Triangularization of a Nonsymmetric Matrix. Journal of the ACM, 5(4), 339–342.

    Article  MathSciNet  MATH  Google Scholar 

  • Iandoli, L., Klein, M., & Zollo, G. (2009). Enabling on-line deliberation and collective decision-making through large-scale argumentation: a new approach to the design of an Internet-based mass collaboration platform. International Journal of Decision Support System Technology, 1(1), 69–91.

    Article  Google Scholar 

  • Johnson, E. M., & Halpin, S. M. (1974). Multistage inference models for intelligence analysis.

  • Jonassen, D., & Jr, H. R. (2005). Mapping alternative discourse structures onto computer conferences. International Journal of Knowledge and Learning, 1(1/2), 113–129.

    Article  Google Scholar 

  • Karacapilidis, N., Loukis, E., & Dimopoulos, S. (2004). A Web-Based System for Supporting Structured Collaboration in the Public Sector. LECTURE NOTES IN COMPUTER SCIENCE, 218–225.

  • Kirschner, P. A., Shum, S. J. B., & Carr, C. S. (2003). Visualizing Argumentation: Software tools for collaborative and educational sense-making. Springer.

  • Kittur, A., Suh, B., Pendleton, B. A., & Chi, E. H. (2007). He says, she says: conflict and coordination in Wikipedia. SIGCHI Conference on Human Factors in Computing Systems.

  • Klein, M. (2003). A Knowledge-Based Methodology for Designing Reliable Multi-Agent Systems. In P. Giorgini, J. P. Mueller, & J. Odell (Eds.), Agent-Oriented Software Engineering IV (Vol. 2935, pp. 85–95). Springer-Verlag.

  • Klein, M. (2007). The MIT Collaboratorium: Enabling Effective Large-Scale Deliberation for Complex Problems.

  • Klein, M., & Bernstein, A. (2004). Towards High-Precision Service Retrieval. IEEE Internet Computing Journal, 8(1), 30–36.

    Article  Google Scholar 

  • Klein, M., & Iandoli, L. (2008). Supporting Collaborative Deliberation Using a Large-Scale Argumentation System: The MIT Collaboratorium. Directions and Implications of Advanced Computing; Conference on Online Deliberation (DIAC-2008/OD2008).

  • Klein, M., Sayama, H., Faratin, P., & Bar-Yam, Y. (2003). The Dynamics of Collaborative Design: Insights From Complex Systems and Negotiation Research. Concurrent Engineering Research & Applications, 11(3), 201–210.

    Article  Google Scholar 

  • Kramer, K., & King, J. (1988). Computer-Based Systems for Coperative Work and Group Decision Making. Computing Surveys, 20(June), 115–146.

    Article  Google Scholar 

  • Lakhani, K. R., & Wolf, R. G. (2005). Why Hackers Do What They Do: Understanding Motivation and Effort in Free/Open Source Software Projects. In J. Feller, B. Fitzgerald, S. Hissam, & K. R. Lakhani (Eds.), Perspectives on Free and Open Source Software. MIT Press.

  • Lowrance, J. D., Harrison, I. W., & Rodriguez, A. C. (2001). Capturing Analytic Thought. First International Conference on Knowledge Capture, 84–91.

  • Luppicini, R. (2007). Review of computer mediated communication research for education. Instructional Science, 35(2), 141–185.

    Article  Google Scholar 

  • Macaulay, L. A., & Alabdulkarim, A. (2005). Facilitation of e-Meetings: State-of-the-Art Review. e-Technology, e-Commerce and e-Service (EEE’05).

  • Moor, A. d., & Aakhus, M. (2006). Argumentation Support: From Technologies to Tools. Communications of the ACM, 49(3), 93.

    Article  Google Scholar 

  • Nisbet, D. (2004). Measuring the Quantity and Quality of Online Discussion Group Interaction. Journal of eLiteracy, 1, 122–139.

    Google Scholar 

  • Pervan, G. P., & Atkinson, D. J. (1995). GDSS research: An overview and historical analysis. Group Decision and Negotiation, 4(6), 475–483.

    Article  Google Scholar 

  • Poole, M. S., Holmes, M., & DeSanctis, G. (1988). Conflict Management and Group Decision Support Systems. Proceedings from Proceedings of Computer Supported Cooperative Work.

  • Powell, A., Piccoli, G., & Ives, B. (2004). Virtual teams: a review of current literature and directions for future research. ACM SIGMIS Database, 35(1), 6–36.

    Article  Google Scholar 

  • Rahwan, I. (2008). Mass argumentation and the semantic web. Journal of Web Semantics, 6(1), 29–37.

    Article  Google Scholar 

  • Reagan-Cirincione, P. (1994). Improving the accuracy of group judgment: a process intervention combining group facilitation, social judgment analysis, and information technology. Organizational Behavior and Human Decision Processes, 58(2), 246–270.

    Article  Google Scholar 

  • Roberts, J. E. F. F. R. E. Y., Hann, I. L.-H. O. R. N., & Slaughter, S. A. N. D. R. A. (2006). Understanding the Motivations, Participation and Performance of Open Source Software Developers: A Longitudinal Study of the Apache Projects. Management Science, 52(7), 984–999.

    Article  Google Scholar 

  • Salganik, M. J., Dodds, P. S., & Watts, D. J. (2006). Experimental Study of Inequality and Unpredictability in an Artificial Cultural Market. Science, 311(5762), 854–856.

    Article  Google Scholar 

  • Schulz-Hardt, S., Frey, D., Lüthgens, C., & Moscovici, S. (2000). Biased information search in group decision making. Journal of Personality and Social Psychology, 78(4), 655–669.

    Article  Google Scholar 

  • Shrager, J., Billman, D., Convertino, G., Massar, J. P., & Pirolli, P. (2010). Soccer Science and the Bayes Community: Exploring the Cognitive Implications of Modern Scientific Communication. Topics in Cognitive Science, 2(1), 53–72.

    Article  Google Scholar 

  • Shum, S. B., Liddo, A. D., Iandoli, L., & Quinto, I. (2012). A Debate Dashboard to Support the Adoption of Online Knowledge Mapping Tools. VINE Journal of information and Knowledge Management Systems.

  • Shum, S. J. B., Selvin, A. M., Sierhuis, M., Conklin, J., & Haley, C. B. (2006). Hypermedia Support for Argumentation-Based Rationale: 15 Years on from gIBIS and QOC. In A. H. Dutoit, R. McCall, I. Mistrik, & B. Paech (Eds.), Rationale Management in Software Engineering. Springer-Verlag.

  • Smallman, H. S. (2008). JIGSAW – Joint Intelligence Graphical Situation Awareness Web for collaborative intelligence analysis. In M. P. Letsky, N. Warner, S. Fiore, & C. A. P. Smith (Eds.), Macrocognition in Teams: Theories and Methodologies (pp. 321–337). Aldershot, UK.: Ashgate.

  • Spatariu, A., Hartley, K., & Bendixen, L. D. (2004). Defining and Measuring Quality in Online Discussions. The Journal of Interactive Online Learning, 2(4).

  • Steenbergen, M. R., Bachtiger, A., Sporndli, M., & Steiner, J. (2003). Measuring political deliberation: a discourse quality index. Comparative European Politics, 1(1), 21–48.

    Article  Google Scholar 

  • Stromer-Galley, J. (2007). Measuring deliberation’s content: A coding scheme. Journal of Public Deliberation, 3(1), 12.

    Google Scholar 

  • Sunstein, C. R. (2006). Infotopia: How Many Minds Produce Knowledge. Oxford University Press.

  • Surowiecki, J. (2005). The Wisdom of Crowds. Anchor.

  • Tapscott, D., & Williams, A. D. (2006). Wikinomics: How Mass Collaboration Changes Everything. Portfolio Hardcover.

  • Trénel, M. (2004). Measuring the quality of online deliberation. Coding scheme 2.4. Social Science Research Center Berlin, Germany. Available at: www. wz-berlin. de/online-mediation/files/publications/quod_2_4. pdf.

  • Tversky, A., & Kahneman, D. (1974). Judgments under uncertainty. Heuristics and biases. Science, 185(4157), 1124–1131.

    Article  Google Scholar 

  • Viegas, F. B., Wattenberg, M., & Dave, K. (2004). Studying cooperation and conflict between authors with history flow visualizations. SIGCHI conference on Human factors in computing systems.

  • Walton, D. N. (2005). Fundamentals of Critical Argumentation (Critical Reasoning and Argumentation). Cambridge (MA): Cambridge University Press.

    Book  Google Scholar 

  • Walton, D. N., & Krabbe, E. C. W. (1995). Commitment in dialogue: Basic concepts of interpersonal reasoning. Albany, NY: State University of New York Press.

    Google Scholar 

Download references

Acknowledgements

The author would like to gratefully acknowledge the many useful conversations he has had on the topic of deliberation metrics with Prof Ali Gurkan (Ecole Centrale Paris), Prof. Luca Iandoli (University of Naples), and Prof. Haji Reijers (Eindhoven University of Technology).

The work has been supported by the National Science Foundation.

Author information

Authors and Affiliations

  1. MIT Center for Collective Intelligence, Cambridge, MA, USA

    Mark Klein

Authors
  1. Mark Klein
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mark Klein.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Klein, M. Enabling Large-Scale Deliberation Using Attention-Mediation Metrics. Comput Supported Coop Work 21, 449–473 (2012). https://doi.org/10.1007/s10606-012-9156-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10606-012-9156-4

Key words

Search

Navigation