Elsevier

Computers in Human Behavior

Volume 99, October 2019, Pages 205-218
Computers in Human Behavior

Full length article
Advice reification, learning, and emergent collective intelligence in online health support communities

https://doi.org/10.1016/j.chb.2019.05.028Get rights and content

Highlights

  • Online health communities refine the advice they provide to newcomers over time.

  • This is similar to the reification of practice in a community of practice.

  • We introduce an algorithm to extract advice refinement arrayed across posts.

  • We identify three different types of advice refinement.

  • These different types reflect differences in the status of each type of advice for the community.

Abstract

Online health support forums utilize straightforward online discussion designs to create a sociotechnical space where people can seek social support from others. The advice generated in these forums exists as an archival resource for future health information seekers. The present study uses mixed methods to investigate how invisible social processes lead advice to be adapted over time by forum members. Drawing on the construct of ‘reification’ from the communities of practice (COP) literature, we operationalize the reification of advice (RoA) as a process by which advice is developed across multiple discussion threads, and construct an algorithmic procedure to extract posts that trace this process. We evaluate our algorithm with crowd-workers, and perform an inductive, qualitative analysis to identify different modes of advice reification. We suggest that RoA could be used as the basis of a mid-level theory that treats online support communities and bundles of advice trajectories embedded in a shifting sociotechnical context. In our closing analysis, we propose that our approach might be a first step in an algorithmic procedure for assessing advice quality, drawing on the idea that reified advice may be considered a product of the collective intelligence of an online health support community.

Introduction

People who participate in online social platforms leave digital traces of their activities, which can in turn become a resource for others (e.g. articles on Wikipedia, code on GitHub, answers on StackOverflow). Social computing platforms employ sociotechnical infrastructures designed to scaffold longitudinal process that adapt and, hopefully, improve the quality of these resources (Bryant et al., 2005, Crowston et al., 2006, Parnin et al., 2012, Solomon and Wash, 2014). Highlighting their ability to harness the collective efforts of online contributors, such social computing platforms have been described as a kind of designed collective intelligence (Kittur and Kraut, 2008, Malone et al., 2009, Quinn and Bederson, 2011).

In other cases, resources generated by online platforms are a side effect of online engagement. Such is often the case with online health support communities (OHC). OHCs are places where people can go to find social support for health conditions. In many cases OHCs are organized as Question & Answer style discussions, and built upon well-established threaded discussion forum technologies (Davison et al., 2000, Maloney-Krichmar and Preece, 2005). The first post in each threaded conversation is usually a question (a request for some kind of support) and the remainder of the conversation is devoted in part to responding to the initial question while also providing long-time forum members with an opportunity for richer interactions (Introne, Semaan, & Goggins, 2016). These conversations often yield advice for the original asker, and this advice may also be considered a resource for future seekers.

Unlike social computing platforms, the processes that generate advice in an OHC are not governed by any designed sociotechnical infrastructures. However, research has shown that in some OHCs, the same small, tightly knit group of people—the community's core—responds to thousands of requests from a much larger population of support seekers (Bambina, 2007, Introne et al., 2016). Given this, it seems likely that advice stabilizes as these tightly knit groups of people revisit the same kinds of repeated support requests. Unlike the designed collective intelligence of other social computing platforms, the stabilization and refinement of advice suggests a form of emergent collective intelligence in OHCs.

The primary aim of this paper is to develop evidence for the existence of such emergent collective intelligence. To do this, we draw heavily on Wenger's “communities of practice” (COP) construct (Lave and Wenger, 1991, Wenger, 1998), and the concomitant process of reification that occurs therein. A COP is a group of people who share a craft or profession, and its participants reify their practice-oriented experiences over time. Wenger (1998) defines the term reification thusly:

“With the term reification I mean to cover a wide range of processes that include making, designing, representing, naming, encoding and describing, as well as perceiving, interpreting, using, reusing, decoding and recasting. Reification occupies much of our collective energy: from entries in a journal to historical records, from poems to encyclopaedias, from names to classification systems, from dolmens to space probes … In all these cases, aspects of human experience and practice are congealed into fixed forms and given the status of object.”

Although reification may produce a concrete object, it is not primarily a process of objectification. Rather, it is a process through which aspects of participation in a community of practice are made more concrete, enabling these aspects to become foci for further negotiation (Wenger, 1998, p. 58). We apply this notion of reification to describe the processes by which members in an OHC produce and adapt health advice. Health advice in an OHC is a complex object that emerges from individuals’ shared practice of living with a health condition, and includes personal experiences with that condition and the healthcare system, sensibilities about how to guide others, and collective sensemaking around these things both on and offline. We argue that the advice produced online reifies this amalgam of experiences via the active participation of individuals grappling with a similar health condition in the virtual space of an OHC.

Adopting this lens, we use this paper first to identify and then classify instances of advice reification across multiple forums hosted by the population an online health information service, WebMD. Our data is drawn from five years across fifty-five different, health-condition specific forums. To perform our analysis, we introduce a novel algorithmic approach to extracting sequences of posts—which we call meta-conversations (MCs)—that are arrayed across discussion threads and are likely vehicles for advice reification. We use a content analytic procedure to validate that these sequences of posts are indeed likely to trace advice reification processes. Then, we use qualitative methods to examine the structure and content of these MCs in order to isolate specific genres (Andersen and van Leeuwen, 2017, Caple and Knox, 2017) of advice reification. Finally, in our closing discussion, we consider both the theoretical implications of our findings as well as the practical application of our techniques to the problem of assessing advice quality online.

Online health support platforms are online spaces where people go to find some form of social support from others (Davison et al., 2000, Maloney-Krichmar and Preece, 2005). Access to social support can have a range of clinical benefits for people with health conditions, and in particular those with chronic conditions (Fox & Purcell, 2010). It is thought that because online support reduces barriers to access and can insulate people from stigma, that it may be a valuable tool for many different patient populations (Davison et al., 2000, Johnson and Ambrose, 2006). An important component of online social support involves advice about how to manage a health condition.

A pressing concern for medical professionals and researchers is whether or not advice found online is of high quality. The existence of misleading or inaccurate information about health issues on the web (though not necessarily within online social support platforms) has been well documented. For example, a systematic review of online health communities found that 55 of 79 distinct studies found information quality to be a problem (Eysenbach, Powell, Kuss, & Sa, 2002). However, online advice is not uniformly bad, and evidence suggests that different communities may develop practices that influence its quality (Hartzler & Huh, 2016). Indeed, several studies have found that in some forums, members actively monitor for inaccuracies and correct ‘bad’ advice (Deshpande and Jadad, 2006, Esquivel et al., 2006).

An important, but as yet underdeveloped line of research is how of social structures and practices that develop in OHCs bear upon the production of advice. Members in OHCs can come to play social roles that interact with advice production in more or less direct ways. For example, Maloney-Krichmar and Preece (2005) identified 17 social roles in a support group for people with knee problems, and a subset of these were explicitly focused on advice and its quality (e.g. “evaluator-critics” and “information givers”). More generally, OHCs often stratify into core and more peripheral users (Bambina, 2007, Introne et al., 2016) who engage in different kinds of behaviors. Core members typically develop strong and convivial relationships with one another. Although these core groups are quite small—Introne et al. (2016) reports core sizes of tens of members or less—they can form persistent groups in OHCs that provide the bulk of advice to thousands of more peripheral members and newcomers (Introne et al., 2016).

A similar stratification from central to more peripheral participants can be found in communities of practice (COP) (Brown and Duguid, 2001, Lave and Wenger, 1991, Wenger, 1998). A COP may be considered to be a community that is characterized by its participants' mutual interest and participation in a common practice (Silva et al., 2009, Wenger, 1998, p. 72). As described by (Lave & Wenger, 1991), newcomers in a COP accumulate experience by working alongside more experienced members (a process referred to as legitimate peripheral participation) possibly becoming masters themselves who will guide subsequent newcomers. This process transforms the community's practice–based repertoire of knowledge, routines, and artifacts as newcomers potentially bring diverse perspectives and older members integrate, curate, and disseminate their accumulated repertoire.

Several researchers have called out both the differences and similarities between offline COPs and OHCs. Jones and Preece (2006) introduced the term “Community of Interest” to highlight the fact that the locus of an OHC is a common health concern, rather than a specific practice. Similarly, Johnston et al. (Johnston, Worrell, Di Gangi, & Wasko, 2013) distinguish OCHs from COPs because they lack a clear professional jargon. On the other hand, Kimmerle et al. (2013) focused on aspects of identity and knowledge management to argue that one online alternative health community could be seen as an excellent fit for a COP.

Unlike Jones and Preece (2006), we take the position that managing a health condition is indeed a central practice shared by members of at least some OHCs, and this is a critical part of what binds the people who find one another in these communities. Managing diabetes, or breast cancer, or fibromyalgia, or multiple sclerosis whilst trying to live a fulfilling life is not a dispassionate interest for these individuals. What and when to eat, how to navigate networks of medical specialists, and grappling with the emotional tumult of a debilitating condition all require the development of a complex web of strategies and behaviors, and sharing these with others who face the same challenges can be an important vehicle for learning and improvement.

We consider advice to be a concrete instantiation of the practice of managing a health condition. We anticipate that the core members of an OHC adapt advice over time for three reasons. First, an OHC is a public space, so the production of advice is visible and thereby becomes a potential target for deliberative engagement. Second, thousands of incoming requests from more peripheral individuals create opportunities for the core to produce the same kinds of advice time after time. Finally, core members are continuously learning about how to manage their own health conditions, and questions or offerings from newcomers may bring new information for consideration.

The concretization and re-concretization of the practice of condition management through the vehicle of advice-giving fits well with Wenger's construct of reification. We briefly discuss reification and its potential relationship to advice in OHCs below.

Reification and participation are the two central processes that lie at the core of Wenger's COP framework (Wenger, 1998). Wenger considers these two processes as a duality rather than a dichotomy; they are intertwined, synergistic, and both fundamental to how COPs negotiate meaning and learn. Participation is the more easily defined of the two, and Wenger's usage is consonant with its dictionary definition: “To have or take a part or share with others (in some activity, enterprise, etc.)” (Wenger, 1998, p. 55). In the context of a COP, the activity or enterprise is social, and stems from membership in social communities.

Reification is more slippery, and in Wenger's usage is a process through which aspects of practice ‘congeal’ to something more stable. Some literature conflates the output of reification with the production of physical artifacts, but Wenger's definition is more nuanced:

Reification can take a great variety of forms: a fleeting smoke signal, or an age-old pyramid, an abstract formula or a concrete truck… a telling glance or a long silence, a private knot on a handkerchief or a controversial statue on a public square… What is important about all these objects is that they are only the tip of an iceberg, which indicates larger contexts of significance realized in human practices… Properly speaking, the products of reification are not simply concrete, material objects. Rather they are reflections of these practices, tokens of vast expanses of human meanings. (Wenger, 1998, pp. 60–61)

Two additional aspects of reification are important for our application of the concept to online advice. First, Wenger uses the term reification to refer to both the object of reification and the process that produces it, explaining that “if meaning exists only in its negotiation, at the level of meaning, the process and the product are not distinct” (Wenger, 1998, p. 60). Thus, we will use the term reification to refer to both advice and the processes that produce it. Second, while reification may “freeze” an aspect of practice, it does not also freeze meaning, any more than the meaning of the U.S. Constitution is frozen by the document itself. Rather, the reified object can help focus the negotiation of meaning for a COP, which is itself an ongoing process. That is, “the process of reification thus compels us to renegotiate the meaning of its past products, in the same way that a scar keeps bringing a past foolishness or heroic deed into conversations” (Wenger, 1998, p. 88).

Several analyses have hinted at this ongoing negotiation around advice in OHCs. For example, Huh and Ackerman (2012) describe how members in an online diabetes support community engage in a kind of collective sensemaking, through which they continuously refine their understanding of diabetes and how to manage it. In one example, forum members share a “startup solution kit,” containing various resources and simple, easy to follow strategies that had been “well-polished through repetitive use” (Huh & Ackerman, 2012, p. 856). In a similar vein, Mamkykina et al. (2015) suggest that reification occurs in part via lateral engagement among users of a diabetes forum, which can lead to a transformation of ideas. Here, lateral engagement refers to the exchanges among heterogeneous users with different perspectives in long discussion threads.

Centering our analysis on the reification of advice in an online, discussion-based communities hold particular interest because of the particular form advice takes. The articulation of advice in a post is at once a reification, but this permanent artifact is itself somewhat ephemeral for the persistent, active members of the forum. While it may be recovered (e.g., via a search or bookmark), it is not ensconced in some highly salient artifact like a wiki article, and the focal point of member participation will move on to new conversations. Yet similar advice may be reified the next time a new seeker posts a similar request, affording an opportunity for adaptation. It is across a sequence of such reifications that we might observe the continuous negotiation of meaning, and perhaps learning. Our aim here to isolate and examine such longitudinal sequences. In the following sections, we outline our approach.

To trace the longitudinal reification of advice (RoA), we sought to identify sequential instances of the same advice embedded in posts that are arrayed across multiple threads. Prior studies (e.g., Huh and Ackerman, 2012, Mamykina et al., 2015) have examined individual discussion threads for related instances of collective information processing, but this constrains the time-scale of the processes these studies reveal. An important aim of our work is to elucidate sequences of RoA and the potential renegotiation of meaning these sequences imply across longer periods of time.

The primary virtual site of participation in most OHCs is the thread-based, asynchronous conversation. Conversation threads are usually initiated with a request for some kind of support, followed by a series of replies from multiple (possibly many) individuals. As described above, we anticipate that repeated encounters by the same members in the context of similar advice requests affords an opportunity for a continuous negotiation of meaning around advice. To operationalize this idea, we focus on two characteristics of this process—first, the posts that transcribe repeated instances of reification of the same advice will be roughly “about” the same thing, though the content may drift as advice is refined; second, these posts will proceed in sequence across, rather than within threads, and will be authored by individuals that have jointly participated in other conversations. We refer to such a set of posts as meta-conversations (MCs). Fig. 1 offers a schematic illustration of how a pair of posts in an MC is arranged across threads.

Note that we do not expect MCs to follow the organization of typical conversation; e.g., we do not expect turn-taking to occur (Sacks, Schegloff, & Jefferson, 1974), or for references to be resolved or maintained across posts (H. Clark & Marshall, 1981). Instead, we intend the term MC to describe something less than a conversation to coordinate understanding about a specific topic, but more than the epidemiological diffusion of a viral video.

More formally, an MC is a set of posts, arranged as a directed acyclic graph. Each post has four features: the time (t) at which it occurred, the thread in which it appears (T), the identity of the poster (u), and the topic (o) of the post. For us, a topic is an algorithmically derived indicator of what a post is “about,” and hence a basis for an assumption about the similarity content across posts. A single link in the graph can be inferred from a tuple consisting of three posts:[Pt1(Ti,ui,oi),Pt2(Ti,uj,),Pt3(Tj,uj,oi)]where ij for all subscripts, and subject to the following constraints:t1<t2<t3;t3t2<kSim(Pt1,Pt2)<s

The constraints k and s capture two distinct intuitions about the nature of an MC. The constraint k indicates that after some period of time, we presume uj has either forgotten about the original post, or is not part of an MC. The constraint s is an additional measure of lexical similarity, reflecting the idea that algorithmically derived topics may be too imprecise a classification to establish the joint membership of posts in a single MC.

A link in an MC can be thought of as a possible transfer of knowledge about reified advice between two users. The potential (observed) transfer occurs between the first and last post in the preceding tuple; the second post merely serves as evidence that the two posters (ui and uj) shared a common context and is not part of the meta-conversation. That is, there is a potential for transfer any time two people (ui and uj) participate (as evidenced by their posting activity) in a single thread, and then the latter person subsequently (uj) posts in a different thread about the topic (oi) originally posted by the first poster (ui). In the methods section, we describe how we applied the above model to our dataset, and in particular, how we inferred values for s (lexical similarity between posts) and k (maximum time distance between two posts in different threads with a reasonable probability of those posts constituting a meta-conversation).

Section snippets

Corpus description

We drew our data from discussion communities hosted by WebMD, a relatively popular and long-standing site for health information. Several prior studies have examined these specific support communities (Huh, 2015, Introne et al., 2016, Kanthawala et al., 2016, Ridings and Wasko, 2010), and our work builds on that described by Introne et al. (2016) in that it adopts a characterization of coreness; we describe users as being more core like as we traverse a scale from extra-periphery (individuals

RQ1: identifying sequences of RoA

We found that roughly 5% of all posts examined belonged to meta-conversations consisting of at least two posts, and component sizes were in general distributed as a long tail. A total of 6,578 meta-conversations were found. Here, we report several measures obtained from the crowd-based coding procedure to validate that these algorithmically identified sequences. First, we examined the Inter-Class Correlation (ICC) statistic, which as described above is appropriate for randomly assigned coders (

Discussion & future work

With this paper, we have introduced an algorithmic method for extracting posts that are arrayed across multiple threads and demonstrated that these posts trace a continual process of advice reification. Our perspective is orthogonal to that taken in prior studies that examine collective sensemaking in OHCs, in two senses. It is literally orthogonal because meta-conversations move across the boundaries of designed affordances in the system (i.e. thread-based discussions). It is metaphorically

Conclusion

Online health support is an important tool in the evolving landscape of digital health. Yet, while many have rushed to embrace the role of big data and personal informatics, the expertise of those who successfully manage their own health conditions and support others is often marginalized, and even disparaged. We believe this expertise to be an untapped resource, especially when it is pooled and enhanced through its online expression in the form of advice that is freely given. Skepticism about

References (60)

  • D.M. Blei et al.

    Dynamic topic models

  • J.S. Brown et al.

    Knowledge and organization: A social-practice perspective

    Organization Science

    (2001)
  • S.L. Bryant et al.

    Becoming wikipedian: Transformation of participation in a collaborative online encyclopedia

  • J. Chang et al.

    Reading tea leaves: How humans interpret topic models

    Advances in Neural Information Processing Systems

    (2009)
  • D.V. Cicchetti

    Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology

    Psychological Assessment

    (1994)
  • H. Clark et al.

    Definite reference and mutual knowledge

  • K. Crowston et al.

    Core and periphery in free/libre and open source software team communications

  • J.D. Culver et al.

    Medical information on the internet

    Journal of General Internal Medicine

    (1997)
  • C. Danescu-Niculescu-Mizil et al.

    No country for old members: User lifecycle and linguistic change in online communities

  • K.P. Davison et al.

    Who talks? The social psychology of illness support groups

    The American Psychologist

    (2000)
  • A.J. Devitt

    Writing genres

    (2004)
  • J. Eisenstein et al.

    A latent variable model for geographic lexical variation

  • A. Esquivel et al.

    Accuracy and self correction of information received from an internet breast cancer list: Content analysis

    BMJ British Medical Journal

    (2006)
  • G. Eysenbach et al.

    Empirical studies assessing the quality of health information for consumers on the world wide web: A systematic review

    Journal of the American Medical Association

    (2002)
  • S. Fox et al.

    Chronic disease and the internet

    (2010)
  • R. Goel et al.

    The social dynamics of language change in online networks

  • S.P. Goggins et al.

    Group informatics: A methodological approach and ontology for sociotechnical group research

    Journal of the Association for Information Science and Technology

    (2013)
  • S.P. Goggins et al.

    Creating a model of the dynamics of socio-technical groups

    User Modeling and User-Adapted Interaction

    (2013)
  • W. Gomaa et al.

    A survey of text similarity approaches

    International Journal of Computer Applications

    (2013)
  • D. Greene et al.

    Exploring the political agenda of the european parliament using a dynamic topic modeling approach

    Political Analysis

    (2017)
  • Cited by (0)

    View full text