From associations to sarcasm: Mining the shift of opinions regarding the Supreme Court on twitter

Abstract

Twitter is a valuable source for learning about public opinion and political communication. Applying data mining to Twitter content (i.e., “Twitter mining”) offers a way to analyze large numbers of tweets to help us understand political associations the public makes. However, the use of incivility and sarcasm in political discourse may pose a challenge for Twitter mining in the context of politics. In this study, we apply Twitter mining to the 2018 confirmation of Judge Brett Kavanaugh to the Supreme Court to look for possible changes in public opinion of the Court in the wake of the confirmation hearing and to determine whether sarcasm in political messages on Twitter can alter the results of computational methods when using large datasets. Examining two waves of tweets, one in the days immediately following the confirmation and one a month later, we find evidence of a shift in public opinion as associations between the Supreme Court and partisanship emerge only in the latter period. Using sentiment analysis, we also demonstrate that sarcasm led to over-categorization of positive tweets, which altered the results by suggesting that the public viewed partisanship on the Supreme Court favorably.

Introduction

Research on Twitter and politics covers both elite and public use of Twitter. Elite-centered studies consider which political officials use Twitter, how, and for what purposes. Who uses Twitter has been of particular interest in studies of congressional members [1], [2], [3], and campaign-focused research has looked at gender differences in congressional candidates’ Twitter use [4]. The content of elected officials’ tweets can be a key to understanding what activities officials prioritize in office (e.g. [5], [6]). Scholars have also relied on tweets to gain insight into the relationship between elected officials and their constituents or followers on Twitter (e.g. [7], [8]). Recognizing journalists’ reliance on Twitter, some studies focus on elite Twitter users’ influence on media coverage in the context of political campaigns [9] and in governing [10].

Many scholars have turned to Twitter to learn more about the political behavior, opinion, and communication of the public. Three areas of study have gained considerable attention: Twitter in the context of campaigns and in the development of social movements, the effects of Twitter on political interest, and the rhetoric and tone in political tweets. Campaign studies have examined the effectiveness of candidate messages delivered via Twitter compared to messages voters receive from traditional news media [11]. Tweets in campaigns have been analyzed in several ways [12] including to provide evidence of ideological echo chambers [13], compare the spread of fake news and corrections of such stories [14], and study citizens’ perceptions of candidates [15] and responses to candidate speeches [16]. Numerous studies exist on the role of Twitter in social movements, particularly the use of social media to react to events and encourage the formation of collective identities that result in activism (e.g. [17], [18], [19], [20]). And there have been efforts to demonstrate the political effects of Twitter on the public; for example, Bode and Dalrymple [21] found in a survey that Twitter users were more interested in politics and less trusting of mainstream media, suggesting important implications for political communication. Twitter is an excellent source for rhetorical studies with scholars using tweets to examine incivility, sarcasm, and the use of humor in political discourse [22], [23], [24].

The messages produced on Twitter (i.e., tweets) may be analyzed qualitatively as researchers read them, or they may be automatically examined through a computational lens. Automatically finding patterns in the data produced on social websites is known as social web mining, and specifically Twitter mining when the social platform of interest is only Twitter [25]. A very large number of tweets is one of the key reasons to perform Twitter mining instead of, or in complement to, a qualitative approach. For instance, the phenomenon of interest may only be observed through a massive amount of data, such as tracking the monthly opinions of various communities in each US state over months. Although Twitter mining may lack the depth or nuance of qualitative methods on individual tweets, coping with datasets consisting of millions of tweets requires a computational method and special infrastructure [26]. Such large datasets are either impossible to process for a team of researchers, or would limit researchers to simple forms of qualitative analyses that can be accurately performed via crowdsourcing. Twitter mining is a well-established approach in health behaviors [27], [28], [29], for instance by analyzing foods consumed using 503 million tweets [30]. Our approach applies Twitter mining to the 2018 confirmation of Judge Brett Kavanaugh. This topic was previously analyzed by Darwish, who used 23 million tweets to analyze the polarization of 687,194 users [31]. While the work of Darwish focuses on polarization around Kavanaugh, we analyze a less direct effect: whether the Kavanaugh case changes public opinion about the Supreme Court itself. Our first research question thus contributes to the literature on politics and media:

• Is there a shift in the public opinion of the Supreme Court before and after the confirmation of Judge Brett Kavanaugh?

  The analysis of political discourse on Twitter has its own challenges, in part due to the use of incivility, sarcasm, and humor. Sarcasm has been the topic of numeroxml us papers on Twitter mining [32], whose methodologies range from a reliance on self-disclosure of sarcasm (e.g., via the hashtags #sarcasm or #sarcastic [33], [34]) to the use of advanced artificial tools such as neural networks [35]. In this paper, our second research question is whether sarcasm does matter for Twitter mining in the context of political discourse. Formally, our question is:

• Can sarcasm in political messages posted on Twitter significantly alter the results of computational methods even when using large datasets?

  The remainder of this paper is organized as follows. In Section 2, we provide a succinct background to the context of our case study, that is, the confirmation of Judge Brett Kavanaugh. Given this context, we introduce our methods in Section 3, in line with the framework that we recently used in Sandhu et al. [36]. In Section 4, we provide our results on the case study. The final section contextualizes our results with respect to our two research questions, and concludes with suggestions for future work.