A Multi-Agent System for guiding users in on-line social environments

https://doi.org/10.1016/j.engappai.2020.103740Get rights and content

Highlights

  • Design and implementation of sentiment, stress, and combined analyses on texts.

  • Proposal of a Multi-Agent system (MAS) for user guiding in on-line social sites.

  • Integration of the different analyses implemented in the MAS.

  • Experiments to discover the most useful analyses to predict future bad outcomes.

  • Experiments to test our MAS in a real-life environment.

Abstract

The present work is a study of the detection of negative affective or emotional states, the high-stress levels that people have using social network sites (SNSs), and the effect that this negative state or stress level has on the repercussions of posted messages. We aim to discover to what extent a user that has a state detected as negative by an analyzer (Sentiment analyzer and Stress analyzer) can affect other users and generate negative repercussions, and also determine whether it is more suitable to predict a future negative situation using different analyzers. We propose two different methods for creating a combined model of sentiment and stress, and we use them in our experimentation to discern which one is more suitable for predicting future negative situations that could arise from the interaction between users, and in what context. Additionally, we designed a Multi-Agent System (MAS) that integrates the analyzers to protect or advise users on a SNS. We have conducted this study to help build future systems that prevent negative situations where a user that has a negative state creates a repercussion in the SNS. This can help users avoid getting into a bad mood or help avoid privacy issues (e.g. a user that has a negative state posting information that the user does not really want to post).

Introduction

In our current society, people are immersed in an environment of on-line applications, of which social networks or Social Network Sites (SNSs) are the most important and most ubiquitous. One question that arises from this social interaction between users is whether or not users are safe. In Vanderhoven et al. (2016), risks and negative outcomes arising from the interaction between users in a SNS have been reviewed. In De Moor et al. (2008), and Livingstone et al. (2011), important risk factors are reviewed. Some of the most important risk factors are content risks, which are the risks of receiving inappropriate content, which can be varied (e.g. pornography, violence, and racism). Other important risks are contact risks, which are the risks that arise from meeting strangers and interacting with them. This can lead to cyber-harassment, privacy issues, and potentially harmful chat contacts. There are also commercial risks, which involve people receiving spam or getting asked for personal information, which also can lead to spam or aggressive marketing. It is also important to note that teenagers face several risks on SNSs and have characteristics that make them more vulnerable to them (Vandenhoven et al., 2014).

The decision making of users of on-line social platforms determines the way they interact, and an unfortunate choice may lead to incurring the risks mentioned above. For example publishing a post about private aspects of the user that attracts sexual predators, thus falling into contact risks, or publishing a post about the violence that could attract unwanted content about violence, incurring content risk. Decision making has been shown to be affected by the emotional state of the person making the decision. The effect of incidental moods, discrete emotions, integral affect, and regret on decision making have been reviewed in George and Dane (2016). By incidental moods and discrete emotions, we mean affective states that are not directly linked with the task at hand and that can arise from other sources (e.g., emotion arising from making decisions, thinking of someone, talking to someone that is not directly linked to the task being performed). On the other hand, integral affect is generated from the task being worked on. Finally, regret is a negative and conscious emotional reaction to self decision-making. In the review, the authors show that incidental moods affect decision making by altering people’s perception, and also that discrete emotions, integral affect, and regret affect decision making (regret acts as anticipated regret, thinking of the negative outcome before it actually happens). Moreover, stress has also been observed to be associated with a specific emotional state (high arousal and negative valence) and has been used in Thelwall (2017) to construct an adaptation (TensiStrength) of the sentiment strength detection software called SentiStrength (Thelwall et al., 2010) for detecting stress and relaxation magnitude in texts. Taking into account the previous, stress levels may be suitable for building a system that analyzes the state of the users along with sentiment values.

As stated above, the emotional state of the user can influence decision making. This can lead to future problems in a social on-line environment and may also make users fall into risks derived from their interaction. As an example of decision making resulting in a negative outcome, in Christofides et al. (2012), the authors show that when a user publishes a post, it can lead to regret and have negative consequences. Thus, it would be desirable for a system to be able to detect this sentimental state of the user and to react to it by trying to advise or protect him or her from possible future negative outcomes that could arise from his or her behavior.

Following the general idea of a system that analyzes the emotional state and stress levels of a user when he or she is interacting on on-line sites, in this work, we present a Multi-Agent System (MAS) for assessing guiding users in SNS by performing sentiment analysis, stress level analysis, and a combined analysis on user posts, and potentially giving them feedback if necessary. The system is built as a MAS to allow the tasks of different analyses to be performed separately and also, to allow the system to start processing new user input while still analyzing a previous one. This is possible due to the pipeline of agents that is built into the architecture, which is shown and discussed in Section 3. Different agents perform distinct analyses, and there are also other agents for the interaction with the users in the on-line social environment and for advising and retaining/retrieving data. This system has been integrated into a SNS to guide the users in their experience through the social environment by advising them when they are going to post messages, analyzing the text of the message with the different analyzers, and warning the user (or not), depending on the results of the analyzers in order to prevent a possible bad outcome (e.g., triggering an argument with other people or publishing content that the user does not really want to make public because of cognitive distortions). This MAS is a modification of a previous prototype presented in Aguado et al. (2018), where the analyzers were built using a Bayesian classifier. In the current version, we built the analyzers using feed-forward Artificial Neural Networks (ANN), which have been coded using the Tensorflow1 and Keras2 libraries with the programming language Python. We used ANNs to improve the classification accuracy and performance of the system since machine learning techniques have been used for aspect-based sentiment analysis achieving state-of-the-art accuracies (Schouten and Frasincar, 2016). In Aguado et al. (2018), we conducted a set of experiments with data from Twitter.com to determine which analysis was able to detect a state of the users that was propagated the most to the replies of the messages. We used the most present value in the replies as a metric of propagation so the analyzers detecting a state of the user that has high propagation would be more useful for detecting messages that generate problems in the future in a SNS. Since none of the analyzers showed a significant difference against the others, in this paper, we present new experiments with the new analyzers, using a new version of the combined analysis, and also show that one of the versions of combined analysis achieved to perform significantly better than the others.

The contribution of the present work is twofold. On the one hand, we constructed a new version of our MAS introducing new analyzers using ANN, and we used our MAS in experiments in a laboratory with a SNS called Pesedia (Bordera, 2016) that was used by a set of children, whose ages were between twelve and fifteen years old, and we were able to draw conclusions about how the proposed MAS works in a real-life environment. On the other hand, we extracted conclusions from experiments performed with data from Twitter.com to determine which analyzer predicts a state of the user that propagates more to the replies of the messages.

Regarding the advantages of our proposal comparing it to the state-of-the-art works, our proposed approach leverages the use of both MAS technologies and ANN to try to accomplish the task of prevention of potential issues, negative outcomes or propagation of negative sentiment polarity or high-stress levels on an on-line social environment, using for this purpose two sources of data, which are the sentiment polarity and stress levels of users interacting with the social environment, and proposing a combined analysis with two modalities. To the best of our knowledge, the state-of-the-art works only use one of those input data sources to prevent negative outcomes in SNSs. We also performed experiments to discover which of the analyses, including the combined modalities, should be used to be more informative in the system and in which cases. This is not the case on the current state-of-the-art works. One of the modalities of the combined analysis shown in the experiments performed in the current work that it can detect a state of the user that significantly propagates more in the network than the other analyzers, which is an advantage when creating a system that warns users based on the analyzed state on their messages. Related to the limitations of the current approach, as we created a system to be used integrated into a SNS for people of young age, we used a dataset made from texts written and labeled by people aged between twelve and fifteen years old for training the machine learning models. Using more datasets made from people of varied ages for creating different models and testing them could improve the performance of the system. Nevertheless, our experiments have shown that the system is able to perform as intended, as will be shown in Sections 4 Experimentation with the social network pesedia, 5 Experimentation with data from twitter.

The rest of the paper is structured as follows. Section 2 gives a description of the state-of-the-art works related to the topic of this paper. Section 3 describes the MAS proposed for guiding users in SNSs. Section 4 explains an experiment conducted with a SNS called Pesedia with known users at a laboratory. Section 5 describes the experiments performed with data from Twitter.com. Finally, Section 6 presents our conclusions and possible future lines of work.

Section snippets

Related work

Since our goal is to build a MAS with agents that implement sentiment analysis, stress analysis, and combined analysis to guide users in on-line social environments in an attempt to prevent possible future issues by analyzing the user state, we will discuss previous approaches for sentiment and stress analysis as well as risk prevention and privacy aiding in SNSs. We will also review previous approaches on modeling the user state, where the state of the user is used by the system to make

System description

We designed the system as a MAS that helps users by analyzing the data from the written messages that they post on social media, using different agents to perform different kinds of analyses (sentiment, stress, or combined) to determine if there should be feedback such as a warning displayed to the users to protect them from potential negative outcomes that could arise from their interaction. We used the SPADE multi-agent platform (Gregori et al., 2006) to implement the agents of the system

Experimentation with the social network pesedia

We performed experiments with real users of a social network called Pesedia, with the proposed MAS integrated as functionality for analyzing the emotional state and stress levels of the users and advising them at the moment of posting messages on the network. This social network was used by children, who ranged in age between twelve and fifteen years old. Pesedia was made with the social networking engine Elgg.3 The network is structured into diverse plug-ins that build

Experimentation with data from twitter

Since the data we collected in the experiments with the Pesedia SNS was not large (as it was generated in a short span of time by only 122 children), and one of our intentions was to discover how the system worked if it was used in different environments, we conducted experiments with data from Twitter.com. The goal of these experiments is to be able to decide what analysis or analyses should be considered to be more informative than others and in which cases a warning should be raised in the

Conclusions and future work

In this work, a MAS for protecting and guiding users through the analysis of their emotional state and stress levels has been presented. The MAS integrates analyzers that use text data from users to determine their sentiment polarity (Sentiment analyzer), stress level (Stress analyzer), and a combined analysis that uses both outputs, proposing two different forms of it (the ‘or’ and the ‘and’ version of the combined analysis). The analyzers are created using ANNs and the Tensorflow (see

CRediT authorship contribution statement

G. Aguado: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Software, Supervision, Validation, Visualization, Writing - original draft, Writing - review & editing. V. Julian: Conceptualization, Formal analysis, Investigation, Methodology, Visualization, Writing - review & editing. A. Garcia-Fornes: Conceptualization, Formal analysis, Investigation, Methodology, Visualization, Writing - review & editing. A. Espinosa: Conceptualization, Formal analysis,

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgment

This work was supported by the project TIN2017-89156-R of the Ministry of Economy, Industry and Competitiveness, Government of Spain .

References (37)

  • AguadoG. et al.

    Towards aiding decision-making in social networks by using sentiment and stress combined analysis

    Information

    (2018)
  • AlrubaianM. et al.

    Credibility in online social networks: A survey

    IEEE Access

    (2018)
  • BleiD.M. et al.

    Latent dirichlet allocation

    J. Mach. Learn. Res.

    (2003)
  • BorderaJ.A.

    PESEDIA. Red social para concienciar en privacidad

    (2016)
  • BuccafurriF. et al.

    Experimenting with certified reputation in a competitive multi-agent scenario

    IEEE Intell. Syst.

    (2015)
  • ChristofidesE. et al.

    Risky disclosures on facebook: The effect of having a bad experience on online behavior

    J. Adolesc. Res.

    (2012)
  • De MoorS. et al.

    Teens and ICT: Risks and opportunities. Belgium: TIRO

    (2008)
  • FeldmanR.

    Techniques and applications for sentiment analysis

    Commun. ACM

    (2013)
  • Cited by (0)

    View full text