On Simulating the Propagation and Countermeasures of Hate Speech in Social Networks

Round 1

Reviewer 1 Report

This paper describes an approach for simulating the spread of hate speech in social media by considering different aspects.

At least a brief description of what "Gab" is must be added. This is not well-known like Twitter.

The paragraph in lines "141-145" must be relocated. In this position is quite confusing to read first a description of Figure 5, and then introducing Figure 1, and so on.

On lines 158-159, what is the meaning of "society"? It refers to users in the "dataset" or individuals in general terms.

When introducing terms like "clique", it is better to briefly define them and include a more reliable reference than Wikipedia.

Which could be the impact of including some natural language processing techniques for generating/evaluating hate speech content in the proposed approach?

Author Response

We thank the reviewer for the comments and suggestions. Next, we reply them here. Moreover, for easy reference, we highlight in blue the changes made in the paper, which is attached here.

Comments:

This paper describes an approach for simulating the spread of hate speech in social media by considering different aspects.

Point 1: At least a brief description of what "Gab" is must be added. This is not well-known like Twitter.

Response 1: Indeed, Gab is less known than Twitter. Gab.com is an American social networking service launched publicly in May 2017 that is known for its far-right userbase. Gab is critizised for using free speech as a shield for users and groups who have been banned from other social media. As suggested by the reviewer, we have included this brief description the first time Gab is mentioned (see third footnote in page 1).

Point 2: The paragraph in lines "141-145" must be relocated. In this position is quite confusing to read first a description of Figure 5, and then introducing Figure 1, and so on.

Response 2: We agree that it is confusing to read, in page 4 (former lines 141-145, now lines 149-50), a reference to Figure 5 before introducing Figure 1. To overcome this, we have included here an additional figure depicting the gamma distribution referenced in these lines. This new figure (now Figure 1 in page 4) is thus referenced instead of Figure 5.

Point 3: On lines 158-159, what is the meaning of "society"? It refers to users in the "dataset" or individuals in general terms.

Response 3: When using society in the “swap to a hateful society” in page 5 (former line 159, now line 163), we mean all the users in the social network. As suggested, we clarify this in the paper in footnote 6.

Point 4: When introducing terms like "clique", it is better to briefly define them and include a more reliable reference than Wikipedia.

Response 4: A clique is a maximal complete subgraph of a graph. In this manner, every two distinct vertices in the clique are adjacent. We have included such definition in subsection 4.1 Network construction (page 6) and a reference to the journal paper by Moon, J.W. and Moser, L. entitled “On cliques in graphs”. Thus, as suggested, we have removed the Wikipedia reference.

Point 5: Which could be the impact of including some natural language processing techniques for generating/evaluating hate speech content in the proposed approach?

Response 5: We appreciate the question about the impact of including some natural language processing techniques for generating/evaluating hate speech content in the proposed approach. However, it is worth noticing that since we are simulating the network, we apply several simplifications to the modelling process (as it is usually done, see e.g., [12]). In particular, we do not include actual messages in the posts, we just model opinion diffusion considering the hate score of the users that are connected and how their opinions change (see equation 1 in page 8). Similarly, instead of identifying a post based on its content, we define a hateful posts based on the users who authored them (as explained in section 6, page 11). However, this does not prevent us from simulating different success rates in identifying hate content and their consequences, and this is exactly what we study when varying the probability of deferring content p_{defer}. Thus, if we actually implemented the natural language processing methods to classify a post as hateful or not, we would be just implementing one specific value in the x axes of the plots in figure 9 (previous figure 8) in page 16. We explain this at the beginning of section 6.2.2 (now in page 15), which reads: “The simulations aimed at studying the effect of deferring hateful content are conducted by varying the deferring probability p_{defer} and considering that a value of 0.7 deems realistic if we take into account the state-of-the-art accuracy  in recognition of hate speech [47]”.

In order to stress the simplifications done by our simulations, we have changed the introduction so to mention in the second paragraph of page 2 that: “However, real social networks are too complex to experiment on, and therefore, this paper is devoted to propose an agent-based model [12] as a virtual experiment for the simulation and comparison of countermeasures against the spread of hatred. Although multi-agent based simulations encompass several simplifications, they are definitely useful to conduct what-if analysis to assess the system’s behaviour under various situations [14–16], which in our case correspond to different countermeasures.”

Author Response File: Author Response.pdf

Reviewer 2 Report

The authors use an agent-based model to investigate the spread of hate-speech and possible counter measures. The construction of the model, especially, the set-up of a preferential graph with sensible assumptions on new connections is well executed.

I do have some reservations when it comes to the contagiousness of hate, which seems grossly over-estimated in this model:
The model uses hatefulness as a contagious and stable attribute, such as opinions and positions that impacts the users around a vertex. This inevitably leads to the "hate cores" you describe. But how valid is such an assumption? If that was how hate spread on the internet, there would be more than 1% hateful users out there. This seems to be a misspecification of the model. Of course, there are people that (temporarily) get infected with hate speech and deal out salty responses, but they don't become contagious. They will calm down over time.
The lacking external validity of the model also becomes apparent when models may become 30% hateful and turn the whole population to hateful users. If such a thing was possible, it would have happened many years ago. In the introduction, you mention that the number of hateful users is relatively stable but the amount of hate speech increases. A model that captures these phenomena would have to have a relatively stable amount of hateful users that increase their edges continuously and a larger group of not so hateful persons that may sometimes engage in hate speech, especially if they are triggered. The model should be amended to resemble the summarized phenomena.

A different but very important issue is the nomenclature of factors: Calling the skewness of the distribution of hatefulness "Education" is highly misleading. This is not about education, it's just the shape of society that may vary in the degree of almost hateful citizens. You could call it "Temperament", "Restraint", "Social Norms" or "Lead Poisoning". All of that would account for different shapes for the Gamma-distribution. Calling it "Education" invites social inferences that are just not supported or even addressed by the model.

Minor points:
Line 7: "withinsocial" should be 2 words.
Line 45: "sandbox" is a patronising term for such a model. I would suggest "virtual experiment".

Author Response

We thank the reviewer for the comments and suggestions, which have been used to improve the paper. Next, we reply them here. Moreover, for easy reference, we highlight in blue the changes made in the paper, which is attached here (please, consider that it also includes the changes made due to the other reviewer).

Comments:

The authors use an agent-based model to investigate the spread of hate-speech and possible counter measures. The construction of the model, especially, the set-up of a preferential graph with sensible assumptions on new connections is well executed.

** Point 1:

I do have some reservations when it comes to the contagiousness of hate, which seems grossly over-estimated in this model:

The model uses hatefulness as a contagious and stable attribute, such as opinions and positions that impacts the users around a vertex. This inevitably leads to the "hate cores" you describe. But how valid is such an assumption? If that was how hate spread on the internet, there would be more than 1% hateful users out there. This seems to be a misspecification of the model. Of course, there are people that (temporarily) get infected with hate speech and deal out salty responses, but they don't become contagious. They will calm down over time.

The lacking external validity of the model also becomes apparent when models may become 30% hateful and turn the whole population to hateful users. If such a thing was possible, it would have happened many years ago. In the introduction, you mention that the number of hateful users is relatively stable but the amount of hate speech increases. A model that captures these phenomena would have to have a relatively stable amount of hateful users that increase their edges continuously and a larger group of not so hateful persons that may sometimes engage in hate speech, especially if they are triggered. The model should be amended to resemble the summarized phenomena.

** Response 1:

We appreciate your comment on our modelling of the contagiousness of hate. However, it is worth noticing that, since we are simulating the social network, we apply several simplifications to the modelling process (as it is usually done, see e.g., [12]). For instance, we do not include actual messages in the posts, we just model opinion diffusion considering the hate score of connected users. In particular, we model opinion change by means of equation 1 in section 4.2. As introduced in the related work section, the literature is populated with several models of opinion diffusion, and, as explained in section 4.2, we borrow equation 1 from one such models: the Deffuant-Weisbuch (DW) model [25]. In brief, this equation models how opinions change based on the received posts, and thus, as the reviewer rightly points out, hatefulness is not a static property. We use this model because the aim of this paper is, rather than providing a new model for the propagation of hate speech, to reuse insights from the research literature to build a baseline model upon which we can analyse how some countermeasures can contain the spread of hatred.

According to the literature, and in line of what the reviewer mentions, people can express or not express their opinion depending on their perceived environment and situation, thus, crying out their opinion emotionally and calming down a few days later. We captured this aspect by defining a reposting probability. However, this does not subsequently mean that rather hateful people (e.g. rather racist) will become normal (non-racists) after uttering a hateful (racist) post. In fact, those people tend to gather in specific social networks, such as Gab, which provide shield for users that have been banned from other social media (see new footnote number 3 in first page).

Due to all the simplifications made, multi-agent based simulations are not meant to be (and cannot be) accurate on the models, but they are definitely useful to conduct what-if analysis to assess the system’s behaviour under various situations (in our case, different countermeasures) [Page et al. 2013, Ahrweiler et al. 2015, Sulis and Terna 2021].

[Page et al. 2013] Le Page, C., Bazile, D., Becu, N., Bommel, P., Bousquet, F., Etienne, M., ... & Weber, J. (2013). Agent-based modelling and simulation applied to environmental management. In Simulating social complexity (pp. 499-540). Springer, Berlin, Heidelberg.

[Ahrweiler et al. 2015] Ahrweiler, Petra, Schilperoord, Michel, Pyka, Andreas and Gilbert, Nigel (2015) 'Modelling Research Policy:  Ex-Ante Evaluation of Complex Policy Instruments' Journal of Artificial Societies and Social Simulation 18 (4) 5 <http://jasss.soc.surrey.ac.uk/18/4/5.html>. doi: 10.18564/jasss.2927

[Sulis and Terna 2021] Sulis, E., & Terna, P. (2021). An Agent-based Decision Support for a Vaccination Campaign. Journal of Medical Systems, 45(11), 1-7.

Specifically, we made simplifications in many aspects, time and size being two major ones. Firstly, we discretize time by means of a simulation time unit called ticks. Ticks are managed by the multi-agent simulation frameworks, and thus, using them is common practice in agent-based modelling (e.g. Schieb and Preuß 2018, Stefanelli and Seidl 2017). Secondly, due to our simulation limitations, the sizes of our simulated networks are much smaller than real social networks. Moreover, it is widely known that convergence depends on the structure of the network (Golub et al. 2007). For this reason, we simulate multiple networks, but we should just take our results as tendencies rather than exact numbers. For example, when considering the "swaps to a hateful society", there are some real work examples, such as xenophobia in Poland (Bilewicz et. al 2020), where whole countries fell into hateful mode. However, as we model single communities (since our simulations just consider one-component networks, and thus disregard other complex topologies), it cannot be strictly compared to big societies of whole countries.

In order to stress the simplifications done by our simulations, we have changed the introduction so to mention in the second paragraph of page 2 that: “However, real social networks are too complex to experiment on, and therefore, this paper is devoted to propose an agent-based model [12] as a virtual experiment for the simulation and comparison of countermeasures against the spread of hatred. Although multi-agent based simulations encompass several simplifications, they are definitely useful to conduct what-if analysis to assess the system’s behaviour under various situations [14–16], which in our case correspond to different countermeasures.”

 

** Point 2: A different but very important issue is the nomenclature of factors: Calling the skewness of the distribution of hatefulness "Education" is highly misleading. This is not about education, it's just the shape of society that may vary in the degree of almost hateful citizens. You could call it "Temperament", "Restraint", "Social Norms" or "Lead Poisoning". All of that would account for different shapes for the Gamma-distribution. Calling it "Education" invites social inferences that are just not supported or even addressed by the model.

** Response 2: We agree with the reviewer that calling the skewness of the distribution of hatefulness ``Education'' can be misleading. However, we abuse of language and use it to align our simplified model with the measures proposed by researchers and politicians (as mentioned in the introduction). Following a “what-if” analysis to assess the system’s behaviour under various situations, what we aim to study with the modelling of this countermeasure is how the system evolves when users have different hateful profiles that we assume could be the long term result of an education on media literacy. Thus, rather than simulating the education of complex concepts such as critical thinking, recognition of hate speech, democratic values, or tolerance, we simplify our simulation and just model the resulting media literacy.              

Point 3: Minor points:

Line 7: "withinsocial" should be 2 words.

Line 45: "sandbox" is a patronising term for such a model. I would suggest "virtual experiment".

Response 3: Thanks for spotting these, we have changed them accordingly in the text (see pages 1 and 2).

Author Response File: Author Response.pdf