Abstract: AI-based NLP literature has explored antipathy toward the marginalized section of society, such as migrants, and their social acceptance. Broadly, extant literature has conceptualized this as an online hate speech detection task and employed predictive ML models. However, a crucial omission in this literature is the genesis (or causality) of online hate, i.e., why do we hate migrants? Drawing insights from social science literature, we have identified three antecedents of online hate: Cultural, Economic, and Security concerns. Subsequently, we probe -which of these concerns triggers higher toxicity on online platforms? Initially, we consider OLS-based regression analysis and SHAP framework to identify the predictors of toxicity, and subsequently, we use Double Machine Learning (DML)-based casual analysis to investigate whether good predictors of toxicity are also causally significant. We find that the causal effect of Cultural concerns on toxicity is higher than Security and Economic concerns.

CCS CONCEPTS

• Human-centered computing →Social media; • Computing methodologies →Natural language processing.

Online Hate, Causality, Toxicity, Double machine Learning

AI-based opinion mining of social media data has emerged as one of the dominant streams of computational social science research [9, 13, 37, 60, 64]. Interestingly, a significant portion of this extant literature has explored “various practical applications of opinion mining, such as product pricing, competitive intelligence, market prediction, election forecasting, nation relationship analysis, and risk detection in banking systems”[59, p.10], and relatively a “little

research has tried to understand opinions in the social and geopolitical context” [13, p.75]. One such relatively under-studied domain is the cross-border movements of migrants from one country to another [20, 63]. Migration has been a defining feature of human history. Cross-border migration may happen for a variety of reasons, including economic opportunities in the host country, war-induced political instability in the home country, or reunification with family members. Generally, migrants are driven by an aspiration for a better life, and consequently, they are willing to face significant risks in pursuit of that aspiration [30, 32]. However, migrants often face discrimination and hostility in their new country [6, 8, 33]. These hostilities, in the host country, often get accentuated for a variety of reasons, ranging from cultural differences to socioeconomic concerns [34]. Overall, migration is a multifaceted, complex, and contentious concern across the world, and cross-border movements of migrants will continue to be a source of political debate and discussion over the years. Thus, we need to have a nuanced understanding of societal opinions toward migrants. The number of international migrants has steadily increased over the last few decades (from 153 million in 1990 to 280.6 million in 2020) [49]. However, the international migrant stock as a percentage of the total population has marginally increased from 2.9% in 1990 to 3.6% in 2020 [49]. Country-wise international migrants as a percentage of the total population are broadly in the range of 10% to 20% for most European nations like Austria (19.3%), Belgium (17.3%), Denmark (12.4%), France (13.1%), Germany (18.8%), Italy (10.6%), Netherlands (13.8%), Spain (14.6%), and Sweden (19.8%). This percentage is also similar for countries like Canada (21.3%), the UK (13.8%), and the USA (15.3%) [49]. However, the perceptions of many of these advanced nations about migrants do not confirm these realities. For instance, the Depth Index of Globalization report pointed out that “Western Europeans across 8 countries, on average, believe immigrants comprise 25% of their country's population, while the actual figures average to only 12%. In the United States, citizens estimated that 42% of the country's population was born abroad, versus the actual ratio of only 14%” [23, p.69]. Interestingly, Americans overestimate the share of the black and Hispanic population [57], and most European countries overestimate the share of the Muslim population [42]. Thus, Carl (2016) concluded that "public beliefs about immigrants and immigration are widely regarded as erroneous ... public typically overestimate the immigrant fraction of the population by 10–15 percentage points" [10, p.2]. Perception matters, irrespective of whether it is factually correct or not, because not only the perception impacts the social media deliberations, like hatred toward migrants, but also the political discourses and policies [58]. An apprehensive view toward migrants, especially by far-right political parties, is gaining momentum across

countries and affecting political mandates [36, 47, 58]. Accordingly, a relatively recent Pew Research Centre survey of 27 nations also indicates that Europeans, especially from “Greece (82%), Hungary (72%), Italy (71%) and Germany (58%) say fewer immigrants or no immigrants at all should be allowed to move to their countries” [16]. Overall, this survey also reports that “a median of 45% say fewer or no immigrants should be allowed to move to their country, while 36% say they want about the same number of immigrants. Just 14% say their countries should allow more immigrants”[16]. Consequently, migration-related policies are getting stringent over the years, especially in advanced nations [49]. Societal perceptions about migrants also get reflected in usergenerated social media content. A handful of studies noted bipolar opinions such as deserving vis-à-vis undeserving migrants, securityrelated concerns vis-à-vis humanitarian concerns, or critical visà-vis positive opinions [24, 26, 29, 43]. Broadly, social media data reveals two diametrically opposite views: humanitarian concerns (because migrants face discrimination in terms of societal acceptance or employment opportunities or the lack of access to basic health or education facilities) and threat perceptions (because migrants can be a cultural as well as security threats, or they can be the economic burden for the host country) [29, 33, 56]. These threat perceptions often get expressed through offensive and aggressive language. This type of offensive content, such as swear words, is labeled as hate speech in the literature. We note that a significant portion of the migration literature probed hate speech against migrants. Apparently, apprehensive views or apathy towards international migrants on social media platforms reconfirm the findings of the Depth Index of Globalization or the Pew Research Centre survey [16, 23]. However, to the best of our knowledge, AI-based opinion mining literature has rarely explored - why do we hate migrants? There can be a wide range of answers. The host country might be apprehensive about the possible change in the social fabric due to migrants, or they may have negative stereotypes about migrants [33]. A segment of the host country might be concerned about the possible economic impact (due to the influx of migrants). Social science literature offers multiple theoretical lenses to explain these apprehensive views or xenophobia. However, AI-based research has rarely employed these theoretical lenses. Also, it is necessary to move beyond binary hate speech detection and understand the toxicity levels of hate speech. Additionally, we also need to probe – which specific societal concern leads to higher toxicity? Hence, our study probes the causality between societal concerns and toxicity levels of hate tweets. We employ a Double Machine Learning-based approach to investigate the same. To sum up, our study draws theoretical insights from xenophobia-related social science literature and employs a causal machine learning-based approach to explain the toxicity levels of hate tweets against migrants.

The watershed moment of hate speech detection against migrants literature was Task 5 of the 2019 International Workshop on Semantic Evaluation commonly referred to as SemEval-2019. Task 5 of SemEval-2019 was basically a two-class (or binary) classification

task to predict whether a tweet was hateful or not against immigrants and women, i.e., Subtask A, and the subsequent task was to predict whether the hate tweet was aggressive or not, i.e., Subtask B [6]. Basile et al. [6, p.54] pointed out that this was “one of the most popular tasks in SemEval-2019 with a total of 108 submitted runs for Subtask A and 70 runs for Subtask B, from a total of 74 different teams”. They concluded that this high number of participation “confirms the growing interest of the community around abusive language in social media and hate speech detection in particular” toward the marginalized section of society (e.g., migrants)[6, p. 62]. The scope of this paper doesn't allow us to discuss all these papers. Hence, we are focusing on a few interesting papers for brevity. Öztürk and Ayvaz [46] investigated the public opinions toward the Syrian refugees on Twitter. This bilingual study noted that Turkish tweets (35% positive) displayed more positive sentiments than English tweets (12% positive tweets). English tweets were primarily neutral (48%) or expressed negative sentiment (40%). It is worth noting that Turkey welcomed many Syrian refugees, and it got reflected in social media deliberations. Similarly, Kreis [36] analyzed tweets with the hashtag #refugeesnotwelcome, and finds that the “argumentation lies on sharing and recirculating events, stories, articles, or images where refugees and immigrants are depicted as criminals and exploiters” and the social media deliberations “discursively construct a national or European identity on the one hand, and migrants as ‘criminals' and ‘out-group members' on the other, which underscores racist and ethnocentric ideologies and discourses circulated by nationalist-conservative Europeans” [36, p.511]. Siapera et al. [56, p.1] also noted that the dominant frames are “revolving around security and safety on one hand and humanitarianism on the other.” Security and safety concerns are triggered not only due to the depiction of refugees and immigrants as criminals but also due to violence by refugees or migrants. In this context, Ekman [18, p.1] probed the emergence of vigilante gangs, who “claim to protect citizens from alleged violent and sexual attacks by refugees” in Europe, and these “racist actors use social media to mobilize and organize street politics targeting refugees/immigrants”. Overall, Ekman [18] concluded that social media communication aids these racist actors in anti-refugee mobilization, but generally, they lack public support and media framing is negative about these vigilant gangs. Sanguinetti et al. [55] developed an annotated corpus of about 6,000 Italian tweets, and it has identified more refined categories of online hate: aggressiveness, offensiveness, irony, and stereotype. Arcila-Calderón et al. [3, p.1] analyzed Spanish tweets related to the Boat Aquarius event and observed that “a significant part of messages expressed rejection or hate—often supported by stereotypes and lies—towards refugees and migrants and towards politicians”. Similarly, Calderón et al. [8, p.40] also explored Spanish tweets to understand verbal rejections and they found that “rejection toward migrants was significantly bigger than over refugees”. Arcila-Calderón et al. [4, p.33]pointed out that “regions with greater support recorded a lower level of hate speech on Twitter.” Recently, Poletto et al. [48] performed a systematic review of various resources and benchmark corpora for online hate detection. Broadly, a significant portion of this literature has conceptualized this as an online hate-speech detection or classification task.

Causality of Online Hate: A Research Gap? A handful of recent studies employed causal approaches in the context of online hate. For instance, Chandrasekharan et al. [12] examined the effects of banning subreddits on the activity, language, and migration trends of former members. This study used synthetic controls and difference-in-differences (DiD) regression analysis to establish causality. Similarly, Munger [41] investigated the impact of social sanctioning on race-based harassment on Twitter. This paper employed a randomized control experiment to evaluate the effects of varying influence and identity of the harassers and the anonymity of the platform. Olteanu et al. [45] explored the impact of Islamophobia and Islamist terrorist attacks on hateful content on Twitter and Reddit. This study used synthetic counterfactual time series analysis to establish causal inference. Accordingly, Müller and Schwarz [40] also investigated the impact of high Twitter usage on anti-Muslim hate crimes in the USA. This study considered fixedeffects panel regression to establish causal inference. von Essen and Jansson [62] probed the relationship between web anonymity and the risk of web exposure to hateful posts. This study also used DiD approach to establish causality. Chandrasekharan et al. [11] studied the impact of quarantining subreddits on member participation and language, as well as new member influx. This study employed interrupted time series (ITS) and bootstrapping tests to establish causal inference. Another study by Thomas et al. [61] examined the impact of external events and regulatory actions on member activity and attrition on Reddit. They used Mahalanobis Distance Matching (MDM), DiD analysis, and multivariate Bayesian changepoint analysis to establish causality. Ananthakrishnan and Tucker [2] investigated the presence and propagation (or "virality") of hate speech on YouTube. This paper employed instrumental variables to establish causal inference. To sum up, a survey by Founta and Specia [22] pointed out that prior studies mostly considered platforms like Twitter, Reddit, Facebook, and YouTube; and primarily employed methodological tools like DiD approach, Experimentation, regression analysis, MDM, Bootstrapping, etc. Overall, the literature is scant and relatively recent. None of the prior studies, to the best of our knowledge, employed a state-of-the-art methodology like Double Machine Learning to probe - why do we hate migrants? Also, Founta and Specia [22] categorically mentioned that none of the prior studies probed the toxicity level of hate speech. Our paper attempts to address these Research Gaps by using DML for probing toxicity level of hate speech.

Peterie and Neil [47, p.23] conceptualized Xenophobia as a ‘fear of the stranger', and they pointed out that "several causal models have been proposed to explain this fear and the hostility it produces. However, disciplinary boundaries have limited productive dialogue between these approaches." Hence, we adopt an interdisciplinary approach to identify appropriate theoretical lenses to investigate the causality between the antecedents of xenophobia and the toxicity levels of hate tweets. Our first theoretical lens is (New) Racism Perspective. This theoretical lens emphasizes that asylum seekers with different skin colors are ‘Cultural Others' and they can be a threat to the ‘National Self'. In other words, negative

perceptions about migrants revolve around notions of cultural and religious differences between ‘Cultural Others' and the ‘National Self' [47]. Consequently, a segment of the host nation would be worried about cultural changes, such as the loss of traditional values of their communities. Khatua and Nejdl [31] juxtaposed generic migrant-related deliberations and Ukrainian migrant-specific discussions on the YouTube platform, and this study broadly confirms this ‘cultural others' vis-a-vis the ‘national self' hypothesis. In other words, (new) racism theory suggests that apathy toward migrants can be triggered by Cultural Concerns. For instance, Muslim migrants are facing hostility in advanced economies like the USA and European countries. Existing literature has conceptualized this as Islamophobia. In the context of the USA, Argyle et al. [5, p.1] pointed out that the "common explanation for growing antipathy is the perceived link between Muslims and terrorism". Interestingly, the media (specifically the right-wing media) plays a crucial role in shaping public perceptions about migrants. However, "the perception of Muslims as a cultural threat may generate more anti-Muslim hostility than their association with political violence" [5, p.1]. This sort of biased perception may also lead to Security Concerns. Carl [10] found a strong correlation between apathy (or opposition) to immigrants with crime statistics (like immigrant arrest rates or arrest rates for violent crime). According to False Beliefs Lens, these security concerns get accentuated by the propagation of "erroneous information regarding the threat posed by asylum seekers" and "detaining asylum seekers in closed (often prison-like) facilities sends a strong message that asylum seekers are dangerous criminals and would-be terrorists" - in other words, these are "politically and discursively manufactured fears" [47, p.24-26]. Accordingly, Stansfield and Stone [58, p.4] also argued that the "criminalization of immigrants" is a political construct. The Sociobiology Theory offers an alternate explanation to these cultural and security concerns. This theory argues that xenophobic behaviors are deeply ingrained in our subconscious mind. Earlier homo sapiens used to form a small cohort and these groups used to compete against each other for survival. Thus, an individual used to trust her fellow group members and they were hostile toward outsiders beyond their cohort. This territorial behavior resembles the animal kingdom where resources are generally limited. In other words, a segment of the host nation can be concerned about the potential strain on public services, such as healthcare and education. In communities with high levels of migration, there may be increased demand for these services, which can put pressure on the existing infrastructure. So, the sociobiology stream explains xenophobia through this territorial behavior and resource scarcity perspective. The above resource scarcity point also hints at the potential economic impact of migration, which can be complex and can vary depending on a wide range of factors - ranging from the skills and qualifications of the migrants to job market conditions in the host economy. A sudden surge in the labor supply due to the inflow of migrants might adversely affect wages as well as employment opportunities, and this becomes an existential threat [39]. Migrants may be seen as taking jobs away from local workers, which can create tension and resentment [58]. Consequently, host nations, especially the lower strata of society or lower-wage workers, may be apprehensive about migrants, particularly in sectors with high levels of unemployment. Capitalist Globalization Lens offers a

theoretical explanation of these concerns. It argues that "xenophobic fear and hostility ... is a function of structural inequalities" [47, p.30]. According to Oxfam 2022 report, "since 1995, the top 1% have captured nearly 20 times more of global wealth than the bottom 50% of humanity, "and this "inequality contributes to the death of at least one person every four seconds" [1]. Multinational corporations, the torchbearers of capitalist globalization, have made the working class vulnerable. Right-wing politicians are grabbing this opportunity and they propagate the narrative - surplus workers due to migration is compromising the job security of host nation workers. This insecurity triggers Economic Concerns. Consequently, xenophobic feelings against migrants gain momentum. This propagation of a partially erroneous political narrative is also in sync with the false belief argument. Many times our perceptions about migrants are far from reality, and this explains why Europeans or Americans overestimated the migration statistics [42, 57]. However, it is also worth noting that host nations are not homogenous. Messing and Ságvári [39, p.8] pointed out that some countries, such as Hungary and the Czech Republic, are apprehensive of "migration in all respects, while Sweden, Norway, Germany and Estonia see migration as more enriching than endangering their country in terms of economic or cultural quality". In short, the reasons for the ‘fear of strangers' or hatred toward migrants will vary from person to person or from one host nation to another host nation.

The previous section has identified cultural concerns, security concerns and economic concerns as casual antecedents of hate, and explained - why these concerns can trigger hatred towards migrants, through multiple theoretical lenses. As aforementioned, we need to investigate - which specific concern triggers a higher level of toxicity? To answer the same, we are employing Double Machine Learning-based (DML) Casual Inference approach [14]. This DML-based causal inference approach helps us to precisely pinpoint the direct causal effect of concern by modeling the interactions between various input features, and subsequently, it allow us to answer which specific concern leads to a higher toxicity level. Specifically, we are employing the Neyman-Rubin Causal Model of Potential Outcomes to address our research question. Keith et al. [28] provides a holistic review of prior studies which has employed a causal inference approach to text data and considered various linguistic features as treatment and control. Our problem formulation follows a similar approach. Problem Formulation: We consider N data points, i.e., N tweets for our analysis, some of which have received a treatment, i.e., expressed one of the concerns, and we estimate the outcome, i.e., the level of toxicity. Let us assume that 𝑇𝑖denotes the treatment indicator with 1 for treatment and 0 for control for 𝑖𝑡ℎunit. We define 𝑌𝑖(1) and 𝑌𝑖(0) as the potential outcomes of unit i under treatment and confounders, respectively. Then we can calculate the individual treatment effect as follows: 𝑇𝑖= 𝑌𝑖(1) −𝑌𝑖(0). Conceptually, we need to know both 𝑌𝑖(1) and 𝑌𝑖(0) for the same data point, but this is practically not feasible. For instance, it is not possible to know two potential outcomes (say, with drugs and

without drugs) for the same person. This is commonly known as fundamental problem of causal inference. The challenge is precisely the same for our research problem. Practically, we cannot have two toxicity levels (i.e., two potential outcomes) of a specific tweet with and without a specific concern. Because of this fundamental problem of causal inference, unit-level causal effects cannot be directly observed. The potential outcomes approach of the Rubin Causal Model was widely used by prior studies for probing causal inference. Interestingly, randomized experiments allow us to estimate population-level causal effects. A randomized experiment assigns units (or human beings for the clinical trial of a drug) randomly to treatments (i.e., with drugs or without drugs). Because of this random assignment, the groups are (on average) equivalent to each other, and the difference between the two groups allows us to estimate the population-level causal effects. Similarly, we argue that N tweets with broadly even distribution of concerns are similar to randomized experiments. In other words, the difference in toxicity between two groups of tweets with-the-concern and without-the-concern will indicate the population-level causal effect of that specific concern. Subsequently, an estimate of the average causal effect (also referred to as the average treatment effects) can be obtained by computing the difference in means between the treated and control samples. In our case, it will be the difference in toxicity levels between tweets with-the-concern and without-the-concern. This average treatment effects (ATE) is:

𝐴𝑇𝐸= 𝐸[𝑌𝑖1 −𝑌𝑖0]

It is worth noting that the above ATE estimation is based on the assumption groups are (on average) equivalent to each other. This might not be always feasible. For instance, due to ethical concerns randomized experiments (to test the effect of poison or virus on human beings) may not be suitable (or ethical) in real-life contexts. To tackle such non-random assignments, literature suggested using propensity score matching, a statistical approach, to appropriately decipher the causal inference [14, 28, 53]. This approach rectifies the biases due to non-randomized experiments by matching control units similar to treatment units. Thus, we use the DML model to calculate the ATE [14, 15]. DML is efficient for accurately estimating treatment effects in the presence of many potential confounders - especially when all potential confounders can have a direct effect on the treatment as well as on the outcome. DML models are suitable for high-dimensional settings with a large number of covariates [27]. In other words, DML models are suitable when the causal inference problem has too many features, and when the outcome cannot be satisfactorily modeled by parametric functions due to the unknown data distribution[14, 15]. Following Chernozhukov et al. [14], we also consider the estimation of ATE and ATTE (i.e., Average Treatment Effects on the Treated) under the unconfoundedness assumption of Rosenbaum and Rubin [52]. Accordingly, our treatment effects are heterogeneous and the treatment variable, D, is binary, 𝐷∈{0, 1}. If Y denotes the outcome variable of interest and Z denotes a set of control variables, then our DML models can be represented using the random vectors of (Y, D, Z) as follows: 𝑌= 𝑔0(𝐷,𝑍) + 𝜁, 𝐸[𝜁|𝑍, 𝐷] = 0,

𝐷= 𝑚0(𝑍) + 𝜐, 𝐸[𝜐|𝑍] = 0.

Following Chernozhukov et al. [14, p.1], D is not additively separable, this model allows for very general heterogeneity in treatment effects. Common target parameters 𝜃0 in this model are the Average Treatment Effects (ATE) 𝜃0 = 𝐸[𝑔0(1,𝑍) −𝑔0(0,𝑍)], and the Average Treatment Effects on the Treated (ATTE) 𝜃0 = 𝐸[𝑔0(1,𝑍) −𝑔0(0,𝑍)|𝐷= 1]. The confounding factor(s) Z affects the treatment variable D via the propensity score, 𝑚0(𝑍) := 𝐸[𝐷|𝑍], and the outcome variable via the function 𝑔0(𝐷,𝑍). Both these functions are unknown and potentially complicated, and thus, we employ ML methods to estimate these functions.

Data: We have considered the SemEval-2019 Hate speech dataset [6] for our causal inference analysis. However, this was a multilingual hate speech detection task against immigrants and women on Twitter. Hence, we have considered a subset of this dataset for our analysis. First, we have considered English tweets. Second, we have considered immigrant-related tweets. Third, our focus is to investigate the toxicity. So, we considered the tweets which were labeled as hate tweets. Finally, our corpus comprises 1352 hate tweets against migrants or immigrants. Toxicity: Following prior studies in the context of online abuse and harassment, we employed a widely used toxicity detection tool, i.e., Google's Perspective API4. This tool reports the level of toxicity of an English text, with a score ranging from 0 to 1. The Perspective API is an open-source toxicity detection tool, accessible via Google's website1. Extant literature has extensively employed the Perspective API to investigate toxicity on social media platforms like YouTube [44], Facebook [25] and Twitter [21]. For example, ElSherief et al. [19] considered two aspects of this API, the toxicity and attack on commenter, to explore whether hate is aimed at a particular individual or entity, or generalized against a group of individuals sharing common characteristics. Similarly, another study used this API to identify toxic conversations and individual user behavior, in comparison to non-toxic conversations [54]. Cuthbertson et al. [17] used this API to develop a Twitter bot that combats abusive tweets directed towards prominent female leaders in the Canadian political system by disseminating supportive tweets about them. Cultural, Economic, and Security Concerns: Drawing insights from our review, we have considered three core concerns for our casual inference analysis (refer to Table 1).2 As noted, when a tweet explicitly or implicitly indicates migrants as Cultural Others, we have labeled it as a Cultural Concerns-related tweet [31, 47]. For instance, we find some users are referring to migrants as illegal aliens who are invading the host nation. Accordingly, tweet #1 in

1https://www.perspectiveapi.com/ 2Annotation was done through a rigorous and iterative process. In the first round, we explained these concerns to a post-graduate student and asked him to read a few scholarly articles to have an in-depth understanding. Simultaneously, one of the authors, who had significant experience in handling migrant-related tweets, also performed this annotation task. The inter-annotator agreement was around 0.75. Mostly, the disagreements occurred for tweets that expressed more than one concern, such as tweet #7. To tackle the same, in the second round, both annotators deliberated regarding these tweets and adopted a multi-labeling approach, e.g. tweet #7. This has generated a corpus of 1352 migrant-related hate tweets.

indicating that migrants who are not getting culturally assimilated into the host country should go back. On the other hand, tweet #2 is explicitly expressing an anti-Muslim view. Hence, both these tweets are classified as Cultural Concerns-related tweets. Economic Concerns are mostly coming from resource scarcity or job market perspective [39, 58]. For example, tweet#3 opines that U.S Taxpayers' money should not fund welfare-related expenses of immigrants. Similarly, tweet#4 argues that the inflow of immigrants, as cheap foreign labor, can disrupt the job market of the host economy. Hence, we labeled tweets #3 and #4 as Economic Concerns-related tweets. Security Concerns of the host economies mostly revolve around the fear of strangers. As noted, prior studies discovered an association between online hate and the violent activities of migrants [10, 36, 56]. Accordingly, multiple users, like tweets #5 and #6, share violent and criminal activities by migrants or refugees. Thus, we labeled these tweets as Security concerns-related tweets. Interestingly, we also find some tweets expressing multiple concerns. For instance, tweet #7 in indicating that migrants are terrorizing the citizens of the host nation, but this tweet also associates this concern with their religious background. Hence, we labeled this tweet for both Security-related as well as Cultural-related concerns. Finally, we also have hate tweets, such as tweet #8, in our corpus which are not specifically referring to either of these concerns. These tweets are becoming the base category for our analysis. Linguistic Input Features: A careful scrutiny of our corpus reveals that some of our anti-migrant tweets are objective or factual - especially tweets by right-wing media. Contrarily, the tweets by common users are more subjective or emotional, and consequently, the toxicity scores are high. Toxicity scores of objective tweets are low or insignificant because they are not opinionated or not using any harsh or swear words. Hence, to control the emotional quotient of a tweet in our analysis, we have considered the LIWC-based affect score. Basically, this is the cumulative score of aspects like sadness, anger, anxiety, etc. Additionally, the SemEval-2019Task [6] also labeled the aggressiveness of the tweets through a binary variable (1- aggressive, and 0 otherwise). So, we have also controlled the Aggressiveness of a tweet. Our brief review reveals that the genesis of Cultural Concerns is Otherness. Based on the usage of pronouns, LIWC reports first-person, second-person, and third-person scores of text contents. To capture the Cultural Others or third-person perspective, we have considered the summation of ‘third-person singular' (e.g., he, she, her, etc.) and ‘third-person plural' (e.g., they, their, etc.). We have also considered the combined scores of the first-person singular, first-person plural, and second-person - as another variable to capture the Cultural Us aspect. Prior studies associated male migrants more with Security Concerns compared to female migrants [51]. For instance, Rakotonarivo [50] argued that "female migrants are less likely to be employed than male migrants". Generally, female migrants are associated with basic occupations, such as domestic help or cleaning services [50]. Thus, the host nation may not associate female migrants with Economic or Security Concerns. Hence, we have considered LIWC-based female and male score. Additionally, we also considered two psycholinguistic attributes of our corpus: Cognitive Process and Perception from LIWC. The cognitive attribute captures aspects like insight, discrepancy, differentiation, etc.; and perception captures aspects like feeling, insight, etc.

Table 1: Representative Tweets and Concerns

4 Economic growth via immigration shifts wealth ... by flooding the market with cheap foreign labor. It spikes profits & stocks by cutting salaries for manual & skilled labor offered by blue-collar and white-collar employees

5 Skyrocketing rapes in Sweden that correlate with increased immigration? 2000+ women raped on New Year's by migrants in Cologne? Paris becoming littered with tent cities? Acid attacks and ethnic clusters in London?

(a) (b)

By leveraging the power of model interpretation tools such as OLS regression with SHAP explanation [38] and EconML [35], we interpret the black box model from both correlation and causation perspectives. In other words, first, we need to know which input features are good predictors of toxicity (through OLS regression and SHAP framework); and next, we want to see whether these features are also causally significant (through a DML-based approach). OLS-based Regression Analysis: Initially, we began with linear regression to investigate the relationships between toxicity and three core concerns and various linguistic features. Table 2 reports the descriptive statistics and the OLS regression results.

Economic

Security

Figure 1a reports the correlation between these variables, and significantly low correlations between variables nullify the chances of multicollinearity. Table 2 indicates a good distribution for most of our input variables. For instance, 50% tweets of our corpus are aggressive. Also, we have a balanced distribution of three concerns: Cultural (28.5%), Economic (19.8%), and Security (24.4%); and the coefficients of all these three concerns are positive and statistically significant, i.e., all three concerns lead to higher toxicity. Other variables are mostly statistically significant. Thus, empirical findings broadly confirm the theory-driven identification of various input features. Overall, our OLS model has good prediction power with an Adjusted R-squared score of 0.794.

Figure 1: Correlation heat map of the variables (left figure) & Features according to their SHAP importance (right figure)

Table 2: Descriptive Statistics & OLS Regression Results

Mean S.D. Min. Max. Coefficient S.E. t-value P>|t|

Toxicity (Dependent Variable) 0.329 0.163 0.03 0.92 Cultural Concerns 0.285 0.451 0.00 1.00 0.126 0.010 12.475 0.000 Economic Concerns 0.198 0.399 0.00 1.00 0.032 0.011 2.830 0.005 Security Concerns 0.244 0.430 0.00 1.00 0.716 0.011 6.531 0.000 Affect 5.003 4.281 0.00 25.00 0.0146 0.001 14.491 0.000 Aggressiveness 0.507 0.500 0.00 1.00 0.1026 0.009 11.783 0.000 Cultural Us 2.697 3.694 0.00 22.22 0.0102 0.001 8.215 0.000 Cultural Others 2.078 3.061 0.00 16.67 0.0099 0.002 6.130 0.000 Female 0.412 1.557 0.00 16.67 0.0122 0.003 3.959 0.000 Male 0.497 1.691 0.00 16.67 0.0049 0.003 1.716 0.086 Cognitive Process 7.803 5.602 0.00 31.58 0.0062 0.001 8.608 0.000 Perception 1.494 2.696 0.00 27.27 0.0073 0.002 4.359 0.000

No. of Observations - 1352; Adj.R-squared: 0.794; ∗∗∗𝑝< 0.001; ∗∗𝑝< 0.01; ∗𝑝< 0.05

Correlation Interpretation (through SHAP Framework): Next, we trained a fine-tuned lightGBM regression model as a predictive ML model. Subsequently, we utilized the Shapley Additive exPlanations (SHAP) [38] framework to understand the importance of each feature for the model outcome and to better understand the correlations between features and the target variable. SHAP Values use a game theoretic approach to fairly distribute a reward among a set of players contributing to a certain outcome [38]. In the context of ML models, SHAP values provide an importance score for each input feature, indicating their contribution to the overall model outcome. In Figure 1b, the most critical features for predicting toxicity are sorted according to their level of importance from top to bottom. This plot indicates that Cultural Concerns, Aggressiveness, Affect, Cultural Us, and Cultural Others have a greater impact on the level of toxicity. Causal Interpretation: As aforementioned in Section 4, DML can control potential confounders (i.e., factors that simultaneously can have a direct effect on the treatment decision and the observed outcome). DML also tackles high-dimensional data and non-randomized distribution using statistical techniques like nonparametric modeling [14]. Additionally, DML performs two predictive tasks: predicting the outcome from the controls, and predicting the treatment from the controls [14, 15]. Next, DML combines these two predictive models in the final stage to efficiently estimate the heterogeneous treatment effect. In the process, DML also addresses various statistical concerns such as small mean squared error, asymptotic normality, or construction of confidence interval [7]. The EconMl package offers several variants for the final model estimation. It also provides valid inference (i.e., confidence interval construction) for measuring the uncertainty of the learned model [7]. Thus, we employ EconML to investigate whether the important features of the SHAP Values are also causally significant or not. We have seen which features are good predictors of toxicity, but are these features also causally significant? Table 3 reports the causal effects of Cultural, Economic, and Security Concerns and other input features on toxicity level. Table 3 sorts the features by causal significance (p-value). If we compare this causal summary table to

the ordering of the SHAP plot (Figure 1b), we notice that mostly the top predictive features are also causally significant. However, the orders are not exactly the same. For instance, unlike the SHAP plot, Affect is causally more important than Aggressiveness. Similarly, Security concerns came up in the order and casually this became more significant than Economic Concerns or Cultural Otherness. Probably, the correlation between toxicity and Economic Concerns is high, but after controlling other input features Economic Concerns is causally less significant than Security concerns. In the context of causality analysis, we may consider that these concerns (e.g. Cultural concerns) are similar to Average Treatment Effect (ATE) for toxicity.

Conditional Average Treatment Effect (CATE): Prior analysis revealed the direct effect of various concerns and input features on toxicity level at an average level. However, the conditional effect (or treatment) of these input features (i.e., Cultural, Economic, and Security Concerns) on toxicity levels may not be the same. Thus, we need to probe how toxicity levels may increase with the addition of a feature/treatment like Cultural Concerns. Or, how does the overall toxicity level (conditionally) change if every tweet refers to Cultural Concerns? In other words, for a given treatment (or concern), we want to know the counterfactuals if we intervene in a different way. We are estimating 𝜃0(𝑥) to understand this conditional average effect of a concern/treatment. 𝜃0(𝑥) can be estimated as follows: 𝜃0(𝑥) = E[𝑌(1) −𝑌(0)|𝑋= 𝑥] for a low-dimensional feature X, where 𝑌(𝑑) the potential outcome with 𝑑∈{0, 1} Figures 2, 3, and 4 report the CATE analysis. Here, in the left heterogeneity tree images, dark green (burgundy) means a higher (lower) level of toxicity. Similarly, in the right images, the orange (blue) histogram reports the level of toxicity without (with) the concern. Histogram analysis reveals that Cultural and Security concerns (as a treatment) positively impact the toxicity level, but this is not the case for Economic concerns. The heterogeneity tree

Table 3: Features sorted by Causal Significance of Toxicity based on 𝑝−𝑣𝑎𝑙𝑢𝑒

Feature point SE Z Stat 𝑝−𝑣𝑎𝑙𝑢𝑒 ci-lower ci-upper

Cultural Concerns 0.082344 0.010980 7.499602 6.401167e-14 0.060824 0.103864 Affect 0.007840 0.001266 6.193526 5.883303e-10 0.005359 0.010320 Aggressiveness 0.056699 0.009265 6.119631 9.379235e-10 0.038539 0.074858 Cultural Us 0.005376 0.001449 3.710332 2.069876e-04 0.002536 0.008217 Security Concerns 0.034536 0.010595 3.259575 1.115793e-03 0.013770 0.055302 Economic Concerns -0.034558 0.012585 -2.745881 6.034860e-03 -0.059225 -0.009891 Cultural Others 0.004312 0.001616 2.668535 7.618283e-03 0.001145 0.007479

Figure 2: Causal interpretation of Cultural Concerns: Heterogeneity tree and Histogram

Figure 3: Causal interpretation of Security Concerns: Heterogeneity tree and Histogram

of Cultural Concerns reveals that the absence of Economic Concerns, combined with a lower Cognitive attribute, but high Cultural Others enhance toxicity in Figure 2. This prominent role of Cultural Others aspect broadly confirms the (new) Racism Theory [47]. Similarly, Figure 3 indicates tweets with low female scores, but highly emotional contents (i.e., high affect scores) lead to higher toxicity. Contrarily, tweets with high female scores combined with Cultural Others (i.e., tweets about female migrants) reduce threat perception or Security Concerns, and subsequently, toxicity, and

this confirms prior studies like Rettberg and Gajjala [51]. Finally, Economic concerns analysis in Figure 4 reveals that tweets with the absence of Cultural concerns coupled with high female scores lower toxicity. This indicates female migrants are not economic threats - as pointed out by Rakotonarivo [50]. However, Cultural Concerns coupled with high Cultural Other scores, and tweets with high perceptual scores lead to higher toxicity - this broadly confirms that perception matters [33] and the Capitalist Globalization lens can explain economic threats [47].

Figure 4: Causal interpretation of Economic Concerns: Heterogeneity tree and Histogram

Our study was an attempt to explore the black box of online hate. We referred to various theoretical lenses to understand the genesis of online hate toward migrants. Extant literature suggests that the depiction of migrants as criminals or cultural threats is driven by racist ideologies [36, 47, 58]. Unfortunately, deviant behaviors (e.g. violent or criminal activities) by a few desperate migrants and distorted narratives around these events reinforce the ‘Cultural Others' ideology. Similarly, conceptualizing migrants as ‘economic threats' can also be a political construct [1, 39, 47, 58]. Our OLSbased regression analysis reveals that these threat perceptions are good predictors of online toxicity. Moreover, we note additional linguistic attributes, such as aggressiveness, subjectivity or affect score, female references, or third-person perspective, are also good predictors of online toxicity. These findings from predictive ML models are insightful, but from the policy-making perspective, we need to know the direct causal effect of our input features on toxicity level, instead of just predicting the toxicity using a set of input features. Hence, we have employed state-of-the-art Double Machine Learning (DML) models to probe - do certain concerns have a higher causal effect on the toxicity level? We find that Cultural concerns lead to higher toxicity in comparison to Security and Economic concerns (Table 3).

We do not endorse or support any form of online hate directed toward any section of society. Using quoted hate tweets is solely for research purposes, aiming to understand the nature and extent of such harmful content on digital platforms. Including these tweets in our research does not reflect or represent our views in any manner. On the contrary, we vehemently condemn all forms of online hate and discrimination. Our paper aims to raise awareness, combat hate speech, and promote safe and inclusive social media for everyone.

Funding for this paper was, in part, provided by the Federal Ministry of Education and Research (BMBF), Germany under the project LeibnizKILabor with grant No. 01DD20003.

[1] Nabil Ahmed, Anna Marriott, Nafkote Dabi, Megan Lowthers, Max Lawson, and Leah Mugehera. 2022. Inequality kills: The unparalleled action needed to combat unprecedented inequality in the wake of COVID-19. (2022).

[2] Uttara M Ananthakrishnan and Catherine E Tucker. 2022. The drivers and virality of hate speech online. Available at SSRN 3793801 (2022).

[3] Carlos Arcila-Calderón, David Blanco-Herrero, Maximiliano Frías-Vázquez, and Francisco Seoane-Pérez. 2021. Refugees welcome? Online hate speech and sentiments in Twitter in Spain during the reception of the boat Aquarius. Sustainability 13, 5 (2021), 2728.

[4] Carlos Arcila-Calderón, Patricia Sánchez-Holgado, Cristina Quintana-Moreno, Javier J Amores, and David Blanco-Herrero. 2022. Hate speech and social acceptance of migrants in Europe: analysis of tweets with geolocation//Discurso de odio y aceptación social hacia migrantes en Europa: análisis de tuits con geolocalización. Comunicar 30, 71 (2022), 21–35.

[5] Lisa Argyle, Ian Gray, Matti Nelimarkka, and Rochelle Terman. 2016. Cultural Threats and Islamophobia in American News Media. In NCAPSA American Politics Workshop. 1–44.

[6] Valerio Basile, Cristina Bosco, Elisabetta Fersini, Debora Nozza, Viviana Patti, Francisco Manuel Rangel Pardo, Paolo Rosso, and Manuela Sanguinetti. 2019. Semeval-2019 task 5: Multilingual detection of hate speech against immigrants and women in twitter. In Proceedings of the 13th international workshop on semantic evaluation. 54–63.

[7] Keith Battocchi, Eleanor Dillon, Maggie Hei, Greg Lewis, Paul Oka, Miruna Oprescu, and Vasilis Syrgkanis. 2019. EconML: a Python package for ML-based heterogeneous treatment effects estimation. GitHub (2019).

[8] Carlos Arcila Calderón, David Blanco-Herrero, and María Belén Valdez Apolo. 2020. Rejection and hate speech in Twitter: Content analysis of Tweets about migrants and refugees in Spanish. Revista Española de Investigaciones Sociológicas (REIS) 172, 172 (2020), 21–56.

[9] Erik Cambria, Björn Schuller, Yunqing Xia, and Catherine Havasi. 2013. New avenues in opinion mining and sentiment analysis. IEEE Intelligent systems 28, 2 (2013), 15–21.

[10] Noah Carl. 2016. Net opposition to immigrants of different nationalities correlates strongly with their arrest rates in the UK. Open Quantitative Sociology & Political Science (2016).

[11] Eshwar Chandrasekharan, Shagun Jhaver, Amy Bruckman, and Eric Gilbert. 2022. Quarantined! Examining the effects of a community-wide moderation intervention on Reddit. ACM Transactions on Computer-Human Interaction (TOCHI) 29, 4 (2022), 1–26.

[12] Eshwar Chandrasekharan, Umashanthi Pavalanathan, Anirudh Srinivasan, Adam Glynn, Jacob Eisenstein, and Eric Gilbert. 2017. You can't stay here: The efficacy of reddit's 2015 ban examined through hate speech. Proceedings of the ACM on Human-Computer Interaction 1, CSCW (2017), 1–22.

[13] Hsinchun Chen and David Zimbra. 2010. AI and opinion mining. IEEE Intelligent Systems 25, 3 (2010), 74–80.

[14] Victor Chernozhukov, Denis Chetverikov, Mert Demirer, Esther Duflo, Christian Hansen, and Whitney Newey. 2017. Double/debiased/neyman machine learning of treatment effects. American Economic Review 107, 5 (2017), 261–265.

[15] Victor Chernozhukov, Denis Chetverikov, Mert Demirer, Esther Duflo, Christian Hansen, Whitney Newey, and James Robins. 2018. Double/debiased machine learning for treatment and structural parameters.

[16] Phillip Connor and Jens Manuel Krogstad. 2018. Many worldwide oppose more migration–both into and out of their countries. (2018).

[17] Lana Cuthbertson, Alex Kearney, Riley Dawson, Ashia Zawaduk, Eve Cuthbertson, Ann Gordon-Tighe, and Kory W Mathewson. 2019. Women, politics and Twitter: Using machine learning to change the discourse. arXiv preprint arXiv:1911.11025 (2019).

[18] Mattias Ekman. 2018. Anti-refugee mobilization in social media: The case of soldiers of Odin. Social Media+ Society 4, 1 (2018), 2056305118764431.

[19] Mai ElSherief, Vivek Kulkarni, Dana Nguyen, William Yang Wang, and Elizabeth Belding. 2018. Hate lingo: A target-based linguistic analysis of hate speech in social media. In Proceedings of the International AAAI Conference on Web and Social Media, Vol. 12.

[20] Lee Fiorio, Guy Abel, Jixuan Cai, Emilio Zagheni, Ingmar Weber, and Guillermo Vinué. 2017. Using twitter data to estimate the relationship between short-term mobility and long-term migration. In Proceedings of the 2017 ACM on web science conference. 103–110.

[21] Paula Fortuna, Juan Soler, and Leo Wanner. 2020. Toxic, hateful, offensive or abusive? what are we really classifying? an empirical analysis of hate speech datasets. In Proceedings of the 12th language resources and evaluation conference. 6786–6794.

[22] Antigoni-Maria Founta and Lucia Specia. 2021. A Survey of Online Hate Speech through the Causal Lens. arXiv preprint arXiv:2109.08120 (2021).

[23] Pankaj Ghemawat and S Altman. 2013. Depth index of globalization–And the big shift to emerging economies. Pankaj Ghemawat/IESE Report, University of Navarra, Spain (2013).

[24] Jeanine PD Guidry, Lucinda L Austin, Kellie E Carlyle, Karen Freberg, Michael Cacciatore, Shana Meganck, Yan Jin, and Marcus Messner. 2018. Welcome or not: Comparing# refugee posts on Instagram and Pinterest. American Behavioral Scientist 62, 4 (2018), 512–531.

[25] Samuel S Guimarães, Julio CS Reis, Filipe N Ribeiro, and Fabrício Benevenuto. 2020. Characterizing toxicity on facebook comments in brazil. In Proceedings of the Brazilian Symposium on Multimedia and the Web. 253–260.

[26] Asmelash Teka Hadgu, Kaweh Djafari Naini, and Claudia Niederée. 2016. Welcome or not-welcome: Reactions to refugee situation on social media. arXiv preprint arXiv:1610.02358 (2016).

[27] Paul Hünermund, Beyers Louw, and Itamar Caspi. 2021. Double Machine Learning and Automated Confounder Selection–A Cautionary Tale. arXiv preprint arXiv:2108.11294 (2021).

[28] Katherine A Keith, David Jensen, and Brendan O'Connor. 2020. Text and causal inference: A review of using text to remove confounding from causal estimates. arXiv preprint arXiv:2005.00649 (2020).

[29] Aparup Khatua and Wolfgang Nejdl. 2021. Analyzing European Migrant-related Twitter Deliberations. In Companion Proceedings of the Web Conference 2021. 166–170.

[30] Aparup Khatua and Wolfgang Nejdl. 2021. Struggle to Settle down! Examining the Voices of Migrants and Refugees on Twitter Platform. In Companion Publication of the 2021 Conference on Computer Supported Cooperative Work and Social Computing. 95–98.

[31] Aparup Khatua and Wolfgang Nejdl. 2022. Endorsement Analysis of Migrantrelated Deliberations on YouTube: Prior to and During 2022 Ukrainian crisis. In Open Challenges in Online Social Networks. 31–38.

[32] Aparup Khatua and Wolfgang Nejdl. 2022. Rites de Passage: Elucidating Displacement to Emplacement of Refugees on Twitter. In Proceedings of the 33rd ACM Conference on Hypertext and Social Media. 214–219.

[33] Aparup Khatua and Wolfgang Nejdl. 2022. Unraveling Social Perceptions & Behaviors towards Migrants on Twitter. In Proceedings of the International AAAI Conference on Web and Social Media, Vol. 16. 512–523.

[34] Aparup Khatua, Emilio Zagheni, and Ingmar Weber. 2023. Host-Centric Social Connectedness of Migrants in Europe on Facebook. In Proceedings of the International AAAI Conference on Web and Social Media, Vol. 17. 1143–1147.

[35] Emre Kiciman, Eleanor Wiske Dillon, Darren Edge, Adam Foster, Agrin Hilmkil, Joel Jennings, Chao Ma, Robert Ness, Nick Pawlowski, Amit Sharma, et al. 2022. A Causal AI Suite for Decision-Making. In NeurIPS 2022 Workshop on Causality for Real-world Impact.

[36] Ramona Kreis. 2017. # refugeesnotwelcome: Anti-refugee discourse on Twitter. Discourse & Communication 11, 5 (2017), 498–514.

[37] Bing Liu. 2012. Sentiment analysis and opinion mining. Synthesis lectures on human language technologies 5, 1 (2012), 1–167.

[38] Scott M Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. Advances in neural information processing systems 30 (2017).

[39] Vera Messing and Bence Ságvári. 2018. Looking behind the culture of fear. Cross-national analysis of attitudes towards migration. (2018).

[40] Karsten Müller and Carlo Schwarz. 2020. From hashtag to hate crime: Twitter and anti-minority sentiment. Available at SSRN 3149103 (2020).

[41] Kevin Munger. 2017. Tweetment effects on the tweeted: Experimentally reducing racist harassment. Political Behavior 39 (2017), 629–649.

[42] Alberto Nardelli and George Arnett. 2014. Today's key fact: you are probably wrong about almost everything. The Guardian 29 (2014).

[43] Adina Nerghes and Ju-Sung Lee. 2018. The refugee/migrant crisis dichotomy on Twitter: A network and sentiment perspective. In Proceedings of the 10th ACM conference on web science. 271–280.

[44] Adewale Obadimu, Esther Mead, Muhammad Nihal Hussain, and Nitin Agarwal. 2019. Identifying toxicity within youtube video comment. In Social, Cultural, and Behavioral Modeling: 12th International Conference, SBP-BRiMS 2019, Washington, DC, USA, July 9–12, 2019, Proceedings 12. Springer, 214–223.

[45] Alexandra Olteanu, Carlos Castillo, Jeremy Boy, and Kush Varshney. 2018. The effect of extremist violence on hateful speech online. In Proceedings of the international AAAI conference on web and social media, Vol. 12.

[46] Nazan Öztürk and Serkan Ayvaz. 2018. Sentiment analysis on Twitter: A text mining approach to the Syrian refugee crisis. Telematics and Informatics 35, 1 (2018), 136–147.

[47] Michelle Peterie and David Neil. 2020. Xenophobia towards asylum seekers: A survey of social theories. Journal of Sociology 56, 1 (2020), 23–35.

[48] Fabio Poletto, Valerio Basile, Manuela Sanguinetti, Cristina Bosco, and Viviana Patti. 2021. Resources and benchmark corpora for hate speech detection: a systematic review. Language Resources and Evaluation 55 (2021), 477–523.

[49] Migration Data Portal. 2023. Migration statistics. https://www. migrationdataportal.org/. [Online; accessed 15-March-2023].

[50] A Rakotonarivo. 2020. Who are the women on the move? A portrait of female migrant workers. ILO, 18 December 2020.

[51] Jill Walker Rettberg and Radhika Gajjala. 2016. Terrorists or cowards: negative portrayals of male Syrian refugees in social media. Feminist Media Studies 16, 1 (2016), 178–181.

[52] Paul R Rosenbaum and Donald B Rubin. 1983. The central role of the propensity score in observational studies for causal effects. Biometrika 70, 1 (1983), 41–55.

[53] Paul R Rosenbaum and Donald B Rubin. 1984. Reducing bias in observational studies using subclassification on the propensity score. Journal of the American statistical Association 79, 387 (1984), 516–524.

[54] Nazanin Salehabadi, Anne Groggel, Mohit Singhal, Sayak Saha Roy, and Shirin Nilizadeh. 2022. User Engagement and the Toxicity of Tweets. arXiv preprint arXiv:2211.03856 (2022).

[55] Manuela Sanguinetti, Fabio Poletto, Cristina Bosco, Viviana Patti, and Marco Stranisci. 2018. An italian twitter corpus of hate speech against immigrants. In Proceedings of the eleventh international conference on language resources and evaluation (LREC 2018).

[56] Eugenia Siapera, Moses Boudourides, Sergios Lenis, and Jane Suiter. 2018. Refugees and network publics on Twitter: Networked framing, affect, and capture. Social Media+ Society 4, 1 (2018), 2056305118764437.

[57] John Sides and Jack Citrin. 2007. European opinion about immigration: The role of identities, interests and information. British journal of political science 37, 3 (2007), 477–504.

[58] Richard Stansfield and Brenna Stone. 2018. Threat perceptions of migrants in Britain and support for policy. Sociological Perspectives 61, 4 (2018), 592–609.

[59] Shiliang Sun, Chen Luo, and Junyu Chen. 2017. A review of natural language processing techniques for opinion mining systems. Information fusion 36 (2017), 10–25.

[60] Jiliang Tang, Yi Chang, and Huan Liu. 2014. Mining social media with social theories: a survey. ACM Sigkdd Explorations Newsletter 15, 2 (2014), 20–29.

[61] Pamela Bilo Thomas, Daniel Riehm, Maria Glenski, and Tim Weninger. 2021. Behavior change in response to subreddit bans and external events. IEEE Transactions on Computational Social Systems 8, 4 (2021), 809–818.

[62] Emma von Essen and Joakim Jansson. 2020. Misogynistic and xenophobic hate language online: a matter of anonymity. (2020).

[63] Emilio Zagheni, Venkata Rama Kiran Garimella, Ingmar Weber, and Bogdan State. 2014. Inferring international and internal migration patterns from twitter data. In Proceedings of the 23rd international conference on world wide web. 439–444.

[64] Daniel Zeng, Hsinchun Chen, Robert Lusch, and Shu-Hsing Li. 2010. Social media analytics and intelligence. IEEE Intelligent Systems 25, 6 (2010), 13–16.