Identifying Biases in Politically Biased Wikis through Word Embeddings

Authors

Markus Knoche∗

RWTH Aachen University markus.knoche@rwth-aachen.de

Florian Lemmerich RWTH Aachen University florian.lemmerich@cssh.rwth-aachen.de

ABSTRACT

With the increase of biased information available online, the im- portance of analysis and detection of such content has also signif- icantly risen. In this paper, we aim to quantify different kinds of social biases using word embeddings. Towards this goal we train such embeddings on two politically biased MediaWiki instances, namely RationalWiki and Conservapedia. Additionally we included Wikipedia as an online encyclopedia, which is accepted by the general public. Utilizing and combining state-of-the-art word em- bedding models with WEAT and WEFAT, we display to what extent biases exist in the above-mentioned corpora. By comparing embed- dings we observe interesting differences between different kinds of wikis.

CCS CONCEPTS

• Information systems →Web mining.

KEYWORDS

Bias, Conservapedia, Embeddings, RationalWiki, Stereotypes, WEAT

ACM Reference Format: Markus Knoche, Radomir Popović, Florian Lemmerich, and Markus Stroh- maier. 2019. Identifying Biases in Politically Biased Wikis through Word Embeddings. In 30th ACM Conference on Hypertext and Social Media (HT ’19), September 17–20, 2019, Hof, Germany. ACM, New York, NY, USA, 5 pages. https://doi.org/10.1145/3342220.3343658

1 INTRODUCTION

Cognitive biases can be defined as “cases in which human cognition reliably produces representations that are systematically distorted compared to some aspect of objective reality” [12]. An important subcategory of these are stereotypical biases, for example gender biases, racial biases, or religious biases. These biases often exist subconsciously, i.e., they influence people without them noticing

∗Both authors contributed equally to this research.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. HT ’19, September 17–20, 2019, Hof, Germany © 2019 Copyright is held by the owner/author(s). Publication rights licensed to ACM. ACM ISBN 978-1-4503-6885-8/19/09. . . $15.00 https://doi.org/10.1145/3342220.3343658

Radomir Popović∗

RWTH Aachen University radomir.popovic@rwth-aachen.de

Markus Strohmaier RWTH Aachen University & GESIS markus.strohmaier@cssh.rwth-aachen.de

which poses a threat to our society. Arguably, such effects might get reinforced by “filter bubbles” in online search results and social media. To obtain information online, wikis have established them- selves as a primary technology for the collaborative creation of contents. In this paper we tackle the wikis Conservapedia and Ra- tionalWiki which can be both considered to be biased. As its name suggest, the former one is converging towards a conservative point of view and audience, while the latter is more liberal. Thus, we ask in this on-going work how stereotypical biases with respect to gen- der, race, and religion differ in-between these ideologically diverse wikis and from the more mainstream shaped English Wikipedia. Approach. We firstly trained word embedding models on these corpora, which assign a high-dimensional vector to each word in some semantical structured way. The idea is that the corpora’s biases are also represented in the embeddings, which allows to measure and quantify them. This measurement is done using the Word Embedding Association Test (WEAT) by Caliskan et al. in [5] and a modification based on vectors proposed by ourselves. Contributions and Findings. Using this methods we found in- triguing cases of biases, via both WEAT and its modification, that surprisingly occur not only in Conservapedia and RationalWiki, but in Wikipedia as well. The results show gender related biases such as males being more associated with intellectual and career oriented terms, while females being related to appearance and fam- ily terms. We further quantified racial and religious biases, which are strongest in Conservapedia.

2 RELATED WORK

This section first summarizes existing literature on the analysis of biases in Wiki-based corpora – specifically Wikipedia – based on lexical approaches. Then, it provides an overview on research about bias detection in word embeddings.

2.1 Biases in Wikis

Recasens et al. trained a model which detects the word of a sentence which introduces the bias [16]. In order to do so they extracted those edits from Wikipedia where only one word was changed and where the editors mention the neutral point of view in the edit’s comment. Using this as the training data they build a regression model which gets as additional input several linguistic analysis results, for example whether the word is a weak subjective based on a list of weak subjectives. Their system achieves a result slightly lower compared with humans in a crowdworker-based study.

Wagner et al. tackled gender inequality in several Wikipedia language editions [17]. Among other approaches they compared how often words of several word sets are contained in articles about famous male and female persons. They found that articles about female persons contain gender-related words more often. Furthermore, they detected a larger amount of relationship- and family-related words in articles of women.

Graells-Garrido et al. came to similar results [9]: words related to arts, gender and family roles appear more prominent in articles about females, whereas articles about males are dominant regard- ing sport-related words. They further found a strong difference between words related to sexuality, which occur more often in female person’s articles, and words related to cognitive processes, which appear more often in article of men.

A political bias in Wikipedia was measured by Greenstein and Zhu [10]. They used phrases which allow to distinguish between democrats and republicans based on congressional records and counted these in Wikipedia articles. They found that especially articles from the years 2002 and 2003 use democrat’s phrases more often, but this trend decreases over time. Nevertheless these phrases still appeared more often in 2011 when their paper was published, especially in topics related to civil rights, government and social security.

2.2 Bias Analysis based on Word Embeddings

Word embeddings, i.e., the representation of individual words by a vector of real numbers, have been applied to improve a wide range of natural language processing tasks. Apart from classical tasks such as clustering or prediction, it has also been employed to measure biases in text corpora.

In that direction, Caliskan et al. reproduced the Implicit Associa- tion Test (IAT) by Greenwald et al. [11] with word embeddings [5]. They propose the Word Embedding Association Test (WEAT) and show that this test allows to measure the same biases as the IAT if it is trained on a large scale crawl of the internet with billions of words. We utilize their methods in this work in order to compare biases from different corpora, see Section 3.3.1 for more details.

Garg et al. measured bias using the relative norm difference [8]. For representative words of two opposite groups, for example asian and whites represented by common last names, they compute the mean vector. Then for each word of a neutral word set, for example different occupations, they measure the distances to both means, average them and take the difference. This way they get a single value which states how biased the neutral word list is between both groups.

They used this method to compare embeddings which are trained on data from different time periods which allows to analyze the development of bias over time. Their results coincide with real data: occupation bias between men and women or asians and whites changes similar to the proportion of women or asians in jobs. They further show that historical events change the biases in the embed- dings, for example the woman’s rights movement, the Immigration and Nationality Act or the September 11 attacks.

Bolukbasi et al. suggested to debias an embedding by removing the gender subspace of gender neutral words [4]. To identify this subspace they apply PCA to the vectors between word pairs like

“king” and “queen”. They further propose a method to measure

bias based on the cosine similarity between the gender subspace and gender neutral words. We decided to not use this method for measuring the bias, because the creation of the neutral words relies on a linear classifier which would propagate potential errors, but the idea of a gender subspace is related to our modification WEATvec of WEAT.

3 APPROACH

In this section, we describe the acquisition and pre-processing of the data employed in this work as well as the methods for computing word embeddings and determining biases in embeddings.

3.1 Dataset and Preprocessing

Two ideologically biased wiki systems are studied and compared with each other and with a sample of the well-known English Wikipedia. In order to do so we retrieved the full contents of Con- servapedia, and RationalWiki. Conservapedia1 was founded by An- drew Schlafly in November 2006. It is intended as a conservative alternative to Wikipedia, in which they see a “liberal bias”. In April 2019, it contained about 47,000 content pages with overall more than 700 million pageviews (according to their own statistics) and had approximately 120 active members. By contrast, RationalWiki2

was created as a reaction to Conservapedia in May 2007. They state as their goal to tackle pseudo- and anti-science as well as crank ideas, authoritarianism and fundamentalism. It had roughly 6,800 content pages and around 300 active registered users in April 2019. Its contents often include sarcasm and satire, posing special chal- lenges to statistical analysis. As a third dataset for comparison, we consider a sample of 50,000 randomly selected articles of the English Wikipedia3 as well.

For each Wiki, we collected the text content as HTML via the provided MediaWikis API. Then, we removed HTML tags and non- continuous text structures (e.g., tables, image captions, enumer- ations and references) to obtain the raw text for every article in the three wikis. To further prepare the data for embedding models, we tokenize the raw text using the Natural Language Toolkit4 and removed numbers, punctuation and other non-alphabetic charac- ters. Following suggestions by Mikolov et al. in [15], we remove all words that occur less than five times in a wiki.

3.2 Computing Word Embeddings

We measure biases in the investigated wikis using word embeddings. In general, word embeddings map each word contained in a corpus to a vector ®v ∈Rn. The application of word embeddings improved many natural language processing tasks in the last decade deci- sively, such as part-of-speech tagging [18], language modelling [2], or named entity recognition [13]. Due to its high applicability a wide range of diverse methods have been introduced to compute word embeddings for a given corpus. In this work, we decided to use FastText [3], a popular state-of-the-art method for this task, which extends word embeddings by including character n-grams.

1https://www.conservapedia.com 2https://rationalwiki.org 3https://en.wikipedia.org 4https://www.nltk.org/

However, we acknowledge that the application of other techniques could be regarded as equally valid. We used the implementation by Gensim5 with an exponential learning rate decay. We stopped training when the performance of the model did not improve for 5 subsequent epochs w.r.t. to the word analogy task described by Mikolov et al. [14]. The dimension of the embedding vectors and the exponential learning rate decay were set to 168 and 0.95 respec- tively since these parameters optimized the performance of this analogy task.

3.3 Measuring Bias in Embeddings

Word embeddings – even when trained from seemingly neutral text corpora – have been shown to inherently reflect biases contained in the underlying texts. As one example, embeddings trained from the Google News corpus has been shown to associate the male pronoun “he” more with professions such as “philosopher” and “architect”, the female pronoun “she” is more closely associated with “homemaker”, “nurse”, or “hairdresser” [4]. While those biases are

typically regarded as a problem for downstream tasks, we exploit the manifestation of biases in word embeddings to measure implicit associations in the underlying text corpora. In particular, we employ the Word Embedding Association Test (WEAT) as proposed by Caliskan et al. in [5] as well as a novel adaptation to it that we call WEATvec. We summarize both methods next.

3.3.1 WEAT and WEFAT. The Word Embedding Association Test (WEAT) uses four inputs: two sets of attribute words (A, B), e.g., sets of male and female names, and two equally sized sets of target words (X, Y), e.g., one related to family and one related to career. Our goal is then to measure if the relative association of the target sets to the attribute sets is the same. For that purpose, we compute a test statistic s that quantifies how much X is associated to A in comparison to the association between Y and B. It is defined as

s(X,Y,A, B) =

h(®x,A, B) −

h(®y,A, B),

®x ∈X

®y ∈Y

with h( ®w,A, B) = mean®a∈A cos( ®w, ®a) −mean®b ∈B cos( ®w, ®b). That is, we compute a value that describes the relation of ®w to the attribute sets. It ranges from −2 to 2, depending on whether ®w is more related to B or A respectively. These values are then aggregated over all vectors contained in X and Y. Consequently, the resulting value is larger the more X is related to A and Y is related to B. The statistical significance can then be assessed using a permutation test, see the original publication for details [5]. Following this work, we compute an effect size based on Cohen’s d [6] to measure how distinct the groups are, normalized with the population’s standard deviation:

mean®x ∈X h(®x,A, B) −mean®y ∈Y h(®y,A, B)

d(X,Y,A, B) =

stdw ∈W h( ®w,A, B)

For visualization purposes, we also use a variation of WEAT called Word Embedding Factual Association Test (WEFAT). It aims to calculate the relative association of a single target word (®v) to two sets of attribute words (A, B). In essence, it corresponds to h( ®w,A, B) normalized by the standard deviation. As before, we refer to [5] for more details.

5https://radimrehurek.com/gensim/

®a ®b

®x1 ®y1

®x2 ®y2

Figure 1: Comparison of WEAT and WEATvec. It is visible that the angle relations change which changes the result of WEAT, but the distance between the projected points stays the same, so WEATvec stays the same as well.

Figure 1: Comparison of WEAT and WEATvec. It is visible that the angle relations change which changes the result of WEAT, but the distance between the projected points stays the same, so WEATvec stays the same as well.

3.3.2 WEATvec. Traditionally, the relation between words is de- fined as the vector between the corresponding embeddings [14]. A potential issue with the WEAT method is that for two pairs of target vectors (®x1, ®y1) and (®x2, ®y2) with the same relation to a pair of attribute vectors (®a, ®b), WEAT might return different results for them depending on the absolute position of vectors, cf. Figure 1.

To counteract this phenomenon, we introduce a variation of WEAT that we call WEATvec. It tackles this issue by not measuring the difference in cosine similarities, but instead the differences in the projections of ®xi and ®yi to the vector from ®a to ®b. This makes the metric independent from the absolute position of ®x and ®y and only considers the relation between them, cf. [4] for a similar idea in a different setting. As a result, the definition of h in the formulas to compute the test statistic s and Cohen’s d is modified to:

mean®b ∈B ®b −mean®a∈A ®a mean®b ∈B ®b −mean®a∈A ®a

hvec( ®w,A, B) = ®w ·

2 Unfortunately, the stronger emphasis on the relation compared to the angle of embedding vectors also comes with a downside: two vectors of different lengths lying on the same direction, can output different scalar projection on some third vector, even though the angle (cosine similarity) is the same. Thus, we consider both varia- tions to capture slightly different viewpoints of measuring biases, i.e., relation-based vs. similarity-based that can be used alongside each other for a more complete picture. It turns out that in our setting both variations lead to similar results.

3.3.3 Selecting attribute and target sets. We combined and extended sets which were previously used to identify biases by Greenwald et al. (IAT)[11], Caliscan et al. [5], and Garg et al. [8]. The full list of attribute and target words is available online6, we just present a few examples here. The target sets male and female are created by com- bining a set of gendered words (“father”, “he”) with a set of given names (“kevin”). Sets for white and black consist of indicative given names (“katie”, “josh” vs. “shereen”, “jamal”). Highly diverse words are used for the attribute sets pleasant and unpleasant, ranging from “peace” over “rainbow” to “joy” or from “assault” over “vomit” to “agony”, respectively. Words like “math” and “einstein” are part of

science, art contains words like “poetry” and “symphony”. The sets career and family contain words like “corporation” and “salary” or “cousin” and “marriage”. Attribute sets for appearance and intellect

6https://github.com/MKnoche/wikibiasembedding

Table 1: Effect sizes of various WEATs and WEATvecs. A negative d corresponds to an inverted relation, for example Conser- vapedia associates science closer to atheism then to christianity with d = −2.060 according to WEATvec. Insignificant values (Bonferroni corrected p > 0.05) are shown in gray, highly significant (p < 0.001) values are bold.

attribute sets wiki male/female black/white christ./islam christ./atheism d dvec d dvec d dvec d dvec pleasant/ conservapedia 0.012 0.061 0.727 0.623 0.547 0.361 1.025 0.901 unpleasant rationalwiki −0.112 −0.131 0.257 0.236 0.271 0.220 0.321 0.345 wikipedia −0.114 0.014 0.305 0.211 0.273 0.179 0.419 0.352 science/ conservapedia 1.626 1.710 0.304 0.287 0.128 0.250 −2.123 −2.060 art rationalwiki 0.732 0.864 −0.516 −0.555 0.362 0.433 −1.837 −1.911 wikipedia 1.324 1.260 −0.043 0.152 −0.610 −0.460 −2.197 −2.293 intellect/ conservapedia 0.882 0.905 −0.282 −0.212 0.545 0.492 −0.338 −0.376 appearance rationalwiki 1.560 1.592 −0.150 −0.159 0.360 0.353 −1.258 −1.206 wikipedia 0.334 0.348 0.096 0.167 0.072 0.050 −1.219 −1.177 career/ conservapedia 2.420 2.284 −0.220 −0.263 −1.095 −0.814 −1.320 −0.923 family rationalwiki 1.734 1.710 −0.002 0.037 −0.848 −0.770 −1.291 −1.178 wikipedia 2.432 2.321 0.010 0.156 −0.199 −0.045 −1.087 −0.930

include “beautiful” and “ugly” vs. “inventive” and “foolish”. The tar- get sets islam and christianity use words like “allah” and “ramadan” or “jesus” and “salvation”. We defined the set atheism ourselves with words like “atheism”, “agnostic”, or “scepticism”.

3.3.4 Robustness of embeddings. An issue of word embeddings is their potential instability due to randomized starting conditions and instance selection steps in the calculation of the embeddings, cf. [1]. The resulting embeddings will exhibit certain variations even when applied to identical algorithms and datasets. To minimize the influence of random noise we trained eight embeddings for each dataset and computed effect size and statistical significance separately. We aggregated the p-values using Fisher’s method[7, p. 104f]. Effect sizes were averaged by their the mean value.

4 RESULTS

This section presents the outcome of our bias detection methods for RationalWiki, Conservapedia, and a sample of the English Wikipedia. Table 1 shows our main results, i.e., the effect sizes d for WEAT and WEATvec. Positive values indicate an associa- tion between the first words of the two pairs, e.g., pleasant/male. Highly significant results (using a Bonferroni correction with factor n = 192) are printed in bold, very significant ones in normal font. Gender Bias. All three wikis show significant biases with respect to gender, but with varying intensity. Wikipedia and Conserva- pedia show a strong association between male and science (and between female and art, resp.), while in RationalWiki the difference is smaller. With respect to intellect vs. appearance only Wikipedia does not show a clear distinction between male and female. Figure 2 visualizes these results in Wikipedia and the ideological Wikis us- ing the WEFAT. It shows a larger overlap in RationalWiki between science/art and a clear separation in the other two wikis. All wikis show a large spread of the appearance set, but in Conservapedia and RationalWiki the intellect-related words are closer to male.

The table further shows the association of gender with fam- ily/career. The former set is strongly associated with female, while career-related words are closer to male words. This bias is less pronounced for RationalWiki. There is, however, no clear bias of gendered words on a pleasant/unpleasant scale.

These results show nevertheless that classical gender stereotypes exist across all three datasets. Especially Conservapedia shows large gaps which is not surprising given its more traditional orientation. Racial Bias. The second column of Table 1 shows associations between white and black to the different attribute pairs in the three wikis. Nearly all values are insignificant, except that Conservapedia associates black people’s names closer to unpleasant words and white people’s names closer to pleasant words. Religious Bias. The last two columns shows the results of reli- gious bias tests. The set christianity is compared with both islam and atheism, again using all the other sets. In Conservapedia, islam is considered less related to pleasant compared with christianity. This association is present much weaker (and partially insignifi- cant) in the other two wikis. In comparison to science and art only Wikipedia shows significant results: islam is linked closer to sci- ence and christianity closer to art. Conservapedia is the only wiki showing a strong link between islam and appearance-related words and christianity and intellect-related words. Both Conservapedia and RationalWiki show highly significant results if compared to the sets career and family. This is most likely caused by a strong link between christianity and family-related words.

If atheism is considered, both Conservapedia and Wikipedia associate pleasant with christianity, possibly indicating a violation of Wikipedias neutral point of view policy. Nevertheless, the effect sizes in Conservapedia are more than twice as large. RationalWiki shows no significant results. We furthermore see strong associations of atheism with science, intellect and career while christianity is connected to art, appearance, and family. One reason for this is that the atheism set contains words like “darwin” and “university”. Summary. Overall, we observe similar biases in the three inves- tigated wikis, but with varying strengths. Regarding the religious bias Conservapedia stands out, which is plausible considering their conservative point of view. From a technical perspective, we see that both WEAT and WEATvec lead to similar results in all settings.

6A numerical scale is not shown because these values are a property introduced by the embedding algorithm and not something derived from the data. The important observation is how mixed the two sets are.

Figure 2: WEFAT values7 for the target set male and female. The horizontal position represents the value originating from Wikipedia, the vertical from Conservapedia or RationalWiki, respectively. Dots show the mean values of the sets. Additional histograms visualize the one dimensional distribution of WEFAT values. We observe a stronger relation of male to science and female to art on Wikipedia and Conservapedia, whereas RationalWiki has a larger overlap between both sets. By con- trast, Wikipedia shows little gender bias for intellect/appearance, but RationalWiki and Conservapedia both associate the set intellect closer to male. For the sake of readability some words have been replaced by small dots in (b).

Figure 2: WEFAT values7 for the target set male and female. The horizontal position represents the value originating from Wikipedia, the vertical from Conservapedia or RationalWiki, respectively. Dots show the mean values of the sets. Additional histograms visualize the one dimensional distribution of WEFAT values. We observe a stronger relation of male to science and female to art on Wikipedia and Conservapedia, whereas RationalWiki has a larger overlap between both sets. By con- trast, Wikipedia shows little gender bias for intellect/appearance, but RationalWiki and Conservapedia both associate the set intellect closer to male. For the sake of readability some words have been replaced by small dots in (b).

female Conservapedia male

female RationalWiki male

female Conservapedia male

numbers

science

computation

calculus

science

computation

addition

equations

math

addition

art

algebra geometry calculus equations

literature novel

art

sculpture

numbers

algebra geometry

symphony

symphony drama

literature

sculpture shakespeare

dance

attractive

shakespeare

poetry

novel

dance

poetry

drama

female Wikipedia male

(a) Association of the sets art and science.

5 SUMMARY AND CONCLUSION

This paper describes ongoing work on analyzing ideologically bi- ased communities. We specifically investigated gender, race and religious biases in two agenda-driven wiki systems (RationalWiki and Conservapedia) and compared them to the more mainstream Wikipedia with approaches for bias detection in word embeddings. With our approach, we could find that similar biases are present in all three wikis, but with varying degree. For example, women are more associated with words related to art, appearance and family whereas men are closer to words related to science, intellect and career. Furthermore we found biases in Conservapedia viewing Christianity in a more positive light compared to Islam or Atheism. Our results show that it is in general possible to find and measure bi- ases using embeddings, even if these are only based on comparably small amounts of data.

At its current stage, we see several limitations of our work, which we will approach in the future. First, we did not include the full Wikipedia, but relied on a random sample. Next, the trained em- beddings are influenced by complex semantical structures, such as satire and sarcasm which is found often especially in Rational- Wiki, posing challenges to our applied methodology. Furthermore, a more in-depth theoretical analysis of the proposed WEATvec method should be performed in order to find practical differences with the established WEAT approach. Finally, we acknowledge that this work focuses on two more niche communities on the Web. In that respect, it will be interesting to study further communities not in-line with mainstream political orientations.

REFERENCES

[1] Maria Antoniak and David Mimno. 2018. Evaluating the stability of embedding-

based word similarities. Transactions of the Association of Computational Linguis- tics 6 (2018), 107–119. [2] Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Jauvin. 2003. A

neural probabilistic language model. Journal of machine learning research 3, Feb (2003), 1137–1155. [3] Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. En-

riching Word Vectors with Subword Information. Transactions of the Association

thoughtful

female Conservapedia male

female RationalWiki male

weak

logical

smart

discerning

strong

dull

dumb idiotic foolish

feeble

thin

. venerable .

judicious

handsome

logical handsome

dull apt

foolish

fashionable

sage adaptable naive

unwise

feeble

athletic

fashionable

.unwise discerning

apt reflective

naive

strong

bald

smart

. fat

bald

thoughtful

idiotic

sage adaptable

slender

weak

reflective

venerable

dumb

thin

healthy

slender

fat ugly

ugly

athletic

gorgeous

healthy

attractive

gorgeous

sensual

beautiful

attractive

female Wikipedia male

(b) Association of the sets appearance and intellect.

for Computational Linguistics 5 (2017), 135–146. [4] Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, and Adam

Kalai. 2016. Man is to Computer Programmer As Woman is to Homemaker? De- biasing Word Embeddings. In Advances in Neural Information Processing Systems, Vol. 29. 4356–4364. [5] Aylin Caliskan, Joanna J. Bryson, and Arvind Narayanan. 2017. Semantics derived

automatically from language corpora contain human-like biases. Science 356, 6334 (2017), 183–186. [6] Jacob Cohen. 2001. Statistical Power Analysis for the Behavioral Sciences (2nd ed.).

Lawrence Erlbaum Associates. [7] Ronald A. Fisher. 1934. Statistical Methods for research Workers (5th ed.). Oliver

& Boyd. [8] Nikhil Garg, Londa Schiebinger, Dan Jurafsky, and James Zou. 2018. Word

embeddings quantify 100 years of gender and ethnic stereotypes. Proceedings of the National Academy of Sciences 115, 16 (2018), E3635–E3644. [9] Eduardo Graells-Garrido, Mounia Lalmas, and Filippo Menczer. 2015. First women,

second sex: Gender bias in Wikipedia. In Proceedings of the 26th ACM Conference on Hypertext & Social Media. 165–174. [10] Shane Greenstein and Feng Zhu. 2012. Is Wikipedia Biased? American Economic

Review 102, 3 (2012), 343–48. [11] Anthony G. Greenwald, Debbie E. McGhee, and Jordan L. K. Schwartz. 1998.

Measuring individual differences in implicit cognition: The implicit association test. Journal of Personality and Social Psychology 74, 6 (1998), 1464–1480. [12] Martie G. Haselton, Daniel Nettle, and Damian R. Murray. 2015. The Evolution of

Cognitive Bias. John Wiley & Sons, Inc, Chapter 41, 968–987. [13] Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami,

and Chris Dyer. 2016. Neural architectures for named entity recognition. arXiv:1603.01360 (2016). [14] Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient

estimation of word representations in vector space. (2013). arXiv:1301.3781 [15] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S. Corrado, and Jeff Dean. 2013.

Distributed Representations of Words and Phrases and their Compositionality. In Advances in Neural Information Processing Systems, Vol. 26. 3111–3119. [16] Marta Recasens, Cristian Danescu-Niculescu-Mizil, and Dan Jurafsky. 2013. Lin-

guistic Models for Analyzing and Detecting Biased Language. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, Vol. 1. Association for Computational Linguistics, 1650–1659. [17] Claudia Wagner, David Garcia, Mohsen Jadidi, and Markus Strohmaier. 2015. It’s

a man’s Wikipedia? Assessing gender inequality in an online encyclopedia. In Ninth International AAAI conference on Web and Social Media. [18] Peilu Wang, Yao Qian, Frank K Soong, Lei He, and Hai Zhao. 2015. A unified tag-

ging solution: Bidirectional lstm recurrent neural network with word embedding. arXiv:1511.00215 (2015).

Rendered source tables

Table 1 rendered from the source PDF