Our article “Discovering and Categorising Language Biases in Reddit” was accepted last week at the International Conference on Web and Social Media 2021 (ICWSM 2021). Although the proceedings will not be ready until early 2021, you can find the author’s version of the paper here.
We present a method to explore language bias in various Reddit communities by comparing the words most closely correlated with different concepts leveraging the embedding space of each community. In this way, we study gender bias in r/TheRedPill, religion bias in r/Atheism, and ethnicity bias in r/The_Donald, and discover important biases that, for instance in r/TheRedPill, picture women with words related to externality and physical appearance such as flirtatious and fuckable, and men through descriptive adjectives serving as indicators of subjectivity such as visionary and tactician (see our Bias Visualisation Tool).
Also, in case you are interested in analysing language biases in your own datasets, we are also sharing the code here. Please use it and feel free to ask any questions or report any errors/problems you find on Github!
Abstract. We present a data-driven approach using word embeddings to discover and categorise language biases on the discussion platform Reddit. As spaces for isolated user communities, platforms such as Reddit are increasingly connected to issues of racism, sexism and other forms of discrimination. Hence, there is a need to monitor the language of these groups. One of the most promising AI approaches to trace linguistic biases in large textual datasets involves word embeddings, which transform text into high-dimensional dense vectors and capture semantic relations between words. Yet, previous studies require predefined sets of potential biases to study, e.g., whether gender is more or less associated with particular types of jobs. This makes these approaches unfit to deal with smaller and community-centric datasets such as those on Reddit, which contain smaller vocabularies and slang, as well as biases that may be particular to that community. This paper proposes a data-driven approach to automatically discover language biases encoded in the vocabulary of online discourse communities on Reddit. In our approach, protected attributes are connected to evaluative words found in the data, which are then categorised through a semantic analysis system. We verify the effectiveness of our method by comparing the biases we discover in the Google News dataset with those found in previous literature. We then successfully discover gender bias, religion bias, and ethnic bias in different Reddit communities. We conclude by discussing potential application scenarios and limitations of this data-driven bias discovery method.