Machine translation (MT) has realized huge improvements in quality from the successful application and development of neural methods Kalchbrenner and Blunsom (2013); Cho et al. (2014); Vaswani et al. (2017); Johnson et al. (2017); Chen et al. (2018). As the community has explored this enhanced performance, many have noted the outstanding challenge of modeling and handling gendered language Kuczmarski (2018); Escudé Font and Costa-jussà (2019). Where previous work identifies issues using manually-curated synthetic examples in a target language Stanovsky et al. (2019); Cho et al. (2019), we expand the scope to analyze real world text across a range of languages to understand challenges for deployed systems.
In this paper, we explore the class of issues which surface when a neutral reference to a person is translated to a gendered form (e.g. in Table 1, where the English counselor and nurse are translated into the masculine conseiller and feminine infirmiére in French). Despite the English being unspecified for gender, the current MT task definition requires a system to produce a single translation, thus exposing a model’s preferred gender for the reference form. That is, our examples form an analysis benchmark that detects gendered representations of personal references in MT models.
With this scope, we make two key contributions. First, we design and implement an automatic pipeline for detecting examples of our class of gender issues in real world input, using a BERT-based perturbation method novel to this work. A key advantage of our pipeline beyond previous work is its extensibility: a) beyond word lists; b) to different language pairs and c) parts of speech. Second, using our new pipeline, we compile a dataset that we will make publicly available to serve as a benchmark for future work. We focus on English as the source language, and explore four target gendered languages across three language families (French, German, Spanish, and Russian).
Source Sentence (En)
so is that going to affect my chances of becoming a counselor?
|Alors, est - ce que cela va affecter mes chances de devenir conseiller?||M|
so is that going to affect my chances of becoming a nurse?
|Alors, est - ce que cela va affecter mes chances de devenir infirmiére?||F|
2 Gender Marking Languages
Gender-marking languages have rich grammatical systems for expressing gender Corbett (1991). To produce a valid sentence in a gender-marking language, gender may need to be marked not only on pronouns (he, she), as it is in English, but also nouns and even verbs, as well as words linked to these gendered nouns and verbs, including determiners and adjectives. This means that translating from a language like English, with little gender marking, to a gender-marking language like Spanish, requires a system to produce gender markings that may not have explicit evidence in the source. For instance, The tall teacher from English could be translated into the Spanish La maestra alta (feminine) or El maestro alto (masculine).
3 Automatic Detection of Gender Issues
The class of issues we are interested in are when translation to a gender-marking language exposes a model’s preference that certain personal references have particular genders. The examples we find that demonstrate this are English sentence-pairs, a minimal pair differing by only a single word, e.g. doctor being replaced by nurse. In each of our examples, this minimal perturbation does not change the gender of the source but gives rise to gender differences upon translation, e.g. doctor becoming masculine and nurse feminine.
In this section, we present a simple, extensible method to mine such examples from real-world text. Our method does not require expensive manually-curated word lists for each target language, which enables us to discover new kinds of entities that are susceptible to model bias but are not usually thought of this way. Indeed, while we demonstrate its utility with nouns with four target languages, our method is naturally extensible to new language pairs and parts of speech with no change in design.
Filtering source sentences
Our first step is to identify sentences that are gender neutral and that include a single human entity, e.g. A doctor works in a hospital. We focus on human entities since these have been the target of previous studies and present the largest risk of gender issues in translation.
We use a BERT-based Named Entity Recognition (NER) model that identifies human entities, and exclude sentences that have more than one token tagged as such. We also remove sentences in which the entity is a gendered term in English111https://github.com/tolga-b/debiaswe/blob/master/data/gender_specific_full.json
https://github.com/uclanlp/gn_glove/tree/master/wordlist (e.g. mother, nephew), a name, or not a noun.
Note that all the sentences we get are naturally occurring sentences, and that we do not use any templates or predefined lists of target words that we want to handle.
Perturbations using BERT
We use BERT as a masked language model to find words which can substitute for the human entity identified in the previous filtering step, e.g. doctor nurse. We do not use predefined substitutions to get natural-sounding output and maintain extensibility. We cap our search to the first 100 candidates BERT returns, accepting the first 10 which are tagged as person.
We translate each of the generated sentences into our target languages using an in-house tool. A doctor/nurse works in a hospital Un doctor/Una enfermera trabaja en un hospital.
We align tokens in the original and translated sentences using fast-align Dyer et al. (2010). This is needed in order to know which token in the translation output is the focus entity in the source sentence, whose gender we want to analyze.
We use a morphological analyzer, implemented following Kong et al. (2017), to tag the gender of the target word.
The final step of our pipeline is identifying pairs of sentences to include in our data set, pairs where different genders are assigned to the human entity. Our example would be included since doctor is translated with the masculine form Un doctor while nurse is translated with the feminine Una enfermera.
4 Challenge Dataset
We compile our final dataset from the output of this pipeline, and explore its properties to understand the issues it represents for deployed systems.
4.1 Random Sampling
In our final dataset, we include both examples that passed the final example identification step above (at risk), as well as a random selection that did not (not at risk). We do this in order to not be constrained too heavily by our choice of translation model; if we did not, we would have no chance of inspecting examples that our system did not spot as at risk but other models might have.
|Source Sentence (En)||Translation||M/F|
|Fr||also should i ask the manager what the pay would be if i got the job prior to flying out?||De plus, devrais - je demander au gestionnaire quel serait le salaire si je obtenais le poste avant de prendre l’avion?||M|
|also should i ask the secretary what the pay would be if i got the job prior to flying out?||De plus, devrais - je demander à la secrétaire quel serait le salaire si je obtenais le poste avant mon départ?||F|
|De||currently thinking about learning a trade (mostly a electrician).||Derzeit über das Erlernen eines Gewerbes nachdenken (meistens Elektriker).||M|
|currently thinking about learning a trade (mostly a cook).||Derzeit über das Erlernen eines Gewerbes nachdenken (meistens eine Köchin).||F|
|Es||- decided to become a teacher: spent a year working 2 jobs and doing prerequisites for a masters in education.||- Decidí ser maestra: pasé un año trabajando en 2 trabajos y haciendo requisitos previos para una maestría en educación.||F|
|- decided to become a lecturer : spent a year working 2 jobs and doing prerequisites for a masters in education.||- Decidí ser profesor: pasé un año trabajando en 2 trabajos y haciendo requisitos previos para una maestría en educación.||M|
|Ru||i read about a psychologist who upgraded into becoming a m.d.||Я читал о психологе, который превратился в Md.||M|
|i read about a nurse who upgraded into becoming a m.d.||Я читал о медсестре, которая превратилась в доктора медицины.||F|
4.2 Fixed Grammatical Gender Rating
When we inspected the examples identified as at risk by our pipeline, the major source of error we found pertained to the issue of fixed grammatical gender. Consider the example in Figure 1:
In this example, the word victim in the first English sentence is identified by our tagger as a human entity. However, its French translation victime is feminine by definition, and cannot be assigned another gender regardless the context, causing a false positive result.
We attempted to filter these examples automatically but came across a number of challenges. Most critically, we found no high-quality, comprehensive dictionary that included the required information for all languages, and heuristics we applied were noisy and not reliable. We observed that the underlying reason for these challenges was that there is no closed list of grammatically-fixed words as languages are evolving to be more gender-inclusive. In order to maximize and guarantee data quality, and to be sensitive to the nuances of language change, we decided to add a manual filtering step after our pipeline to select the positive (at risk) examples.
We note that the problem of fixed grammatical gender is particular to nouns. Our pipeline is naturally extensible across parts of speech and we would not expect the same issues in future work perturbing adjectives or verbs.
4.3 Dataset Statistics
To create our dataset we mine text from the subreddit ‘‘career’’.222https://www.reddit.com/r/Career/ From 29,330 sentences, we found 4,016 which referred to a single, non-gendered human entity. Introducing perturbations with BERT into these 4,016 sentences yielded 40,160 pairs. Out of those, 592 to 1,012 pairs are identified as at risk by our pipeline, depending on the target language. We asked humans to manually identify 100 true at risk examples for the final dataset, which was achieved for all languages except Russian, where we have 59 pairs. To this 100, we add a further 100 randomly sampled negative examples for each language. Table 2 shows a representative example for each language-pair.
4.4 Exploratory Analysis
Table 3 lists the most frequent focus personal reference in each language-pair among the positive (at risk) and negative (not at risk) examples, along with the ratio between times the reference form was translated as masculine compared to feminine. Words with extreme values of this ratio indicate cases where a model has a systematic preference for one gender over another, i.e. a gendered representation.
Among the negative examples, we see a prior for masculine translations across all terms. Positive examples break from this prior by exposing reference forms with a feminine preference: nurse and secretary are the most consistently feminine forms, consistent with the Bureau of Labor statistics used in previous work Caliskan et al. (2017).
5 Related Work
Our study builds on the literature around gender bias in machine translation. Cho et al. (2019) use sentence templates to probe for differences in Korean pronouns. Prates et al. (2019) and Stanovsky et al. (2019) also use sentence templates, but filled with word lists, of professions and adjectives in the former, and professions in the latter. A separate but related line of work focuses on generating correct inflections when translating to gender-marking languages Vanmassenhove et al. (2018); Moryossef et al. (2019).
The primary contribution of our work is a novel, automatic method for identifying gender issues in machine translation. By performing BERT-based perturbations on naturally-occurring sentences, we are able to identify sentence pairs that behave differently upon translation to gender-marking languages. We demonstrate our technique over human reference forms and discover new sources of risk beyond the word lists used previously. Furthermore, the novelty of our approach is its natural extensibility to new language pairs, text genres, and different parts of speech. We look forward to future work exploring such applications.
Using our new method, we compile a dataset across four languages from three language families. By publicly releasing our dataset, we hope to enable the community to work together towards solutions that are inclusive and equitable to all.
We thank Melvin Johnson for his helpful feedback throughout this project, Dan Garrette for helping with some parts of the pipeline, and Dani Mitropolsky, Vitaly Nikolaev and Marisa Rossmann for the help with filtering the dataset.
- Caliskan et al. (2017) Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183–186.
- Chen et al. (2018) Mia Xu Chen, Orhan Firat, Ankur Bapna, Melvin Johnson, Wolfgang Macherey, George Foster, Llion Jones, Mike Schuster, Noam Shazeer, Niki Parmar, Ashish Vaswani, Jakob Uszkoreit, Lukasz Kaiser, Zhifeng Chen, Yonghui Wu, and Macduff Hughes. 2018. The best of both worlds: Combining recent advances in neural machine translation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 76–86, Melbourne, Australia. Association for Computational Linguistics.
Cho et al. (2014)
Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau,
Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014.
representations using RNN encoder–decoder for statistical machine
Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1724–1734, Doha, Qatar. Association for Computational Linguistics.
- Cho et al. (2019) Won Ik Cho, Ji Won Kim, Seok Min Kim, and Nam Soo Kim. 2019. On measuring gender bias in translation of gender-neutral pronouns. In Proceedings of the First Workshop on Gender Bias in Natural Language Processing.
- Corbett (1991) Greville G Corbett. 1991. Gender.
- Dyer et al. (2010) Chris Dyer, Adam Lopez, Juri Ganitkevitch, Jonathan Weese, Ferhan Ture, Phil Blunsom, Hendra Setiawan, Vladimir Eidelman, and Philip Resnik. 2010. cdec: A decoder, alignment, and learning framework for finite-state and context-free translation models. In Proceedings of the ACL 2010 System Demonstrations.
Escudé Font and Costa-jussà (2019)
Joel Escudé Font and Marta R. Costa-jussà. 2019.
Equalizing gender bias in neural machine translation with word embeddings techniques.In Proceedings of the First Workshop on Gender Bias in Natural Language Processing.
- Johnson et al. (2017) Melvin Johnson, Mike Schuster, Quoc V. Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Viégas, Martin Wattenberg, Greg Corrado, Macduff Hughes, and Jeffrey Dean. 2017. Google’s multilingual neural machine translation system: Enabling zero-shot translation. Transactions of the Association for Computational Linguistics, 5:339–351.
- Kalchbrenner and Blunsom (2013) Nal Kalchbrenner and Phil Blunsom. 2013. Recurrent continuous translation models. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1700–1709, Seattle, Washington, USA. Association for Computational Linguistics.
- Kong et al. (2017) Lingpeng Kong, Chris Alberti, Daniel Andor, Ivan Bogatyy, and David Weiss. 2017. Dragnn: A transition-based framework for dynamically connected neural networks. arXiv preprint arXiv:1703.04474.
- Kuczmarski (2018) James Kuczmarski. 2018. Reducing gender bias in google translate.
- Moryossef et al. (2019) Amit Moryossef, Roee Aharoni, and Yoav Goldberg. 2019. Filling gender & number gaps in neural machine translation with black-box context injection. In Proceedings of the First Workshop on Gender Bias in Natural Language Processing.
- Prates et al. (2019) Marcelo OR Prates, Pedro H Avelar, and Luís C Lamb. 2019. Assessing gender bias in machine translation: a case study with google translate. Neural Computing and Applications.
- Stanovsky et al. (2019) Gabriel Stanovsky, Noah A. Smith, and Luke Zettlemoyer. 2019. Evaluating gender bias in machine translation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.
- Vanmassenhove et al. (2018) Eva Vanmassenhove, Christian Hardmeier, and Andy Way. 2018. Getting gender right in neural machine translation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.
- Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30, pages 5998–6008. Curran Associates, Inc.