`wefe.debias.double_hard_debias`.DoubleHardDebias

class wefe.debias.double_hard_debias.DoubleHardDebias(pca_args: Dict[str, Any] = {'n_components': 10}, verbose: bool = False, criterion_name: Optional[str] = None, incremental_pca: bool = True, n_words: int = 1000, n_components: int = 4)[source]

Double Hard Debias Method.

This method allow reducing the bias of an embedding model through geometric operations between embeddings. This method is binary because it only allows 2 classes of the same bias criterion, such as male or female.

The main idea of this method is:

1. Identify a bias subspace through the defining sets. In the case of gender, these could be e.g. [['woman', 'man'], ['she', 'he'], ...]

2. Find the dominant directions of the entire set of vectors by doing a Principal components analysis over it.

3. Get the target words by finding the most biased words, this is the words tha are closest to the representation of each bias group. In case of gender 'he' and 'she'.

3. Try removing each component resulting of PCA and remove also the bias direction to every vector in the target set and find which component reduces bias the most.

4. Remove the dominant direction that most reduces bias and remove also the bias direction of the vectors in the target set.

References

[1]: Wang, Tianlu, Xi Victoria Lin, Nazneen Fatema Rajani, Bryan McCann, Vicente Or-donez y Caiming Xiong.

Double-Hard Debias: Tailoring Word Embeddings for GenderBias Mitigation.

CoRR, abs/2005.00965, 2020.https://arxiv.org/abs/2005.00965.

[2]: https://github.com/uvavision/Double-Hard-Debias

Examples

The following example shows how to execute Double Hard Debias method that reduces bias in a word embedding model:

>>> from wefe.debias.double_hard_debias import DoubleHardDebias
>>> from wefe.utils import load_test_model
>>> from wefe.datasets import fetch_debiaswe
>>>
>>> # load the model (in this case, the test model included in wefe)
>>> model = load_test_model()
>>> # load definitional pairs, in this case definitinal pairs included in wefe
>>> debiaswe_wordsets = fetch_debiaswe()
>>> definitional_pairs = debiaswe_wordsets["definitional_pairs"]
>>>
>>> # instance and fit the method including bias representation words,
>>> # in case of gender definitional_pairs=[['he', 'she'], ...]
>>> dhd = DoubleHardDebias().fit(
...     model=model,
...     definitional_pairs=definitional_pairs,
...     bias_representation=['he','she'])
>>> # execute the debias, if you don't want a set of words to be debiased
>>> # include them in the ignore set.
>>> gender_specific = debiaswe_wordsets["gender_specific"]
>>>
>>> debiased_model = dhd.transform(
...     model=model, ignore=gender_specific
... )
Copy argument is True. Transform will attempt to create a copy of the original
model. This may fail due to lack of memory.
Model copy created successfully.

If you want the debiased to be performed over a specific set of words you can specify them in the target parameter

>>> debiased_model = dhd.transform(
... model=model, target = ['doctor','nurse','programmer','teacher']
... )
Copy argument is True. Transform will attempt to create a copy of the original
model. This may fail due to lack of memory.
Model copy created successfully.

__init__(pca_args: Dict[str, Any] = {'n_components': 10}, verbose: bool = False, criterion_name: Optional[str] = None, incremental_pca: bool = True, n_words: int = 1000, n_components: int = 4) → None[source]

Initialize a Double Hard Debias instance.

Parameters:

pca_argsDict[str, Any], optional: Arguments for the PCA that is calculated internally in the identification of the bias subspace, by default {“n_components”: 10}
verbosebool, optional: True will print informative messages about the debiasing process, by default False.
criterion_nameOptional[str], optional: The name of the criterion for which the debias is being executed, e.g., ‘Gender’. This will indicate the name of the model returning transform, by default None
incremental_pca: bool, optional: If True, incremental pca will be used over the entire set of vectors. If False, pca will be used over the entire set of vectors. WARNING: Running pca over the entire set of vectors may raise to MemoryError, by default True.
n_words: int, optional: Number of target words to be used for each bias group. By default 1000
n_components: int, optional: Numbers of components of PCA to be used to explore the one that reduces bias the most. Usually the best one is close to embedding dimension/100. By default 4.

fit(model: WordEmbeddingModel, definitional_pairs: List[List[str]], bias_representation: List[str], **fit_params) → BaseDebias[source]

Compute the bias direction and get the principal components of the vectors.

Parameters:

modelWordEmbeddingModel: The word embedding model to debias.
definitional_pairsList[List[str]]: A sequence of string pairs that will be used to define the bias direction. For example, for the case of gender debias, this list could be [[‘woman’, ‘man’], [‘girl’, ‘boy’], [‘she’, ‘he’], [‘mother’, ‘father’], …].
bias_representation: List[str]: Two words that represents each bias group. In case of gender “he” and “she”.

Returns:

BaseDebias: The debias method fitted.

get_target_words(model: WordEmbeddingModel, target: List[str], ignore: List[str], n_words: int, bias_representation: List[str]) → List[str][source]

Obtain target words to be debiased.

This is done by searching the “n_words” most biased words by obtaining the words closest to each word in the bias_representation set (in case of gender “he” and “she”).

Parameters:

model: WordEmbeddingModel: The word embedding model to debias.
ignore: List[str]: Set of words to be ignored from the debias process.
n_words: int: number of target words to use.
bias_representation: List[str]: Two words that represents de bias groups.

Returns:

List[str]: List of target words for each bias group

transform(model: WordEmbeddingModel, target: Optional[List[str]] = None, ignore: Optional[List[str]] = [], copy: bool = True) → WordEmbeddingModel[source]

Execute hard debias over the provided model.

Parameters:

modelWordEmbeddingModel: The word embedding model to debias.
ignoreList[str], optional: If set of words is specified in ignore, the debias method will perform the debias in all target words except those specified in this set, by default [].
copybool, optional: If True, the debias will be performed on a copy of the model. If False, the debias will be applied on the same model delivered, causing its vectors to mutate. WARNING: Setting copy with True requires RAM at least 2x of the size of the model, otherwise the execution of the debias may raise to MemoryError, by default True.

Returns:

WordEmbeddingModel: The debiased embedding model.

wefe.debias.double_hard_debias.DoubleHardDebias

`wefe.debias.double_hard_debias`.DoubleHardDebias