wefe.HardDebias

class wefe.HardDebias(pca_args: Dict[str, Any] = {'n_components': 10}, verbose: bool = False, criterion_name: Optional[str] = None)

Hard Debias debiasing method.

This method allow reducing the bias of an embedding model through geometric operations between embeddings. This method is binary because it only allows 2 classes of the same bias criterion, such as male or female. For a multiclass debias (such as for Latinos, Asians and Whites), it is recommended to visit MulticlassHardDebias class.

The main idea of this method is:

1. Identify a bias subspace through the defining sets. In the case of gender, these could be e.g. {‘woman’, ‘man’}, {‘she’, ‘he’}, …

2. Neutralize the bias subspace of embeddings that should not be biased. First, it is defined a set of words that are correct to be related to the bias criterion: the criterion specific gender words. For example, in the case of gender, gender specific words are: {‘he’, ‘his’, ‘He’, ‘her’, ‘she’, ‘him’, ‘him’, ‘She’, ‘man’, ‘women’, ‘men’…}.

Then, it is defined that all words outside this set should have no relation to the bias criterion and thus have the possibility of being biased. (e.g. for the case of gender: {doctor, nurse, …}). Therefore, this set of words is neutralized with respect to the bias subspace found in the previous step.

The neutralization is carried out under the following operation:

• u : embedding

• v : bias direction

First calculate the projection of the embedding on the bias subspace. - bias_subspace = v • (v • u) / (v • v)

Then subtract the projection from the embedding. - u’ = u - bias_subspace

3. Equalizate the embeddings with respect to the bias direction.. Given an equalization set (set of word pairs such as [she, he], [men, women], …, but not limited to the definitional set) this step executes, for each pair, an equalization with respect to the bias direction. That is, it takes both embeddings of the pair and distributes them at the same distance from the bias direction, such that neither is closer to the bias direction than the other.

References

[1]: Bolukbasi, T., Chang, K. W., Zou, J. Y., Saligrama, V., & Kalai, A. T. (2016).

Man is to computer programmer as woman is to homemaker? debiasing word embeddings.

Advances in Neural Information Processing Systems.

[2]: https://github.com/tolga-b/debiaswe

__init__(pca_args: Dict[str, Any] = {'n_components': 10}, verbose: bool = False, criterion_name: Optional[str] = None) → None

Initialize a Hard Debias instance.

Parameters

pca_argsDict[str, Any], optional

Arguments for the PCA that is calculated internally in the identification of the bias subspace, by default {“n_components”: 10}

verbosebool, optional

True will print informative messages about the debiasing process, by default False.

criterion_nameOptional[str], optional

The name of the criterion for which the debias is being executed, e.g., ‘Gender’. This will indicate the name of the model returning transform, by default None

fit(model: wefe.word_embedding_model.WordEmbeddingModel, definitional_pairs: Sequence[Sequence[str]], equalize_pairs: Optional[Sequence[Sequence[str]]] = None, **fit_params) → wefe.debias.base_debias.BaseDebias

Compute the bias direction and obtains the equalize embedding pairs.

Parameters

modelWordEmbeddingModel

The word embedding model to debias.

definitional_pairsSequence[Sequence[str]]

A sequence of string pairs that will be used to define the bias direction. For example, for the case of gender debias, this list could be [[‘woman’, ‘man’], [‘girl’, ‘boy’], [‘she’, ‘he’], [‘mother’, ‘father’], …].

equalize_pairsOptional[Sequence[Sequence[str]]], optional

A list with pairs of strings, which will be equalized. In the case of passing None, the equalization will be done over the word pairs passed in definitional_pairs, by default None.

Returns

BaseDebias

The debias method fitted.

transform(model: wefe.word_embedding_model.WordEmbeddingModel, target: Optional[List[str]] = None, ignore: Optional[List[str]] = None, copy: bool = True) → wefe.word_embedding_model.WordEmbeddingModel

Execute hard debias over the provided model.

Parameters

modelWordEmbeddingModel

The word embedding model to debias.

targetOptional[List[str]], optional

If a set of words is specified in target, the debias method will be performed only on the word embeddings of this set. If None is provided, the debias will be performed on all words (except those specified in ignore). by default None.

ignoreOptional[List[str]], optional

If target is None and a set of words is specified in ignore, the debias method will perform the debias in all words except those specified in this set, by default None.

copybool, optional

If True, the debias will be performed on a copy of the model. If False, the debias will be applied on the same model delivered, causing its vectors to mutate. WARNING: Setting copy with True requires RAM at least 2x of the size of the model, otherwise the execution of the debias may give rise to MemoryError, by default True.

Returns

WordEmbeddingModel

The debiased embedding model.