wefe.debias.half_sibling_regression.HalfSiblingRegression

class wefe.debias.half_sibling_regression.HalfSiblingRegression(verbose: bool = False, criterion_name: Optional[str] = None)

Half Sibling Debias method.

This method proposes to learn spurious gender information via causal inference by utilizing the statistical dependency between gender-biased word vectors and gender definition word vectors. The learned spurious gender information is then subtracted from the gender-biased word vectors to achieve gender-debiasing as the following where \(V_n\) are the debiased word vectors, Vn are non gender definition and \(G\) is the approximated gender information:

\[V_n' := V_n - G\]

G is obtained by predicting Non gender definition word vectors (\(V_n\)) using the gender-definition word vectors (\(V_d\)):

\[G := E[V_n|V_d]\]

The Prediction is done by a Ridge Regression following the next steps:

1. Compute the weight matrix of a Ridge Regression using two sets of words

\[W = ((V_d)^T V_d + \alpha I)^{-1} (V_d)^TV_n\]

2. Compute the gender information:

\[G = V_d W\]

3. Subtract gender information from non gender definition words:

\[V_n' = V_n - G\]

This method is binary because it only allows 2 classes of the same bias criterion, such as male or female. For a multiclass debias (such as for Latinos, Asians and Whites), it is recommended to visit MulticlassHardDebias class.

Warning

This method requires three times the memory of the model when a copy of the model is made and two times the memory of the model if not. Make sure this much memory is available.

References

[1]: Yang, Zekun y Juan Feng: A causal inference method for reducing

gender bias in word embedding relations.

In Proceedings of the AAAI Conference on Artificial Intelligence, volumen 34, pages 9434–9441, 2020

[2]: https://github.com/KunkunYang/GenderBiasHSR

[3]: Bernhard Sch ̈olkopf, David W. Hogg, Dun Wang,

Daniel Foreman-Mackey, Dominik Jan-zing, Carl-Johann Simon-Gabriel, and Jonas Peters.

Modeling confounding by half-sibling regression.

Proceedings of the National Academy of Sciences, 113(27):7391–7398, 2016

Examples

The following example shows how to execute Half Sibling Regression Debias method that reduces bias in a word embedding model:

>>> from wefe.debias.half_sibling_regression import HalfSiblingRegression
>>> from wefe.utils import load_test_model
>>> from wefe.datasets import fetch_debiaswe
>>>
>>> # load the model (in this case, the test model included in wefe)
>>> model = load_test_model()
>>> # load gender specific words, in this case the ones included in wefe
>>> debiaswe_wordsets = fetch_debiaswe()
>>> gender_specific = debiaswe_wordsets["gender_specific"]
>>>
>>> # instance and fit the method
>>> hsr = HalfSiblingRegression().fit(
...     model=model, definitional_words=gender_specific
... )
>>> # execute the debias on the words not included in the gender definition set
>>> debiased_model = hsr.transform(model = model)
Copy argument is True. Transform will attempt to create a copy of the original
model. This may fail due to lack of memory.
Model copy created successfully.
>>>
>>>
>>> # if you want the debias over a specific set of words  you can
>>> #include them in the target parameter
>>> debiased_model = hsr.transform(
...     model=model, target=["doctor", "nurse", "programmer"]
... )
Copy argument is True. Transform will attempt to create a copy of the original
model. This may fail due to lack of memory.
Model copy created successfully.
>>>
>>> # if you want to exclude a set of words from the debias process
>>> # you can include them in the ignore parameter
>>> debiased_model = hsr.transform(
...     model=model, ignore=["dress", "beard", "niece", "nephew"]
... )
Copy argument is True. Transform will attempt to create a copy of the original
model. This may fail due to lack of memory.
Model copy created successfully.

__init__(verbose: bool = False, criterion_name: Optional[str] = None) → None

Initialize a Half Sibling Regression Debias instance.

Parameters:

verbosebool, optional

True will print informative messages about the debiasing process, by default False.

criterion_nameOptional[str], optional

The name of the criterion for which the debias is being executed, e.g., ‘Gender’. This will indicate the name of the model returning transform, by default None

fit(model: WordEmbeddingModel, definitional_words: List[str], alpha: float = 60) → BaseDebias

Compute the weight matrix and the bias information.

Parameters:

model: WordEmbeddingModel

The word embedding model to debias.

definitional_words: List[str]

List of strings. This list contains words that embody bias information by definition.

alpha: float

Ridge Regression constant. By default 60.

Returns:

BaseDebias

The debias method fitted.

transform(model: WordEmbeddingModel, target: Optional[List[str]] = None, ignore: Optional[List[str]] = None, copy: bool = True) → WordEmbeddingModel

Substracts the gender information from vectors.

Parameters:

modelWordEmbeddingModel

The word embedding model to mitigate.

targetOptional[List[str]], optional

If a set of words is specified in target, the debias method will be performed only on the word embeddings of this set. If None is provided, the debias will be performed on all non gender specific words (except those specified in ignore). Target words must not be included in the gender specific set. by default None.

ignoreOptional[List[str]], optional

If target is None and a set of words is specified in ignore, the debias method will perform the debias in all non gender specific words except those specified in this set, by default [].

copybool, optional

If True, the debias will be performed on a copy of the model. If False, the debias will be applied on the same model delivered, causing its vectors to mutate. WARNING: Setting copy with True requires RAM at least 2x of the size of the model, otherwise the execution of the debias may raise to MemoryError, by default True.

Returns:

WordEmbeddingModel

The debiased embedding model.