Neural representations often contain information about attributes considered private or should not play a role in
the decision-making process. This problem leads to the question: can we remove such information from neural representations to make the
decision-making process not rely on such information, often spurious, to make such a process fairer and more precise?
This is where the Spectral Attribute RemovaL (SAL) comes to play. SAL erases information from neural representations by projecting
them into a subspace where the correlation between these attributes and the projected representations is minimal. There is a simple
way to do it by “flipping singular value decomposition on its head” – we start with the covariance matrix of the neural representations
coupled with the protected attributes, and then prune away the principal directions (which are found through SVD) with the largest
There are limitations to such a method, most notably that the erasing projection is linear. Therefore there might still be
information lurking in the representations that could be adversarially used. One way to partially solve that is by using rich
transformations on the neural representations and see whether, after these transformations, the information is there and can be
removed. This is done through a derivative of SAL, kSAL (for kernelized SAL), which uses kernel methods to create rich
Finally, it is often the case that we would like to remove information from a neural representation. Still, we might be
unable to pair each representation with its corresponding attribute during training, as SAL would require. This is where the Assignment-Maximization SAL
algorithm comes in to help us. It helps erase information from neural representations by looking at overall population statistics
(such as the priors of different classes of protected attributes). In two iterative steps, it first (A) assigns each (projected) neural representation
an attribute, and then (M) finds a projection to maximize the correlation between the attributes and the representations. By
iterating between these two steps, not unlike the EM algorithm (but without a log-likelihood function), it aims at finding
an assignment of a protected attribute to each neural representation, which makes them readily available for a final step.
The figure below illustrates the impact of SAL on word representations and, specifically, its effectiveness in removing gender information
from a profession classification dataset. This is a dataset that demonstrates the problem of relying on gender attributes when predicting
from a biography what the profession of the person in the biography is.
We examined how similar gender-stereotypical words (male and female) are with respect to each word in their profession cluster - male(blue)/female(red)
associated words before and after we applied SAL on the representations. Similarly, we calculated inter-cluster similarity. A score
higher than one means the similarities are higher after we use SAL, and a lower than one means the similarities are lower.
Our analysis revealed that, on average, words become less similar to others in their corresponding profession cluster and more similar to words
in the opposite cluster after applying SAL, making it more challenging for classifiers to use gender-related information for profession
classification where biased prediction may lead to unfair automated hiring procedure.