SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2502.01406 · ICLR · 2025

GRADIEND: Feature Learning within Neural Networks Exemplified through Biases

Encoder Decoder, Jonathan Drechsel, Steffen Herbold, ∇ -Wm Wm

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 11 functions out of this paper's own repositories and ran 6 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

RepositoryRoleRan
aieng-lab/gradiend-bias canonical 4 of 8
aieng-lab/gradiend — 2 of 3
FunctionStatusWhere it lives
LargeLinear Ran aieng-lab/gradiend-bias/gradiend/model.py
code served (permissive licence) · get_code("2fce88cdf9c23568")
LargeLinear Ran aieng-lab/gradiend/gradiend/model/model.py
code served (permissive licence) · get_code("f0ae40797e7c4249")
convert_tuple_keys_recursively Ran aieng-lab/gradiend-bias/gradiend/model.py
code served (permissive licence) · get_code("20e1c499fda1de6c")
get_activation Ran aieng-lab/gradiend-bias/gradiend/model.py
code served (permissive licence) · get_code("c056e040a07e6c67")
get_activation Ran aieng-lab/gradiend/gradiend/model/model.py
code served (permissive licence) · get_code("b5d368803b6b5a6c")
hash_it Ran aieng-lab/gradiend-bias/gradiend/model.py
code served (permissive licence) · get_code("e9a1e0ae7afb92c6")
GradiendModel Not yet run aieng-lab/gradiend-bias/gradiend/model.py
code served (permissive licence) · get_code("f3498d317263f1a7")
GradiendModel Not yet run aieng-lab/gradiend/gradiend/model/model.py
code served (permissive licence) · get_code("0b09e27b255b08fa")
compute_bias_score Not yet run aieng-lab/gradiend-bias/gradiend/evaluation/analyze_decoder.py
code served (permissive licence) · get_code("7c76d06aaa408115")
compute_bias_score_v1 Not yet run aieng-lab/gradiend-bias/gradiend/evaluation/analyze_decoder.py
code served (permissive licence) · get_code("63880b46142f096d")
compute_bias_score_v2 Not yet run aieng-lab/gradiend-bias/gradiend/evaluation/analyze_decoder.py
code served (permissive licence) · get_code("4f6a9dbc21fa4d20")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

AI systems frequently exhibit and amplify social biases, leading to harmful consequences in critical areas. This study introduces a novel encoder-decoder approach that leverages model gradients to learn a feature neuron encoding societal bias information such as gender, race, and religion. We show that our method can not only identify which weights of a model need to be changed to modify a feature, but even demonstrate that this can be used to rewrite models to debias them while maintaining other capabilities. We demonstrate the effectiveness of our approach across various model architectures and highlight its potential for broader applications.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2502.01406")
get_code_for_paper("2502.01406")
have("2502.01406")

Connect an agent — have() is free.