SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2211.11665 · ICLR · 2023

Representational dissimilarity metric spaces for stochastic neural networks

Jingyang Zhou, Alex Williams, Josue Nassar, Lyndon Duong, Jules Berman, Jeroen Olieslagers

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 7 functions out of this paper's own repositories and ran 7 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

RepositoryRoleRan
ahwillia/netrep canonical 7 of 7
FunctionStatusWhere it lives
align Ran ahwillia/netrep/netrep/utils.py
code served (permissive licence) · get_code("e3a13a9dbac537a1")
check_equal_shapes Ran ahwillia/netrep/netrep/validation.py
code served (permissive licence) · get_code("d93cab147768102d")
cross_distances Ran ahwillia/netrep/netrep/multiset.py
code served (permissive licence) · get_code("8c975d8811c0e60e")
pairwise_distances Ran ahwillia/netrep/netrep/multiset.py
code served (permissive licence) · get_code("91093e5cd3d80b03")
posdefsqrt Ran ahwillia/netrep/netrep/utils.py
code served (permissive licence) · get_code("27a5bb3f60533931")
split Ran ahwillia/netrep/netrep/metrics/stochastic_process.py
code served (permissive licence) · get_code("1e8f7903c244e8ab")
sq_bures_metric_slow Ran ahwillia/netrep/netrep/utils.py
code served (permissive licence) · get_code("66514727b6e365f6")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Quantifying similarity between neural representations-e.g. hidden layer activation vectors-is a perennial problem in deep learning and neuroscience research. Existing methods compare deterministic responses (e.g. artificial networks that lack stochastic layers) or averaged responses (e.g., trial-averaged firing rates in biological data). However, these measures of deterministic representational similarity ignore the scale and geometric structure of noise, both of which play important roles in neural computation. To rectify this, we generalize previously proposed shape metrics (Williams et al., 2021) to quantify differences in stochastic representations. These new distances satisfy the triangle inequality, and thus can be used as a rigorous basis for many supervised and unsupervised analyses. Leveraging this novel framework, we find that the stochastic geometries of neurobiological representations of oriented visual gratings and naturalistic scenes respectively resemble untrained and trained deep network representations. Further, we are able to more accurately predict certain network attributes (e.g. training hyperparameters) from its position in stochastic (versus deterministic) shape space.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2211.11665")
get_code_for_paper("2211.11665")
have("2211.11665")

Connect an agent — have() is free.