We lifted 6 functions out of this paper's own repositories and ran 6 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| copy not recorded | — | 6 of 6 |
| Function | Status | Where it lives |
|---|---|---|
| compute_ccas | Ran | this paper's copy was not recorded; identical code first harvested from google/svcca pointer only · get_code("01472465bed8b268") |
| positivedef_matrix_sqrt | Ran | this paper's copy was not recorded; identical code first harvested from google/svcca pointer only · get_code("ea98389a1755bb3f") |
| remove_small | Ran | this paper's copy was not recorded; identical code first harvested from google/svcca pointer only · get_code("ffcbd13c7094948e") |
| svcca_distance | Ran | this paper's copy was not recorded; identical code first harvested from moskomule/cca.pytorch pointer only · get_code("38d282dc70b65fd7") |
| svd_reduction | Ran | this paper's copy was not recorded; identical code first harvested from moskomule/cca.pytorch pointer only · get_code("25452d41e8e4d737") |
| zero_mean | Ran | this paper's copy was not recorded; identical code first harvested from moskomule/cca.pytorch pointer only · get_code("eff74a29f37795b8") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Comparing different neural network representations and determining how representations evolve over time remain challenging open questions in our understanding of the function of neural networks. Comparing representations in neural networks is fundamentally difficult as the structure of representations varies greatly, even across groups of networks trained on identical tasks, and over the course of training. Here, we develop projection weighted CCA (Canonical Correlation Analysis) as a tool for understanding neural networks, building off of SVCCA, a recently proposed method (Raghu et al., 2017). We first improve the core method, showing how to differentiate between signal and noise, and then apply this technique to compare across a group of CNNs, demonstrating that networks which generalize converge to more similar representations than networks which memorize, that wider networks converge to more similar solutions than narrow networks, and that trained networks with identical topology but different learning rates converge to distinct clusters with diverse representations. We also investigate the representational dynamics of RNNs, across both training and sequential timesteps, finding that RNNs converge in a bottom-up pattern over the course of training and that the hidden state is highly variable over the course of a sequence, even when accounting for linear transforms. Together, these results provide new insights into the function of CNNs and RNNs, and demonstrate the utility of using CCA to understand representations.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("1806.05759")
get_code_for_paper("1806.05759")
have("1806.05759")
Connect an agent — have() is free.