SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2106.02624 · 2021

ViViT: Curvature access through the generalized Gauss-Newton's low-rank structure

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 15 functions out of this paper's own repositories and ran 0 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

RepositoryRoleRan
pwlo3k46/vivit canonical 0 of 9
jbzre7bp/vivit canonical 0 of 4
zv5dmjq5/vivit canonical 0 of 2
FunctionStatusWhere it lives
approximate_boundaries Not yet run jbzre7bp/vivit/vivit/hessianfree/lanczos.py
code served (permissive licence) · get_code("8bcff92887677b71")
approximate_boundaries_abs Not yet run jbzre7bp/vivit/vivit/hessianfree/lanczos.py
code served (permissive licence) · get_code("e4d94e72d53429eb")
fast_lanczos Not yet run jbzre7bp/vivit/vivit/hessianfree/lanczos.py
code served (permissive licence) · get_code("c38616bc34dad46d")
get_letters Not yet run pwlo3k46/vivit/vivit/utils/gram.py
code served (permissive licence) · get_code("3b29da7c5bb6f1c4")
is_subset Not yet run pwlo3k46/vivit/vivit/utils/subsampling.py
code served (permissive licence) · get_code("b4944136575f91af")
make_top_k_criterion Not yet run jbzre7bp/vivit/exp/utils/vivit.py
code served (permissive licence) · get_code("e157c276ee749806")
merge_subsamplings Not yet run pwlo3k46/vivit/vivit/utils/subsampling.py
code served (permissive licence) · get_code("26c378565ba49153")
pairwise2_dot Not yet run pwlo3k46/vivit/vivit/utils/gram.py
code served (permissive licence) · get_code("0049e73ca8f9bee3")
pairwise_dot Not yet run pwlo3k46/vivit/vivit/utils/gram.py
code served (permissive licence) · get_code("42d68d5c80b5db4e")
sample_output_mapping Not yet run pwlo3k46/vivit/vivit/utils/subsampling.py
code served (permissive licence) · get_code("2e09290d345742fd")
shift_diag Not yet run pwlo3k46/vivit/vivit/utils/eig.py
code served (permissive licence) · get_code("facdac15f2c804c0")
stable_symeig Not yet run pwlo3k46/vivit/vivit/utils/eig.py
code served (permissive licence) · get_code("e044b617cfd2c895")
stable_symeig Not yet run zv5dmjq5/vivit/vivit/utils/eig.py
code served (permissive licence) · get_code("7cf1a1d2a94fd903")
symeig_psd Not yet run pwlo3k46/vivit/vivit/utils/eig.py
code served (permissive licence) · get_code("6667b9a2a60b7fe1")
symeig_psd Not yet run zv5dmjq5/vivit/vivit/utils/eig.py
code served (permissive licence) · get_code("484f66d87c0c4751")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Curvature in form of the Hessian or its generalized Gauss-Newton (GGN) approximation is valuable for algorithms that rely on a local model for the loss to train, compress, or explain deep networks. Existing methods based on implicit multiplication via automatic differentiation or Kronecker-factored block diagonal approximations do not consider noise in the mini-batch. We present ViViT, a curvature model that leverages the GGN's low-rank structure without further approximations. It allows for efficient computation of eigenvalues, eigenvectors, as well as per-sample first- and second-order directional derivatives. The representation is computed in parallel with gradients in one backward pass and offers a fine-grained cost-accuracy trade-off, which allows it to scale. We demonstrate this by conducting performance benchmarks and substantiate ViViT's usefulness by studying the impact of noise on the GGN's structural properties during neural network training.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2106.02624")
get_code_for_paper("2106.02624")
have("2106.02624")

Connect an agent — have() is free.