SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2409.11995 · 2024

Unraveling the Hessian: A Key to Smooth Convergence in Loss Function Landscapes

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 4 functions out of this paper's own repositories and ran 4 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

RepositoryRoleRan
kisnikser/landscape-hessian canonical 4 of 4
FunctionStatusWhere it lives
calculate_ema Ran kisnikser/landscape-hessian/code/get_loss_values.py
code served (permissive licence) · get_code("cf5d50a254b9b86e")
calculate_ema Ran kisnikser/landscape-hessian/code/plot_differences.py
code served (permissive licence) · get_code("a7406e6dd4491589")
criterion_params Ran kisnikser/landscape-hessian/code/get_loss_values.py
code served (permissive licence) · get_code("549ca336ff33b04d")
get_loss_abs_differences Ran kisnikser/landscape-hessian/code/get_loss_values.py
code served (permissive licence) · get_code("1c3c27dd0b929d96")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

The loss landscape of neural networks is a critical aspect of their training, and understanding its properties is essential for improving their performance. In this paper, we investigate how the loss surface changes when the sample size increases, a previously unexplored issue. We theoretically analyze the convergence of the loss landscape in a fully connected neural network and derive upper bounds for the difference in loss function values when adding a new object to the sample. Our empirical study confirms these results on various datasets, demonstrating the convergence of the loss function surface for image classification tasks. Our findings provide insights into the local geometry of neural loss landscapes and have implications for the development of sample size determination techniques.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2409.11995")
get_code_for_paper("2409.11995")
have("2409.11995")

Connect an agent — have() is free.