Bastian Rieck, Marc Niethammer, Florian Graf, Sebastian Zeng, Roland Kwitt
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
We study the excess capacity of deep networks in the context of supervised classification. That is, given a capacity measure of the underlying hypothesis class -in our case, empirical Rademacher complexity -to what extent can we (a priori) constrain this class while retaining an empirical error on a par with the unconstrained regime? To assess excess capacity in modern architectures (such as residual networks), we extend and unify prior Rademacher complexity bounds to accommodate function composition and addition, as well as the structure of convolutions. The capacitydriving terms in our bounds are the Lipschitz constants of the layers and an (2, 1) group norm distance to the initializations of the convolution weights. Experiments on benchmark datasets of varying task difficulty indicate that (1) there is a substantial amount of excess capacity per task, and (2) capacity can be kept at a surprisingly similar level across tasks. Overall, this suggests a notion of compressibility with respect to weight norms, complementary to classic compression via weight pruning. Source code is available at https://github.com/rkwitt/excess_capacity.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2202.08070")
get_code_for_paper("2202.08070")
have("2202.08070")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2202.08070.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2202.08070)
Connect an agent — have() is free.