Weijie Su, Shuxiao Chen, Hangfeng He
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. If it links a repository, it is listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
As a popular approach to modeling the dynamics of training overparametrized neural networks (NNs), the neural tangent kernels (NTK) are known to fall behind real-world NNs in generalization ability. This performance gap is in part due to the label agnostic nature of the NTK, which renders the resulting kernel not as locally elastic as NNs (He & Su, 2020). In this paper, we introduce a novel approach from the perspective of label-awareness to reduce this gap for the NTK. Specifically, we propose two label-aware kernels that are each a superimposition of a labelagnostic part and a hierarchy of label-aware parts with increasing complexity of label dependence, using the Hoeffding decomposition. Through both theoretical and empirical evidence, we show that the models trained with the proposed kernels better simulate NNs in terms of generalization ability and local elasticity. 1
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2010.11775")
get_code_for_paper("2010.11775")
have("2010.11775")
Connect an agent — have() is free.