Chinmay Hegde, Jiangyuan Li, Thanh Nguyen, Raymond Wong
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
We study the implicit regularization of gradient descent towards structured sparsity via a novel neural reparameterization, which we call a "diagonally grouped linear neural network". We show the following intriguing property of our reparameterization: gradient descent over the squared regression loss, without any explicit regularization, biases towards solutions with a group sparsity structure. In contrast to many existing works in understanding implicit regularization, we prove that our training trajectory cannot be simulated by mirror descent. We analyze the gradient dynamics of the corresponding regression problem in the general noise setting and obtain minimax-optimal error rates. Compared to existing bounds for implicit sparse regularization using diagonal linear networks, our analysis with the new reparameterization shows improved sample complexity. In the degenerate case of size-one groups, our approach gives rise to a new algorithm for sparse linear regression. Finally, we demonstrate the efficacy of our approach with several numerical experiments 1 .
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2301.12540")
get_code_for_paper("2301.12540")
have("2301.12540")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2301.12540.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2301.12540)
Connect an agent — have() is free.