Kai Tan, Pierre Bellec
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
This paper studies the generalization performance of iterates obtained by Gradient Descent (GD), Stochastic Gradient Descent (SGD) and their proximal variants in high-dimensional robust regression problems. The number of features is comparable to the sample size and errors may be heavy-tailed. We introduce estimators that precisely track the generalization error of the iterates along the trajectory of the iterative algorithm. These estimators are provably consistent under suitable conditions. The results are illustrated through several examples, including Huber regression, pseudo-Huber regression, and their penalized variants with non-smooth regularizer. We provide explicit generalization error estimates for iterates generated from GD and SGD, or from proximal SGD in the presence of a non-smooth regularizer. The proposed risk estimates serve as effective proxies for the actual generalization error, allowing us to determine the optimal stopping iteration that minimizes the generalization error. Extensive simulations confirm the effectiveness of the proposed generalization error estimates. 0 min(1, x) dx, the Pseudo-Huber loss ρ(r; δ) = δ 2 ( 1 + (r/δ) 2 -1) or other Lipschitz loss functions to combat the possible heavy-tails of the additive noise. Typical examples of penalty functions include the L1/Lasso [27] penalty g(b) = λ∥b∥ 1 , group-Lasso penalty [29] for grouped variables, or their non-convex variants including for instance SCAD [12] or MCP [30]. In order to solve the optimization problem (2), practitioners resort to iterative algorithms, for instance gradient descent, accelerated gradient descent, stochastic gradient descent, and the corresponding proximal methods [20] in the presence of a non-smooth regularizer. Let the algorithm starts with 38th Conference on Neural Information Processing Systems (NeurIPS 2024).
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2410.02629")
get_code_for_paper("2410.02629")
have("2410.02629")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2410.02629.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2410.02629)
Connect an agent — have() is free.