Qing Li, Lianbo Ma, Yuee Zhou, Jianlun Ma, Guo Yu
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Weight quantization is an effective technique to compress deep neural networks for their deployment on edge devices with limited resources. Traditional loss-aware quantization methods commonly use the quantized gradient to replace the full-precision gradient. However, we discover that the gradient error will lead to an unexpected zig-zagging-like issue in the gradient descent learning procedures, where the gradient directions rapidly oscillate or zig-zag, and such issue seriously slows down the model convergence. Accordingly, this paper proposes a one-step forward and backtrack way for loss-aware quantization to get more accurate and stable gradient direction to defy this issue. During the gradient descent learning, a one-step forward search is designed to find the trial gradient of the next-step, which is adopted to adjust the gradient of current step towards the direction of fast convergence. After that, we backtrack the current step to update the full-precision and quantized weights through the current-step gradient and the trial gradient. A series of theoretical analysis and experiments on benchmark deep models have demonstrated the effectiveness and competitiveness of the proposed method, and our method especially outperforms others on the convergence performance.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2401.16760")
get_code_for_paper("2401.16760")
have("2401.16760")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2401.16760.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2401.16760)
Connect an agent — have() is free.