Srinandan Dasmahapatra, Christopher Subia-Waud
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Modern iterations of deep learning models contain millions (billions) of unique parameters-each represented by a b-bit number. Popular attempts at compressing neural networks (such as pruning and quantisation) have shown that many of the parameters are superfluous, which we can remove (pruning) or express with b < b bits (quantisation) without hindering performance. Here we look to go much further in minimising the information content of networks. Rather than a channel or layer-wise encoding, we look to lossless whole-network quantisation to minimise the entropy and number of unique parameters in a network. We propose a new method, which we call Weight Fixing Networks (WFN) that we design to realise four model outcome objectives: i) very few unique weights, ii) low-entropy weight encodings, iii) unique weight values which are amenable to energy-saving versions of hardware multiplication, and iv) lossless task-performance. Some of these goals are conflicting. To best balance these conflicts, we combine a few novel (and some well-trodden) tricks; a novel regularisation term, (i, ii) a view of clustering cost as relative distance change (i, ii, iv), and a focus on wholenetwork re-use of weights (i, iii). Our Imagenet experiments demonstrate lossless compression using 56x fewer unique weights and a 1.9x lower weight-space entropy than SOTA quantisation approaches. Code and model saves can be found at github.com/subiawaud/Weight Fix Networks.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2210.13554")
get_code_for_paper("2210.13554")
have("2210.13554")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2210.13554.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2210.13554)
Connect an agent — have() is free.