Kai Han, Yunhe Wang, Yehui Tang, Jianyuan Guo, Zhi-Hong Deng, Shibo Jie
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Vision Transformers (ViTs) by reducing the number of the redundant tokens, e.g., pruning inattentive tokens or merging similar tokens. However, when applied to downstream tasks, these approaches suffer from significant performance drop when the compression degrees are mismatched between training and inference stages, which limits the application of token compression on off-the-shelf trained models. In this paper, we propose a model arithmetic framework to decouple the compression degrees between the two stages. In advance, we additionally perform a fast parameter-efficient self-distillation stage on the pre-trained models to obtain a small plugin, called Token Compensator (ToCom), which describes the gap between models across different compression degrees. During inference, ToCom can be directly inserted into any downstream off-the-shelf models with any mismatched training and inference compression degrees to acquire universal performance improvements without further training. Experiments on over 20 downstream tasks demonstrate the effectiveness of our framework. On CIFAR100, fine-grained visual classification, and VTAB-1k benchmark, ToCom can yield up to a maximum improvement of 2.3%, 1.5%, and 2.0% in the average performance of DeiT-B, respectively.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2408.06798")
get_code_for_paper("2408.06798")
have("2408.06798")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2408.06798.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2408.06798)
Connect an agent — have() is free.