FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformer

Code that ran

We lifted 3 functions out of this paper's own repositories and ran 3 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

Repository	Role	Ran
megvii-research/FQ-ViT	canonical	2 of 2
copy not recorded	—	1 of 1

Function	Status	Where it lives
accuracy	Ran	this paper's copy was not recorded; identical code first harvested from ChristophReich1996/Semantic_Pyramid_for_Image_Generation pointer only · get_code("18badd533b52adb3")
str2model	Ran	megvii-research/FQ-ViT/test_quant.py code served (permissive licence) · get_code("e4b6f33d7173cf96")
validate	Ran	megvii-research/FQ-ViT/test_quant.py code served (permissive licence) · get_code("0e215bc8d251972f")

Repositories linked to this paper

megvii-research/FQ-ViT — marked official in Papers with Code; named in the paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Network quantization significantly reduces model inference complexity and has been widely used in real-world deployments. However, most existing quantization methods have been developed mainly on Convolutional Neural Networks (CNNs), and suffer severe degradation when applied to fully quantized vision transformers. In this work, we demonstrate that many of these difficulties arise because of serious inter-channel variation in LayerNorm inputs, and present, Power-of-Two Factor (PTF), a systematic method to reduce the performance degradation and inference complexity of fully quantized vision transformers. In addition, observing an extreme non-uniform distribution in attention maps, we propose Log-Int-Softmax (LIS) to sustain that and simplify inference by using 4-bit quantization and the BitShift operator. Comprehensive experiments on various transformer-based architectures and benchmarks show that our Fully Quantized Vision Transformer (FQ-ViT) outperforms previous works while even using lower bit-width on attention maps. For instance, we reach 84.89% top-1 accuracy with ViT-L on ImageNet and 50.8 mAP with Cascade Mask R-CNN (Swin-S) on COCO. To our knowledge, we are the first to achieve lossless accuracy degradation (~1%) on fully quantized vision transformers. The code is available at https://github.com/megvii-research/FQ-ViT.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2111.13824")
get_code_for_paper("2111.13824")
have("2111.13824")

Connect an agent — have() is free.