Sangdoo Yun, Dongyoon Han, Alexander Hauptmann, Jaegul Choo, Changdae Oh, Kyungwoo Song, Hyesu Lim, Mijoo Kim, Zhi-Qi Cheng
We lifted 12 functions out of this paper's own repositories and ran 6 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| MLAI-Yonsei/CaRot | canonical | 6 of 12 |
| Function | Status | Where it lives |
|---|---|---|
| basic_clean | Ran | MLAI-Yonsei/CaRot/clip/tokenizer.py pointer only (licence: NONE) · get_code("98f385d847636a3e") |
| compute_calibration | Ran | MLAI-Yonsei/CaRot/src/visualize.py pointer only (licence: NONE) · get_code("6e97baf23bf18650") |
| get_pairs | Ran | MLAI-Yonsei/CaRot/clip/tokenizer.py pointer only (licence: NONE) · get_code("d919ae32e5e4e616") |
| maybe_dictionarize | Ran | MLAI-Yonsei/CaRot/src/datasets_/common.py pointer only (licence: NONE) · get_code("b1e9e0d3d7d7e615") |
| project_logits | Ran | MLAI-Yonsei/CaRot/src/datasets_/imagenet.py pointer only (licence: NONE) · get_code("9fbea808db47dc66") |
| whitespace_clean | Ran | MLAI-Yonsei/CaRot/clip/tokenizer.py pointer only (licence: NONE) · get_code("9542161e9640b858") |
| build_model | Not yet run | MLAI-Yonsei/CaRot/clip/model.py pointer only (licence: NONE) · get_code("b64f67bec6a093b9") |
| gather_features | Not yet run | MLAI-Yonsei/CaRot/clip/loss.py pointer only (licence: NONE) · get_code("ddcbd45e940484ee") |
| get_features | Not yet run | MLAI-Yonsei/CaRot/src/datasets_/common.py pointer only (licence: NONE) · get_code("35ec3a8e556cc61c") |
| get_features_helper | Not yet run | MLAI-Yonsei/CaRot/src/datasets_/common.py pointer only (licence: NONE) · get_code("74edffa1dfc39e58") |
| reliability_diagram | Not yet run | MLAI-Yonsei/CaRot/src/visualize.py pointer only (licence: NONE) · get_code("d558de6dd71f4220") |
| reliability_diagrams | Not yet run | MLAI-Yonsei/CaRot/src/visualize.py pointer only (licence: NONE) · get_code("435288e8d2fd445a") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Improving out-of-distribution (OOD) generalization during in-distribution (ID) adaptation is a primary goal of robust fine-tuning of zero-shot models beyond naive fine-tuning. However, despite decent OOD generalization performance from recent robust fine-tuning methods, confidence calibration for reliable model output has not been fully addressed. This work proposes a robust fine-tuning method that improves both OOD accuracy and confidence calibration simultaneously in vision language models. Firstly, we show that both OOD classification and OOD calibration errors have a shared upper bound consisting of two terms of ID data: 1) ID calibration error and 2) the smallest singular value of the ID input covariance matrix. Based on this insight, we design a novel framework that conducts finetuning with a constrained multimodal contrastive loss enforcing a larger smallest singular value, which is further guided by the self-distillation of a moving-averaged model to achieve calibrated prediction as well. Starting from empirical evidence supporting our theoretical statements, we provide extensive experimental results on ImageNet distribution shift benchmarks that demonstrate the effectiveness of our theorem and its practical implementation. Our code is available here.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2311.01723")
get_code_for_paper("2311.01723")
have("2311.01723")
Connect an agent — have() is free.