Lucas Beyer, Alexander Kolesnikov, Xiaohua Zhai, Basil Mustafa
We lifted 29 functions out of this paper's own repositories and ran 14 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| apple/ml-mobileclip | — | 7 of 16 |
| mlfoundations/open_clip | — | 3 of 5 |
| copy not recorded | — | 2 of 4 |
| merveenoyan/siglip | pwc_unofficial | 2 of 2 |
| ramanakshay/clip | — | 0 of 2 |
| Function | Status | Where it lives |
|---|---|---|
| GlobalPool | Ran | apple/ml-mobileclip/mobileclip/clip.py pointer only (licence: NOASSERTION) · get_code("af515d1162a90440") |
| GlobalPool2D | Ran | apple/ml-mobileclip/mobileclip/clip.py pointer only (licence: NOASSERTION) · get_code("21e0abb626a7ae62") |
| LayerNormFP32 | Ran | apple/ml-mobileclip/mobileclip/clip.py pointer only (licence: NOASSERTION) · get_code("c4c619fef7d59d34") |
| LearnablePositionalEmbedding | Ran | apple/ml-mobileclip/mobileclip/clip.py pointer only (licence: NOASSERTION) · get_code("0748e821730b3fc1") |
| MultiHeadAttention | Ran | apple/ml-mobileclip/mobileclip/clip.py pointer only (licence: NOASSERTION) · get_code("88f8bcf4b76c2844") |
| NeighbourExchange | Ran | mlfoundations/open_clip/src/open_clip/loss.py pointer only (licence: NOASSERTION) · get_code("997398ad737e7cd2") |
| NeighbourExchangeBidir | Ran | mlfoundations/open_clip/src/open_clip/loss.py pointer only (licence: NOASSERTION) · get_code("7caa2b64e6a6f021") |
| PositionalEmbedding | Ran | apple/ml-mobileclip/mobileclip/clip.py pointer only (licence: NOASSERTION) · get_code("1e38402044297bef") |
| SigLipLoss | Ran | mlfoundations/open_clip/src/open_clip/loss.py pointer only (licence: NOASSERTION) · get_code("2d2c2d56e476e772") |
| get_normalization_layer | Ran | apple/ml-mobileclip/mobileclip/clip.py pointer only (licence: NOASSERTION) · get_code("59434b67e9a74429") |
| postprocess | Ran | merveenoyan/siglip/clip_siglip.py code served (permissive licence) · get_code("3586d486792b020d") |
| postprocess_siglip | Ran | merveenoyan/siglip/clip_siglip.py code served (permissive licence) · get_code("ad49d2b1bad09561") |
| relic_loss | Ran | this paper's copy was not recorded; identical code first harvested from filipbasara0/relic pointer only · get_code("eb62100582261eb4") |
| siglip_loss | Ran | this paper's copy was not recorded; identical code first harvested from filipbasara0/simple-clip pointer only · get_code("67c4ef5004396e02") |
| CLIP | Not yet run | ramanakshay/clip/src/model/model.py code served (permissive licence) · get_code("7f362a66c1752126") |
| CLIP | Not yet run | apple/ml-mobileclip/mobileclip/clip.py pointer only (licence: NOASSERTION) · get_code("a8fa6ab303249a21") |
| CLIPModel | Not yet run | ramanakshay/clip/src/model/model.py code served (permissive licence) · get_code("e80ed1f8e5c3f5a8") |
| ConvFFN | Not yet run | apple/ml-mobileclip/mobileclip/clip.py pointer only (licence: NOASSERTION) · get_code("df9eaf41a5c09a07") |
| MCi | Not yet run | apple/ml-mobileclip/mobileclip/clip.py pointer only (licence: NOASSERTION) · get_code("579a47ae5bfe8248") |
| MobileOneBlock | Not yet run | apple/ml-mobileclip/mobileclip/clip.py pointer only (licence: NOASSERTION) · get_code("695f06cd58c0597e") |
| RepMixer | Not yet run | apple/ml-mobileclip/mobileclip/clip.py pointer only (licence: NOASSERTION) · get_code("43cafd406a2da3f3") |
| RepMixerBlock | Not yet run | apple/ml-mobileclip/mobileclip/clip.py pointer only (licence: NOASSERTION) · get_code("efcf8d9368ee9d08") |
| SEBlock | Not yet run | apple/ml-mobileclip/mobileclip/clip.py pointer only (licence: NOASSERTION) · get_code("49fac99e1f1a0270") |
| TextTransformer | Not yet run | apple/ml-mobileclip/mobileclip/clip.py pointer only (licence: NOASSERTION) · get_code("88fa799b7ff64ccb") |
| TransformerEncoder | Not yet run | apple/ml-mobileclip/mobileclip/clip.py pointer only (licence: NOASSERTION) · get_code("35094b7d325013c5") |
| neighbour_exchange | Not yet run | this paper's copy was not recorded; identical code first harvested from lxa9867/imagefolder pointer only · get_code("e332856e3c2fc814") |
| neighbour_exchange_bidir | Not yet run | this paper's copy was not recorded; identical code first harvested from lxa9867/imagefolder pointer only · get_code("5b1fd364afcf3c05") |
| neighbour_exchange_bidir_with_grad | Not yet run | mlfoundations/open_clip/src/open_clip/loss.py pointer only (licence: NOASSERTION) · get_code("4a8006bab44bb658") |
| neighbour_exchange_with_grad | Not yet run | mlfoundations/open_clip/src/open_clip/loss.py pointer only (licence: NOASSERTION) · get_code("12c8728ab37fdce8") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
We propose a simple pairwise sigmoid loss for imagetext pre-training. Unlike standard contrastive learning with softmax normalization, the sigmoid loss operates solely on image-text pairs and does not require a global view of the pairwise similarities for normalization. The sigmoid loss simultaneously allows further scaling up the batch size, while also performing better at smaller batch sizes. With only four TPUv4 chips, we can train a Base CLIP model at 4 k batch size and a Large LiT model at 20 k batch size, the latter achieves 84.5% ImageNet zero-shot accuracy in two days. This disentanglement of the batch size from the loss further allows us to study the impact of examples vs pairs and negative to positive ratio. Finally, we push the batch size to the extreme, up to one million, and find that the benefits of growing batch size quickly diminish, with a more reasonable batch size of 32 k being sufficient. We hope our research motivates further explorations in improving the quality and efficiency of language-image pre-training.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2303.15343")
get_code_for_paper("2303.15343")
have("2303.15343")
Connect an agent — have() is free.