SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2409.13079 · 2024

Embedding Geometries of Contrastive Language-Image Pre-Training

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 15 functions out of this paper's own repositories and ran 4 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

RepositoryRoleRan
eify/open_clip canonical 4 of 15
FunctionStatusWhere it lives
convert_to_custom_text_state_dict Ran eify/open_clip/src/open_clip/model.py
pointer only (licence: NONE) · get_code("f19962ebb134b3d7")
get_1d_sincos_pos_embed_from_grid Ran eify/open_clip/src/open_clip/pos_embed.py
pointer only (licence: NONE) · get_code("4bdd36eab04c3f62")
get_cast_dtype Ran eify/open_clip/src/open_clip/model.py
pointer only (licence: NONE) · get_code("dcd422d66b0581d8")
get_input_dtype Ran eify/open_clip/src/open_clip/model.py
pointer only (licence: NONE) · get_code("b476c8cfbf0f1f47")
gather_features Not yet run eify/open_clip/src/open_clip/loss.py
pointer only (licence: NONE) · get_code("ddcbd45e940484ee")
get_2d_sincos_pos_embed Not yet run eify/open_clip/src/open_clip/pos_embed.py
pointer only (licence: NONE) · get_code("9417ae492629cf06")
get_2d_sincos_pos_embed_from_grid Not yet run eify/open_clip/src/open_clip/pos_embed.py
pointer only (licence: NONE) · get_code("4fd80de79832745d")
get_model_config Not yet run eify/open_clip/src/open_clip/factory.py
pointer only (licence: NOASSERTION) · get_code("491eae38899d6610")
load_openai_model Not yet run eify/open_clip/src/open_clip/openai.py
pointer only (licence: NOASSERTION) · get_code("8811c5d06f3a2eaa")
load_state_dict Not yet run eify/open_clip/src/open_clip/factory.py
pointer only (licence: NONE) · get_code("ef8f5bfd8696abff")
neighbour_exchange Not yet run eify/open_clip/src/open_clip/loss.py
pointer only (licence: NONE) · get_code("e332856e3c2fc814")
neighbour_exchange_bidir Not yet run eify/open_clip/src/open_clip/loss.py
pointer only (licence: NONE) · get_code("5b1fd364afcf3c05")
parse_model_name Not yet run eify/open_clip/src/open_clip/factory.py
pointer only (licence: NONE) · get_code("898cc6c8249c06a0")
prepare_inputs_for_generation Not yet run eify/open_clip/src/open_clip/coca_model.py
pointer only (licence: NONE) · get_code("fb651d0a97fd4d3f")
register_pooler Not yet run eify/open_clip/src/open_clip/hf_model.py
pointer only (licence: NONE) · get_code("2a377da4a76a2d44")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Since the publication of CLIP, the approach of using InfoNCE loss for contrastive pre-training has become widely popular for bridging two or more modalities. Despite its wide adoption, CLIP's original design choices of L2 normalization and cosine similarity logit have rarely been revisited. We have systematically experimented with alternative geometries and softmax logits for language-image pre-training and identified that variants with intuitive Euclidean geometry, Euclidean CLIP (EuCLIP), match or exceed the performance of CLIP and support hierarchical relationships at least as well as more complicated hyperbolic alternative.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2409.13079")
get_code_for_paper("2409.13079")
have("2409.13079")

Connect an agent — have() is free.