SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2402.01832 · 2024

SynthCLIP: Are We Ready for a Fully Synthetic CLIP Training?

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 12 functions out of this paper's own repositories and ran 11 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

RepositoryRoleRan
hammoudhasan/synthclip canonical 11 of 12
FunctionStatusWhere it lives
all_gather_batch Ran hammoudhasan/synthclip/Training/utils.py
pointer only (licence: NONE) · get_code("d888047d3f40a1a5")
balance_sampling Ran hammoudhasan/synthclip/TextGen/balancing.py
pointer only (licence: NONE) · get_code("14ba694ed500c3d5")
basic_clean Ran hammoudhasan/synthclip/Training/tokenizer.py
pointer only (licence: NONE) · get_code("98f385d847636a3e")
dist_func Ran hammoudhasan/synthclip/TextGen/substr_matching.py
pointer only (licence: NONE) · get_code("2918369c57cf9ccd")
get_imagenet Ran hammoudhasan/synthclip/Training/datasets.py
pointer only (licence: NONE) · get_code("76b6f8ebe3d85394")
get_pairs Ran hammoudhasan/synthclip/Training/tokenizer.py
pointer only (licence: NONE) · get_code("d919ae32e5e4e616")
get_rewrites_from_file Ran hammoudhasan/synthclip/ImageGen/batched_txt2img.py
pointer only (licence: NONE) · get_code("2d570f88cec83aee")
scaled_all_reduce Ran hammoudhasan/synthclip/Training/utils.py
pointer only (licence: NONE) · get_code("dc80fec8c333239e")
spacing Ran hammoudhasan/synthclip/TextGen/substr_matching.py
pointer only (licence: NONE) · get_code("cc85393791526ee3")
substr_matching Ran hammoudhasan/synthclip/TextGen/substr_matching.py
pointer only (licence: NONE) · get_code("c0af2ad3ff3eea0e")
whitespace_clean Ran hammoudhasan/synthclip/Training/tokenizer.py
pointer only (licence: NONE) · get_code("9542161e9640b858")
get_model Not yet run hammoudhasan/synthclip/Training/utils.py
pointer only (licence: NONE) · get_code("2ec9ac675733a9ac")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

We present SynthCLIP, a CLIP model trained on entirely synthetic text-image pairs. Leveraging recent text-to-image (TTI) networks and large language models (LLM), we generate synthetic datasets of images and corresponding captions at scale, with no human intervention. In this work, we provide an analysis on CLIP models trained on synthetic data. We provide insights on the data generation strategy, number of samples required, scaling trends, and resulting properties. We also introduce SynthCI-30M, a purely synthetic dataset comprising 30 million captioned images. Our code, trained models, and data, are released as open source at https://github.com/hammoudhasan/SynthCLIP

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2402.01832")
get_code_for_paper("2402.01832")
have("2402.01832")

Connect an agent — have() is free.