We lifted 5 functions out of this paper's own repositories and ran 5 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| ashwinipokle/contrastive_landscape | canonical | 5 of 5 |
| Function | Status | Where it lives |
|---|---|---|
| D | Ran | ashwinipokle/contrastive_landscape/models/model_utils.py code served (permissive licence) · get_code("d73bba2d00bfe495") |
| gen_z | Ran | ashwinipokle/contrastive_landscape/data_model/gen_sparse_coding_data.py code served (permissive licence) · get_code("c9084c59df73470b") |
| gen_z_one_hot | Ran | ashwinipokle/contrastive_landscape/data_model/gen_sparse_coding_data.py code served (permissive licence) · get_code("b697a417ca50012e") |
| gen_z_random | Ran | ashwinipokle/contrastive_landscape/data_model/gen_sparse_coding_data.py code served (permissive licence) · get_code("4dc470bb2a694eb3") |
| str2bool | Ran | ashwinipokle/contrastive_landscape/common_args.py code served (permissive licence) · get_code("7c508037b40522af") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
A lot of recent advances in unsupervised feature learning are based on designing features which are invariant under semantic data augmentations. A common way to do this is contrastive learning, which uses positive and negative samples. Some recent works however have shown promising results for non-contrastive learning, which does not require negative samples. However, the non-contrastive losses have obvious "collapsed" minima, in which the encoders output a constant feature embedding, independent of the input. A folk conjecture is that so long as these collapsed solutions are avoided, the produced feature representations should be good. In our paper, we cast doubt on this story: we show through theoretical results and controlled experiments that even on simple data models, non-contrastive losses have a preponderance of non-collapsed bad minima. Moreover, we show that the training process does not avoid these minima.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2203.15702")
get_code_for_paper("2203.15702")
have("2203.15702")
Connect an agent — have() is free.