We lifted 18 functions out of this paper's own repositories and ran 13 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| salesforce/jaxformer | canonical | 9 of 14 |
| salesforce/progen | canonical | 2 of 2 |
| copy not recorded | — | 2 of 2 |
| Function | Status | Where it lives |
|---|---|---|
| apply_rotary_pos_emb | Ran | this paper's copy was not recorded; identical code first harvested from salesforce/CodeGen pointer only · get_code("2f6618868885e7ca") |
| cast | Ran | salesforce/jaxformer/jaxformer/hf/sample.py code served (permissive licence) · get_code("58a861ed3065cdb3") |
| duplicate_interleave | Ran | salesforce/jaxformer/jaxformer/hf/codegen/modeling_codegen.py code served (permissive licence) · get_code("03e99c761c545617") |
| fixed_pos_embedding | Ran | salesforce/progen/progen2/models/progen/modeling_progen.py code served (permissive licence) · get_code("72fa614b434d062b") |
| fixed_pos_embedding | Ran | salesforce/jaxformer/jaxformer/hf/codegen/modeling_codegen.py code served (permissive licence) · get_code("7ea21b9da7ef5be6") |
| loss | Ran | salesforce/jaxformer/jaxformer/models/decoder/inter/model.py code served (permissive licence) · get_code("943b81269283516d") |
| rotate_every_two | Ran | this paper's copy was not recorded; identical code first harvested from salesforce/CodeGen pointer only · get_code("c66149010337c505") |
| rotate_every_two | Ran | salesforce/jaxformer/jaxformer/hf/codegen/modeling_codegen.py code served (permissive licence) · get_code("621a8a98538cd46d") |
| run_return | Ran | salesforce/jaxformer/jaxformer/utils.py code served (permissive licence) · get_code("127d82bb96d01476") |
| sample | Ran | salesforce/progen/progen2/likelihood.py code served (permissive licence) · get_code("5f6ee6a61b771e9f") |
| set_default_config | Ran | salesforce/jaxformer/jaxformer/run/trainer.py code served (permissive licence) · get_code("28250166becef619") |
| sh_ret | Ran | salesforce/jaxformer/jaxformer/utils.py code served (permissive licence) · get_code("36f7d178ef808204") |
| tree_flatten_with_names | Ran | salesforce/jaxformer/jaxformer/hf/convert.py code served (permissive licence) · get_code("a0fd9e51c4dc05fd") |
| apply_rotary_pos_emb | Not yet run | salesforce/jaxformer/jaxformer/models/decoder/inter/positional.py code served (permissive licence) · get_code("5679796dda9bd742") |
| create_master | Not yet run | salesforce/jaxformer/jaxformer/run/trainer.py code served (permissive licence) · get_code("f1a5638f90df6384") |
| include_whitespace | Not yet run | salesforce/jaxformer/jaxformer/hf/sample.py code served (permissive licence) · get_code("f5761899e7fefe36") |
| run_loop | Not yet run | salesforce/jaxformer/jaxformer/utils.py code served (permissive licence) · get_code("365cda55aef11b01") |
| tree_leaves_with_names | Not yet run | salesforce/jaxformer/jaxformer/hf/convert.py code served (permissive licence) · get_code("2c193137483f61dd") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Attention-based models trained on protein sequences have demonstrated incredible success at classification and generation tasks relevant for artificial intelligence-driven protein design. However, we lack a sufficient understanding of how very large-scale models and data play a role in effective protein model development. We introduce a suite of protein language models, named ProGen2, that are scaled up to 6.4B parameters and trained on different sequence datasets drawn from over a billion proteins from genomic, metagenomic, and immune repertoire databases. ProGen2 models show state-of-the-art performance in capturing the distribution of observed evolutionary sequences, generating novel viable sequences, and predicting protein fitness without additional finetuning. As large model sizes and raw numbers of protein sequences continue to become more widely accessible, our results suggest that a growing emphasis needs to be placed on the data distribution provided to a protein sequence model. We release the ProGen2 models and code at https://github.com/salesforce/progen.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2206.13517")
get_code_for_paper("2206.13517")
have("2206.13517")
Connect an agent — have() is free.