We lifted 8 functions out of this paper's own repositories and ran 7 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| neelsjain/neftune | canonical | 0 of 1 |
| akjindal53244/arithmo | pwc_unofficial | 7 of 7 |
| Function | Status | Where it lives |
|---|---|---|
| extract_ground_truth_answer | Ran | akjindal53244/arithmo/eval/gsm8k/gsm8k_compute_metric_zero_shot_CoT.py code served (permissive licence) · get_code("466b972ae1ee4368") |
| extract_predcited_answer | Ran | akjindal53244/arithmo/eval/gsm8k/gsm8k_compute_metric_zero_shot_CoT.py code served (permissive licence) · get_code("d5cb757080457d06") |
| extract_python_program | Ran | akjindal53244/arithmo/eval/gsm8k/gsm8k_write_zero_shot_PoT_outputs.py code served (permissive licence) · get_code("bccdf81fed5534bd") |
| is_equiv | Ran | akjindal53244/arithmo/eval/MATH/MATH_compute_metric_zero_shot_CoT.py code served (permissive licence) · get_code("20063c675d638a77") |
| last_boxed_only_string | Ran | akjindal53244/arithmo/eval/MATH/MATH_compute_metric_zero_shot_CoT.py code served (permissive licence) · get_code("0b14c648516c38a7") |
| remove_boxed | Ran | akjindal53244/arithmo/eval/MATH/MATH_compute_metric_zero_shot_CoT.py code served (permissive licence) · get_code("f3bbe264b05aadd3") |
| similar | Ran | akjindal53244/arithmo/data_prep/prepare_model_traininig_data.py code served (permissive licence) · get_code("5e4b02e5bc753357") |
| generate_responses_batched | Not yet run | neelsjain/neftune/experiment_code/eval_generate.py code served (permissive licence) · get_code("903c643febb1d508") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
We show that language model finetuning can be improved, sometimes dramatically, with a simple augmentation. NEFTune adds noise to the embedding vectors during training. Standard finetuning of LLaMA-2-7B using Alpaca achieves 29.79% on AlpacaEval, which rises to 64.69% using noisy embeddings. NEFTune also improves over strong baselines on modern instruction datasets. Models trained with Evol-Instruct see a 10% improvement, with ShareGPT an 8% improvement, and with OpenPlatypus an 8% improvement. Even powerful models further refined with RLHF such as LLaMA-2-Chat benefit from additional training with NEFTune.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2310.05914")
get_code_for_paper("2310.05914")
have("2310.05914")
Connect an agent — have() is free.