We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Large language models (LLMs) have shown promise for automatic summarization but the reasons behind their successes are poorly understood. By conducting a human evaluation on ten LLMs across different pretraining methods, prompts, and model scales, we make two important observations. First, we find instruction tuning, and not model size, is the key to the LLM's zero-shot summarization capability. Second, existing studies have been limited by low-quality references, leading to underestimates of human performance and lower few-shot and finetuning performance. To better evaluate LLMs, we perform human evaluation over high-quality summaries we collect from freelance writers. Despite major stylistic differences such as the amount of paraphrasing, we find that LMM summaries are judged to be on par with human written summaries.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2301.13848")
get_code_for_paper("2301.13848")
have("2301.13848")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2301.13848.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2301.13848)
Connect an agent — have() is free.