We lifted 13 functions out of this paper's own repositories and ran 3 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| 0-kaikai-0/sh2 | canonical | 3 of 13 |
| Function | Status | Where it lives |
|---|---|---|
| extract_answer_from_output | Ran | 0-kaikai-0/sh2/factor_eval.py pointer only (licence: NONE) · get_code("96fe095fcc6b821a") |
| get_config_class_from_processor_class | Ran | 0-kaikai-0/sh2/transformers/utils/create_dummy_models.py pointer only (licence: NONE) · get_code("3d999b95009c7537") |
| get_processor_types_from_config_class | Ran | 0-kaikai-0/sh2/transformers/utils/create_dummy_models.py pointer only (licence: NONE) · get_code("0c0b81b081919d8e") |
| download_url | Not yet run | 0-kaikai-0/sh2/factor_eval.py pointer only (licence: NONE) · get_code("0f14b819bc361c25") |
| format_best | Not yet run | 0-kaikai-0/sh2/tfqa_mc_eval.py pointer only (licence: NONE) · get_code("158b8847881a64b3") |
| get_architectures_from_config_class | Not yet run | 0-kaikai-0/sh2/transformers/utils/create_dummy_models.py pointer only (licence: NONE) · get_code("4ac2af420b64a13a") |
| load_csv | Not yet run | 0-kaikai-0/sh2/tfqa_keys.py pointer only (licence: NONE) · get_code("2a4c0ecd291e1795") |
| load_csv | Not yet run | 0-kaikai-0/sh2/factor_eval.py pointer only (licence: NONE) · get_code("c47f09f5ffb69e69") |
| load_csv | Not yet run | 0-kaikai-0/sh2/tfqa_mc_eval.py pointer only (licence: NONE) · get_code("91ad5d3cac5d67de") |
| load_jsonl | Not yet run | 0-kaikai-0/sh2/halusum_eval.py pointer only (licence: NONE) · get_code("2b8c437d3604845c") |
| num_tokens_from_message | Not yet run | 0-kaikai-0/sh2/halusum_eval.py pointer only (licence: NONE) · get_code("b3a67ee5030c2ffe") |
| split_multi_answer | Not yet run | 0-kaikai-0/sh2/tfqa_mc_eval.py pointer only (licence: NONE) · get_code("c852bd4d2face0a1") |
| truncate_message | Not yet run | 0-kaikai-0/sh2/halusum_eval.py pointer only (licence: NONE) · get_code("dc51f8f0031a298a") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Large language models (LLMs) demonstrate great performance in text generation. However, LLMs are still suffering from hallucinations. In this work, we propose an inference-time method, Self-Highlighted Hesitation (SH2), to help LLMs decode more truthfully. SH2 is based on a simple fact rooted in information theory that for an LLM, the tokens predicted with lower probabilities are prone to be more informative than others. Our analysis shows that the tokens assigned with lower probabilities by an LLM are more likely to be closely related to factual information, such as nouns, proper nouns, and adjectives. Therefore, we propose to ''highlight'' the factual information by selecting the tokens with the lowest probabilities and concatenating them to the original context, thus forcing the model to repeatedly read and hesitate on these tokens before generation. During decoding, we also adopt contrastive decoding to emphasize the difference in the output probabilities brought by the hesitation. Experimental results demonstrate that our SH2, requiring no additional data or models, can effectively help LLMs elicit factual knowledge and distinguish hallucinated contexts. Significant and consistent improvements are achieved by SH2 for LLaMA-7b, LLaMA2-7b and Mistral-7b on multiple hallucination tasks.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2401.05930")
get_code_for_paper("2401.05930")
have("2401.05930")
Connect an agent — have() is free.