We lifted 1 functions out of this paper's own repositories and ran 1 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| ipouyall/can-llms-be-lateral-thinkers | canonical | 1 of 1 |
| Function | Status | Where it lives |
|---|---|---|
| read_data | Ran | ipouyall/can-llms-be-lateral-thinkers/experiments/finetune/inference.py pointer only (licence: NONE) · get_code("7060d3600a2bccbe") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Inspired by human cognition, Jiang et al.(2023c) create a benchmark for assessing LLMs' lateral thinking-thinking outside the box. Building upon this benchmark, we investigate how different prompting methods enhance LLMs' performance on this task to reveal their inherent power for outside-the-box thinking ability. Through participating in SemEval-2024, task 9, Sentence Puzzle sub-task, we explore prompt engineering methods: chain of thoughts (CoT) and direct prompting, enhancing with informative descriptions, and employing contextualizing prompts using a retrieval augmented generation (RAG) pipeline. Our experiments involve three LLMs including GPT-3.5, GPT-4, and Zephyr-7B-beta. We generate a dataset of thinking paths between riddles and options using GPT-4, validated by humans for quality. Findings indicate that compressed informative prompts enhance performance. Dynamic in-context learning enhances model performance significantly. Furthermore, fine-tuning Zephyr on our dataset enhances performance across other commonsense datasets, underscoring the value of innovative thinking.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2404.02474")
get_code_for_paper("2404.02474")
have("2404.02474")
Connect an agent — have() is free.