We lifted 15 functions out of this paper's own repositories and ran 14 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| damo-nlp-sg/coi-agent | canonical | 14 of 15 |
| Function | Status | Where it lives |
|---|---|---|
| encode_image | Ran | damo-nlp-sg/coi-agent/LLM.py code served (permissive licence) · get_code("f41cb1a19b154297") |
| extract | Ran | damo-nlp-sg/coi-agent/utils.py code served (permissive licence) · get_code("f650e65cc3083d84") |
| extract | Ran | damo-nlp-sg/coi-agent/searcher/sementic_search.py code served (permissive licence) · get_code("4abf0ae6b3c8d8de") |
| extract_json | Ran | damo-nlp-sg/coi-agent/utils.py code served (permissive licence) · get_code("8da3d9ba5f65d0b8") |
| fetch | Ran | damo-nlp-sg/coi-agent/searcher/sementic_search.py code served (permissive licence) · get_code("4872fb8c1582e598") |
| get_content_between_a_b | Ran | damo-nlp-sg/coi-agent/LLM.py code served (permissive licence) · get_code("a6d4e225abfd096e") |
| get_deep_check_idea_novel_search_query_prompt | Ran | damo-nlp-sg/coi-agent/prompts/deep_research_agent_prompts.py code served (permissive licence) · get_code("62498ce66167479b") |
| get_deep_rewrite_query_prompt | Ran | damo-nlp-sg/coi-agent/prompts/deep_research_agent_prompts.py code served (permissive licence) · get_code("9f075963c52cb8a9") |
| get_deep_search_query_prompt | Ran | damo-nlp-sg/coi-agent/prompts/deep_research_agent_prompts.py code served (permissive licence) · get_code("954ecd542ae11bc3") |
| get_judge_experiment_all_prompt | Ran | damo-nlp-sg/coi-agent/prompts/juder_prompts.py code served (permissive licence) · get_code("4ddd0eb717fa5b8e") |
| get_judge_idea_all_prompt | Ran | damo-nlp-sg/coi-agent/prompts/juder_prompts.py code served (permissive licence) · get_code("8f8af080a9c99830") |
| get_review_experiment_design_suggestions_prompt | Ran | damo-nlp-sg/coi-agent/prompts/review_agent_prompts.py code served (permissive licence) · get_code("455aefd097ad94ca") |
| get_review_search_related_paper_prompt | Ran | damo-nlp-sg/coi-agent/prompts/review_agent_prompts.py code served (permissive licence) · get_code("ec479392ccee5103") |
| get_review_suggestions_from_papers_prompt | Ran | damo-nlp-sg/coi-agent/prompts/review_agent_prompts.py code served (permissive licence) · get_code("7480cf0c2cd3d240") |
| get_openai_url | Not yet run | damo-nlp-sg/coi-agent/LLM.py code served (permissive licence) · get_code("d79e88c565eff1fa") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Effective research ideation is a critical step for scientific research. However, the exponential increase in scientific literature makes it challenging for researchers to stay current with recent advances and identify meaningful research directions. Recent developments in large language models~(LLMs) suggest a promising avenue for automating the generation of novel research ideas. However, existing methods for idea generation either trivially prompt LLMs or directly expose LLMs to extensive literature without indicating useful information. Inspired by the research process of human researchers, we propose a Chain-of-Ideas~(CoI) agent, an LLM-based agent that organizes relevant literature in a chain structure to effectively mirror the progressive development in a research domain. This organization facilitates LLMs to capture the current advancements in research, thereby enhancing their ideation capabilities. Furthermore, we propose Idea Arena, an evaluation protocol that can comprehensively evaluate idea generation methods from different perspectives, aligning closely with the preferences of human researchers. Experimental results indicate that the CoI agent consistently outperforms other methods and shows comparable quality as humans in research idea generation. Moreover, our CoI agent is budget-friendly, with a minimum cost of \$0.50 to generate a candidate idea and its corresponding experimental design.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2410.13185")
get_code_for_paper("2410.13185")
have("2410.13185")
Connect an agent — have() is free.