Qi Zhang, Xuanjing Huang, Junjie Ye, Tao Gui, Songyang Gao, Guanyu Li, Caishuang Huang, Yilong Wu, Sixian Li
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Tool learning is widely acknowledged as a foundational approach for deploying large language models (LLMs) in real-world scenarios. While current research primarily emphasizes leveraging tools to augment LLMs, it frequently neglects emerging safety considerations tied to their application. To fill this gap, we present ToolSword, a comprehensive framework dedicated to meticulously investigating safety issues linked to LLMs in tool learning. Specifically, ToolSword delineates six safety scenarios for LLMs in tool learning, encompassing malicious queries and jailbreak attacks in the input stage, noisy misdirection and risky cues in the execution stage, and harmful feedback and error conflicts in the output stage. Experiments conducted on 11 open-source and closed-source LLMs reveal enduring safety challenges in tool learning, such as handling harmful queries, employing risky tools, and delivering detrimental feedback, which even GPT-4 is susceptible to. Moreover, we conduct further studies with the aim of fostering research on tool learning safety. The data is released in https://github.com/ Junjie-Ye/ToolSword.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2402.10753")
get_code_for_paper("2402.10753")
have("2402.10753")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2402.10753.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2402.10753)
Connect an agent — have() is free.