We lifted 1 functions out of this paper's own repositories and ran 1 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| opensafetylab/salad-bench | canonical | 1 of 1 |
| Function | Status | Where it lives |
|---|---|---|
| read_example_data | Ran | opensafetylab/salad-bench/examples/example.py code served (permissive licence) · get_code("fb5c20ebd49dd8a2") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
In the rapidly evolving landscape of Large Language Models (LLMs), ensuring robust safety measures is paramount. To meet this crucial need, we propose \emph{SALAD-Bench}, a safety benchmark specifically designed for evaluating LLMs, attack, and defense methods. Distinguished by its breadth, SALAD-Bench transcends conventional benchmarks through its large scale, rich diversity, intricate taxonomy spanning three levels, and versatile functionalities.SALAD-Bench is crafted with a meticulous array of questions, from standard queries to complex ones enriched with attack, defense modifications and multiple-choice. To effectively manage the inherent complexity, we introduce an innovative evaluators: the LLM-based MD-Judge for QA pairs with a particular focus on attack-enhanced queries, ensuring a seamless, and reliable evaluation. Above components extend SALAD-Bench from standard LLM safety evaluation to both LLM attack and defense methods evaluation, ensuring the joint-purpose utility. Our extensive experiments shed light on the resilience of LLMs against emerging threats and the efficacy of contemporary defense tactics. Data and evaluator are released under https://github.com/OpenSafetyLab/SALAD-BENCH.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2402.05044")
get_code_for_paper("2402.05044")
have("2402.05044")
Connect an agent — have() is free.