SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2407.10058 · 2024

Learning to Refuse: Towards Mitigating Privacy Risks in LLMs

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 6 functions out of this paper's own repositories and ran 6 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

RepositoryRoleRan
zhliu0106/learning-to-refuse canonical 6 of 6
FunctionStatusWhere it lives
filter_out_person Ran zhliu0106/learning-to-refuse/process_data/filtering.py
pointer only (licence: NONE) · get_code("c1a969d76dae6c3d")
generate_prompt Ran zhliu0106/learning-to-refuse/process_data/augmentation.py
pointer only (licence: NONE) · get_code("a4dd4eb5709c2a36")
load_filtered_personqa Ran zhliu0106/learning-to-refuse/process_data/construction.py
pointer only (licence: NONE) · get_code("fe88e7af8e255442")
load_model_prediction Ran zhliu0106/learning-to-refuse/process_data/filtering.py
pointer only (licence: NONE) · get_code("30889d3bca9a6056")
load_question_template Ran zhliu0106/learning-to-refuse/process_data/augmentation.py
pointer only (licence: NONE) · get_code("d551e8dc744dd824")
split_train_eval Ran zhliu0106/learning-to-refuse/process_data/construction.py
pointer only (licence: NONE) · get_code("725ec8b577008818")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Large language models (LLMs) exhibit remarkable capabilities in understanding and generating natural language. However, these models can inadvertently memorize private information, posing significant privacy risks. This study addresses the challenge of enabling LLMs to protect specific individuals' private data without the need for complete retraining. We propose \return, a Real-world pErsonal daTa UnleaRNing dataset, comprising 2,492 individuals from Wikipedia with associated QA pairs, to evaluate machine unlearning (MU) methods for protecting personal data in a realistic scenario. Additionally, we introduce the Name-Aware Unlearning Framework (NAUF) for Privacy Protection, which enables the model to learn which individuals' information should be protected without affecting its ability to answer questions related to other unrelated individuals. Our extensive experiments demonstrate that NAUF achieves a state-of-the-art average unlearning score, surpassing the best baseline method by 5.65 points, effectively protecting target individuals' personal data while maintaining the model's general capabilities.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2407.10058")
get_code_for_paper("2407.10058")
have("2407.10058")

Connect an agent — have() is free.