Colin Raffel, U Berkeley, Nicholas Carlini, Florian Tramèr, Eric Wallace, Adam Roberts, Dawn Song, Katherine Lee, Matthew Jagielski, Ariel Herbert-Voss, Alina Oprea, Tom Brown, and 1 more
We lifted 4 functions out of this paper's own repositories and ran 3 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| ftramer/LM_Memorization | canonical | 3 of 3 |
| shreyansh26/Extracting-Training-Data-from-Large-Langauge-Models | reimplementation | 0 of 1 |
| Function | Status | Where it lives |
|---|---|---|
| calculatePerplexity | Ran | ftramer/LM_Memorization/extraction.py code served (permissive licence) · get_code("f21497e100719c5a") |
| parse_arguments | Ran | ftramer/LM_Memorization/extraction.py code served (permissive licence) · get_code("2a6129e2c84f477d") |
| parse_commoncrawl | Ran | ftramer/LM_Memorization/extraction.py code served (permissive licence) · get_code("7a30c79a1d5e467c") |
| calculate_perplexity | Not yet run | shreyansh26/Extracting-Training-Data-from-Large-Langauge-Models/extraction_top_n.py code served (permissive licence) · get_code("c59c1720bffdb7a5") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
It has become common to publish large (billion parameter) language models that have been trained on private datasets. This paper demonstrates that in such settings, an adversary can perform a training data extraction attack to recover individual training examples by querying the language model. We demonstrate our attack on GPT-2, a language model trained on scrapes of the public Internet, and are able to extract hundreds of verbatim text sequences from the model's training data. These extracted examples include (public) personally identifiable information (names, phone numbers, and email addresses), IRC conversations, code, and 128-bit UUIDs. Our attack is possible even though each of the above sequences are included in just one document in the training data. We comprehensively evaluate our extraction attack to understand the factors that contribute to its success. Worryingly, we find that larger models are more vulnerable than smaller models. We conclude by drawing lessons and discussing possible safeguards for training large language models.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2012.07805")
get_code_for_paper("2012.07805")
have("2012.07805")
Connect an agent — have() is free.