Chao Chen, Songzhu Zheng, Tengfei Ma, Weimin Lyu
We lifted 1 functions out of this paper's own repositories and ran 1 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| weimin17/attention_abnormality_in_trojaned_berts | — | 1 of 1 |
| Function | Status | Where it lives |
|---|---|---|
| identify_trigger_head | Ran | weimin17/attention_abnormality_in_trojaned_berts/attn_utils.py pointer only (licence: NONE) · get_code("e36331a1e43a19f5") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Trojan attacks raise serious security concerns. In this paper, we investigate the underlying mechanism of Trojaned BERT models. We observe the attention focus drifting behavior of Trojaned models, i.e., when encountering an poisoned input, the trigger token hijacks the attention focus regardless of the context. We provide a thorough qualitative and quantitative analysis of this phenomenon, revealing insights into the Trojan mechanism. Based on the observation, we propose an attention-based Trojan detector to distinguish Trojaned models from clean ones. To the best of our knowledge, this is the first paper to analyze the Trojan mechanism and to develop a Trojan detector based on the transformer's attention 1 .
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2205.08305")
get_code_for_paper("2205.08305")
have("2205.08305")
Connect an agent — have() is free.