Wei Wu, Yuhao Zhou, Qi Zhang, Xuanjing Huang, Rong Bao, Rui Zheng, Tao Gui, Di Liang, Sirui Wang
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. If it links a repository, it is listed below.
Recent works on Lottery Ticket Hypothesis have shown that pre-trained language models (PLMs) contain smaller matching subnetworks (winning tickets) which are capable of reaching accuracy comparable to the original models. However, these tickets are proved to be not robust to adversarial examples, and even worse than their PLM counterparts. To address this problem, we propose a novel method based on learning binary weight masks to identify robust tickets hidden in the original PLMs. Since the loss is not differentiable for the binary mask, we assign the hard concrete distribution to the masks and encourage their sparsity using a smoothing approximation of L 0 regularization. Furthermore, we design an adversarial loss objective to guide the search for robust tickets and ensure that the tickets perform well both in accuracy and robustness. Experimental results show the significant improvement of the proposed method over previous work on adversarial robustness evaluation.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2211.03013")
get_code_for_paper("2211.03013")
have("2211.03013")
Connect an agent — have() is free.