CIARD: Cyclic Iterative Adversarial Robustness Distillation

Xu Zheng, Xiang Gu, Yunhuai Liu, Liming Lu, Shuchao Pang, Anan Du, Yongbin Zhou

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 1 functions out of this paper's own repositories and ran 1 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

Repositories linked to this paper

• eminentgu/CIARD

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Adversarial robustness distillation (ARD) aims to transfer both performance and robustness from teacher model to lightweight student model, enabling resilient performance on resource-constrained scenarios. Though existing ARD approaches enhance student model's robustness, the inevitable by-product leads to the degraded performance on clean examples. We summarize the causes of this problem inherent in existing methods with dual-teacher framework as: 1 ⃝ The divergent optimization objectives of dualteacher models, i.e., the clean and robust teachers, impede effective knowledge transfer to the student model, and 2 ⃝ The iteratively generated adversarial examples during training lead to performance deterioration of the robust teacher model. To address these challenges, we propose a novel Cyclic Iterative ARD (CIARD) method with two key innovations: 1 ⃝ A multi-teacher framework with contrastive push-loss alignment to resolve conflicts in dualteacher optimization objectives, and 2 ⃝ Continuous adversarial retraining to maintain dynamic teacher robustness against performance degradation from the varying adversarial examples. Extensive experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet demonstrate that CIARD achieves remarkable performance with an average 3.53% improvement in adversarial defense rates across various attack scenarios and a 5.87% increase in clean sample accuracy, establishing a new benchmark for balancing model robustness and generalization. Our code is available at https://github.com/eminentgu/CIARD.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2509.12633")
get_code_for_paper("2509.12633")
have("2509.12633")

Connect an agent — have() is free.