Bahram Zonooz, Elahe Arani, Prashant Bhat, Kishaan Jeeveswaran, Nat Robustness
We lifted 4 functions out of this paper's own repositories and ran 3 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| NeurAI-Lab/BiRT | canonical | 1 of 1 |
| neurai-lab/birt | — | 2 of 3 |
| Function | Status | Where it lives |
|---|---|---|
| BiRT | Ran | neurai-lab/birt/continual/birt.py code served (permissive licence) · get_code("ef9e47a1ad77522f") |
| ContinualClassifier | Ran | neurai-lab/birt/continual/birt.py code served (permissive licence) · get_code("195ac27332a19667") |
| eval_training_finetuning | Ran | NeurAI-Lab/BiRT/continual/birt.py code served (permissive licence) · get_code("d00cae076f919bb2") |
| freeze_parameters | Not yet run | neurai-lab/birt/continual/birt.py code served (permissive licence) · get_code("6fb6b6f1f48b2180") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
The ability of deep neural networks to continually learn and adapt to a sequence of tasks has remained challenging due to catastrophic forgetting of previously learned tasks. Humans, on the other hand, have a remarkable ability to acquire, assimilate, and transfer knowledge across tasks throughout their lifetime without catastrophic forgetting. The versatility of the brain can be attributed to the rehearsal of abstract experiences through a complementary learning system. However, representation rehearsal in vision transformers lacks diversity, resulting in overfitting and consequently, performance drops significantly compared to raw image rehearsal. Therefore, we propose BiRT, a novel representation rehearsal-based continual learning approach using vision transformers. Specifically, we introduce constructive noises at various stages of the vision transformer and enforce consistency in predictions with respect to an exponential moving average of the working model. Our method provides consistent performance gain over raw image and vanilla representation rehearsal on several challenging CL benchmarks, while being memory efficient and robust to natural and adversarial corruptions. 1
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2305.04769")
get_code_for_paper("2305.04769")
have("2305.04769")
Connect an agent — have() is free.