We lifted 8 functions out of this paper's own repositories and ran 5 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| google-research/mnist-c | canonical | 1 of 1 |
| testingautomated-usi/fashion-mnist-c | pwc_unofficial | 4 of 6 |
| copy not recorded | — | 0 of 1 |
| Function | Status | Where it lives |
|---|---|---|
| clipped_zoom | Ran | google-research/mnist-c/corruptions.py code served (permissive licence) · get_code("f210d4ac7d4969d7") |
| disk | Ran | testingautomated-usi/fashion-mnist-c/mnist_c.py code served (permissive licence) · get_code("07a9b58ec8d910eb") |
| flip_sides | Ran | testingautomated-usi/fashion-mnist-c/additional_corruptions.py code served (permissive licence) · get_code("3f1558360c090b44") |
| flip_up_down | Ran | testingautomated-usi/fashion-mnist-c/additional_corruptions.py code served (permissive licence) · get_code("01cd0fbb90369028") |
| turn_left | Ran | testingautomated-usi/fashion-mnist-c/additional_corruptions.py code served (permissive licence) · get_code("20f8669ba3834517") |
| generate_mix_dataset | Not yet run | testingautomated-usi/fashion-mnist-c/generator.py code served (permissive licence) · get_code("712d7aea3437c370") |
| line_from_points | Not yet run | testingautomated-usi/fashion-mnist-c/mnist_c.py code served (permissive licence) · get_code("9f67f6434ab46c83") |
| plasma_fractal | Not yet run | this paper's copy was not recorded; identical code first harvested from hendrycks/robustness pointer only · get_code("3fe212e00cbe6fb3") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
We introduce the MNIST-C dataset, a comprehensive suite of 15 corruptions applied to the MNIST test set, for benchmarking out-of-distribution robustness in computer vision. Through several experiments and visualizations we demonstrate that our corruptions significantly degrade performance of state-of-the-art computer vision models while preserving the semantic content of the test images. In contrast to the popular notion of adversarial robustness, our model-agnostic corruptions do not seek worst-case performance but are instead designed to be broad and diverse, capturing multiple failure modes of modern models. In fact, we find that several previously published adversarial defenses significantly degrade robustness as measured by MNIST-C. We hope that our benchmark serves as a useful tool for future work in designing systems that are able to learn robust feature representations that capture the underlying semantics of the input.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("1906.02337")
get_code_for_paper("1906.02337")
have("1906.02337")
Connect an agent — have() is free.