We lifted 4 functions out of this paper's own repositories and ran 0 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| mchong6/FID_IS_infinity | canonical | 0 of 4 |
| Function | Status | Where it lives |
|---|---|---|
| calculate_FID_infinity | Not yet run | mchong6/FID_IS_infinity/score_infinity.py code served (permissive licence) · get_code("1938147133587a22") |
| calculate_FID_infinity_path | Not yet run | mchong6/FID_IS_infinity/score_infinity.py code served (permissive licence) · get_code("1c83672ee4fd3628") |
| calculate_IS_infinity | Not yet run | mchong6/FID_IS_infinity/score_infinity.py code served (permissive licence) · get_code("aadcb9db697e528d") |
| load_inception_net | Not yet run | mchong6/FID_IS_infinity/inception.py code served (permissive licence) · get_code("28d83c8f5084a8e1") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
This paper shows that two commonly used evaluation metrics for generative models, the Fréchet Inception Distance (FID) and the Inception Score (IS), are biased -- the expected value of the score computed for a finite sample set is not the true value of the score. Worse, the paper shows that the bias term depends on the particular model being evaluated, so model A may get a better score than model B simply because model A's bias term is smaller. This effect cannot be fixed by evaluating at a fixed number of samples. This means all comparisons using FID or IS as currently computed are unreliable. We then show how to extrapolate the score to obtain an effectively bias-free estimate of scores computed with an infinite number of samples, which we term $\overline{\textrm{FID}}_\infty$ and $\overline{\textrm{IS}}_\infty$. In turn, this effectively bias-free estimate requires good estimates of scores with a finite number of samples. We show that using Quasi-Monte Carlo integration notably improves estimates of FID and IS for finite sample sets. Our extrapolated scores are simple, drop-in replacements for the finite sample scores. Additionally, we show that using low discrepancy sequence in GAN training offers small improvements in the resulting generator.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("1911.07023")
get_code_for_paper("1911.07023")
have("1911.07023")
Connect an agent — have() is free.