SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 1511.04581 · 2015

A Test of Relative Similarity For Model Selection in Generative Models

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 3 functions out of this paper's own repositories and ran 1 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

RepositoryRoleRan
eugenium/MMD canonical 1 of 3
FunctionStatusWhere it lives
grbf Ran eugenium/MMD/mmd.py
pointer only (licence: NONE) · get_code("36f96addab29ba19")
MMD_3_Sample_Test Not yet run eugenium/MMD/mmd.py
pointer only (licence: NONE) · get_code("3cc92f05cbac7f13")
MMD_Diff_Var Not yet run eugenium/MMD/mmd.py
pointer only (licence: NONE) · get_code("4fa0660abad83e3e")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Probabilistic generative models provide a powerful framework for representing data that avoids the expense of manual annotation typically needed by discriminative approaches. Model selection in this generative setting can be challenging, however, particularly when likelihoods are not easily accessible. To address this issue, we introduce a statistical test of relative similarity, which is used to determine which of two models generates samples that are significantly closer to a real-world reference dataset of interest. We use as our test statistic the difference in maximum mean discrepancies (MMDs) between the reference dataset and each model dataset, and derive a powerful, low-variance test based on the joint asymptotic distribution of the MMDs between each reference-model pair. In experiments on deep generative models, including the variational auto-encoder and generative moment matching network, the tests provide a meaningful ranking of model performance as a function of parameter and training settings.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("1511.04581")
get_code_for_paper("1511.04581")
have("1511.04581")

Connect an agent — have() is free.