Anh Nguyen, Michael Alcorn
We lifted 5 functions out of this paper's own repositories and ran 2 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| airalcorn2/deformer | canonical | 2 of 5 |
| Function | Status | Where it lives |
|---|---|---|
| DEformer | Ran | airalcorn2/deformer/deformer.py pointer only (licence: NONE) · get_code("6c121782411e0793") |
| generate_mask | Ran | airalcorn2/deformer/deformer.py pointer only (licence: NONE) · get_code("3237bc793452728d") |
| generate_mask | Not yet run | airalcorn2/deformer/deformer.py pointer only (licence: NONE) · get_code("b23e01cf590819cb") |
| generate_multi_mask | Not yet run | airalcorn2/deformer/deformer.py pointer only (licence: NONE) · get_code("8d35033845c6d813") |
| get_preds_labels | Not yet run | airalcorn2/deformer/train_deformer.py pointer only (licence: NONE) · get_code("daeff0bb3e842355") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Order-agnostic autoregressive distribution (density) estimation (OADE), i.e., autoregressive distribution estimation where the features can occur in an arbitrary order, is a challenging problem in generative machine learning. Prior work on OADE has encoded feature identity by assigning each feature to a distinct fixed position in an input vector. As a result, architectures built for these inputs must strategically mask either the input or model weights to learn the various conditional distributions necessary for inferring the full joint distribution of the dataset in an orderagnostic way. In this paper, we propose an alternative approach for encoding feature identities, where each feature's identity is included alongside its value in the input. This feature identity encoding strategy allows neural architectures designed for sequential data to be applied to the OADE task without modification. As a proof of concept, we show that a Transformer trained on this input (which we refer to as "the DEformer" 2 , i.e., the distribution estimating Transformer) can effectively model binarized-MNIST, approaching the performance of fixed-order autoregressive distribution estimating algorithms while still being entirely order-agnostic. Additionally, we find that the DEformer surpasses the performance of recent flow-based architectures when modeling a tabular dataset.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2106.06989")
get_code_for_paper("2106.06989")
have("2106.06989")
Connect an agent — have() is free.