Stefano Ermon, Andy Shih, Dorsa Sadigh
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Conditional inference on arbitrary subsets of variables is a core problem in probabilistic inference with important applications such as masked language modeling and image inpainting. In recent years, the family of Any-Order Autoregressive Models (AO-ARMs) -closely related to popular models such as BERT and XL-Net -has shown breakthrough performance in arbitrary conditional tasks across a sweeping range of domains. But, in spite of their success, in this paper we identify significant improvements to be made to previous formulations of AO-ARMs. First, we show that AO-ARMs suffer from redundancy in their probabilistic model, i.e., they define the same distribution in multiple different ways. We alleviate this redundancy by training on a smaller set of univariate conditionals that still maintains support for efficient arbitrary conditional inference. Second, we upweight the training loss for univariate conditionals that are evaluated more frequently during inference. Our method leads to improved performance with no compromises on tractability, giving state-of-the-art likelihoods in arbitrary conditional modeling on text (Text8), image (CIFAR10, ImageNet32), and continuous tabular data domains. 36th Conference on Neural Information Processing Systems (NeurIPS 2022).
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2205.13554")
get_code_for_paper("2205.13554")
have("2205.13554")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2205.13554.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2205.13554)
Connect an agent — have() is free.