Ole Winther, H Kim, Valentin Liévin, Andreas Geert Motzfeldt, Ida Jensen, D Yang, E Chang, E Kim, F Martínez-Plumed, De Melo, G Kruszewski, G Wang, and 7 more
We lifted 1 functions out of this paper's own repositories and ran 1 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| VodLM/vod | canonical | 1 of 1 |
| Function | Status | Where it lives |
|---|---|---|
| combine_datasets | Ran | VodLM/vod/src/vod_datasets/postprocessing.py code served (permissive licence) · get_code("0c2a3c350e11f679") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Retrieval-augmented models have proven to be effective in natural language processing tasks, yet there remains a lack of research on their optimization using variational inference. We introduce the Variational Open-Domain (VOD) framework for end-to-end training and evaluation of retrieval-augmented models, focusing on opendomain question answering and language modelling. The VOD objective, a self-normalized estimate of the Rényi variational bound, approximates the task marginal likelihood and is evaluated under samples drawn from an auxiliary sampling distribution (cached retriever and/or approximate posterior). It remains tractable, even for retriever distributions defined on large corpora. We demonstrate VOD's versatility by training readerretriever BERT-sized models on multiple-choice medical exam questions. On the MedMCQA dataset, we outperform the domain-tuned Med-PaLM by +5.3% despite using 2.500× fewer parameters. Our retrieval-augmented BioLinkBERT model scored 62.9% on the MedMCQA and 55.0% on the MedQA-USMLE. Last, we show the effectiveness of our learned retriever component in the context of medical semantic search.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2210.06345")
get_code_for_paper("2210.06345")
have("2210.06345")
Connect an agent — have() is free.