Oleksandr Polozov, Chia-Hsuan Lee, Matthew Richardson
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
The goal of database question answering is to enable natural language querying of real-life relational databases in diverse application domains. Recently, large-scale datasets such as Spider and WikiSQL facilitated novel modeling techniques for text-to-SQL parsing, improving zero-shot generalization to unseen databases. In this work, we examine the challenges that still prevent these techniques from practical deployment. First, we present KaggleDBQA, a new cross-domain evaluation dataset of real Web databases, with domain-specific data types, original formatting, and unrestricted questions. Second, we re-examine the choice of evaluation tasks for text-to-SQL parsers as applied in real-life settings. Finally, we augment our in-domain evaluation task with database documentation, a naturally occurring source of implicit domain knowledge. We show that KaggleDBQA presents a challenge to state-ofthe-art zero-shot parsers but a more realistic evaluation setting and creative use of associated database documentation boosts their accuracy by over 13.2%, doubling their performance.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2106.11455")
get_code_for_paper("2106.11455")
have("2106.11455")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2106.11455.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2106.11455)
Connect an agent — have() is free.