OpenTab: Advancing Large Language Models as Open-domain Table Reasoners

Kezhi Kong, Balasubramaniam Srinivasan, George Karypis, Christos Faloutsos, Huzefa Rangwala, Jiani Zhang, Zhengyuan Shen, Chuan Lei

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 5 functions out of this paper's own repositories and ran 5 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

Repositories linked to this paper

• amazon-science/llm-open-domain-table-reasoner — marked official in Papers with Code; named in the paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Large Language Models (LLMs) trained on large volumes of data excel at various natural language tasks, but they cannot handle tasks requiring knowledge that has not been trained on previously. One solution is to use a retriever that fetches relevant information to expand LLM's knowledge scope. However, existing textual-oriented retrieval-based LLMs are not ideal on structured table data due to diversified data modalities and large table sizes. In this work, we propose OPENTAB, an open-domain table reasoning framework powered by LLMs. Overall, OPENTAB leverages table retriever to fetch relevant tables and then generates SQL programs to parse the retrieved tables efficiently. Utilizing the intermediate data derived from the SQL executions, it conducts grounded inference to produce accurate response. Extensive experimental evaluation shows that OPENTAB significantly outperforms baselines in both open-and closed-domain settings, achieving up to 21.5% higher accuracy. We further run ablation studies to validate the efficacy of our proposed designs of the system. We open source our implementation at https://github.com/amazon-science/ llm-open-domain-table-reasoner.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2402.14361")
get_code_for_paper("2402.14361")
have("2402.14361")

Connect an agent — have() is free.