Zhiyu Chen, Wenhu Chen, William Wang, Hong Wang, Hanwen Zha, Wenhan Xiong
We lifted 15 functions out of this paper's own repositories and ran 12 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| wenhuchen/HybridQA | canonical | 3 of 3 |
| teacherpeterpan/Unsupervised-Multi-hop-QA | pwc_unofficial | 7 of 10 |
| copy not recorded | — | 2 of 2 |
| Function | Status | Where it lives |
|---|---|---|
| compute_exact | Ran | wenhuchen/HybridQA/evaluate_script.py code served (permissive licence) · get_code("9026d9da5d085bd4") |
| get_bridge_entities | Ran | teacherpeterpan/Unsupervised-Multi-hop-QA/MQA_QG/HybridQA/operations.py code served (permissive licence) · get_code("a745217943d329bd") |
| get_tokens | Ran | this paper's copy was not recorded; identical code first harvested from saptarshi059/cdqa-project pointer only · get_code("aa6bc32d9c17a2f1") |
| include_bridge_entity | Ran | teacherpeterpan/Unsupervised-Multi-hop-QA/MQA_QG/HotpotQA/operations.py code served (permissive licence) · get_code("c7994a1f51a22564") |
| isNaN | Ran | wenhuchen/HybridQA/train_stage12.py code served (permissive licence) · get_code("86df5eac4985fe0e") |
| load_all_questions | Ran | teacherpeterpan/Unsupervised-Multi-hop-QA/MQA_QG/run_ppl_ranking.py code served (permissive licence) · get_code("199bd77e59857d70") |
| load_hotpotQA | Ran | teacherpeterpan/Unsupervised-Multi-hop-QA/prep_data_hotpotQA.py code served (permissive licence) · get_code("b8a1108847521ccb") |
| normalize_answer | Ran | this paper's copy was not recorded; identical code first harvested from saptarshi059/cdqa-project pointer only · get_code("6a96435eba311b08") |
| phrase_match | Ran | teacherpeterpan/Unsupervised-Multi-hop-QA/MQA_QG/HotpotQA/operations.py code served (permissive licence) · get_code("fd256c34b37a0e87") |
| phrase_overlap | Ran | teacherpeterpan/Unsupervised-Multi-hop-QA/MQA_QG/HybridQA/utils.py code served (permissive licence) · get_code("8a1e9b4aa699789a") |
| title_match | Ran | teacherpeterpan/Unsupervised-Multi-hop-QA/prep_data_hotpotQA.py code served (permissive licence) · get_code("8f0d111c90b80c13") |
| url2text | Ran | wenhuchen/HybridQA/train_stage12.py code served (permissive licence) · get_code("38c3f4fa8e089762") |
| construct_sample | Not yet run | teacherpeterpan/Unsupervised-Multi-hop-QA/MQA_QG/run_multihop_generation.py code served (permissive licence) · get_code("f1a4954ff85e0b97") |
| convert_question_into_desc | Not yet run | teacherpeterpan/Unsupervised-Multi-hop-QA/MQA_QG/HybridQA/operations.py code served (permissive licence) · get_code("79bc6415d0faf9c6") |
| get_passage | Not yet run | teacherpeterpan/Unsupervised-Multi-hop-QA/MQA_QG/HybridQA/operations.py code served (permissive licence) · get_code("04a978d51168a6db") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Existing question answering datasets focus on dealing with homogeneous information, based either only on text or KB/Table information alone. However, as human knowledge is distributed over heterogeneous forms, using homogeneous information alone might lead to severe coverage problems. To fill in the gap, we present HybridQA 1 , a new large-scale question-answering dataset that requires reasoning on heterogeneous information. Each question is aligned with a Wikipedia table and multiple free-form corpora linked with the entities in the table. The questions are designed to aggregate both tabular information and text information, i.e., lack of either form would render the question unanswerable. We test with three different models: 1) a table-only model. 2) text-only model. 3) a hybrid model that combines heterogeneous information to find the answer. The experimental results show that the EM scores obtained by two baselines are below 20%, while the hybrid model can achieve an EM over 40%. This gap suggests the necessity to aggregate heterogeneous information in HybridQA. However, the hybrid model's score is still far behind human performance. Hence, HybridQA can serve as a challenging benchmark to study question answering with heterogeneous information.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2004.07347")
get_code_for_paper("2004.07347")
have("2004.07347")
Connect an agent — have() is free.