Huan Sun, Yu Su, Xiang Deng, You Wu, Alyssa Lees, Yu Cong
We lifted 11 functions out of this paper's own repositories and ran 3 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| sunlab-osu/reasonbert | canonical | 3 of 11 |
| Function | Status | Where it lives |
|---|---|---|
| exact_match_score | Ran | sunlab-osu/reasonbert/model/metric.py code served (permissive licence) · get_code("f6c275d6a18330a9") |
| f1_score | Ran | sunlab-osu/reasonbert/model/metric.py code served (permissive licence) · get_code("fdd9a4d20f5a9d6f") |
| normalize_answer | Ran | sunlab-osu/reasonbert/model/metric.py code served (permissive licence) · get_code("c6a80c065d2e4851") |
| MRQA_preprocess | Not yet run | sunlab-osu/reasonbert/data_loader/data_loaders.py code served (permissive licence) · get_code("ac2fa2763f9ff92e") |
| cross_entropy_with_onehot | Not yet run | sunlab-osu/reasonbert/model/model.py code served (permissive licence) · get_code("46aaba4ebd9f132b") |
| flatten | Not yet run | sunlab-osu/reasonbert/data_loader/tapas_modelling.py code served (permissive licence) · get_code("c2c2e8507e9edd19") |
| gather | Not yet run | sunlab-osu/reasonbert/data_loader/tapas_modelling.py code served (permissive licence) · get_code("ead963170369d948") |
| get_linear_schedule_with_warmup | Not yet run | sunlab-osu/reasonbert/model/loss.py code served (permissive licence) · get_code("afba2c06a42074fe") |
| get_tokenized_loc | Not yet run | sunlab-osu/reasonbert/data_loader/data_loaders.py code served (permissive licence) · get_code("8c5fc643936371bd") |
| nll_loss | Not yet run | sunlab-osu/reasonbert/model/loss.py code served (permissive licence) · get_code("8c2db3d4435816c0") |
| tmp_func | Not yet run | sunlab-osu/reasonbert/data_loader/data_loaders.py code served (permissive licence) · get_code("dd644affb7c645e2") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
We present ReasonBERT, a pre-training method that augments language models with the ability to reason over long-range relations and multiple, possibly hybrid, contexts. Unlike existing pre-training methods that only harvest learning signals from local contexts of naturally occurring texts, we propose a generalized notion of distant supervision to automatically connect multiple pieces of text and tables to create pre-training examples that require long-range reasoning. Different types of reasoning are simulated, including intersecting multiple pieces of evidence, bridging from one piece of evidence to another, and detecting unanswerable cases. We conduct a comprehensive evaluation on a variety of extractive question answering datasets ranging from single-hop to multi-hop and from text-only to table-only to hybrid that require various reasoning capabilities and show that ReasonBERT achieves remarkable improvement over an array of strong baselines. Fewshot experiments further demonstrate that our pre-training method substantially improves sample efficiency. 1
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2109.04912")
get_code_for_paper("2109.04912")
have("2109.04912")
Connect an agent — have() is free.