Yi Su, Gerard De Melo, Xiang Hu, Zujie Wen, Haitao Mi, Jing Zheng, Yafang Wang
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Human language understanding operates at multiple levels of granularity (e.g., words, phrases, and sentences) with increasing levels of abstraction that can be hierarchically combined. However, existing deep models with stacked layers do not explicitly model any sort of hierarchical process. This paper proposes a recursive Transformer model based on differentiable CKY style binary trees to emulate the composition process. We extend the bidirectional language model pre-training objective to this architecture, attempting to predict each word given its left and right abstraction nodes. To scale up our approach, we also introduce an efficient pruned tree induction algorithm to enable encoding in just a linear number of composition steps. Experimental results on language modeling and unsupervised parsing show the effectiveness of our approach. 1 * Equal contribution.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2107.00967")
get_code_for_paper("2107.00967")
have("2107.00967")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2107.00967.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2107.00967)
Connect an agent — have() is free.