We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
This research note combines two methods that have recently improved the state of the art in language modeling: Transformers and dynamic evaluation. Transformers use stacked layers of self-attention that allow them to capture long range dependencies in sequential data. Dynamic evaluation fits models to the recent sequence history, allowing them to assign higher probabilities to re-occurring sequential patterns. By applying dynamic evaluation to Transformer-XL models, we improve the state of the art on enwik8 from 0.99 to 0.94 bits/char, text8 from 1.08 to 1.04 bits/char, and WikiText-103 from 18.3 to 16.4 perplexity points.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("1904.08378")
get_code_for_paper("1904.08378")
have("1904.08378")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/1904.08378.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/1904.08378)
Connect an agent — have() is free.