SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2007.15135 · 2020

The Return of Lexical Dependencies: Neural Lexicalized PCFGs

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 4 functions out of this paper's own repositories and ran 3 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

RepositoryRoleRan
zhaoyanpeng/xcfg extension 3 of 4
FunctionStatusWhere it lives
build_spans Ran zhaoyanpeng/xcfg/xcfg/data/baseline.py
pointer only (licence: NONE) · get_code("5535820241c1032e")
get_stats Ran zhaoyanpeng/xcfg/xcfg/data/baseline.py
pointer only (licence: NONE) · get_code("d4b3f50524ef7245")
random_tree Ran zhaoyanpeng/xcfg/xcfg/data/baseline.py
pointer only (licence: NONE) · get_code("d2a5db5244d18f43")
remove_morph_feature Not yet run zhaoyanpeng/xcfg/xcfg/data/treebank.py
pointer only (licence: NONE) · get_code("d733a35f871f4e54")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

In this paper we demonstrate that $\textit{context free grammar (CFG) based methods for grammar induction benefit from modeling lexical dependencies}$. This contrasts to the most popular current methods for grammar induction, which focus on discovering $\textit{either}$ constituents $\textit{or}$ dependencies. Previous approaches to marry these two disparate syntactic formalisms (e.g. lexicalized PCFGs) have been plagued by sparsity, making them unsuitable for unsupervised grammar induction. However, in this work, we present novel neural models of lexicalized PCFGs which allow us to overcome sparsity problems and effectively induce both constituents and dependencies within a single model. Experiments demonstrate that this unified framework results in stronger results on both representations than achieved when modeling either formalism alone. Code is available at https://github.com/neulab/neural-lpcfg.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2007.15135")
get_code_for_paper("2007.15135")
have("2007.15135")

Connect an agent — have() is free.