Language-Agnostic Representation Learning of Source Code from Structure and Context

Jure Leskovec, Stephan Ünnemann, Daniel Ügner, Tobias Kirschstein, Michele Catasta

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 8 functions out of this paper's own repositories and ran 6 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

Repositories linked to this paper

• danielzuegner/code-transformer — marked official in Papers with Code

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Source code (Context) and its parsed abstract syntax tree (AST; Structure) are two complementary representations of the same computer program. Traditionally, designers of machine learning models have relied predominantly either on Structure or Context. We propose a new model, which jointly learns on Context and Structure of source code. In contrast to previous approaches, our model uses only language-agnostic features, i.e., source code and features that can be computed directly from the AST. Besides obtaining state-of-the-art on monolingual code summarization on all five programming languages considered in this work, we propose the first multilingual code summarization model. We show that jointly training on non-parallel data from multiple programming languages improves results on all individual languages, where the strongest gains are on low-resource languages. Remarkably, multilingual training only from Context does not lead to the same improvements, highlighting the benefits of combining Structure and Context for representation learning on code.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2103.11318")
get_code_for_paper("2103.11318")
have("2103.11318")

Connect an agent — have() is free.