SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2207.01903 · NeurIPS · 2022

Betti numbers of attention graphs is all you really need

Laida Kushnareva, Dmitri Piontkovski, Irina Piontkovskaya

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

We apply methods of topological analysis to the attention graphs, calculated on the attention heads of the BERT model (Devlin et al. (2019)). Our research shows that the classifier built upon basic persistent topological features (namely, Betti numbers) of the trained neural network can achieve classification results on par with the conventional classification method. We show the relevance of such topological text representation on three text classification benchmarks. For the best of our knowledge, it is the first attempt to analyze the topology of an attention-based neural network, widely used for Natural Language Processing.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2207.01903")
get_code_for_paper("2207.01903")
have("2207.01903")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2207.01903.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2207.01903.svg)](https://syntology.ai/paper/2207.01903)

Connect an agent — have() is free.