SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2110.06388 · EMNLP · 2021

HETFORMER: Heterogeneous Transformer with Sparse Attention for Long-Text Extractive Summarization

Lifang He, Philip Yu, Congying Xia, Yao Wan, Ye Liu, Jian-Guo Zhang

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

To capture the semantic graph structure from raw text, most existing summarization approaches are built on GNNs with a pre-trained model. However, these methods suffer from cumbersome procedures and inefficient computations for long-text documents. To mitigate these issues, this paper proposes HET-FORMER, a Transformer-based pre-trained model with multi-granularity sparse attentions for long-text extractive summarization. Specifically, we model different types of semantic nodes in raw text as a potential heterogeneous graph and directly learn heterogeneous relationships (edges) among nodes by Transformer. Extensive experiments on both single-and multi-document summarization tasks show that HETFORMER achieves stateof-the-art performance in Rouge F1 while using less memory and fewer parameters.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2110.06388")
get_code_for_paper("2110.06388")
have("2110.06388")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2110.06388.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2110.06388.svg)](https://syntology.ai/paper/2110.06388)

Connect an agent — have() is free.