Pratyush Maini, Danish Pruthi, Keshav Kolluru
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Pooling-based recurrent neural architectures consistently outperform their counterparts without pooling on sequence classification tasks. However, the reasons for their enhanced performance are largely unexamined. In this work, we explore three commonly used pooling techniques (mean-pooling, max-pooling, and attention 1 ), and propose max-attention, a novel variant that captures interactions among predictive tokens in a sentence. Using novel experiments, we demonstrate that pooling architectures substantially differ from their nonpooling equivalents in their learning ability and positional biases: (i) pooling facilitates better gradient flow than BiLSTMs in initial training epochs, and (ii) BiLSTMs are biased towards tokens at the beginning and end of the input, whereas pooling alleviates this bias. Consequently, we find that pooling yields large gains in low resource scenarios, and instances when salient words lie towards the middle of the input. Across several text classification tasks, we find max-attention to frequently outperform other pooling techniques. 2
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2005.00159")
get_code_for_paper("2005.00159")
have("2005.00159")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2005.00159.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2005.00159)
Connect an agent — have() is free.