Boxing Chen, Derek Wong, Lidia Chao, Dayiheng Liu, Baosong Yang, Rong Xiao, Yu Wan, Haibo Zhang
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Attention mechanism has become the dominant module in natural language processing models. It is computationally intensive and depends on massive power-hungry multiplications. In this paper, we rethink variants of attention mechanism from the energy consumption aspects. After reaching the conclusion that the energy costs of several energyfriendly operations are far less than their multiplication counterparts, we build a novel attention model by replacing multiplications with either selective operations or additions. Empirical results on three machine translation tasks demonstrate that the proposed model, against the vanilla one, achieves competitable accuracy while saving 99% and 66% energy during alignment calculation and the whole attention procedure. Code is available at: https://github.com/NLP2CT/E-Att.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2204.13353")
get_code_for_paper("2204.13353")
have("2204.13353")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2204.13353.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2204.13353)
Connect an agent — have() is free.