SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 1805.02867 · 2018

Online normalizer calculation for softmax

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

The Softmax function is ubiquitous in machine learning, multiple previous works suggested faster alternatives for it. In this paper we propose a way to compute classical Softmax with fewer memory accesses and hypothesize that this reduction in memory accesses should improve Softmax performance on actual hardware. The benchmarks confirm this hypothesis: Softmax accelerates by up to 1.3x and Softmax+TopK combined and fused by up to 5x.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("1805.02867")
get_code_for_paper("1805.02867")
have("1805.02867")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/1805.02867.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/1805.02867.svg)](https://syntology.ai/paper/1805.02867)

Connect an agent — have() is free.