SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 1504.00941 · 2015

A Simple Way to Initialize Recurrent Networks of Rectified Linear Units

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Learning long term dependencies in recurrent networks is difficult due to vanishing and exploding gradients. To overcome this difficulty, researchers have developed sophisticated optimization techniques and network architectures. In this paper, we propose a simpler solution that use recurrent neural networks composed of rectified linear units. Key to our solution is the use of the identity matrix or its scaled version to initialize the recurrent weight matrix. We find that our solution is comparable to LSTM on our four benchmarks: two toy problems involving long-range temporal structures, a large language modeling problem and a benchmark speech recognition problem.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("1504.00941")
get_code_for_paper("1504.00941")
have("1504.00941")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/1504.00941.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/1504.00941.svg)](https://syntology.ai/paper/1504.00941)

Connect an agent — have() is free.