Francesco Faccio, Jürgen Schmidhuber, Kazuki Irie
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Neural ordinary differential equations (ODEs) have attracted much attention as continuous-time counterparts of deep residual neural networks (NNs), and numerous extensions for recurrent NNs have been proposed. Since the 1980s, ODEs have also been used to derive theoretical results for NN learning rules, e.g., the famous connection between Oja's rule and principal component analysis. Such rules are typically expressed as additive iterative update processes which have straightforward ODE counterparts. Here we introduce a novel combination of learning rules and Neural ODEs to build continuous-time sequence processing nets that learn to manipulate short-term memory in rapidly changing synaptic connections of other nets. This yields continuous-time counterparts of Fast Weight Programmers and linear Transformers. Our novel models outperform the best existing Neural Controlled Differential Equation based models on various time series classification tasks, while also addressing their fundamental scalability limitations. Our code is public. 1
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2206.01649")
get_code_for_paper("2206.01649")
have("2206.01649")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2206.01649.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2206.01649)
Connect an agent — have() is free.