Dmitry Krotov, Cengiz Pehlevan, Jacob Zavatone-Veth, Hamza Chaudhry
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Sequence memory is an essential attribute of natural and artificial intelligence that enables agents to encode, store, and retrieve complex sequences of stimuli and actions. Computational models of sequence memory have been proposed where recurrent Hopfield-like neural networks are trained with temporally asymmetric Hebbian rules. However, these networks suffer from limited sequence capacity (maximal length of the stored sequence) due to interference between the memories. Inspired by recent work on Dense Associative Memories, we expand the sequence capacity of these models by introducing a nonlinear interaction term, enhancing separation between the patterns. We derive novel scaling laws for sequence capacity with respect to network size, significantly outperforming existing scaling laws for models based on traditional Hopfield networks, and verify these theoretical results with numerical simulation. Moreover, we introduce a generalized pseudoinverse rule to recall sequences of highly correlated patterns. Finally, we extend this model to store sequences with variable timing between states' transitions and describe a biologically-plausible implementation, with connections to motor neuroscience.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2306.04532")
get_code_for_paper("2306.04532")
have("2306.04532")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2306.04532.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2306.04532)
Connect an agent — have() is free.