We lifted 6 functions out of this paper's own repositories and ran 3 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| elicassion/StARformer | canonical | 3 of 6 |
| Function | Status | Where it lives |
|---|---|---|
| discount_cumsum | Ran | elicassion/StARformer/gym/experiment.py code served (permissive licence) · get_code("0161b27cbe3cc58d") |
| rgb2gray | Ran | elicassion/StARformer/atari_and_dmc/run_star_dmc.py code served (permissive licence) · get_code("9dacc5f0ddc36c98") |
| top_k_logits | Ran | elicassion/StARformer/atari_and_dmc/utils.py code served (permissive licence) · get_code("0f0021cdea13e4da") |
| evaluate_episode | Not yet run | elicassion/StARformer/gym/evaluation/evaluate_episodes.py code served (permissive licence) · get_code("5d44a44afbb9c4ba") |
| evaluate_episode_rtg | Not yet run | elicassion/StARformer/gym/evaluation/evaluate_episodes.py code served (permissive licence) · get_code("db29bf553fde82c3") |
| load_tf_weights_in_gpt2 | Not yet run | elicassion/StARformer/gym/models/trajectory_gpt2.py code served (permissive licence) · get_code("00a33466c69c5705") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Reinforcement Learning (RL) can be considered as a sequence modeling task: given a sequence of past state-action-reward experiences, an agent predicts a sequence of next actions. In this work, we propose State-Action-Reward Transformer (StARformer) for visual RL, which explicitly models short-term state-action-reward representations (StAR-representations), essentially introducing a Markovian-like inductive bias to improve long-term modeling. Our approach first extracts StAR-representations by self-attending image state patches, action, and reward tokens within a short temporal window. These are then combined with pure image state representations -- extracted as convolutional features, to perform self-attention over the whole sequence. Our experiments show that StARformer outperforms the state-of-the-art Transformer-based method on image-based Atari and DeepMind Control Suite benchmarks, in both offline-RL and imitation learning settings. StARformer is also more compliant with longer sequences of inputs. Our code is available at https://github.com/elicassion/StARformer.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2110.06206")
get_code_for_paper("2110.06206")
have("2110.06206")
Connect an agent — have() is free.