We lifted 5 functions out of this paper's own repositories and ran 5 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| vwxyzjn/invalid-action-masking | canonical | 1 of 1 |
| Stable-Baselines-Team/stable-baselines3-contrib | pwc_unofficial | 3 of 3 |
| copy not recorded | — | 1 of 1 |
| Function | Status | Where it lives |
|---|---|---|
| conjugate_gradient_solver | Ran | Stable-Baselines-Team/stable-baselines3-contrib/sb3_contrib/common/utils.py code served (permissive licence) · get_code("b720173db270f08b") |
| discount_cumsum | Ran | vwxyzjn/invalid-action-masking/invalid_action_masking/ppo_10x10.py code served (permissive licence) · get_code("43887c9edd549830") |
| flat_grad | Ran | Stable-Baselines-Team/stable-baselines3-contrib/sb3_contrib/common/utils.py code served (permissive licence) · get_code("530d3c238c8e12f0") |
| quantile_huber_loss | Ran | Stable-Baselines-Team/stable-baselines3-contrib/sb3_contrib/common/utils.py code served (permissive licence) · get_code("8a3b5a2760bcc10a") |
| update_mean_var_count_from_moments | Ran | this paper's copy was not recorded; identical code first harvested from joonleesky/train-procgen-pytorch pointer only · get_code("3ebb420a5d10cc64") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
In recent years, Deep Reinforcement Learning (DRL) algorithms have achieved state-of-the-art performance in many challenging strategy games. Because these games have complicated rules, an action sampled from the full discrete action distribution predicted by the learned policy is likely to be invalid according to the game rules (e.g., walking into a wall). The usual approach to deal with this problem in policy gradient algorithms is to "mask out" invalid actions and just sample from the set of valid actions. The implications of this process, however, remain under-investigated. In this paper, we 1) show theoretical justification for such a practice, 2) empirically demonstrate its importance as the space of invalid actions grows, and 3) provide further insights by evaluating different action masking regimes, such as removing masking after an agent has been trained using masking. The source code can be found at https://github.com/vwxyzjn/invalid-action-masking
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2006.14171")
get_code_for_paper("2006.14171")
have("2006.14171")
Connect an agent — have() is free.