Man Yung, Russell Wong
We lifted 16 functions out of this paper's own repositories and ran 13 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| russellwmy/affinity-is-not-enough | canonical | 13 of 16 |
| Function | Status | Where it lives |
|---|---|---|
| compute_metrics | Ran | russellwmy/affinity-is-not-enough/prototype/routing_entropy.py code served (permissive licence) · get_code("2ddba7ae7e49d0cf") |
| domain_signal | Ran | russellwmy/affinity-is-not-enough/prototype/beta_routing.py code served (permissive licence) · get_code("773ddf39c6cda539") |
| fmt | Ran | russellwmy/affinity-is-not-enough/prototype/format_tables.py code served (permissive licence) · get_code("28cb7991e57a1649") |
| get_noise_stds | Ran | russellwmy/affinity-is-not-enough/prototype/precision_gating.py code served (permissive licence) · get_code("7a78fcbfc4e835a3") |
| identify_domain_experts | Ran | russellwmy/affinity-is-not-enough/prototype/lm_experiment.py code served (permissive licence) · get_code("70b48a866e60e61f") |
| load | Ran | russellwmy/affinity-is-not-enough/prototype/format_tables.py code served (permissive licence) · get_code("b226d3e0e0e56328") |
| make_batch | Ran | russellwmy/affinity-is-not-enough/prototype/ablation.py code served (permissive licence) · get_code("898413a5c9423425") |
| make_early_signal_batch | Ran | russellwmy/affinity-is-not-enough/prototype/beta_routing.py code served (permissive licence) · get_code("948ec14415a90164") |
| make_switch_batch | Ran | russellwmy/affinity-is-not-enough/prototype/beta_routing.py code served (permissive licence) · get_code("6ab8b5b26264ed89") |
| mean_std | Ran | russellwmy/affinity-is-not-enough/prototype/ablation.py code served (permissive licence) · get_code("b2a4256fd27b02e6") |
| sample_domain_segment | Ran | russellwmy/affinity-is-not-enough/prototype/lm_experiment.py code served (permissive licence) · get_code("bddd134c3514c8b3") |
| train_anticipatory | Ran | russellwmy/affinity-is-not-enough/prototype/anticipatory_routing.py code served (permissive licence) · get_code("f254faf20e6dab90") |
| train_standard | Ran | russellwmy/affinity-is-not-enough/prototype/anticipatory_routing.py code served (permissive licence) · get_code("231601653d8c8552") |
| emit_inline_numbers | Not yet run | russellwmy/affinity-is-not-enough/prototype/format_tables.py code served (permissive licence) · get_code("04df650672f71678") |
| evaluate | Not yet run | russellwmy/affinity-is-not-enough/prototype/routing_entropy.py code served (permissive licence) · get_code("9b9d4654d1049eb5") |
| train | Not yet run | russellwmy/affinity-is-not-enough/prototype/ablation.py code served (permissive licence) · get_code("8224a4d3a0cba015") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Sparse Mixture-of-Experts (MoE) routing fails at exactly the points where routing matters most -domain transitions where the current token belongs to one distribution and the next belongs to another. In a controlled routing experiment (4 experts, 5 seeds), standard affinity-based routing assigns only 0.006 ± 0.001 probability mass to the correct expert at the transition step. We show that three lightweight modifications to the routing gate raise this to 0.748 ± 0.002 -a 124× increase, reducing the experts needed for 99% routing coverage from infeasible to a small constant: temporal memory (β) -a per-expert LIF membrane potential that accumulates routing context across tokens; precision-weighted gating (Π) -per-expert inverse variance of recent prediction error, yielding a 31× contrast between reliable and unreliable experts; and anticipatory routing -a next-state predictor conditioned on the β-accumulated hidden state. The three mechanisms are motivated by Friston's Free Energy Principle and instantiated using LIF dynamics from spiking neural networks. An ablation across all 2 3 mechanism subsets reveals a super-additive β × Ant interaction: anticipation alone gives nothing (+0.000 ± 0.001); β alone gives modest gain (+0.295 ± 0.013); combined they close 75% of the oracle gap (+0.741 ± 0.002, exceeding the sum of individual gains by +0.446 ± 0.014). This is a structural finding -a stateless predictor cannot detect approaching domain transitions, because pre-transition tokens are distributionally identical to within-domain tokens. In a character-level MoE language model (5 seeds), β-routing reduces BPC at the transition step from 6.56 ± 0.01 (Standard MoE) to 4.01 ± 0.15 (β-MoE); the β + Ant gate places 0.86 ± 0.02 probability on the correct domain expert before that domain becomes visible in the input, versus 0.42 ± 0.12 for Standard MoE. Reference implementations are ∼200 lines each and released as prototype/ alongside the paper.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2605.00604")
get_code_for_paper("2605.00604")
have("2605.00604")
Connect an agent — have() is free.