SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2409.16040 · ICLR · 2025

Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts

Ming Jin, Shiyu Wang, Qingsong Wen, Xiaoming Shi, Yuqi Nie, Dianqi Li, Zhou Ye

arXiv · PDF · Open in the Atlas

Code that ran

14 of 20 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (9 of them constructed an object rather than computing a result; 2 honoured the contract we drafted, 0 violated it and 9 have no contract verdict; 3 failures of our instrument; 6 no recorded run); 2 of them were also run on the host, not in the jail, by the peer-agreement lane that set their level, measured 2026-11-22; repository links from the Papers with Code archive attributed under CC BY-SA, no affiliation; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.

"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2409.16040.json · badge: /api/ran/2409.16040.svg

Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.

RepositoryRoleRan
Time-MoE/Time-MoE canonical (classifier label)
classifier basis: classifier_2025
4 ran · 0 no recorded run
time-moe/time-moe marked official in Papers with Code
Papers with Code archive
10 ran · 6 no recorded run
FunctionStatusWhere it lives
count_num_tensor_elements Ran
ran, honoured the contract we drafted
Time-MoE/Time-MoE/run_eval.py
code served (permissive licence) · get_code("1ac8347912b2415e")
How it ran

Ran in Syntology's jail on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned int, ndim 0, dtype family int, shape () as a later run recorded it. The image it ran in was not recorded. The comparison behind this sample's level is a different run: the peer-agreement lane ran it again on the host, not in the jail (report dated 2026-09-04T22:45:11Z). If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# The function takes a tensor and counts total number of elements
# by multiplying all dimensions in tensor.shape
tensor = torch.randn(2, 4, 3)

ARGS = {
    "tensor": tensor
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type int, ndim 0, dtype family int, shape () — shape () as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
repeat_kv Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
Time-MoE/Time-MoE/time_moe/models/modeling_time_moe.py
code served (permissive licence) · get_code("3c76e52815c5401d")
How it ran

Ran in Syntology's jail on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned Tensor, ndim 4, dtype family float, shape (2, 6, 5, 4) as a later run recorded it. The image it ran in was not recorded. The comparison behind this sample's level is a different run: the peer-agreement lane ran it again on the host, not in the jail (report dated 2026-09-04T23:23:17Z). If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# Create a small tensor with shape (batch, num_key_value_heads, seqlen, head_dim)
# Using distinct sizes: batch=2, num_key_value_heads=3, seqlen=5, head_dim=4
hidden_states = torch.randn(2, 3, 5, 4)

# n_rep is the repetition factor
n_rep = 2

ARGS = {
    "hidden_states": hidden_states,
    "n_rep": n_rep,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 4, dtype family float, shape (2, 6, 5, 4) — shape (2, 6, 5, 4) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
TimeMoeConfig Ran
ran on a synthesized input · constructed an object; no result was computed
time-moe/time-moe/time_moe/models/modeling_time_moe.py
code served (permissive licence) · get_code("3f901a4c755b778e")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, TimeMoeConfig: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
from transformers import PretrainedConfig

torch.manual_seed(0)

ARGS = {
    'input_size': 1,
    'hidden_size': 256,
    'intermediate_size': 512,
    'horizon_lengths': [1, 2],
    'num_hidden_layers': 2,
    'num_attention_heads': 8,
    'num_key_value_heads': 8,
    'hidden_act': 'silu',
    'num_experts_per_tok': 2,
    'num_experts': 4,
    'max_position_embeddings': 512,
    'initializer_range': 0.02,
    'rms_norm_eps': 1e-6,
    'use_cache': True,
    'use_dense': True,
    'rope_theta': 10000,
    'attention_dropout': 0.0,
    'apply_aux_loss': False,
    'router_aux_loss_factor': 0.02,
    'tie_word_embeddings': False,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch, transformers — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type TimeMoeConfig — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a TimeMoeConfig output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
TimeMoeInputEmbedding Ran
ran on a synthesized input · constructed an object; no result was computed
time-moe/time-moe/time_moe/models/modeling_time_moe.py
code served (permissive licence) · get_code("e958505dd9a77a19")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, TimeMoeInputEmbedding: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
from transformers.activations import ACT2FN
from transformers import PretrainedConfig

torch.manual_seed(0)

class TimeMoeConfig(PretrainedConfig):
    model_type = 'time_moe'
    keys_to_ignore_at_inference = ['past_key_values']

    def __init__(self, input_size: int=1, hidden_size: int=4096, intermediate_size: int=22016, horizon_lengths: list=None, num_hidden_layers: int=32, num_attention_heads: int=32, num_key_value_heads: int=None, hidden_act: str='silu', num_experts_per_tok: int=2, num_experts: int=1, max_position_embeddings: int=32768, initializer_range: float=0.02, rms_norm_eps: float=1e-06, use_cache: bool=True, use_dense: bool=False, rope_theta: int=10000, attention_dropout: float=0.0, apply_aux_loss: bool=True, router_aux_loss_factor: float=0.02, tie_word_embeddings: bool=False, **kwargs):
        if horizon_lengths is None:
            horizon_lengths = [1]
        self.input_size = input_size
        self.hidden_size = hidden_size
        self.intermediate_size = intermediate_size
        self.max_position_embeddings = max_position_embeddings
        self.num_hidden_layers = num_hidden_layers
        self.num_attention_heads = num_attention_heads
        if num_key_value_heads is None:
            num_key_value_heads = num_attention_heads
        self.num_key_value_heads = num_key_value_heads
        self.hidden_act = hidden_act
        if isinstance(horizon_lengths, int):
            horizon_lengths = [horizon_lengths]
        self.horizon_lengths = horizon_lengths
        self.num_experts_per_tok = num_experts_per_tok
        self.num_experts = num_experts
        self.initializer_range = initializer_range
        self.rms_norm_eps = rms_norm_eps
        self.use_cache = use_cache
        self.use_dense = use_dense
        self.rope_theta = rope_theta
        self.attention_dropout = attention_dropout
        self.apply_aux_loss = apply_aux_loss
        self.router_aux_loss_factor = router_aux_loss_factor
        assert self.use_dense ^ self.apply_aux_loss, 'Both use_dense and apply_aux_loss cannot be set to True or False at the same time.'
        kwargs.pop('tie_word_embeddings', None)
        super().__init__(tie_word_embeddings=tie_word_embeddings, **kwargs)

# Create a TimeMoeConfig instance
config = TimeMoeConfig(
    input_size=3,
    hidden_size=8,
    intermediate_size=16,
    horizon_lengths=[1],
    num_hidden_layers=2,
    num_attention_heads=2,
    num_key_value_heads=2,
    hidden_act='silu',
    num_experts_per_tok=1,
    num_experts=1,
    max_position_embeddings=128,
    initializer_range=0.02,
    rms_norm_eps=1e-6,
    use_cache=True,
    use_dense=False,
    rope_theta=10000,
    attention_dropout=0.0,
    apply_aux_loss=True,
    router_aux_loss_factor=0.02,
    tie_word_embeddings=False
)

ARGS = {
    'config': config
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch, transformers — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type TimeMoeInputEmbedding — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a TimeMoeInputEmbedding output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
TimeMoeMLP Ran
ran on a synthesized input · constructed an object; no result was computed
time-moe/time-moe/time_moe/models/modeling_time_moe.py
code served (permissive licence) · get_code("45c683c6147f9aeb")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, TimeMoeMLP: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
from transformers.activations import ACT2FN

torch.manual_seed(0)

# TimeMoeMLP requires: hidden_size, intermediate_size, hidden_act
# These are all used in __init__ to set up Linear layers and activation function

hidden_size = 4
intermediate_size = 8
hidden_act = "gelu"

ARGS = {
    "hidden_size": hidden_size,
    "intermediate_size": intermediate_size,
    "hidden_act": hidden_act,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch, transformers — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type TimeMoeMLP — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a TimeMoeMLP output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
TimeMoeOutputLayer Ran
ran on a synthesized input · constructed an object; no result was computed
time-moe/time-moe/time_moe/models/modeling_time_moe.py
code served (permissive licence) · get_code("db5943a1300c151a")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, TimeMoeOutputLayer: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

# Define the arguments for TimeMoeOutputLayer
ARGS = {
    "hidden_size": 16,
    "horizon_length": 8,
    "input_size": 3,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type TimeMoeOutputLayer — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a TimeMoeOutputLayer output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
TimeMoePreTrainedModel Ran
ran on a synthesized input · constructed an object; no result was computed
time-moe/time-moe/time_moe/models/modeling_time_moe.py
code served (permissive licence) · get_code("0da41a274c8a465a")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, TimeMoePreTrainedModel: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
from transformers import PreTrainedModel, Cache, DynamicCache, StaticCache
from transformers import PretrainedConfig
from typing import Optional, Tuple, List, Union

# Define TimeMoeConfig
class TimeMoeConfig(PretrainedConfig):
    model_type = 'time_moe'
    keys_to_ignore_at_inference = ['past_key_values']

    def __init__(self, input_size: int=1, hidden_size: int=4096, intermediate_size: int=22016, horizon_lengths: List[int]=None, num_hidden_layers: int=32, num_attention_heads: int=32, num_key_value_heads: int=None, hidden_act: str='silu', num_experts_per_tok: int=2, num_experts: int=1, max_position_embeddings: int=32768, initializer_range: float=0.02, rms_norm_eps: float=1e-06, use_cache: bool=True, use_dense: bool=False, rope_theta: int=10000, attention_dropout: float=0.0, apply_aux_loss: bool=True, router_aux_loss_factor: float=0.02, tie_word_embeddings: bool=False, **kwargs):
        self.input_size = input_size
        self.hidden_size = hidden_size
        self.intermediate_size = intermediate_size
        self.max_position_embeddings = max_position_embeddings
        self.num_hidden_layers = num_hidden_layers
        self.num_attention_heads = num_attention_heads
        if num_key_value_heads is None:
            num_key_value_heads = num_attention_heads
        self.num_key_value_heads = num_key_value_heads
        self.hidden_act = hidden_act
        if horizon_lengths is None:
            horizon_lengths = [1]
        if isinstance(horizon_lengths, int):
            horizon_lengths = [horizon_lengths]
        self.horizon_lengths = horizon_lengths
        self.num_experts_per_tok = num_experts_per_tok
        self.num_experts = num_experts
        self.initializer_range = initializer_range
        self.rms_norm_eps = rms_norm_eps
        self.use_cache = use_cache
        self.use_dense = use_dense
        self.rope_theta = rope_theta
        self.attention_dropout = attention_dropout
        self.apply_aux_loss = apply_aux_loss
        self.router_aux_loss_factor = router_aux_loss_factor
        assert self.use_dense ^ self.apply_aux_loss, 'Both use_dense and apply_aux_loss cannot be set to True or False at the same time.'
        kwargs.pop('tie_word_embeddings', None)
        super().__init__(tie_word_embeddings=tie_word_embeddings, **kwargs)

# Define TimeMoePreTrainedModel
class TimeMoePreTrainedModel(PreTrainedModel):
    config_class = TimeMoeConfig
    base_model_prefix = 'model'
    supports_gradient_checkpointing = True
    _no_split_modules = ['TimeMoeDecoderLayer']
    _skip_keys_device_placement = 'past_key_values'
    _supports_flash_attn_2 = True
    _supports_sdpa = False
    _supports_cache_class = True

    def _init_weights(self, module):
        std = self.config.initializer_range
        if isinstance(module, torch.nn.Linear):
            module.weight.data.normal_(mean=0.0, std=std)
            if module.bias is not None:
                module.bias.data.zero_()
        elif isinstance(module, torch.nn.Embedding):
            module.weight.data.normal_(mean=0.0, std=std)
            if module.padding_idx is not None:
                module.weight.data[module.padding_idx].zero_()

torch.manual_seed(0)

# Create a minimal config
config = TimeMoeConfig(
    input_size=1,
    hidden_size=64,
    intermediate_size=128,
    num_hidden_layers=2,
    num_attention_heads=4,
    num_key_value_heads=4,
    use_cache=True,
    use_dense=False,
    apply_aux_loss=True
)

# Create the ARGS dictionary for TimeMoePreTrainedModel.__init__
ARGS = {
    'config': config,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch, transformers — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type TimeMoePreTrainedModel — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a TimeMoePreTrainedModel output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
TimeMoeRMSNorm Ran
ran on a synthesized input · constructed an object; no result was computed
time-moe/time-moe/time_moe/models/modeling_time_moe.py
code served (permissive licence) · get_code("17cafe312739a7dd")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, TimeMoeRMSNorm: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
from torch import nn

torch.manual_seed(0)

# The TimeMoeRMSNorm class takes:
# 1. hidden_size: int - the dimensionality of the hidden states
# 2. eps: float (optional, default=1e-06) - variance epsilon for numerical stability

# Looking at the forward method:
# - It expects hidden_states tensor with shape (..., hidden_size)
# - It computes variance over the last dimension
# - It applies weight which should have shape (hidden_size,)

hidden_size = 4

# Create a test hidden_states tensor with shape (batch_size=2, seq_len=3, hidden_size=4)
hidden_states = torch.randn(2, 3, hidden_size)

ARGS = {
    "hidden_size": hidden_size,
    "eps": 1e-06,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type TimeMoeRMSNorm — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a TimeMoeRMSNorm output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
TimeMoeRotaryEmbedding Ran
ran on a synthesized input · constructed an object; no result was computed
time-moe/time-moe/time_moe/models/modeling_time_moe.py
code served (permissive licence) · get_code("6ce7138c76e559cd")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, TimeMoeRotaryEmbedding: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

ARGS = {
    'dim': 4,
    'max_position_embeddings': 2048,
    'base': 10000,
    'device': None,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type TimeMoeRotaryEmbedding — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a TimeMoeRotaryEmbedding output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
TimeMoeSparseExpertsLayer Ran
ran on a synthesized input · constructed an object; no result was computed
time-moe/time-moe/time_moe/models/modeling_time_moe.py
code served (permissive licence) · get_code("5642e200205692a9")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, TimeMoeSparseExpertsLayer: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
from torch import nn

torch.manual_seed(0)

class Config:
    def __init__(self):
        self.hidden_size = 4
        self.intermediate_size = 8
        self.num_experts = 2
        self.num_experts_per_tok = 1
        self.hidden_act = "relu"

config = Config()

ARGS = {
    "config": config
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch, transformers — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type TimeMoeSparseExpertsLayer — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a TimeMoeSparseExpertsLayer output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
TimeMoeTemporalBlock Ran
ran on a synthesized input · constructed an object; no result was computed
time-moe/time-moe/time_moe/models/modeling_time_moe.py
code served (permissive licence) · get_code("8689182c3214eb30")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, TimeMoeTemporalBlock: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
from transformers.activations import ACT2FN

torch.manual_seed(0)

# Construct arguments for TimeMoeTemporalBlock.__init__
ARGS = {
    "hidden_size": 4,
    "intermediate_size": 8,
    "hidden_act": "gelu"
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch, transformers — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type TimeMoeTemporalBlock — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a TimeMoeTemporalBlock output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
_get_unpad_data Ran
ran; our contract draft was wrong, not the code
time-moe/time-moe/time_moe/models/modeling_time_moe.py
code served (permissive licence) · get_code("180b7fb4f14752ec")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn.functional as F

torch.manual_seed(0)

# Create a small attention mask tensor
# Shape: (batch_size=2, seq_len=4)
# attention_mask should be binary (0s and 1s) or boolean
attention_mask = torch.tensor([
    [1, 1, 1, 0],
    [1, 1, 0, 0]
], dtype=torch.int64)

ARGS = {
    "attention_mask": attention_mask
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple — a tuple output has no array shape of its own; none is stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
load_balancing_loss_func Ran
ran, honoured the contract we drafted
Time-MoE/Time-MoE/time_moe/models/modeling_time_moe.py
code served (permissive licence) · get_code("e0d3c24aaf39d620")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned Tensor, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn.functional as F

torch.manual_seed(0)

# Create gate_logits as a tuple of tensors
# Each tensor represents logits from a layer with shape [batch_size * sequence_length, num_experts]
batch_size = 2
sequence_length = 3
num_experts = 4
num_hidden_layers = 2

gate_logits = tuple(
    torch.randn(batch_size * sequence_length, num_experts)
    for _ in range(num_hidden_layers)
)

# top_k parameter
top_k = 2

# num_experts parameter
num_experts_param = num_experts

# attention_mask parameter - shape [batch_size, sequence_length]
attention_mask = torch.ones(batch_size, sequence_length, dtype=torch.long)

ARGS = {
    "gate_logits": gate_logits,
    "top_k": top_k,
    "num_experts": num_experts_param,
    "attention_mask": attention_mask,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 0, dtype family float, shape () — shape () as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
rotate_half Ran
ran; our contract draft was wrong, not the code
Time-MoE/Time-MoE/time_moe/models/modeling_time_moe.py
code served (permissive licence) · get_code("e03d53ba9d4f9ae5")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned Tensor, ndim 2, dtype family float, shape (2, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# The function takes a single parameter 'x' which is a tensor
# It accesses x[..., :x.shape[-1] // 2] and x[..., x.shape[-1] // 2:]
# So x needs to be a tensor with a last dimension that can be split in half
# Let's create a small tensor with batch size 2 and feature dimension 4

x = torch.randn(2, 4)

ARGS = {
    "x": x
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 2, dtype family float, shape (2, 4) — shape (2, 4) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
TSGenerationMixin Not yet run
unverified (harvested, no recorded run)
time-moe/time-moe/time_moe/models/modeling_time_moe.py
code served (permissive licence) · get_code("324cdcce054c2bef")
How it ran

no invocation recorded.

TimeMoeAttention Not yet run
unverified (harvested, no recorded run)
time-moe/time-moe/time_moe/models/modeling_time_moe.py
code served (permissive licence) · get_code("b789fb96aa9249b6")
How it ran

no invocation recorded.

TimeMoeDecoderLayer Not yet run
unverified (harvested, no recorded run)
time-moe/time-moe/time_moe/models/modeling_time_moe.py
code served (permissive licence) · get_code("45df40b90039e227")
How it ran

no invocation recorded.

TimeMoeFlashAttention2 Not yet run
unverified (harvested, no recorded run)
time-moe/time-moe/time_moe/models/modeling_time_moe.py
code served (permissive licence) · get_code("3342550a58b73daa")
How it ran

no invocation recorded.

TimeMoeForPrediction Not yet run
unverified (harvested, no recorded run)
time-moe/time-moe/time_moe/models/modeling_time_moe.py
code served (permissive licence) · get_code("fa1448790e7ba45b")
How it ran

no invocation recorded.

TimeMoeModel Not yet run
unverified (harvested, no recorded run)
time-moe/time-moe/time_moe/models/modeling_time_moe.py
code served (permissive licence) · get_code("e25a1475ceb73726")
How it ran

no invocation recorded.

Also present in other papers' code

Of 20 samples lifted from this paper's linked repositories, 20 carry a function identity and 0 do not. Of the 20 distinct functions with an identity, 4 are also present in other papers whose code we hold (3 with at least one paper dated earlier, 2 the same month, 4 later) and 16 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).

Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.

FunctionAlso present inEarliest-datedOthers named
rotate_half
brick of this paper's assembly: helper
360 other papers
178 dated earlier, 5 the same month, 168 later, 6 same year, order not determinable, 3 undated · 30 only as an assembly brick
1901.02860 (2019-01)
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context
2203.08913 (2022-03)
Memorizing Transformers
2302.13971 (2023-02)
LLaMA: Open and Efficient Foundation Language Models
and 357 more not named here
repeat_kv
brick of this paper's assembly: helper
243 other papers
114 dated earlier, 4 the same month, 121 later, 2 same year, order not determinable, 2 undated · 22 only as an assembly brick
1901.02860 (2019-01)
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context
2203.08913 (2022-03)
Memorizing Transformers
2305.17888 (2023-05)
LLM-QAT: Data-Free Quantization Aware Training for Large Language Models
and 240 more not named here
_get_unpad_data
brick of this paper's assembly: helper
3 other papers
1 dated earlier, 2 later · 3 only as an assembly brick
2402.14905 (2024-02)
MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases
2410.05357 (2024-10)
Model-GLUE: Democratized LLM Scaling for A Large Model Zoo in the Wild
2504.10462 (2025-04)
The Scalability of Simplicity: Empirical Analysis of Vision-Language Learning with a Single Transformer
load_balancing_loss_func
brick of this paper's assembly: loss
1 other paper
1 later
2025.emnlp-main.231 (2025 (year only))
TS-CLIP: Time Series Understanding by CLIP

Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Deep learning for time series forecasting has seen significant advancements over the past decades. However, despite the success of large-scale pre-training in language and vision domains, pre-trained time series models remain limited in scale and operate at a high cost, hindering the development of larger capable forecasting models in real-world applications. In response, we introduce TIME-MOE, a scalable and unified architecture designed to pre-train larger, more capable forecasting foundation models while reducing inference costs. By leveraging a sparse mixture-of-experts (MoE) design, TIME-MOE enhances computational efficiency by activating only a subset of networks for each prediction, reducing computational load while maintaining high model capacity. This allows TIME-MOE to scale effectively without a corresponding increase in inference costs. TIME-MOE comprises a family of decoder-only transformer models that operate in an autoregressive manner and support flexible forecasting horizons with varying input context lengths. We pre-trained these models on our newly introduced large-scale data Time-300B, which spans over 9 domains and encompassing over 300 billion time points. For the first time, we scaled a time series foundation model up to 2.4 billion parameters, achieving significantly improved forecasting precision. Our results validate the applicability of scaling laws for training tokens and model size in the context of time series forecasting. Compared to dense models with the same number of activated parameters or equivalent computation budgets, our models consistently outperform them by large margin. These advancements position TIME-MOE as a state-of-the-art solution for tackling real-world time series forecasting challenges with superior capability, efficiency, and flexibility.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2409.16040")
get_code_for_paper("2409.16040")
have("2409.16040")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2409.16040.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2409.16040.svg)](https://syntology.ai/paper/2409.16040)

Connect an agent — have() is free.