SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2306.11249 · NeurIPS · 2023

OpenSTL: A Comprehensive Benchmark of Spatio-Temporal Predictive Learning

Convlstm Convlstm, Zicheng Liu, Siyuan Li, Lirong Wu, Stan Li, Cheng Tan, Zhangyang Gao, Zedong Wang, Wenfei Guan, Convlstm St-Lstm, 𝐶 ! #, Incepunet Conv, and 2 more

arXiv · PDF · Open in the Atlas

Code that ran

4 of 4 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (0 of them constructed an object rather than computing a result; 2 honoured the contract we drafted, 0 violated it and 0 have no contract verdict; 2 failures of our instrument; 0 no recorded run), measured 2026-11-22; repository links from the Papers with Code archive attributed under CC BY-SA, no affiliation; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.

"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2306.11249.json · badge: /api/ran/2306.11249.svg

Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.

RepositoryRoleRan
chengtan9907/simvpv2 canonical (classifier label)
classifier basis not recorded on the link
4 ran · 0 no recorded run
FunctionStatusWhere it lives
MAE Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
chengtan9907/simvpv2/openstl/core/metrics.py
pointer only (licence: Apache-2.0); the stored copy get_code serves, from whynot-zyl/CloDS, is pointer only (licence: NONE) · get_code("05ac0a71a8992041")
How it ran

The body of this sample is withheld: this paper's link allows the body inline (Apache-2.0), but get_code serves the stored copy, first harvested from whynot-zyl/CloDS, and GitHub detected no licence in that repository when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

MSE Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
chengtan9907/simvpv2/openstl/core/metrics.py
pointer only (licence: Apache-2.0); the stored copy get_code serves, from whynot-zyl/CloDS, is pointer only (licence: NONE) · get_code("c2bfee96fb97d16c")
How it ran

The body of this sample is withheld: this paper's link allows the body inline (Apache-2.0), but get_code serves the stored copy, first harvested from whynot-zyl/CloDS, and GitHub detected no licence in that repository when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

drop_scheduler Ran
ran, honoured the contract we drafted
chengtan9907/simvpv2/openstl/core/drop_scheduler.py
pointer only (licence: Apache-2.0); the stored copy get_code serves, from facebookresearch/dropout, is pointer only (licence: NOASSERTION) · get_code("5e1695210460b9d6")
How it ran

The body of this sample is withheld: this paper's link allows the body inline (Apache-2.0), but get_code serves the stored copy, first harvested from facebookresearch/dropout, and GitHub could not classify the licence of that repository (NOASSERTION). The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

rescale Ran
ran, honoured the contract we drafted
chengtan9907/simvpv2/openstl/core/metrics.py
code served (permissive licence) · get_code("5e7f6606163474d5")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 2, dtype family float, shape (2, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# The function rescale takes a single parameter x
# It calls x.max(), x.min() on x, which suggests x is a tensor
# It performs arithmetic operations on x

x = torch.randn(2, 4)

ARGS = {
    "x": x
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 2, dtype family float, shape (2, 4) — shape (2, 4) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block

Also present in other papers' code

Of 4 samples lifted from this paper's linked repositories, 4 carry a function identity and 0 do not. Of the 4 distinct functions with an identity, 4 are also present in other papers whose code we hold (1 with at least one paper dated earlier, 0 the same month, 4 later) and 0 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).

Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.

FunctionAlso present inEarliest-datedOthers named
drop_scheduler 4 other papers
1 dated earlier, 3 later
2303.01500 (2023-03)
Dropout Reduces Underfitting
2311.05589 (2023-11)
A Coefficient Makes SVRG Effective
2404.06773 (2024-04)
Adapting LLaMA Decoder to Vision Transformer
and 1 more not named here
MAE 2 other papers
2 later
Li_Met2Net_A_Decoupled_Two-Stage_Spatio-Temporal_Forecasting_Model_for_Complex_Meteorological_ICCV_2025_paper (2025 (year only))
Met 2 Net: A Decoupled Two-Stage Spatio-Temporal Forecasting Model for Complex Meteorological Systems
2602.01844 (2026-02)
CloDS: Visual-Only Unsupervised Cloth Dynamics Learning in Unknown Conditions
MSE 2 other papers
2 later
Li_Met2Net_A_Decoupled_Two-Stage_Spatio-Temporal_Forecasting_Model_for_Complex_Meteorological_ICCV_2025_paper (2025 (year only))
Met 2 Net: A Decoupled Two-Stage Spatio-Temporal Forecasting Model for Complex Meteorological Systems
2602.01844 (2026-02)
CloDS: Visual-Only Unsupervised Cloth Dynamics Learning in Unknown Conditions
rescale 1 other paper
1 later
Li_Met2Net_A_Decoupled_Two-Stage_Spatio-Temporal_Forecasting_Model_for_Complex_Meteorological_ICCV_2025_paper (2025 (year only))
Met 2 Net: A Decoupled Two-Stage Spatio-Temporal Forecasting Model for Complex Meteorological Systems

Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Spatio-temporal predictive learning is a learning paradigm that enables models to learn spatial and temporal patterns by predicting future frames from given past frames in an unsupervised manner. Despite remarkable progress in recent years, a lack of systematic understanding persists due to the diverse settings, complex implementation, and difficult reproducibility. Without standardization, comparisons can be unfair and insights inconclusive. To address this dilemma, we propose OpenSTL, a comprehensive benchmark for spatio-temporal predictive learning that categorizes prevalent approaches into recurrent-based and recurrent-free models. OpenSTL provides a modular and extensible framework implementing various state-of-the-art methods. We conduct standard evaluations on datasets across various domains, including synthetic moving object trajectory, human motion, driving scenes, traffic flow, and weather forecasting. Based on our observations, we provide a detailed analysis of how model architecture and dataset properties affect spatio-temporal predictive learning performance. Surprisingly, we find that recurrent-free models achieve a good balance between efficiency and performance than recurrent models. Thus, we further extend the common MetaFormers to boost recurrent-free spatial-temporal predictive learning. We open-source the code and models at https://github.com/chengtan9907/OpenSTL.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2306.11249")
get_code_for_paper("2306.11249")
have("2306.11249")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2306.11249.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2306.11249.svg)](https://syntology.ai/paper/2306.11249)

Connect an agent — have() is free.