SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2406.05815 · 2024

What Can We Learn from State Space Models for Machine Learning on Graphs?

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 11 functions out of this paper's own repositories and ran 6 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

RepositoryRoleRan
graph-com/gssc canonical 6 of 11
FunctionStatusWhere it lives
augment_seq Ran graph-com/gssc/gssc/layer/gps_layer.py
pointer only (licence: NONE) · get_code("3fc527802fe80aad")
get_activation Ran graph-com/gssc/gssc/layer/Exphormer.py
pointer only (licence: NONE) · get_code("b888018e8acd3f1d")
get_final_pretrained_ckpt Ran graph-com/gssc/gssc/finetuning.py
pointer only (licence: NONE) · get_code("1bb331bf70a78136")
init_model_from_pretrained Ran graph-com/gssc/gssc/finetuning.py
pointer only (licence: NONE) · get_code("9e4db46ba20af98e")
permute_nodes_within_identity Ran graph-com/gssc/gssc/layer/gps_layer.py
pointer only (licence: NONE) · get_code("c872f11ec358e25c")
sort_rand_gpu Ran graph-com/gssc/gssc/layer/gps_layer.py
pointer only (licence: NONE) · get_code("79306f9d63c3e8fa")
apply_chunking_to_forward Not yet run graph-com/gssc/gssc/layer/bigbird_layer.py
pointer only (licence: NONE) · get_code("ae56bcfaae96da3c")
is_seed Not yet run graph-com/gssc/gssc/agg_runs.py
pointer only (licence: NONE) · get_code("c98a702be0660ac9")
is_split Not yet run graph-com/gssc/gssc/agg_runs.py
pointer only (licence: NONE) · get_code("438635c08adfe2aa")
join_list Not yet run graph-com/gssc/gssc/agg_runs.py
pointer only (licence: NONE) · get_code("40e98e1ecbf39f34")
load_pretrained_model_cfg Not yet run graph-com/gssc/gssc/finetuning.py
pointer only (licence: NONE) · get_code("a969312556ec39bb")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Machine learning on graphs has recently found extensive applications across domains. However, the commonly used Message Passing Neural Networks (MPNNs) suffer from limited expressive power and struggle to capture long-range dependencies. Graph transformers offer a strong alternative due to their global attention mechanism, but they come with great computational overheads, especially for large graphs. In recent years, State Space Models (SSMs) have emerged as a compelling approach to replace full attention in transformers to model sequential data. It blends the strengths of RNNs and CNNs, offering a) efficient computation, b) the ability to capture long-range dependencies, and c) good generalization across sequences of various lengths. However, extending SSMs to graph-structured data presents unique challenges due to the lack of canonical node ordering in graphs. In this work, we propose Graph State Space Convolution (GSSC) as a principled extension of SSMs to graph-structured data. By leveraging global permutation-equivariant set aggregation and factorizable graph kernels that rely on relative node distances as the convolution kernels, GSSC preserves all three advantages of SSMs. We demonstrate the provably stronger expressiveness of GSSC than MPNNs in counting graph substructures and show its effectiveness across 11 real-world, widely used benchmark datasets. GSSC achieves the best results on 6 out of 11 datasets with all significant improvements compared to the state-of-the-art baselines and second-best results on the other 5 datasets. Our findings highlight the potential of GSSC as a powerful and scalable model for graph machine learning. Our code is available at https://github.com/Graph-COM/GSSC.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2406.05815")
get_code_for_paper("2406.05815")
have("2406.05815")

Connect an agent — have() is free.