We lifted 13 functions out of this paper's own repositories and ran 8 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| zhouc20/k-transformer | canonical | 8 of 13 |
| Function | Status | Where it lives |
|---|---|---|
| Delta_0 | Ran | zhouc20/k-transformer/Hodge_decomposition.py code served (permissive licence) · get_code("a59edb7a016ef8b4") |
| PyG_collate | Ran | zhouc20/k-transformer/train_count_substructure.py code served (permissive licence) · get_code("589458845725ad19") |
| cumsum_pad0 | Ran | zhouc20/k-transformer/graphgps/network/k_simplicial_transformer_model.py code served (permissive licence) · get_code("8134b52cd16bee5c") |
| get_final_pretrained_ckpt | Ran | zhouc20/k-transformer/graphgps/finetuning.py code served (permissive licence) · get_code("1bb331bf70a78136") |
| get_log_deg | Ran | zhouc20/k-transformer/graphgps/layer/tensorized_transformer_layer.py code served (permissive licence) · get_code("b5c9e7acf599a98a") |
| init_model_from_pretrained | Ran | zhouc20/k-transformer/graphgps/finetuning.py code served (permissive licence) · get_code("9a4410b8a6c0cc47") |
| new_optimizer_config | Ran | zhouc20/k-transformer/train_count_substructure.py code served (permissive licence) · get_code("585566394225e457") |
| num2batch | Ran | zhouc20/k-transformer/graphgps/network/k_simplicial_transformer_model.py code served (permissive licence) · get_code("9655691d41dabccf") |
| divergence | Not yet run | zhouc20/k-transformer/Hodge_decomposition.py code served (permissive licence) · get_code("a0f064817b33d41f") |
| is_seed | Not yet run | zhouc20/k-transformer/graphgps/agg_runs.py code served (permissive licence) · get_code("c98a702be0660ac9") |
| is_split | Not yet run | zhouc20/k-transformer/graphgps/agg_runs.py code served (permissive licence) · get_code("438635c08adfe2aa") |
| join_list | Not yet run | zhouc20/k-transformer/graphgps/agg_runs.py code served (permissive licence) · get_code("40e98e1ecbf39f34") |
| load_pretrained_model_cfg | Not yet run | zhouc20/k-transformer/graphgps/finetuning.py code served (permissive licence) · get_code("91c1f5bb90bbe0f3") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Graph transformers have recently received significant attention in graph learning, partly due to their ability to capture more global interaction via self-attention. Nevertheless, while higher-order graph neural networks have been reasonably well studied, the exploration of extending graph transformers to higher-order variants is just starting. Both theoretical understanding and empirical results are limited. In this paper, we provide a systematic study of the theoretical expressive power of order-$k$ graph transformers and sparse variants. We first show that, an order-$k$ graph transformer without additional structural information is less expressive than the $k$-Weisfeiler Lehman ($k$-WL) test despite its high computational cost. We then explore strategies to both sparsify and enhance the higher-order graph transformers, aiming to improve both their efficiency and expressiveness. Indeed, sparsification based on neighborhood information can enhance the expressive power, as it provides additional information about input graph structures. In particular, we show that a natural neighborhood-based sparse order-$k$ transformer model is not only computationally efficient, but also expressive -- as expressive as $k$-WL test. We further study several other sparse graph attention models that are computationally efficient and provide their expressiveness analysis. Finally, we provide experimental results to show the effectiveness of the different sparsification strategies.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2404.03380")
get_code_for_paper("2404.03380")
have("2404.03380")
Connect an agent — have() is free.