SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2407.19394 · 2024

Depth-Wise Convolutions in Vision Transformers for Efficient Training on Small Datasets

arXiv · PDF · Open in the Atlas

Code that ran

13 of 16 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (0 of them constructed an object rather than computing a result; 2 honoured the contract we drafted, 3 violated it and 0 have no contract verdict; 8 failures of our instrument; 3 no recorded run); 1 of them was also run on the host, not in the jail, by the peer-agreement lane that set its level, measured 2026-11-22; repository links from the Papers with Code archive attributed under CC BY-SA, no affiliation; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.

"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2407.19394.json · badge: /api/ran/2407.19394.svg

Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.

RepositoryRoleRan
ztx-100/efficient_vit_with_dw canonical (classifier label)
classifier basis: census
13 ran · 3 no recorded run
FunctionStatusWhere it lives
window_partition Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
ztx-100/efficient_vit_with_dw/models/swin_transformer.py
pointer only (licence: NONE) · get_code("144d10b49baeb8a6")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, ztx-100/efficient_vit_with_dw when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

auto_resume_helper Ran
ran; our contract draft was wrong, not the code
ztx-100/efficient_vit_with_dw/utils.py
pointer only (licence: NONE); the stored copy get_code serves, from leaplabthu/inline, is pointer only (licence: NONE) · get_code("0c32459bef92ff8c")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, ztx-100/efficient_vit_with_dw when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

build_optimizer Ran
ran; our contract draft was wrong, not the code
ztx-100/efficient_vit_with_dw/optimizer.py
pointer only (licence: NONE) · get_code("9fe330952c4ee761")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, ztx-100/efficient_vit_with_dw when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

build_scheduler Ran
ran; our contract draft was wrong, not the code
ztx-100/efficient_vit_with_dw/lr_scheduler.py
pointer only (licence: NONE) · get_code("0fbdf3257aea9bcc")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, ztx-100/efficient_vit_with_dw when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

check_keywords_in_name Ran
ran, violated the contract we drafted
ztx-100/efficient_vit_with_dw/optimizer.py
pointer only (licence: NONE); the stored copy get_code serves, from leaplabthu/inline, is pointer only (licence: NONE) · get_code("b0a5beb34716d5a6")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, ztx-100/efficient_vit_with_dw when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

create_logger Ran
ran; our contract draft was wrong, not the code
ztx-100/efficient_vit_with_dw/logger.py
pointer only (licence: NONE); the stored copy get_code serves, from ghrua/seqpe, is pointer only (licence: NONE) · get_code("80450600f7f09b0a")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, ztx-100/efficient_vit_with_dw when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

dropout_layers Ran
ran; our contract draft was wrong, not the code
ztx-100/efficient_vit_with_dw/models/cait.py
pointer only (licence: NONE) · get_code("5c177b03eb4157b4")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, ztx-100/efficient_vit_with_dw when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

exists Ran
ran, violated the contract we drafted
ztx-100/efficient_vit_with_dw/models/cait.py
pointer only (licence: NONE); the stored copy get_code serves, from ThomasMrY/VCT, is pointer only (licence: NONE) · get_code("aa5486a3650902d8")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, ztx-100/efficient_vit_with_dw when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

get_grad_norm Ran
ran, honoured the contract we drafted
ztx-100/efficient_vit_with_dw/utils.py
pointer only (licence: NONE) · get_code("eec1e7cba51d5e8e")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, ztx-100/efficient_vit_with_dw when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

load_checkpoint Ran
ran, honoured the contract we drafted
ztx-100/efficient_vit_with_dw/utils.py
pointer only (licence: NONE) · get_code("59c3a4f0d92e6970")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, ztx-100/efficient_vit_with_dw when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

pair Ran
ran, violated the contract we drafted
ztx-100/efficient_vit_with_dw/models/vit.py
pointer only (licence: NONE) · get_code("6ba8cee9f5daea41")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, ztx-100/efficient_vit_with_dw when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

set_weight_decay Ran
ran; our contract draft was wrong, not the code
ztx-100/efficient_vit_with_dw/optimizer.py
pointer only (licence: NONE) · get_code("b33222a09fc93bec")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, ztx-100/efficient_vit_with_dw when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

window_reverse Ran
ran; our contract draft was wrong, not the code
ztx-100/efficient_vit_with_dw/models/swin_transformer.py
pointer only (licence: NONE) · get_code("61bf152e6a42a184")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, ztx-100/efficient_vit_with_dw when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

build_model Not yet run
unverified (harvested, no recorded run)
ztx-100/efficient_vit_with_dw/models/build.py
pointer only (licence: NONE) · get_code("7cc4ee2a64e93062")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, ztx-100/efficient_vit_with_dw when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

get_config Not yet run
unverified (harvested, no recorded run)
ztx-100/efficient_vit_with_dw/config.py
pointer only (licence: NONE) · get_code("759a6b3cb8363354")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, ztx-100/efficient_vit_with_dw when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

update_config Not yet run
unverified (harvested, no recorded run)
ztx-100/efficient_vit_with_dw/config.py
pointer only (licence: NONE) · get_code("a876af529e9074d6")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, ztx-100/efficient_vit_with_dw when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

Also present in other papers' code

Of 16 samples lifted from this paper's linked repositories, 16 carry a function identity and 0 do not. Of the 16 distinct functions with an identity, 14 are also present in other papers whose code we hold (13 with at least one paper dated earlier, 8 the same month, 13 later) and 2 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).

Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.

FunctionAlso present inEarliest-datedOthers named
exists 486 other papers
362 dated earlier, 8 the same month, 105 later, 9 same year, order not determinable, 2 undated · 78 only as an assembly brick
1603.08155 (2016-03)
Perceptual Losses for Real-Time Style Transfer and Super-Resolution
1611.07004 (2016-11)
Image-to-Image Translation with Conditional Adversarial Networks
1804.04637 (2018-04)
EMBER: An Open Dataset for Training Static PE Malware Machine Learning Models
and 483 more not named here
window_reverse 227 other papers
181 dated earlier, 3 the same month, 34 later, 9 same year, order not determinable · 34 only as an assembly brick
1608.03983 (2016-08)
SGDR: Stochastic Gradient Descent with Warm Restarts
1711.10925 (2017-11)
Deep Image Prior
2003.13678 (2020-03)
Designing Network Design Spaces
and 224 more not named here
window_partition 223 other papers
177 dated earlier, 3 the same month, 34 later, 9 same year, order not determinable · 32 only as an assembly brick
1706.03762 (2017-06)
Attention Is All You Need
1711.10925 (2017-11)
Deep Image Prior
2003.13678 (2020-03)
Designing Network Design Spaces
and 220 more not named here
pair 90 other papers
74 dated earlier, 1 the same month, 13 later, 1 same year, order not determinable, 1 undated · 8 only as an assembly brick
2004.13649 (2020-04)
Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels
2010.11929 (2020-10)
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
2103.15691 (2021-03)
ViViT: A Video Vision Transformer
and 87 more not named here
get_config 51 other papers
37 dated earlier, 10 later, 4 same year, order not determinable
2103.14030 (2021-03)
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
2106.03650 (2021-06)
Shuffle Transformer: Rethinking Spatial Shuffle for Vision Transformer
2106.03746 (2021-06)
Efficient Training of Visual Transformers with Small Datasets
and 48 more not named here
create_logger 35 other papers
28 dated earlier, 5 later, 2 same year, order not determinable
2104.13840 (2021-04)
Twins: Revisiting the Design of Spatial Attention in Vision Transformers
2106.03650 (2021-06)
Shuffle Transformer: Rethinking Spatial Shuffle for Vision Transformer
2106.03746 (2021-06)
Efficient Training of Visual Transformers with Small Datasets
and 32 more not named here
check_keywords_in_name 31 other papers
23 dated earlier, 6 later, 2 same year, order not determinable
2103.14030 (2021-03)
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
2106.03650 (2021-06)
Shuffle Transformer: Rethinking Spatial Shuffle for Vision Transformer
2106.03746 (2021-06)
Efficient Training of Visual Transformers with Small Datasets
and 28 more not named here
get_grad_norm 23 other papers
15 dated earlier, 1 the same month, 5 later, 2 same year, order not determinable
1910.13616 (2019-10)
Multimodal Model-Agnostic Meta-Learning via Task-Aware Modulation
2103.14030 (2021-03)
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
2106.03650 (2021-06)
Shuffle Transformer: Rethinking Spatial Shuffle for Vision Transformer
and 20 more not named here
auto_resume_helper 20 other papers
14 dated earlier, 1 the same month, 3 later, 2 same year, order not determinable
2103.14030 (2021-03)
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
2106.03650 (2021-06)
Shuffle Transformer: Rethinking Spatial Shuffle for Vision Transformer
2106.04263 (2021-06)
On the Connection between Local Attention and Dynamic Depth-wise Convolution
and 17 more not named here
set_weight_decay 14 other papers
13 dated earlier, 1 later
2103.14030 (2021-03)
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
2106.03650 (2021-06)
Shuffle Transformer: Rethinking Spatial Shuffle for Vision Transformer
2106.03746 (2021-06)
Efficient Training of Visual Transformers with Small Datasets
and 11 more not named here
load_checkpoint 4 other papers
2 dated earlier, 1 the same month, 1 later
2310.02642 (2023-10)
GET: Group Event Transformer for Event-Based Vision
2406.08773 (2024-06)
DenoiseRep: Denoising Model for Representation Learning
2407.10240 (2024-07)
xLSTMTime : Long-term Time Series Forecasting With xLSTM
and 1 more not named here
build_optimizer 3 other papers
2 dated earlier, 1 later
2306.05175 (2023-06)
Large-scale Dataset Pruning with Dynamic Uncertainty
2406.08773 (2024-06)
DenoiseRep: Denoising Model for Representation Learning
ijcai2025_0271 (2025 (year only))
Efficient Visual Representation Learning with Heat Conduction Equation
build_scheduler 3 other papers
1 the same month, 2 later
2407.10240 (2024-07)
xLSTMTime : Long-term Time Series Forecasting With xLSTM
2410.15091 (2024-10)
Spatial-Mamba: Effective Visual State Space Models via Structure-Aware State Fusion
dropout_layers 2 other papers
2 dated earlier
2110.09057 (2021-10)
Training Deep Neural Networks with Adaptive Momentum Inspired by the Quadratic Optimization
2308.03258 (2023-08)
APBench: A Unified Benchmark for Availability Poisoning Attacks and Defenses

Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

The Vision Transformer (ViT) leverages the Transformer's encoder to capture global information by dividing images into patches and achieves superior performance across various computer vision tasks. However, the self-attention mechanism of ViT captures the global context from the outset, overlooking the inherent relationships between neighboring pixels in images or videos. Transformers mainly focus on global information while ignoring the fine-grained local details. Consequently, ViT lacks inductive bias during image or video dataset training. In contrast, convolutional neural networks (CNNs), with their reliance on local filters, possess an inherent inductive bias, making them more efficient and quicker to converge than ViT with less data. In this paper, we present a lightweight Depth-Wise Convolution module as a shortcut in ViT models, bypassing entire Transformer blocks to ensure the models capture both local and global information with minimal overhead. Additionally, we introduce two architecture variants, allowing the Depth-Wise Convolution modules to be applied to multiple Transformer blocks for parameter savings, and incorporating independent parallel Depth-Wise Convolution modules with different kernels to enhance the acquisition of local information. The proposed approach significantly boosts the performance of ViT models on image classification, object detection, and instance segmentation by a large margin, especially on small datasets, as evaluated on CIFAR-10, CIFAR-100, Tiny-ImageNet and ImageNet for image classification, and COCO for object detection and instance segmentation. The source code can be accessed at https://github.com/ZTX-100/Efficient_ViT_with_DW.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2407.19394")
get_code_for_paper("2407.19394")
have("2407.19394")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2407.19394.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2407.19394.svg)](https://syntology.ai/paper/2407.19394)

Connect an agent — have() is free.