MearcAll papers

arXiv 2103.14030 · cs.CV cs.LG

Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, and 2 more

Claims were extracted automatically by reducer-1 on 2026-11-12. Claims are quoted in full under the paper's license.

No reproduction recordedclm_d0583574
To address these differences, we propose a hierarchical Transformer whose representation is computed with Shifted windows.

Abstract · page 1 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_51122543
Its performance surpasses the previous state-of-theart by a large margin of +2.7 box AP and +2.6 mask AP on COCO, and +3.2 mIoU on ADE20K, demonstrating the potential of Transformer-based models as vision backbones.

Abstract · page 1 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_7b65ca52
To overcome these issues, we propose a generalpurpose Transformer backbone, called Swin Transformer, which constructs hierarchical feature maps and has linear computational complexity to image size.

Introduction · page 2 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_b832cbaf
Our experiments show that the proposed 1The query and key are projection vectors in a self-attention layer.

Introduction · page 2 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_a7eeab87
It also achieves a top-1 accuracy of 87.3% on ImageNet-1K image classification.

Introduction · page 2 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_8e6f5afc
Self-attention in non-overlapped windows For efficient modeling, we propose to compute self-attention within local windows.

Shifted Window based Self-Attention · page 4 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_55dccde6
To introduce cross-window connections while maintaining the efficient computation of non-overlapping windows, we propose a shifted window partitioning approach which alternates between two partitioning configurations in consecutive Swin Transformer blocks.

Shifted Window based Self-Attention · page 4 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_5ec40aed
We observe significant improvements over counterparts without this bias term or that use absolute position embedding, as shown in Table 4.

Shifted Window based Self-Attention · page 5 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_fd40a8f4
Compared with the previous best results for ImageNet-22K pre-training, our models achieve significantly better speed-accuracy trade-offs: Swin-B obtains 86.4% top-1 accuracy, which is 2.4% higher than that of ViT with similar inference throughput (84.7 vs. 85.9 images/sec) and slightly lower FLOPs (47.0G vs. 55.4G).

Experiments · page 6 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_bc691832
The larger Swin-L model achieves 87.3% top-1 accuracy, +0.9% better than that of the Swin-B model.

Experiments · page 6 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_73664b7e
For system-level comparison, we adopt an improved HTC [9] (denoted as HTC++) with instaboost [22], stronger multi-scale training [7], 6x schedule (72 epochs), soft-NMS [5], and ImageNet-22K pre-trained model as initialization.

Object Detection on COCO · page 6 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_83f17d6c
Swin Transformer achieves a high detection accuracy of 51.9 box AP and 45.0 mask AP, which are significant gains of +3.6 box AP and +3.3 mask AP over ResNeXt101- 64x4d, which has similar model size, FLOPs and latency.

Object Detection on COCO · page 7 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_2b8b152e
On a higher baseline of 52.3 box AP and 46.0 mask AP using an improved HTC framework, the gains by Swin Transformer are also high, at +4.1 box AP and +3.1 mask AP (see Table 2(c)).

Object Detection on COCO · page 7 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_30894bbf
Our best model achieves 58.7 box AP and 51.1 mask AP on COCO test-dev, surpassing the previous best results by +2.7 box AP (Copy-paste [26] without external data) and +2.6 mask AP (DetectoRS [46]).

Object Detection on COCO · page 7 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_e3f80096
Our Swin-L model with ImageNet-22K pre-training achieves 53.5 mIoU on the val set, surpassing the previous best model by +3.2 mIoU (50.3 mIoU by SETR [81] which has a larger model size).

Semantic Segmentation on ADE20K · page 8 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_e3a4b3a3
Swin-T with the shifted window partitioning outperforms the counterpart built on a single window partitioning at each stage by +1.1% top-1 accuracy on ImageNet-1K, +2.8 box AP/+2.2 mask AP on COCO, and +2.8 mIoU on ADE20K.

Ablation Study · page 8 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_3a341010
Also note that while the inclusion of absolute position embedding improves image classification accuracy (+0.4%), it harms object detection and semantic segmentation (-0.2 box/mask AP on COCO and -0.6 mIoU on ADE20K).

Ablation Study · page 8 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_6d5eeb30
Overall, the Swin Transformer architectures built on shifted windows are 4.1/1.5, 4.0/1.5, 3.6/1.5 times faster than variants built on sliding windows for Swin-T, Swin-S, and Swin-B, respectively.

Ablation Study · page 8 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_69ce43f7
For other resolutions such as 3842, we fine-tune the models trained at 2242 resolution, instead of training from scratch, to reduce GPU consumption.

Conclusion · page 9 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_47f65b70
For system-level comparison, we adopt an improved HTC [9] (denoted as HTC++) with instaboost [22], stronger multi-scale training [7] (resizing the input such that the shorter side is between 400 and 1400 while the longer side is at most 1600), 6x schedule (72 epochs with the learning rate decayed at epochs 63 and 69 by a factor of 0.1), soft- NMS [5], and an extra global self-attention layer appended at the output of last stage and ImageNet-22K pre-trained 9 downsp. rate (output size) Swin-T Swin-S Swin-B Swin-L stage 1 4×(56×56) concat 4×4, 96-d, LN concat 4×4, 96-d, LN concat 4×4, 128-d, LN concat 4×4, 192-d, LN[ win. sz.

Conclusion · page 9 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_64265450
In general, a larger input resolution leads to better top-1 accuracy but with slower inference speed.

Conclusion · page 10 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedclm_2e7a2b57
While SGD is used as a default optimizer for Cascade Mask R-CNN framework, we generally observe improved accuracy by replacing it with an AdamW optimizer, particularly for smaller backbones.

Conclusion · page 10 · Automated extraction, not reviewed · 0 reviewed reproductions

No reproduction recordedres_4a041eb6
Swin-Mixer performs significantly better than MLP-Mixer (81.3% vs. 76.4%) using slightly smaller computation budget (10.4G vs. 12.7G).

Conclusion · page 11 · Automated extraction, not reviewed · 0 reviewed reproductions

Reproduced one of these claims?

Record your result in the Mearc app on your own machine, then use Publish this result with this paper's id (2103.14030) and the claim id shown next to the claim. Mearc creates a file that holds only what you choose to publish. Add it to the public claim data by pull request; a maintainer other than you reviews it before it counts.