clm_d0583574To address these differences, we propose a hierarchical Transformer whose representation is computed with Shifted windows.
arXiv 2103.14030 · cs.CV cs.LG
Read on arXiv · CC BY 4.0
Claims were extracted automatically by reducer-1 on 2026-11-12. Claims are quoted in full under the paper's license.
clm_d0583574To address these differences, we propose a hierarchical Transformer whose representation is computed with Shifted windows.
res_51122543Its performance surpasses the previous state-of-theart by a large margin of +2.7 box AP and +2.6 mask AP on COCO, and +3.2 mIoU on ADE20K, demonstrating the potential of Transformer-based models as vision backbones.
clm_7b65ca52To overcome these issues, we propose a generalpurpose Transformer backbone, called Swin Transformer, which constructs hierarchical feature maps and has linear computational complexity to image size.
clm_b832cbafOur experiments show that the proposed 1The query and key are projection vectors in a self-attention layer.
res_a7eeab87It also achieves a top-1 accuracy of 87.3% on ImageNet-1K image classification.
clm_8e6f5afcSelf-attention in non-overlapped windows For efficient modeling, we propose to compute self-attention within local windows.
clm_55dccde6To introduce cross-window connections while maintaining the efficient computation of non-overlapping windows, we propose a shifted window partitioning approach which alternates between two partitioning configurations in consecutive Swin Transformer blocks.
res_5ec40aedWe observe significant improvements over counterparts without this bias term or that use absolute position embedding, as shown in Table 4.
res_fd40a8f4Compared with the previous best results for ImageNet-22K pre-training, our models achieve significantly better speed-accuracy trade-offs: Swin-B obtains 86.4% top-1 accuracy, which is 2.4% higher than that of ViT with similar inference throughput (84.7 vs. 85.9 images/sec) and slightly lower FLOPs (47.0G vs. 55.4G).
res_bc691832The larger Swin-L model achieves 87.3% top-1 accuracy, +0.9% better than that of the Swin-B model.
res_73664b7eFor system-level comparison, we adopt an improved HTC [9] (denoted as HTC++) with instaboost [22], stronger multi-scale training [7], 6x schedule (72 epochs), soft-NMS [5], and ImageNet-22K pre-trained model as initialization.
res_83f17d6cSwin Transformer achieves a high detection accuracy of 51.9 box AP and 45.0 mask AP, which are significant gains of +3.6 box AP and +3.3 mask AP over ResNeXt101- 64x4d, which has similar model size, FLOPs and latency.
res_2b8b152eOn a higher baseline of 52.3 box AP and 46.0 mask AP using an improved HTC framework, the gains by Swin Transformer are also high, at +4.1 box AP and +3.1 mask AP (see Table 2(c)).
res_30894bbfOur best model achieves 58.7 box AP and 51.1 mask AP on COCO test-dev, surpassing the previous best results by +2.7 box AP (Copy-paste [26] without external data) and +2.6 mask AP (DetectoRS [46]).
res_e3f80096Our Swin-L model with ImageNet-22K pre-training achieves 53.5 mIoU on the val set, surpassing the previous best model by +3.2 mIoU (50.3 mIoU by SETR [81] which has a larger model size).
res_e3a4b3a3Swin-T with the shifted window partitioning outperforms the counterpart built on a single window partitioning at each stage by +1.1% top-1 accuracy on ImageNet-1K, +2.8 box AP/+2.2 mask AP on COCO, and +2.8 mIoU on ADE20K.
res_3a341010Also note that while the inclusion of absolute position embedding improves image classification accuracy (+0.4%), it harms object detection and semantic segmentation (-0.2 box/mask AP on COCO and -0.6 mIoU on ADE20K).
res_6d5eeb30Overall, the Swin Transformer architectures built on shifted windows are 4.1/1.5, 4.0/1.5, 3.6/1.5 times faster than variants built on sliding windows for Swin-T, Swin-S, and Swin-B, respectively.
res_69ce43f7For other resolutions such as 3842, we fine-tune the models trained at 2242 resolution, instead of training from scratch, to reduce GPU consumption.
res_47f65b70For system-level comparison, we adopt an improved HTC [9] (denoted as HTC++) with instaboost [22], stronger multi-scale training [7] (resizing the input such that the shorter side is between 400 and 1400 while the longer side is at most 1600), 6x schedule (72 epochs with the learning rate decayed at epochs 63 and 69 by a factor of 0.1), soft- NMS [5], and an extra global self-attention layer appended at the output of last stage and ImageNet-22K pre-trained 9 downsp. rate (output size) Swin-T Swin-S Swin-B Swin-L stage 1 4×(56×56) concat 4×4, 96-d, LN concat 4×4, 96-d, LN concat 4×4, 128-d, LN concat 4×4, 192-d, LN[ win. sz.
res_64265450In general, a larger input resolution leads to better top-1 accuracy but with slower inference speed.
clm_2e7a2b57While SGD is used as a default optimizer for Cascade Mask R-CNN framework, we generally observe improved accuracy by replacing it with an AdamW optimizer, particularly for smaller backbones.
res_4a041eb6Swin-Mixer performs significantly better than MLP-Mixer (81.3% vs. 76.4%) using slightly smaller computation budget (10.4G vs. 12.7G).
Record your result in the Mearc app on your own machine, then use Publish this result with this paper's id (2103.14030) and the claim id shown next to the claim. Mearc creates a file that holds only what you choose to publish. Add it to the public claim data by pull request; a maintainer other than you reviews it before it counts.