Mask2Former

VW-Mask2Former

Installation

See installation instructions.

Datasets

See Preparing Datasets for MaskFormer.

Train

More Utilization: See Getting Started with MaskFormer.

Swin-Tiny

python ./train_net.py \
--resume --num-gpus 2 --dist-url auto \
--config-file configs/ade20k/semantic-segmentation/swin/vw/vw_maskformer2_swin_tiny_bs16_160k.yaml \
OUTPUT_DIR path/to/tiny TEST.EVAL_PERIOD 10000 MODEL.MASK_FORMER.SIZE_DIVISIBILITY 64

Swin-Tiny with Deformable Attention

python ./train_net.py \
--resume --num-gpus 2 --dist-url auto \
--config-file configs/ade20k/semantic-segmentation/swin/vw/vw_deformattn_maskformer2_swin_tiny_bs16_160k.yaml \
OUTPUT_DIR path/to/tiny TEST.EVAL_PERIOD 10000 MODEL.MASK_FORMER.SIZE_DIVISIBILITY 64

Swin-Small

python ./train_net.py \
--resume --num-gpus 4 --dist-url auto \
--config-file configs/ade20k/semantic-segmentation/swin/vw/vw_maskformer2_swin_small_bs16_160k.yaml \
OUTPUT_DIR path/to/small TEST.EVAL_PERIOD 10000 MODEL.MASK_FORMER.SIZE_DIVISIBILITY 64

Swin-Small with Deformable Attention

python ./train_net.py \
--resume --num-gpus 4 --dist-url auto \
--config-file configs/ade20k/semantic-segmentation/swin/vw/vw_deformattn_maskformer2_swin_small_bs16_160k.yaml \
OUTPUT_DIR path/to/small TEST.EVAL_PERIOD 10000 MODEL.MASK_FORMER.SIZE_DIVISIBILITY 64

Swin-Base

python ./train_net.py \
--resume --num-gpus 8 --dist-url auto \
--config-file configs/ade20k/semantic-segmentation/swin/vw/vw_maskformer2_swin_base_IN21k_384_bs16_160k_res640.yaml \
OUTPUT_DIR path/to/base TEST.EVAL_PERIOD 10000 MODEL.MASK_FORMER.SIZE_DIVISIBILITY 64

Swin-Base with Deformable Attention

python ./train_net.py \
--resume --num-gpus 8 --dist-url auto \
--config-file configs/ade20k/semantic-segmentation/swin/vw/vw_deformattn_maskformer2_swin_base_IN21k_384_bs16_160k_res640.yaml \
OUTPUT_DIR path/to/base TEST.EVAL_PERIOD 10000 MODEL.MASK_FORMER.SIZE_DIVISIBILITY 64

Swin-Large

python ./train_net.py \
--resume --num-gpus 16 --dist-url auto \
--config-file configs/ade20k/semantic-segmentation/swin/vw/vw_maskformer2_swin_large_IN21k_384_bs16_160k_res640.yaml \
OUTPUT_DIR path/to/large TEST.EVAL_PERIOD 10000 MODEL.MASK_FORMER.SIZE_DIVISIBILITY 64

Swin-Large with Deformable Attention

python ./train_net.py \
--resume --num-gpus 16 --dist-url auto \
--config-file configs/ade20k/semantic-segmentation/swin/vw/vw_deformattn_maskformer2_swin_large_IN21k_384_bs16_160k_res640.yaml \
OUTPUT_DIR path/to/large TEST.EVAL_PERIOD 10000 MODEL.MASK_FORMER.SIZE_DIVISIBILITY 64

Evaluation

python ./train_net.py \
--eval-only --num-gpus NGPUS --dist-url auto \
--config-file path/to/config \
MODEL.WEIGHTS path/to/weight TEST.AUG.ENABLED True MODEL.MASK_FORMER.SIZE_DIVISIBILITY 64

Model

Name	Backbone	crop size	lr sched	mIoU	mIoU (ms+flip)	download
VW-Mask2Former	Swin-T	512x512	160k	48.2	50.5	model
VW-Mask2Former	Swin-S	512x512	160k	52.1	53.7	model
VW-Mask2Former	Swin-B	640x640	160k	54.6	56.0	model
VW-Mask2Former	Swin-L	640x640	160k	56.5	57.8	model

Swin Transformer with Deformable Attention

Name	Backbone	crop size	lr sched	mIoU	mIoU (ms+flip)	download
VW-Mask2Former	Swin-T	512x512	160k	48.5	50.3	model
VW-Mask2Former	Swin-S	512x512	160k	52.0	53.6	model
VW-Mask2Former	Swin-B	640x640	160k	55.2	56.5	model
VW-Mask2Former	Swin-L	640x640	160k	56.9	58.3	model

Citing VW-Mask2Former

@inproceedings{yan2023multi,
  title={Multi-Scale Representations by Varing Window Attention for Semantic Segmentation},
  author={Yan, Haotian and Wu, Ming and Zhang, Chuang},
  booktitle={The Twelfth International Conference on Learning Representations},
  year={2023}
}