This repo contains the implementation of the paper, Mind the Backbone: Minimizing Backbone Distortion for Robust Object Detection .
Download weights for efficientnet, resnet50 instagram model, and convnext. For other models, we use timm or see model_list.txt to download.
mkdir pre_models
cd pre_models
wget https://github.com/rwightman/pytorch-image-models/releases/download/v0.1-weights/tf_efficientnet_b2_ns-00306e48.pth
wget https://dl.fbaipublicfiles.com/semiweaksupervision/model_files/semi_weakly_supervised_resnet50-16a12f1b.pth
wget https://dl.fbaipublicfiles.com/convnext/convnext_base_22k_224.pth
Download from here
See following links for the details of each dataset,
BDD * Json split used for evaluation is here.
FoggyCityscapes * beta 0.02 is used for evaluation
Put directories under directory named VOCdevkit as follows:
|-- VOC2007
| |-- Annotations
| |-- ImageSets
| |-- JPEGImages
|-- VOC2012
|-- VOC_Clipart
|-- VOC_Water
|-- VOC_Comic
|-- bdd100k
|-- images
|-- 100k
|-- 10k
` |-- labels
|-- cityscapes
|-- gtFine
|-- leftImg8bit
|-- foggy_images
|-- train
|-- val
Run following commands in this directory before starting the training or use pre-trained decoder provided above.
export DETECTRON2_DATASETS=$VOCdevkit # it needs to be the path to VOCdevkit.
export PYTHONPATH="$PWD:$PYTHONPATH"
Run following commands in this directory.
python tools/train.py --config configs/Pascal/base.yaml --num-gpus 1 \\
OUTPUT_DIR tmp MODEL.PRETRAIN_NAME fbnet TRAIN.PRETRAIN True
## Train with RGN regularization
python tools/train.py --config configs/Pascal/base.yaml --num-gpus 1 \\
OUTPUT_DIR path_to_dir MODEL.PRETRAIN_NAME fbnet MODEL.WEIGHTS pretrained_detector/pascal_fbnet_pretrain.pth TRAIN.REG.RGN True \\
SOLVER.COEFF 0.1
TRAIN.REG.RGN == True => Apply RGN weighted regularization.
TRAIN.REG.EWC == True => Apply EWC weighted regularization.
TRAIN.REG.PLAINDIST == True => Apply plain weighted regularization.
Run following commands in this directory.
python tools/train.py --config configs/Pascal/base.yaml --num-gpu 1 --eval-only MODEL.WEIGHTS path_to_weight
If you find this repository useful for your publications, please consider citing our paper.
@article{saito2023mind,
title={Mind the Backbone: Minimizing Backbone Distortion for Robust Object Detection},
author={Saito, Kuniaki and Kim, Donghyun and Teterwak, Piotr and Feris, Rogerio and Saenko, Kate},
journal={arXiv preprint arXiv:2303.14744},
year={2023}
}