This repo is a collection of the challenging panoptic segmentation, including papers, codes, and benchmark results, etc.
from UPSNet.
Generally, the datasets which contains both semantic and instance annotations can be used to solve the challenging panoptic task.
PQare the standard metrics described in Panoptic Segmentation.
PCare the standard metrics described in DeeperLab.
- AutoNUE 2019 Panoptic Segmentation Challenge (ICCV 2019 Workshop, Closed)
- COCO 2019 Panoptic Segmentation Task (ICCV 2019 Workshop, Closed)
- Mapillary 2019 Panoptic Segmentation Task (ICCV 2019 Workshop, Closed)
- Cityscapes Panoptic Semantic Labeling Task (Open)
- COCO 2018 Panoptic Segmentation Task (ECCV 2018 Workshop, Closed)
- Mapillary Vistas 2018 Panoptic Segmentation Task (ECCV 2018 Workshop, Closed)
| Method | Backbone | PQ | PQ-Thing | PQ-Stuff | SQ | RQ | mIoU | AP-Mask | PC | e2e |
|---|---|---|---|---|---|---|---|---|---|---|
| SOGNet | ResNet-50 | 43.7 | 50.6 | 33.2 | 78.7 | 53.5 | 54.56 | 34.2 | - | ✅ |
| UPSNet | ResNet-50 | 42.5 | 48.6 | 33.4 | - | - | 54.3 | 34.3 | - | ✅ |
| OANet | ResNet-101 | 41.3 | 50.4 | 27.7 | - | - | - | - | - | ✅ |
| OCFusion | ResNet-50 | 41.0 | 49.0 | 29.0 | 77.1 | 50.6 | - | - | - | ✅ |
| Panoptic FPN | ResNet-101 | 40.9 | 48.3 | 29.7 | - | - | - | - | - | ✅ |
| AUNet | ResNet-50 | 39.6 | 49.1 | 25.2 | - | - | 45.1 | 34.7 | - | ✅ |
| AdaptIS | ResNet-101 | 37.0 | 41.8 | 29.9 | - | - | - | - | - | ✅ |
| DeeperLab | Xception-71 | 34.3 | 37.5 | 29.6 | 77.1 | 43.1 | - | - | 56.8 | ✅ |
| Method | Backbone | PQ | PQ-Thing | PQ-Stuff | SQ | RQ | mIoU | AP-Mask | PC | e2e |
|---|---|---|---|---|---|---|---|---|---|---|
| Panoptic(Merge) | - | 61.2 | 66.4 | 54.0 | 80.9 | 74.4 | - | - | - | ❎ |
| AdaptIS | ResNet-101 | 60.6 | 58.7 | 64.4 | - | - | 79.2 | 36.3 | - | ✅ |
| SOGNet | ResNet-50 | 60.0 | 56.7 | 62.5 | - | - | - | - | - | ✅ |
| Seamless | ResNet-50 | 59.8 | 53.4 | 64.5 | - | - | 75.4 | 31.9 | - | ✅ |
| UPSNet | ResNet-50 | 59.3 | 54.6 | 62.7 | 79.7 | 73.0 | 75.2 | 33.3 | - | ✅ |
| TASCNet | ResNet-101 | 59.2 | 56 | 61.5 | - | - | 77.8 | 37.6 | - | ✅ |
| AUNet | ResNet-101 | 59.0 | 54.8 | 62.1 | - | - | 75.6 | 34.4 | - | ✅ |
| Panoptic FPN | ResNet-101 | 58.1 | 52.0 | 62.5 | - | - | 75.7 | 33.0 | - | ✅ |
| DeeperLab | Xception-71 | 56.5 | - | - | - | - | - | - | 75.6 | ✅ |
| Method | Backbone | PQ | PQ-Thing | PQ-Stuff | SQ | RQ | mIoU | AP-Mask | PC | e2e |
|---|---|---|---|---|---|---|---|---|---|---|
| Panoptic(Merge) | - | 38.3 | 41.8 | 35.7 | 73.6 | 47.7 | - | - | - | ❎ |
| Seamless | ResNet-50 | 37.2 | 33.2 | 42.5 | - | - | 50.2 | 16.3 | - | ✅ |
| AdaptIS | ResNet-101 | 33.4 | 28.3 | 40.3 | - | - | - | - | - | ✅ |
| TASCNet | ResNet-101 | 32.6 | 31.3 | 34.4 | - | - | 35.0 | 18.5 | - | ✅ |
| DeeperLab | Xception-71 | 32.0 | - | - | - | - | - | - | 55.3 | ✅ |
- SOGNet: Yibo Yang, Hongyang Li, Xia Li, Qijie Zhao, Jianlong Wu, Zhouchen Lin.
"SOGNet: Scene Overlap Graph Network for Panoptic Segmentation." AAAI (2020). [paper]
-
AdaptIS: Konstantin Sofiiuk, Olga Barinova, Anton Konushin.
"AdaptIS: Adaptive Instance Selection Network." ICCV (2019). [paper] -
Cheng-Yang Fu, Tamara L. Berg, Alexander C. Berg.
"IMP: Instance Mask Projection for High Accuracy Semantic Segmentation of Things." ICCV (2019). [paper] -
Bowen Cheng, Maxwell D. Collins, Yukun Zhu, Ting Liu, Thomas S. Huang, Hartwig Adam, Liang-Chieh Chen.
"Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation Bowen." ICCVW (2019). [paper]
-
Panoptic Segmentation: Alexander Kirillov, Kaiming He, Ross Girshick, Carsten Rother, Piotr Dollár.
"Panoptic Segmentation." CVPR (2019). [paper] -
Panoptic FPN: Alexander Kirillov, Ross Girshick, Kaiming He, Piotr Dollár.
"Panoptic Feature Pyramid Networks." CVPR (2019 oral). [paper] [unofficial code][detectron2] -
AUNet: Yanwei Li, Xinze Chen, Zheng Zhu, Lingxi Xie, Guan Huang, Dalong Du, Xingang Wang.
"Attention-guided Unified Network for Panoptic Segmentation." CVPR (2019). [paper] -
UPSNet: Yuwen Xiong, Renjie Liao, Hengshuang Zhao, Rui Hu, Min Bai, Ersin Yumer, Raquel Urtasun.
"UPSNet: A Unified Panoptic Segmentation Network." CVPR (2019 oral). [paper] [code] -
DeeperLab: Tien-Ju Yang, Maxwell D. Collins, Yukun Zhu, Jyh-Jing Hwang, Ting Liu, Xiao Zhang, Vivienne Sze, George Papandreou, Liang-Chieh Chen.
"DeeperLab: Single-Shot Image Parser." CVPR (2019 oral). [paper] [project] [code] -
OANet: Huanyu Liu, Chao Peng, Changqian Yu, Jingbo Wang, Xu Liu, Gang Yu, Wei Jiang.
"An End-to-End Network for Panoptic Segmentation." CVPR (2019). [paper] -
Eirikur Agustsson, Jasper R. R. Uijlings, Vittorio Ferrari .
"Interactive Full Image Segmentation by Considering All Regions Jointly." CVPR (2019). [paper] -
Seamless: Lorenzo Porzi, Samuel Rota Bulo, Aleksander Colovic, Peter Kontschieder.
"Seamless Scene Segmentation." CVPR (2019) (Extended Version). [paper][code]
- Qizhu Li, Anurag Arnab, Philip H.S. Torr.
"Weakly- and Semi-Supervised Panoptic Segmentation." ECCV (2018). [paper] [code]
-
Rohit Mohan, Abhinav Valada.
"EfficientPS: Efficient Panoptic Segmentation." arXiv (2020). [paper] -
Rui Hou, Jie Li, Arjun Bhargava, Allan Raventos, Vitor Guizilini, Chao Fang, Jerome Lynch, Adrien Gaidon.
"Real-Time Panoptic Segmentation from Dense Detections." arXiv (2019). [paper] -
Mark Weber, Jonathon Luiten, Bastian Leibe.
"Single-Shot Panoptic Segmentation." arXiv (2019). [paper] -
Qiang Chen, Anda Cheng, Xiangyu He, Peisong Wang, Jian Cheng.
"SpatialFlow: Bridging All Tasks for Panoptic Segmentation." arXiv (2019). [paper] -
Sagi Eppel, Alan Aspuru-Guzik.
"Generator evaluator-selector net: a modular approach for panoptic segmentation." arXiv (2019). [paper] -
Jasper R. R. Uijlings, Mykhaylo Andriluka, Vittorio Ferrari.
"Panoptic Image Annotation with a Collaborative Assistant." arXiv (2019). [paper] -
OCFusion: Justin Lazarow, Kwonjoon Lee, Zhuowen Tu.
"Learning Instance Occlusion for Panoptic Segmentation." arXiv (2019). [paper] -
PEN: Yuan Hu, Yingtian Zou, Jiashi Feng.
"Panoptic Edge Detection." arXiv (2019). [paper] -
TASCNet: Jie Li, Allan Raventos, Arjun Bhargava, Takaaki Tagawa, Adrien Gaidon.
"Learning to Fuse Things and Stuff." arXiv (2018). [paper] -
Daan de Geus, Panagiotis Meletis, Gijs Dubbelman.
"Panoptic Segmentation with a Joint Semantic and Instance Segmentation Network." arXiv (2018). [paper] -
Daan de Geus, Panagiotis Meletis, Gijs Dubbelman.
"Single Network Panoptic Segmentation for Street Scene Understanding." arXiv (2019). [paper] -
David Owen, Ping-Lin Chang.
"Detecting Reflections by Combining Semantic and Instance Segmentation." arXiv (2019). [paper] -
Gaku Narita, Takashi Seno, Tomoya Ishikawa, Yohsuke Kaji.
"PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things." arXiv (2019, IROS). [paper]
- CVPR 2019 Tutorial on Visual Recognition and Beyond. [slides] [homepage]
- COCO 2017 Workshop. [slides]
- Megvii(Face++) Detection Team. [zhihu]