Learning Semantic Segmentation from Multiple Datasets with Label Shifts (arXiv)

/in /by

Publication Date: 2/4/2022

Event: arXiv

Reference: https://arxiv.org/abs/2202.14030

Authors: Dongwan Kim, Seoul National University, NEC Laboratories America, Inc., Yi Hsuan Tsai, NEC Laboratories America, Inc., Yumin Suh, NEC Laboratories America, Inc., Masoud Faraki, NEC Laboratories America, Inc., Sparsh Garg, NEC Laboratories America, Inc., Manmohan Chandraker, NEC Laboratories America, Inc., Bohyung Ha, Seoul National University

Abstract: With increasing applications of semantic segmentation, numerous datasets have been proposed in the past few years. Yet labeling remains expensive, thus, it is desirable to jointly train models across aggregations of datasets to enhance data volume and diversity. However, label spaces differ across datasets and may even be in conflict with one another. This paper proposes UniSeg, an effective approach to automatically train models across multiple datasets with differing label spaces, without any manual relabeling efforts. Specifically, we propose two losses that account for conflicting and co occurring labels to achieve better generalization performance in unseen domains. First, a gradient conflict in training due to mismatched label spaces is identified and a class independent binary cross entropy loss is proposed to alleviate such label conflicts. Second, a loss function that considers class relationships across datasets is proposed for a better multi dataset training scheme. Extensive quantitative and qualitative analyses on road scene datasets show that UniSeg improves over multi dataset baselines, especially on unseen datasets, e.g., achieving more than 8% gain in IoU on KITTI averaged over all the settings.

Publication Link: https://arxiv.org/pdf/2202.14030.pdf

You might also like
5 Accepted Papers CVPR2022! Check out our Computervision Papers at #CVPR2022
5 Accepted Papers CVPR2022! Active Adversarial Domain Adaptation
5 Accepted Papers CVPR2022! Adaptation Across Extreme Variations using Unlabeled Bridges
5 Accepted Papers CVPR2022! Adversarial Learning of Privacy-Preserving and Task-Oriented Representations
5 Accepted Papers CVPR2022! Channel Recurrent Attention Networks for Video Pedestrian Retrieval
5 Accepted Papers CVPR2022! Controllable Dynamic Multi-Task Architectures
5 Accepted Papers CVPR2022! Coordinated Joint Multimodal Embeddings for Generalized Audio-Visual Zero-shot Classification and Retrieval of Videos
5 Accepted Papers CVPR2022! Cross-Domain Similarity Learning for Face Recognition in Unseen Domains