Domain Adaptive Semantic Segmentation
Using Weak Labels

Sujoy Paul1 Yi-Hsuan Tsai2 Samuel Schulter2
Amit K. Roy-Chowdhury1 Manmohan Chandraker2,3
1 UC Riverside 2 NEC Laboratories America 3 UC San Diego
ECCV 2020
[Paper]
Our work introduces two key ideas to adapt semantic segmentation models across domains. I: Using image-level weak annotations for domain adaptation, either estimated, i.e., pseudo-weak labels (Unsupervised Domain Adaptation, UDA) or acquired from a human oracle (Weakly-supervised Domain Adaptation, WDA). II: We utilize weak labels to improve the category-wise feature alignment between the source and target domains.
Learning semantic segmentation models requires a huge amount of pixel-wise labeling. However, labeled data may only be available abundantly in a domain different from the desired target domain, which only has minimal or no annotations. In this work, we propose a novel framework for domain adaptation in semantic segmentation with image-level weak labels in the target domain. The weak labels may be obtained based on a model prediction for unsupervised domain adaptation (UDA), or from a human annotator in a new weakly-supervised domain adaptation (WDA) paradigm for semantic segmentation. Using weak labels is both practical and useful, since (i) collecting image-level target annotations is comparably cheap in WDA and incurs no cost in UDA, and (ii) it opens the opportunity for category-wise domain alignment. Our framework uses weak labels to enable the interplay between feature alignment and pseudo-labeling, improving both in the process of domain adaptation. Specifically, we develop a weak-label classification module to enforce the network to attend to certain categories, and then use such training signals to guide the proposed category-wise alignment method. In experiments, we show considerable improvements with respect to the existing state-of-the-arts in UDA and present a new benchmark in the WDA setting.

Main Results

Performance comparison on GTA5-to-Cityscapes with different levels of supervision on target images: no target labels (“No Adapt” and “UDA”), weak image labels (30 seconds), one point labels (45 seconds), and fully-supervised setting with all pixels labeled (“All Pixels Labeled”).

Paper

Sujoy Paul, Yi-Hsuan Tsai, Samuel Schulter, Amit K. Roy-Chowdhury and Manmohan Chandraker
Domain Adaptive Semantic Segmentation Using Weak Labels
In European Conference on Computer Vision (ECCV) 2020
[PDF]  [Bibtex]

Short Summarization



Video Results on GTA5-to-Cityscapes (Sequence 1)



Video Results on GTA5-to-Cityscapes (Sequence 2)



Video Results on GTA5-to-Cityscapes (Sequence 3)