Our work introduces two key ideas to adapt semantic segmentation models across domains. I: Using image-level weak annotations for domain adaptation,
either estimated, i.e., pseudo-weak labels (Unsupervised Domain Adaptation, UDA) or acquired from a human oracle (Weakly-supervised Domain Adaptation, WDA).
II: We utilize weak labels to improve the category-wise feature alignment between the source and target domains.
Learning semantic segmentation models requires a huge amount of pixel-wise labeling. However, labeled data may only be available abundantly
in a domain different from the desired target domain, which only has minimal or no annotations. In this work, we propose a novel framework
for domain adaptation in semantic segmentation with image-level weak labels in the target domain. The weak labels may be obtained based on
a model prediction for unsupervised domain adaptation (UDA), or from a human annotator in a new weakly-supervised domain adaptation (WDA)
paradigm for semantic segmentation. Using weak labels is both practical and useful, since (i) collecting image-level target annotations is
comparably cheap in WDA and incurs no cost in UDA, and (ii) it opens the opportunity for category-wise domain alignment. Our framework uses
weak labels to enable the interplay between feature alignment and pseudo-labeling, improving both in the process of domain adaptation.
Specifically, we develop a weak-label classification module to enforce the network to attend to certain categories, and then use such training
signals to guide the proposed category-wise alignment method. In experiments, we show considerable improvements with respect to the existing
state-of-the-arts in UDA and present a new benchmark in the WDA setting.
Performance comparison on GTA5-to-Cityscapes with different levels of supervision on target images: no target labels
(“No Adapt” and “UDA”), weak image labels (30 seconds), one point labels (45 seconds), and fully-supervised
setting with all pixels labeled (“All Pixels Labeled”).
Sujoy Paul, Yi-Hsuan Tsai, Samuel Schulter, Amit K. Roy-Chowdhury and Manmohan Chandraker
Domain Adaptive Semantic Segmentation Using Weak Labels
In European Conference on Computer Vision (ECCV) 2020 [PDF][Bibtex]